Category Archives: Linux

Install and run Freenetis in a Docker container

According to wikipedia, Docker is an open-source project that automates the deployment of applications inside software containers, by providing an additional layer of abstraction and automation of operating-system-level virtualization on Linux.

Freenetis is an open source information system for managing community networks.

Why to learn Docker and container technologies in general?

I am fascinated with the idea behind the Docker. It enables you to just create your own docker image with your application and you don’t have to care about deployment details at all. It can be deployed locally, on your server or directly into a cloud. So the first benefit is the significant reduce of requirements with respect to deployment.

You can achieve isolation benefits like with virtual machines, but with significantly smaller overhead and memory footprint. Docker allows you to use configuration as a code – so called Dockerfile – which is lightweight and makes it easy to share container definitions. It also offers simple way to distribute ready-to-deploy containers.

There are also disadvantages, it is currently targeted more only to Linux hosts and images, but this may change in time.

Current possibilities to install Freenetis

Freenetis can be installed currently in two ways. Either it can be installed directly by checking out sources and configuring everything from scratch or it can be installed automatically using the provided deb packages. The first option is substantially difficult and requires deep knowledge of the Freenetis system. The second option to use prepared deb packages is much more user friendly.

I currently use Fedora on my laptop and it uses rpm packaging system. So there is currently no simple way to install Freenetis on my machine, as the only option for me is to install manually from the sources. However there is a solution. I can install Docker, pull there an image of Debian operating system, start the image as a container and install there Freenetis using the steps provided. Currently only in Czech. That’s it! I don’t have to install heavy-weight full virtualization of Debian.

Installing Docker

The first thing is to install docker. The following commands are applicable and were tested on Fedora 20. You need to install docker-io package. To make further work easier, it is convenient to add a system user to a docker group, that will make your life easier and you will not have to work ordinarily with Docker as a superuser.

# yum install docker-io
# usermod -a -G docker user_id

We can look for all debian images available in a central repository and install the basic one with the second command:

$ docker search debian
$ docker pull debian

I have to mention several useful commands here. The first one is to list currently installed Docker images and the second one is to remove unnecessary images. This will be useful, as after each update of the upstream Docker image, the old one is preserved.

$ docker images
$ docker rmi c9fa20ecce88

The last command is to check if the image is working properly. It should print the content of a default directory:

$ docker run -t debian ls

Installing Freenetis

At the beginning the container already needs to be running, we can achieve this by starting the container in interactive mode:

$ docker run -it -p 12345:80 debian /bin/bash

Remember that all changes done after starting the container are automatically lost. To save them, we need to create a container snapshot and save it as a new image. That’s why number of container images may grow quickly and it is useful to use Dockerfiles or run commands in batch. Let’s return to the command itself. Container will be run in interactive mode, with a new console open in the same terminal window. Port 80 of the container will be mapped to port 12345 of the host system. Console of the container will be started by definining the last command, which tells the docker to start bash console in the container.

Let’s install necessary tools for installation:

root@fa644d58b14:/# apt-get update && apt-get install vim wget

We have to add new Debian package repository:

root@fa644d58b14:/# vim /etc/apt/sources.list
deb http://repository.freenetis.org/debian/stable/ wheezy main

Then we can install Freenetis itself:

wget -O - http://repository.freenetis.org/debian/freenetis_repo.gpg.key | apt-key add -
apt-get update && apt-get install freenetis

During Freenetis installation I chose option one to install Freenetis into location http://localhost/freenetis.

In the end I had manually start Apache 2 httpd server:

root@5949a66e334c:/# service apache2 start

Now I am ready on my host computer to access Freenetis installed on the following page http://localhost:12345/freenetis.

Summary

In this article I’ve tried to sum up benefits of container technologies and Docker in particular. My current aim is to install Freenetis easily on unsupported non-debian system like Fedora. However the implications of Docker are big, for example a cloud that supports Docker should be easily able to consume images containing Freenetis. So try it now and let me know your feedback, I’ll appreciate it!

MariaDB tips for Fedora 20

Installation

When I did installation of MySQL in Fedora 20, it automatically installed MariaDB database. In order to install MariaDB now, the current commands will be for example:

$ sudo su -
# yum install mariadb mariadb-server phpmyadmin
# sudo systemctl enable mariadb.service
# sudo systemctl start mariadb.service

In order to setup root password for MariaDB after installation you have to do the following:

$ mysql -u root
MariaDB [(none)]> set password for root@localhost = password('your_password');
MariaDB [(none)]> exit;

Now you can login into http://localhost/phpmyadmin as a user root using the defined password. In order to login into MariaDB console again, you have to change the command to request password authentication:

$ mysql -u root -p

Usage

The database has been installed, set up to start every time after reboot. What else can be done? In order to use it as a developer, we have to create also a new database inside. The simplest way is to login to phpmyadmin manager of MariaDB database. Select tab ‘Users’ and click on the button ‘Add user’. Fill in the user name, password, repeat the password and check option ‘Create database with same name and grant all privileges.’. That’s it! You’ll see the all SQL commands, which were executed. If you are able to type the commands into the console, you can use it as well, for example:

MariaDB [(none)]> create database jbpm;
MariaDB [(none)]> use jbpm;

For example now we can deploy and run any arbitrary application, which uses jbpm schema for persistence. The schema will be generated using DDL scripts automatically by hibernate. Later we can inspect the content of the database using other SQL commands.

Working with PostgreSQL

Installation

PostgreSQL installation command on Fedora:

# yum -y install postgresql-server phpPgAdmin

It assumes that you have already Apache web server installed (package httpd)

Setup

PostgreSQL database needs to be initialized first.

# postgresql-setup initdb
# su postgres		
bash-4.2$ createuser --superuser -P pgadmin
bash-4.2$ createdb --encoding=utf-8 --owner=pgadmin pgadmin
# systemctl start postgresql.service
# systemctl enable postgresql.service

Also it is necessary to enable logging into phpPgAdmin:

# vim /etc/phpPgAdmin/config.inc.php
$conf['extra_login_security'] = false;

To change the PostgreSQL password:

$ sudo -u postgres psql
password pgadmin
q

Setting up PostgreSQL database can be tricky. Just remember that after installation there is a user called postgres, who is a user in the operating system. The most comfortable way is to login as superuser and then as postgres user. After that you are able to create a database superuser, for example pgadmin. It’s necessary to create a database for him. This is not enough, you have to disable default authentication methods in /var/lib/pgsql/data/pg_hba.conf and change them to md5. After that and restarting the service I was able to login into phpPgAdmin as a pgadmin user.

# su postgres
bash-4.2$ vim /var/lib/pgsql/data/pg_hba.conf
local   all             all                                     md5
host    all             all             127.0.0.1/32            md5
host    all             all             ::1/128                 md5
bash-4.2$ exit
# systemctl restart postgresql.service

Creating users and their databases

The easiest way is to login to phpPgAdmin as pgadmin user. First you need to create a role, which seems to be the same as user in phpPgAdmin terminology. Fill in the role’s name, password, password confirmation and don’t forget to check at least “Can login?” check box. Then create a database for this user, for simplicity it is good habit to use the same name as the name of the user (role). The latest thing is to grant privileges for the selected user on already created database.

Running SQL commands from CLI

This example will focus on a simple Bash script, which is able to reset (drop and create) a PostgreSQL database. The script may look like this:

#!/bin/bash
# PostgreSQL 9.1 pg_stat_activity.procpid, 9.2 has just pg_stat_activity.pid
psql -U pgadmin -c "update pg_database set datallowconn = 'false' where datname = '$DATABASE'; SELECT pg_terminate_backend(pg_stat_activity.procpid) FROM pg_stat_activity WHERE datname = '$DATABASE';"
psql -U pgadmin -c "drop database $DATABASE;"
psql -U pgadmin -c "create database $DATABASE owner $DATABASE encoding 'UTF8';"

Script assumes that you have already set up a super user pgadmin. Still there are few things to be done to run the script. The first thing is to edit the file pg_hba.conf and change authentication method to md5.

# vim /var/lib/pgsql/data/pg_hba.conf
local   all             all                                     md5
host    all             all             127.0.0.1/32            md5
host    all             all             ::1/128                 md5

Now you should be able to run the script, but for each command you would have to fill in the password. To remove this necessity, you can save the password in your home directory.

$ vim ~/.pgpass
localhost:5432:*:pgadmin:password

Please note that this is not safe to save the password as a plain text, but I believe that for a local testing instance of PostgreSQL database it is a satisfactory solution.

 

Caveats

Hints

  • This utility will remove orphaned large objects from the database, its diagnosis mode is:
    /usr/pgsql-9.3/bin/vacuumlo -n -v database_name

Tips for working with git

This article describes basic steps regarding to working with git and also gives several tips, which can be useful and are not usually covered in tutorials.

Typical workflow

To start working with git you can create own repo with git init or you can clone an existing repository, for example:

git clone git@github.com:droolsjbpm/jbpm.git

It’s recommended to prefer SSH url, because HTTPS connection is less reliable.

You can make changes to your project and add them to next commit. And then commit the changes. You should always write a brief description of your commit.

git add --all
git commit -m "Refactored module X to work with Y."

After making necessary commits you can share your changes with other by pushing them to the server. Usually the remote on the server is called origin or upstream. The convention is that main repo of the project for all people is usually called upstream and personal forked repo on Github is called origin. I think it is a good habit to also specify a branch you want to push. For example:

git push origin master

It is good idea to regularly check for changes from the others to work with current code and prevent conflicts. The first one just downloads the changes. The second one adds the changes of others to your master branch and puts your commits on the top as the latest ones. This principle keeps linear history and is called rebasing.

git fetch upstream
git rebase upstream/master

You may easily end with unwanted changes in the repository and there seem to be no steps to revert the state back. In this cases reset can help, but it clears your changes and makes your repository the same as remote.

git reset --hard

Getting information about your repository

Here are several useful commands to get more information about your repository. So to check status of your repository, list all branches or remotes:

git status
git branch -a
git remote -v

Sometimes it is necessary to check commit history of your project.  Log is the basic command, but much better is to install tool called tig. It allows you to inspect history in command line interface in more comfortable way than simple log.

git log
tig

Interested in who screwed something up? You can check who changed which lines in selected file.

git blame README.md

Working with branches

Branches are powerful feature of Git. Instead of Subversion, the branches are cheap and simple to use. To create a new branch “MyBranch” and start working with it, just type:

git checkout -b MyBranch

This will create new branch by copying the current branch and automatically the MyBranch is switched. To change branch, for example to work again with master:

git checkout master

You can create topic branches, make changes, push them to your origin and in Github web interface make pull requests to upstream. Sometimes there are changes done in the upstream during the work on a pull request. Then you cannot simply push, because when you rebase updates from master to your topic branch, the commit IDs will change. You can use a force push here:

git checkout MyTopicBranch
git rebase master
git push --force origin MyTopicBranch

However after you are done, you may need to delete old branches. The first one deletes remotely, the second one locally.

git push origin :MyTopicBranch
git branch -D MyTopicBranch

How to rebase a Github pull request to your project

Imagine the situation. Your project is hosted on Github. Somebody forks your repository and creates a pull request to your repo. Github offers you a green button “Merge pull request” if there are no conflicts to automatically merge the changes of a contributor to your repository. This will add the changes to your upstream but will create an additional merge commit. Better approach can be the git rebase operation, which has some significant advantages over merge. The main reason is clean and linear history of the commits. I do not know why, but Github does not have a button “Rebase pull request”. So to utilize advantages of rebase approach the following git command can simplify your life (taken from blog of Stuart Douglas):

git pull --rebase upstream pull/$no/head
git push upstream master

Where the upstream is the name of the Github remote where the pull request was created and $no is the identification number of the pull request. This way you can rebase the changes to your local working repository, check them, make corrections if needed, run unit tests and then push back to your upstream/origin on the server. Pushing to repository also automatically closes the pull request.

Creating a patch file

One of the simplest approaches is to commit your changes, create a patch file (for example for your co-workers) and reset hard to a previous commit:

git format-patch -M -1
git reset --hard previous_commit_id

Applying a patch

Later if we want to apply a patch file to our repository the steps are following:

git apply patch_file_name.patch

Analyze chess games on Linux

Club chess players are familiar with chess analysis. I started playing chess when I was a child. Later I began to play regular games for my local chess club. After a game you are usually encircled by a group of other players and they tend to show you possible better moves, especially when you lost. This can sometimes help you a bit. Unfortunately there are often too many hands, which change the position quickly and you can be confused. Or at least easily caught thinking about a less probable position.

Almost every player is interested in improving his technique and skills. Computer is your friend and is able to find the moves, that could be better in a given game position. Not need to mention that current computers are able to defeat even the best chess players. It’s not that surprising, if you realize that computers are better at tactics and also contain huge knowledge of many openings and thousands known games previously played. Human grandmasters were able to defeat them only by using more sophisticated strategy and planning. For me it was always boring to play against the computers due to obvious reasons. However I have learned a lot from my own mistakes, which were shown to me by a chess engine in analysis mode. For me the first such chess engine was Fritz.

Since I have been using Linux for several years now, I have tried free and open source chess engine Crafty. My steps summary is based on the following discussion. By the way I have found a really nice game played by two computers – Fritz against Crafty. Fritz won in that match, but I guess that no human player would be so brave to play like that!

Installation of packages

In Ubuntu type the following command:

sudo apt-get install crafty eboard scid

Crafty is the chess engine itself, Eboard is a graphical frontend for chess board and Scid is an analysis tool.

Adding the engine to Scid and running the analysis

Now you can simply run the chess analysis. Type scid in terminal or create a desktop icon for this application. In Tools->Analysis engine add the Crafty. Possible configuration:

  • Name: Crafty
  • Command: crafty
  • Directory: ~/.eboard
  • Uncheck UCI engine

That’s all. Now double click Crafty in the Analysis engine and the analysis runs!

Improving the engine

There are several ways how to improve the quality of Crafty. They are described in the linked Ubuntu forum post and simply said you have to download and build a database of chess games which improves Crafty’s strength based on the statistics of many already played opening positions.

My steps after Fedora installation

I’ve wanted to write a post about setting up my Linux environment for a long time. It’s for my personal purposes, so it doesn’t claim to be the best guide that you are looking for. The main motivation is to have easy-to-access place (like this blog), where I can find useful and well-arranged information quickly. I’ve googled many things many times and I would like to avoid repeating of my search queries. Code snippets are sometimes specific to Fedora 17, but generally the commands will be helpful with future releases too.

Do not count on Linux distribution upgrade feature

I’ve upgraded several Linux distributions many times in the past and rarely with success. Very often there have been problems with broken dependencies, kernel upgrades, hardware, Java, etc. The best solution is to divide your partition when doing the first installation (fortunately Fedora installer can partition the drive, if you are not sure):

  • root /
  • home dir /home
  • swap

This division of your hard drive partition enables you to just format your root partition and reinstall Linux there without any problems. All of your user data will be kept on your /home partition including configuration of all applications used before. This tip can save you several hours.

Install additional software

I am used to software of my choice and often I don’t like the default software for distribution chosen by someone else. So I install several types of packages:

$ su -
# yum -y install audacious audacious-plugins-freeworld-mp3 audacious-plugins-freeworld-ffaudio gimp git htop keepassx krb5-workstation nfs-utils pidgin subversion terminator thunderbird tig vim wireshark-gnome

Java

# yum -y install java-1.7.0-openjdk java-1.7.0-openjdk-devel

This command installs both Java runtime environment and development kit.

Services

# yum -y install httpd phpmyadmin mysql-server postgresql-server phpPgAdmin

Services include MySQL (mysqld) and PostgreSQL (postgresql) databases and web server (httpd). The services can be controlled like this:

# service httpd|mysqld|postgresql start|restart|stop

To start services at each system boot it is necessary to run:

# chkconfig postgresql on

Phpmyadmin might refuse to login. Then it’s necessary to reset MySQL database password:

$ mysql -u root
mysql> set password for root@localhost = password('yourpassword');
mysql> exit;

Later you need to login into MySQL using the password authentication.

$ mysql -u root -p

Unpackaged software

This kind of software includes many tools and application useful for Java developer.

  • Intellij Idea – instructions how to add GUI icon for Idea
  • Maven – just unzip and add $M2_HOME system variable and $M2_HOME/bin directory to your $PATH variable
  • Eclipse

Yes, the question is where to put these kind of software. You have probably successfully downloaded the archive into $HOME/Downloads. If your computer is and will remain single user, then the easiest solution is to put these applications inside something like ~/Apps. For multiuser machine you should use /opt, but this dir means additional configuration of access rights for root. After unzipping and moving the app directory you just need to add bin dir of the apps to your system $PATH variable.

Unofficial Fedora repository – RPM Fusion

Fedora strongly complies to the ideas of free software, so it doesn’t package nonfree software. But there are still many sites and data formats which require such nonfree software. To solve this problem you can add RPM Fusion repository, which contains such nonfree, but sometimes necessary software. You can follow their guide. Or just use these commands:

$ su -c 'yum localinstall --nogpgcheck http://download1.rpmfusion.org/free/fedora/rpmfusion-free-release-stable.noarch.rpm http://download1.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-stable.noarch.rpm'
$ su -c 'yum install gstreamer-{ffmpeg,plugins-{good,ugly,bad{,-free,-nonfree}}}'

Install Adobe Flash

I have taken the steps from here.

# rpm -ivh http://linuxdownload.adobe.com/adobe-release/adobe-release-x86_64-1.0-1.noarch.rpm
# rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-adobe-linux
# yum install flash-plugin nspluginwrapper alsa-plugins-pulseaudio libcurl

Printer autodiscovery

This demonstrates how to configure cups to automatically discover all printers available in the local network.

# vim /etc/cups/cupsd.conf
Browsing On
BrowsePoll cups.site.company.com:631
# service cups restart

Sometimes it’s necessary to change user and group id, especially after system reinstallation (Fedora 20):

systemctl isolate multi-user.target
groupmod -g GID group
usermod -u UID -g GID user
systemctl isolate graphical.target
# find / -uid OLD_UID -exec chown -h UID  '{}' \+
# find / -gid OLD_GID -exec chgrp -h GID '{}' \+