Monthly Archives: February 2013

First release of jBPM workflow plugin

Yesterday I released the first version of my jBPM workflow plugin. I started research and development of this JenkinsjBPM integration at the university in November 2011. And my company let me to continue work on my project. That’s great, because this is one of the things, which keeps my job from becoming monotonous and also I have learned a lot of knowledge by working on this project.

Continuous integration helps many companies in their software development. Jenkins is one of the most widespread CI servers. It’s used for several purposes – building software, running tests, publishing results, software analysis, etc. As a quality engineer I have mostly experience only with Jenkins as a tool for running many matrix tests. Axis include operating systems, Java development kits (JDK), databases, Java content repositories (JCR) and web browsers. Many test types are included too like unit, regression, integration, performance and others. And as you continuously work on something, you tend to optimize your steps.

So back to start – I had to apply for a master project and found an interesting topic. The task was to create a Jenkins plugin, which would enable to run test plans by using business processes to improve efficiency of the quality engineering department. If you spend less time with starting, monitoring and evaluating Jenkins jobs, then you have more time for other tasks, for example improving test coverage of a product. I defended the master’s project and continued to improve it.

Now the early release of the plugin has been done. However current solution still has several limitations. It can fully run only in JBoss AS 7 container. What is meant by fully? That means running business process with persistence enabled. That’s necessary if you want to easily recover the latest state from database in case of unexpected outage. Theoretically the plugin should run without persistence on any container, but I have not verified that. And also it’s necessary to mention that the JBoss AS needs its jpa subsystem to be disabled. That can be real limitation if you host more applications than Jenkins in one application server instance.

Problems that had to be solved

I’ll try to summarize problems I faced with the implementation of the plugin. The problems were often related to classloading.

Classloading issues

I had to face many classloading problems. For example I was not able to run the tests, I had to always use -DskipTests option in Maven command. One of the reasons was the XStream library – both jBPM and Jenkins use it. Unfortunately Jenkins uses its own changed version and jBPM another version. Only one of them was classloaded, so I could not run tests because Jenkins test suite could not find its specific method in its changed XStream library.

Could not find persistence.xml

This happened because of complexity of combination of Jenkins and JBoss classloading. Jenkins uses several classloaders and one of them is dedicated for plugins. So the Jenkins .war is deployed in the application server, but deploys .hpi file by its own mechanism. This was solved by changing context classloader of the plugin thread to that dedicated classloader.

Need to eliminate provided persistence provider as a result of inability to configure it properly

After many failures I have decided to supply hibernate .jars directly with the plugin and od not depend on the ones provided by JBoss AS. But running two Hibernate frameworks together could not work. I have tried many things how to disable classloading of Hibernate into Jenkins or into my plugin using the jboss-deployment-structure.xml. This file should exclude or include specific classpath elements specific for your application. Unfortunately without success. So I had to disable jpa subsystem in standalone.xml configuration file. This way correct Hibernate is loaded and is able to find the persistence.xml file. If you know, how to better control classloading (for example by jboss-deployment-structure.xml), feel free to leave me a comment, thanks.

Transaction issues

I had also issues with transactions, but honestly I don’t remember them exactly. The solution was to implement own way for looking up a transaction manager, since I have decided not to depend no provided persistence system.

Future of the plugin

Current implementation of the plugin has its drawbacks, these include:

  • need for setup an additional Jenkins job for launching the business process build step with with test plan
  • only JBoss AS 7 with disabled jpa subsystem is currently usable for persistence

The best approach to solve this can jBPM remote handler implementation. This handler implements a work item handler in jBPM for Jenkins Task (service task). It’s the same service task as in jBPM workflow plugin, or at least with the same interface. You can supply it in the same way with the job name to launch, optionally with parameters in case of parameterized build and receive an object as a result. This is great since you can use both integration approaches and your business processes will remain the same. Only handler implementation changes.

Remote handler as the name suggest will start and evaluate Jenkins jobs remotely. Business process instance will run in jBPM console (simple process server) or a user application, which uses jBPM engine inside. jBPM console can also handle human tasks, for example for decision of the QA engineer like what to do if basic tests failed etc. The main benefit will be decreased complexity of the integration implementation and also there will be no limitations on application server for Jenkins.

Conclusion

I have learned a lot of technical stuff on this project. I have understood many concepts. But most importantly I have learned that it’s not a good idea to integrate two complex technologies directly on the application level. At least in Java you’ll end up with many classloading problems. If you can oppose this, please leave me a comment, thanks.

Advertisements

My first release!

Yesterday I released the first version of my jBPM workflow plugin. I started research and development of this JenkinsjBPM integration at the university in November 2011. And my company let me to continue work on my project. That’s great, because this is one of the things, which keeps my job from becoming monotonous and also I have learned a lot of knowledge by working on this project.

Continuous integration helps many companies in their software development. Jenkins is one of the most widespread CI servers. It’s used for several purposes – building software, running tests, publishing results, software analysis, etc. As a quality engineer I have mostly experience only with Jenkins as a tool for running many matrix tests. Axis include operating systems, Java development kits (JDK), databases, Java content repositories (JCR) and web browsers. Many test types are included too like unit, regression, integration, performance and others. And as you continuously work on something, you tend to optimize your steps.

So back to start – I had to apply for a master project and found an interesting topic. The task was to create a Jenkins plugin, which would enable to run test plans by using business processes to improve efficiency of the quality engineering department. If you spend less time with starting, monitoring and evaluating Jenkins jobs, then you have more time for other tasks, for example improving test coverage of a product. I defended the master’s project and continued to improve it.

Now the early release of the plugin has been done. However current solution still has several limitations. It can fully run only in JBoss AS 7 container. What is meant by fully? That means running business process with persistence enabled. That’s necessary if you want to easily recover the latest state from database in case of unexpected outage. Theoretically the plugin should run without persistence on any container, but I have not verified that. And also it’s necessary to mention that the JBoss AS needs its jpa subsystem to be disabled. That can be real limitation if you host more applications than Jenkins in one application server instance.

Problems that had to be solved

I’ll try to summarize problems I faced with the implementation of the plugin.

Jboss AS 7 Hibernate was not able to find persistence.xml of my plugin

This happened because of complexity of combination of Jenkins and JBoss classloading. Jenkins uses several classloaders and one of them is dedicated for plugins. So the Jenkins .war is deployed in the application server, but deploys .hpi file by its own mechanism. This was solved by changing context classloader of the plugin thread to that dedicated classloader.

JBoss AS 7 had a tendency to supply its own Hibernate

After many failures I have decided to supply hibernate .jars directly with the plugin and od not depend on the ones provided by JBoss AS. But running two Hibernate frameworks together could not work. I have tried many things how to disable classloading of Hibernate into Jenkins or into my plugin using the jboss-deployment-structure.xml. This file should exclude or include specific classpath elements specific for your application. Unfortunately without success. So I had to disable jpa subsystem in standalone.xml configuration file. This way correct Hibernate is loaded and is able to find the persistence.xml file.

Transaction issues

I had also issues with transactions, but honestly I don’t remember them exactly. The solution was to implement own way for looking up a transaction manager.

Future of the plugin

Current implementation of the plugin has its drawbacks, these include:

  • need for setup an additional Jenkins job for launching the business process build step with with test plan
  • only JBoss AS 7 with disabled jpa subsystem is usable for persistence

The best approach to solve this can jBPM remote handler implementation. This handler implements a work item handler in jBPM for JenkinsJob service task. It’s the same service task  as in jBPM workflow plugin, or at least with the same interface. You can supply it in the same way with the job name to launch, optionally with parameters in case of parameterized build and receive an object as a result. This is great since you can use both integration approaches and your business processes will remain the same. Only handler implementation changes.

Remote handler as the name suggest will start and evaluate Jenkins jobs remotely. Business process instance will run in jBPM console (simple process server) or a user application, which uses jBPM engine inside. jBPM console can also handle human tasks, for example for decision of the QA engineer like what to do if basic tests failed etc. The main benefit will be decreased complexity of the integration implementation and also there will be no limitations on application server for Jenkins.

Conclusion

I have learned a lot of technical stuff on this project. I have understood many concepts. But most importantly I have learned that it’s not a good idea to integrate two complex technologies directly on the application level. At least in Java you’ll end up with many classloading problems. If you can oppose this, please leave me a comment, thanks.

Analyze chess games on Linux

Club chess players are familiar with chess analysis. I started playing chess when I was a child. Later I began to play regular games for my local chess club. After a game you are usually encircled by a group of other players and they tend to show you possible better moves, especially when you lost. This can sometimes help you a bit. Unfortunately there are often too many hands, which change the position quickly and you can be confused. Or at least easily caught thinking about a less probable position.

Almost every player is interested in improving his technique and skills. Computer is your friend and is able to find the moves, that could be better in a given game position. Not need to mention that current computers are able to defeat even the best chess players. It’s not that surprising, if you realize that computers are better at tactics and also contain huge knowledge of many openings and thousands known games previously played. Human grandmasters were able to defeat them only by using more sophisticated strategy and planning. For me it was always boring to play against the computers due to obvious reasons. However I have learned a lot from my own mistakes, which were shown to me by a chess engine in analysis mode. For me the first such chess engine was Fritz.

Since I have been using Linux for several years now, I have tried free and open source chess engine Crafty. My steps summary is based on the following discussion. By the way I have found a really nice game played by two computers – Fritz against Crafty. Fritz won in that match, but I guess that no human player would be so brave to play like that!

Installation of packages

In Ubuntu type the following command:

sudo apt-get install crafty eboard scid

Crafty is the chess engine itself, Eboard is a graphical frontend for chess board and Scid is an analysis tool.

Adding the engine to Scid and running the analysis

Now you can simply run the chess analysis. Type scid in terminal or create a desktop icon for this application. In Tools->Analysis engine add the Crafty. Possible configuration:

  • Name: Crafty
  • Command: crafty
  • Directory: ~/.eboard
  • Uncheck UCI engine

That’s all. Now double click Crafty in the Analysis engine and the analysis runs!

Improving the engine

There are several ways how to improve the quality of Crafty. They are described in the linked Ubuntu forum post and simply said you have to download and build a database of chess games which improves Crafty’s strength based on the statistics of many already played opening positions.