jBPM 6 presentation at devconf.cz 2014

Hello all those who are interested in jBPM 6! I had a presentation called ‘Integration with jBPM 6’ at Developer Conference organized by Red Hat Czech Republic in Brno.

Main focus of the presentation was to shortly describe new major features in jBPM 6 business process suite and also to demonstrate its capabilities when it comes to embedding jBPM as a library into a web application.

The presented web application was rewards-jsf example, which demonstrates usage of new RuntimeManager API and CDI interfaces used together with Java Server Faces web framework.

My talk was recorded and you can see it on youtube:

Advertisements

jBPM 6 web application examples

After a long time, let’s see new technology in action. jBPM 6 was released in the end of the last year, so it’s quite fresh and still lacks good examples to easily start with. I’ll focus on my recent example projects, which demonstrate jBPM 6 in use as a workflow engine embedded inside a web application.

rewards-basic application

So far the git repo contains just two projects. The first one is called rewards-basic. It was developed by Toshiya Kobayashi, who created the original application using jBPM 5 and Java Enterprise Editon (EE) 5. I’ve rewritten his application to work with new jBPM 6 and also later standard Java EE 6. There are many useful improvements in the new engine. For example now you can use runtime managers with advanced session strategies to better separate process contexts or to gain a better performance, especially in concurrent environment. Seamless integration with Java EE world is done by Context and Dependency Injection (CDI). Many people still don’t like CDI, but once you learn it, you may realize its advantages. You may even continue to use jBPM without CDI and initialize your environment using ordinary Java constructors or factories.

Both applications are build around one simple business process. After start it creates a task for user John and after he approves his task, the process creates a second task for user Mary. After she approves her task, the process finishes. Example applications provide web interfaces to these operations – in particular to start a business process, to list human tasks available and to approve them. More information including steps how to run the programs can be found on Github link above.

ProcessBean class

This article focuses on internal structure suitable to software developers. Let’s start and take a look on a code snippet from a process service Enterprise Java Bean (EJB):

@Inject
@Singleton
private RuntimeManager singletonManager;

This demostrates CDI and how it can be used. You can just inject an object of defined class to your bean and you don’t have to care about how and where it is initialized. Don’t worry, we’ll get to that! The second annotation is interesting as well. You may choose also others for advanced session strategies like @PerProcessInstance and @PerRequest. Their names are self-explanatory, just to be sure – the first one keeps session (context) for a process instance and the second one is just stateless, no state is kept. What can be done with the runtime manager then?

RuntimeEngine runtime = singletonManager.getRuntimeEngine(EmptyContext.get());
KieSession ksession = runtime.getKieSession();
...
ProcessInstance processInstance = ksession.startProcess("com.sample.rewards-basic", params);

Keep in mind that runtime manager usually follows singleton design pattern in your application. For each request you should get an instance of runtime engine from it based on the context. Note that runtime engine is just an encapsulation of a kie session and a task service. The session can be used to start a process instance. The task service is automatically linked to this session and can be used for human task related queries and commands. The last command just starts a process instance, so the business process is finally running.

The last thing that can be noted are user transactions. You don’t have to use user transactions, but they are useful in many cases. For example you may want to save some information into a corporate database at the same time with the execution of the process engine. This way you ensure that all operations are either committed or rolled back together.

TaskBean class

Similarly you can just simply inject a task service. Do you remember jBPM 5? The task service was decoupled from the process engine and had to be connected to the engine for each process run in order to support human tasks. jBPM 6 integrated the task service back to the engine.

@Inject
TaskService taskService;
...
List list = taskService.getTasksAssignedAsPotentialOwner(actorId, "en-UK");

As you can see in TaskBean, you can easily run an arbitrary task service method.

CDI producer classes

In order to get CDI working you have to provide producers for injected class instance where necessary. One of the CDI advantages is the loose coupling. For example you may use a service in your beans, which functionality is dependent on running application server. This is ok, but how to easily unit test your application without setting up awkward application server? For example you can easily write a mock service for unit testing and don’t even have to modify a Java code. All you would have to do is just to change an alternative class in CDI configuration file beans.xml.

On the other hand debugging a CDI application may be difficult as the errors of unsatisfied dependencies are thrown mostly during run time, which slows the development process. If you know how to cope with this, please leave me a comment, thanks!

Application scoped producers

These producers can be found in ApplicationScopedProducer class. @ApplicationScoped annotation tells the application server (or more precisely the CDI container), that instances produced here should be instantiated for the whole life cycle of this web application deployment. It’s logical – for example the persistence unit (database connection) won’t change during the runtime of the web application.

Very important is also the RuntimeEnvironment producer, which provides whole setup for runtime manager, for example including our “com.sample.rewards-basic” process definition to be used. Runtime manager is injected from a jBPM library, but this same library also internally injects RuntimeEnvironment instance to get our environment that it doesn’t know.

You may also notice RewardsUserGroupCallback class. It is a simple user group callback and is defined as an alternative. This demonstrates that by using alternatives you may use your own classes, if you are not satisfied with prepared implementations coming from jBPM libraries.

Presentation layer

Web presentation layer is really simple. It uses Java Server Pages (JSP) technology and Java Servlets to handle presentation logic. Web user interface is just plain HTML, because the focus was put on demonstrating the internal service logic.

rewards-jsf

The second example application stays internally mostly the same, based on the same concepts. However Java Server Faces (JSF) technology including CDI is used for presentation layer.

Summary

I have described briefly two simple web applications demonstrating together jBPM 6 and Java EE 6. Please write a comment if you’ve liked them or if you have some feedback, I would be happy to hear your opinion. Thanks.

Working with PostgreSQL

Installation

PostgreSQL installation command on Fedora:

# yum -y install postgresql-server phpPgAdmin

It assumes that you have already Apache web server installed (package httpd)

Setup

PostgreSQL database needs to be initialized first.

# postgresql-setup initdb
# su postgres		
bash-4.2$ createuser --superuser -P pgadmin
bash-4.2$ createdb --encoding=utf-8 --owner=pgadmin pgadmin
# systemctl start postgresql.service
# systemctl enable postgresql.service

Also it is necessary to enable logging into phpPgAdmin:

# vim /etc/phpPgAdmin/config.inc.php
$conf['extra_login_security'] = false;

To change the PostgreSQL password:

$ sudo -u postgres psql
password pgadmin
q

Setting up PostgreSQL database can be tricky. Just remember that after installation there is a user called postgres, who is a user in the operating system. The most comfortable way is to login as superuser and then as postgres user. After that you are able to create a database superuser, for example pgadmin. It’s necessary to create a database for him. This is not enough, you have to disable default authentication methods in /var/lib/pgsql/data/pg_hba.conf and change them to md5. After that and restarting the service I was able to login into phpPgAdmin as a pgadmin user.

# su postgres
bash-4.2$ vim /var/lib/pgsql/data/pg_hba.conf
local   all             all                                     md5
host    all             all             127.0.0.1/32            md5
host    all             all             ::1/128                 md5
bash-4.2$ exit
# systemctl restart postgresql.service

Creating users and their databases

The easiest way is to login to phpPgAdmin as pgadmin user. First you need to create a role, which seems to be the same as user in phpPgAdmin terminology. Fill in the role’s name, password, password confirmation and don’t forget to check at least “Can login?” check box. Then create a database for this user, for simplicity it is good habit to use the same name as the name of the user (role). The latest thing is to grant privileges for the selected user on already created database.

Running SQL commands from CLI

This example will focus on a simple Bash script, which is able to reset (drop and create) a PostgreSQL database. The script may look like this:

#!/bin/bash
# PostgreSQL 9.1 pg_stat_activity.procpid, 9.2 has just pg_stat_activity.pid
psql -U pgadmin -c "update pg_database set datallowconn = 'false' where datname = '$DATABASE'; SELECT pg_terminate_backend(pg_stat_activity.procpid) FROM pg_stat_activity WHERE datname = '$DATABASE';"
psql -U pgadmin -c "drop database $DATABASE;"
psql -U pgadmin -c "create database $DATABASE owner $DATABASE encoding 'UTF8';"

Script assumes that you have already set up a super user pgadmin. Still there are few things to be done to run the script. The first thing is to edit the file pg_hba.conf and change authentication method to md5.

# vim /var/lib/pgsql/data/pg_hba.conf
local   all             all                                     md5
host    all             all             127.0.0.1/32            md5
host    all             all             ::1/128                 md5

Now you should be able to run the script, but for each command you would have to fill in the password. To remove this necessity, you can save the password in your home directory.

$ vim ~/.pgpass
localhost:5432:*:pgadmin:password

Please note that this is not safe to save the password as a plain text, but I believe that for a local testing instance of PostgreSQL database it is a satisfactory solution.

 

Caveats

Hints

  • This utility will remove orphaned large objects from the database, its diagnosis mode is:
    /usr/pgsql-9.3/bin/vacuumlo -n -v database_name

Tips for working with git

This article describes basic steps regarding to working with git and also gives several tips, which can be useful and are not usually covered in tutorials.

Typical workflow

To start working with git you can create own repo with git init or you can clone an existing repository, for example:

git clone git@github.com:droolsjbpm/jbpm.git

It’s recommended to prefer SSH url, because HTTPS connection is less reliable.

You can make changes to your project and add them to next commit. And then commit the changes. You should always write a brief description of your commit.

git add --all
git commit -m "Refactored module X to work with Y."

After making necessary commits you can share your changes with other by pushing them to the server. Usually the remote on the server is called origin or upstream. The convention is that main repo of the project for all people is usually called upstream and personal forked repo on Github is called origin. I think it is a good habit to also specify a branch you want to push. For example:

git push origin master

It is good idea to regularly check for changes from the others to work with current code and prevent conflicts. The first one just downloads the changes. The second one adds the changes of others to your master branch and puts your commits on the top as the latest ones. This principle keeps linear history and is called rebasing.

git fetch upstream
git rebase upstream/master

You may easily end with unwanted changes in the repository and there seem to be no steps to revert the state back. In this cases reset can help, but it clears your changes and makes your repository the same as remote.

git reset --hard

Getting information about your repository

Here are several useful commands to get more information about your repository. So to check status of your repository, list all branches or remotes:

git status
git branch -a
git remote -v

Sometimes it is necessary to check commit history of your project.  Log is the basic command, but much better is to install tool called tig. It allows you to inspect history in command line interface in more comfortable way than simple log.

git log
tig

Interested in who screwed something up? You can check who changed which lines in selected file.

git blame README.md

Working with branches

Branches are powerful feature of Git. Instead of Subversion, the branches are cheap and simple to use. To create a new branch “MyBranch” and start working with it, just type:

git checkout -b MyBranch

This will create new branch by copying the current branch and automatically the MyBranch is switched. To change branch, for example to work again with master:

git checkout master

You can create topic branches, make changes, push them to your origin and in Github web interface make pull requests to upstream. Sometimes there are changes done in the upstream during the work on a pull request. Then you cannot simply push, because when you rebase updates from master to your topic branch, the commit IDs will change. You can use a force push here:

git checkout MyTopicBranch
git rebase master
git push --force origin MyTopicBranch

However after you are done, you may need to delete old branches. The first one deletes remotely, the second one locally.

git push origin :MyTopicBranch
git branch -D MyTopicBranch

How to rebase a Github pull request to your project

Imagine the situation. Your project is hosted on Github. Somebody forks your repository and creates a pull request to your repo. Github offers you a green button “Merge pull request” if there are no conflicts to automatically merge the changes of a contributor to your repository. This will add the changes to your upstream but will create an additional merge commit. Better approach can be the git rebase operation, which has some significant advantages over merge. The main reason is clean and linear history of the commits. I do not know why, but Github does not have a button “Rebase pull request”. So to utilize advantages of rebase approach the following git command can simplify your life (taken from blog of Stuart Douglas):

git pull --rebase upstream pull/$no/head
git push upstream master

Where the upstream is the name of the Github remote where the pull request was created and $no is the identification number of the pull request. This way you can rebase the changes to your local working repository, check them, make corrections if needed, run unit tests and then push back to your upstream/origin on the server. Pushing to repository also automatically closes the pull request.

Creating a patch file

One of the simplest approaches is to commit your changes, create a patch file (for example for your co-workers) and reset hard to a previous commit:

git format-patch -M -1
git reset --hard previous_commit_id

Applying a patch

Later if we want to apply a patch file to our repository the steps are following:

git apply patch_file_name.patch

jBPM 5 Developer Guide – review

In this article I am going to describe my impressions from the book jBPM 5 Developer Guide by Packt Publishing. I think I should start with introducing myself. Of course, you can check my About page, but I’ll go a little bit deeper. I work as a jBPM quality engineer. I started with quality engineering of rule engine Drools, but later I’ve moved my attention towards jBPM 5, which had to be productized. I’ve started to learn jBPM in the end of 2011 and also I’ve registered my master project at school, where I focused on using jBPM to control Jenkins jobs, for example to execute test plans on Jenkins. So I know the jBPM only from the technical perspective, I have no experience with adopting Business Process Management (BPM) solutions at customers.

Why to use business processes?

After reading the book I have learned many new things, which I missed before. The book clearly describes motivation for learning and reasons for adopting BPM techniques. What I found most useful were the tips given by the authors, how to convince the employees of the target company to deploy such BPM solution instead of their traditional paperwork. I can imagine how hard this task may be as I have also some experience from developing and deploying a network management information system. The book is written from the developer’s perspective and gives you hints how to deal with these early problems, which you will surely face (as the IT experts).

Business process basics

From the start the book is not limited only to jBPM, but business processes are described in general. The same applies for the general description of the whole business process management system. jBPM 5 uses Business Process Management and Notation (BPMN) 2.0 standard and that’s one of the biggest differences from the last widely used version jBPM 3. The chapter about BPMN 2.0 could have also contained references to other books, which describe business process modeling more in depth, because one chapter cannot cover everything. In later chapter the jBPM 5 components are described, including differences from jBPM 3. Unfortunately sometimes I found some parts of the text describing too obvious things, which were easily visible from the many pictures. From these jBPM 5 components only the Designer is described in depth in its own chapter. The steps how to draft a business process are described from simple processes to complex ones, with advanced concepts like for example swimlanes,  messages, events and subprocesses.

Features important for everyone

If you were already familiar with some of the business process suites you could have skipped the previous chapters without severe complications, but chapters describing service tasks, human tasks, persistence and transactions are a must to read. The stress is put on the features, which can be implemented by a developer on its own on the top of the existing APIs. Author explains that you are not tied to the offered tooling, for example proposes you a way how to define your own frontends to your human tasks. These chapters also contain description of the services which are not present in the current jBPM 5.4 yet, but can be implemented in the future versions, so you can be prepared for these future features as well. Or which you can implement just now on your own.

Benefit of business rules and complex event processing

Next two chapters are more advanced topics, which you may or don’t have to use. It’s up to you and your use cases. They describe how to take advantage of the well working integration of jBPM with the rule engine Drools Expert and Drools Fusion (complex event processing). Rule engine is useful for evaluating complex business logic which may often change (for example mortgage calculations, complex discount logic, …) and event processing agent can be used to detect a complex event from an event stream and based on that start a business process (for example detecting a fraud and launching business process for its investigation).

Summary

To sum it up – it’s definitely a very good book for everyone who is new to the jBPM 5 and its ecosystem. The information, guides and problem solutions are often scattered from documentation, wiki pages, blogs to user forum and this book gives you all important information nicely put together in one place.

Pros:

  • All important information nicely put together, currently there is nothing better to start learning jBPM 5 from
  • Developer’s perspective and hints
  • Not only about jBPM, but also describes business rules, complex events and integration options to give a complete picture

Cons:

  • Sometimes there are empty sentences telling nothing new
  • Some Java code examples are not obvious and easy to understand
  • Lack of clustering topics, steps how to scale jBPM

First release of jBPM workflow plugin

Yesterday I released the first version of my jBPM workflow plugin. I started research and development of this JenkinsjBPM integration at the university in November 2011. And my company let me to continue work on my project. That’s great, because this is one of the things, which keeps my job from becoming monotonous and also I have learned a lot of knowledge by working on this project.

Continuous integration helps many companies in their software development. Jenkins is one of the most widespread CI servers. It’s used for several purposes – building software, running tests, publishing results, software analysis, etc. As a quality engineer I have mostly experience only with Jenkins as a tool for running many matrix tests. Axis include operating systems, Java development kits (JDK), databases, Java content repositories (JCR) and web browsers. Many test types are included too like unit, regression, integration, performance and others. And as you continuously work on something, you tend to optimize your steps.

So back to start – I had to apply for a master project and found an interesting topic. The task was to create a Jenkins plugin, which would enable to run test plans by using business processes to improve efficiency of the quality engineering department. If you spend less time with starting, monitoring and evaluating Jenkins jobs, then you have more time for other tasks, for example improving test coverage of a product. I defended the master’s project and continued to improve it.

Now the early release of the plugin has been done. However current solution still has several limitations. It can fully run only in JBoss AS 7 container. What is meant by fully? That means running business process with persistence enabled. That’s necessary if you want to easily recover the latest state from database in case of unexpected outage. Theoretically the plugin should run without persistence on any container, but I have not verified that. And also it’s necessary to mention that the JBoss AS needs its jpa subsystem to be disabled. That can be real limitation if you host more applications than Jenkins in one application server instance.

Problems that had to be solved

I’ll try to summarize problems I faced with the implementation of the plugin. The problems were often related to classloading.

Classloading issues

I had to face many classloading problems. For example I was not able to run the tests, I had to always use -DskipTests option in Maven command. One of the reasons was the XStream library – both jBPM and Jenkins use it. Unfortunately Jenkins uses its own changed version and jBPM another version. Only one of them was classloaded, so I could not run tests because Jenkins test suite could not find its specific method in its changed XStream library.

Could not find persistence.xml

This happened because of complexity of combination of Jenkins and JBoss classloading. Jenkins uses several classloaders and one of them is dedicated for plugins. So the Jenkins .war is deployed in the application server, but deploys .hpi file by its own mechanism. This was solved by changing context classloader of the plugin thread to that dedicated classloader.

Need to eliminate provided persistence provider as a result of inability to configure it properly

After many failures I have decided to supply hibernate .jars directly with the plugin and od not depend on the ones provided by JBoss AS. But running two Hibernate frameworks together could not work. I have tried many things how to disable classloading of Hibernate into Jenkins or into my plugin using the jboss-deployment-structure.xml. This file should exclude or include specific classpath elements specific for your application. Unfortunately without success. So I had to disable jpa subsystem in standalone.xml configuration file. This way correct Hibernate is loaded and is able to find the persistence.xml file. If you know, how to better control classloading (for example by jboss-deployment-structure.xml), feel free to leave me a comment, thanks.

Transaction issues

I had also issues with transactions, but honestly I don’t remember them exactly. The solution was to implement own way for looking up a transaction manager, since I have decided not to depend no provided persistence system.

Future of the plugin

Current implementation of the plugin has its drawbacks, these include:

  • need for setup an additional Jenkins job for launching the business process build step with with test plan
  • only JBoss AS 7 with disabled jpa subsystem is currently usable for persistence

The best approach to solve this can jBPM remote handler implementation. This handler implements a work item handler in jBPM for Jenkins Task (service task). It’s the same service task as in jBPM workflow plugin, or at least with the same interface. You can supply it in the same way with the job name to launch, optionally with parameters in case of parameterized build and receive an object as a result. This is great since you can use both integration approaches and your business processes will remain the same. Only handler implementation changes.

Remote handler as the name suggest will start and evaluate Jenkins jobs remotely. Business process instance will run in jBPM console (simple process server) or a user application, which uses jBPM engine inside. jBPM console can also handle human tasks, for example for decision of the QA engineer like what to do if basic tests failed etc. The main benefit will be decreased complexity of the integration implementation and also there will be no limitations on application server for Jenkins.

Conclusion

I have learned a lot of technical stuff on this project. I have understood many concepts. But most importantly I have learned that it’s not a good idea to integrate two complex technologies directly on the application level. At least in Java you’ll end up with many classloading problems. If you can oppose this, please leave me a comment, thanks.

My first release!

Yesterday I released the first version of my jBPM workflow plugin. I started research and development of this JenkinsjBPM integration at the university in November 2011. And my company let me to continue work on my project. That’s great, because this is one of the things, which keeps my job from becoming monotonous and also I have learned a lot of knowledge by working on this project.

Continuous integration helps many companies in their software development. Jenkins is one of the most widespread CI servers. It’s used for several purposes – building software, running tests, publishing results, software analysis, etc. As a quality engineer I have mostly experience only with Jenkins as a tool for running many matrix tests. Axis include operating systems, Java development kits (JDK), databases, Java content repositories (JCR) and web browsers. Many test types are included too like unit, regression, integration, performance and others. And as you continuously work on something, you tend to optimize your steps.

So back to start – I had to apply for a master project and found an interesting topic. The task was to create a Jenkins plugin, which would enable to run test plans by using business processes to improve efficiency of the quality engineering department. If you spend less time with starting, monitoring and evaluating Jenkins jobs, then you have more time for other tasks, for example improving test coverage of a product. I defended the master’s project and continued to improve it.

Now the early release of the plugin has been done. However current solution still has several limitations. It can fully run only in JBoss AS 7 container. What is meant by fully? That means running business process with persistence enabled. That’s necessary if you want to easily recover the latest state from database in case of unexpected outage. Theoretically the plugin should run without persistence on any container, but I have not verified that. And also it’s necessary to mention that the JBoss AS needs its jpa subsystem to be disabled. That can be real limitation if you host more applications than Jenkins in one application server instance.

Problems that had to be solved

I’ll try to summarize problems I faced with the implementation of the plugin.

Jboss AS 7 Hibernate was not able to find persistence.xml of my plugin

This happened because of complexity of combination of Jenkins and JBoss classloading. Jenkins uses several classloaders and one of them is dedicated for plugins. So the Jenkins .war is deployed in the application server, but deploys .hpi file by its own mechanism. This was solved by changing context classloader of the plugin thread to that dedicated classloader.

JBoss AS 7 had a tendency to supply its own Hibernate

After many failures I have decided to supply hibernate .jars directly with the plugin and od not depend on the ones provided by JBoss AS. But running two Hibernate frameworks together could not work. I have tried many things how to disable classloading of Hibernate into Jenkins or into my plugin using the jboss-deployment-structure.xml. This file should exclude or include specific classpath elements specific for your application. Unfortunately without success. So I had to disable jpa subsystem in standalone.xml configuration file. This way correct Hibernate is loaded and is able to find the persistence.xml file.

Transaction issues

I had also issues with transactions, but honestly I don’t remember them exactly. The solution was to implement own way for looking up a transaction manager.

Future of the plugin

Current implementation of the plugin has its drawbacks, these include:

  • need for setup an additional Jenkins job for launching the business process build step with with test plan
  • only JBoss AS 7 with disabled jpa subsystem is usable for persistence

The best approach to solve this can jBPM remote handler implementation. This handler implements a work item handler in jBPM for JenkinsJob service task. It’s the same service task  as in jBPM workflow plugin, or at least with the same interface. You can supply it in the same way with the job name to launch, optionally with parameters in case of parameterized build and receive an object as a result. This is great since you can use both integration approaches and your business processes will remain the same. Only handler implementation changes.

Remote handler as the name suggest will start and evaluate Jenkins jobs remotely. Business process instance will run in jBPM console (simple process server) or a user application, which uses jBPM engine inside. jBPM console can also handle human tasks, for example for decision of the QA engineer like what to do if basic tests failed etc. The main benefit will be decreased complexity of the integration implementation and also there will be no limitations on application server for Jenkins.

Conclusion

I have learned a lot of technical stuff on this project. I have understood many concepts. But most importantly I have learned that it’s not a good idea to integrate two complex technologies directly on the application level. At least in Java you’ll end up with many classloading problems. If you can oppose this, please leave me a comment, thanks.