fredag 19 november 2010

Comparing OSS offerings, like comparing apples to oranges

Rounding up the OSS integration frameworks described earlier is not easy and by no means a task of selecting an overall "winner". I've only scratched the surface on them, and most of the experience comes from testing simple samples, reading forums and blogs and in some cases implementing small solutions. There are also a bunch of frameworks that I haven't had time to test yet, so I'll focus on Mule, WSO2 and Apache Camel. Note that these are my personal reflections and experiences, so if you disagree completely, feel free to post so and call me a moron (maybe you're right :-)

Mule ESB
Marketed as the "most popular" open source ESB this was a good alternative to start with.
Mule is quoted as a stand-alone ESB and offers a wealth of connectivity options. The Mulesoft company behind it are very commited to the product and posts regular updates to the documentation as well as blogs.
However, the configurations are all designed by editing a quite cumbersome XML files which might offput some users wishing for less "bracket coding". Editing these files is only more constrained by the amount of schemas that needs to be included, i.e. each transport has its own schema and restrictions.
It supports a stand-alone runtime, hot deployments and extra functions such as registries and management consoles if you opt for the Enterprise license.
Mule was not created with EIP in mind from the startup, but updates in the most recent version made it easier to design flows through the "flow" construct even though I've already found some bugs while using it.

WSO2 ESB
Out of the three, wso2 is the only one that requires sort of an application server to run in the form of the Carbon server.
However, the management console is outstanding with its included support for statistics and registry. You also have the option to graphically design your integrations which is always good for beginners.
There are three things that bother me with WSO2 at the moment though:

- Big focus on the WS-stack, which is not surprising given the WSO2 team background. If you're after a webservice framework, this product is perhaps the most promising of the three, but as far as I see it might not be perfectly suited for other integration scenarios.

- Synapse. WSO2 is built upon Apache Synapse, a project that doesn't have as much community activity as the other two alternatives. Only the future will tell if it will grow in the same pace as the other ones, but I'm at least a bit concerned about its future.

- The need for an application server
This might be a political issue, but I'd think it will be more difficult to sell in a new application server to your management than a simple API runtime. Other might see this as a strength, but when you're already sitting on a stack of servers you'll undoubtley will get the question why you'll need another one.

Apache Camel
The most lightweight alternative of the three I've tested and probably the most flexible as well.
Since Camel is built as an implementation of the EIP patterns it fits right in to creating integration flows, which is the logical way of implementing integration.
With Camel you get alternatives wherever you go. Do you wish to develop your flows through XML? Or maybe java? Or perhaps Scala or Groovy? Would you like to execute it in a spring environemnt? Or as a routing component inside the ServiceMix OSGi runtime? Or inside a web application? Options everywhere... Which is acutally a good thing! Hopefully your team and organisation can settle with one and develop best practices using the one you prefer. Otherwise you could end up with a mess of implementations, but this holds true for all types of development environments, so its not a problem per se.

The only downside I could possibly think of when using camel is that since it's so easy to use you might start putting integration logic all over your projects instead of at a central ICC team of integrators and designers. Integration is a very complex area and should be handled by dedicated staff with end-to-end focus in mind. Camel with it flexibility and pragmatic approach might lead you to solving integration tasks as part of application projects, but as you can see I'm only nitpicking for a possible future problem here.

I especially like the fact that they've started to experiment with Scala as a a first-class citizen as I personally see a great future in Scala. Having this support from the start gives me hope that this component will have a long future even after Java has been declared as "dead" or legacy.

As noted these are just my first impressions of the frameworks and I might be proven wrong or have missunderstood some parts of them, so take these words with a grain of salt.
There are also a lot of alternatives that I haven't been able to try out extensively yet, for example JBoss ESB, ServiceMix and Spring Integration 2.0 which looks promising. So many toys, so little time...

See you soon!

onsdag 17 november 2010

Climbing the Camel


I remember my first ride with a live camel during one of our family vacations half a lifetime ago. It was a quite challenging task just to get up on the beast, and even hairier when this huge animal started to move. Rocketing back and forth it eventually managed to instill some sense of trust as it started to swaggle on with me sitting high atop on its back. After the first terror waned, the experience was actually simply a blast!



Well, recently I had the chance to climb up to the camel's back again, but now in another context. This camel was maybe even more scarier as I had never seen something quite like it and I was first startled by its complexity. But again, once it started moving I understood the pure beauty of the beast and felt safe and secure again.

I'm of course talking about Apache Camel, the lightweight OSS framework for designing and executing integration flows. It takes a pragmatic aproach to integration and has a very clean and efficient way of defining your integration logic between separate endpoints.
Camel is all about lightweight integration, and even though it does not qualify as a complete ESB since it misses some important parts such as manageability and hot deployments of your solutions, it certainly has some features which puts it right into the Enterprise integration space.

First of all one has to understand that camel is not an integration product per se, but rather an API for defining integration patterns. It does not execute by itself but should preferably be bundled together with some existing hosting environment i.e. JBI, Spring applications, your messaging engine, an application server or what not.

Setting up a Camel project is easy, since you could start out from Maven archetypes found on the Camel website. These will help you download all dependencies and create your Spring context. Camel does not force you to use either Maven or Spring, but much of the samples and tutorials on the net (and the documentation) assumes that you're already an experienced Maven/Spring user. After passing these first hurdles (which really had mostly to do with setting Maven up correctly) I was able to try out my first flows.

You have several options for choosing how to define your integration logic. The two major ones are through either Spring xml files similar to i.e. Mule ESB or a fluid Java API. There are also options for Scala and Groovy if that's your fashion. No matter which, the flow executes the same way as the design languages are only a means to model your integrations.

This separation is quite brilliant as it allows you to select the technology you are most proficient with, and the runtime will adapt. It also means that cross-functions such as automatic documentation is supported no matter which language you choose.

Our first flow


So, enough babbling. Lets have a look at an example:


public class ExampleRouteBuilder extends RouteBuilder {

/**
* A main() so we can easily run these routing rules in our IDE
*/
public static void main(String... args) throws Exception {
Main.main(args);
}

/**
* Lets configure the Camel routing rules using Java code...
*/
public void configure() {
from("jms:queue:purchaseOrder?concurrentConsumers=5")
.to("log:fromJms")
.split().xpath("//lineItem")
.choice()
.when(body().contains("Pen"))
.to("jms:queue:Pencils")
.when(body().contains("Paper"))
.to("jms:queue:Papers")
.otherwise()
.to("file:target/messages/invalid");
}
}

So, what is going on here?
Well, the first thing you'll notice is that we're extending the RouteBuilder abstract class and overide its configure() method. RouteBuilders are classes that configure your integration flows, or "routes" in camel language. The route itself starts with the from() method call and is described by chaining methods together.

Regular code completion assist then shows you which types of "Processors" you can apply to the route. A processor is basically some part that does something with a message exchange in a pipe and filter architecture. Apache comes bundled with processors for most of the EIP patterns such as the split defined in the example above, but you can of course add extra processors by just implementing the org.apache.camel.Processor interface or by sending the message to a spring bean method.

As you can see we've managed to include at least 4 EIP patterns in just a couple of lines (tecnically just one line of java, but you get the idea...) of code, namely the competing consumers, wiretap (the log endpoint), splitter and content-based router patterns. Now that's impressive!

The same flow could as stated above be modeled using XML in a Spring-like fashion by just including the Camel namespace and using content-assist in your favourite XML editor with the exact same outcome, so it's up to you to decide which method that suits you best.

Transports

Camel has the concept of adressing endpoints as URI's with the endpoint type first followed by the address and optional parameters on a query format. In our example we're instructing the route to use 5 concurrent threads to get the messages from the purchaseOrder jms queue.

The API relies on third party providers to implement the endpoint factories called components, so for our jms provider we could bind the jms transport to a component realised by i.e. Websphere MQ. This could either be performed by declaring the component through code, or as below as a spring bean in your context configuration.

<bean id="jms" class="org.apache.camel.component.jms.JmsComponent">
<property name="connectionFactory">
<bean class="com.ibm.mq.jms.MQXAQueueConnectionFactory">
<property name="hostName" value="localhost"/>
<property name="queueManager" value="QM_ESB"/>
</bean>
</property>
</bean>

This snippet declares that the MQXAQueueConnectionFactory should be the queueconnectionfactory for our jms endpoints. Just make sure that the application user has access rights to the queue manager by including the user either in the mqm group or some other group with connect/read access.

You could also define your endpoints as beans in the Spring XML file and just reference them by name, making the actual endpoints changeable depending on i.e. your different test environments setup.

So you could define your endpoints in a Camel context xml as:

<camel:endpoint id="PurchaseOrders.IN" uri="jms:queue:purchaseOrder?concurrentusers=5"/>

.. and then just reference this endpoint in your route as:

from("PurchaseOrders.IN")

This gives you a very good flexibility for choosing the actual endpoint location and even transport outside the actual flow.

Other cool features that Camel brings are for example

- Automatic type conversion:
Note how we could just use an Xpath expression even though we did not know the exact inbound object format? Camel automatically type converts between a bunch of different standard formats so you dont need to know whether the data is delivered as i.e. a bytearray, string, jaxb objects, xml stream etc.
If you don't find support for your particular format you can always write your own converters and easily plug these in to the runtime.

- A bunch of transport components out of the box:
We only saw two very common components in the example, but have a look at the http://camel.apache.org/components.html and you'll find that even the most obscure transport options are available.

- Bean support for i.e. defining a Spring bean as a service.

So, this was just a quick intro on the capabilities of Camel. I advise you to take a look at the number of articles written about it and get a feeling for this awesome framework at http://camel.apache.org/articles.html

'til next time,

Over and out!

tisdag 2 november 2010

First look at WSO2 ESB

Earlier this year I promised myself to educate me on the open source offerings for ESB's and BPM products out in the field.

As you may have noticed I took a first step into Mule ESB recently, a product that clearly suprises me, even though it is quite cumbersome to work with since it requires some fiddling around with XML files and a quite comprehensive stack of XML schemas. I guess it'll all become second nature to you once you get the hang of it.

Well, today I downloaded WSO2 ESB for the first time and my initial impression was just "WOW!". I knew that the guys at WSO2 are experts in their field, but for being an open source offering this product just instantly blew me away.

Right out of the box you have a solution with features such as Web management, tracing and statistics, pattern catalogs, registry, EIP support for advanced features such as splitting and aggregation, transformation options (XSLT, XQuery, Smooks, Custom java/ruby just to name a few) as well as the option to add in extra transport adapters as you see fit. And so the list goes on...

I've not yet scoured the full transport offerings, but to me it seems that Mule certainly has an upper hand when it comes to out-of-the box transports. But WSO2 wins my heart any day when it comes to manageability and ease of use.

I also noticed that WSO2 has it's own IDE plugins to Eclipse which is always handy. I'm currently downloading it and hope to see if it eases the integration configuration even more.

I'll also take a quick look into the BPEL editor and their runtime to see what's in store in that area as well.

So, have you used WSO2 ESB or any other Carbon products? If so, how do you compare it to other OS offerings such as Mule, JBoss, ServiceMix, Camel etc? I'd be interested to hear your thoughts on the stack since it does not matter how many bells and whistles it brings if it has not been proved in production.