Why Standards Matter

A month ago, the Why BPEL Matters article was posted on IT|Redux, and triggered a wave of discussions rarely seen in the little BPM microcosm. Initially, they focused on arcane mathematical considerations supported by pseudo-scientific arguments on both sides of the fence (here and there). But as the debate progressed, what is at stake for those opposed to BPEL as standard process execution language became clear, and the motivations for supporting their side of the argument painfully evident: disregard for the value of standards, and focus on the needs of vendors rather than the interest of customers (Cf. this article and that one). Let’s take a closer look at what this healthy debate brought under the spotlight.

In reading through these articles (here, here, here, and there), one can see that BPEL opponents are all essentially saying the same thing: “BPEL is an awfully complicated language,” and “I don’t see what BPEL brings to the table.” The first argument is supported by cute little BPMN diagrams and hugly pieces of BPEL code showing how difficult it is to generate valid BPEL code from BPMN flows. The second comes in many flavors, like “BPEL is only supported by a handful of vendors” (IBM, Oracle, and SAP among them, mind you), or “it does not matter what BPMN is translated into, as long as you can execute the process.”

First of all, let us agree with the cohort of BPEL opponents once in a while: BPEL is indeed an awfully complicated language, and generating BPEL code from BPMN flows is not for the faint of heart. We’ve said many times (here, here, and there) that BPEL was never designed to be used by human beings directly. It is to be produced by code generators automatically. Saying that some piece of code is awfully complicated or downright ugly is like saying that a fragment of Java Bytecode lacks the elegance of a few lines of Smalltalk code. Nobody will argue against that, but who should care about it? Nobody. Yes indeed, BPEL is complicated, and so is the generation of BPEL code from BPMN flows, which might be the reason why so very few companies managed to make it work (IBM, Oracle, SAP, and little Intalio). But the same is true for many powerful technologies, like a Java JIT compiler, or a VHDL synthesizer.

Second, let’s focus on the core question, which really is: “do we need a standard process execution language?” Forget about BPEL for a moment, and focus on this question instead. Whatever the standard could be, do we really need one? Answer this question, and the debate is essentially closed. Here, BPEL opponents split into two groups: the ones who could not care less about standards, and the ones who pay lip service to them.

Those who do not care about standards (like Tom Bayens from JBoss, Cf. article), are not only saying that a standard process execution language (like BPEL) does not matter, but a standard notation for business processes (like BPMN) does not really matter either:

“We believe that analysis diagrams can be created in many notations, BPMN being one of them.”
—Tom Bayens, Red Hat JBoss

I thought that BPMN was the one things that we could all agree on. Obviously, I was wrong. That being said, I’m not sure that such a position is considered of the BPM industry at large. After all, it is expressed by someone who works for a company developing tools for Java developers, and these do not care about notations for business processes. All they care about is code, and the only standards that matter to them are the ones defined by Sun and IBM under the Java umbrella. Here again, I will reach across the aisle, and agree that developers do not care about BPMN, and certainly do not want to write BPEL code by hand. Instead, they need a simpler language, and a way to bind it to any programming language (not just Java). This thing is being developed today by the Apache Software Foundation, and is aptly named SimPEL.

Those who pay lip service to standards follow the tradition of the Workflow Management Coalition and only support standards that support the interchange of non-executable models (like XPDL) and interoperability across proprietary products (like Wf-XML). To them, any standard process execution language is a threat, for it would open their niche markets to the competition, and brutally level the playing field. As much as I have sympathy for their fate, I cannot put my Darwinism faith aside. Customer needs matter more than vendor survival.

Keith Swenson concluded one of his articles by asking “Why again do analysts recommend BPEL?” This tells me two things: One, analysts have started to promote BPEL as a standard that vendors must support, and this is great news. Two, the question he is not asking is: “Why do we need a standard process execution language?” Please let me answer this all too important question, for it will be asked again and again.

Validity
Using a standard execution language for processes defined with a standard notation such as BPMN provides a deterministic way of validating the correctness of executables against models. Once the rules that define the translation of a model to an executable have been set and agreed upon, validation tools for code generators can be developed and leverage sets of standard unit tests developed by the industry. Such is the way mature industries like the database industry (SQL) or the electronics industry (Verilog, VHDL) are working today, and the BPM industry should definitely follow their lead.

Predictability
Using a standard process execution language grants predictability for the execution of process models. A given set of standard graphical process constructs enhanced with a proper set of standard execution descriptors will lead to a standard fragment of executable code, in a perfectly unambiguous way. Such predictability of execution is mandatory when processes are used in mission critical environments like air traffic control, military operations, or space exploration (all use cases currently supported by products based on BPMN and BPEL).

Portability
Using a standard process execution language provides portability of executables across process engines offered by different vendors. While portability of process models is desirable at the process modeling tool level, it is not enough without a standard way of translating these models into executables. Such portability of executables is especially important in the OEM market in order to allow Independent Software Vendors (ISVs) to build their applications on top of a process engine. The lack of a standard process execution language would lock ISVs to specific middleware vendors, thereby reducing the appeal of leveraging a third-party process execution engine. To a very large extent, the success of the ERP industry with vendors like SAP and PeopleSoft (now Oracle) hinged on the availability of a standard in the database industry (SQL), and the same will be true for the next generation of application vendors that will look for standards-based platform to build their applications on top of. Similarly, end-customers are increasingly looking for standards-based solutions that will give them independence from any particular vendor, and the ability to migrate their end-to-end process assets from one product to another will become a critical decision factor in their selection of a Business Process Platform (BPP).

Benchmarking
Using a standard process execution language allows process engines from different vendors to be benchmarked against each others, in an objective fashion. While business analysts might not care about things like performance, scalability, fault-tolerance, and quality of service, the people in charge of deploying and supporting the systems that execute business processes do. Without a standard process execution language, benchmarking proprietary process engines is akin to comparing apples to oranges. On the contrary, the availability of a standard process execution language will allow standard benchmarks to be developed. Such benchmarks were developed in the database industry (Cf. TPC), and provided invaluable benefits to customers looking for the database product that would best suit their needs. Similar benchmarks should be developed for the BPM industry, and the availability of a standard process execution language is a mandatory pre-requisite.

Progress
Adopting a standard process execution language will allow the industry to move forward faster, and better address the higher-level requirements of customers looking for ways to implement the vision of the process managed enterprise. The longer the industry debates about which standard is most appropriate for describing executable processes (a fairly low-level question to be sure), or whether such a standard is necessary to begin with, the longer it will take for the industry to tackle the challenges that should come next. To a large extent, the success of the Internet hinged on the availability of a well-accepted standard for exchanging packets of information across computer networks (TCP-IP). This standard was not perfect (some would argue that ATM would have been better suited), nor was it directly useful to business users (I cannot remember the last time I looked at a TCP-IP packet), but it was necessary for a complete stack of standards to emerge, and a plethora of interoperable products to become available. Without such a standard, companies like Cisco or Google would not exist today. The BPM industry needs a standard process execution language. Such a standard exists today. It is called BPEL, it is good enough, and it should be adopted as widely as possible.

The case has been made: we need a standard process execution language. While most industry observers agree that BPMN has estabished itself as the standard notation for business processes, it is not enough to provide a standard execution model for business processes, as explained in this article. In other words, an Executable BPMN Specification can only be defined on top of an executable process language, and the best candidate for such a job is BPEL, also known as WS-BPEL. Once we agree on such an approach, a lot of work has to be done in order to turn the vision into reality. Such an effort is advocated by BPMLab, and outlined in a set of clearly-defined projects. We invite you to join us in this effort.

What is BPM 2.0?

BPM 2.0 is a term originally coined by Bruce Silver (BPMS Watch) and defined by Ismael Ghalimi on this post. The concept was later refined through this other post. Many industry analysts agree with this definition, including its reference to WS-BPEL, which we believe is a requirement for the development of an executable BPMN model (Cf. BPMI.org Redux). What follows is a summary of its guiding principles.

Please let us know what is missing, or what might be unnecessary.

BPMI.org Redux

This article is an unedited version of this original article posted on IT|Redux.

A breakthrough, finally! After almost a month of back and forth discussions started by the now-infamous why BPEL matters post, and continued on InfoQ, the Workflow Patterns Google Group, and countless posts on this very blog, we finally reached a conclusion, drawn by Bruce Silver and relayed by David French. They are calling for the industry to develop a “Compliance Specification for BPMN” that would define the “elements, attributes, and flow patterns that must be supported to claim BPMN support.” What a great idea! Now, let’s take a look at how such a specification should be developed, and who could take the lead for such a project.

As of today, several things are missing to make BPMN a true standard in the industry. First, as Bruce Silver pointed out, BPMN’s semantics are “vague” and need to be clarified, especially with respect to the elements of the specification that should drive the generation of executable processes. Second, there is no standard interchange format for BPMN diagrams that would allow a diagram designed in one tool to be used in another. Third, if BPMN is to be used as overarching framework for describing executable processes, all “annotations” to the specifications that are required to turn a pretty picture into an executable process should be standardized as well, as well as the way to serialize (translation: translate into textual form) all these annotations.

To a certain extent, all this refers to the vision of “native BPMN engines” that has been promoted by some vendors lately. While the vision looks good on paper, what these vendors fail to realize (or advertise) is the fact that BPMN only deals with one dimension of processes: the flow of execution. But it totally overlooks another critical dimension of processes: data. How data flows through the process, how it is exchanged between processes, how it drives the execution of business rules throughout the process, etc. And guess what? Properly adding this data dimension to BPMN will lead to the complete re-invention of something that already exists: BPEL. So why not start from it, by using BPEL as default execution model for executable BPMN processes?

What Bruce and David call a “Compliance Specification for BPMN,” I would refer to as “Executable BPMN Specification.” It would unambiguously describe the elements, attributes, and flow patterns that must be translated into BPEL in order to support proper execution, and the ones that are there for documentation purposes only. With respect to the former, this specification should formally describe which BPEL code must be generated for which elements, attributes, and flow patterns, making the underlying BPMN to BPEL code generation fully deterministic and predictable. Such an effort should be supported by the proper formalisms, leveraging recent scientific papers such as the Formal semantics and analysis of control flow in WS-BPEL by Ouyang, Chun and Verbeek, Eric and van der Aalst, Wil M.P. and Breutel, Stephan and Dumas, Marlon and ter Hofstede, Arthur H.M. (2007).

In parallel to this effort, a group should work on a standard serialization format for BPMN. This format should be based on XML, canonically map the BPMN diagram’s graph structure, and be fully extensible so that vendors could add their proprietary decorations to it (font attributes, shape colors, icons, etc.). That’s the easy part. The difficult part is in understanding that we need two serialization formats, one for non-executable BPMN, and one for executable BPMN, the former being a subset of the later. A serialization format for non-executable BPMN can be developed right away, and should be a fairly straightforward exercise. All it would require is a thorough review of the current BPMN specification, and a fair amount of discussions among tool vendors in order to define the mechanisms by which the specification could be extended in order to support proprietary decorations. A serialization format for executable BPMN would not only require the completion of the first specification (for non-executable BPMN), but also the completion of the Executable BPMN Specification as well. Eventually, the three efforts should be combined into a single specification.

If this makes sense to you, please let me make some recommendations:

Properly define both ends of the equation
If your goal is to define a specification for executable BPMN, make sure to define what “executable” means, and there is no easier way to do that than by picking an execution language that already exists. As of today, BPEL is the most complete, most robust, and most widely supported, hence should be considered as the best candidate. Furthermore, this default execution language should be the only one you care about. The minute you start adding other options (like XPDL), the project goes off tracks, for you need to define an unnecessary abstraction layer. You go from meta to meta-meta, then from meta-meta to meta-meta-meta, and before you know it you pulled a German dictionary from your bookshelf in order to understand what über-meta means. Instead, just consider this exercise as solving an equation. For the equation to be solvable, you need to define both ends, BPMN on one side, BPEL on the other. Once you solved the equation, feel free to add variations to it, and see if the work you’ve done could be used with other target execution models (like XPDL). But make sure to solve the original equation first!

Do not worry about round-tripping between BPMN and BPEL
Round-tripping is a futile and wasteful exercise when attempted between two languages or notations that have a significant semantic gap between them, and so is the case for BPMN and any process execution language optimized for computers (like BPEL). The same is true between UML and Java by the way. Round-tripping should be set as a goal only between two languages or notations that are essentially representing the same thing, such as BPMN and its future serialization format for example. What this means is that there must exist a way to fully serialize BPMN diagrams on one hand, and a way to graphically display serialized BPMN on the other. And all this should work in a deterministic and predictable way, uniformly across tools.

Do not worry about the ugliness of the BPEL code
BPEL is to be produced by code generators and consumed by process virtual machines, not written nor read by human beings, so do not pay attention to the verbosity of the BPEL code resulting from the translation of executable BPMN. Of course, the BPEL code should be correct and as succinct as possible so that we can more easily apply formal validation techniques to it, but do not try to make it any simpler, for there is no point to it.

Improve BPEL whenever necessary
As many pundits indicated, some BPMN flow patterns cannot be translated to BPEL. In some cases, it’s because they should not, for the BPMN diagram is wrong — as in, it makes no sense at all. A smart BPMN modeling tool should detect such flawed patterns, and warn the business analyst that while beautiful, this flower-shaped graph is utterly meaningless. In other cases, the BPMN diagram is simply representing something that will not be executed by any kind of IT system, and no BPEL code should be produced. In yet other cases, the BPMN diagram can be translated into BPEL, but the resulting BPEL code will be ugly, and it would be preferable for the business analyst to change the BPMN diagram slightly in order to support cleaner BPEL code generation. A very smart BPMN modeling tool should detect such occurences, and make suggestions that could improve both the BPMN diagram and resulting BPEL code. Last but not least, in a few cases, the BPEL language might not be powerful enough to represent some BPMN constructs. I have yet to come across such instances, but let’s assume that they could arise, just for the sake of the exercise. If such were to be the case, take note of the limitation, and send it to the authors of the BPEL specification. If they do a good job, they might address the limitation you found in a subsequent version of the specification — BPEL 3.0, anyone? In the meantime, the non-executable pattern you stumbled upon should be added to the non-executable BPMN section of the specification, and marked as “not yet” executable.

If you follow these guidelines, you might get something out of the exercise.

Now, how do we get there? Well, if I’m the only one thinking that such an exercise would benefit the industry, there is not much point continuing the discussion, and all we need is for Intalio to better document the way we translate BPMN into BPEL. We already have a serialization format for BPMN (we donated it to the Eclipse Foundation), and we already generate valid BPEL 2.0 code for pretty much any BPMN diagram that makes sense.

But if there are other people out there who think that vendors and customers would both benefit from such a set of standards, we better start working on them now, and this begs the following question: where such a piece of work should be conducted? Which standards organization should host it? Should we propose it to an existing one, or should we create an ad hoc organization from scratch, and run with it?

Well, I have a fair bit of experience in this area, having founded the Business Process Management Initiative (BPMI.org) back in 2000, and chaired it for about four years. I have helped orchestrate its integration within the Object Management Group (OMG), and remotely followed their work for the past few years. I have also witnessed the evolutions of the Workflow Management Coalition, the W3C, and OASIS, and learned a few things along the way. From all this, I can safely say that the proposed exercise will fail if it is not conducted within a new group, which sole objective is to make it succeed.

I will leave to others the task of discussing why the aforementioned organizations are not suitable candidates for hosting such a project, and will just explain where the challenge lies: in trying to build this executable BPMN model, one has to bridge the gap between two specifications that are owned by two different groups: BPMN owned by the OMG, and BPEL owned by OASIS. Furthermore, this work has to be done in isolation from other specifications which integration would lead to failure, first among them XPDL owned by the WfMC. To make it perfectly clear, I have nothing against any of these fine groups of people, or the specifications they produced. I have strong opinions about their usefulness, but these should be for me to keep. The only thing I care about is the success of the project, for it will take significant resources to complete, and these resources do not come cheap, especially today.

So here is what I am proposing. If you’re interested by the notion of executable BPMN and believe that the easiest and fastest way to get there is to build it on top of BPEL first, then I invite you to join us (Intalio and a few others) in the creation of a new group tentatively called BPMLab. It will be a non-profit organization managed like the BPMI.org of the early days, which mission will be to develop standards for BPM. Membership will be open to anyone, will be affordable, and will get you in touch with people who share the same passion about BPM as you do. But make no mistake, BPMLab won’t be an advocacy group for BPM, or even less a place to learn about Business Process Re-engineering. There are plenty such groups out there, and there is very little we could add to their work. Instead, BPMLab will be a group of experts who can make meaninful contributions to the field of executable BPMN, in a constructive and pragmatic way.

BPMLab, anyone?

Why BPEL Matters

This article is an edited version of this original article posted on IT|Redux.

Following my recent post on BPM 2.0, Sandy Kemsley, one of the few independent analysts covering the BPM space, was quick to agree with our updated definition of BPM 2.0, at the exception of one point (out of sixteen): support for BPEL. The question of BPEL vs. XPDL, (or BPEL vs. nothing) has been one of the most debated topics in the decade-long history of BPM. Here is why BPEL matters.

First, let me make something clear, once and for all: nobody should ever try to write BPEL code by hand, unless they really have to. As far as I’m concerned, the ones that really have to are employed by BPM vendors whose products support BPEL. BPEL is an extremely powerful yet very complex language, and I don’t think there are more than a few hundred people around the world who can write decent BPEL code by hand. BPEL is a precise yet verbose language that was conceived to be produced automatically by code generators, not written by hand. In other words, BPEL is for computers, not human beings, at least not the ones I usually hang out with.

Second, should you really want to write code by hand, you should use a simplified version of BPEL, based on a less verbose syntax, and such a language is currently being developed by Intalio as part of the Apache ODE project. It’s called SimPEL, and it’s something that Ruby or Ruby on Rails folks will love.

Third, should you not want to write any code, you should move a step up the stack, and produce your BPEL code with a BPMN designer rather than a visual BPEL editor. The reason for it is pretty simple: BPEL is a complex language because of its semantics, not just because of its verbose syntax (which makes it complicated, but not necessarily complex), and a visual BPEL editor does very little to alleviate this complexity.

Now that we all agree that we should stay as far away from BPEL as possible, why on Earth does it matter that the core process engine runs BPEL code vs. XPDL, Java, or any proprietary process execution language devised by some BPM vendor? There are very many reasons for it, so let’s go through them one by one.

BPEL won the war of standards
Whether you like it or not, BPEL won the war of standards. While we (Intalio) lost a battle when BPML was supplanted by BPEL (because we had failed to secure IBM’s support for it), the war was eventually won when BPEL 2.0 was released (with Intalio’s support), and all major vendors adopted it against XPDL, including Microsoft, Oracle, and SAP. Some legacy workflow vendors might find delight in trying to re-play lost battles of the past (like some nostalgic French patriot dreaming of what could have been had Bonaparte’s army won the Battle of Waterloo — I know, I’m French), it’s time to move on. Really.

BPMN is not enough
Many people who fought for XPDL five years ago are now promoting the idea that neither BPEL nor XPDL really matters, and that all we need is BPMN. After all, BPM is for business analysts (wrong), and all they care about is what the process looks like. This argument is disingenuous at best. Executable processes cannot be designed with BPMN alone. A BPMN diagram is the process’ skeleton. For the process to walk (or run), it needs flesh around its skeleton. BPEL is the DNA that is used to produce it. Pick the wrong underlying language, and you end up with the wrong DNA. What’s missing in BPMN for it to describe executable processes? Data. Why is data important? Because a process without data is like a living being without language: it might walk or run, but it can’t speak nor listen. If your BPM vendor of choice is advocating that only BPMN matters, they’re using proprietary ways to make your processes fully executable, usually requiring a lot of custom code development, and this is the last thing you want for your processes. Whenever you find yourself listening to such a misleading marketing pitch, simply ask the following question: “Can you show me the code behind the boxes and arrows?”

BPEL is built upon the right mathematical model
A BPM vendor has three options to make processes executable: BPEL, XPDL, or their own language. If they use BPEL, they’re leveraging the Pi-Calculus mathematical model for supporting the execution of distributed and collaborative processes, taking advantage of one of its very powerful features, channel passing. If they use XPDL, they do not benefit from such a capability. If they use their own language or execution model, they do not benefit from the academic work that is validating and strengthening existing standards.

BPEL supports distributed transactions
Thanks to the notion of Scope, BPEL 2.0 supports distributed transactions that can be executed through the battle-tested two-phase commit protocol. Neither XPDL nor any proprietary process execution model I know of supports it, which makes them unsuitable for a broad range of enterprise applications. Again, you could hard-code distributed transactions within software components like Enterprise Java Beans, but this is not BPM (at least not BPM 2.0), it’s old-school OLTP.

BPEL is the standard
BPEL won the war of standards, and as such became the industry standard, and standards matter. You might not care to know about them, but you definitely care to know that your system is built on them. If you own a car, you’re glad that it’s running on standard gasoline, and that you can get your refill at any gas pump, operated by any oil & gas company — even though you hate the fact that gas is so expensive nowadays. It’s the same for a BPMS. If it’s built on top of BPEL, you’ll be able to take the processes you designed with its tool, and run them on another vendor’s engine. You cannot do that with a BPMS built on top of a proprietary execution model. Saying that BPEL does not matter when selecting a BPMS is like saying that SQL does not matter when selecting an RDBMS. Who in their right mind would buy a database engine that does not support SQL today? Why should it be any different with BPM and BPEL? This totally escapes me.

Executable BPMN does not exist
Whenever challenged with the argument above, non-BPEL vendors make the point that BPMN is all you need, and that executable BPMN is coming soon. Supposedly, OMG’s BPMI.org is working on something like that, and if you read the Business Process Definition Metamodel (BPDM) with a strong-enough dose of wishful thinking, you might find it between the lines of this arcane specification that nobody supports. Reality is, there is nothing like executable BPMN, and should anyone develop such a thing and did a good job at it, they would end up re-inventing BPEL 2.0, or something very close to it. In other words, the combination of BPMN and BPEL is all you need to make processes executable. The two members of this duo play very nicely together, but separate one from the other, and the whole thing falls apart.

Welcome to BPMLab

Eight years ago, I founded the Business Process Management Initiative (BPMI.org), which aim was to develop standards for Business Process Management (BPM). In the early days, it largely succeeded, and delivered the Business Process Modeling Language (BPML) and the Business Process Modeling Notation (BPMN). Eventually, BPMI.org merged within the Object Management Group (OMG), and did not manage to innovate beyond the release of BPMN 2.0. BPMLab is an attempt at continuing BPMI.org’s original effort, developing an integrated stack of standards for BPM based on BPMN and BPEL.