Manage process and service dependencies in an SOA environment
Background Do you know which services a BPEL process depends on? If different versions of a BPEL process are used, the dependencies between the two can quickly become more complex. The complexity of dependency management increases if we take into account the Enterprise Service Bus (ESB) services invoked by the BPEL process. Complexity makes deployment and testing time-consuming, difficult, and error-prone.
Often we end up using the Microsoft Visio modeling tool to manually diagram dependencies and scramble to update dependencies after every change in the process. This is a major hindrance to the agility of service-oriented architecture (SOA) infrastructures, which are designed to enable agile changes to business processes.
In this technical note, you'll learn how to successfully improve your build process and implement automatic generation of process dependency graphs.
Our challenge is to implement an Oracle SOA Suite demonstration project for a customer that contains many BPEL processes and references many BPEL sub-processes and ESB services. We ended up using a dozen BPEL processes and ESB services (which were defined as public services and shared on the service registry) as well as other proprietary BPEL processes and ESB services.
First, we decided to create an Ant-based deployment for all the services of the project, deploy the BPEL processes (including the test cases that execute them) to different environments (testing, integration, production), and also deploy the ESB services to the Ant-based these environments. Computer e-books free download
Requirements After completing the first project release, we have some requirements:
,
When a BPEL process or ESB service changes, we don't want to deploy all services of the project. So we need to move from project-centric deployment to a public service-centric deployment approach.
When deploying a BPEL process, all dependent (proprietary) sub-processes and ESB services are also automatically deployed.
To prevent overwriting a specific version of a process, deploy that version of the process only if it is not already deployed on the server. Overwriting will cause all instance flow information to be lost.
A visual graph of all process and service dependencies should be automatically created during deployment without the need to maintain this information separately. The visualization should look like this.
Open Questions In response to these new requirements, we asked the following questions and provided the following answers:
Question: Where is the information from process and service dependencies stored? Need to add more information to create a complete dependency graph?
Answer: All services called by the BPEL process are stored in the bpel.xml file marked by partnerLinkBinding. The called BPEL process and version information are also encoded into the URL. For example: Free download of computer e-books
<property name="wsdlRuntimeLocation">
${domain_url}/CustomerAccount_BES/1.3/CustomerAccount_BES?wsdl
</property>
The current version of the BPEL process to be deployed can be found in the build.properties file. For version maintenance of the BPEL process, we only need to change the rev property in the build.properties file.
To differentiate between public and private BPEL processes, the client's partner link within the bpel.xml file provides a new attribute "type" with a value of "public" or "private", as shown here:
<partnerLinkBinding name="client">
<property name="wsdlLocation">CustomerAccount_BES.wsdl</property>
<property name="type">public</property>
</partnerLinkBinding>
Question: Is there a tool or framework that can dynamically generate dependency graphs?
Answer: Yes, Graphviz ( www.graphviz.org ), an open source tool, can generate graphs from input files in text format.
Question: What are the options for referencing other BPEL processes?
Answer: The options are:
Reference the default version Reference a specific version Reference a UDDI service key (with or without encoded version information)
Question: How can all these new deployment requirements be implemented?
Answer: These new deployment requirements are best achieved using custom Ant tasks.
With the getting started questions answered satisfactorily, let's start by creating a custom Ant task. The bpel.xml file is parsed in the Ant task. For all partnerLinkBindings tags found, parsing of the corresponding bpel.xml file will begin recursively. In addition to parsing, the current value of the version property ("ref") is extracted from the build.properties file of each found BPEL project. One of the challenges of recursive parsing is skipping cyclic dependencies.
Our new "type" attribute and recursive parsing in the bpel.xml file implement all our needs for deployment with the service in mind.
Even better, we use Graphviz to solve the problem of automatic generation of dependency graphs. To achieve this, we will merge all the dependencies obtained from the recursive bpel.xml parsing into a single XML file as shown below. Computer e-books free download
<?xml version="1.0" encoding="UTF-8"?>
<BPELSuitcase>
<BPELProcess id="Resource_BAS_SetForAccount" src=" http://www.oracle.com/technology/tech/soa/soa-suite-best-practices/Resource_BAS_SetForAccount.bpel ">
<partnerLinkBindings>
<partnerLinkBinding name="client">
<property name="wsdlLocation">Resource_BAS_SetForAccount.wsdl</property>
<property name="type">public</property>
<property name="version">1.2</property>
</partnerLinkBinding>
<partnerLinkBinding name="RemoveFromAccount">
<property name="wsdlLocation">Resource_BAS_RemoveFromAccount.wsdl</property>
<property name="version">1.5</property>
</partnerLinkBinding>
...
</partnerLinkBindings>
</BPELProcess>
<BPELProcess id="Resource_BAS_RemoveFromAccount" src=" http://www.oracle.com/technology/tech/soa/soa-suite-best-practices/Resource_BAS_RemoveFromAccount.bpel ">
<partnerLinkBindings>
<partnerLinkBinding name="client">
<property name="wsdlLocation">Resource_BAS_RemoveFromAccount.wsdl</property>
<property name="type">public</property>
<property name="version">1.5</property>
</partnerLinkBinding>
<partnerLinkBinding name="CheckAvailability">
<property name="wsdlLocation">Resource_BES_CheckAvailability.wsdl</property>
<property name="version">1.1</property>
</partnerLinkBinding>
...
</partnerLinkBindings>
</BPELProcess>
<BPELProcess id="Resource_BES_CheckAvailability" src=" http://www.oracle.com/technology/tech/soa/soa-suite-best-practices/Resource_BES_CheckAvailability.bpel ">
<partnerLinkBindings>
<partnerLinkBinding name="client">
<property name="wsdlLocation">Resource_BES_CheckAvailability.wsdl</property>
<property name="type">public</property>
<property name="version">1.1</property>
</partnerLinkBinding>
...
</partnerLinkBindings>
</BPELProcess>
...
</BPELSuitcase>
Use XSLT to convert the merged XML files into a target format (.dot), which can be used as input to the Graphviz generator to generate a PNG image, as shown below:
digraph structs {
node [shape=record,fontname="Arial",fontsize="10"];
edge [fontname="Arial",fontsize="8"];
rankdir=LR;
labeljust=l;
"Resource_BAS_SetForAccount_1_2" [shape=record,label="{Resource_BAS_SetForAccount}|{1.2|public}",
fillcolor=yellowgreen,style=filled];
"Resource_BAS_RemoveFromAccount_1_5" [shape=record,label="{Resource_BAS_RemoveFromAccount}|{1.5|public}",
fillcolor=yellowgreen,style=filled];
"Resource_BES_CheckAvailability_1_1" [shape=record,label="{Resource_BES_CheckAvailability}|{1.1|public}",
fillcolor=yellowgreen,style=filled];
"Resource_BAS_SetForAccount_1_2" -> "Resource_BAS_RemoveFromAccount_1_5" [decorate=true,label=""];
"Resource_BAS_RemoveFromAccount_1_5" -> "Resource_BES_CheckAvailability_1_1" [decorate=true,label=""];
}
The dependency graph generated by the above DOT file looks like this:
The service name, version, and type are shown in the box. The box color indicates the service type. In this diagram we have three public (green) services. The diagram at the beginning of this article also shows the ESB service (white) and the BPEL service (orange).
After generating the ideal diagram for each public BPEL process, we need to implement automatic deployment of the relevant processes. For this service-centric deployment, we used the process list we generated earlier. The standard deployment target is called for each process in the list. To prevent overwriting existing process versions on the server, the deployment target is only called if there is no current version of the process on the server.
You can determine whether a specific process version is available by opening an HTTP connection to the process WSDL URL and then checking the returned status code (HTTP status 200 ? Deployed, HTTP status 404 ? Not yet deployed). Computer e-books free download
The output of our custom Ant task (containing recursive parsing), PNG image generation, and service-centric deployment looks like this:
...
[mkdir] Created dir: /MyProject/Resource_BAS_SetForAccount/doc
[bpeltask] ** STARTING DeployWithDependencesTask **
[bpeltask] Resource_BAS_SetForAccount-1.2
[bpeltask] ** PARAMETER **
[bpeltask] dotfilename: doc/bpel-recursiv-all.dot
[bpeltask] dotfilenamepublic: doc/bpel-recursiv-public.dot
[bpeltask] deploytarget: deploy
[bpeltask] deployment: true
[bpeltask] overwriting: false
[bpeltask] stoponalreadydeployed: false
[bpeltask] ** RESOVLING RECURSIVLY ALL bpel.xml FILES **
[bpeltask] [Resource_BAS_SetForAccount-1.2]
[bpeltask] [Resource_BAS_SetForAccount-1.2, Resource_BAS_RemoveFromAccount-1.5]
[bpeltask] [Resource_BAS_SetForAccount-1.2, Resource_BAS_RemoveFromAccount-1.5, Resource_BES_CheckAvailability-1.1]
[bpeltask] ** WRITING CENTRAL bpel.xml FILE **
[bpeltask] ** TRANSFORMING DOT FILE **
[bpeltask] ** SERVICE DEPLOYMENT **
[bpeltask] =============================================== =========================
[bpeltask] SERVICE ALREADY DEPLOYED (HTTP/1.1 200 OK) 'Resource_BES_CheckAvailability' in version '1.1' http://localhost:8888/orabpel/default/Resource_BES_CheckAvailability/1.1/ Resource_BES_CheckAvailability?wsdl
[bpeltask] =============================================== =========================
[bpeltask] =============================================== =========================
[bpeltask] SERVICE ALREADY DEPLOYED (HTTP/1.1 200 OK) 'Resource_BAS_RemoveFromAccount' in version '1.5' http://localhost:8888/orabpel/default/Resource_BAS_RemoveFromAccount/1.5/ Resource_BAS_RemoveFromAccount?wsdl
[bpeltask] =============================================== =========================
[bpeltask] =============================================== =========================
[bpeltask] SERVICE NOT YET DEPLOYED (HTTP/1.1 404 Not Found) http://localhost:8888/orabpel/default/Resource_BAS_SetForAccount/1.2/ Resource_BAS_SetForAccount?wsdl
[bpeltask] STARTING SERVICE DEPLOYMENT of Resource_BAS_SetForAccount in version 1.2
[bpeltask] =============================================== =========================
...
Conclusion Service-centric deployment that automatically generates dependency graphs meets all of our needs for SOA infrastructure transparency. Because the deployment is entirely based on Ant, we can also use it for our Luntbuild nightly and production build jobs (continuous build environment).
To support our customers' SOA governance, we will expose our public process and its documentation, along with the resulting dependency graph, in the customer's wiki, as shown here:
Wiki pages are dynamically generated when a lookup in the service registry retrieves the version of a currently registered public service. The figures in the wiki page provide links to the source repository (Subversion), which contains actual-size figures of the documentation and registered versions of our public services. Click on the thumbnail to retrieve the corresponding file from Subversion via WebDAV