Tuesday, December 3, 2013

Creating async request reply web service over jms on WebSphere (SOAP over JMS)

Introduction

I started with this developerworks tutorial and its files.

It has a section Asynchronous JAX-WS Web service invocations with SOAP over JMS, but the article is not doing any implementation, it is just a theoretical explanation. Let me show you how to call the web service asynchronously using soap over jms.

Software used:
  • IBM Integration Designer 7.5.1 
  • IBM Process Server 7.5.1.0 on WAS 7.0.0.19

Service code generation

Generate your server code as described in the tutorial. Use "Top down EJB Web Service" to create a soap over jms service. The same service code is used for both sync and async calls.

Client code generation

Right click the wsdl file in RAD or WID, and select web services > generate client

Make sure you flag "enable asynchronous invocation for generated client"



Now in the client code, the *Async operations are important. Three methods are generated for each request reply operation in the wsdl.

This wsdl does not contain one way (fire and forget) operations, but if it did, it would result in one method in the client code. Because one way operations are alway async when using JMS, they have only one method in the generated class.

One method (without Async postfix) is for sync calling, one returns a response handler (for polling), and the third accepts a callbackhandler.

The standard synchronous method

public EchoStringResponse echoOperation(EchoStringInput parameter);

The asynchronous method with polling

public Response<EchoStringResponse> echoOperationAsync(EchoStringInput parameter);

The asynchronous method with callback

public Future<?> echoOperationAsync(EchoStringInput parameter, AsyncHandler<EchoStringResponse> asyncHandler);

Calling with response

Use the response wrapper to check if the async call is finished. While it is not finished, other tasks can be executed.

Response<EchoStringResponse> asyncresp = service.getEchoServicePort().echoOperationAsync(input);

while (!asyncresp.isDone()){
    System.out.println("no response yet, let's have a nap");
    try {
        Thread.sleep(1000);
    } catch (InterruptedException e) {

         System.out.println("failed to nap :(");
    }
}
try {
    System.out.println("I got a response! request send: " + request.getParameter("input") + ", got response : " + asyncresp.get().getEchoResponse());
} catch (Exception e) {
    e.printStackTrace();
}

Calling with callbackhandler

You can do the same with a callbackhandler, if it is not necessary to process the response in the same thread.

input.setEchoInput(input.getEchoInput() + " with callback");

service.getEchoServicePort().echoOperationAsync(input, new AsyncHandler<EchoStringResponse>() {

    @Override
    public void handleResponse(Response<EchoStringResponse> res) {

        try {
            System.out.println("I got a callback response! " + res.get().getEchoResponse());
        } catch (Exception e) {
            e.printStackTrace();

        }
    }});

Resources

This article in the infocenter talks about setting JVM properties for the async message listener, but I don't really understand what it is about, or why you should set these properties, it works fine without them, just define a reply queue: http://pic.dhe.ibm.com/infocenter/wasinfo/v7r0/topic/com.ibm.websphere.nd.multiplatform.doc/info/ae/ae/twbs_jmsasyncresplistener.html

More info on Invoking JAX-WS Web services asynchronously: http://pic.dhe.ibm.com/infocenter/wasinfo/v7r0/topic/com.ibm.websphere.base.doc/info/aes/ae/twbs_jaxwsclientasync.html

It makes the following statement:
To enable asynchronous mappings, you can add the jaxws:enableAsyncMapping binding declaration to the WSDL file.

I added that to the wsdl, and after I did the generated client will ALWAYS had the asynchronous method, even when I didn't check "enable asynchronous invocation for generated client" in the client generation wizard.


code:
<jaxws:bindings>
    <jaxws:enableAsyncMapping>true</jaxws:enableAsyncMapping>
</jaxws:bindings>

Don't forget to add the namespace declaration to the wsdl


code:
xmlns:jaxws="http://java.sun.com/xml/ns/jaxws"


If the server code is generated with the enableAsyncMapping extension added to the wsdl, you will find that also in the web service implementation class async methods have been generated

public Response<EchoStringResponse> echoOperationAsync(EchoStringInput parameter){
    // TODO Auto-generated method stub
    return null;
}

These are not valid at server side. You will get a stacktrace like this in the systemout :

[2/12/13 11:28:41:442 CET] 00000083 JMSListenerMD E   WSWS3018E: Caught exception during request processing: WSWS3163E: Error: The Web services engine could not find a target service to invoke!  targetService is EchoServicePortType
[2/12/13 11:28:42:173 CET] 0000007f SystemErr     R java.util.concurrent.ExecutionException: javax.xml.ws.soap.SOAPFaultException: WSWS3163E: Error: The Web services engine could not find a target service to invoke!  targetService is EchoServicePortType
[2/12/13 11:28:42:173 CET] 0000007f SystemErr     R         at org.apache.axis2.jaxws.client.async.AsyncResponse.onError(AsyncResponse.java:141)
[2/12/13 11:28:42:173 CET] 0000007f SystemErr     R         at org.apache.axis2.jaxws.client.async.PollingFuture.onError(PollingFuture.java:95)
[2/12/13 11:28:42:173 CET] 0000007f SystemErr     R         at org.apache.axis2.description.OutInAxisOperationClient$NonBlockingInvocationWorker.run(OutInAxisOperation.java:470)
[2/12/13 11:28:42:173 CET] 0000007f SystemErr     R         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:897)
[2/12/13 11:28:42:173 CET] 0000007f SystemErr     R         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919)
[2/12/13 11:28:42:174 CET] 0000007f SystemErr     R         at java.lang.Thread.run(Thread.java:736)


You can remove these *Async postfixed methods in the implementation and interface to resolve this issue.

Thursday, June 20, 2013

"Magic map" in WESB mediation flows

IBM's Integration Designer (I'm using version 7.5.1) has many quirks, which you will soon discover as you start making large and complex mediation flows.

With trial and error I learned that for a certain type of errors, a "magic map" helps. What is this magic map you say? Nothing more than an xsl transformation that moves the root elements to the root elements:

Magic map, sounds nice! Suits well with my flowers and unicorns.

But what kind of errors does it solve then? It seems that after executing an xslt, the esb's internal dataobject representation can go corrupt.

When does this happen? I don't know. Sometimes with simple maps, sometimes complex maps,  sometimes when the message enters the flow and gets deserialized.

How do I know when it happens? Well, let me show you some examples.

Example 1: when you test your component with the integration test client, and you have a fine-grained trace running, you can see the result of each primitive in this "mediation message" window on the right. If you click on a a primitive in your trace, and you are missing an SMO element (context, headers or body), you know something is wrong with the DataObject:


If you try to print the DataObject with a trace primitive, or you do anything else with it that causes serialization, you will get:

CWSXM3154E: Mediation primitive java.lang.RuntimeException: Error serializing BO

Putting a magic map right after the primitive that "looses" one of the SMO's root elements, will fix the DataObject.


Example 2: Take a look at this mediation flow
How simple can it be? The map just moves the input. However, this exception ruins the fun when executing:

CWSXM0202E: An
unexpected exception occurred when processing mediation flow for component
MyService_MED in module mymodule: CWSXM1025E: An unexpected exception occurred
when the flow was called: IXJXE0920E: [ERR API0136] Execution terminated
because of the following error: 'IXJXE0465E: [ERR 0414][ERR XTDE0410] The
result sequence used to construct the content of an element node contains a
namespace node or attribute node named 'xsi:type' that is preceded in the
sequence by a node that is neither a namespace node nor an attribute node.  Attribute and namespace nodes must precede
all other kinds of nodes in the sequence used to construct the content of an
element.; SystemID: xsltcl:/xslt/MyService/myOperation/map_req_1.xsl; Line#:
96; Column# 149'
Refer to the server logs for more information.

Indeed, the input type is actually an abstract type, and an inheriting type should be used. In the xml of the message, xsi:type indicates the instance's "class". Hmm, so somehow the internal representation is wrong? Magic map!


With the magic map the mediation flow works like a charm. Pun not intended.


Monday, April 22, 2013

Encrypt Message Using WS-Security in Websphere 7: InvalidKeyException

I tried to encrypt a soap message using WS-Security with an SSL certificate I got from the service provider.

My runtime is a WebSphere ESB 7.5.1.1 running on WAS 7.0.0.27

After configuring the policy set and the client policy set binding, I couldn't get rid of the following exception when executing the web service client:

Exception:
javax.xml.ws.WebServiceException:
com.ibm.wsspi.wssecurity.core.SoapSecurityException: CWWSS5612E: Encrypting the
data produced the following exception: Wrong length: 162:
java.security.InvalidKeyException: Wrong length: 162 Message: com.ibm.wsspi.wssecurity.core.SoapSecurityException:
CWWSS5612E: Encrypting the data produced the following exception: Wrong length:
162: java.security.InvalidKeyException: Wrong length: 162

After trying a lot of configuration changes (which is very tedious because each change in a policy set or client policy set binding requires a appserver restart), I finally found the cause: in the encrypter configuration I assumed "Data encryption" was the correct setting for encrypting the data of the message, but the infocenter clearly states that "Data encryption" is used for symmetric encryption only.


Changing the setting to "Key encryption" solved the InvalidKeyException!

Information center info about the setting:

Usage of key information reference
This field is available on this panel if you are configuring encryption protection and it specifies that the encryption key information is either data encryption key information or key encryption key information. Select Data encryption for symmetric algorithms and Key encryption for asymmetric algorithms.
 

Thursday, March 28, 2013

Tips and tricks for WebSphere ESB development - Always use xsl transformations on root level


I'm currently working in Integration Designer 7.5 on WebSphere ESB 7.5

Creating xsl maps I always end up having trouble with xsl transformations that specify a different root path. For example, a transformation with message root  "/context" works fine locally using "test map", but running on the server it fails with this error:

MyService_MED in module mymed: CWSXM1025E: An unexpected exception occurred when the flow was called: 
com.ibm.ws.box.bomodel.impl.BusinessObjectImpl incompatible with com.ibm.websphere.sibx.smobo.ContextType

Remaking the transformation with root "/" solved the issue (no other modifications were made).

New xsl transformation file dialog
New xsl transformation file dialog
 
I had other similar issues with xls transformations on "/body" or "/context" message root level before, so I made it a best practice: always define the root "/" as message root!

Besides, if you only need to modify the context, it is very easy to copy the body element with one move transform statement!

Tuesday, February 12, 2013

Troubleshooting IBM WebSphere ESB Development: Cannot retrieve information for class in XSLT

Working on a mediation module in IBM Integration Designer 7.5.1, I was testing my module continuously, and all of the sudden following stacktraces  were showing in the SystemOut.log

[12/02/13 15:10:23:802 CET] 00000047 StaticMethodI W com.ibm.xltxe.rnm1.xylem.instructions.StaticMethodInvocationInstruction typeCheck IXJXE0678E: [ERR 0614] The processor has encountered an internal error condition.  Please report the problem and provide the following information: Cannot retrieve information for class com.blogger.arendatwork.DateTool and method formatDate
java.lang.NullPointerException
at com.ibm.xltxe.rnm1.xylem.instructions.StaticMethodInvocationInstruction.typeCheck(StaticMethodInvocationInstruction.java:443)
at com.ibm.xltxe.rnm1.xylem.instructions.LetBaseInstruction.typeCheck2(LetBaseInstruction.java:245)
at com.ibm.xltxe.rnm1.xylem.instructions.LetBaseInstruction.typeCheck(LetBaseInstruction.java:224)
at com.ibm.xltxe.rnm1.xylem.instructions.ChooseInstruction.typeCheck(ChooseInstruction.java:247)
at com.ibm.xltxe.rnm1.xylem.instructions.LetBaseInstruction.typeCheck2(LetBaseInstruction.java:245)
at com.ibm.xltxe.rnm1.xylem.instructions.LetBaseInstruction.typeCheck(LetBaseInstruction.java:224)
at com.ibm.xltxe.rnm1.xylem.Function.typeCheck(Function.java:485)
at com.ibm.xltxe.rnm1.xylem.Function.ensureTypechecked(Function.java:981)
at com.ibm.xltxe.rnm1.xylem.Function.instantiate(Function.java:772)

Republishing or restarting the server brought no solace.

There was only one xslt map that used the specified java class, so I had a look at that map. 


Trying to test the map using the build in test map function,failed, giving a similar message. Now that I used test map, the compiler finally marked errors in the xslt map file. Turned out that by adding a submap in the map, the editor had deleted the import of the java class. Somehow this gave no errors, and the module could be deployed without problems. At runtime the class could not be found for the mapping, and stacktraces were printed. However, this does not fail your mediation.


It is a good practice to test your xslt mapping locally, this will compile and test your xslt file before deploying, and thus saves a lot of time debugging and troubleshooting the transformation.


Friday, February 8, 2013

Tips and tricks for WebSphere ESB development - Start Your Mediation Flow On Paper


Starting with this article: a new series of tips and tricks for WebSphere ESB development, in Integration designer.

I'm currently working in Integration Designer 7.5 on WebSphere ESB 7.5, in short ID75 en WESB75.

Creating mediation flows in the designer might seem simple when looking at a demo, or following some tutorial. But real life implementations are not always that easy, and ID75 has its own quirks.

So my first tip is about starting your mediation flow. The integration designer in Business integration perspective is a  tool that visualizes your flow, and the pitfall here is that you want to develop straight away. It gets harder and harder along the way, you add some fan outs, need extra variables in your shared and transient context. 
Integration Designer 7.5 does not like it when business object change all the time. Sometimes the changes are "detected" in the visual editors like the xsl transformation primitive or the message element setter primitive, but most of the time ID75 doesn't  recognize the extra fields in the auto-complete, and you are obliged to open the mediation flow in text mode to add the primitive settings to the XML manually.

It helps A LOT to draw your mediation flow ... on a piece of paper, before starting development in ID75. Draw the blocks you have in mind, detect fan outs, recognize where you will need variables in the different contexts.


  • A fan out with a service call in it: copy the list over which you iterate to transient context
  • If you fan out and need a result of each iteration, copy the result to a shared context variable
  • And so on. Implement a few mediation flows, and you will automatically recognize the fields you will need.


Make a list of these variables, create a DataObject for each context in ID75 and add the variables as fields.
Define your context DataObjects in the details properties of the input node, for easy recognition and auto-complete during implementation.



Drawing a flow on paper speeds up your development, you make less mistakes, you have less hassle with ID75 not recognizing added fields, and you will have less rework because you forgot a loop somewhere.

 

Friday, January 18, 2013

WebSphere Lombardi Edition on SQLServer 2010 - No message engine was found

I installed WebSphere Lombardi Edition 7.2 win32 edition using a Microsoft SQL Server 2010 database.

After installation the database tables and content were verified (the default users should be present, otherwise you cannot log in, see previous blogpost). All tables were created, with the data present.

However when starting the twprocsvr profile, the following stacktrace came up:
Caused by:
com.lombardisoftware.client.delegate.BusinessDelegateException: CWSIA0241E: An
exception was received during the call to the method
JmsManagedConnectionFactoryImpl.createConnection:
com.ibm.websphere.sib.exception.SIResourceException: CWSIT0008E: A successful
connection was made to the bootstrap server at
srv-lom-01:7277:BootstrapBasicMessaging but the server returned an error
condition: CWSIT0088E: There are currently no messaging engines in bus
twprocsvr_bus running. Additional failure information: CWSIT0103E: No messaging
engine was found that matched the following parameters: bus=twprocsvr_bus,
targetGroup=null, targetType=BusMember, targetSignificance=Preferred,
transportChain=InboundBasicMessaging, proximity=Bus..
at
com.lombardisoftware.client.delegate.BusinessDelegateException.asBusinessDelegateException(BusinessDelegateException.java:41)
at
com.lombardisoftware.client.delegate.common.WebsphereDelegateHelper.doAsCurrentSubjectContextSensitive(WebsphereDelegateHelper.java:132)
at
com.lombardisoftware.client.delegate.EventQueueManagerDelegateDefault.browseErrorQueue(EventQueueManagerDelegateDefault.java:70)
at
com.ibm._jsp._cs_5F_event_5F_mgr_5F_queue._jspService(_cs_5F_event_5F_mgr_5F_queue.java:230)
...
53 more

No message engine running? Hmm, let's check the administration console. twprocsvr profile has no administration console installed, so I started server1 profile for problem determination.

Turns out that the message engine on twprocsvr is configured to use the datasource configured, with schema 'sa'. This is the user used I specified for authentication. There is no schema 'sa' however, the default schema on SQLServer is 'dbo'. Therefore the message engine could not create the tables required for the message engine. They can be recognized by their prefix "SIB":


I found out by looking for SIB in the SystemOut after reading on the web that this error can be caused by invalid SIB* tables.

0000000d SibMessage    E   [twprocsvr_bus:ProcessCenter01.twprocsvr-twprocsvr_bus] CWSIS0002E: The messaging engine encountered an exception while starting. Exception: com.ibm.ws.sib.msgstore.PersistenceException: CWSIS1501E: The data source has produced an unexpected exception: com.jnetdirect.jsql.JSQLNonFatalException: The specified schema name "sa" either does not exist or you do not have permission to use it.

0000000d SibMessage    E   [twprocsvr_bus:ProcessCenter01.twprocsvr-twprocsvr_bus] CWSID0035E: Messaging engine ProcessCenter01.twprocsvr-twprocsvr_bus cannot be started; detected error reported during com.ibm.ws.sib.msgstore.impl.MessageStoreImpl start()

0000000d SibMessage    E   [twprocsvr_bus:ProcessCenter01.twprocsvr-twprocsvr_bus] CWSID0027E: Messaging engine ProcessCenter01.twprocsvr-twprocsvr_bus cannot be restarted because a serious error has been reported.

0000000d SibMessage    I   [twprocsvr_bus:ProcessCenter01.twprocsvr-twprocsvr_bus] CWSID0016I: Messaging engine ProcessCenter01.twprocsvr-twprocsvr_bus is in state Stopped.

0000000d SibMessage    I   [twprocsvr_bus:ProcessCenter01.twprocsvr-twprocsvr_bus] CWSID0016I: Messaging engine ProcessCenter01.twprocsvr-twprocsvr_bus is in state Joined.

0000000d SibMessage    E   [twprocsvr_bus:ProcessCenter01.twprocsvr-twprocsvr_bus] CWSID0039E: HAManager-initiated activation has failed, messaging engine ProcessCenter01.twprocsvr-twprocsvr_bus will be disabled. Reason Refer to earlier messages

I fixed it by adding a schema 'sa' to my database in sqlserver


I changed it on database level because I could not change the schema setting on the message engine, when logging in using the tw_admin user. Hmm, I have no credentials for wasadmin... only the default lombardi credentials.