Using the Coherence Adapter in SOA Suite 12C

Retrieving data from a back-end system or executing something in a business rule engine can cost quite some time. When we build services we want to strife for performance but sometimes you make services which you just can’t make any faster. One thing to keep in mind is that it is possible to cache in SOA Suite using the coherence adapter. For example retrieving data from a back-end system in a composite and enriching it takes 1400ms average but during a 24 hours, you might get 15% call with the same request, which should give exactly the same answer! To do the retrieving and enrichment every time seems like a shame. This is where the Coherence adapter comes in.

The Coherence adapter can store certain pieces of data for a certain amount of time. This overview consists of 2 parts. The first one is the configuration of the coherence adapter in weblogic, the second part is the usage of the coherence adapter in a composite. Now lets get started.

Configuring the Coherence adapter

  1. The Coherence adapter isn’t active out of the the box. It is not targeted to any managed server so the first thing you have to do is to target it.

    Go to your SOA server’s console, go to Deployments and click the CoherenceAdapter. Go the the tab Targets. Look if it is targeted. If it is not….target it. Click Lock & Edit and check soa_cluster checkbox, click Save and Activate Changes: Target

  2. The next step is to add cache configuration file. This step requires that you have to physically put an XML on the file system of the server. In our example it is a file called ProductServiceCache-Configuration.xml and sat in location /u01/domains/dev_soa_domain. On a clustered environment the file have to be on both servers! My configuration file looks like:
    <?xml version="1.0"?>
    <!DOCTYPE cache-config SYSTEM "cache-config.dtd">
    <cache-config>
      <caching-scheme-mapping>
        <cache-mapping>
          <cache-name>ProductServiceCache</cache-name>
          <scheme-name>transactional</scheme-name>
        </cache-mapping>
      </caching-scheme-mapping>
      <caching-schemes>
        <transactional-scheme>
          <scheme-name>transactional</scheme-name>
          <service-name>DistributedCache</service-name>
          <autostart>true</autostart>
        </transactional-scheme>
      </caching-schemes>
    </cache-config>
    

    It is wise to create separate caches for separate services in case you want to set different cache timers or in case of the occasional flush.

  3. The next step is creating a Coherence Adapter Connection Factory. Go to your SOA environment console, deployments and click on the Coherence Adapter. Go to the tab Outbound Connection Pools. Click New. Choose javax.resource.cci.ConnectionFactory. Input your JNDI name. I will have eis/Coherence/ProductService and click Finish. Now select the connection factory you just created and choose the Properties tab. Set the following values:ConnectionFactorySettings
  4. As a last step, you will have to update the CoherenceAdapter with the settings. You can do this by stopping en starting the CoherenceAdapter. Go to Deployments and click Lock & Edit. Select the checkbox of the CoherenceAdapter and click Update and the Finish.
    CoherenceAdapter

Oke now we have configured Coherence including a cache for the ProductService.

Making use of the Coherence adapter in a composite

To make use of the adapter in your BPEL, you will have to add 2 JCA connections.
Coherence-JCA

We are going to call them retrieveResult and writeResult. Lets start of with the retrieveResult. Drag the Coherence Adapter onto the right part of the canvas and follow the wizard:

RetrieveResult

On the step 4 when pressing Finish it will ask you No cache key specified. No worries as we will do this later on.

No we will create a writeResult JCA adapter:
WriteResult I set the Time To Live of the cache to 10.000 milliseconds for the test.

Now wire your BPEL to your 2 adapters like you would normally do. This would look like this then.

CompleteComposite

Now in your BPEL you want to do 2 things:

  • Check if something is in the cache based on your key, and if it is return the result.
  • If it is not in there, do your normal thing and write the result into the cache

The reading and writing can take place like normal call-outs as you can see. Just drag an Invoke action onto your BPEL and wire it the correct Partnerlink. Lets first do the retrieve. Invoke the retrieveResult and create input and output variables like you would normally do. For the unique key, go to the Properties tab and add property jca.coherence.Key. Here you define the unique key for your result. I just concatenated some string from the request.

retrieveCall

In your BPEL you can check if the retrieve call gave you some data back. If it did, you can skip all the normal processing and just return the result. If you ended up with nothing, you want to do normal processing and at the end, write the result to the cache. Use the same Property again and make sure you assign the expression or variable which you used to retrieve something.

retrieveCall

It is that easy. My composite will look something like this now:
final

Now just build and deploy and see if it works!

My first soap call takes about 1777 ms….and that is without a cache hit. Now lets call it again within 10 seconds. There we go….800 ms. Now let check the EM to see if we hit the actual cache:

EM

There we go!

Some considerations:

  • This is an easy way to make things faster. Do keep checking with Performance, Load and Stress tests what will happen to you CPU and Memory usage on big loads. The A-team have some good articles about this.
  • You might want to have the Time-To-Live different per environment. So for example 10 minutes on Dev but 1 hour on Test and 24 houres on Prod. You can easily do this making use of server tokens. Add them in the EM under tokens and use them in the JCA files like ${ProductServiceCacheTime} for example.

References:

  • http://www.ateam-oracle.com/using-the-12c-coherence-adapter-to-access-a-non-transactional-local-cache/
  • https://docs.oracle.com/middleware/1221/adapters/develop-soa-adapters/GUID-EA82A9A3-656E-464A-B764-E076C172BAEF.htm#TKADP2495

OFM 12C: Slow SB deployment issue

Working with 12C a while, we started noticing that the deployment of our SharedResources SB project started to slow down. This project contains re-usable resources and contracts. The strange thing was that it was only this specific SB project. All the other projects where fine. Both deploying in JDeveloper and deploying using Maven started at around 40 seconds but after a while, it went up to 10 minutes even which is of course not workable.

After some proper investigation and contact with Oracle Support we came to the conclusion that the Maven deployments where causing this. See bug 22051706: Maven OSB deploy causes open activation sessions on OSB in case of failure, which if unresolved cause slow deployment performance. So basically when your maven deployment fails, it causes an open session. If the session contains a lot of items, which our SharedResources project has, then even 1 or 2 open sessions will drastic slow your deployments down.

sessions

The solutions is either to use the servicebus console to navigate to the sessions tab at the bottom, select a session, take control of it and discard it. I think that also the $DOMAIN_HOME/osb folder holds the sessions which can be deleted on the server.

Another way is to remove them using a piece of code:

/* To run this, make sure the setDomainEnv.sh is sourced:
    cd $DOMAIN_HOME/bin
    . setDomainEnv.sh
*/

import weblogic.management.mbeanservers.domainruntime.DomainRuntimeServiceMBean;
import weblogic.management.jmx.MBeanServerInvocationHandler;
import javax.management.MBeanServerConnection;
import javax.management.remote.JMXConnector;
import javax.management.remote.JMXServiceURL;
import javax.management.remote.JMXConnectorFactory;
import javax.management.ObjectName;
import com.bea.wli.sb.management.configuration.SessionManagementMBean;
import java.util.Set;
import java.util.Hashtable;
import java.util.HashSet;
import java.net.MalformedURLException;
import javax.naming.Context;
import java.io.IOException;

public class DiscardOSBSessions {
    public static final String hostname = "MY_HOSTNAME";
    public static final int port = 7001;
    public static final String username = "MY_USERNAME";
    public static final String password = "MY_PASSWORD";

    static public void main(String[] args)
    {
        JMXConnector conn = null;
        
        try
        {
            // get the jmx connector
            conn = initConnection(hostname, port, username, password);

            // get mbean connection
            MBeanServerConnection mbconn = conn.getMBeanServerConnection();

            // get the Session names:
            //com.bea:Name=ALSBConfiguration.weblogic,Location=AdminServer,Type=com.bea.wli.sb.management.configuration.ALSBConfigurationMBean
            ObjectName mbeanQuery = new ObjectName("*:Name=ALSBConfiguration.*,Type=com.bea.wli.sb.management.configuration.ALSBConfigurationMBean,Location=AdminServer");
            Set<ObjectName> mbeans = mbconn.queryNames(mbeanQuery, null);
            Set<String> sessionNames = new HashSet<String>();
            for (ObjectName mbeanName : mbeans) {
                sessionNames.add(mbeanName.getKeyPropertyList().get("Name").replace("ALSBConfiguration.",""));
            }
            
            System.out.println(sessionNames.size()+" open sessions:");
            if (!sessionNames.isEmpty()){
                for (String sessionName : sessionNames) {
                    System.out.println(" - "+sessionName);
                }

                System.out.println();
                System.out.println("Destroying the sessions:");
                
                // get domain service mbean. This is the topmost mbean
                DomainRuntimeServiceMBean domainService = (DomainRuntimeServiceMBean) MBeanServerInvocationHandler.newProxyInstance(mbconn, new ObjectName(DomainRuntimeServiceMBean.OBJECT_NAME));

                // obtain session management mbean to destroy a session.
                // This mbean instance can be used more than once to create/discard/commit many sessions
                SessionManagementMBean sm = (SessionManagementMBean) domainService.findService(SessionManagementMBean.NAME, SessionManagementMBean.TYPE, null);

                for (String sessionName : sessionNames) {
                    System.out.println(" - "+sessionName);
                    sm.discardSession(sessionName);
                }
            }
            System.out.println();
            System.out.println("Successful completion");
        }
        catch (Exception e) {
            e.printStackTrace();
        } finally {
            if (conn != null)
                try {
                    conn.close();
                } catch (Exception e) {                
                    e.printStackTrace();
                }
        }
    }

    private static JMXConnector initConnection(String hostname, int port, String username, String password) throws IOException,MalformedURLException
    {
        JMXServiceURL serviceURL = new JMXServiceURL("t3", hostname, port, "/jndi/" + DomainRuntimeServiceMBean.MBEANSERVER_JNDI_NAME);
        Hashtable<String, String> h = new Hashtable<String, String>();
        h.put(Context.SECURITY_PRINCIPAL, username);
        h.put(Context.SECURITY_CREDENTIALS, password);
        h.put(JMXConnectorFactory.PROTOCOL_PROVIDER_PACKAGES, "weblogic.management.remote");
        return JMXConnectorFactory.connect(serviceURL, h);
    }
}

As you can see, you can only run this on the appropriate server with the right classes on the classpath. I tried to Mavenize it into a maven project with the proper dependencies but I wasn’t able to find the right jar files for:

  • weblogic.management.mbeanservers.domainruntime.DomainRuntimeServiceMBean
  • weblogic.management.jmx.MBeanServerInvocationHandler
  • com.bea.wli.sb.management.configuration.SessionManagementMBean

If anyone can help me out here….that would be great!

A feature request has been made to add the ‘discard session’ parameter to the Maven properties so hopefully this will come in future releases.

The Oracle Maven repository available now!

For ages, software companies have had maven repositories to manage their artifacts. Using maven’s dependencies system, you can easily add new libraries to your project without needing to know which other libraries you needed and maven will even download them for you. So the times of adding a library…compiling…class-not-found…adding another library…compiling…next-class-not-found where over. Unless you needed some oracle libraries!

logo

Up to now, Oracle did not have a public repository so when we needed libraries, you had to download the libraries yourself and stick them into your local repository. Or use the tool provided by Oracle to do this for you see last part of this weblog. But with the coming of the 12C version where Oracle has gotten more Maven enabled, it would not be long before the Oracle maven repository would come to life and taataaaaaa! Oracle now has a maven repository: http://maven.oracle.com. You will have to register/login and accept the user terms before you can access the repository. The repository only holds release-level artifacts, such as 12.1.2 and 12.1.3 for now.

Lets see if it works connecting straight to the Oracle repository using maven. Download Maven 3.2.5 or later as only these versions are supported by the repository it seems. Go to settings.xml and add the oracle server to the servers section.

<servers>
  <server>
    <id>maven.oracle.com</id>
    <username>YOURUSERNAME</username>
    <password>YOURPASSWORD</password>
    <configuration>
      <basicAuthScope>
        <host>ANY</host>
        <port>ANY</port>
        <realm>OAM 11g</realm>
      </basicAuthScope>
      <httpConfiguration>
        <all>
          <params>
            <property>
              <name>http.protocol.allow-circular-redirects</name>
              <value>%b,true</value>
            </property>
          </params>
        </all>
      </httpConfiguration>
    </configuration>
  </server>

Now go to a 12C service bus project you have lying around and add the next section to your pom.xml:

<repositories>
	<repository>
		<id>maven.oracle.com</id>
		<releases>
			<enabled>true</enabled>
		</releases>
		<snapshots>
			<enabled>false</enabled>
		</snapshots>
		<url>https://maven.oracle.com</url>
		<layout>default</layout>
	</repository>
</repositories>
<pluginRepositories>
	<pluginRepository>
		<id>maven.oracle.com</id>
		<url>https://maven.oracle.com</url>
	</pluginRepository>
</pluginRepositories>

Now we have configured the Oracle repository for the project and in the maven config we have set the password. So lets do a mvn clean and see if the dowloads starts…..

D:\Amazon EC2\SVN\trunk\osb\HelloService_1.0>mvn clean
[INFO] Scanning for projects...
Downloading: https://maven.oracle.com/com/oracle/servicebus/sbar-project-common/12.1.3-0-0/sbar-project-common-12.1.3-0-0.pom
Downloaded: https://maven.oracle.com/com/oracle/servicebus/sbar-project-common/12.1.3-0-0/sbar-project-common-12.1.3-0-0.pom (0 B at 0.0 KB/sec)
Downloading: https://maven.oracle.com/com/oracle/servicebus/client/maven-metadata.xml
Downloaded: https://maven.oracle.com/com/oracle/servicebus/client/maven-metadata.xml (381 B at 0.2 KB/sec)
Downloading: https://maven.oracle.com/com/oracle/weblogic/weblogic-server-pom/maven-metadata.xml
...
Downloaded: https://maven.oracle.com/com/oracle/servicebus/core/module-common/maven-metadata.xml (393 B at 0.2 KB/sec)
Downloading: https://maven.oracle.com/com/oracle/servicebus/core/module-configfwk/maven-metadata.xml
Downloaded: https://maven.oracle.com/com/oracle/servicebus/core/module-configfwk/maven-metadata.xml (396 B at 0.1 KB/sec)
Downloading: https://maven.oracle.com/com/oracle/servicebus/core/module-kernel-api/maven-metadata.xml
Downloaded: https://maven.oracle.com/com/oracle/servicebus/core/module-kernel-api/maven-metadata.xml (397 B at 0.2 KB/sec)
[WARNING]
[WARNING] Some problems were encountered while building the effective model for nl.redrock:HelloService_1.0:sbar:1.0-SNAPSHOT
[WARNING] 'parent.relativePath' points at RR-ServiceBusApplication:RR-ServiceBusApplication instead of com.oracle.servicebus:sbar-project-common, please verify your project structure @ line 6, column 13
[WARNING]
[WARNING] It is highly recommended to fix these problems because they threaten the stability of your build.
[WARNING]
[WARNING] For this reason, future Maven versions might no longer support building such malformed projects.
[WARNING]
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building HelloService_1.0 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ HelloService_1.0 ---
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 09:58 min
[INFO] Finished at: 2015-01-14T20:00:20+01:00
[INFO] Final Memory: 9M/120M
[INFO] ------------------------------------------------------------------------
D:\Amazon EC2\SVN\trunk\osb\HelloService_1.0>

So perfect….it works!

The next thing we want to do is proxy the repository in your Nexus for example. We can add the server in Nexus but we can’t seem to browse it. After some looking around it seems a lot of repository managers can’t handle the authentication way of the Oracle repository and have to be patched. See a comment of Mark Nelson about this:

Hi Folks,

We are aware that some of you would want to proxy the Oracle Maven Repository with repository managers like Artifactory and Nexus. We have tested the three common ones, and found that they all need small patches to understand the authentication we are using. We are working with them to patch these issues. We will let you know the JIRA ID’s for these issues when they are available. If you want to help, you can upvote the issues, or if you are using commercial versions, you could contact your representative to let them know you want these patches.

Best regards,

Mark Nelson

So work in progress but definately a step in the right direction!

References: