Add client certificate for outgoing OSB call

Simple use case……you want to connect to a customer system over the internet. The customers system has an API but requires 2-way SSL. This means we have to send a client certificate along to make sure the SSL handshake can be completed. If your server has already a server certificate installed, it will send this one along by default but the customers system won’t accept it as it is different then what it trusts. In the next section, I will explain how to add a client certificate to an outgoing OSB call.

SSL

Let’s say I want to connect to my favourite climbing shop www.mountaingear.com as they have a nice backend api to take orders. The guys from mountaingear.com created a certificate for me to send along with the call.

First I am going to generate a keystore with a private key in it, to simulate the certificate which the third party gave to me.

keytool -genkey -keyalg RSA -alias climber -keystore keystore.jks -storepass password -validity 360 -keysize 2048

If I open the created JKS using keystore explorer I see a keypair we just created.

MyCertificate

Copy the jks to my OracleHome say ORACLE_HOME/keystores.

The next thing we have to do is to create an PrivateKeyInfrastructure Provider. Go to weblogic console and then:

  • Security Realms
  • myrealm
  • Providers
  • Credential Mapping
  • New –> Set your name here. Mine is MountainGearPKIProvider. Set the type to PKICredentialMapper. Click OK to finish.

PKIProvider

Restart the Admin server for the changes to take effect.

When the admin server has restarted we are going to configure the PrivateKeyInfrastructure Provider we just created. Go to weblogic console and then:

  • Security Realms
  • myrealm
  • Providers
  • Credential Mapping
  • MountainGearPKIProvider
  • Provider Specific –> KeystoreType = JKS, Keystore File Name = The location of your just created jks…mine is G:\Oracle\Middleware\OracleHome\keystore\MountainGear.jks, KeyStorePassphrase = password, Confirm KeyStorePassphrase = password. Click Save to finish
    • PKIProviderConfiguration

      Restart the Admin server again for the changes to take effect.

      After this, go to the SBConsole and start a new session. In a project, create a folder which will hold the ServiceKeyProvider. Mine will be in my Accounts project. Create a ServiceKeyProvider in this new folder and select using SSL Client Authentication the right certificate. Set the password, save it and activate it.

      ServiceKeyProvider

      The reason why we do this in the OSB console is that it cannot resolve the PKIProvider in JDeveloper. The only work around is to create it on the server and then export your project and import it into JDev. After you have done this, you can see your ServiceKeyProvider.

      The last step now is to make sure it is send along with a flow. The only thing we have to do, to accomplish this is to select the proxy service and go to the Security tab. Here you can select the Service Key Provider.

      ProxyService

Using JMX to list installed 12C OSB and SOA projects including versions using Java

Lately we where looking for a way to easily list all the services (OSB and SOA) installed on our DTAP machines. A CMDB works as well as it is maintained. As this is tedious and manual labour, we all know from experience that the CMDB is never fully correct. So how do we know then what is installed where? We can open the EM and the sbconsole and have a look there but wouldn’t it be more fancy if we would just select an environment, run some script and it would give you a list of the services?

Ok, how do we obtain al this information then? Weblogic has a JMX (Java Management eXtensions) interface which we can use to access mbeans which should hold all the information we need. Let’s start of with our SOA server first. You can use WLST but as I’m a Java fan, I will use Java. Starting of with a simple Maven project I first need to setup a connection to a server, lookup the right mbean which can give me a list of the installed services, access it and fill some result. Let’s first init the connection and the use the connection to lookup the right mbean.

   /**
     * Initialize the connection with the server
     * @param hostname the host
     * @param port the port
     * @param username the username
     * @param password the password
     * @return a JMXConnector
     * @throws IOException
     * @throws MalformedURLException
     */
    public static JMXConnector initConnection(String hostname, int port, String username, String password) throws IOException, MalformedURLException {
        JMXServiceURL serviceURL = new JMXServiceURL("t3", hostname, port, "/jndi/" + DomainRuntimeServiceMBean.MBEANSERVER_JNDI_NAME);
        Hashtable<String, String> h = new Hashtable<String, String>();
        h.put(Context.SECURITY_PRINCIPAL, username);
        h.put(Context.SECURITY_CREDENTIALS, password);
        h.put(JMXConnectorFactory.PROTOCOL_PROVIDER_PACKAGES, "weblogic.management.remote");
        return JMXConnectorFactory.connect(serviceURL, h);
    }

    /**
     * List the SOA services
     *
     * @throws Exception
     */
    public static List<DeployedService> getSOA(String host, int port, String username, String password) throws Exception {

        List<DeployedService> result = new ArrayList();

        JMXConnector connector = initConnection(host, port, username, password);
        // get mbean connection
        MBeanServerConnection mbconn = connector.getMBeanServerConnection();

        ObjectName queryObject = new ObjectName("oracle.soa.config:Location=soa_ms01,name=soa-infra,j2eeType=CompositeLifecycleConfig,Application=soa-infra,*");
        Set<ObjectName> queryObjectName = mbconn.queryNames(queryObject, null);
        Iterator iterator = queryObjectName.iterator();

        while (iterator.hasNext()) {
            ObjectName compositeObjectName = (ObjectName) iterator.next();

            CompositeData[] composites = (CompositeData[]) mbconn.getAttribute(compositeObjectName, "DeployedComposites");
            for (CompositeData composite : composites) {
                String dn = (String) composite.get("DN");
                Boolean isDefault = (Boolean) composite.get("isDefault");
                String state = (String) composite.get("state");
                String mode = (String) composite.get("mode");

                if (isDefault && state.equalsIgnoreCase("on") && mode.equalsIgnoreCase("active")) {
                    DeployedService service = new DeployedService();
                    service.setName(dn.substring(dn.indexOf('/') + 1, dn.indexOf('!')));
                    service.setVersion(dn.substring(dn.indexOf('!') + 1, dn.indexOf('*')));
                    if (!result.contains(service)) {
                        result.add(service);
                    }
                }
            }
        }
        connector.close();
        return result;
    }

As you can see the CompositeData data has all of the data we need. We can check if the composite is Active and if it is Default and we can extract the version from the DN. As for dependencies, I suspected I would need the wlfullclient and probably some other jars. Netbeans neatly suggested the dependencies for me from my local maven repository and it seemed I only needed 1 which was soa-infra-mgmt. So my first dependcies where:

        <dependency>
            <groupId>com.oracle.weblogic</groupId>
            <artifactId>wlfullclient</artifactId>
            <version>12.1.3</version>
        </dependency>
        <dependency>
            <groupId>com.oracle.soa</groupId>
            <artifactId>soa-infra-mgmt</artifactId>
            <version>12.1.3-0-0</version>
            <type>jar</type>
        </dependency>

Next but is the OSB. This was a bit more tricky as the OSB doesn’t support versioning. We worked around this issue by adding version info in the description field of the project. See here for an explanation. So it was kind of obvious there was no mbean to retrieve this info. The best way I found was just to export all the project, unzip them and just loop through all the projects and check their description field. The code looks like this:

    /**
     * List all the OSB services
     *
     * @throws Exception
     */
    public static List<DeployedService> getOSB(String host, int port, String username, String password) throws Exception {

        List<DeployedService> result = new ArrayList();

        // get the jmx connector
        JMXConnector connector = initConnection(host, port, username, password);

        // get mbean connection
        MBeanServerConnection mbconn = connector.getMBeanServerConnection();

        // get domain service mbean. This is the topmost mbean
        DomainRuntimeServiceMBean domainService = (DomainRuntimeServiceMBean) MBeanServerInvocationHandler.
                newProxyInstance(mbconn, new ObjectName(DomainRuntimeServiceMBean.OBJECT_NAME));

        // Obtain MBean for peforming read only operations. Notice that the name
        // of the mbean for the core data does not contain any session name.
        ALSBConfigurationMBean alsbCore = (ALSBConfigurationMBean) domainService.findService(ALSBConfigurationMBean.NAME, ALSBConfigurationMBean.TYPE, null);

        Set<Ref> refs = alsbCore.getProjects();
        byte[] contentsProj = alsbCore.exportProjects(new ArrayList(refs), null);

        //write the projects to the target directory
        FileOutputStream fos = new FileOutputStream("target/export-sbconfig.jar");
        fos.write(contentsProj);
        fos.close();

        //unzip the zip file
        File unzipDir = new File("target/export-sbconfig");
        FileUtils.unZipIt("target/export-sbconfig.jar", unzipDir);

        //loop through them and read the value
        File[] files = unzipDir.listFiles();
        DeployedService service;
        for (File file : files) {
            //name of the directory
            String name = file.getName();
            if (name.contains("_")) {
                String description = XMLUtils.readDescription(file.getAbsolutePath() + "/_projectdata.LocationData");
                service = new DeployedService();
                service.setNaam(name);
                service.setVersie(description);
                result.add(service);
            }
        }

        //delete the zip and the unzip dir
        FileUtils.delete(unzipDir);
        FileUtils.delete(new File("target/export-sbconfig.jar"));

        connector.close();
        return result;
    }

As you can see I export the zip file to the target directory, unzip it there and then loop through them and checking the _projectdata.LocationData file. What about the dependencies here? Netbeans found the obvious ones for me which where module-kernel-api and module-configfwk both in com.oracle.servicebus.core. So

        <dependency>
            <groupId>com.oracle.servicebus.core</groupId>
            <artifactId>module-kernel-api</artifactId>
            <version>12.1.3-0-0</version>
            <type>jar</type>
        </dependency>
        <dependency>
            <groupId>com.oracle.servicebus.core</groupId>
            <artifactId>module-configfwk</artifactId>
            <version>12.1.3-0-0</version>
            <type>jar</type>
        </dependency>

Well that seemed about right. I tried to give it a spin but unfortunately I ran into a ClassNotFoundException……..specificaly com.bea.wli.sb.management.configuration.DelegatedALSBConfigurationMBean. It seemed I needed something else at runtime. After some Googling, I found out the jar I needed was sb-kernel-impl.jar but I couldn’t find it anywhere on the 12C installation. In a 11G installation, it was under OSB_DOMAIN/lib so I spinned up an old 11G VM and grabbed the jar and uploaded it into my maven repo. Again, give it a try…..and there we go….it work. Next I tweaked it a bit so I could start it from Jenkins and it would output the list in easy readable format. The result look like this.

overview

SOA Suite 12C: Add version information to your ServiceBus projects using custom maven plugin

As you have seen in my previous posts it is possible to build your SB and SOA components using Maven. See here. One of the issues we encountered is that building ServiceBus projects supports some very basic maven stuff. For example the description property in the maven pom file is not mapped to the description field of a service bus project which would have been nice as this is the only extra field which we can use to put some extra information….for example version info!

For SOA composites you are able to input some versioning info by updating the composite.xml using a Google maven plugin. Just add this plugin to build:

<!--Needed to replace the revision in the composite.xml due to bug (20553998) which causes not to update the revision correctly -->
<plugin>
	<groupId>com.google.code.maven-replacer-plugin</groupId>
	<artifactId>replacer</artifactId>
	<version>1.5.3</version>
	<executions>
		<execution>
			<phase>initialize</phase>
			<goals>
				<goal>replace</goal>
			</goals>                   
		</execution>
	</executions>
	<configuration>
		<ignoreMissingFile>true</ignoreMissingFile>
		<file>${scac.input}</file>
		<xpath>//composite/@revision</xpath>
		<token>^.*$</token>
		<value>${composite.revision}</value>
	</configuration>
</plugin>

which result into:
versioningSoa

For SB projects there is no such thing. How can we then see which version we have?! Well the only field which we can use is the description field. The only problem is that you can not update this using Maven. The things you have to:

  • Unzip the sbconfig.sbar
  • Update the _projectdata.LocationData file which holds a proj:description tag
  • Zip the sbconfig.sbar again

Not too difficult at all. You probably can do this by using Ant but that is so 2001! We have Maven now so why not write a custom Maven plugin which does this all for you?! Well I won’t bother you with the Java details but you can download the plugin jar here!

Just install it into your Maven repository by running:

    mvn install:install-file -Dfile=version-information-plugin-1.0.jar -DgroupId=nl.redrock.maven.plugins.servicebus -DartifactId=version-information-plugin -Dversion=1.0 -Dpackaging=jar

Now that you have the plugin installed you can wire it to the package phase of your service bus project by adding the plugin to your service bus pom file. Mine looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd"
         xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <modelVersion>4.0.0</modelVersion>
  <parent>
    <groupId>com.oracle.servicebus</groupId>
    <artifactId>sbar-project-common</artifactId>
    <version>12.1.3-0-0</version>
  </parent>
  <groupId>nl.redrock</groupId>
  <artifactId>ConversionRateService</artifactId>
  <version>1.1.2.5</version>
  <packaging>sbar</packaging>
  <description></description>
  <build>
    <plugins>
     <plugin>
        <groupId>nl.redrock.maven.plugins.servicebus</groupId>
        <artifactId>version-information-plugin</artifactId>
        <version>1.0</version>
        <executions>
          <execution>
            <phase>package</phase>
            <goals>
              <goal>version</goal>
            </goals>
          </execution>
        </executions>
        <configuration>
          <description>${project.version}</description>
        </configuration>
      </plugin>
    </plugins>
  </build>
</project>

This will now call the plugin after the normal packaging has been completed. In the configuration you can now set the description. In my case I fill it with the versioning info of the component itself. Now just build your service bus project using Maven:

mavenbuild

and deploy and go to the sbconsole and voila:

sbconsole

OFM 12C: Slow SB deployment issue

Working with 12C a while, we started noticing that the deployment of our SharedResources SB project started to slow down. This project contains re-usable resources and contracts. The strange thing was that it was only this specific SB project. All the other projects where fine. Both deploying in JDeveloper and deploying using Maven started at around 40 seconds but after a while, it went up to 10 minutes even which is of course not workable.

After some proper investigation and contact with Oracle Support we came to the conclusion that the Maven deployments where causing this. See bug 22051706: Maven OSB deploy causes open activation sessions on OSB in case of failure, which if unresolved cause slow deployment performance. So basically when your maven deployment fails, it causes an open session. If the session contains a lot of items, which our SharedResources project has, then even 1 or 2 open sessions will drastic slow your deployments down.

sessions

The solutions is either to use the servicebus console to navigate to the sessions tab at the bottom, select a session, take control of it and discard it. I think that also the $DOMAIN_HOME/osb folder holds the sessions which can be deleted on the server.

Another way is to remove them using a piece of code:

/* To run this, make sure the setDomainEnv.sh is sourced:
    cd $DOMAIN_HOME/bin
    . setDomainEnv.sh
*/

import weblogic.management.mbeanservers.domainruntime.DomainRuntimeServiceMBean;
import weblogic.management.jmx.MBeanServerInvocationHandler;
import javax.management.MBeanServerConnection;
import javax.management.remote.JMXConnector;
import javax.management.remote.JMXServiceURL;
import javax.management.remote.JMXConnectorFactory;
import javax.management.ObjectName;
import com.bea.wli.sb.management.configuration.SessionManagementMBean;
import java.util.Set;
import java.util.Hashtable;
import java.util.HashSet;
import java.net.MalformedURLException;
import javax.naming.Context;
import java.io.IOException;

public class DiscardOSBSessions {
    public static final String hostname = "MY_HOSTNAME";
    public static final int port = 7001;
    public static final String username = "MY_USERNAME";
    public static final String password = "MY_PASSWORD";

    static public void main(String[] args)
    {
        JMXConnector conn = null;
        
        try
        {
            // get the jmx connector
            conn = initConnection(hostname, port, username, password);

            // get mbean connection
            MBeanServerConnection mbconn = conn.getMBeanServerConnection();

            // get the Session names:
            //com.bea:Name=ALSBConfiguration.weblogic,Location=AdminServer,Type=com.bea.wli.sb.management.configuration.ALSBConfigurationMBean
            ObjectName mbeanQuery = new ObjectName("*:Name=ALSBConfiguration.*,Type=com.bea.wli.sb.management.configuration.ALSBConfigurationMBean,Location=AdminServer");
            Set<ObjectName> mbeans = mbconn.queryNames(mbeanQuery, null);
            Set<String> sessionNames = new HashSet<String>();
            for (ObjectName mbeanName : mbeans) {
                sessionNames.add(mbeanName.getKeyPropertyList().get("Name").replace("ALSBConfiguration.",""));
            }
            
            System.out.println(sessionNames.size()+" open sessions:");
            if (!sessionNames.isEmpty()){
                for (String sessionName : sessionNames) {
                    System.out.println(" - "+sessionName);
                }

                System.out.println();
                System.out.println("Destroying the sessions:");
                
                // get domain service mbean. This is the topmost mbean
                DomainRuntimeServiceMBean domainService = (DomainRuntimeServiceMBean) MBeanServerInvocationHandler.newProxyInstance(mbconn, new ObjectName(DomainRuntimeServiceMBean.OBJECT_NAME));

                // obtain session management mbean to destroy a session.
                // This mbean instance can be used more than once to create/discard/commit many sessions
                SessionManagementMBean sm = (SessionManagementMBean) domainService.findService(SessionManagementMBean.NAME, SessionManagementMBean.TYPE, null);

                for (String sessionName : sessionNames) {
                    System.out.println(" - "+sessionName);
                    sm.discardSession(sessionName);
                }
            }
            System.out.println();
            System.out.println("Successful completion");
        }
        catch (Exception e) {
            e.printStackTrace();
        } finally {
            if (conn != null)
                try {
                    conn.close();
                } catch (Exception e) {                
                    e.printStackTrace();
                }
        }
    }

    private static JMXConnector initConnection(String hostname, int port, String username, String password) throws IOException,MalformedURLException
    {
        JMXServiceURL serviceURL = new JMXServiceURL("t3", hostname, port, "/jndi/" + DomainRuntimeServiceMBean.MBEANSERVER_JNDI_NAME);
        Hashtable<String, String> h = new Hashtable<String, String>();
        h.put(Context.SECURITY_PRINCIPAL, username);
        h.put(Context.SECURITY_CREDENTIALS, password);
        h.put(JMXConnectorFactory.PROTOCOL_PROVIDER_PACKAGES, "weblogic.management.remote");
        return JMXConnectorFactory.connect(serviceURL, h);
    }
}

As you can see, you can only run this on the appropriate server with the right classes on the classpath. I tried to Mavenize it into a maven project with the proper dependencies but I wasn’t able to find the right jar files for:

  • weblogic.management.mbeanservers.domainruntime.DomainRuntimeServiceMBean
  • weblogic.management.jmx.MBeanServerInvocationHandler
  • com.bea.wli.sb.management.configuration.SessionManagementMBean

If anyone can help me out here….that would be great!

A feature request has been made to add the ‘discard session’ parameter to the Maven properties so hopefully this will come in future releases.

Custom 11G XPath function not showing up in JDeveloper 12C

You can write your own XPath lib in Java to make certain things easier and re-usable. For an example see here.

From the documentation of 12C, nothing indicated that things would have been changed so I stuck in an old library and restarted JDev. When I opened the xslt mapper, I didn’t see my functions under User Defined. This made me wonder if maybe something did have changed. Last week I attended the OFM Summercamp in Lisbon and Wilfred van der Deijl mentioned that JDev caches a lot so maybe clearing the cache would solve the problem. Someone also mentioned you could use -clean as a parameter when starting JDev but to be quite thorough I just renamed the cache folder which is default under C:\Users\\AppData\Roaming\JDeveloper from 12.1.3.0.0 to 12.1.3.0.0.old. Then I started again and voilà…..there it was. Cheers Wilfred!

UserDefined

Automated acceptance testing using Fitnesse, Java and the OSB

When I was at Xebicon this year I saw a talk from Dave Farley. At 26 minutes or so he starts talking about Domain Specific Languages. Making an abstraction layer on top of your system where your tests run against so when the underlying system changes, you only have to adjust your layer in between instead of all the tests. This idea got me thinking of how we could be able to do this as I am mainly working in the integration business building services. This lead me to Fitnesse. Fitnesse has been around for many years now but I never had the chance to do something with it. But what is Fitnesse?

fitnesse

Fitnesse is a lightweight, open-source framework that makes it easy for software teams to:

  • Collaboratively define Acceptance Tests, web pages containing simple tables of inputs and expected outputs.
  • Run those tests and see the results.

I was interested if I could make a simple abstraction from a service which I build using the OSB and got Fitnesse to run tests against it. Also I was interested if you could then possibly wire it into a build/release pipeline.

The first thing was to get a simple webservice running. I used an old 12C webservice of mine which I build in a previous post. It is a conversion rate webservices with a SOAP or REST entry. I will be using the SOAP one for now. The service takes two parameters, a from and a to currency and the fetches the conversionrate. I deployed it and tested it and as you can see, it works.

soap-service

Next is to get Fitnesse up and running. This is as simple as downloading it and running it using Java like java -jar fitnesse-standalone.jar. When it is up, you can go to http://localhost:80 to see the Fitnesse wiki startpage.

fitnesse-wiki

The next thing is to create a test page. Click the Add link and choose Test Page. Give the page a name and a description. If you start a page name with Test….Fitnesse will automatically recognize it as a test page. Save it for now because we first have to make a piece of Java which will be the abstraction layer between the tests and our OSB webservice. Open your favourite IDE and create a simple Java project based on Maven. I am going to use jax-ws to create a webservice client and write a fixture. A fixture is a Java class which can be extended so Fitnesse can inject the test variables and retrieve the results making use of reflection and such. So we need to add a Fitnesse dependency in our pom and I am going to use the jax-ws maven plugin to generate classes based on the service WSDL. You can do this in various ways but for now I am going to use plain Java. My pom looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>nl.redrock</groupId>
    <artifactId>FitnesseTest</artifactId>
    <version>1.0</version>
    <packaging>jar</packaging>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <maven.compiler.source>1.7</maven.compiler.source>
    </properties>

    <dependencies>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.10</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.fitnesse</groupId>
            <artifactId>fitnesse</artifactId>
            <version>20081201</version>
        </dependency>
        <dependency>
            <groupId>org.codehaus.mojo</groupId>
            <artifactId>jaxws-maven-plugin</artifactId>
            <version>1.12</version>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <!-- usage of jax-ws maven plugin-->
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>jaxws-maven-plugin</artifactId>
                <version>1.12</version>
                <executions>
                    <execution>
                        <id>wsimport-from-jdk</id>
                        <goals>
                            <goal>wsimport</goal>
                        </goals>
                    </execution>
                </executions>
                <configuration>
                    <!--wsdls file directory -->
                    <wsdlDirectory>src/main/resources/wsdls</wsdlDirectory>
                    <!-- which wsdl file -->
                    <wsdlFiles>
                        <wsdlFile>conversionrateservice.wsdl</wsdlFile>
                    </wsdlFiles>
                    <!-- Keep generated files -->
                    <keep>true</keep>
                    <!-- Package name -->
                    <packageName>nl.redrock.fitnessetest</packageName>
                    <!-- generated source files destination-->
                    <sourceDestDir>target/generated-code/src</sourceDestDir>
                </configuration>
            </plugin>
            <!-- adding generated source -->
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>build-helper-maven-plugin</artifactId>
                <executions>
                    <execution>
                        <id>add-source</id>
                        <phase>generate-sources</phase>
                        <goals>
                            <goal>add-source</goal>
                        </goals>
                        <configuration>
                            <sources>
                                <source>target/generated-code/src</source>
                            </sources>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
</project>

As you can see I refer to src/main/resources/wsdls where my service WSDL is located. I can now just run mvn generate-sources to generate the classes I need for calling the OSB service. When I ran mvn generate-sources I have the following classes:

generated-sources

I can now write the fixture and use JUnit to test it. I am going to use a simple fixture but as your can see here there are many fixtures which you can extend. Mine looks like this:

package nl.redrock.fitnessetest.fixtures;

import fit.*;
import nl.redrock.fitnessetest.ConversionRateService_Service;
import nl.redrock.fitnessetest.Currency;
import nl.redrock.fitnessetest.GetConversionRateRequest;
import nl.redrock.fitnessetest.GetConversionRateResponse;

/**
 *
 * @author Hugo
 */
public class ConversionRateServiceFixture extends ColumnFixture {
    
    public String from;
    public String to;
    
    public double getConverionRate() throws Exception
    {
        ConversionRateService_Service client = new ConversionRateService_Service();
        GetConversionRateRequest req = new GetConversionRateRequest();
        req.setFromCurrency(Currency.fromValue(this.from));
        req.setToCurrency(Currency.fromValue(this.to));
        
        GetConversionRateResponse response = client.getConversionRateServicePort().getConversionRate(req);
        return response.getConversionRateResult();
    }
    
}

As you can see it extends the class ColumnFixture and I have 2 public variables to and from.

The next thing is to write a JUnit test to see if the fixture works:

package nl.redrock.fitnessetest;

import nl.redrock.fitnessetest.fixtures.ConversionRateServiceFixture;
import org.junit.Test;

public class ConversionRateServiceTest {

    @Test
    public void ConversionRateClientTest() throws Exception {
        
        ConversionRateServiceFixture fix = new ConversionRateServiceFixture();
        fix.from="EUR";
        fix.to="USD";
        System.out.println(fix.getConverionRate());
    }
}

I have no assertion here, just a System.out to see if the service works. Lets run it and see the magic.

junit-test

Perfect. The last thing we have to do now is to package everything into a jar so we can tell Fitnesse to put it on the classpath. Just run mvn clean package and it should create your jar in the target directory. Now lets get back to Fitnesse. Open the page you have just create and edit it.

Add the following code to the wiki page:

!path D:\Workspace\Java\FitnesseTest\target\FitnesseTest-1.0.jar

!|nl.redrock.fitnessetest.fixtures.ConversionRateServiceFixture |
| to | from | getConverionRate? |
| USD | EUR | 1.111 |
| EUR | EUR | 1.0 |

The !path will tell Fitnesse that it should put this jar on its classpath when running the tests. The
!|nl.redrock.fitnessetest.fixtures.ConversionRateServiceFixture | will tell which is the fixture class. After that it is just easy. The first two columns say which parameters can be used as input, and the third on for output. Notice the questionmark at the end of getConverionRate?.
The last 2 rows are just testcases.

Now save the page and press the Test button at the top of the screen.

fitnesse-test

There we go. We ran the 2 testcases and got the results back. I checked Jenkins for plugins and there is a plugin which enables you to run the Fitnesse tests so you easily add them to release pipeline.

So

  • The abstraction layer between your service and the tests is quite a big benefit. Lets say the contract of the service has a breaking change. We add a mandatory field. Normally you would have to change all your existing testcases. With the abstraction layer you would be able to just fix this in the fixture (if your existing testcases are unaffected by the added field of course)
  • another nice feature is that anyone can make testcases without any knowledge of soap/xml/webservice. If your fixture is nicely programmed, even business people could write tests. Quite often acceptence testers are not used to testing services without a GUI. With this its possible to do that.
  • last one is that you completely automate it. Quite often acceptance tests are done by hand which take quite some time and effort. Now you automate them and reduce the time you can possibly deliver your software to production.

Some considerations though are:

  • You do need to put in the effort to make the abstraction layer. In the beginning this will take more effort but you will earn this back quite fast when you need to regression tests or so. In the long run….this will pay of!
  • Fitnesse can be used by different languages such as .Net and Java but you will still need people who are capable of this.

CI using Oracle Fusion Middleware 12C: Part 2. Building a SB and SOA project using maven and the MDS

In the part 1 I have shown how to setup a simple CI environment and how to build a Service Bus project using Maven. In this part I will try to make a release pipeline which builds, deploys, tests, packages and release a whole service using Jenkins and if all successful and finally install the artifact in Nexus.

Lets start where we left of. Startup Tomcat and log into Jenkins. We need some sort of plugin to be able to run multiple actions in a sequence. Jenkins has alot of plugins but the one which I am going to use is the MultiJob one. Go to Manage Jenkins->Manage plugins, choose the available tab, check the Multijob plugin and click Install without restart.

Multijob

The multi-job plugin can chain jobs together and share variables and artifact between jobs. You can make very intricate jobs but for now I will keep it simple. I will make 1 job that will:

  1. Build the service bus component which also refers to a SharedObjects project and deploy it to my server
  2. Build the soa component which also refers to the MDS and deploy it to my server
  3. Run the matching soap ui test
  4. If succesfull, install the artifact to nexus

I have created a simple HelloService which first goes to the SB and then routes to a SOA component. The SB component makes use of a SharedObjects SB project which holds the WSDL and XSD. This project is setup so you don’t have to sync between this project and the MDS. This because the SB isn’t able yet to access the MDS. The SOA component does nothing else the return a string response. So the setup will look like this:

HelloService

So the first job we have to create is one which builds and deploys an SB project using maven. We will have to have 2 jobs as we want to deploy the SharedObjects project and we want to deploy the SB service afterwards. Lets make the SharedObject first. This is an easy one as it has no references to any other project.

Create a new Jenkins job called OSB – SharedObjects which is based on a maven project. The .pom file is very simple. This will look like this:

<?xml version="1.0" encoding="UTF-8"?>
<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd"
         xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
	<modelVersion>4.0.0</modelVersion>

	<parent>
		<groupId>com.oracle.servicebus</groupId>
		<artifactId>sbar-project-common</artifactId>
		<version>12.1.3-0-0</version>
	</parent>

	<groupId>nl.cvgz</groupId>
	<artifactId>GedeeldeObjecten</artifactId>
	<version>1.0</version>
	<packaging>sbar</packaging>
	<description/>
</project>

As a maven goal we can just use the following command to build and deploy the SharedObject project to the designated server:

clean pre-integration-test -DoracleHome=${oracleHome} -DoracleServerUrl=${osbServername} -DoracleUsername=${osbUsername} -DoraclePassword=${osbPassword}

You can insert the variables from a property file as you can see. Now lets try this first job and see how it runs.

SharedObjects-jenkins

Started by user Hugo Hendriks
[EnvInject] - Loading node environment variables.
Building in workspace D:\tomcatfiles\.jenkins\jobs\OSB - GedeeldeObjecten\workspace
Cleaning local Directory GedeeldeObjecten
Checking out svn://hendriksh@u10023o/mmi/trunk/OSB/GedeeldeObjecten at revision '2015-02-18T19:18:29.973 +0100'
A         GedeeldeObjecten.jpr
A         apps
A         apps\services
A         apps\services\HelloService
A         apps\services\HelloService\1.0
A         apps\services\HelloService\1.0\HelloService.wsdl
A         apps\services\HelloService\1.0\cdm.xsd
A         apps\services\HelloService\1.0\HelloService.xsd
A         servicebus.sboverview
A         alerts
A         alerts\Log.alert
A         alerts\ErrorDestination.alert
A         pom.xml
 U        .
At revision 927
Cleaning local Directory Build
Checking out svn://hendriksh@u10023o/mmi/trunk/build at revision '2015-02-18T19:18:29.973 +0100'
A         build-env-TST.properties
A         build-env-ONT.properties
At revision 927
no change for svn://hendriksh@u10023o/mmi/trunk/OSB/GedeeldeObjecten since the previous build
no change for svn://hendriksh@u10023o/mmi/trunk/build since the previous build
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties file path 'Build/build-env-ONT.properties'
[EnvInject] - Variables injected successfully.
Parsing POMs
[GedeeldeObjecten] $ D:\ProgramFiles\Java\jdk1.7.0_71/bin/java -cp D:\tomcatfiles\.jenkins\plugins\maven-plugin\WEB-INF\lib\maven31-agent-1.5.jar;D:\ProgramFiles\Apache\apache-maven-3.2.3\boot\plexus-classworlds-2.5.1.jar;D:\ProgramFiles\Apache\apache-maven-3.2.3/conf/logging jenkins.maven3.agent.Maven31Main D:\ProgramFiles\Apache\apache-maven-3.2.3 D:\ProgramFiles\Apache\Tomcat8.0\webapps\jenkins\WEB-INF\lib\remoting-2.48.jar D:\tomcatfiles\.jenkins\plugins\maven-plugin\WEB-INF\lib\maven31-interceptor-1.5.jar D:\tomcatfiles\.jenkins\plugins\maven-plugin\WEB-INF\lib\maven3-interceptor-commons-1.5.jar 58198
<===[JENKINS REMOTING CAPACITY]===>���channel started
Executing Maven:  -B -f D:\tomcatfiles\.jenkins\jobs\OSB - GedeeldeObjecten\workspace\GedeeldeObjecten\pom.xml clean pre-integration-test -DoracleHome=[MY-ORACLEHOME] -DoracleServerUrl=http://[MY-SERVER] -DoracleUsername=[MY-USER] -DoraclePassword=[MY-PASSWORD]
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building GedeeldeObjecten 1.0
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ GedeeldeObjecten ---
[INFO] 
[INFO] --- oracle-servicebus-plugin:12.1.3-0-0:package (default-package) @ GedeeldeObjecten ---
[INFO] 
[INFO] --- oracle-servicebus-plugin:12.1.3-0-0:deploy (default-deploy) @ GedeeldeObjecten ---
[INFO] Service Bus Archive deployed using session Service_Bus_Maven-GedeeldeObjecten-1424283534524.
Diagnostic XML Bean debug log file created: C:\Windows\TEMP\xmlbeandebug7850265000274769465.log
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 02:37 min
[INFO] Finished at: 2015-02-18T19:21:15+01:00
[INFO] Final Memory: 22M/362M
[INFO] ------------------------------------------------------------------------
[JENKINS] Archiving D:\tomcatfiles\.jenkins\jobs\OSB - GedeeldeObjecten\workspace\GedeeldeObjecten\pom.xml to nl.cvgz/GedeeldeObjecten/1.0/GedeeldeObjecten-1.0.pom
[JENKINS] Archiving D:\tomcatfiles\.jenkins\jobs\OSB - GedeeldeObjecten\workspace\GedeeldeObjecten\.data\maven\sbconfig.sbar to nl.cvgz/GedeeldeObjecten/1.0/GedeeldeObjecten-1.0.sbar
D:/tomcatfiles/.jenkins/jobs/OSB - GedeeldeObjecten/workspace/GedeeldeObjecten/pom.xml is not inside D:/tomcatfiles/.jenkins/jobs/OSB - GedeeldeObjecten/workspace/GedeeldeObjecten/GedeeldeObjecten/; will archive in a separate pass
D:/tomcatfiles/.jenkins/jobs/OSB - GedeeldeObjecten/workspace/GedeeldeObjecten/.data/maven/sbconfig.sbar is not inside D:/tomcatfiles/.jenkins/jobs/OSB - GedeeldeObjecten/workspace/GedeeldeObjecten/GedeeldeObjecten/; will archive in a separate pass

channel stopped
Finished: SUCCESS

As you can see….success!

The next bit will be to make a job which deploys the HelloService SB component. This job will actually be quite the same as the previous one except for the fact that it references the SharedObjects project. If you run the normal maven build, it will complain that it can’t find the shared resources. The way to solve this, is to also checkout the SharedObject project during the build in your workspace. This way it can find the project compile time.

Started by user Hugo Hendriks
[EnvInject] - Loading node environment variables.
Building in workspace D:\tomcatfiles\.jenkins\jobs\OSB - Service\workspace
Cleaning local Directory HelloService_v1.0
Checking out svn://hendriksh@u10023o/mmi/trunk/OSB/HelloService_v1.0 at revision '2015-02-18T19:31:12.398 +0100'
A         pom.xml
A         Business
A         Business\HelloService.bix
A         LogPipeline.pipeline
A         HelloPipeline.pipeline
A         Proxy
A         HelloService_v1.0.jpr
A         ValidatePipeline.pipeline
A         HelloService.proxy
A         servicebus.sboverview
At revision 927
Cleaning local Directory GedeeldeObjecten
Checking out svn://hendriksh@u10023o/mmi/trunk/OSB/GedeeldeObjecten at revision '2015-02-18T19:31:12.398 +0100'
A         GedeeldeObjecten.jpr
A         apps
A         apps\services
A         apps\services\HelloService
A         apps\services\HelloService\1.0
A         apps\services\HelloService\1.0\HelloService.wsdl
A         apps\services\HelloService\1.0\cdm.xsd
A         apps\services\HelloService\1.0\HelloService.xsd
A         servicebus.sboverview
A         alerts
A         alerts\Log.alert
A         alerts\ErrorDestination.alert
A         pom.xml
 U        .
At revision 927
Cleaning local Directory Build
Checking out svn://hendriksh@u10023o/mmi/trunk/build at revision '2015-02-18T19:31:12.398 +0100'
A         build-env-TST.properties
A         build-env-ONT.properties
At revision 927
Cleaning local Directory Customization
Checking out svn://hendriksh@u10023o/mmi/trunk/OSB/Customization at revision '2015-02-18T19:31:12.398 +0100'
A         all_BS_ACC.xml
A         all_BS_TST.xml
A         all_BS_ONT.xml
At revision 927
no revision recorded for svn://hendriksh@u10023o/mmi/trunk/OSB/HelloService_v1.0 in the previous build
no change for svn://hendriksh@u10023o/mmi/trunk/build since the previous build
no change for svn://hendriksh@u10023o/mmi/trunk/OSB/Customization since the previous build
[EnvInject] - Injecting environment variables from a build step.
[EnvInject] - Injecting as environment variables the properties file path 'Build/build-env-ONT.properties'
[EnvInject] - Variables injected successfully.
Parsing POMs
Modules changed, recalculating dependency graph
[HelloService_v1.0] $ D:\ProgramFiles\Java\jdk1.7.0_71/bin/java -cp D:\tomcatfiles\.jenkins\plugins\maven-plugin\WEB-INF\lib\maven31-agent-1.5.jar;D:\ProgramFiles\Apache\apache-maven-3.2.3\boot\plexus-classworlds-2.5.1.jar;D:\ProgramFiles\Apache\apache-maven-3.2.3/conf/logging jenkins.maven3.agent.Maven31Main D:\ProgramFiles\Apache\apache-maven-3.2.3 D:\ProgramFiles\Apache\Tomcat8.0\webapps\jenkins\WEB-INF\lib\remoting-2.48.jar D:\tomcatfiles\.jenkins\plugins\maven-plugin\WEB-INF\lib\maven31-interceptor-1.5.jar D:\tomcatfiles\.jenkins\plugins\maven-plugin\WEB-INF\lib\maven3-interceptor-commons-1.5.jar 58247
<===[JENKINS REMOTING CAPACITY]===>���channel started
Executing Maven:  -B -f D:\tomcatfiles\.jenkins\jobs\OSB - Service\workspace\HelloService_v1.0\pom.xml clean pre-integration-test -DoracleHome=[MY-ORACLEHOME] -DoracleServerUrl={MY-SERVER} -DoracleUsername=[MY-USER] -DoraclePassword=[MY-PASSWORD]
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building HelloService 1.0
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ HelloService ---
[INFO] 
[INFO] --- oracle-servicebus-plugin:12.1.3-0-0:package (default-package) @ HelloService ---
[INFO] 
[INFO] --- oracle-servicebus-plugin:12.1.3-0-0:deploy (default-deploy) @ HelloService ---
[INFO] Service Bus Archive deployed using session Service_Bus_Maven-HelloService-1424284297240.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 34.955 s
[INFO] Finished at: 2015-02-18T19:31:56+01:00
[INFO] Final Memory: 23M/226M
[INFO] ------------------------------------------------------------------------
Waiting for Jenkins to finish collecting data
[JENKINS] Archiving D:\tomcatfiles\.jenkins\jobs\OSB - Service\workspace\HelloService_v1.0\pom.xml to nl.cvgz/HelloService/1.0/HelloService-1.0.pom
[JENKINS] Archiving D:\tomcatfiles\.jenkins\jobs\OSB - Service\workspace\HelloService_v1.0\.data\maven\sbconfig.sbar to nl.cvgz/HelloService/1.0/HelloService-1.0.sbar
channel stopped
Finished: SUCCESS

Also success! So we now have 2 separate jobs:

      OSB – SharedObjects = which builds and deploys your SharedObject project.
      OSB – Service = which builds and deploys your HelloService project.

The next thing is to do the same for the SOA component. I actually want a SOA – MDS jenkins job which zips and deploys the MDS but it seems Oracle doesn’t support the deployment of the MDS through maven yet. I have heard rumors though that you can zip it and deploy it using the normal maven plugins. I am able to zip the apps directory but I haven’t been able to deploy it yet so if anyone knows how to do this….let me know :)

The last job is a SOA – Service job which builds and deploys the SOA component. When you create a SOA project you get a pom with it. To make this work with the MDS you do have to tweak it here and there. The problem is that the MDS reference is made through the application which references the adf-config.xml. Lets start of with the adf-config.xml which is located in .adf\META-INF. You should make a MDS like this:

<adf-mds-config xmlns="http://xmlns.oracle.com/adf/mds/config">
    <mds-config xmlns="http://xmlns.oracle.com/mds/config">
      <persistence-config>
        <metadata-namespaces>
          <namespace path="/apps" metadata-store-usage="mstore-usage_apps"/>
        </metadata-namespaces>
        <metadata-store-usages>
          <metadata-store-usage id="mstore-usage_apps">
            <metadata-store class-name="oracle.mds.persistence.stores.file.FileMetadataStore">
              <property name="metadata-path" value="${soamds.apps.home}" />
            </metadata-store>
          </metadata-store-usage>
        </metadata-store-usages>
      </persistence-config>
    </mds-config>
  </adf-mds-config>

As you can see we made the metadata-path for the /apps settable using a variable soamds.apps.home. So now you can set the directory of your apps of your application which holds the soa project. Now for the last bit….the .pom file of you soa project. As I said, you will have to tweak some settings here as it doesn’t completely work out of the box. You should make the following adjustments:

<?xml version="1.0" encoding="UTF-8"?>
<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
         <!--
           This POM was automatically generated during SOA project creation.           
           This POM relates to this SOA Composite, i.e. the one in this same directory.
           There is another POM in the SOA Application directory which handles
           the whole SOA Application, which may contain additional projects. 
        -->
    <modelVersion>4.0.0</modelVersion>
    <groupId>VGZ-SOA-Application</groupId>
    <artifactId>HelloService</artifactId>
    <version>1.0</version>
    <packaging>sar</packaging>
    
    <!--
           The parent points to the common SOA parent POM.  That is a special POM that is
           shipped by Oracle as a point of customization (only). You can add default values
           for properties like serverUrl, etc. to the SOA common parent POM, so that you
           do not have to specify them over and over in every project POM.
    --> 
    <parent>
        <groupId>com.oracle.soa</groupId>
        <artifactId>sar-common</artifactId>
        <version>12.1.3-0-0</version>
    </parent>
    
    <properties>
        <!-- These parameters are used by the compile goal -->
        <scac.input.dir>${project.basedir}\SOA/</scac.input.dir>
        <scac.output.dir>${project.basedir}/target</scac.output.dir>
        <scac.input>${scac.input.dir}/composite.xml</scac.input>
        <scac.output>${scac.output.dir}/out.xml</scac.output>
        <scac.error>${scac.output.dir}/error.txt</scac.error>
        <scac.displayLevel>1</scac.displayLevel>
        <!-- These parameters are used by the deploy and undeploy goals --> 
        <composite.name>${project.artifactId}</composite.name>
        <composite.revision>${project.version}</composite.revision>
        <composite.partition>default</composite.partition>        
        <serverUrl>${oracleServerUrl}</serverUrl>        
        <user>${oracleUsername}</user>
        <password>${oraclePassword}</password>
        <overwrite>true</overwrite>
        <forceDefault>true</forceDefault>
        <regenerateRulebase>false</regenerateRulebase>
        <keepInstancesOnRedeploy>false</keepInstancesOnRedeploy>
        
        <!-- These parameters are used by the test goal 
         if you are using the sca-test (test) goal, you need to uncomment the following
             line and point it to your jndi.properties file. --> 
             
        <!--<jndi.properties.input>UNDEFINED</jndi.properties.input>-->
        <scatest.result>${scac.output.dir}/testResult</scatest.result>
        <!--  input is the name of the composite to run test suties against -->
        <input>${project.artifactId}</input>
        
        <!--<scac.ant.buildfile>${env.MW_HOME}/soa/bin/ant-sca-compile.xml</scac.ant.buildfile>
        <sca.ant.testfile>${env.MW_HOME}/soa/bin/ant-sca-test.xml</sca.ant.testfile>
        -->
        
    </properties>
    <build>
        <plugins>
            <plugin>
                <groupId>com.oracle.soa.plugin</groupId>
                <artifactId>oracle-soa-plugin</artifactId>
                <version>12.1.3-0-0</version>
                <configuration>
                    <compositeName>${project.artifactId}</compositeName>
                    <composite>${scac.input}</composite>
                    <sarLocation>${scac.output.dir}/sca_${project.artifactId}_rev${composite.revision}.jar</sarLocation>
                    <serverUrl>${serverUrl}</serverUrl>
                    <appHome>../../../</appHome>
                    <!-- Note: compositeRevision is needed to package, revision is needed to undeploy -->
                    <compositeRevision>${project.version}</compositeRevision>
                    <revision>${composite.revision}</revision>
					<scacInputDir>${scac.input.dir}</scacInputDir>
					<user>${user}</user>
                    <password>${password}</password>                    
                    <input>${input}</input> 
                </configuration>
                 <!-- extensions=true is needed to use the custom sar packaging type -->
                <extensions>true</extensions>
            </plugin>
        </plugins>
    </build>
</project>

You should add

<appHome>../../../</appHome>

to you plugin. This will have to point to the place where your application is located and thus where your .adf/META-INF/adf-config.xml is also. In my case, that is 3 folders up when I do a checkout of my svn. You also have to fix ${project.version} as it will by default generates a 1.0-SNAPSHOT version but it will look for a 1.0 to deploy.

After you have done all this lets create a Jenkins job called SOA – Service and run the following maven command:

clean pre-integration-test -DoracleServerUrl=${soaServername} -DoracleUsername=${soaUsername} -DoraclePassword=${soaPassword}

Don’t forget to set soamds.apps.home=${workspace}\GedeeldeObjecten as environment variable and to also checkout the application.jws including .adf/META-INF/adf-config.xml. Now lets give the job a spin:

Started by user Hugo Hendriks
[EnvInject] - Loading node environment variables.
Building in workspace D:\tomcatfiles\.jenkins\jobs\SOA - Service\workspace
Cleaning local Directory .
Checking out svn://hendriksh@u10023o/mmi/trunk/services at revision '2015-02-18T20:23:09.684 +0100'
A         VGZ-SOA-Application.jws
 U        .
At revision 927
Cleaning local Directory .adf
Checking out svn://hendriksh@u10023o/mmi/trunk/services/.adf at revision '2015-02-18T20:23:09.684 +0100'
A         META-INF
A         META-INF\adf-config.xml
At revision 927
Cleaning local Directory GedeeldeObjecten/apps
Checking out svn://hendriksh@u10023o/mmi/trunk/OSB/GedeeldeObjecten/apps at revision '2015-02-18T20:23:09.684 +0100'
A         services\HelloService
A         services\HelloService\1.0
A         services\HelloService\1.0\HelloService.xsd
A         services\HelloService\1.0\HelloService.wsdl
A         services\HelloService\1.0\cdm.xsd
 U        .
At revision 927
Cleaning local Directory HelloService_v1.0/soa/HelloService_v1.0
Checking out svn://hendriksh@u10023o/mmi/trunk/services/HelloService_v1.0/soa/HelloService_v1.0 at revision '2015-02-18T20:23:09.684 +0100'
A         HelloService_v1.0.jpr
A         SOA
A         SOA\composite.xml
A         SOA\Schemas
A         SOA\Events
A         SOA\Mediators
A         SOA\Mediators\HelloMediator.mplan
A         SOA\measurements.xml
A         SOA\Transformations
A         SOA\WSDLs
A         SOA\testsuites
A         SOA\testsuites\fileList.xml
A         pom.xml
 U        .
At revision 927
Cleaning local Directory Build
Checking out svn://hendriksh@u10023o/mmi/trunk/build at revision '2015-02-18T20:23:09.684 +0100'
A         build-env-TST.properties
A         build-env-ONT.properties
At revision 927
no change for svn://hendriksh@u10023o/mmi/trunk/services/.adf since the previous build
no revision recorded for svn://hendriksh@u10023o/mmi/trunk/services/HelloService_v1.0/soa/HelloService_v1.0 in the previous build
no change for svn://hendriksh@u10023o/mmi/trunk/build since the previous build
[EnvInject] - Injecting environment variables from a build step.
[EnvInject] - Injecting as environment variables the properties file path 'Build/build-env-ONT.properties'
[EnvInject] - Variables injected successfully.
[EnvInject] - Injecting as environment variables the properties content 
soamds.apps.home=${workspace}\GedeeldeObjecten

[EnvInject] - Variables injected successfully.
Parsing POMs
Modules changed, recalculating dependency graph
[HelloService_v1.0] $ D:\ProgramFiles\Java\jdk1.7.0_71/bin/java -cp D:\tomcatfiles\.jenkins\plugins\maven-plugin\WEB-INF\lib\maven31-agent-1.5.jar;D:\ProgramFiles\Apache\apache-maven-3.2.3\boot\plexus-classworlds-2.5.1.jar;D:\ProgramFiles\Apache\apache-maven-3.2.3/conf/logging jenkins.maven3.agent.Maven31Main D:\ProgramFiles\Apache\apache-maven-3.2.3 D:\ProgramFiles\Apache\Tomcat8.0\webapps\jenkins\WEB-INF\lib\remoting-2.48.jar D:\tomcatfiles\.jenkins\plugins\maven-plugin\WEB-INF\lib\maven31-interceptor-1.5.jar D:\tomcatfiles\.jenkins\plugins\maven-plugin\WEB-INF\lib\maven3-interceptor-commons-1.5.jar 58351
<===[JENKINS REMOTING CAPACITY]===>���channel started
Executing Maven:  -B -f D:\tomcatfiles\.jenkins\jobs\SOA - Service\workspace\HelloService_v1.0\soa\HelloService_v1.0\pom.xml clean pre-integration-test -DoracleServerUrl=[MY-SERVER] -DoracleUsername=[MY-USER] -DoraclePassword=[MY-PASSWORD]
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building HelloService 1.0
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ HelloService ---
[INFO] 
[INFO] --- maven-resources-plugin:2.7:resources (default-resources) @ HelloService ---
[WARNING] Using platform encoding (Cp1252 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory D:\tomcatfiles\.jenkins\jobs\SOA - Service\workspace\HelloService_v1.0\soa\HelloService_v1.0\src\main\resources
[INFO] 
[INFO] --- oracle-soa-plugin:12.1.3-0-0:compile (default-compile) @ HelloService ---
[INFO] ------------------------------------------------------------------------
[INFO] ORACLE SOA MAVEN PLUGIN - COMPILE
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] ABOUT TO RUN oracle.soa.scac.ValidateComposite...
[INFO] compile: Executing: [cmd:[D:\ProgramFiles\Java\jdk1.7.0_71\bin\java, -Djava.protocol.handler.pkgs=oracle.mds.net.protocol|oracle.fabric.common.classloaderurl.handler|oracle.fabric.common.uddiurl.handler, oracle.soa.scac.ValidateComposite, D:\tomcatfiles\.jenkins\jobs\SOA - Service\workspace\HelloService_v1.0\soa\HelloService_v1.0\SOA//composite.xml, -level=1, -appHome=../../../]]
[INFO] Process being executed, waiting for completion.
[INFO] [exec] 2015-02-18 20:23:35.950/4.421 Oracle Coherence 12.1.3.0.0 <Info> (thread=main, member=n/a): Loaded operational configuration from "jar:file:/D:/ProgramFiles/Apache/apache-maven-repository/com/oracle/coherence/coherence/12.1.3-0-0/coherence-12.1.3-0-0.jar!/tangosol-coherence.xml"
[INFO] [exec] 2015-02-18 20:23:36.091/4.562 Oracle Coherence 12.1.3.0.0 <Info> (thread=main, member=n/a): Loaded operational overrides from "jar:file:/D:/ProgramFiles/Apache/apache-maven-repository/com/oracle/coherence/coherence/12.1.3-0-0/coherence-12.1.3-0-0.jar!/tangosol-coherence-override-dev.xml"
[INFO] [exec] 2015-02-18 20:23:36.091/4.562 Oracle Coherence 12.1.3.0.0 <D5> (thread=main, member=n/a): Optional configuration override "/tangosol-coherence-override.xml" is not specified
[INFO] [exec] 2015-02-18 20:23:36.107/4.578 Oracle Coherence 12.1.3.0.0 <D5> (thread=main, member=n/a): Optional configuration override "cache-factory-config.xml" is not specified
[INFO] [exec] 2015-02-18 20:23:36.107/4.578 Oracle Coherence 12.1.3.0.0 <D5> (thread=main, member=n/a): Optional configuration override "cache-factory-builder-config.xml" is not specified
[INFO] [exec] 2015-02-18 20:23:36.107/4.578 Oracle Coherence 12.1.3.0.0 <D5> (thread=main, member=n/a): Optional configuration override "/custom-mbeans.xml" is not specified
[INFO] [exec] 
[INFO] [exec] Oracle Coherence Version 12.1.3.0.0 Build 52031
[INFO] [exec]  Grid Edition: Development mode
[INFO] [exec] Copyright (c) 2000, 2014, Oracle and/or its affiliates. All rights reserved.
[INFO] [exec] 
[INFO] [exec] 2015-02-18 20:23:36.247/4.718 Oracle Coherence GE 12.1.3.0.0 <Info> (thread=main, member=n/a): Created cache factory com.tangosol.net.DefaultConfigurableCacheFactory
[INFO] [exec] Feb 18, 2015 8:23:37 PM oracle.fabric.common.wsdl.SchemaManager isIncrementalBuildSupported
[INFO] [exec] INFO: XMLSchema incremental build enabled.
[INFO] [exec] Mediators/HelloMediator.mplan: warning: Assigning property/constant "concat("Hallo ", $in.payload/tns:HelloRequest/tns:Naam)" to element "$out.payload/tns:HelloResponse/tns:begroeting". Please make sure target is single leaf node, otherwise non-leaf node will contain only string value which may generate non-valid xml as per the xsd.
[INFO] compile: [cmd:[D:\ProgramFiles\Java\jdk1.7.0_71\bin\java, -Djava.protocol.handler.pkgs=oracle.mds.net.protocol|oracle.fabric.common.classloaderurl.handler|oracle.fabric.common.uddiurl.handler, oracle.soa.scac.ValidateComposite, D:\tomcatfiles\.jenkins\jobs\SOA - Service\workspace\HelloService_v1.0\soa\HelloService_v1.0\SOA//composite.xml, -level=1, -appHome=../../../]] exit code=0
[INFO] SOA COMPILE DONE
[INFO] 
[INFO] --- maven-resources-plugin:2.7:testResources (default-testResources) @ HelloService ---
[WARNING] Using platform encoding (Cp1252 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory D:\tomcatfiles\.jenkins\jobs\SOA - Service\workspace\HelloService_v1.0\soa\HelloService_v1.0\src\test\resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.2:testCompile (default-testCompile) @ HelloService ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.18.1:test (default-test) @ HelloService ---
[INFO] No tests to run.
[JENKINS] Recording test results
[INFO] 
[INFO] --- oracle-soa-plugin:12.1.3-0-0:sar (default-sar) @ HelloService ---
[INFO] Building sar: D:\tomcatfiles\.jenkins\jobs\SOA - Service\workspace\HelloService_v1.0\soa\HelloService_v1.0\target\sca_HelloService_rev1.0.jar
[INFO] 
[INFO] --- oracle-soa-plugin:12.1.3-0-0:deploy (default-deploy) @ HelloService ---
[INFO] ------------------------------------------------------------------------
[INFO] ORACLE SOA MAVEN PLUGIN - DEPLOY COMPOSITE
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] setting user/password..., user=weblogic
Processing sar=D:\tomcatfiles\.jenkins\jobs\SOA - Service\workspace\HelloService_v1.0\soa\HelloService_v1.0/target/sca_HelloService_rev1.0.jar
Adding sar file - D:\tomcatfiles\.jenkins\jobs\SOA - Service\workspace\HelloService_v1.0\soa\HelloService_v1.0\target\sca_HelloService_rev1.0.jar
INFO: Creating HTTP connection to host:[MY-SERVER], port:[MY-PORT]
INFO: Received HTTP response from the server, response code=200
---->Deploying composite success.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 21.328 s
[INFO] Finished at: 2015-02-18T20:23:41+01:00
[INFO] Final Memory: 27M/356M
[INFO] ------------------------------------------------------------------------
Waiting for Jenkins to finish collecting data
[JENKINS] Archiving D:\tomcatfiles\.jenkins\jobs\SOA - Service\workspace\HelloService_v1.0\soa\HelloService_v1.0\pom.xml to VGZ-SOA-Application/HelloService/1.0/HelloService-1.0.pom
[JENKINS] Archiving D:\tomcatfiles\.jenkins\jobs\SOA - Service\workspace\HelloService_v1.0\soa\HelloService_v1.0\target\sca_HelloService_rev1.0.jar to VGZ-SOA-Application/HelloService/1.0/HelloService-1.0.jar
channel stopped
Archiving artifacts
Finished: SUCCESS

Perfect! As a final we can make a job to run a unit test against the just deployed server. I already made a description of how to do this here.

Now for the multijob. This will be a Release-Service job which calls all of the above step in a specific order. In the end you can use the CopyArtifact plugin to copy all of the generated artifacts into the Release-Service job. You can then zip these. I’m using the promote plugin to authorize the upload of the zip into Nexus. So the job looks like this now:

Release-Service

The stars on the left side will tell you that these where builds which where promoted and ended up in Nexus. I’m using a windows batch command to upload the zip to Nexus like this:

mvn deploy:deploy-file -Durl=[MY-SERVER] -DrepositoryId=local-nexus -DgroupId=nl.cvgz.mmi.services -DartifactId=%servicenaamkort% -Dversion=%serviceversie% -Dpackaging=zip -Dfile=target/%servicenaam%.zip

And finally they will end up in Nexus then. A zip which will contain:

  • OSB/SharedObject.sbar
  • OSB/Service.sbar
  • OSB/Customizations. Directory with customizations files for DTAP
  • SOA/MDS.zip
  • SOA/Service.jar

Nexus

So this could possibly be a way to automatically build, deploy, test, package and release a whole service using Jenkins. Some issues which I haven’t solved yet:

  • It seems adding a customization file during the deploy of the SB component gives an error. Could be a bug though.
  • Deploying the MDS.zip to the soa server through maven. If you know how to do this….drop me a line please

Consuming a service in Oracle Service Bus 12C using the REST adapter

In a previous post I have shown how to create a REST service in 12C. That was easy. But how do we consume a REST service? In this next bit I will show how to do that. The REST service we are going to use is Google’s Geocoding API. You can use this API to input address data and retrieve geo data in both JSON or XML. For example:

GoogleGeo

To consume this service in 11G we had to do quite some work….see here. Lets see if it has become more easy now in 12C.

First we are going to create the service how we are going to expose our service. We have the XSD and WSDL from the previous project so we are just going to reuse that. Create a new ServiceBus project GeoService and drag a new Pipeline onto your canvas. Name your pipeline GeoServicePipeline and click next. Next define its interface by selecting the WSDL using the icon with the green arrow. As you can see my WSDL is located in a folder called wsdl. Also check the box below ‘Expose as a Proxy Service’ and click Finish.

WSDL

As you can see we have now a simple proxy service doing nothing.

Skeleton

We now have to hook it up to the Google backend. As seen in the previous post, a WADL is a description of how to use a REST service, comparable to a WSDL for SOAP services. So basically I am looking for a WADL for the Geocode API. After some looking around I found a SoapUI project containing a few Google API services with their created WADL’s in there. So basically I grabbed the WADL from there and I did some tweaking and tuning as I saw that when creating the REST adapter in JDeveloper, it was picky with some settings. The WADL I created looks as followed:

<application xmlns="http://wadl.dev.java.net/2009/02"
    xmlns:soa="http://www.oracle.com/soa/rest" 
    xmlns:xsd="http://www.w3.org/2001/XMLSchema" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:sch="http://www.google.com/geocode/service/schema">
   <doc xml:lang="en" title="Geocoding API"/>
   <grammars>
      <xsd:schema targetNamespace="http://www.google.com/geocode/service/imports" elementFormDefault="qualified" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
            <xsd:import schemaLocation="GoogleGeocode.xsd" namespace="http://www.google.com/geocode/service/schema"/>
        </xsd:schema>
   </grammars>
   <resources base="http://maps.googleapis.com">
      <resource path="maps/api/geocode/xml" id="geocode">
         <doc xml:lang="en" title="geocode"/>
         <param name="address" style="query" soa:expression="$msg.parameters/sch:address" default="" type="xsd:string"/>
         <param name="sensor" style="query" soa:expression="$msg.parameters/sch:sensor" default="false" type="xsd:string"/>
         <param name="language" style="query" soa:expression="$msg.parameters/sch:language" default="EN" type="xsd:string"/>
         <method name="GET" id="GET">
            <doc xml:lang="en" title="GET"/>
            <request/>
            <response status="200">
               <representation mediaType="application/xml" element="sch:GeocodeResponse" xmlns:sch="http://www.google.com/geocode/service/schema"/>
            </response>
         </method>
      </resource>
   </resources>
</application>

with matching XSD:

<?xml version = '1.0' encoding = 'UTF-8'?>
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" elementFormDefault="qualified"
            xmlns:nxsd="http://xmlns.oracle.com/pcbpel/nxsd" nxsd:encoding="US-ASCII"
            xmlns:tns="http://www.google.com/geocode/service/schema"
            targetNamespace="http://www.google.com/geocode/service/schema">
  <xsd:element name="address" type="xsd:string"/>
  <xsd:element name="language" type="xsd:string"/>
  <xsd:element name="sensor" type="xsd:string"/>
  <xsd:element name="GeocodeResponse">
    <xsd:complexType>
      <xsd:sequence>
        <xsd:element name="status" type="xsd:string"/>
        <xsd:element name="result">
          <xsd:complexType>
            <xsd:sequence>
              <xsd:element name="type" type="xsd:string"/>
              <xsd:element name="formatted_address" type="xsd:string"/>
              <xsd:element name="address_component" maxOccurs="unbounded">
                <xsd:complexType>
                  <xsd:sequence>
                    <xsd:element name="long_name" type="xsd:string"/>
                    <xsd:element name="short_name" type="xsd:string"/>
                    <xsd:element name="type" maxOccurs="unbounded" type="xsd:string"/>
                  </xsd:sequence>
                </xsd:complexType>
              </xsd:element>
              <xsd:element name="geometry">
                <xsd:complexType>
                  <xsd:sequence>
                    <xsd:element name="location">
                      <xsd:complexType>
                        <xsd:sequence>
                          <xsd:element name="lat" type="xsd:double"/>
                          <xsd:element name="lng" type="xsd:double"/>
                        </xsd:sequence>
                      </xsd:complexType>
                    </xsd:element>
                    <xsd:element name="location_type" type="xsd:string"/>
                    <xsd:element name="viewport">
                      <xsd:complexType>
                        <xsd:sequence>
                          <xsd:element name="southwest">
                            <xsd:complexType>
                              <xsd:sequence>
                                <xsd:element name="lat" type="xsd:double"/>
                                <xsd:element name="lng" type="xsd:double"/>
                              </xsd:sequence>
                            </xsd:complexType>
                          </xsd:element>
                          <xsd:element name="northeast">
                            <xsd:complexType>
                              <xsd:sequence>
                                <xsd:element name="lat" type="xsd:double"/>
                                <xsd:element name="lng" type="xsd:double"/>
                              </xsd:sequence>
                            </xsd:complexType>
                          </xsd:element>
                        </xsd:sequence>
                      </xsd:complexType>
                    </xsd:element>
                  </xsd:sequence>
                </xsd:complexType>
              </xsd:element>
            </xsd:sequence>
          </xsd:complexType>
        </xsd:element>
      </xsd:sequence>
    </xsd:complexType>
  </xsd:element>
</xsd:schema>

Stick these 2 files in a folder thirdparty under you project. Now open your composite and drag the REST Adapter component to the External Service swimlane.

RestAdapter

Input the name and then click the green + at the bottom and select ‘Add operations based on WADL service’. Now select the WADL in the thirdparty folder and click OK. As you can see, the resource path and operation bindings are filled in. Also when you select the GET binding and click the pencil. You can see that both the request and response parameters are set.

RequestResponse

Now click on OK and JDeveloper will generate a business service, a WADL and a WSDL for you. The WADL and WSDL are under Resources but the business service is in the root. To keep it clean I move the business service to the business folder. It might disappear from your swim lane but you can just drag it onto it again from your project.

Composite

Now lets hook at all together. Connect the Pipeline with the REST adapter and open the pipeline. Add a Replace action in the Request route pipeline with the following xpath.

XpathInput

Don’t forget the namespaces. Now also input a Replace action in the Response route pipeline with the following xpath.

OutputXpath

In the Route set the Operation to GET.

Route

Save the whole project and deploy it. Open your EM and lets see if the service works.

Test

As you can see, the longitude and latitude are returned in the correct manner.

So, basically the conclusion is that the new 12C REST Adapter really makes your life easier when consuming a REST service. The only thing you might struggle with a bit is to get the WADL correct but once you get the hang if it, it is not too bad.

Creating a REST service in Oracle Service Bus 12C

In this first REST post I will demonstrate how to make easily expose a REST service in the Service Bus 12C. I will not explain REST as quite some people have already written tons of stuff about this….Google is your friend :) . Exposing a service in a REST way using the new 12C release is quite simple. Lets try and expose the ConversionRateService we previously build as a REST service.

Right click the Pipeline you want to expose and select Expose as REST.

ExposeAsREST

A popup opens where you can define the name of the service and it’s resource path. In our case I will name it ConversionRateRestService and will set its resource path to /json. You can do this by pressing the little pencil on the right.

ResourcePath

The only thing we now have to do is set the operational bindings. Click the GetConversionRate operation below and click the pencil. Another popup opens where you can set the HTTP verb which will be GET in our case. Down below you can see the request parameters and the expression of how to use them.

Binding

Now click the Response tab to define in which form we want our data returned. De-select the XML box and select the JSON box as we want our service to return JSON instead of XML.

JSON

Then press OK twice and voila. You have exposed your service!

If you want you can also expose another REST service which will return XML. Follow the previous steps but you have to remember that you cannot have two proxy services with the same Endpoint URI. So what I did, I created 2 proxy services with a different URI…..for example /financial/conversionraterestservice/1.0/xml for the xml version and /financial/conversionraterestservice/1.0/json for the JSON version. When you do that, you can leave the Resource Path of the REST binding on /

Binding

This way you can easily expose your functionality in three ways: SOAP

ExposeAsSOAP

or REST with JSON

RESTWithJSON

or REST with XML

RESTWithXML

In the next post I will show how to consume a REST service using the 12C Service Bus

Creating a template in Oracle Service Bus 12C

In this article I will show how to use the new templating functionality in Oracle Service Bus 12C. When building a lot of services through different projects, you will always duplicate certain functionality such as Logging, Validation etc… All conform your current development rules and guidelines of the project. This means tedious copying and pasting of certain pieces of code and the old OEPE IDE isn’t always precise with that. 12C has a nice feature now which gives you the opportunity to template certain pieces of code and re-use them in other projects. Lets see how that works.

I am using the project I created in my previous post. You can download a zip containing the project at the bottom so you can start from there. The conversionrate service is still very plain. As I would like to follow the VETRO pattern, I would like to add validation. So lets add a validation pipeline to the flow. Drag the Pipeline icon from the resource pallet onto the Piplelines/Split Joins lane. Choose a name and choose next. Select our own WDSL located in the wsdl folder and unselect Expose as a Proxy Service option as we already have a proxy.

validationPipeline

Next delete the wire between the proxy and the ConversionRateServicePipeline and connect the proxy to the new ValidationPipeline and the ValidationPipeline to the ConversionRateServicePipeline.

pipeline

Now lets implement the validation. Double click the ValidationPipeline. As you can see it has already set the routing to the ConversionRateServicePipeline for you. Lets add a ValidationPipelinePairNode and a request and response stage as you would normally do and add the validations steps.

validatePipelinePair

Next deploy and run a SoapUI test with an invalid enumeration type. You should get the following result:

validationFault

But we want some custom reply instead of a SoapFault so lets make that happen. We add an errorhandler to our Pipeline and catch the specific OSB-382505 error which is a validation fault. In that case we transform the Fault into a neat message for the client. The Replace would look like this.

<sch:Messages>
{
  for $message in $fault/ctx:details/con1:ValidationFailureDetail/con1:message
  return
    <sch:Message>
      <sch:Code>E-001</sch:Code>
      <sch:Description>{data($message/text())}</sch:Description>
    </sch:Message>
}
</sch:Messages>

Don’t forget to add the namespace with the value http://www.bea.com/wli/sb/stages/transform/config and the namespace with value http://www.redrock-it.nl/conversionrate/service/schema. Now, if you save and deploy and use SoapUI to make an invalid call, you will see the following result.

validationFaultProper

Now for the template creation part. We will want to re-use the ValidationPipeline in other services so what we do we click New->Pipeline Template. We give the template a name and select a location. Unfortunately it seems we cannot create it in a directory outside the project so lets just choose a template directory we just created. Select next and select for any SOAP and click finish and voila……we have a template for validation.

createTemplate

But how do we use it? Very simple. When you add a new Pipeline to the Pipelines/Split Join lane, you can choose a template at the bottom of the dialog. Choose the template you just created and voila….you have a completely filled Pipeline ready to go.

usingTemplate