Advanced java code dealing with real world problems.

Friday, September 24, 2010

Java COBOL data exchange - Quick Start Guide

Java has rapidly grown into enterprise landscape and the needs for integrating with mainframe world through JMS/MQ or CICS Transaction Gateway has become a critical piece to many enterprise IT stacks.

JavaCobolExchanger - an open source google code project is developed to address the needs.

Here is the link to the project: http://code.google.com/a/eclipselabs.org/p/java-cobol-exchanger/

To get started first we need to understand COBOL copy books. A copy book is a data description of data layout in fixed format. For example:

01 MY-EXCHANGE-RECORD.
  05 USER-ID          PIC X(20).
  05 USER-AGE         PIC 9(3).
  05 USER_NAME.
    10 FIRST-NAME     PIC X(20).
    10 LAST-NAME      PIC X(20).
    10 MIDDLE_INITIAL PIC X(1).
  05 AMOUNT-RECEIVED  PIC 9(5).99.
  05 DATE-REGISTERED.
    10 DATE-CCYY      PIC 9(4).
    10 DATE-SEP1      PIC X.
    10 DATE-MM        PIC 9(2).
    10 DATE-SEP2      PIC X.
    10 DATE-DD        PIC 9(2).

This copy book tells us that the first 20 characters are reserved for user Id, the next 3 characters are reserved for user age and it must be numeric. The next 41 characters are reserved for a group called USER-NAME and the group contains three elements. Followed after the group is a decimal element with equivalent decimal format of  "00000.00". The last 10 characters are reserved for a date element, which again is a group consisting of five elements, with equivalent date format of "yyyy-MM-dd".

To construct a java exchange instance of above COBOL copy book:

public class CobolCopybook extends ExchangeRecord {

    public CobolCopybook() {
        // define user name group
        BaseElement userNameGroup[] = {
                new StringElement("firstName", 20),
                new StringElement("lastName", 20),
                new StringElement("middleInitial",1),
        };
    
        // define the exchange record
        list.add(new StringElement("userId", 20));
        list.add(new IntegerElement("userAge", 3));
        list.add(new StructElement("userName", userNameGroup));
        list.add(new DecimalElement("amount", "00000.00"));
        list.add(new DateTimeElement("date", "yyyy-MM-dd"));
    }
}

To load the instance with data, simply add the following code:


ExchangeRecord bean = new CobolCopybook();
bean.getElement("userId").setValue("test user");
bean.getElement("userAge").setValue("35");
StructElement userNameGroup = (StructElement) bean.getElement("userName");
userNameGroup.getElement("firstName").setValue("John");
userNameGroup.getElement("lastName").setValue("Smith");
bean.getElement("amount").setValue("199.99");
bean.getElement("date").setValue("2010-01-01");






Now you can export the instance to a fixed format string which can then be consumed by a COBOL program:
 
String cobolString = bean.exportToString();
// XXX: add your code here to send it to mainframe

The data exchange can be performed in either direction, we just demonstrated a data exchange from java to COBOL. Now let's see how we handle exchange from a COBOL output:

First we need create an java exchange instance:

ExchangeRecord bean = new ComplexCopybook();

Then we import the COBOL output to the instance:

bean.importFromString();

And now we can access the COBOL data using getValue method:

String userId = bean.getElement("userId").getValue();
Integer userAge = (Integer) bean.getElement("userAge").getValue();

Thursday, September 2, 2010

A fast XML schema validator

JAXP 1.3 introduced a SchemaFactory by which you can compile a schema from a xsd file, and use the compiled schema to create a Validator that can be used to validate a XML document. Since the schema compilation takes some time, it would be beneficial in a service oriented environment to reuse the compiled schemas for future requests. Presented here is a simple SchemaValidator class that implements the idea with a hash table.

 /*
 * blog/javaclue/xml/SchemaValidator.java
 *
 * Copyright (C) 2009 JackW
 *
 * This program is free software: you can redistribute it and/or modify it under the terms of the
 * GNU Lesser General Public License as published by the Free Software Foundation, either version 3
 * of the License, or (at your option) any later version.
 *
 * This library is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without
 * even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
 * Lesser General Public License for more details.
 *
 * You should have received a copy of the GNU Lesser General Public License along with this library.
 * If not, see .
 */

package blog.javaclue.xml;

import java.io.IOException;
import java.io.InputStream;
import java.net.URL;
import java.util.Hashtable;
import java.util.Map;

import javax.xml.XMLConstants;
import javax.xml.parsers.ParserConfigurationException;
import javax.xml.transform.Source;
import javax.xml.transform.dom.DOMSource;
import javax.xml.transform.stream.StreamSource;
import javax.xml.validation.Schema;
import javax.xml.validation.SchemaFactory;
import javax.xml.validation.Validator;

import org.apache.log4j.Logger;
import org.w3c.dom.Document;
import org.w3c.dom.Element;
import org.xml.sax.SAXException;

/**
 * Validate XML against schema using SchemaFactory (JAXP 1.3).
 * Schema compilation usually takes time, this class saves the compiled schemas
 * in a hash table so they can be reused by future requests and other instances.
 */
public class SchemaValidator {
    protected static Logger logger = Logger.getLogger(SchemaValidator.class);
    protected static boolean isDebugEnabled = logger.isDebugEnabled();
  
    private final String schema_path;
    private final String schema_file;
    private static final Map schemaPool = new Hashtable();
  
    public SchemaValidator(String schema_path, String schema_file) {
        this.schema_path = schema_path;
        this.schema_file = schema_file;
    }

    /**
     * Validate xmlStream against schema
     *
     * @throws IOException
     * @throws SAXException
     * @throws ParserConfigurationException
     */
    public void validate(Document xml_doc) throws ParserConfigurationException, SAXException,
            IOException {
        validate(xml_doc.getDocumentElement());
    }

    public void validate(Element element) throws SAXException, IOException {
        Schema schema = compileSchema(schema_path + schema_file);
      
        // create a Validator instance, which can be used to validate an
        // instance document
        Validator validator = schema.newValidator();
        validator.setErrorHandler(new SaxErrorHandler());
      
        // validate the DOM tree
        validator.validate(new DOMSource(element));
    }

    private static Schema compileSchema(String schemaFile) throws SAXException {
        if (!schemaPool.containsKey(schemaFile)) {
            if (isDebugEnabled) {
                logger.info("compileSchema() - compile schema file: " + schemaFile);
            }
            // create a SchemaFactory capable of understanding W3C schemas
            SchemaFactory factory = SchemaFactory.newInstance(XMLConstants.W3C_XML_SCHEMA_NS_URI);

            factory.setErrorHandler(new SaxErrorHandler());
           
            ClassLoader loader = Thread.currentThread().getContextClassLoader();
            URL url = loader.getResource(schemaFile);
            if (url == null) {
                throw new RuntimeException("Could not find Schema file: " + schemaFile);
            }
            InputStream is = loader.getResourceAsStream(schemaFile);
            // load a schema, represented by a Schema instance
            Source schemasrc = new StreamSource(is, url.getPath());
            Schema schema = factory.newSchema(schemasrc);
            schemaPool.put(schemaFile, schema);
        }
        return schemaPool.get(schemaFile);
    }

    public static synchronized void removeFromPool(String schemaFile) {
        schemaPool.remove(schemaFile);
    }

    public static synchronized void clearPool() {
        schemaPool.clear();
    }
  
    public static void main(String[] args) {
        SchemaValidator test = new SchemaValidator("Schemas/", "MySampleSchema.xsd");
        try {
            Document doc = XMLHelper.loadDocument("Test_xmls/MySampleXml.xml");
            test.validate(doc);
        }
        catch (Exception e) {
            e.printStackTrace();
        }
    }
}

Simple IBM MQ Message Age Monitor

For those that are still running IBM MQ (or Websphere MQ) on mainframes, presented here is a simple java message age monitor program that polls a queue periodically for how long the oldest message has been living in the queue. This sample code will output a message to the console when the oldest message in the queue has been living in there for more that 60 seconds at the moment of polling. You can change it to send out an email or snmp alert instead.
In order to compile and run this program, PCF package (MS0B) is needed and can be downloaded from IBM. You will also need "com.ibm.mq.jar" and IBM's "j2ee.jar". They can be found from IBM's WSAD or RAD IDE development tools.
/*
 * blog/javaclue/ibmmq/MessageAgeMonitor.java
 * 
 * Copyright (C) 2009 JackW
 * 
 * This program is free software: you can redistribute it and/or modify it under the terms of the
 * GNU Lesser General Public License as published by the Free Software Foundation, either version 3
 * of the License, or (at your option) any later version.
 * 
 * This library is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without
 * even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
 * Lesser General Public License for more details.
 * 
 * You should have received a copy of the GNU Lesser General Public License along with this library.
 * If not, see <http://www.gnu.org/licenses/>.
 */
package blog.javaclue.ibmmq;

import java.util.GregorianCalendar;

import org.apache.log4j.Logger;

import com.ibm.mq.MQC;
import com.ibm.mq.MQEnvironment;
import com.ibm.mq.MQException;
import com.ibm.mq.MQGetMessageOptions;
import com.ibm.mq.MQMessage;
import com.ibm.mq.MQQueue;
import com.ibm.mq.MQQueueManager;

/**
 * Simple message age monitor program that uses IBM MQ Java to read PCF-format
 * messages from a queue.
 */
public class MessageAgeMonitor implements Runnable {
 protected static Logger logger = Logger.getLogger(MessageAgeMonitor.class);
 protected static boolean isDebugEnabled = logger.isDebugEnabled();
 
 final private String qmgrName;
 final private String host;
 final private int port;
 final private String queueName;
 final private int alertAge; // in seconds
 final private String channel;
 
 final static int Polling_Freq = 30 * 1000; // 30 seconds

 MessageAgeMonitor(String qmgrName, String host, int port, String queueName, String channel,
   int alertAge) {
  this.qmgrName = qmgrName;
  this.host = host;
  this.port = port;
  this.channel = channel;
  this.queueName = queueName;
  this.alertAge = alertAge;
 }
 
 public void run() {
  if (isDebugEnabled)
   logger.debug("Starting Message Age monitor for " + queueName + "...");
  while (true) {
   checkAge();
   try {
    Thread.sleep(Polling_Freq); // sleep for 30 seconds
   }
   catch (InterruptedException e) {
    logger.info("The monitor has been interrupted, exit...");
    break;
   }
  }
 }

 private void checkAge() {
  MQQueueManager qm = null;
  MQQueue queue = null;
  if (isDebugEnabled) {
   logger.debug("Connecting to " + qmgrName + " at " + host + ":" + port + " over " + channel);
  }
        try {
   // Turn off unnecessary output before we start
   MQEnvironment.disableTracing ();
         MQException.log = null;
   MQEnvironment.hostname = host;
   MQEnvironment.port = port;
   MQEnvironment.channel = channel;
   qm = new MQQueueManager ("");
        queue = qm.accessQueue (queueName, MQC.MQOO_BROWSE | MQC.MQOO_FAIL_IF_QUIESCING);
            MQMessage message = new MQMessage ();
            MQGetMessageOptions gmo = new MQGetMessageOptions ();
            
            gmo.options = MQC.MQGMO_BROWSE_FIRST | MQC.MQGMO_NO_WAIT | MQC.MQGMO_CONVERT;
         
         message.messageId = null;
         message.correlationId = null;
         queue.get (message, gmo);
         // get message put date time
         GregorianCalendar cal = message.putDateTime;
         long ageInMillis = new java.util.Date().getTime() - cal.getTime().getTime();
         int ageInSeconds = (int) ageInMillis/1000;
         if (isDebugEnabled)
    logger.debug("Put Date: " + cal.getTime() + " age in seconds: " + ageInSeconds);
   if (ageInSeconds > alertAge) {
    logger.info(qmgrName + "/" + queueName + " age = " + ageInSeconds
      + ", exceeded alert threshold: " + alertAge);
    // XXX: add your code here to send out alert
   }
        }
        catch (MQException mqe) {
         if (mqe.reasonCode == MQException.MQRC_NO_MSG_AVAILABLE) {
          if (isDebugEnabled) {
           logger.debug("Queue " + qmgrName + "/" + queueName + " is empty.");
          }
         }
         else {
          logger.error("MQException caught", mqe);
         }
        }
        finally {
         if (queue != null && queue.isOpen()) {
          try {
           queue.close();
          }
          catch (Exception e) {
     logger.error("Exception caught during queue.close()", e);
          }
         }
   if (qm != null) {
    if (qm.isOpen()) {
     try {
      qm.close();
     }
     catch (Exception e) {
      logger.error("Exception caught during qm.close()", e);
     }
    }
    if (qm.isConnected()) {
     try {
      qm.disconnect();
     }
     catch (Exception e) {
      logger.error("Exception caught during qm.disconnect()", e);
     }
    }
   }
        }
    }

 public static void main (String [] args) {
  String qmgrName = "QMGR";
  String host = "localhost";
  int port = 1450;
  String channel = "SYSTEM.DEF.SVRCONN";
  String queueName = "TEST_QUEUE";
  
  MessageAgeMonitor monitor = new MessageAgeMonitor(qmgrName, host, port, queueName, channel, 60);
  new Thread(monitor).start();
 }
}

Followers

About Me

An IT professional with more than 20 years of experience in enterprise computing. An Audio enthusiast designed and built DIY audio gears and speakers.