Roll your own RFH2 header for WebSphere MQ

Do you have an application that is sending messages to an IBM WebSphere Message Queue? And the receiving application is JMS based (Java Message Service) while the sending application is a traditional non-JMS (ie native) WebSphere MQ application.

In this case you may run into the same problem that I encountered. Specifically that the JMS application requires the presence of an MQRFH2 header. This header generally just contains some superfluous information, but apparently it is a requirement for JMS applications. It may be possible to configure the JMS application to ignore this header, however I don’t have any information on that.

I spent several days researching online and testing various methods and generally trying everything out until I had something that now works. Below I will give you the most important facts.

 

Structure of the RFH Header

This article is very useful at explaining at how the MQ message is structured in the case of JMS applications:

http://middleware.its.state.nc.us/middleware/Documentation/en_US/htm/csqzaw09/csqzaw0937.htm

It also explains the format of the MQRFH2 header.  The format consists of a fixed part (of 36 bytes length) which has a set of bytes to indicate the total length, the encoding used, character sets etc. See the article above for the full breakdown of this fixed part.

Example of Fixed Part of the Header

StrucId (MQCHAR4)     Structure identifier.  Must be MQRFH_STRUC_ID (value: "RFH ") (initial value).
Version (MQLONG)     Structure version number.     Must be MQRFH_VERSION_2 (value: 2) (initial value).
StrucLength (MQLONG)     Total length of MQRFH2, including the NameValueData fields.
Encoding (MQLONG)    Data encoding.
CodedCharSetId (MQLONG)    Coded character set identifier.
Format (MQCHAR8)    Format name.
Flags (MQLONG)    Flags.     MQRFH_NO_FLAGS =0. No flags set.
NameValueCCSID (MQLONG)

The second part of the header is the Variable part which is generally made up from a few XML folders which you can fill with some user data (usr section) or jms specific data (jms section)

Example of Variable Part of the Header

<mcd><Msd>jms_text</Msd></mcd><jms><Dst>queue:///APPS/BOB/TEST</Dst><Tms>1297042644307</Tms><Dlv>2</Dlv></jms><usr><msg_id>445566</msg_id><receiver_id>TestReceiver</receiver_id></usr>

 

Sample Code Creating JMS Headers

Here are some examples of code that I had a look at in my investigations on how to create this header. The first is an example showing some Java code which creates a message (this code uses the JMS library so the header is auto generated). The second example is some straight C code which uses the native interface to inject the header manually.

 

Java  Sample Code

The guys we were working with, who had developed the JMS application which was picking up our messages and which required the JMS header, sent us some sample code to show how they were creating messages with headers. Note this is Java code which uses JMS libraries. There is no actual code to create the header, this is done automatically for you just by using the JMS APIs.

 

C Sample Code

There used to be a lot of samples for MQ on the IBM site, however  because they are so old they have been taken off and now are no longer available. However someone has kindly downloaded everything and placed it on their site:

http://www.capitalware.biz/mq_code_c.html

The most interesting one from our point of view is:  Jsmqput.c  This one is a C source code file (very tricky to read if you aren’t used to C or C++...) but it does show the general mechanism behind putting together a message with an RFH header. One thing I noticed on this one is that they pad each folder within the variable part of the message to a multiple of four. In the end my version only pads the entire variable part, not each section as well. The other interesting thing that I picked up from that sample is that each section within the variable part of the header is preceded with a length, this wasn’t obvious from other documentation that I had read.

An example of the code for generating the header:

 These samples where useful from a reference point of view, however I still had to come up with a way to do the same thing in .NET which is the environment that I am using.

Possible Solutions

Option A – The Sensible Way – Use the IBM XMS Interface

This option is probably the way to go, however you do need to upgrade to the version 7 MQ Client libraries (if you haven’t already) and then install the IBM Message Service Client library (XMS). This then adds some new DLLs which you can reference from your code (specifically IBM.XMS.dll) and this provides a wrapper for the JMS API which can be used in a .NET environment.

Here is where you can download this library from:

http://www-01.ibm.com/support/docview.wss?rs=171&uid=swg24011756&loc=en_US&cs=utf-8&lang=en

There are also sample programs which come with this to illustrate its use, for example here is some code:

You can see that there is no need to do anything specific to create the header, simple just set the variables you want passed in the header itself.

Unfortunately I was restricted from upgrading our system to version 7, we were still running version 6 of the client libraries, therefore I had to move on to the next best option...

Option B – The Mad Way – Roll your own MQRFH2 Header

If you are using version 6 of the MQ Client libraries, then you are limited to this option, which is to handcraft the header yourself and insert it into the message as you send the message. This is the approach that I have used, and it was difficult to get it working, but it means no need to upgrade anything. You could use this option for any version of the client libraries.

First of all I assume that you have some existing code to send messages into the queue, for example something like this:

If you look towards the end of the above screenshot you can see where we actually write the message body to the MQMessage before it is sent to the Queue.

I added another line before this WriteString where I send the MQMessage to a new RfhHeaderProvider class (injected into the above class using our dependency injection framework being StructureMap).

Then the code looks more like this:

The header provider class is where the crux of the logic sits. We load the RFH header details from a config file, this includes a switch to turn it on/off and the xml for the three folders that we include in the variable part of the message. Have a look at the code below:

   public class RfhHeaderProvider : IHeaderProvider
    {
        #region Fields

        private readonly IRfhConfigurationProvider _rfhHeaderDetailsConfiguration;

        #endregion Fields

        #region Constructors
     
        /// <summary>
        /// Initializes a new instance of the <see cref="RfhHeaderProvider"/> class.
        /// </summary>
        /// <param name="rfhHeaderDetailsConfiguration">The RFH header details configuration.</param>
        public RfhHeaderProvider(IRfhConfigurationProvider rfhHeaderDetailsConfiguration)
        {
            _rfhHeaderDetailsConfiguration = rfhHeaderDetailsConfiguration;
        }

        #endregion Constructors
   
        /// <summary>
        /// Injects an optional RFH header, the settings for this are loaded via the config file.
        /// This header is usually only used by JMS applications, ie Java Websphere MQ applications using the JMS apis.
        /// since the retrieving part of the system is written by HP using JMS they would like this header so their end
        /// can process the messge without modifying their code
        /// INFO ON RFH Header
        /// http://middleware.its.state.nc.us/middleware/Documentation/en_US/htm/csqzaw09/csqzaw0937.htm
        /// There is an alternative to handcrafting this header, and that is by using the following library:
        /// http://www-01.ibm.com/support/docview.wss?rs=171&uid=swg24011756&loc=en_US&cs=utf-8&lang=en.
        /// However this requires us to install this package and the v7 MQ client libraries.
        /// The structure of the header is also defined here:
        /// http://publib.boulder.ibm.com/infocenter/wmbhelp/v6r1m0/index.jsp?topic=/com.ibm.etools.mft.doc/aq06930_.htm
        /// If you want to modify this structure and examine whether the message is still valid then I recommend RfhUtils.exe
        /// This is a tool that can be run on the server to browse messages on the MQ to view their data and RFH header properties:
        /// http://sites.google.com/site/ibmmiddleware/rfhutils
        /// </summary>
        /// <param name="mqMsg">The message.</param>
        public void WriteHeader(MQMessage mqMsg)
        {
            // THE HEADER IS NOT ENABLED, THEREFORE NOTHING TO DO
            if (_rfhHeaderDetailsConfiguration.HeaderDetails.IsEnabled == false) return;

            var encoding = new System.Text.UTF8Encoding();

            // OLD HARD CODED STRINGS
            //var mcdHeader = @"<mcd><Msd>jms_text</Msd></mcd>";
            //var jmsHeader = @"<jms><Dst>queue:///APPS/OOT/REQUEST</Dst><Tms>1297042644307</Tms><Dlv>2</Dlv></jms>";
            //var usrHeader = @"<usr><msg_id>1122</msg_id><receiver_id>TestReceiver</receiver_id></usr>";

            // NEW DYNAMIC CONFIG LOADED STRINGS
            var mcdHeader = _rfhHeaderDetailsConfiguration.HeaderDetails.McdHeader;
            var jmsHeader = _rfhHeaderDetailsConfiguration.HeaderDetails.JmsHeader;
            var usrHeader = _rfhHeaderDetailsConfiguration.HeaderDetails.UsrHeader;

            var rfhHeader = mcdHeader + jmsHeader + usrHeader;

            while (encoding.GetByteCount(rfhHeader) % 4 != 0)
            {   rfhHeader = rfhHeader + " ";                        // add space here for tracking total length (don't actually do anything with this one)
                usrHeader = usrHeader + " ";                        // last item in name value section gets the extra spaces
            }
            var rfhHeaderLen = encoding.GetByteCount(rfhHeader);

            Int32 iStrucLength = MQC.MQRFH_STRUC_LENGTH_FIXED_2 + rfhHeaderLen + 12; // 3 header lengths * 4 bytes

            // RFH HEADER 
            mqMsg.Format = MQC.MQFMT_RF_HEADER_2;   // Msg Format 
            mqMsg.WriteBytes(MQC.MQRFH_STRUC_ID);   // Structure identifier   
            mqMsg.WriteInt4(MQC.MQRFH_VERSION_2);   // Structure version number
            mqMsg.WriteInt4(iStrucLength);          // Total length of MQRFH2 including NameValueData
            mqMsg.WriteInt4(MQC.MQENC_NATIVE);      // Numeric encoding of data that follows NameValueData   
            mqMsg.WriteInt4(MQC.MQCCSI_DEFAULT);    // Character set identifier of data that follows NameValueData
            mqMsg.WriteBytes(MQC.MQFMT_STRING);     // Format name of data that follows NameValueData
            mqMsg.WriteInt4(MQC.MQRFH_NO_FLAGS);    // Flags
            mqMsg.WriteInt4(1208);                  // Character set identifier of NameValueData, 1208 = UTF-8

            // Optional NameValueData folders and their content must occur in the sequence (length, data)

            mqMsg.WriteInt4(encoding.GetByteCount(mcdHeader));
            mqMsg.WriteBytes(mcdHeader);

            mqMsg.WriteInt4(encoding.GetByteCount(jmsHeader));
            mqMsg.WriteBytes(jmsHeader);

            mqMsg.WriteInt4(encoding.GetByteCount(usrHeader));
            mqMsg.WriteBytes(usrHeader);         
        }
    }

 

The first part of the WriteHeader method pads out the variable part of the header to a multiple of four. Note that I can make this code a bit more efficient (ie no need to append spaces to the rfhHeader variable since I am not using it anyway, could keep a count there instead), however I have left it as is since I now want to write something up about this code before I start optimising it further. Once the variable part has been padded, we need to work out the structure length. This is quite critical to get the length right otherwise you will get an error message when you try to retrieve the message from the queue.

The structure length is made up from the length of the fixed part of the header, plus the length of the padded variable part of the header, plus 4 (bytes) for each header length we insert before each folder.

The second part of the method writes the fixed part of the header, this includes things like the encoding, character sets used, formats etc.  I haven’t changed it too much from the default, and you should probably find that using the same settings as mine will work fine for you as well.

The last part of the method writes the variable part of the header. Each xml folder (I am using three, ie <mcd>, <jms> and <usr>) needs to be pre-pended with its length.

Once you have implemented the code as per above, you will also be able to write your own RFH header into your MQ message for use by JMS applications!

 

Tools for Analysing and Debugging the RFH Header

There is a useful (although quite hard to understand) tool out there for pulling messages of a WebSphere MQ. This is the RfhUtils.exe application which is available from here:

http://www-1.ibm.com/support/docview.wss?rs=849&context=SSKM8N&dc=D400&uid=swg24000637&loc=en_US&cs=UTF-8%E2%8C%A9=en&rss=ct849websphere

There is a useful article on how to use it at this location:

http://sites.google.com/site/ibmmiddleware/rfhutils

The best way to start this thing is to write a few lines into a cmd file as per the instructions. (ie where you set the required paths and pass in the IP address etc).

Once you have started the application it will look something like this:

The options that I used most frequently were “Browse Q” this will show you any messages in the queue, and specifically if there are any errors or not. If there aren’t any errors the message will be loaded and you will be able to see the data breakdown (ie data size) of the body versus the RFH header. You can then also switch to the RFH tab to see the RFH header details, and also to the jms and usr tabs to see the content of these xml folders.

There is an option called “Purge Q” which is useful if you want to clear the messages before you put another one, or you could hit “Start Browse” which steps you through all the messages if you have more than one and want to step through them all.

Hope that helps and let me know if I have missed anything?