Recently I had to integrate with Oracle db using the BizTalk 2006 Oracle adapter (Note: just our vanilla Oracle adapter, not the WCF one). I experience huge challenge setting it up on a Windows 2003 64 bit machine. It's mainly around getting the correct Oracle drivers, using the 64 bit version of ODBC data source interface, giving permission to certain folders, etc.

We had to do this on a production server on limited window, so its crucial to get the steps spot on. Here are the steps that?s required.

1. Download Oracle Data Access Component
2. Install Oracle Data Access Component
3. Create/Configure tnsnames.ora file will connection details
4. Create a System DSN and test the connection
5. Install Oracle Adapter
6. Configure Oracle Adapter
7. Give permission to Oracle Installation folder
8. Restart Host Instances
9. Configure a Receive Port/Receive Location

1 Download Oracle Data Access Component

The first step in configuring the BizTalk 2006 Oracle adapter is to make sure the underlying Oracle ODAC components are correctly configured. You can download the required files from the following location

http://www.oracle.com/technetwork/database/windows/downloads/utilsoft-087491.html

The page will display various links, with various flavours. The one we are interested in (the one we tested successfully) is

Download the Oracle Universal Installer version - ODAC112012.zip

The key thing to look out for in the list is the "Oracle ODBC Driver 11.2.0.1.0"

The other important bit here is not to confuse too much with the 64 bit and 32 bit versions. You only need 32 bit version of the ODAC installer.

2 Install Oracle Data Access Component

Once you have downloaded the ODAC112012.zip (around 270mb), extract the content and identify setup.exe file. Double click the file to start the installation process.

On the first screen select the option "Oracle Data Access Components for Oracle Client 11.2.0.1.2"

In the next screen you need to specify the path, by default it puts the current user name at the end. Just replace the username with "Oracle"

In the component selection screen, select only the following components

1. Oracle Data Provider for .NET 11.2.0.1.2, and
2. Oracle Instant Client 11.2.0.1.0

Note: While unselecting you need to do it from bottom up, else you will get error about dependencies.

3 Create/Configure tnsnames.ora file will connection details

Once the Oracle ODAC components are successfully installed, you need to configure the tnsnames.ora file (found under your install folder d:\app\Oracle\Network\admin) with the environment specific connection string. Example:

ORACLEWEB =

(DESCRIPTION =

(ADDRESS = (PROTOCOL = TCP)(HOST = yourservername)(PORT = 1523))

(CONNECT_DATA =

(SERVER = DEDICATED)

(SERVICE_NAME = FILEMFU1)

)

)

Oracle DBA's should be able to provide these details.

4 Create a System DSN and test the connection

Next we need to create a system DSN to connect to the oracle database. On a 64 bit system it?s important to use the 64bit or Wow64 enabled ODBC datasources interface instead of using the standard one present under Administrative Tools\DataSources ODBC.

In order to do that, open the odbccad32 application found under the following location C:\WINDOWS\SysWOW64\odbcad32.exe

Navigate to System DSN tab and click "Add" button. Then Select "Oracle in OraClient11g_home1" and click finish

In the next screen, Set

Data Source Name = ORACLEWEB

TNS Service Name = ORACLEWEB (This should correspond to the name provided in the tnsnames.ora file)

UserID = <<environment specifi>>

Then click the "Test Connection" button, provide password and make sure the connection is successful.

5 Install Oracle Adapter

Once establishing the successful oracle connection from the BizTalk machine, the next step is to install the BizTalk 2006 Oracle adapter. The installer files are present in a separate CD called "Microsoft BizTalk Server Adapters for Enterprise", if you can?t find the CD you can download it from MSDN using the subscription account.

Once downloaded, open the setup.exe file and select "Install Microsoft BizTalk adapters for Enterprise Application" link.

In the next screen, only select the "Oralce? Database" (not the Oracle? E-Business Suite) and make all other options disabled.

Follow the rest of wizard with standard steps and finish the installation.

6 Configure Oracle Adapter

Once the oracle adapter setup is completed, open up BizTalk Administration console and navigate to Platform Settings\Adapters, right click and select "New"

In the new window, enter "OralceDB" for name and select "Oracle? Database" from the drop down box as shown below:

Click OK.

7 Give permission to Oracle Installation folder

Navigate to the oracle ODAC installation folder ex: "D:\app\Oracle" in the windows explorer. Right-click and select Security. Add the service account user name under which the host instance is running and provide full control

8 Restart Host Instances

Make sure you restart the required host instances. Note: There is no requirement to create a 32 bit host/host instances.

9 Configure a Receive Port/Receive Location

You can test the configuration by creating a new Receive Port, Receive Location.

Open "BizTalk Application 1" and

1. Navigate to Receive Port, right click and select "New - One-Way Receive Port", Name it "Oracle Test".
2. Select "Receive Location" tab and click "New", Name it "Oracle Test"
3. Select "Oracle DB" from the Type combo box and click "Configure". You need to configure 5 main values
   1. Path: Point to "D:\app\Oracle\product\11.2.0\client_1\bin"
   2. Service Name: System DSN name configured earlier
   3. User Name:
   4. Password:
   5. Poll SQL Statement

As shown below:

Click ?Apply? and then ?OK?. Click the ?Configure? button again and this time click on the ?Manage Events? option, which will pop up a new window. On LHS navigate to ?Native SQL?.

Note: If you are able to see the LHS expand as shown below, then it a sign showing everything configured correctly.

Click "OK" all the way through, and enable the "Receive Location". After a while you should see some messages coming into BizTalk server and gets suspended (since there are no subscribers). But this will just prove the oracle configuration is successful.

Reposting after server crash!!

Scenario: In this article we are not going to discuss about how to utilize BAM for your BizTalk applications. I've made the assumption; you created few BAM activities and utilized them inside your BizTalk applications and the system is now running on your productions servers. In my experience, majority of the clients don't take advantage of BAM, when they do they just stick to the basics of creating few BAM activities and utilizing them in their applications. So this article is focused only on this basis and we are going to ignore all the fancy BAM stuff like BAM Views, Alerts, Notifications, real-time data etc.

Production Configuration:

At the very high level your production BAM configuration should like the one shown below

 

Step 1: Make sure you deploy the BAM activities:

This one might look silly :-) but it's possible you might have developed your solution taking advantage of BAM and not deployed BAM activities in your production environment. BAM failures are not treated very seriously by the BizTalk runtime. For example: Your orchestration might have BAM code embedded inside, but you haven't deployed the corresponding BAM activities. In this case the orchestrations will complete successfully without getting suspended but there will be lot of events in the event viewer notifying the BAM failures and also records will start accumulate in the TDDS_FailedTrackingData table in the database. Over a period of time this table will grow substantially, resulting in an unhealthy database and its not nice to see lot of failure events in your event viewer, which will interfere with analysing the real issues.

Step 2: Make sure TDDS sub-service is running.

When you configure a BizTalk host with the option "Allow Host Tracking" to true and created a host instance for that host, then you have created a TDDS sub-service inside that host instance automatically.

The important step is to make sure there is at least one BizTalk Host/Host Instance configured to run "Tracking Data Decode Service (TDDS)" in your environment. When you utilize BAM inside your BizTalk applications (inside your Orchestrations, Pipelines etc) BizTalk runtime will utilize one of the asynchronous event stream to write BAM data for performance reasons. When an asynchronous event stream is used, the BAM data will be first written to the BizTalk message box database and its the job of the TDDS service to move the BAM data from BizTalk message box database to BAMPrimaryImport database. If you don't run the TDDS service the data will start accumulating in the message box database and over a period of time will result in a bloated unhealthy message box database. For more info about Tracking host read this article http://msdn.microsoft.com/en-us/library/ee308950(BTS.10).aspx

Step 3: Setup your active window for the BAM data:

Based on your business needs, it will be critical to keep the historical BAM data for certain period.  The default setting by standard BizTalk installation is to keep the data for 6 months. If you are capturing lot of BAM data, you might need to reduce this value to days, weeks or months. You can change the value by using the bm.exe tool.

You can see the existing value by executing the following command

bm get-activitywindow ?Activity:your_activity_name

You can set the active window to desired value by executing the following command

bm set-activitywindow ? Activity:your_activity_name ?TimeLength:1 ?TimeUnit:Day

Step 4: Make sure BAM_DM_<<activityname>> SSIS packages are configured to run

One of the biggest advantages of BAM is it creates the database infrastructure for you to capture business data. You don't need to manually provision the database tables and views; it's automatically created when you deploy the BAM activities using the management tool bm.exe. For every activity deployed there will be 5 corresponding database tables created in the BAMPrimaryImport database with the postfix _Active, _ActiveRelationships, _Completed, _CompletedRelationships and _Continuations (few SQL views will created to work with these tables and corresponding BAM_DM_<<activity name>> SSIS package will be created).

The main reason for this provisioning is to improve the read/write speed while accessing the BAM data. The number of records in the _Active table will be less; it will contain only in-flight instance data, once the instance complete processing the data will be moved to _Completed tables.

This active/completed partition is good to certain extend, until the number of records in your completed tables started to grow, that's where BAM_DM_<<activity names>> SSIS packages comes into picture. This SSIS packages are mainly responsible for 2 tasks:

1. Taking the database table provisioning to the next level, in order to improve read/write access speed, and
2. Removing the old data from the BAMPrimaryImport database. It can either be moved to BAMArchive database or purged as shown in the picture (beginning of the article).

Whenever you run this SSIS package additional tables are created in BAMPrimaryImport database post fixed by a GUID and old data is removed based on the active window configuration.

The SSIS packages for your BAM activities are created when you deploy your activities, but they are NOT configured automatically in your environment. In your production environment it's critical to create a SQL job to run the SSIS packages in a periodic basis (every day or week based on your BAM data volume).

NOTE: If you have installed your BAM databases on a named SQL instance, then you won't see the BAM_DM_ packages automatically when you connect to Integration services. See the topic "Where are my BAM_DM_ <<BAM Activity Name>> packages?" topic in the end of the article for the solution.

Step 5: Whether to archive it or purge it?

Historical Data:

Historical Data in BAM falls under 2 categories.

1. Based on the active window settings the data will be kept in BAMPrimaryImport database for that period. Example: If your active window is set to 5 days, then 5 days BAM data will be held in various activity tables (partitioned with multiple GUID?s) within BAMPrimaryImport database.
2. Any data that's older than the active window will be held in the BAMArchive database until its cleared manually.

It will be a business decision to keep the historical data for governance purpose.  Until recently, before Microsoft issued the hot fix 971984 (http://support.microsoft.com/kb/971984 ) it was not possible to purge the data directly from BAMPrimaryImport database in BizTalk Server 2006. You always need to move the old historical data to BAMArchive database and then purge it manually (see the script in the bottom of the article for doing it).

With the hot fix (its available by default on BizTalk 2009) you can configure whether or not the data goes into BAMArchive database by using the bm.exe tool.

bm set-archive ?Activity:your_activity_name ?shouldArchive:true|false

Where are my BAM_DM_ <<BAM Activity Name>> packages?

If you have installed BAM databases on a named SQL instance and try to connect to Integration Services and expand the MSDB node, you'll see the following error message.

The SQL server specified in SSIS service configuration is not present or is not available. This might occur when there is no default instance of SQL Server on the computer. For more information, see the topic "Configuring the Integration Services Service" in Server 2005 Books Online.

As noted in the error message the solution is available under Configuring the Integration Services article. You need to open the MsDtsSrvr.ini.xml file located under the folder %ProgramFiles%\Microsoft SQL Server\100\DTS\Binn (90 instead of 100 if you are using SQL 2005) and put the correct server name\instance name inside the <ServerName> element.

You need to restart the SQL Integration Service after making this change.

BAMArchive Purge Script:

There is no clear procedures documented in dealing with the data present in the BAMArchive database. Over a period of time BAMArchive database will also grow to substantial size and it will be required to truncate the data. Here is the script I use to truncate the BAMArchive database, please note this will wipe out all the data present in the BAMArchive database (use it at your own risk), but you will still have the historical data based on your active window configuration in the BAMPrimaryImport database.

truncate table bam_<<your_bam_activityname_1>>_Instances
truncate table bam_<<your_bam_activityname_1>>_Relationships

truncate table bam_<<your_bam_activityname_2>>_Instances
truncate table bam_<<your_bam_activityname_2>>_Relationships

<<repeate the truncate table steps for all of your bam activities>>

Declare @SqlQuery nvarchar(4000)
Declare @num int

Set @num = (Select Count([Name]) From sys.tables
Where [Name] like '%bam_%Instances_200%-%-%' Or
[Name] like '%bam_%Relationships_200%-%-%')

-- Print @Num
Declare @i int
Set @i = 0
While @Num > @i
Begin
    Set @i = @i + 1
    Set @SqlQuery = (Select Top 1 'Drop Table [' + [Name] + ']' From sys.tables
        Where [Name] like '%bam_%Instances_200%-%-%' Or
        [Name] like '%bam_%Relationships_200%-%-%')
    Execute(@SqlQuery)
End

Nandri!

Saravana

Introduction:

In my current client, this is the situation: There are two separate teams involved in two different projects. One utilizing BizTalk Server 2006 building BPM solution and the other utilizing IBM WebSphere Message Broker building an even management system. The day has arrived, where we need to make these two systems talk to each other via MQ. Looking at very high level picture wearing an architecture hat this is what you would have drawn on a white board:

Easypeacy, BizTalk got an MQ adapter, so it can put messages into MQ and WMB will pickup the messages from there and both the system can talk to each other seamlessly. But it just turns out; it's not as simple as that. In order for WMB to pickup the messages that come into MQ, the messages must be PUBLISHED rather than point-to-point PUT and while publishing certain TOPICS should be added. WMB in turn will do all its routing to the subscribers based on the topics that's present in the message.

It's always tricky if you don't know the semantics of glossary terms used by different teams. After some research and with the help of the Utility tool RFHUTIL.exe provided by IBM to write messages into MQ, it turns out the difference between PUBLISH and PUT is just the addition of some extra MQ specific headers. The format of the header is called MQRFH2; one of the segments in the header will contain the PSC commands required for publishing. Relevant TOPICS as name value pair can be added as part of the PSC command.

All these things will make more sense if we see some pictures. I utilized RFHUTIL to place couple of messages into MQ. When you browse the messages in MQ using MQ explorer, it will look like this:

You can spot the difference between the messages, the PUBLSIHED message format is MQRFH2 and the message data now contains some additional RFH information in the beginning.

When you double click and open the message, the first one (regular PUT) will look like this

And the second one (PUBLISH) will look like the one displayed below with additional MQRFH2 properties containing PSC information.

Also, if you look at the data you can see the additional information at the beginning of the message as shown below:

At this stage at least you know, what you are working on, what's your target goal is and now you need to concentrate on how you are going to manage it from BizTalk Server 2006.

Understanding BizTalk Adapters for MQ:

When it comes to BizTalk and MQ the clean way to integrate both of them is by using an adapter. BizTalk server 2006 comes with 2 different set of adapters to talk to MQ. The default MQSeries adapter which gets installed as part of the normal BizTalk installation and the second one MQSC adapter that comes as part of the "BizTalk Adapters for Host System" package. The major difference between the two is the first one can only be used on MQ hosted on windows environment, whereas the second one is used to talk to MQ hosted on windows/non-windows environment. The MQSC adapter is also called as client based adapter because its dependant on IBM MQ's client API's. There are two different flavours of MQSC client adapters, the one which supports transaction ? paid license required and the other without transaction ?free no license required (guarantees at-least once delivery, but cannot guarantee duplicate message delivery). Switching between the two types of client adapters is easy via BizTalk Send Port Configuration.

You can learn more about them here: http://blogs.msdn.com/biztalkcpr/archive/2008/10/07/snap-shot-of-biztalk-2006-solutions-for-mqseries.aspx, and http://msdn.microsoft.com/en-us/library/aa772116(BTS.10).aspx

A bit of research was required to settle for the right adapter, in our case it?s definitely a client based MQSC adapter since MQ was hosted on a non-windows environment.

Bonus Findings!!:

1. You can just install the WebSphere MQ client API's by doing running the WebSphere MQ 7.0 custom setup and selecting Windows client tools as shown below:

2. Once you have settled for the MQSC client adapter, you need to figure out the supported MQ platforms. According to the official document for MQSC adapter (http://download.microsoft.com/download/c/2/2/c22737c1-707e-42a3-ae45-5df40973a0a7/BizTalk%202006%20R2%20Datasheet.pdf) that comes as part of BizTalk Server 2006 (Host integration server 2006) the supported platforms are 5.3 w/ Fix Pack 10 or higher, and 6.0 w/ Fix Pack 1 or higher. But during the prototyping, I figured out the adapter happily talks to MQ 7.0 as long as the MQ 7.0 client is installed. BizTalk 2009 with Host Integration Server 2009 officially supports MQ 7.0.

3. If in case you are working on the MQSeries server based adapter, you will experience lot of issues with strange error codes. Majority of them will be related to transactions and MSDTC. The easiest way to fix it during development and focus on the business problem is to disable transaction on the send and receive port. The other proper option is to configure MSDTC ?Security Configuration correctly as described here http://msdn.microsoft.com/en-us/library/aa544733.aspx

Setting up a development environment:

One of the biggest challenges in an integration project (like this!!) involving two different technology stacks is setting up the development environment. You will lose majority of the time sorting out the environment, rather than working on the real problem. In my case it was not any different and I settled for building a virtual machine with everything required. I installed the software in the following order.

1. Clean Windows XP SP3 Virtual machine (OS choice is yours)

2. Installed SQL Server 2005, and SP1 on top of it

3. Installed Visual Studio 2005 (including VC++ components), and SP1 on top of it

4. Installed BizTalk Server 2006 and configured it

5. Installed MQ 7.0 90 days trial version

6. Installed ?BizTalk Adapter for Host Systems?

7. Downloaded http://www-01.ibm.com/support/docview.wss?uid=swg24000637 IH03.zip file containing RFHUTIL.

Make sure BizTalk is able to communicate to MQ:

Once the environment is setup as explained in previous step, the next thing to do is to make sure BizTalk can communicate to MQ without any trouble. In order to test it, create a BizTalk application with a FILE receive port and MQSC send port and hook them together with appropriate filter condition (hint: BTS.ReceivePortName= ReceivePort1).

To setup MQSC send port, you need to configure 4 basic properties as shown in the below figure

Channel Name: This is the channel name of type server connection and its case sensitive, for testing purposes the easiest way to create a channel is just by enabling "Remote Administration" on the queue manager. Right-Click on the queue manager and select "Remote Administration", on the pop-up window click the button "Create" next to "SYSTEM.ADMIN.SVRCONN channel" and click ok.

NOTE: I don't think this is the safest or right way to create a server connection channel, but on a development environment it certainly works. You should contact your MQ administrator to setup a proper server connection channel.

Connection Name: It's in the format "server name(port number)". The default port when you create queue manager in MQ explorer is 1414.

You need to specify the queue manager name and queue name.

JMS MQRFH2 BizTalk Pipeline Component:

At this stage we have a working BizTalk application that's able to send messages to MQ via the MQSC adapter. Now raises the million dollar question, how are we going to add the required format and headers to message as discussed in the beginning of the article to PUBLISH the message into MQ. JMS MQRFH2 BizTalk Pipeline component, that comes as part of Microsoft ESB Guidance/Toolkit package comes to our rescue. It's worth understanding the significance of this pipeline component before we proceed further.

As part of Microsoft ESB Guidance 1.0 http://msdn.microsoft.com/en-us/library/bb931189.aspx, which was released after BizTalk Server 2006 R2 (later became Microsoft ESB Toolkit 2.0 http://www.microsoft.com/downloads/details.aspx?FamilyID=BC86CF1E-EF29-4B19-95F7-388F64555090&displaylang=en after the release of BizTalk Server 2009), Microsoft released JMS MQRFH2 BizTalk encoder/decoder pipeline component to help customers utilizing Java Messaging Service (JMS) for their integration needs. At this point it's worth understanding the basic structure of MQ message and how JMS integrates with MQ.

WebSphere MQ messages are composed of three components:

• The WebSphere MQ Message Descriptor (MQMD)
• One or more optional message headers (in our case MQRFH2 header)
• The message body.

The message descriptor contains standard message properties, applicable to all WebSphere MQ messages. The second component, optional message headers (example MQRFH2, CICS etc) are application specific. For example when MQ is integrated with JMS, certain information are mapped into the MQRFH2 headers as shown in the below diagram.

The JMS MQRFH2 encoder and decoder BizTalk pipeline component helps customers to extend their JMS integration with Microsoft BizTalk server as shown in the below figure.

Using the encoder component the correct JMS message with relevant MQRFH2 header information can be constructed and submitted into WebSphere MQ, on the other hand BizTalk can receive JMS messages from WebSphere MQ and decode the JMS specific properties present in the MQRFH2 headers and write it to the context properties of the message.

Installing JMS MQRFH2 component for BizTalk Server 2006:

JMS MQRFH2 component was bundled along with ESB Guidance/Toolkit packages which were released and targeted for Microsoft BizTalk Server 2006 R2 and 2009.

One of the constraints I forgot to mention in the beginning was, our environment is still BizTalk Server 2006 (not a big surprise in big enterprise where product migration doesn't happen that often), which mean we are still on .NET Framework 2.0.

This constraint prevented me from installing the latest Microsoft ESB Toolkit 2.0; as soon as you start the installation it will halt straight away asking you to install .NET Framework 3.5. Also, the other important information, ESB Toolkit 2.0 is not an open source project anymore like its predecessor Microsoft ESB Guidance 1.0.

But still there is some good news; ESB Guidance 1.0 is still available for download with complete source code at http://msdn.microsoft.com/en-us/library/bb931189.aspx and the JMS MQRFH2 is a normal pipeline component without any dependency on either BizTalk 2006 R2 or 2009 specific functionalities. That means, you can take the code from the package, compile it and deploy it in your BizTalk 2006 environment.

Once you have installed the downloaded msi file, navigate to "C:\Program Files\Microsoft ESB Guidance 1.0 - November 2007" and unzip the file ESBSource. We are interested in only 3 BizTalk projects highlighted in the below picture.

ESB.JMS.Schemas.Property:

This project contains all the relevant context properties associated with MQRFH2 header. Open the project in Visual Studio 2005, open the project properties and change the application name to "JMS.ESB" (any name you wish). Compile and deploy the project.

ESB.JMS.PipelineComponents:

This is the core component, and compiling this project requires some attention.  This project is developed as a C++ project due to its dependencies with IBM C header files to construct the MQRFH2 headers. If you have installed VC++ on your system, you shouldn't have any problem compiling it. As you probably know, any BizTalk Pipeline component assembly should be present in the GAC, but in order to put the assembly in GAC it needs to be signed with a strong name key. There are some constraints in signing the assembly generated by this project, which is documented here http://msdn.microsoft.com/en-us/library/ee250074(BTS.10).aspx

Once the project is compiled successfully, the easy option to add a strong name key to the compiled assembly is by issuing the following command

SN ?R "C:\Program Files\Microsoft BizTalk Server 2006\Pipeline Components\ Microsoft.Practices.ESB.JMS.PipelineComponents.dll" "C:\Program Files\Microsoft ESB Guidance 1.0 - November 2007\ESBSource\Keys\ Microsoft.Practices.ESB.snk"

The above command will re-sign the assembly with the new key file. You can then put the assembly into the GAC by using gacutils.

ESB.JMS.Pipelines:

This project contains all the support BizTalk pipelines utilizing the encode and decode pipeline components created by EMS.JMS.PipelineComponents assembly. This project is found under the samples\JMS\Source folder, open the project in Visual Studio 2005, open the project properties and change the application name to "JMS.ESB" (or whatever name you have selected while deploying ESB.JMS.Schemas.Property project). Compile and deploy the project.

The JMS MQRFH2 component logic:

The component first inspects the BizTalk context property MQMD_Format and if its equal to "MQHRF2 " (important:with 2 trailing spaces, and its NOT MQRFH2 but MQHRF2 due to header structure), then it applies the logic and adds the MQRFH2 headers else its just pass the incoming message as it is.

The component builds a default MQRFH2 Header using the IBM Header definition file "cmqc.h". Then the Message Context Properties are checked and for each of the MQRFH2_Properties:"MQRFH2_StrucLength" "MQRFH2_Encoding" "MQRFH2_CodedCharSetId? "MQRFH2_Flags" "MQRFH2_NameValueCCSID" "MQRFH2_Format" If the property is set, then the value is parsed to determine if it is the appropriate type, i.e. Integer or 6 Character String. If the value is a correct type, then the value is written to the MQRFH2 Header.

Once each of the fixed values is set, then the length of the "MQRFH2_NameValueData" variable is determined and written to the "MQRFH2.NameValueLength" value in the MQRFH2 structure. Finally the content of the "MQRFH2_NameValueData" is prepended to the outbound MQRFH2 structure.

This final structure is then pre-pended to the outbound message body.

The component also takes care of encoding rules during this process.

Steps to PUBLISH a message into WebSphere MQ?

As discussed in the beginning of the article, publishing a message into WebSphere MQ is nothing more than adding some additional MQRFH2 headers into the message.  With the help of JMS MQRFH2 pipeline component the task is seamless.  Here are the steps to successfully PUBLISH the message into WebSphere MQ

1. In your BizTalk Application add a reference to the BizTalk application "JMS.ESB" (refer to "Installing JMS MQRFH2 component for BizTalk Server 2006" topic), so that you get access to deployed schemas and pipelines.

2. Create a static one-way send port configured with MQSC adapter similar to the one discussed under the topic "Make sure BizTalk is able to communicate with MQ".

3. On the send port change the pipeline to JMS.PassThroughSend

4. Inside your Orchestration set the following context properties on the outgoing message. Build and deploy the orchestration

MSG_OUT(MQSeries.MQMD_Format) = "MQHRF2  ";

MSG_OUT(Microsoft.Practices.ESB.JMS.Schemas.Property.MQRFH2_NameValueData) = "<psc><Command>Publish</Command><Topic>Test</Topic></psc>";

NameValueData is where you specify the psc command with application specific topic information in a XML format.

NOTE: I assumed the readers are familiar with binding and running the BizTalk Application.

Run the application with a sample message and inspect the corresponding WebSphere queue using MQ explorer.

The queue will resemble to the one shown in below figure with all the relevant MQRFH2 properties.

Issues Encountered:

There were couple of issues which returned the error reason code 2142 and 2085. 2142 was mainly due to MQRC_HEADER_ERROR, By hard coding the CodedCharSetId value to 437 the issue was resolved. 2085 was mainly due to UNKNOWN OBJECT, which was resolved by hard coding the StructLength to 96 (based on the psc command length). Still some minor work required to remove these hard coded values, but it should not be a big show stopper.

Conclusion:

The article summarised how custom MQRFH2 headers can be added to the WebSphere MQ messages utilizing the JMS MQRFH2 pipeline component that comes as part of ESB Guidance 1.0 and ESB Toolkit 2.0. The article also explained how the component can be utilized in a BizTalk server 2006 environment by obtaining the source code. Even though the original component was written to transmit and receive JMS specific header in the MQ message, in this article we have seen how it can be utilized to PUBLISH messages into MQ.

The end result is, it would have been impossible to achieve it without the JMS MQRFH2 component that comes as part of ESB Guidance/Toolkit. This article is bit long, but I hope reading a document for couple of hours is  better than spending 3 days finding a solution :-)

Nandri!

Saravana

BizTalk Server 2009 Hyper-V guide is released on 17th April which can be downloaded on Microsoft Download Center.

I'm very happy to be one of the reviewers for this guide. I'm also grateful to Microsoft for mentioning my name on the Acknowledgment section. I just wanted to highlight some of the points

1. The emphasis on this guide is mainly around BizTalk Server performance, it contains details about lot of performance counters and ways you can performance test your BizTalk application. So, just don't get fooled around by the heading.

2. Also, apart from the virtualization topic majority of the performance testing considerations are still valid for older version of BizTalk server like 2006 and 2006 R2.

I guess its going to be hard job to go through the 135 page document, unless otherwise you are thinking about virtualizing your BizTalk environment. For those of you who are interested in getting a very high level impact of BizTalk  on a virtualized environment I guess this graph gives a good indication (taken from the guide - page 75).

 

 

Nandri!

Saravana

In this article I'll explain how you can call a Web Service which requires multiple arguments using a Custom pipeline and a custom pipeline component in a messaging-only scenario without using any Orchestration.

 

Normally, when there is a requirement to call a web service from BizTalk, people tend to take the easy route of calling it via an Orchestration. When we do a web reference inside the orchestration, Orchestration does quite a lot of work for us. It creates all the required schemas, it creates all the required multipart messages, which will be passed to the web service as argument. It makes our life easier. But I guess like me, some of you out there might need to call the web service without using Orchestration. As shown in the above figure. I've one request-response HTTP receive port, and one Solicit response SOAP send port, through this I'm going to call a web service, which expects multiple argument (including one complex type) and return the result back to the caller (HTTP Response). Here are the steps: The attached sample file contains all the required file, I'm just going to explain the key factors in this article.

1. Web Service Definition:

[WebMethod]
public Person GetPersonInfo(Person person, string firstName, string secondName) {
//Some processing
return person;
}

2. Create a general custom pipeline component to construct the multipart message required for the Web Service call 

At run time SOAP Adapter in the send port will map the Biztalk multipart IBaseMessage to the Web Service argument based on the partName of IBaseMessage and argument names of the Webservice. The key factor is how we are going to construct the multipart message in the format required by the SOAP Adapter to make the WebService call. In my previous article I explained how you can easily generate the required IBaseMessage with help of MIME message and MIME Decoder component. But when it comes to reality we don't want to get into another data format like MIME. So, in this article we are going to create custom pipeline component which will construct the correct IBaseMessage required by the SOAP adapter based on the input message and some pipeline design time properties (for more detail on design time properties see my white paper ) .

The custom pipeline component we are going to use has 2 design time properties FirstName and SecondName, which will be passed as parameters to the web service (See webservice definition from Step 1). We'll pass the first webservice argument "Person" as the incoming message via HTTP receive port. The figure below show the custom design time properties configuration window within Biztalk Admin console. 

The code below is the snippet from the custom pipeline component (two important methods Execute and CreateMessage). The Execute method below without the first line of code will be equivalent to a PassThru pipeline component with default Biztalk IBaseMessage. 

#############################################################

public Microsoft.BizTalk.Message.Interop.IBaseMessage Execute(IPipelineContext pc, IBaseMessage inmsg)
{
IBaseMessage msg = CreateMessage(inmsg.BodyPart.GetOriginalDataStream(), pc.GetMessageFactory(),inmsg.Context);
return msg;
}

#############################################################

IBaseMessage CreateMessage(Stream s, IBaseMessageFactory msgFactory, IBaseMessageContext context)
{
IBaseMessage msg = msgFactory.CreateMessage();
IBaseMessagePart part = msgFactory.CreateMessagePart();
part.Data = s;

msg.AddPart("Person", part, true);
msg.Context = context;

//1st Part
IBaseMessagePart partFirstName = msgFactory.CreateMessagePart();
byte[] firstPart = System.Text.Encoding.UTF8.GetBytes(string.Format("<string>{0}</string>", _firstName));
partFirstName.Data = new MemoryStream(firstPart);
partFirstName.Charset = "utf-8"
partFirstName.ContentType = "text/xml"
msg.AddPart("firstName", partFirstName, false);

//2nd Part
IBaseMessagePart partSecondName = msgFactory.CreateMessagePart();
byte[] secondPart = System.Text.Encoding.UTF8.GetBytes(string.Format("<string>{0}</string>", _secondName));
partSecondName.Data = new MemoryStream(secondPart);
partSecondName.Charset = "utf-8"
partSecondName.ContentType = "text/xml"
msg.AddPart("secondName", partSecondName, false);
return msg;
}

#############################################################

Our user defined function CreateMessage will create the required BizTalk IBaseMessage as shown in the below figure

In the above code snippet, the important things to note are highlighted in RED. The incoming message ("Person") will go as the first part (BodyPart) of the IBaseMessage with the name "Person", and then we added two more addional parts "firstName" and "secondName" to the IBaseMessage with correct partNames inline with the web service arguments. The other important thing to note is how the basic data types gets serialized. In our example we got "<string>{0}</string>" as value for firstName and secondName, because they are of type string. If for example you got int as your argument then you need to create the part in the format <int>5</int>.

NOTE: See the web service signature defined in Step 1 for comparison

3. Create a Custom Receive Pipeline using the custom pipeline component

Create a new Biztalk Receive Pipeline and place the custom pipeline component we created in the "Decode" stage of the pipeline.

4. Configure the ports

As shown in our design diagram at the beginning we need 2 ports to send and receive the message, the attached sample file got a binding file, this section is just for explanation, doesn't explain in detail how to configure the ports. Make sure the URL are correct, both on Receive and Send side after importing the binding. You need to configure IIS as well to receive messages via HTTP, follow the link to configure IIS for HTTP receive  http://msdn2.microsoft.com/en-us/library/aa559072.aspx.

Two-Way HTTP Receive Port:

Solicit-Response SOAP Send Port:

We used the .NET Proxy class on our SOAP port to make the call.

Filter Condition on the Send Port

5. Post a Message.

I used WFetch to post the message to BizTalk. You can see on the result pane the request message is posted and you got the response back from the web service synchronously on a two way connection.

Troubleshooting:

Some of the common exceptions you'll see while calling a webservice via SOAP adapter is shown below (from HAT and eventviewer)

1. "Failed to retrieve the message part for parameter "firstName". "

2. "Failed to serialize the message part "firstName" into the type "String" using namespace "". Please ensure that the message part stream is created properly."

The reason for the first error message is due to wrongly named IBaseMessage partName. Read Section 2 carefully to overcome this error.

The reason for the second error message is mainly due to some problem with serializing the IBaseMessage parts to the correct web service arguments. Best approach to overcome this error will be to build a .net console/windows application, add a web reference to the webservice and try to serialize each argument to the corresponding type. For example for this example you can try the following

FileStream fs = new FileStream(@"C:\Documents and Settings\SaravanaK\Desktop\FailedMessages\_Person.out",FileMode.Open,FileAccess.Read);
XmlSerializer serialise = new XmlSerializer(typeof(LH.WebReference.Person));
LH.WebReference.Person per = (LH.WebReference.Person)serialise.Deserialize(fs);
fs.Close();

fs = new FileStream(@"C:\Documents and Settings\SaravanaK\Desktop\FailedMessages\_secondName.out",FileMode.Open,FileAccess.Read);
serialise = new XmlSerializer(typeof(string));
string s2 = (string)serialise.Deserialize(fs);
fs.Close();

The files "_Person.out" and "_secondName.out" are saved from HAT tool. See the exception detail and fix the issue, it will be some namespace issue or data issue.

DOWNLOAD SAMPLE

Read the readme.txt file inside to configure it. Will take approximately 5-20 minutes based on your BizTalk knowledge level.

Related Post

In one of my previous post I explained how to call a web service that has more than one argument using a MIME message in a Content based routing(messaging only) scenario.

Nandri!

Saravana Kumar