Interesting tech stuff I come across day-to-day. Mainly around Biztalk and .NET 3.0

Calling Web Service from BizTalk 2006 in a Messaging only Scenario (aka Content based Routing)

Wednesday, January 31, 2007

In this article I'll explain how you can call a Web Service which requires multiple arguments using a Custom pipeline and a custom pipeline component in a messaging-only scenario without using any Orchestration.

 

Normally, when there is a requirement to call a web service from BizTalk, people tend to take the easy route of calling it via an Orchestration. When we do a web reference inside the orchestration, Orchestration does quite a lot of work for us. It creates all the required schemas, it creates all the required multipart messages, which will be passed to the web service as argument. It makes our life easier. But I guess like me, some of you out there might need to call the web service without using Orchestration. As shown in the above figure. I've one request-response HTTP receive port, and one Solicit response SOAP send port, through this I'm going to call a web service, which expects multiple argument (including one complex type) and return the result back to the caller (HTTP Response). Here are the steps: The attached sample file contains all the required file, I'm just going to explain the key factors in this article.

1. Web Service Definition:

[WebMethod]
public Person GetPersonInfo(Person person, string firstName, string secondName) {
//Some processing
return person;
}

2. Create a general custom pipeline component to construct the multipart message required for the Web Service call 

At run time SOAP Adapter in the send port will map the Biztalk multipart IBaseMessage to the Web Service argument based on the partName of IBaseMessage and argument names of the Webservice. The key factor is how we are going to construct the multipart message in the format required by the SOAP Adapter to make the WebService call. In my previous article I explained how you can easily generate the required IBaseMessage with help of MIME message and MIME Decoder component. But when it comes to reality we don't want to get into another data format like MIME. So, in this article we are going to create custom pipeline component which will construct the correct IBaseMessage required by the SOAP adapter based on the input message and some pipeline design time properties (for more detail on design time properties see my white paper ) .

The custom pipeline component we are going to use has 2 design time properties FirstName and SecondName, which will be passed as parameters to the web service (See webservice definition from Step 1). We'll pass the first webservice argument "Person" as the incoming message via HTTP receive port. The figure below show the custom design time properties configuration window within Biztalk Admin console. 

The code below is the snippet from the custom pipeline component (two important methods Execute and CreateMessage). The Execute method below without the first line of code will be equivalent to a PassThru pipeline component with default Biztalk IBaseMessage. 

#############################################################

public Microsoft.BizTalk.Message.Interop.IBaseMessage Execute(IPipelineContext pc, IBaseMessage inmsg)
{
IBaseMessage msg = CreateMessage(inmsg.BodyPart.GetOriginalDataStream(), pc.GetMessageFactory(),inmsg.Context);
return msg;
}

#############################################################

IBaseMessage CreateMessage(Stream s, IBaseMessageFactory msgFactory, IBaseMessageContext context)
{
IBaseMessage msg = msgFactory.CreateMessage();
IBaseMessagePart part = msgFactory.CreateMessagePart();
part.Data = s;

msg.AddPart("Person", part, true);
msg.Context = context;

//1st Part
IBaseMessagePart partFirstName = msgFactory.CreateMessagePart();
byte[] firstPart = System.Text.Encoding.UTF8.GetBytes(string.Format("<string>{0}</string>", _firstName));
partFirstName.Data = new MemoryStream(firstPart);
partFirstName.Charset = "utf-8"
partFirstName.ContentType = "text/xml"
msg.AddPart("firstName", partFirstName, false);

//2nd Part
IBaseMessagePart partSecondName = msgFactory.CreateMessagePart();
byte[] secondPart = System.Text.Encoding.UTF8.GetBytes(string.Format("<string>{0}</string>", _secondName));
partSecondName.Data = new MemoryStream(secondPart);
partSecondName.Charset = "utf-8"
partSecondName.ContentType = "text/xml"
msg.AddPart("secondName", partSecondName, false);
return msg;
}

#############################################################

Our user defined function CreateMessage will create the required BizTalk IBaseMessage as shown in the below figure

In the above code snippet, the important things to note are highlighted in RED. The incoming message ("Person") will go as the first part (BodyPart) of the IBaseMessage with the name "Person", and then we added two more addional parts "firstName" and "secondName" to the IBaseMessage with correct partNames inline with the web service arguments. The other important thing to note is how the basic data types gets serialized. In our example we got "<string>{0}</string>" as value for firstName and secondName, because they are of type string. If for example you got int as your argument then you need to create the part in the format <int>5</int>.

NOTE: See the web service signature defined in Step 1 for comparison

3. Create a Custom Receive Pipeline using the custom pipeline component

Create a new Biztalk Receive Pipeline and place the custom pipeline component we created in the "Decode" stage of the pipeline.

4. Configure the ports

As shown in our design diagram at the beginning we need 2 ports to send and receive the message, the attached sample file got a binding file, this section is just for explanation, doesn't explain in detail how to configure the ports. Make sure the URL are correct, both on Receive and Send side after importing the binding. You need to configure IIS as well to receive messages via HTTP, follow the link to configure IIS for HTTP receive  http://msdn2.microsoft.com/en-us/library/aa559072.aspx.

Two-Way HTTP Receive Port:

Solicit-Response SOAP Send Port:

We used the .NET Proxy class on our SOAP port to make the call.

Filter Condition on the Send Port

5. Post a Message.

I used WFetch to post the message to BizTalk. You can see on the result pane the request message is posted and you got the response back from the web service synchronously on a two way connection.

Troubleshooting:

Some of the common exceptions you'll see while calling a webservice via SOAP adapter is shown below (from HAT and eventviewer)

1. "Failed to retrieve the message part for parameter "firstName". "

2. "Failed to serialize the message part "firstName" into the type "String" using namespace "". Please ensure that the message part stream is created properly."

The reason for the first error message is due to wrongly named IBaseMessage partName. Read Section 2 carefully to overcome this error.

The reason for the second error message is mainly due to some problem with serializing the IBaseMessage parts to the correct web service arguments. Best approach to overcome this error will be to build a .net console/windows application, add a web reference to the webservice and try to serialize each argument to the corresponding type. For example for this example you can try the following

FileStream fs = new FileStream(@"C:\Documents and Settings\SaravanaK\Desktop\FailedMessages\_Person.out",FileMode.Open,FileAccess.Read);
XmlSerializer serialise = new XmlSerializer(typeof(LH.WebReference.Person));
LH.WebReference.Person per = (LH.WebReference.Person)serialise.Deserialize(fs);
fs.Close();

fs = new FileStream(@"C:\Documents and Settings\SaravanaK\Desktop\FailedMessages\_secondName.out",FileMode.Open,FileAccess.Read);
serialise = new XmlSerializer(typeof(string));
string s2 = (string)serialise.Deserialize(fs);
fs.Close();

The files "_Person.out" and "_secondName.out" are saved from HAT tool. See the exception detail and fix the issue, it will be some namespace issue or data issue.

DOWNLOAD SAMPLE

Read the readme.txt file inside to configure it. Will take approximately 5-20 minutes based on your BizTalk knowledge level.

Related Post

In one of my previous post I explained how to call a web service that has more than one argument using a MIME message in a Content based routing(messaging only) scenario.

Nandri!

Saravana Kumar

Labels:




Difference between (PassThruReceive, PassThruSend) and (XmlReceive, XmlSend) Biztalk Pipelines (For Beginners)

I've recently seen a query in the "Biztalk New Users" newsgroup someone asking the above question. I quickly did a search to find out if there is (are) a decent article out there explaining the difference and didn't find any. I feel its quite important and basic thing every biztalk developer should understand, ofcourse we all learned this by experience, but for new comers it will be a big confusing point. So, let me put a short article here to explain the difference.

 

The above figure show the architecture for Biztalk 2004/2006 (I'm sure you would have come across this picture million times by now, I took it from MSDN). It shows clearly the only way messages can come in and go out of the message system via some kind of pipeline. So, whether you are doing any processing in the pipeline or not you need some default pipeline for the message to come in and go out of Biztalk.

 

Out of the box biztalk provides 2 Receive pipelines (PassThruReceive, XmlReceive) and 2 Send Pipelines (PassThruSend, XmlSend), ofcourse you can go and build more custom pipelines if required. For a detail understanding of pipelines read my white paper published by Microsoft.

 

Now, what's the difference?

The default PassThruReceive and PassThruSend pipelines doesn't do any processing at all with the message. You can treat them as if like no pipelines are present (virtually) inbetween the adapter and the message box. Wheareas the XmlReceive pipeline will have a XmlDisassembler pipeline component on the disassembling stage and the XmlSend pipeline will have a XmlAssembler on the Assemble stage as shown in the above figure.

 

Why do we need a XmlReceive pipeline:
As discussed earlier the only additional component in the XmlReceive pipeline when compared to PassThruReceive pipeline is the addition of XmlDisassembler pipeline component on the disassembling stage. Whenever an Xml message is received via the XmlReceive pipeline the XmlDisassembler will do the following tasks:

1.Promote the "MessageType" context property by taking the combination of TargetNamespace and Rootelement in the format, Targetnamespace#RootElement.
2. Remove Envelopes and disassemble the interchanges
3. Promote the content properties from interchange and individual document into message context based on the confirured distingushed fields and promoted properties.

See the link for detailed explanation of XmlDisassembler (http://msdn2.microsoft.com/en-us/library/ms964545.aspx).

There are 3 important things you need to keep in mind.

1. Maps on the receive side will be applied after the receive pipeline. In order to pick the correct schemas for processing, Maps need the "MessageType" property, which is promoted by the XmlDisassembler. So, if you are using PassThruReceive "MessageType" property won't be present, which in turn will result in Map not being applied.

2. Binding of logical port (Activation port) in the orchestration to a message (inside the orchestration designer), and deploying/enlisting the orchestration will create the subscription for the particular message type. So, if you are using PassThruReceive "MessageType" property won't be present, which in turn will result in not starting the particular Orchestration instance.

3. If you have configured content based routing based on the properties in the message (Filters in SendPorts, Example: BTS.MessageType = http://tempuri#Person), routing won't happen until the correct properties are present in the message context.

Note: There are numerous ways in which you can make all the above 3 points work without XmlReceive pipeline, but they all requires some custom coding.

Is there any rule on the combination of Pipelines you can use on the Receieve and Send side?

Of course, NOT. There is no restriction on the type of pipeline combination you need to use on the Receive and Send side. For example: You can receive a message via XmlReceive pipeline and Send it thru PassThruSend pipeline, this applies to custom pipelines as well. In fact Biztalks pub/sub architecture doesn't put direct connection (or in better terms won't know) between the receive and send side.

Why do we need a XmlSend pipeline?

XmlSend pipeline does the reverse of what a XmlReceive pipeline did. It got a XmlAssembler component in the Assemble stage in addition to the PassThruSend pipeline, which does the following tasks:

1. XML Assembler Builds envelopes as needed and appends XML messages within the envelope.

2. Populates content properties on the message instance and envelopes.

See the link for detailed explanation of XmlAssembler http://msdn2.microsoft.com/en-us/library/ms964547.aspx

For beginners, you can skip all the points related to Envelopes and Interchanges mentioned above.

Labels:




Defining design-time properties using basic .NET data types

Recently I was building a pipeline component which requires a simple design time property. Even though I've written a white paper about it Understanding Design-Time Properties for Custom Pipeline Components in BizTalk Server I couldn't remeber all the syntax on top of my head. So, I opened the document for a quick view. I just copied a section from the article which explains the basic steps and published it here. View the full article for detailed explanation of some of the advanced concepts.

Let’s start with an example where you need three properties—ConnectionString (string), SaveContext (bool), and LogLevel (enum). To define these properties, you need to perform the following steps within your pipeline component:

 

Step 1: Define the enumeration type required.

public enum LogLevelType

{

 Warning,

 Error,

 Information

}

 

Step 2: Define private field variables.

 

private string _connectionString = string.Empty;

private bool _saveContext = false;

private LogLevelType _logLevel = LogLevelType.Information;

 

Step 3: Define public design-time properties.

public string ConnectionString

{

get{return _connectionString;}

set{_connectionString = value;}

}

public bool SaveContext

{

get{return _saveContext;}

set{_saveContext = value;}

}

public LogLevelType LogLevel

{

get{return _ logLevel;}

set{ _logLevel = value;}

}

 

Step 4: Implement the Load method of the IPersistPropertyBag interface.

public virtual void Load(Microsoft.BizTalk.Component.Interop.IPropertyBag pb, int errlog)

       {

            object val = ReadPropertyBag(pb,"ConnectionString");

            if (val != null) _connectionString = (string)val;

 

            val = ReadPropertyBag(pb, "SaveContext");

            if (val != null) _saveContext = (bool)val;

 

            val = ReadPropertyBag(pb, "LogLevel");

            if (val != null) _logLevel = (LogLevelType)val;

        }

 

The helper function ReadPropertyBag is used to read the design-time properties from the property bag.  Error-handling code inside this function is required when the component is loaded inside the pipeline designer for the first time. At that point there are no values associated with the properties, which results in an exception. This helper function catches and suppresses the exception.

 

Private object ReadPropertyBag(Microsoft.BizTalk.Component.Interop.IPropertyBag pb, string propName)

        {

            object val = null;

            try

            {

                pb.Read(propName, out val, 0);

            }

            catch (System.ArgumentException )

            {

                return val;

            }

            catch (System.Exception e)

            {

                throw new System.ApplicationException(e.Message);

            }

            return val;

        }

Step 5: Implement the Save method of the IPersistPropertyBag interface.

public virtual void Save(Microsoft.BizTalk.Component.Interop.IPropertyBag pb, bool fClearDirty, bool fSaveAllProperties)

       {

            object val = _connectionString;

            pb.Write("ConnectionString", ref val);

 

            val = _saveContext;

            pb.Write("SaveContext", ref val);

 

            val = (LogLevelType)_logLevel;

            pb.Write("LogLevel", ref val);

        }

Step 6: Implement the GetClassID and InitNew methods of the IPersistPropertyBag interface.

 

   Public void GetClassID(out System.Guid classid)

        {

            classid = new System.Guid("AC21E483-C9BF-41F1-9AF0-2031528535C6");

        }

The GetClassID method needs to return a unique identifier that represents the component within unmanaged code, which will allow interoperability with unmanaged code.

 

Note: The BizTalk 2004/2006 messaging engine is built using unmanaged code.

 

public void InitNew(){}

 

For basic data types you can leave the InitNew implementation blank, since we don’t have any structure, data, cache, or object to initialize for our sample.  

 

After you compile the component and insert it into a BizTalk pipeline (see “Using the custom pipeline component within a BizTalk pipeline” later in this article), the properties will be displayed in the property grid as shown in the following figure.

 

 

Labels:




Property Promotion inside Orchestration

Monday, January 08, 2007

Promoting properties inside BizTalk is quite common task, after all back bone of BizTalk's underlying routing infrastructure Publish-Subscribe is based on property promotion. There are tons of articles published in the web explaining this concept.

Properties get promoted within BizTalk at different places, Example: some of the properties gets promoted by Adapters, Pipelines Components, Messaging Engine etc,etc. In custom pipeline components ,developers can use either IBaseMessageContext.Write to write context properties or IBaseMessageContext.Promote to promote context properties. The basic difference between writing and promoting (aka distinguished and promoted) properties are, properties that are written cannot be used for message routing, whereas properties that are promoted can be used for message routing. (There are quite few other important differences, but they are not relevant for this article).

Orchestration:

The syntax for promoting properties within Orchestration is

Message(PromotedPropertyName) = value;

Example

MSG_OUT_PERSON(HTTP.ContentType) = "text/xml";

this is the only direct option available within an orchestration to promote a property. Properties promoted by this method are written rather than promoted. That means it can't be used for message routing (filter expressions in Send Port, Orchestration etc).

The sample code contains an orchestration named "PromoteProperty.odx" as shown in below figure, it just receives a message, does property promotion inside the message assignment stage and send the message out.

The following two lines are present inside the "Message Assignment" shape:

MSG_OUT_PERSON = MSG_IN_PERSON;
MSG_OUT_PERSON(PropertyPromotionInsideOrchestration.PropertySchema.ID) = "SK007"

If you look at the outgoing message context, it will look like as shown in figure below, and you can see clearly the property is "NOT PROMOTED"

So, how can you promote property within Orchestration for routing:

Orchestration got the concept of Correlation, basically which routes the response messages to the correct running orchestration instance that initiated the request message. Within Orchestration we use Correlation Set and Correlation type to archive this type of instance routing.

Correlation type is nothing but a set of Properties, Example in our case it's "ID" (you can have multiple properties together to form a single correlation type, Example EmployeeId, EmployeeCompany).

Correlation Set is based on Correlation type and its a set of properties with specific values. When we "Initialize a Correlation set" within orchestration, the orchestration instance automatically promotes those properties in Correlation set into the message context.

We can use this mechanism to promote propeties (rather than just writing properties), there is no need to do a follow up of the Correlations sets we initialized, and it NOT going to create unnecessary subscriptions. (In simple terms you create correlation sets, not for correlation but for property promotion.)

The sample code contains an Orchestration called "PromotePropertyUsingDummyCorrelationSet.odx" which is exact replica of our first Orchestration "PromoteProperty.odx", with one addition. I added a Correlation Type, Correlation Set and Initialized the Correlation set in the Send shape as shown in below figures.

      

Now, if you look at the outgoing message context , it will look like as shown in figure below. Now you can see ID property is PROMOTED. The attached sample clearly explains this concept. Read the Download section for details.

David Hurtado highlighted this trick on his blog. I expanded his trick with proper sample and more explanation, so that it will be helpful to some one in the future. Or atleast to me if in case I can't remember.

Download:

The download contains the complete source code, sample messages, File drop folders and binding file. Follow the steps to setup the sample.

1. Extract "PropertyPromotionInsideOrchestration" solution to "C:\BTSSamples" folder

2. Open it in Visual Studio, Build/Deploy it, it will create a Biztalk Application called "PropertyPromotionInsideOrchestration". Rectify the errors if something pops up.

3. Open Biztalk 2006 Administration console, right-click on the application "PropertyPromotionInsideOrchestration" and import the binding file "binding.xml". (Start the application).

Execution:

1. Drop the message "NoPromotion.xml" into "FileDrop\In" folder, you'll see the output only in "FileDrop\Out" folder.

2. Drop the message "PromotionWillHappen.xml" into "FileDrop\In" folder you'll see the output in both "FileDrop\Out" and "FileDrop\Out.ByID" folder. (Because message is routed via the second send port to this folder by Content based routing.)

Labels:




Microsoft.BizTalk.Gac.Fusion, Access denied Exception and GAC is empty

Once in a while you get the following error while trying to compile/build the BizTalk project (With few more Access denied errors and some HRESULT values).

Error 2 at Microsoft.BizTalk.Gac.Fusion.IAssemblyCache.InstallAssembly(AssemblyCacheInstallFlag flags, String manifestFilePath, FusionInstallReference referenceData)
at Microsoft.BizTalk.Gac.Gac.InstallAssembly(String assemblyPathname, Boolean force)
at Microsoft.BizTalk.Deployment.BizTalkAssembly.GacInstall(String assemblyLocation)
at Microsoft.BizTalk.Deployment.BizTalkAssembly.PrivateDeploy(String server, String database, String assemblyPathname, String applicationName)
at Microsoft.BizTalk.Deployment.BizTalkAssembly.Deploy(Boolean redeploy, String server, String database, String assemblyPathname, String group, String applicationName, ApplicationLog log)

The error messages were very generic and its hard to diagnose anything with those messages. As a developer you tend to try few things like 1. Shutting down Visual Studio, 2. Restarting Biztalk, 3. Restarting IIS, 4. Stopping Virus scanners, etc etc and at one stage even restarting Windows. After doing one, two or all of the above steps, you'll be ok at some point (but you never know the root cause). The project will recompile without any issues this time.  Whenever you are experiencing this issue if you look at the GAC, the GAC will be empty (Yes! believe me, scary stuff, but that's true. I put two pictures below to show that).

You'll experience similar errors once in a while, when you have some Visual Studio "Built Events" scripts that deploy assemblies into GAC. One day I decided to spend some time and find out the root cause for the problem. SysInternals (Oops! now Microsoft's) FileMon came to rescue, I put the filter on FileMon to watch for the GAC folder and started to rebuild the assemblies, all of a sudden I started to see the process cisvc.exe popping up the screen, a bit of search revealed it the "Indexing Service" that blocks the GAC folder temporarily. I don't know whether "Indexing Service" is part of Windows XP or it got installed as part of some other updates. If in case you have it installed on your PC, that's the services you need to start and stop if you are experiencing the problem outlined in this article.

Nandri!

Saravana

Labels:




Understanding Design-Time Properties for Custom Pipeline Components in BizTalk Server

Thursday, January 04, 2007

Recently I been involved in building variety of Biztalk pipeline components with different design-time properties ranging from simple "string" to "complex collections". I struggled a bit to figure out the ways to do it, due to poor error description you get while developing and also due to lack of documentation around it. Normally you'll find some blog articles here and there explaining stuff, but in this particular instance I couldn't find any blog articles either. So, I went ahead and written a white paper explaining design-time properties in Custom Pipeline component, the article explains using complex types and their serialization techniques, development tips and error diagnosis procedures.

You can download the White Paper and the accompanying code from the following location: 

http://www.microsoft.com/downloads/details.aspx?FamilyId=8C2572A2-FB61-4016-A7A7-1C37518B39FF&displaylang=en

I need to thank Angus Foreman (from Microsoft,UK) and Luke Nyswonger for their help in publishing the article. I also would like to thank all the editors.

Nandri!

Saravana

Labels: