Biztalk 2004 Deployment - moaning by a Software engineer
Friday, March 18, 2005
We need to agree with Martin's point on deployment issues we face with Biztalk Server 2004.
Read this post by Martin regarding all the Biztalk Deployment problems.
We mainly use Scott Colestock's NAnt-scripts for deployments, since we quickly concluded that it is absolutely essential to have an install script to perform deploytime configuration. We think it is practically impossible to perform all required steps manually w/o mistakes. It would take a really long time, good nerves and a lot of heavy-duty mousework to deploy manually into the production environment!
Our customers often want BizTalk after having seen the BizTalk Mapper in action, and heard some other appealing integration arguments. They expect/hope their own competent development staff to instantly be productive with this great tool. When we talk them through the finer points of deployment, they get really disappointed that they might need help from external consultants just to handle deployment of some simple change to their BizTalk configuration.
Also they do not like the idea that to deploy the same simple change they might need to completely shut down everything currently running, remove everything, essentially reverting to a blank installation, and then reinstalling everything again. Of course there are ways to partition your solution so that say redeployment of order processing do not interfere with say ongoing customer registration, but that instead complicates the deployment scripts, if for example "OrderProcessingOrchestration" is dependant on the same send port "SendToERP" as is "CustomerRegistrationOrchestration".
Also, the customers are surprised that VS.NET "Deploy" command.
* Does not work remotely.
* Does not handle redeploy for solutions partitioned into separate assemblies like schema-assembly, orchestration-assembly and pipeline-assembly.
* Does not perform bindings.
* Does not help with adding binding information to new (currently undeployed) orchestrations with "specify later"-ports.
I recently did a hands-on ad-hoc demo (=no scripted deploy) including a flat file transform in a pipeline, an orchestration, and a map, all housed inside one single assembly) for a few (competent) developers from a large potential customer, and the number of manual configuration steps, biztalk explorer refreshes, and host instance restarts and deployment related problems I encountered and had to solve made them a bit put down with the entire BizTalk concept we offered. I tried to explain that we would normally setup a deployment script to avoid all this, but had to agree that deployment, even scripted, is much more complicated than most people initially think. They now seem more inclined to handle a lot of "simpler" integration scenarios outside of BizTalk just because it seems too complicated to do really simple things with it.
Below are some of the deployment related problems we face in an ongoing project.
In the production environment:
* How to handle updates to the production environment with messages constantly arriving at the receive locations
* Avoid receiving messages when receive ports are running but orchestrations are stopped/undeployed -> "could not find matching subscription"
* Never, ever, lose messages.
* Minimize downtime.
* A seemingly "simple" redeploy like a new element in a schema often needs to stop and undeploy everything because of interdependencies, assuming one orchestration assembly and one separate schema assembly upon with the orchestration assembly is dependant.
* What happens with currently running/suspended orchestration instances when stopping BizTalk and undeploying orchestrations? Seems they must all be terminated before allowing un/redeploy (unless doing "side-by-side").
* Sometimes it is possible to stop the receive ports, and wait until all running orchestrations have had time to finish, but this is of course impossible for real long-running orchestrations. Needs side-by-side versioning of orchestrations.
* Very delicate ordering requirements of undeploy and deployment of various artifacts.
* Different if side-by-side w/different version numbers, or update with same version.
* Stop relevant ports first
* Make sure no interesting orchestration instances are alive. Else wait!
* Redeploy
* Restart BizTalk to reload assemblies.
* Side-by-side even more complex, but required with long-running orchestrations!
* No samples or external tools (Scott Colestock's NAnt-deployment-scripts) seems to be built for:
* (re)deployment with side-by-side versioning
* correctly handle redeploy with messages constantly arriving at the receive locations
* probably because there are no generic solution, and even getting the specific solution for your specific BizTalk project right is hard.
* We have a few orchestrations that gets executed frequently and runs for a minute or so, so we need to stop the ports, and wait for those to end before redeploying. Unfortunately we have not had time to incorporate this behaviour in our deployment scripts, so we need to remember to do this manually.
* We have not had the time and energy to fully investigate the side-by-side scenario, and change our deployment procedure to that method. Needs a few custom tools and major modifications of the deployment scripts in order to work as well as our current scripts.
* At least with Scott Colestock's NAnt-scripts, redeploy does undeploy based on new bindings (which for example does not handle a rename of a port or orch)
* Need to store deployed config with new name after successful deploy, and undeploy using that.
(Avoid removing/uninstalling that file during installation of update components)
* Probably possible to extract current bindings with BTSDeploy Export, and undeploy using that.
* Deploy HAT Message Property Tracking settings and Orchestration/Pipeline Inbound/Outbound message bodies
* Message property tracking settings are reset on redeploy.
* Had to build custom tool to re-configure message property tracking after each redeploy, but before receiving any messages, since tracking is a requirement.
* However... orchestration and pipeline tracking settings are *not* reset on redeploy.
During development:
* Chicken-or-egg situation when adding new orchestrations with a scripted environment.
* Must have orch in binding file, but cannot export bindings before first deploy.
* Therefore one often has to do some hand-editing of the binding files.
* This has proved to be a major problem for our junior developers. Re-deploying with the NAnt-script works great when they only update the assemblies, but when they need to add or change the configuration of ports/orchs they sometimes get stuck in limbo getting things half deployed, half undeployed, not being able to get out of the mess by themselves. They cannot undeploy because "something" they cannot find among the myriad of existing ports is referencing a schema or pipeline they try to remove, and therefore they cannot deploy and are also unable to revert to an older version!
* Deploy on different machines with different configuration, for example a test server w/o MSMQ/T Adapter
* We have development machines we cannot install MSMQ/T on, because they need dynamic IP and MSMQ.
* Have to replace MSMQ/T with FILE adapter in binding files.
* This particular scenario simplified with the recent release of the MSMQ adapter.
* This is also made simpler by the latest release of Scott Colestock's NAnt-scripts, which we haven't had time to migrate to.
* No way to automatically increment btproj file version and assembly version (current version updaters assume attributes AssemblyInfo.cs file)
* Sometimes the MSI-setup seems to not replace existing DLLs when upgrading (not deploying to biztalk, but rather installing the necessary scripts and assemblies to do so) maybe because the version numbers of the DLLs have not changed, although this might be due to something else. (Not a BizTalk-specific problem in any case)
* No way to automatically increment installer version (and product/package code) - also not a BizTalk-specific problem, but stops us from making a fully automated build and deploy process, without buying or making custom tools.
/Martin
Software Engineer
Softronic Syd AB
www.softronic.se
Sweden
Read this post by Martin regarding all the Biztalk Deployment problems.
We mainly use Scott Colestock's NAnt-scripts for deployments, since we quickly concluded that it is absolutely essential to have an install script to perform deploytime configuration. We think it is practically impossible to perform all required steps manually w/o mistakes. It would take a really long time, good nerves and a lot of heavy-duty mousework to deploy manually into the production environment!
Our customers often want BizTalk after having seen the BizTalk Mapper in action, and heard some other appealing integration arguments. They expect/hope their own competent development staff to instantly be productive with this great tool. When we talk them through the finer points of deployment, they get really disappointed that they might need help from external consultants just to handle deployment of some simple change to their BizTalk configuration.
Also they do not like the idea that to deploy the same simple change they might need to completely shut down everything currently running, remove everything, essentially reverting to a blank installation, and then reinstalling everything again. Of course there are ways to partition your solution so that say redeployment of order processing do not interfere with say ongoing customer registration, but that instead complicates the deployment scripts, if for example "OrderProcessingOrchestration" is dependant on the same send port "SendToERP" as is "CustomerRegistrationOrchestration".
Also, the customers are surprised that VS.NET "Deploy" command.
* Does not work remotely.
* Does not handle redeploy for solutions partitioned into separate assemblies like schema-assembly, orchestration-assembly and pipeline-assembly.
* Does not perform bindings.
* Does not help with adding binding information to new (currently undeployed) orchestrations with "specify later"-ports.
I recently did a hands-on ad-hoc demo (=no scripted deploy) including a flat file transform in a pipeline, an orchestration, and a map, all housed inside one single assembly) for a few (competent) developers from a large potential customer, and the number of manual configuration steps, biztalk explorer refreshes, and host instance restarts and deployment related problems I encountered and had to solve made them a bit put down with the entire BizTalk concept we offered. I tried to explain that we would normally setup a deployment script to avoid all this, but had to agree that deployment, even scripted, is much more complicated than most people initially think. They now seem more inclined to handle a lot of "simpler" integration scenarios outside of BizTalk just because it seems too complicated to do really simple things with it.
Below are some of the deployment related problems we face in an ongoing project.
In the production environment:
* How to handle updates to the production environment with messages constantly arriving at the receive locations
* Avoid receiving messages when receive ports are running but orchestrations are stopped/undeployed -> "could not find matching subscription"
* Never, ever, lose messages.
* Minimize downtime.
* A seemingly "simple" redeploy like a new element in a schema often needs to stop and undeploy everything because of interdependencies, assuming one orchestration assembly and one separate schema assembly upon with the orchestration assembly is dependant.
* What happens with currently running/suspended orchestration instances when stopping BizTalk and undeploying orchestrations? Seems they must all be terminated before allowing un/redeploy (unless doing "side-by-side").
* Sometimes it is possible to stop the receive ports, and wait until all running orchestrations have had time to finish, but this is of course impossible for real long-running orchestrations. Needs side-by-side versioning of orchestrations.
* Very delicate ordering requirements of undeploy and deployment of various artifacts.
* Different if side-by-side w/different version numbers, or update with same version.
* Stop relevant ports first
* Make sure no interesting orchestration instances are alive. Else wait!
* Redeploy
* Restart BizTalk to reload assemblies.
* Side-by-side even more complex, but required with long-running orchestrations!
* No samples or external tools (Scott Colestock's NAnt-deployment-scripts) seems to be built for:
* (re)deployment with side-by-side versioning
* correctly handle redeploy with messages constantly arriving at the receive locations
* probably because there are no generic solution, and even getting the specific solution for your specific BizTalk project right is hard.
* We have a few orchestrations that gets executed frequently and runs for a minute or so, so we need to stop the ports, and wait for those to end before redeploying. Unfortunately we have not had time to incorporate this behaviour in our deployment scripts, so we need to remember to do this manually.
* We have not had the time and energy to fully investigate the side-by-side scenario, and change our deployment procedure to that method. Needs a few custom tools and major modifications of the deployment scripts in order to work as well as our current scripts.
* At least with Scott Colestock's NAnt-scripts, redeploy does undeploy based on new bindings (which for example does not handle a rename of a port or orch)
* Need to store deployed config with new name after successful deploy, and undeploy using that.
(Avoid removing/uninstalling that file during installation of update components)
* Probably possible to extract current bindings with BTSDeploy Export, and undeploy using that.
* Deploy HAT Message Property Tracking settings and Orchestration/Pipeline Inbound/Outbound message bodies
* Message property tracking settings are reset on redeploy.
* Had to build custom tool to re-configure message property tracking after each redeploy, but before receiving any messages, since tracking is a requirement.
* However... orchestration and pipeline tracking settings are *not* reset on redeploy.
During development:
* Chicken-or-egg situation when adding new orchestrations with a scripted environment.
* Must have orch in binding file, but cannot export bindings before first deploy.
* Therefore one often has to do some hand-editing of the binding files.
* This has proved to be a major problem for our junior developers. Re-deploying with the NAnt-script works great when they only update the assemblies, but when they need to add or change the configuration of ports/orchs they sometimes get stuck in limbo getting things half deployed, half undeployed, not being able to get out of the mess by themselves. They cannot undeploy because "something" they cannot find among the myriad of existing ports is referencing a schema or pipeline they try to remove, and therefore they cannot deploy and are also unable to revert to an older version!
* Deploy on different machines with different configuration, for example a test server w/o MSMQ/T Adapter
* We have development machines we cannot install MSMQ/T on, because they need dynamic IP and MSMQ.
* Have to replace MSMQ/T with FILE adapter in binding files.
* This particular scenario simplified with the recent release of the MSMQ adapter.
* This is also made simpler by the latest release of Scott Colestock's NAnt-scripts, which we haven't had time to migrate to.
* No way to automatically increment btproj file version and assembly version (current version updaters assume attributes AssemblyInfo.cs file)
* Sometimes the MSI-setup seems to not replace existing DLLs when upgrading (not deploying to biztalk, but rather installing the necessary scripts and assemblies to do so) maybe because the version numbers of the DLLs have not changed, although this might be due to something else. (Not a BizTalk-specific problem in any case)
* No way to automatically increment installer version (and product/package code) - also not a BizTalk-specific problem, but stops us from making a fully automated build and deploy process, without buying or making custom tools.
/Martin
Software Engineer
Softronic Syd AB
www.softronic.se
Sweden
Labels: BIZTALK
Home
About Me
Email
