30 USEFUL

BIZTALK
SERVER
TIPS

Ricardo Torre

Head of Technical Pre-Sales

& Product Strategist

contents
2

Efficient
Host
Separation

Choose
robust
adapters

Monitor
performance
SLAs with
BAM

12

BAM
lightweight
tracking

Minimize
persistence
points

Performance
Testing is key

10

13

Review
the
performance
of your
environment
regularly

Take advantage
of Multi MessageBox
Configuration

Package
all your
resources
into MSI

Build your
custom pipelines
with components using
default properties

Cluster
host
instances
with
adapter
like
FTP or
POP3

14

15

Health
Check
with
Message
Box
Viewer

11

Create
security
boundaries

Use the
Business Rule
Engine to implement
Business Logic

Split big
messages in the
receive pipeline

16

Use
Visual Studio for
functional and load testing

18

Cluster
the Master
Secret Server

21

Plan
your
Backup and
DR strategy

19

Purge
BAM Data

22

Use
Webgarden
for IIS
scalability

24

27

Use fast
disks for your
BizTalk Database subsystem

20
23

25

Maintain you
Message Box efficiently

Implement a
Enterprise
Service
Bus using the
ESB Toolkit

17

28

30

Use SSO
as the application
configuration store

Configure and
activate the DTA
Purge and Archive Job

26

Consider
Test 100%
to prefer
of your code
XSLT in maps

Avoid
Orchestrations
when possible

Use ETW for high

performance tracking

29

Develop
adapter
using WCF

Take advantage of
Multi MessageBox Configuration

Take advantage of configuring multiple message boxes in BizTalk environment

to overcome SQL server bottlenecks. Once the load goes up in your BizTalk
environment, youll start seeing bottleneck in SQL server. Normally it will require deep SQL knowledge, such as configuring multiple files per file group
to overcome this challenge. The other easy option to solve this problem is by
simply adding multiple message boxes. This tip even works nicely on a single
SQL server instance.
During performance and scalability tests of a high volume BizTalk solution
when you try to determine the Maximum Sustainable Throughput you will
find that logical bottlenecks in the SQL layer are the most common. Because
of these multiple articles have been written on how to best configure the BizTalk Message Box to avoid them. Optimizing Database Performance or even
these old article BizTalk Server Database Optimization are still mostly valid
today and address these concerns.
The single most significant optimization step when dealing with scaling the
BizTalk Message Box is adding new message boxes even when they are created in the same SQL Instance, with the added benefit of being the most simple
to implement. You will need to add three message boxes to achieve the best
performance.

Take advantage of Multi

MessageBox Configuration

TWEET

SHARE

POST

Efficient Host Separation

When configuring hosts always consider the best practices for host separation, start with the separation of artifacts per functionality (receive, send,
orchestration and tracking) then consider isolation (per application, business
unit, etc), then for performance reasons and finally for reliability reasons. A
good Host separation policy will help you scale your environment, ensure stability and security.
Generally it is recommended to start any BizTalk environment with at least 4
hosts, for Receiving, Orchestrations, Sending and Tracking. This follows the
first principle of separation of functionality which will be easier to configure
scalability and easier to control the behaviour such as stopping all receives to
enable an environment drain. Another important host separation technique is
to have a dedicated tracking host, since tracking is a low priority sub-service
if you have tracking in a very busy host it might start accumulating tracking
information on the BizTalk Message Box.
Isolating artifacts with different security requirements is another reason for
host separation, you can assign different Windows Groups, Users and certificates to each host creating security boundaries which will reduce the risk of
running them in the same environment as other artifacts which increases the
return on investment you have done in BizTalk.
You can also separate hosts for security and reliability reasons, since each host
has its own set of dedicated resources such as database tables and windows
services you can separate artifacts or applications that have high performance
requirements that will benefit from these additional resources. If you have

some unreliable artifact you can also use host separation for reducing the
impact of such component (Custom Adapter, Pipeline, etc.) since if that particular host instance crashes it wont affect anything else .
You can read more on this topic here.

Efficient Host Separation

TWEET

SHARE

POST

BAM lightweight tracking

Use BAM as a lightweight tracking mechanism, by disabling default tracking

and enabling BAM activities on a message flow you can reduce the performance impact of tracking. BizTalk default tracking takes an all or nothing approach and using BAM you can decide exactly what needs to be tracked.
When BizTalk default tracking is enabled BizTalk is collecting information on
a lot of different places. All service instances are being tracked for start and
finish, orchestration specifically are being also tracked for message send and
receive as well as shape start and end. This amount of tracking is considerable
and can be increased by activating message body tracking or promoted properties tracking. And while you have the option to disable this type of tracking
you dont have an easy way to control how extensive its.
Using BAM to do tracking gives you the opportunity to track exactly what you
need, you can define your tracking schema (activity) and what you map to
be tracked (tracking profile). The BAM Infrastructure also give you the ability
to define your purging and partitioning criteria allowing you to control how
much data you retain and also OLAP cubes can be created to better analyse
the tracking data. Read more on How to replace Tracking with BAM in BizTalk.

BAM lightweight tracking

TWEET

SHARE

POST

Minimize persistence points

Minimize the amount of persistence points when developing your solution

to guarantee scalability, throughput and low latency. Persisting data to the
database is the single most expensive task in BizTalk, to avoid them you can
group together send shapes in Orchestrations or not using Orchestrations all
together by leveraging static or content based routing.
A persistence point happens whenever certain shapes occur on a BizTalk Orchestration such as the end of a transaction scope, the start Orchestration
shape, the send shape and the end of the Orchestration. Whenever such persistence point happens the BizTalk engine will save the whole state of the Orchestration to the BizTalk Message Box and since writing to disk is the slowest
thing you can do with basic computer resources it comes as a significant cost
to the whole operation.
Reducing these persistence points will have a significant impact in the overall
performance, you can avoid them all together if you dont use Orchestrations
in scenarios where you only do receive, transform and send. You can also reduce the amount of persistence points within a Orchestration if you for example group send shapes inside a atomic scope or use the Call Orchestration
shape instead of the Start Orchestration shape. Read more on this topic here.

Minimize persistence points

TWEET

SHARE

POST

Health Check with

Message Box Viewer

Run Message Box Viewer (MBV) daily to evaluate the health state of your BizTalk environment, this tool brings the knowledge of Microsoft Product Group,
Support and Field Engineers into a single report. You can schedule MBV from
either MBV itself or via BizTalk360 to run every day and be notified if any new
critical or noncritical errors surface.
Microsoft recommends that you run Message Box Viewer every day as part of
the daily operational checklist (see: Checklist: Performing Daily Maintenance
Checks) for checking the health of everything that is BizTalk related such as
the databases, configuration, best practices, registry settings, service packs
and cumulative updates, runtime problems, adapter problems, etc. The likelihood of what you want to check to be done by MBV is very high.
The reports generated are also in a big degree a documentation of your BizTalk environment and can be used to review the BizTalk configuration and
identify message flows within the system. In case problems are identified that
can be solved by the BizTalk Terminator, MBV will generate a configuration file
to load in the BizTalk Terminator and allow you to run the solutions needed
for your BizTalk system.

Health Check with Message Box

TWEET

SHARE

POST

Choose Robust Adapters

Prefer Adapters that implement good error handling capabilities to minimize

the errors inside your integration platform. Handling failures of integration
business processes is key for a trustworthy platform and will push the accountability back to the owners of the data being processed as well as increase transparency.

Choose Robust Adapters

TWEET

SHARE

POST

Performance Testing is Key

Consider performance testing a deployment criteria, never go into production

before knowing what your maximum sustainable throughput is. Assuming
linear scalability is not realistic and the nature of the applications you develop
will dictate how it scales, during performance testing you will have the chance
to optimize your configuration, architecture and code.
There is a great white paper from Microsoft BizTalk Server Performance Optimization Guide which will help you understand various aspects of performance tuning, how to use tools like LoadGen, etc.

Performance Testing is Key

TWEET

SHARE

POST

Package all your resources

into MSI

Include every artifact in your BizTalk Application as a MSI resource for a fully
automated application installation, you can include anything you need including script files that are executed during install and uninstall. This will give you
a single installation package with minimal manual intervention which reduces
the likely would of introducing problems.

Package all your resources

into MSI

TWEET

SHARE

POST

Monitor performance SLAs

with BAM

BAM can be easily used to calculate performance SLAs and provide this information to business owners, you can then enable BAM Alerting and use that
for monitoring your performance SLAs. If you also use the BAM API you will
be able to extend this SLA monitoring capability outside the boundaries of
BizTalk.
By defining an activity in BAM that collects at least two milestones you will be
able to calculate the duration between them using a calculated field in BAM.
Once this is done you now have access to the KPI you need to measure and
you can create a query in BAM to identify instances where the duration was
bigger than your defined threshold. You can also use BAM Alerts to receive a
notification every time your query returns new results.
Read more on this topic BAM to Implement Service and SLA Monitoring

Monitor performance SLAs

with BAM

TWEET

SHARE

POST

10

Build your custom pipelines with

components using default
properties

Build your custom pipelines with components using their default properties
and then use runtime bindings for adjusting them for multiple purposes. This
will reduce the amount of artifacts you need in your solution and will increase
the code re-usage making it easier to test and deploy.
When developing BizTalk applications it is common that developers will create
multiple pipelines just for addressing variations in the configuration of pipeline components. BizTalk allows you to change this properties through configuration and that can be imported and exported with bindings. This in some
cases can mean hundreds of artifacts that you save from creating and deploying; this will also simplify testing your applications.
A very common example of this is the flat file disassembler where you need to
identify the schema to which the flat file being received should be parsed to.

Build your custom pipelines with

components using default properties

TWEET

SHARE

POST

11

Create Security Boundaries

Lock down your BizTalk system by using different security context when accessing sensitive information, you can achieve this by running your hosts and
host instances with different Windows Groups and Users. Reducing the surface of attack with this approach will make your environment less prone to
attacks.
Security is a very important topic when integrating with highly critical applications inside and across organizations. Planning and implementing security
is a demanding task. In BizTalk you will have to assign permissions to receive,
send and query against your most important applications giving the BizTalk
environment a considerable amount of power and at the same time risk of
getting compromised.
An approach recommended is to limit the permissions needed by BizTalk to
just exactly where they are needed, so if my receive host need to be able to
pool a particular SQL Table I should create a BizTalk user account and corresponding Windows Group to assign to the Host and Host Instance that needs
that privilege while at the same time only running artifacts in the same host
that share the same level of permissions. Read more the BizTalk Security

Create Security Boundaries

TWEET

SHARE

POST

12

Review the performance of

your environment regularly

Collect performance counters on a regular basis and analyse them with PAL
for the best performance insight of a BizTalk environment. Collect performance data during your utilization peak time and compare the results with a
baseline to be able to plan for the right capacity in your system.
Monitoring the performance of your BizTalk System is key for a stable and
predictable behaviour. Since the amount of counters available in a BizTalk
System is vast it is hard to know which ones to pay more attention or understanding the correlations of their values. This is where PAL will help you by
providing a list of counters recommended to collect and then parses all these
counters and evaluates them based on the knowledge of experienced BizTalk
engineers and generates alerts when thresholds are being violated. A comprehensive report is generated after running this tool.
Not only will you benefit from the collective knowledge for BizTalk but also
from other software in the stack such as Windows, IIS, SQL Server and more.
Start using it Performance Analysis of Logs (PAL) Tool.

Review the performance of

your environment regularly

TWEET

SHARE

POST

13

Cluster host instances with

adapter like FTP or POP3

Use Microsoft Clustering for adapters that required a single host instance running at a time like FTP or POP3 to avoid duplicate messages while providing
high availability. Failure to implement this will result in either having to deal
with duplicate messages or implementing manual processes for performing
failover.
Due to the nature of some protocols it is impossible to guarantee high availability, scalability and failover. In this cases we normally see customers disabling one of the host instances to prevent duplicate messages. This only
provides manual failover and what should be done is to cluster the hosts
where this adapters run, Windows Clustering will guarantee that at least on
host instance is running and that no more than one is working.
Read more on Improving Fault Tolerance in BizTalk Server by Using a Windows
Server Cluster

Cluster host instances with

adapter like FTP or POP3

TWEET

SHARE

POST

14

Use the Business Rule Engine to

implement Business Logic

Use Business Rule Engine to implement business logic that is modular, reusable and simple. It will allow you to operate on information contained in .NET
objects, database tables and XML Documents. BRE also enables developers to
create and maintain applications with minimal effort.
The Business Rule Engine can be a good way of to modularize the constantly
changing business logic that is often developed inside Orchestrations, making
changes a more smooth process since updating a policy to the latest version
doesnt require any downtime at all. With some training even non BizTalk developers or tech savvy non developers can create and test rules.
Business Rule Engine is also a high performance engine to evaluate business
rules even when compared with WF Rules, read more on this topic WF Rules
and MS BRE Comparing Performance

Use the Business Rule Engine to

implement Business Logic

TWEET

SHARE

POST

15

Split big messages in the

receive pipeline

If you have a big message consider using envelop schemas and the default
pipelines to split the message at the entrance point in BizTalk for best performance and high resource utilization. Using this method also doesnt require
the creation of custom pipelines or very expensive splitting in the orchestrations.

Split big messages in the

receive pipeline

TWEET

SHARE

POST

16

Use Visual Studio for functional

and load testing

Use Visual Studio test capabilities for functional testing and load testing, you
can simulate real load of your environment while at the same time monitoring
performance and behavior of your environment during the test in real time.
When load testing start by doing short duration tests first and then long duration reliability tests.
Here are few great links
Using Visual Studio To Generate Load Against a Two-Way Request-Response
WCF Receive Location
Load Testing BizTalk Server Solutions with Visual Studio

Use Visual Studio for functional

and load testing

TWEET

SHARE

POST

17

Use fast disks for your BizTalk

Database subsystem

SQL Server is vital for any BizTalk environment, ensure maximum performance
for the most demanding databases with a fast disk subsystem, consider using
RAID 1+0 and SSDs for a high performance environment. In the majority of
the BizTalk environments this is the first physical bottleneck you will face.
When doing load testing you will find that after you configure you environment to avoid any logical bottlenecks you will finally find a physical one. Normally the very first physical bottleneck you see is the IO disk subsystem performance on the SQL Server, this makes investing in the disk subsystem the
most important scaling you can do on an environment. Betting on a highly
available RAID system with fast disks is very important.
Read more on Monitoring and Reducing Database I/O Contention

Use fast disks for your BizTalk

Database subsystem

TWEET

SHARE

POST

18

Cluster the Master

Secret Server

Cluster SSO Master Secret Server using Windows Clustering to guarantee this
component is also highly available, you have to do this to achieve full HA. The
best candidate servers on which you can cluster the Master Secret Server is
the SQL Server cluster where BizTalk is installed because you can reuse this
cluster for this low weight service.
If you have not clustered the Master Secret Server there is no high availability
in the BizTalk environment, in this cases the first machine you have installed is
the Master Secret Server and if it becomes unavailable the whole BizTalk environment will not work since it depends on the Master Secret server to read
configuration from the SSO database.
If you want to install the SSO cluster on the BizTalk machines you can only do
it if you also cluster all the hosts in the same Cluster application which means
you will have an active passive BizTalk runtime environment. Read more on
High-Availability SSO Installation Options

Cluster the Master Secret Server

TWEET

SHARE

POST

19

Purge BAM Data

Use the out of the box SSIS packages for purging old data from BAM ensuring
only relevant information is available and that the Performance of the BAMPrimaryImport is the best. You can create SQL Jobs to run this packages which
are automatically generated when you deploy your BAM Activities.

Purge BAM Data

TWEET

SHARE

POST

20

Use SSO as the application

configuration store

For BizTalk Application settings use SSO for secure, scalable and highly available store. This is more secure than the usually used BizTalk configuration file
and is more maintainable since you will have a single place to update your
BizTalk Applications configuration and much more easily you can automate
the deployment of these settings.
Here is a YouTube video showcasing Using SSO for Application Configuration

Use SSO as the application

configuration store

TWEET

SHARE

POST

21

Plan your Backup and

DR strategy

Configure and activate the BizTalk Backup Job to generate the only supported
BizTalk backup files and ensure you are able to restore them by planning and
testing a disaster recovery plan. The success criteria for a Disaster Recovery
plan happens only when you test your scenario.
The only supported way of backing up the BizTalk databases is by using the
out of the box Backup BizTalk Server Job combined the BizTalk Log Shipping
for automatic restoration. It is recommended that you plan the recovery of all
BizTalk server components including the databases, configuration and applications; you should also consider the recovery point objective (RPO) and the
recovery time objective (RTO) when considering about investing in your disaster recovery plan.
Read more on this topic here.

Plan your Backup and DR strategy

TWEET

SHARE

POST

22

Use Webgarden for

IIS scalability

When IIS becomes a bottleneck you can use webgarden for IIS scalability, this
will create multiple hosting processes in the server and IIS will manage the
balancing of load between them. Optimizing your web layer like this is very
simple and highly reliable when compared to changing other internal IIS engine settings.
If you have the need to process a large amount of two way receive port connections through IIS in a BizTalk Application it is likely you will exhaust some
threading capabilities in the IIS engine and hit a logical bottleneck, if you have
access to an IIS expert you can possible reconfigure the engine to overcome
that limitation, or instead using the very scalable and easy to use web garden
capability.
Read On Web Gardens, ASP.NET, and IIS 6.0

Use Webgarden for IIS scalability

TWEET

SHARE

POST

23

Configure and activate the

DTA Purge and Archive Job

Configure and activate the DTA Purge And Archive Job to maintain a healthy
BizTalkDTADb database, just keep the tracking information you need and delete from the live tracking database a soon as possible for the best experience
when searching or understanding what happened to your business processes
in BizTalk.
A big BizTalkDTADb will result in frequent timeouts and a generally bad experience when working with tracking data in BizTalk, the best way to handle
this problem is by configuring and activating the DTA Purge and Archive Job.
If you do need the tracking information and you need to have a smaller database you can use the archiving capabilities for making older tracking information available for querying.
See How to Configure the DTA Purge and Archive Job

Configure and activate the DTA Purge

and Archive Job

TWEET

SHARE

POST

24

Maintain you Message Box

efficiently

Dont let your BizTalk Message Box database grow significantly by accumulating suspended messages, put in place a monitoring mechanism to identify
and resolve suspended messages as soon as possible. When you find a database integrity problem that is causing your databases to grow use the BizTalk
Terminator to resolve this.
Maintaining the health of your BizTalk databases is vital for any BizTalk environment, specially when we are talking about the BizTalkMsgBoxDb, there are
several reasons for the databases to grow like suspended messages that you
should avoid and resolved quickly when they happen. You can also have some
integrity problems identified by using the out of the box Monitor Job or by
running a tool like the Message Box Viewer, in this cases you can run the BizTalk Terminator which is a tool provided by Microsoft to address these issues.
Read about the BizTalk Terminator

Maintain you Message Box efficiently

TWEET

SHARE

POST

25

Consider to prefer
XSLT in maps

When developing maps consider using XSLT as the preferred language for
describing your transformations by either using Custom XSLT, Scripting Functoids or even when developing Custom Functoids. This will result in maps that
are faster to run and easier to debug.
Using the out of the box functoids is sometimes hard or impossible to implement the necessary transformations leaving developers a lot of options on
how to overcome those limitations. Using custom XSLT as a full map, a inline
XSLT scripting functoid or developing a functoid that outputs XSLT is in the
majority of the times the best option due to the end result being just on single XSLT that needs to be compiled and run just like any other simple map
without any need for external components.

Consider to prefer XSLT in maps

TWEET

SHARE

POST

26

Test 100% of Your Code

Guarantee full code coverage by running Orchestration Profiler of tracking

data of your load tests. When testing your BizTalk Applications it is important to guarantee all code is executed, even the error paths. The Orchestration
profiler will help you understand how much of the code you covered with
your test and also give important information on how fast it was.
When testing software one of the important measurements to guarantee a
thorough analysis is code coverage. Since this metric is not possible to measure in Visual Studio for BizTalk Projects Orchestration Profiler has been developed and uses the tracking information generated by BizTalk to identify which
code path have been executed and also includes additional information on
the success rate and volumes going through each shape.
Learn more on Orchestration Profiler and start using it on your projects.

Test 100% of Your Code

TWEET

SHARE

POST

27

Implement a Enterprise
Service Bus using the ESB Toolkit

Use ESB Toolkit to implement a dynamic self-adapting solution. It enables the

implementation of the ESB pattern within BizTalk by maximizing the re-use
of services while at the same time maintaining the flexibility to easily change
solution. The ESB Toolkit extends the capabilities of supporting a loosely coupled and dynamic messaging architecture.
Here is a good place to start working with the ESB Toolkit by reading the
Introduction to the BizTalk ESB Toolkit.
If you want a in-depth video on the ESB Toolkit capabilities watch this video.

Implement a Enterprise
Service Bus using the ESB Toolkit

TWEET

SHARE

POST

28

Avoid Orchestrations
when possible

Use static routing, content based routing or itineraries to avoid using Orchestrations and use routing of failed messages for advance error handling since
messaging doesnt provide a rich error handling capability. This approach will
give you the high performance of messaging and the power of the Orchestrations when necessary.
When a high volume of messages is going to be processed in BizTalk every
cost is important, activating an Orchestration has a significant impact due to
the additional Message Box hop and the Orchestration execution. In a lot of
scenarios you cannot avoid Orchestrations, but in many cases you can avoid
them by making use of the messaging capabilities such as routing or transformations in the ports. Another interesting aspect of this is that if something
goes bad with simple messaging and you need the power of the Orchestrations you can still use them when combined with Routing of Failed Messages.

Avoid Orchestrations when possible

TWEET

SHARE

POST

29

Develop Adapter Using WCF

When developing new adapters create a Custom WCF Channel or use the
WCF LOB SDK as a reference starting point, this will allow you to create a scalable and easy to host adapter that can be used across other .NET solutions.
This level of flexibility will make the adapter more likely to be reused somewhere else besides the BizTalk world and so being more valuable than simply
building it with the BizTalk Adapter framework.
If you have to decide to implement a BizTalk Adapter you will want to make it
so that it can be as relevant as possible and choosing the framework you will
use is an important task. The simple fact that developing adapters for WCF
makes them reusable should be a compelling enough reason to decide.
You can understand the Differences Between the WCF LOB Adapter SDK and
the BizTalk Server Adapter Framework here.

Develop Adapter Using WCF

TWEET

SHARE

POST

30

Use ETW for high

performance tracking

Use Event Tracking for Windows (ETW) for a high performance tracking and
debugging of your BizTalk Applications. Since this method uses some high
performance structures within Windows to host tracking the impact is near
zero if you are not consuming the output and highly valuable if you are troubleshooting a problem.
You can access to an implementation of this type of tracing on Best Practices
for Instrumenting High Performance BizTalk Solutions

Use ETW for high performance tracking

TWEET

SHARE

POST

About the Author

Ricardo loves technology and knowledge in general, if you challenge
him you will get a challenge accepted look back. He had the chance to
work with a lot of the most important BizTalk customers throughout
his years at Microsoft which gave him extended experience around the
needs and practices of the integration space. Explain to him your integration scenario and you will have a proposed architecture within minutes ready for a thorough review! If you dont know how much you are
missing out on what BizTalk360 has to offer you should ask Ricardo for
a travel through the wonders of the product.
Ricardo Torre
Head of Technical Pre-Sales & Product Strategist

www.BizTalk360.com

Microsoft, Microsoft BizTalk Server are either registered trademarks or

trademarks of Microsoft Corporation in the United States and/or other countries.