Toying with those maps

So what i am trying to figure out or prove is XSLT maps are faster than standard BTS maps outputs, reason i believe so is XSLT maps will use standard .NET transformation classes which based on firehose cursor style to process XML messages

however standard maps are basiclly XML document that BTS has a way to process, what i did is i reflected the code for a map that uses XSL and here is what i found
[SchemaReference("LinkDev.BizTalk.Adapters.OracleStaticAdapter.Schemas.CommandExecuteRequest", typeof(CommandExecuteRequest)), SchemaReference("MinistryOfCommerce.eInvestor.BizTalk.Schemas.SuperApprovalSchema", typeof(SuperApprovalSchema))]
public sealed class mapAppFeesSUperAppIn2ExecuteReq : TransformBase
{
// Methods
public mapAppFeesSUperAppIn2ExecuteReq();

// Properties
public override string[] SourceSchemas { get; }
public override string[] TargetSchemas { get; }
public override string XmlContent { get; }
public override string XsltArgumentListContent { get; }

// Fields
private const SuperApprovalSchema _srcSchemaTypeReference0 = null;
private const string _strArgList = "\n";
private const string _strMap = " \n \n \n \n\t --- Rest of XSLT file that i used for the map";
private const string _strSrcSchemasList0 = "MinistryOfCommerce.eInvestor.BizTalk.Schemas.SuperApprovalSchema";
private const string _strTrgSchemasList0 = "LinkDev.BizTalk.Adapters.OracleStaticAdapter.Schemas.CommandExecuteRequest";
private const CommandExecuteRequest _trgSchemaTypeReference0 = null;
}




while trying to reflect a standard map i found:
[SchemaReference("MinistryOfCommerce.eInvestor.BizTalk.Schemas.eInvestorResponseMessage", typeof(eInvestorResponseMessage)), SchemaReference("MinistryOfCommerce.eInvestor.BizTalk.Schemas.eInvestorTransactionMessage", typeof(eInvestorTransactionMessage))]
public sealed class mapInternal2Response : TransformBase
{
// Methods
public mapInternal2Response();

// Properties
public override string[] SourceSchemas { get; }
public override string[] TargetSchemas { get; }
public override string XmlContent { get; }
public override string XsltArgumentListContent { get; }

// Fields
private const eInvestorTransactionMessage _srcSchemaTypeReference0 = null;
private const string _strArgList = "\n";
private const string _strMap = " \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n";
private const string _strSrcSchemasList0 = "MinistryOfCommerce.eInvestor.BizTalk.Schemas.eInvestorTransactionMessage";
private const string _strTrgSchemasList0 = "MinistryOfCommerce.eInvestor.BizTalk.Schemas.eInvestorResponseMessage";
private const eInvestorResponseMessage _trgSchemaTypeReference0 = null;
}



so conclusion so far are:
- even standard BTS maps generates XSLT and being pocessed slightly different check out those var:var statments.
- I still belive that XSLTs are faster and they gave u more flixbility more on this now.
- If i want to create a simple map i will use standard BTS map wizard, while if i want a complex (will be used alot) i will go for XSLT


Kal

Fast BPI, Faster Than What?

Regular weekend cappuccino J trying to get thoughts straight for that blog entry, okay what I am talking about. I am talking about BTS 2004 Orchs performance and optimization.

I happened to be in one of these projects where we are delivering an integrated “fully configurable” workflow, you know the drill external data store contains all the configuration data this configuration data affects your orchs in a couple of areas
- Message process – should your workflow process a just received message or kick it back returning some business validation rule violation error – yeah those errors that you keep in Strings.cs or external resources files

- Decisions shapes – should your orch do X or Y


Anyway during the implementation I found out about a couple of interesting things that would

- Make your life easier during the development
- Let you answer questions from other techy’s/whoever telling you hey that is slow I can write C# code that would do better job and more importantly faster.



Development:
As you have noticed developing orchs using VS.NET 2003 is nasty especially – That one is much, much better in BTS 2006 code named PathFinder as I have learned - if you are implementing a lot of ports and messages in/out. The reason behind that is GDI L. Somewhere in the designer it consumes a lot of memory I found out that:
- Develop using Windows XP or 2003 it – the designer - is more optimized on them (don’t ask me why I am trying to find out why still)
- If you have to do it on win 2000 then get that SP1 it really helps
- Don’t connect to local SQL, Deployment and enlisting generates a lot of traffic on local SQL so why not using one of these big/ fat development servers instead of connecting locally
- Always, always close port surface like:



Reason behind that is it really puts GDI @ ease to eliminate these lines connecting your shapes and ports

- If you are using Virtual PC to do the good stuff, then put it – VMD files - on a different HDD if you are running on a tower pc or put it on USB 2 external HDD if you are running on a laptop – yes these works like a charm

Runtime:
Okay scaling up or out your servers is pretty straight forward, adding more servers and how to add them is published everywhere on MSDN and Microsoft web site. But what I am trying to introduce is ability to get these orchs to act faster and do more.


Let me tell you this the problem that might make your orchs slower isn’t executing code within the orchs it is the state maintenance, by that I mean the fact that BTS saves the orchs during execution so if things went wrong it can compensate, execute from last successful step or whatever you configure it.

Most of the code generated out of orchs is maps call or send/receive calls the heavy calls – other than state maintenance is during sending or receiving (hence pipeline/adapters) can really damage your performance expectation but I am saving that talk for later blog entry.

Before I go on I want to introduce to concepts related to orchs processing within BTS
- Serialization: is done when BTS put a mark on your orch instance, think of it as the last successful action made during you’re your processing, this happens a lot in Send/Receive loops etc..
o How it happens: BTS orch code genereated is just a .NET class that implements ISerializable interface or other seriliazation mechanism, during the execution BTS decides it needs to mark it, it serializes everything to the MSGBOX DB. And it doesn’t take anything out of BTS Host’s memory. Pretty smart stuff eh? J loved that one btw
- Dehydration: is done when BTS decides ok this particular instance is not needed in the memory any more (happens in listen, delay, receive shapes – other than activation). Later on when BTS decides to awaken – hydrates – this instance based on message received, delay is over – those one are message also called timer messages- , more on that later on other blogs.
o Ho it happens: it happens using serialization as well but this time BTS takes everything out of the host’s memory.


As you have noticed both operations entails heavy calls to SQL stored procs, unfortunately you can not control dehydration process as it is totally controlled by BTS – which is good to tell you the truth – however the serialization process is controllable using few tricks.

Btw I found this post to control the dehydration but it is don’t do it at home style



Back to our talk, BTS never serializes scoped operations reason is message processing so scope shapes can be your best friend if you used them well:

Consider the following picture: it is send receive shapes in a scope (BTS will not go to DB in scopes, I have tried atomics and none transactional none synchronized scopes and it works like a charm). This way you minimize calls to SQL hence faster processing.



NOTE: doing that will not let BTS mark the execution so if it is failed somewhere in the middle of the scope it will start the execution before the scope. Hence do that carefully on top of my mind I can tell you
- do that when there is no harm repeating the calls (like Update statements)
- do that in loops if there is no harm repeating your loops starting the counter @ 0.
- Do that if your adapters are transactional.
- Don’t do that when your sending messages to external entities (you never trust how they process repeated messages)
- Don’t do that during calls to web services (unless they implement some transactional mechanism).


Other tips related to performance is parallel execution
- No I am not saying use it, instead don’t use it in message processing. Use them if you are doing some execution that has nothing to do with sending or receiving message. Reason is using that will most probably require some synchronization shapes which hurts your performance. (in addition imagine the number of calls to SQL if you have a couple of branches that send/receive messages)

If you want to test how many times BTS serializes your orch instances create a serializable class have it as amember variable in your orch and let it save counter for each time your serialization ctor called, then try with different shapes (loops receive delays and so on) promise, astonishment!

I will keep this post short will stop here, later I will talk about messages/schemas I want to tell you about optimizing your schemas for performance (that will entails talking about pipelines)


Feel free to communicate with me on khnidk@gmail.com

Enjoy BPI,
Kal

BizTalk Map Tester

This tool test BizTalk maps (out of BTS assemblies), this one comes handy if you suspected that one of your maps isn't doing what it's supposed to do. without going to VS.NET

http://www.gotdotnet.com/Community/UserSamples/Details.aspx?SampleGuid=ac910d58-3cf3-4b01-8e56-79ea86aba2f3


Kal

Deploy, Bind, Enlist & Start. Need to say more?

a very busy week; one of the biggest projects is getting to see the light, alot of tight deadlines and deliveries. anyway it is getting out which is good ;)

Okay so now you decided i wonna go develop some BTS projects to do some functionlity - whatever it's for now- the first question you gonna ask yourself is how am i going to build/structure my solutions you have alot of stuff @ hand orchs maps schemas custom components and whatever that comes into picture there. below are so thoughts i found comes handy intime of needs

Rules:
- Keep your projects small as possible as it will affect the time it takes to deploy to BTS DBs
- Keep Maps in seprate project those little devils really take alot of time to compile. though in size comparson BTS orch project produces bigger assemblies.

BTS Solutions Structure:

Solution 1: Pipelines and thier helperss & support components.
Project A- .NET Class Library:
Contains pipleline components (GAC these components)
Project B- .NET Class library:
Contains .NET classes that your pipleline components consume.
Project C- Pipelines:
Contains your BTS piplelines (this project will be GACed so you should strong name it)
Solution 2: Buisness Process and thier helpers
Project A: Schemas (Strong named as it goes into GAC). this project output will be referenced from almost everything(pipleline assmblers & disassmblers, orchs ...etc).
Project B: Transform
Your Maps and thier XSLTs if your are using them (GACed as well)
Project C: Process
Your Process orchs
Project D: .NET class that you Business process consumes. (GACed as well)

You think your done? the answer is yes if you want just to start. but the real answer is no if you want to make your life is easier moving on

Okay during the development a big percentage of time you will be doing Deployment, binding enlistment and starting.

in to move things from Development environment to Staging Environment i would strongly recommend nAnt deployment with the help of this sample doc

However in to deploy things on your machine you need something easier to configure and very very user friendly.

I have found that batch files are the best (faster and easier to maintain - remember that any code you write you will have to maintain and debug).

here how they should look like (& they should run in the following sequence):

• Termination Exe(terminating any orch intances):a good guy Sherif ElMeteny developed an EXE that terminates any instance of orchs running on local BTS using WMI. Note DON'T run this on production environment. (the exec is avaliable on my workspace). you can run the EXE using the following command: BiztalkWMI.exe Terminate
  
• Stopping Orch Batch: this batch stops the running orchs on Local BTS. should look like

cscript /NoLogo StopOrch.vbs "Fully Qulified Orch Name" "Assembly Name" Unenlist

Note: Keep StopOrch.vbs - found in BTS directory/SDK/Sampls/Admin/ in the same folder

• Undeployment Batch: undeploys certain assemblies: the batch should contains commands like

BTSDEPLOY REMOVE NAME= VERSION=1.0.0.0 Culture=neutral PUBLICKEYTOKEN=XXXXX UNINSTALL=True LOG=BTS.undeploy..process.log

BTSDEPLOY REMOVE NAME= VERSION=1.0.0.0 Culture=neutral PUBLICKEYTOKEN=XXXXX UNINSTALL=True LOG=BTS.undeploy..process.log

BTSDEPLOY REMOVE NAME= VERSION=1.0.0.0 Culture=neutral PUBLICKEYTOKEN=XXXXX UNINSTALL=True LOG=BTS.undeploy..process.log

• Deployment Batch: deploys certain assemblies: the batch should contains commands like

BTSDEPLOY DEPLOY ASSEMBLY="C:\Projects\BuildOut\ProjectNameSpace.Schemas.dll" INSTALL=True LOG=BTS.deploy.ProjectName.schema.log

BTSDEPLOY DEPLOY ASSEMBLY="C:\Projects\BuildOut\ProjectNameSpace.Transform.dll" INSTALL=True LOG=BTS.deploy.ProjectName.Transform.log

BTSDEPLOY DEPLOY ASSEMBLY="C:\Projects\BuildOut\ProjectNameSpace.Process.dll" INSTALL=True LOG=BTS.deploy.ProjectName.Process.log

BTSDEPLOY IMPORT BINDING="C:\Projects\BuildOut\PortBinding.xml" LOG=bts.deploy.Project.binding.log (this file is imported after the deployment using BTS deployment wizard)

• Starting Orchs Batch: this batch starts the orchs after a deployment: should look like

cscript /Nologo EnlistOrch.vbs "Orch Fully qualified name" "Assembly Name" Start

Note: Keep EnlistOrch.vbs - found in BTS directory/SDK/Sampls/Admin/ in the same folder

As you have noticed these batch files are easy to maintain and run on the development environmnt i have noticed that the best thing is to run them then go grap a coffee, that explains my too mainy coffee/smoking breaks ;)

anyway. one challenge is if you renamed a port and you tried to deploy using the old binding file you will get address confilcts, so there is a couple of VBs that i have developed that delets all sendports and all recieve ports. (needless to say don't run those on production environment)

I have published samples to these batchs on the workspace with the ability to add GAC DLLs as well. they might come handy as you have noticed

well, that was a long post! keep me posted with your comments and feedback, use my Gmail address (khnidk@gmail.com)

Enjoy Deployment :)

Kal

Hey and Welcome

Hi,

My name is Khaled Hnidk, Call me Kal as it is easier this way. I am a Technical Consultant working with LinkdotNET a regional - middle east - ISP, based in cairo

Since almost everybody i know have been pushing me around to get this blog up, i finally decided to get it done.

I have been for working for some 10+ years as programmer and related jobs ;)

next, What this blog is all about? it's about BPI - Business Process Integeration - using Microsoft Technologies. stuff like BizTalk Server, BizTalk related frameworks, SPS.. you name it.

currently i own this gotdotnet work space which is about an Oracle Adapter for BizTalk Server 2004. yet i believe i have more to share with world :)

I will be sending more posts about tips, tricks, thoughts and hard aquired experiences about BPI that hopefully would make your BPI experience a bit easier and definitly more fun.

That is it for this one

Kal