I resently got a question on linkedin about using streaming approach in BizTalk pipelines. More precisely how to read a part of the message without reading the complete message in memory.
The underlying goal was to get the message count in a batch message, with envelope . Strangely enough xmlreceivepipeline does not return the message count. The component only returns the index of the specific message plus that the last message is marked as the last message in the interchange.
What i created is a none specific message stream as i wanted it to handle both text and xml messages. For this to be possible a pattern search is made in a streaming fashion of course.
Even though the example uses a custom xml disassembler component the custom stream in it self can be used by any disassembler component.
To count the messages in the batch in a streaming fashion you must replace the original message with the new custom message using the original message as input.
What happens then is that when the disassembler reades the message it realy reades the new custom stream. The stream returns the message as is without modifiyng it in any way but it keeps a count of how many times a specific pattern occurs in the original stream.
As the message is read in chunks by the disassembler the search is done byte by byte. The xml disassembler reads the message two times. Once when searching from promoted properties and the second time when the messages are returned by the GetNext method.
I make sure that the stream only does a pattern search the first time the pipeline component traverses the message.
The pattern match count is save in a private variable called recordCount that is later promoted on each message that is returnedby the GetNext method.
The sample includes a property schema with the specified namespace and property name.
So in a nutschell the biztalk streaming pattern is as follows
- Replace the original biztalk message stream with a custom stream.
- Replace the BodyPart.Data with your custom stream.
- When biztalk now reads the stream as it possible does several times before it reaches the messagebox it actually reads your custom stream which in turn reads the original biztalkmessage.
The custom stream can then do anything with the original stream data even changing the content before it returns it to biztalk.
What you gain by doing this is
- The message dones not need to be traversed more times then neccesarry.
- Low memory consumtion.
The code for the solution can be found in my one-drive in a zip file called ExtendeDisassemblers.zip
fernandodosanjos 