My challenge, opportunity, and resolution for the new year is to update this blog more often. Let's see if I can actually follow through...
I have some new information to post regarding AIF queue management that I've known for a few months now but just have not made the time to formally post.
As with most documentation that you see for setting up a batch server, the instructions either state or imply to run both the AOS service as well as the batch client on the same dedicated server. This may at first seem like a common-sensical notion since the Batch Client, the AOS Service and the data storage all reside on the same system and will ultimately aide in reducing any network bottlenecks of large amounts of batchable transactions running across the network. However, this is not the case because the bottleneck of having both components running on the same system is the memory. Due to the 1.7GB limitation of 32-bit applications, the OS, the AOS, and the client are all contending for the available memory.
Depending on the amount of transactions that you have processing in an hour, or even a day, separating the processes out onto separate servers may not be necessary for you. In our case, we have approximately 1-500 messages measuring between 30-50KB that pass through our AX system via AIF every hour. As a result of the number of messages we process, our 'All-In-One' setup proved to be quite the bottleneck and resulted with us constantly getting 'Out of Memory' errors in our AIF queue. At one point, we had to filter out all messages larger than 200KB simply because even after freeing up all memory available, we still were unable to process those messages and import them through custom file I/O operations. After testing and working Microsoft, we found that separating the AOS Service on a dedicated server and running the client(s) on a separate dedicated server gave us our best potential performance and freed up the client to contend for memory only against the OS and the same with the AOS service.
To further increase our AIF performance, we found that instead of running all four AIF batch processes together in a single client session, separating each of the four out into separate client sessions increased performance. As a result, you are not limited to the four processes being executed in a sequential order but instead they can all three execute independent and even concurrently. Although the issue still exists that the four processes are related and dependent of each other, however, messages that are ready to be processed at the next execution of the batch are not held back from being processed.
For example, the four AIF processing classes are:
- AIFSendGateway
- AIFReceiveGateway
- AIFInboundProcessingService
- AIFOutboundProcessingService
In order to import a message into AX, AIFReceiveGateway must execute to pull the message into the AIF Queue and the AIFInboundProcessingService must execute to process and persist the data to the database. Likewise, to export a message, AIFOutboundProcessingService must retrieve the data from AX package it in the form of a message and write it to the queue before the AIFSendGateway can pick it up and write it to a message. As a result, spliting up the jobs into different batch clients will allow you to make the AIF message passing process closer to a 'real-time' process.
Finally on a separate note, I have just completed reviewing a new book on Dynamics AX that will soon be published. Once the book is officially published, I will post a link to it.
DB
