This may apply to more than the history requests, but that's where I've run into this problem first.

When requesting history data, I will often process a batch of files to back-fill data. To speed up this process, I keep the socket open and go through a lot of request data, read response sequences. If something goofy happens (like getting two !ENDMSG! tokens for one request), it is possible for my software to read the first !ENDMSG! token and think that it's done. The next request would be made and then the second !ENDMSG! would be read and mistakenly matched up with the second request. (Note that the double !ENDMSG! problem is already scheduled to be fixed, but I feel there is still a need for a beginning-of-response token).

There are many ways to solve this, but I feel the most robust way would be to echo the original request back as the start of the requested data. Another possibility would be to add an additional field to the original request that will be passed back at the beginning of the data transmission. That would make it possible to be backwards compatible with the current implementation.

Current Implementation

TX> HD,[Symbol],[Days];
RX> Time Stamp, High, Low, Open, Close, Volume, OpenInterest<CR><LF>
      . . . 
RX> Time Stamp, High, Low, Open, Close, Volume, OpenInterest<CR><LF>
<CR><LF>!ENDMSG!<CR><LF>

Proposed Implementation with user-defined request sequence field. If the Request Sequence field is not provided, then the current functionality remains.

TX> HD,MSFT,10,[Request-Sequence];
RX> !BEGIN_MSG!  Request Sequence<CR><LF>
RX> Time Stamp, High, Low, Open, Close, Volume, OpenInterest<CR><LF>
      . . . 
RX> Time Stamp, High, Low, Open, Close, Volume, OpenInterest<CR><LF>
<CR><LF>!ENDMSG!<CR><LF>

If this isn't clear, let me know and I'll try to explain some more...

P.S. This is similar in concept to the thread http://forums.iqfeed.net/index.cfm?page=topic&topicID=830

Edited by EspressoE on Mar 17, 2006 at 01:48 PM