On TCPIP, it seems impossible to synchronize an intra-day tick history request and watchlist request.

Consider the problem of starting a new intra-day trade tick chart for symbol X after market open. It needs to display today's old ticks and subscribe to all new ticks.

This isn't a problem if you are subscribed before market open and have no crashes or network failures, etc...

Without IQFeed providing at least milliseconds on the Watchlist Update message, its impossible to know where the Watchlist tick messages begin and the Historical tick messages end.

Even then, with milliseconds its still a guess as there may or may not be duplicates at the historical->watchlist juncture.

Better still a proper sequential daily tick identifier on each message is required including the seconds and milliseconds.

The sequential daily tick identifier, starting at 0 for each day, sequentially orders all trades and quotes - either trades and quotes together (in the case of a single trade and quote stream or seperate trade and quote streams) or trades and quotes individually.

Each Watchlist tick (trade and quote) gets a sequential/unique id or counter for the day:
Q,0,MSFT,...HH:MM:SS:ZZZt,....
Q,1,MSFT,...HH:MM:SS:ZZZb,....
Q,2,MSFT,...HH:MM:SS:ZZZa,....

This sequential tick identifier is only a change to the message and doesn't change the API.

Another alternative could be to change the Watchlist API to optionally send historical/today's ticks in addition to updates so it sends all historical ticks and appends a snap/batch complete message before it sends updates for example. But I'm assuming changing the API is a lot more work.


In the problem case listed above, it would be more efficient to only send previous/today's trade ticks and not the previous/today's quotes when building the chart.

In other problem cases like recovery or specialized analytics for example I need to recover previous/today's quotes in addition to trades. But this is still useless without milliseconds and a sequential tick identifier! (at least in the current API the historical tick request will send trade and quote ticks in the same stream - I'm guessing in the order they arrived)

Maybe you could think about splitting historical trade and quote tick requests so they are optional in each request - either Trade or Quote or both. This way you could save on bandwidth at least for the problem case listed above. Then if you request trade or quotes seperately, using the sequential tick identifier you can still match them and order them correctly.

You could also apply this trade or quote or both optionality to the watchlist request. Using the sequential tick identifier we could piece them both back together if for example we want to save them to db for example or enable quotes and ticks half way through the day.


...But in the mean time, to get around this problem with the current API, I guess I have to subscribe to 500, 1300 or 15000+ symbols before market and subsequently waste many megabytes per minute of bandwidth and resources of the DTN servers. And then hope that my app, IQFeed, the PC, or the Internet don't crash.

Jay mentioned DTN are going to add seconds and hopefully milliseconds in the future.

Any comments or tips from DTN if this is wacked out, makes any sense and/or maybe how I can solve this please?

Unless we can work around this problem then I think its a worthy topic for discussion before the next release.

Sasha,

We plan on adding the exchange's Tick Identifier to the historical data and the data feed. It's my understanding that this value is guaranteed to be greater than the last for each new tick.

We also plan on adding second resolution. With the tick identifier, though, I don't really see a need for millisecond resolution here. The exchanges don't transmit time with millisecond precision, and, taking propagation times between the exchanges and the end user into account, any millisecond value would be totally arbitrary and based only on system times here.

---

Tim Russell
Software Engineer
DTN IQ & FinWin

Great to hear Tim!

This will be very useful. Thank you.