Join the 80,000 other DTN customers who enjoy the fastest, most reliable data available. There is no better value than DTN!

(Move your cursor to this area to pause scrolling)




"I started a trial a few weeks back before the market went wild. DTN.IQ didn’t miss anything and beat my other provider. I decided to stay with you because of the great service through all the volatility." - Comment from Mike
"I've been using IQFeed 4 in a multi-threaded situation for the last week or two on 2600 symbols or so with 100 simultaneous daily charts, and I have had 100% responsiveness." - Comment from Scott
"DTN has never given me problems. It is incredibly stable. In fact I've occasionally lost the data feed from Interactive Brokers, but still been able to trade because I'm getting good data from DTN." - Comment from Leighton
"Thank God for your Data Feed as the only Zippers I see are on my pants (LOL), and no more 200 pip spikes to mess up charts." - Comment from Spiro via Email
"I've never had DTN go out on me since switching. ******* would go down a couple times every month when I was using them." - Comment from Bryce in AL.
"Excellent datafeed !!!" - Comment from Arely
"This beats the pants off CQG, I am definitely switching to the ProphetX 3.0!" - Comment from Stephen
"I had always used ******* but for the past 2 weeks have been trying DTN IQFeed. Customer support has been extraordinary. They call just to make sure your problem hasn't recurred." - Comment from Public Forum
"I am enjoying the feed very much - so superior to the broker provided feed I was previously using." - Comment from George
"My broker in Davenport suggested I give you a try as he uses your service and says its the best." - Comment from Bill via RT Chat
Home  Search  Register  Login  Recent Posts

Information on DTN's Industries:
DTN Oil & Gas | DTN Trading | DTN Agriculture | DTN Weather
Follow DTNMarkets on Twitter
DTN.IQ/IQFeed on Twitter
DTN News and Analysis on Twitter
»Forums Index »IQFeed Developer »IQFeed Developer Support »Keeping up with L2 flow during busy market hours
Author Topic: Keeping up with L2 flow during busy market hours (8 messages, Page 1 of 1)

jmv
-Interested User-
Posts: 19
Joined: May 31, 2011


Posted: Jun 16, 2020 10:18 AM          Msg. 1 of 8
Hi there,

recently, I've spent some time modifying my trading application in order to keep a full real-time visual snapshot or "view" of the DOM landscape. While the visual result is very fancy and all of that (d'oh!), in the process of toying around with IQFeed's L2 stream I've come across a problem which I'd like to share with you.

Thing is, L2 can be at times a really busy stream, even more so if you happen to pick a busy symbol (NQ, ES, YM...). I've come to record incoming L2 message ratios well in excess of 1500 L2 messages per second coming from IQFeed during busy market hours (stock maket open at 09:30EST, close at 16:00EST, etc.. Normal, "quiet" periods tend to average 200-500 messages per sec.

The problem I'm experiencing is that, the moment the L2 message ratio reaches aound the 1000 mps (messages per second) mark, my IQClient app starts losing L2 messages, and it will disconnect / reconnect to your L2 servers. Here's an example from the logs:

------------
STATUS Information 36 0 2020-06-16 16:21:38 No data from Level 2 server in 7s. Reconnecting.
STATUS Information 36 0 2020-06-16 16:21:38 Attempting to reconnect to Level 2 server.
STATUS Information 36 0 2020-06-16 16:21:43 Connected to Level 2 server.

STATUS Information 36 0 2020-06-16 16:40:03 No data from Level 2 server in 43s. Reconnecting.
STATUS Information 36 0 2020-06-16 16:40:03 Attempting to reconnect to Level 2 server.
STATUS Information 36 0 2020-06-16 16:41:09 Connected to Level 2 server.

STATUS Information 36 0 2020-06-16 16:41:48 No data from Level 2 server in 39s. Reconnecting.
STATUS Information 36 0 2020-06-16 16:41:48 Attempting to reconnect to Level 2 server.
STATUS Information 36 0 2020-06-16 16:42:11 Connected to Level 2 server.
----------

Sometimes, it will recconect itself in mere seconds (3-4 seconds), while other times one or two minutes can pass until it will recconect itself to the L2 server.

I know that if my end cannot keep up with IQFeed's server, your host will likely drop my connection, hence these disconnections could be due to my system not being able to cope with one of these "L2 storms" which it's happening during market busy hours.

The problem I see here is that I already tried to make a very simple, stripped-down L2 client just to make a proof-of-concept of how many L2 messages per second I would be able to catch. Even with a simple read-and-discard strategy (no L2 message storage, no ascii parsing, no dynamic structures used, no thread locking, no synchronization locks, etc) and then, I keep getting kicked out of your L2 servers during _Really_ busy market events (opening, close, etc.).

So, if the software isn't blocking your stream flow... could it be just a latency problem ? My servers have a not-so-nice 180-190ms ping round trip to IQFeed servers. With as high a latency as that, am I facing a structural problem here, and no matter how fast my software is, I won't be able to keep up with a fast moving L2 stream ? (unless I opt for leaving Europe and collocating my servers within USA, of course..)

thanks for your time,
Edited by jmv on Jun 16, 2020 at 10:18 AM
Edited by jmv on Jun 16, 2020 at 10:18 AM
Edited by jmv on Jun 16, 2020 at 10:20 AM

DTN_Gary_Stephen
-DTN Evangelist-
Posts: 141
Joined: Jul 3, 2019


Posted: Jun 16, 2020 11:15 AM          Msg. 2 of 8
There are no hard-and-fast limits on the number of L2 messages you can receive at once, or anything else that should be causing this behavior. Can you tell me what programming language you are using, and the list of symbols you're requesting that's causing the problem? You may send this information to the IQFeed API support email or support chat if you'd prefer.

Sincerely,
Gary Stephen
DTN IQFeed Implementation Support Specialist

jmv
-Interested User-
Posts: 19
Joined: May 31, 2011


Posted: Jun 16, 2020 11:28 AM          Msg. 3 of 8
The language I'm using is Java (openJDK 14), and I'm asking for seven symbols : ES,NQ,YM,6E,6J,6B, BTC .

I'm just toying a bit with the system's granularity right now... I'm checking the receiver buffer a bit less often (every 25ms instead of 5ms which was before). I'll let you know if this eases the problem.

However, the real test will come when markets close (i.e., 16:00EST ) since this is one of the busiest events in the day, from a L2 perspective. So in three and a half hours we will be able to see whether we still hit this problem again.

For me, it's highly suspicious that the problem only manifests itself in the busiest market moments (i.e., when the L2 message flow is at its peak). Either my receiving software(*) is the bottleneck, or my latency is causing the problem, I'm afraid.

(*)note: as for my software, I've also tried raising the socket buffer size, just to be sure that I wasn't discarding L2 messages due to a network bufer overrun... but that doesn't seem to be the problem, either.
Edited by jmv on Jun 16, 2020 at 11:30 AM
Edited by jmv on Jun 16, 2020 at 11:33 AM
Edited by jmv on Jun 16, 2020 at 11:36 AM
Edited by jmv on Jun 16, 2020 at 11:37 AM

DTN_Gary_Stephen
-DTN Evangelist-
Posts: 141
Joined: Jul 3, 2019


Posted: Jun 16, 2020 12:22 PM          Msg. 4 of 8
If you are willing, we can test your app in-house. This will help eliminate network or machine as possible causes of the delay.

Sincerely,
Gary Stephen
DTN IQFeed Implementation Support Specialist

jmv
-Interested User-
Posts: 19
Joined: May 31, 2011


Posted: Jun 17, 2020 08:42 AM          Msg. 5 of 8
Thanks for your kind offer ,

as of yesterday's experiments, I've found out that tinkering with granularity (i.e., thread locking frequency) helps yielding better results. Yesterday, I got only a final (and single) L2 disconnection event, at market's close (16:00EST) which was to be expected, since it's the busiest moment of day (along with the stock market opening at 09:30EST, this is). This is much better than previous days, when I was getting several L2 disconnections *per hour*.

This tells me that the problem is at my end (most probably, excessive thread locking is still my problem). I think I can fix this by toying a bit more with my system's granularity. Should this not be the case, I will send you the source code of my stripped-out as-simple-as-possible test mule barebones L2 receiver, so that you can check it out for yourself.

thanks for you time and great support,

DTN_Gary_Stephen
-DTN Evangelist-
Posts: 141
Joined: Jul 3, 2019


Posted: Jun 17, 2020 09:03 AM          Msg. 6 of 8
If you decide you would like us to review it, you can get in touch with me via any of the support channels. Ask to speak to Gary (me), and I'll handle everything from there.

Sincerely,
Gary Stephen
DTN IQFeed Implementation Support Specialist

jmv
-Interested User-
Posts: 19
Joined: May 31, 2011


Posted: Jun 22, 2020 04:31 PM          Msg. 7 of 8
Hi Gary,

just a couple of lines in order to let you know that, by means of optimizing our code (mostly trying to avoid race conditions and time-costly locks), I've been able to keep the L2 disconnections at a minimum (perhaps twice or three times a day, in the busiest day moments).

So the problem is _nearly_ solved. However, there's one thing I forgot to mention in my initial problem description: for years, I've been running iqconnect.exe via wine from linux, which I am quite sure it gives some (if not quite a) performance penalty.

So, after all, I'm quite sure right now that those two or three "spikes" would be handled correctly by iqconnect.exe , should it be natively run from within a Windows machine. Within the next few days, I will set up a Windows Sever 2016 instance in order to try whether this is the case.

Other than that, we're pretty satisfied with our performance right now.

thanks for your time and support !

DTN_Gary_Stephen
-DTN Evangelist-
Posts: 141
Joined: Jul 3, 2019


Posted: Jun 24, 2020 06:51 AM          Msg. 8 of 8
Thanks for the update! IQFeed is a native Windows program, and I can't offer a lot of advice on how to run it under Linux or any non-Windows OS. You might also look for any Wine errors or log messages at the times you're having these spikes; in some cases that can clue us in to what's causing it.

Sincerely,
Gary Stephen
DTN IQFeed Implementation Support Specialist
 

 

Time: Tue September 29, 2020 8:22 AM CFBB v1.2.0 15 ms.
© AderSoftware 2002-2003