Join the 80,000 other DTN customers who enjoy the fastest, most reliable data available. There is no better value than DTN!

(Move your cursor to this area to pause scrolling)




"There is no doubt that IQFeed is the best data provider. I am very satisfied with your services. And IQFeed is the only one that I would recommend to my friends. Now, most of them are using your product in China." - Comment from Zhezhe
"Thank God for your Data Feed as the only Zippers I see are on my pants (LOL), and no more 200 pip spikes to mess up charts." - Comment from Spiro via Email
"The people at Nirvana have very nice things to say about your company and I can see why! Price and service is a potent combination." - Comment from Ed
"I am keeping IQFeed, much better reliabilty than *******. I may refer a few other people in the office to switch as well." - Comment from Don
"Thanks for the great product and support. During this week of high volume trading, my QuoteTracker + IQ Feed setup never missed a beat. Also, thanks for your swiftness in responding to data issues. I was on ******* for a few years before I made the switch over early this year, and wish I had done it a long time ago." - Comment from Ken
"Thanks for all of your help. Great customer service deserves to be recognized which one the reasons I've been a customer of DTN for over 10 years!" - Comment from Stuart
"I am very pleased with the DTNIQ system for quotes and news." - Comment from Larry
"I used to have *******, but they are way more money for the same thing. I have had no probs with data from DTN since switching over." - Comment from Public Forum Post
"Awesome response, as usual. It is a sincere and refreshing pleasure to do business with DTN, compared to your competition." - Comment from Ryan
"I was with ******* for 4 years at $230 a month, this is a huge savings for me, GOD BLESS YOU PEOPLE," - Comment from T.S. via Email
Home  Search  Register  Login  Recent Posts
Information on DTN's Industries:
DTN Oil & Gas | DTN Trading | DTN Agriculture | DTN Weather
Follow DTNMarkets on Twitter
DTN.IQ/IQFeed on Twitter
DTN News and Analysis on Twitter
Viewing User Profile for: mac
About Contact
Joined: Apr 6, 2017 08:50 PM
Last Post: May 20, 2023 10:17 AM
Last Visit: May 21, 2023 05:49 PM
Website:  
Location:
Occupation:
Interests:
Email: chapinmark@gmail.com
AIM:
ICQ:
MSN IM:
Yahoo IM:
Post Statistics
mac has contributed to 25 posts out of 21185 total posts (0.12%) in 2,576 days (0.01 posts per day).

20 Most recent posts:
IQFeed API Questions » 16 second delay in receiving response to single bat HIT request May 20, 2023 10:17 AM (Total replies: 4)

Hi, I'm not sure if it's related to wine. I can't imagine is has anything to do with AWS. Hopefully I can test on windows next week.
MAC

IQFeed API Questions » 16 second delay in receiving response to single bat HIT request May 18, 2023 05:49 PM (Total replies: 4)

Yes that would be fine. I'll send an email to support.
MAC

IQFeed API Questions » 16 second delay in receiving response to single bat HIT request May 12, 2023 08:00 AM (Total replies: 4)

I occasionally see long delays receiving a response from a HIT 60 second request. I'm asking for the most recent interval bar (one bar). My requests are sent at 100-700 ms after the minute boundary. I send 35 symbol requests on 7 sockets in this time. I am not rate limited, I receive no "too many requests" error. I repeat this request every minute. In a 12 hour period, I saw 21 requests delays. The delay is always 15-16 seconds.

I'm running on linux/wine in AWS.

FROM CLIENT	LookupRequest	88	2520	2023-05-12 11:47:00	HIT,UNP,60,20230512 074600,20230512 074600,,,,,,,,0
FROM CLIENT	LookupRequest	304	2521	2023-05-12 11:47:00	HIT,@YMM23,60,20230512 074600,20230512 074600,,,,,,,,0
FROM CLIENT	LookupRequest	296	2523	2023-05-12 11:47:00	HIT,UNH,60,20230512 074600,20230512 074600,,,,,,,,0
TO CLIENT	LookupData	360	2517	2023-05-12 11:47:00	LH,2023-05-12 07:47:00,57.7800,57.7500,57.7700,57.7500,14252,5167,0,
TO CLIENT	LookupRequest	360	2517	2023-05-12 11:47:00	!ENDMSG!,
TO CLIENT	LookupData	360	2517	2023-05-12 11:47:00	!ENDMSG!,
TO CLIENT	LookupError	296	2523	2023-05-12 11:47:00	E,!NO_DATA!,,
!ENDMSG!,
TO CLIENT	LookupError	88	2520	2023-05-12 11:47:00	E,!NO_DATA!,,
!ENDMSG!,
TO CLIENT	LookupData	296	2523	2023-05-12 11:47:00	E,!NO_DATA!,,
!ENDMSG!,
TO CLIENT	LookupData	88	2520	2023-05-12 11:47:00	E,!NO_DATA!,,
!ENDMSG!,
TO CLIENT	LookupData	288	2519	2023-05-12 11:47:00	LH,2023-05-12 07:47:00,176.0600,176.0000,176.0200,176.0000,1531419,7866,0,
TO CLIENT	LookupRequest	288	2519	2023-05-12 11:47:00	!ENDMSG!,
TO CLIENT	LookupData	288	2519	2023-05-12 11:47:00	!ENDMSG!,


Response 15-16 seconds later


TO CLIENT	LookupData	304	2521	2023-05-12 11:47:15	LH,2023-05-12 07:47:00,33492.00,33489.00,33491.00,33489.00,24641,20,0,
TO CLIENT	LookupRequest	304	2521	2023-05-12 11:47:15	!ENDMSG!,
TO CLIENT	LookupData	304	2521	2023-05-12 11:47:15	!ENDMSG!,

MAC

IQFeed API Questions » Only send updates for dynamic fields subscribed May 8, 2023 02:18 PM (Total replies: 1)

If I set,

S,SELECT UPDATE FIELDS,Bid,Ask,Message Contents

and w@ESM23

Q,@ESM23,4152.75,4153.00,b,
Q,@ESM23,4152.50,4153.00,b,
Q,@ESM23,4152.50,4152.75,a,
Q,@ESM23,4152.50,4153.00,a,
Q,@ESM23,4152.75,4153.00,b,
Q,@ESM23,4152.50,4153.00,b,
Q,@ESM23,4152.75,4153.00,b,
Q,@ESM23,4152.50,4153.00,b,
Q,@ESM23,4152.75,4153.00,b,
Q,@ESM23,4152.50,4153.00,b,

I see only bid or ask price updates. However, when watching equities, I get mostly duplicate data. These are updates to Size and Market Data Center which are not part of my update fields. Is it possible to filter out these updates in iqconnect so I don't need to process duplicate data. It makes sense to get updates if market data center updates but not size if I'm not interested in this field.

wAAPL

Q,AAPL,173.5900,173.6000,ba,
Q,AAPL,173.5900,173.6000,ba,
Q,AAPL,173.5900,173.6000,ba,
Q,AAPL,173.5900,173.6000,ba,
Q,AAPL,173.5900,173.6000,ba,
Q,AAPL,173.5900,173.6000,ba,
Q,AAPL,173.5900,173.6000,ba,
Q,AAPL,173.5900,173.6000,ba,
Q,AAPL,173.5900,173.6000,ba,
Q,AAPL,173.5900,173.6000,ba,
Q,AAPL,173.5900,173.6000,ba,
Q,AAPL,173.5900,173.6000,ba,


MAC
Edited by mac on May 8, 2023 at 03:40 PM
Edited by mac on May 8, 2023 at 03:41 PM

IQFeed API Questions » Internal socket _IQIF_SBL1 hangs under wine May 8, 2023 01:21 PM (Total replies: 3)

The above approach to closing the socket is not working, iqconnect (or wine) is holding the internal iqconnect 5009 port open even after I disconnect my app.

This is causing me to repeatedly have to restart iqconnect (many times per week). I am currently developing an app to generate my own bars from level 1 data. Once I have this running I will stop connecting to the derivative port in production.

If I can create a docker file that reproduces this issue, could IQ support run this and see if it is reproducible on your end? I only see a problem under heavy market volume, so you might need to run for a few days before the internal port hangs.
MAC

IQFeed API Questions » Internal socket _IQIF_SBL1 hangs under wine Apr 28, 2023 09:44 AM (Total replies: 3)

I'm using the dotnet client so graciously provided here https://github.com/mathpaquette/IQFeed.CSharpApiClient

In this client, I see that the sockets are not explicitly disconnected, they are just disposed. It seems that this results in a quick release of the sockets on the iqconnect side in normal circumstances. I believe this causes the socket to be held onto by iqconnect when it enters this queued state. In the queued state, where the IQ derivative process in iqconnect no longer reads from the internal L1 socket, will hold onto this internal socket even though I am no longer connected to the derivative port. Data just piles up until it crashes when there is no more memory left.

On a positive note, if I explicitly disconnect from the derivative socket before disposing, iqconnect will eventually (30-60 seconds later) release the internal socket to L1. I have been able to reconnect to only the derivative socket without restarting iqconnect. All my sockets that are connected to L1 directly work fine through this problem.

Any insight as to why the derivative process would stop reading from its internal L1 socket would be helpful. I've had to write a bunch of code to circumvent this issue.

I will say, I don't believe this is an issue on Windows, only while running with wine.
MAC

IQFeed API Questions » Internal socket _IQIF_SBL1 hangs under wine Apr 24, 2023 03:49 PM (Total replies: 3)

Hi,

While running iqconnect under wine, I see that the internal _IQIF_SBL1 socket from a Derivative stream hangs causing subscribed bars to stop and for the _IQIF_SBL1 socket to start reporting queued data. This typically occurs during heavy market activity around open. However, I have seen it happen randomly through the day. I'm subscribing to 12 liquid equities and 23 liquid futures contracts. If the KBQueue reaches 10MB, I close all connected and open new ones. All sockets close except for the _IQIF_SBL1_***. These just keep piling up queued data until I restart iqconnect or eventually run out of memory. I'm running this in a docker container. I'm happy to share the Dockerfile if that helps.

I've had this issue for years now. My solution was to just kill the iqconnect process and restart it. This has become an issue for me and I can no longer afford to have interruptions in iqconnect.

In this first section of logs, you can see that the _IQIF_SBL1_*** connection stops delivering data(sent). It just piles up (kbqueued) I see no logged errors prior to this. I'm not sure what the level of logging is set to. If you can tell that debugging is not set, please let me know how to set this in a registry in 6.2 (64bit). I don't have easy access to the Diagnostics app

TO CLIENT Admin 404 4 2023-04-24 13:42:21 S,CLIENTSTATS,3,37,_IQIF_SBLOOKUP_IQMarketDataDeriv-2023-04-23T20:33:41.3079511-06:00,20230424 023332,,,2.11,8.42,0.00,
S,CLIENTSTATS,0,1,,20230423 025714,,,0.00,16810.25,0.00,
S,CLIENTSTATS,0,4,TickIngestAdmin2023-04-22T21:06:23.6679584-06:00,20230423 030623,,,0.10,104482.72,0.00,
S,CLIENTSTATS,0,5,IQMarketDataAdmin-2023-04-23T20:33:41.3085897-06:00,20230424 023332,,,0.11,33988.28,0.00,
S,CLIENTSTATS,1,9,_IQIF_SBL1_IQMarketDataDeriv-2023-04-23T20:33:41.3079511-06:00,20230424 023332,36,0,2.60,1009056.63,0.00,
S,CLIENTSTATS,1,7,IQMarketDataL1-2023-04-23T20:33:41.3091165-06:00,20230424 023331,36,0,1.32,35760.50,0.00,
S,CLIENTSTATS,1,5,TickIngestL12023-04-22T21:06:23.6683403-06:00,20230423 030623,36,0,29.16,1030698.07,0.00,

TO CLIENT Admin 404 5 2023-04-24 13:42:21 S,CLIENTSTATS,3,37,_IQIF_SBLOOKUP_IQMarketDataDeriv-2023-04-23T20:33:41.3079511-06:00,20230424 023332,,,2.11,8.42,0.00,
S,CLIENTSTATS,0,1,,20230423 025714,,,0.00,16810.25,0.00,
S,CLIENTSTATS,0,4,TickIngestAdmin2023-04-22T21:06:23.6679584-06:00,20230423 030623,,,0.10,104482.72,0.00,
S,CLIENTSTATS,0,5,IQMarketDataAdmin-2023-04-23T20:33:41.3085897-06:00,20230424 023332,,,0.11,33988.28,0.00,
S,CLIENTSTATS,1,9,_IQIF_SBL1_IQMarketDataDeriv-2023-04-23T20:33:41.3079511-06:00,20230424 023332,36,0,2.60,1009056.63,0.00,
S,CLIENTSTATS,1,7,IQMarketDataL1-2023-04-23T20:33:41.3091165-06:00,20230424 023331,36,0,1.32,35760.50,0.00,
S,CLIENTSTATS,1,5,TickIngestL12023-04-22T21:06:23.6683403-06:00,20230423 030623,36,0,29.16,1030698.07,0.00,

TO CLIENT Admin 404 1 2023-04-24 13:42:21 S,STATS,12.36.218.206,60004,500,36,7,0,0,1,Apr 23 2:57AM,Apr 24 09:42AM,Connected,6.2.0.25,465642,264332.45,53.47,2.11,2425889.77,498.33,19.39,

TO CLIENT Admin 404 4 2023-04-24 13:42:21 S,STATS,12.36.218.206,60004,500,36,7,0,0,1,Apr 23 2:57AM,Apr 24 09:42AM,Connected,6.2.0.25,465642,264332.45,53.47,2.11,2425889.77,498.33,19.39,

TO CLIENT Admin 404 5 2023-04-24 13:42:21 S,STATS,12.36.218.206,60004,500,36,7,0,0,1,Apr 23 2:57AM,Apr 24 09:42AM,Connected,6.2.0.25,465642,264332.45,53.47,2.11,2425889.77,498.33,19.39,

TO CLIENT Admin 404 4 2023-04-24 13:42:22 S,CLIENTSTATS,3,37,_IQIF_SBLOOKUP_IQMarketDataDeriv-2023-04-23T20:33:41.3079511-06:00,20230424 023332,,,2.11,8.42,0.00,
S,CLIENTSTATS,0,1,,20230423 025714,,,0.00,16810.39,0.00,
S,CLIENTSTATS,0,4,TickIngestAdmin2023-04-22T21:06:23.6679584-06:00,20230423 030623,,,0.10,104483.58,0.00,
S,CLIENTSTATS,0,5,IQMarketDataAdmin-2023-04-23T20:33:41.3085897-06:00,20230424 023332,,,0.11,33989.13,0.00,
S,CLIENTSTATS,1,9,_IQIF_SBL1_IQMarketDataDeriv-2023-04-23T20:33:41.3079511-06:00,20230424 023332,36,0,2.60,1009167.23,150.54,
S,CLIENTSTATS,1,7,IQMarketDataL1-2023-04-23T20:33:41.3091165-06:00,20230424 023331,36,0,1.32,35775.04,0.00,
S,CLIENTSTATS,1,5,TickIngestL12023-04-22T21:06:23.6683403-06:00,20230423 030623,36,0,29.16,1030935.49,0.00,

TO CLIENT Admin 404 5 2023-04-24 13:42:22 S,CLIENTSTATS,3,37,_IQIF_SBLOOKUP_IQMarketDataDeriv-2023-04-23T20:33:41.3079511-06:00,20230424 023332,,,2.11,8.42,0.00,
S,CLIENTSTATS,0,1,,20230423 025714,,,0.00,16810.39,0.00,
S,CLIENTSTATS,0,4,TickIngestAdmin2023-04-22T21:06:23.6679584-06:00,20230423 030623,,,0.10,104483.58,0.00,
S,CLIENTSTATS,0,5,IQMarketDataAdmin-2023-04-23T20:33:41.3085897-06:00,20230424 023332,,,0.11,33989.13,0.00,
S,CLIENTSTATS,1,9,_IQIF_SBL1_IQMarketDataDeriv-2023-04-23T20:33:41.3079511-06:00,20230424 023332,36,0,2.60,1009167.23,150.54,
S,CLIENTSTATS,1,7,IQMarketDataL1-2023-04-23T20:33:41.3091165-06:00,20230424 023331,36,0,1.32,35775.04,0.00,
S,CLIENTSTATS,1,5,TickIngestL12023-04-22T21:06:23.6683403-06:00,20230423 030623,36,0,29.16,1030935.49,0.00,

TO CLIENT Admin 404 1 2023-04-24 13:42:22 S,STATS,12.36.218.206,60004,500,36,7,0,0,1,Apr 23 2:57AM,Apr 24 09:42AM,Connected,6.2.0.25,465642,264390.37,57.93,2.11,2426254.17,364.40,19.39,

TO CLIENT Admin 404 4 2023-04-24 13:42:22 S,STATS,12.36.218.206,60004,500,36,7,0,0,1,Apr 23 2:57AM,Apr 24 09:42AM,Connected,6.2.0.25,465642,264390.37,57.93,2.11,2426254.17,364.40,19.39,

TO CLIENT Admin 404 5 2023-04-24 13:42:22 S,STATS,12.36.218.206,60004,500,36,7,0,0,1,Apr 23 2:57AM,Apr 24 09:42AM,Connected,6.2.0.25,465642,264390.37,57.93,2.11,2426254.17,364.40,19.39,

TO CLIENT Admin 404 4 2023-04-24 13:42:23 S,CLIENTSTATS,3,37,_IQIF_SBLOOKUP_IQMarketDataDeriv-2023-04-23T20:33:41.3079511-06:00,20230424 023332,,,2.11,8.42,0.00,
S,CLIENTSTATS,0,1,,20230423 025714,,,0.00,16810.53,0.00,
S,CLIENTSTATS,0,4,TickIngestAdmin2023-04-22T21:06:23.6679584-06:00,20230423 030623,,,0.10,104484.43,0.00,
S,CLIENTSTATS,0,5,IQMarketDataAdmin-2023-04-23T20:33:41.3085897-06:00,20230424 023332,,,0.11,33989.99,0.00,
S,CLIENTSTATS,1,9,_IQIF_SBL1_IQMarketDataDeriv-2023-04-23T20:33:41.3079511-06:00,20230424 023332,36,0,2.60,1009167.23,362.70,
S,CLIENTSTATS,1,7,IQMarketDataL1-2023-04-23T20:33:41.3091165-06:00,20230424 023331,36,0,1.32,35792.81,0.00,
S,CLIENTSTATS,1,5,TickIngestL12023-04-22T21:06:23.6683403-06:00,20230423 030623,36,0,29.16,1031126.10,0.00,

TO CLIENT Admin 404 5 2023-04-24 13:42:23 S,CLIENTSTATS,3,37,_IQIF_SBLOOKUP_IQMarketDataDeriv-2023-04-23T20:33:41.3079511-06:00,20230424 023332,,,2.11,8.42,0.00,
S,CLIENTSTATS,0,1,,20230423 025714,,,0.00,16810.53,0.00,
S,CLIENTSTATS,0,4,TickIngestAdmin2023-04-22T21:06:23.6679584-06:00,20230423 030623,,,0.10,104484.43,0.00,
S,CLIENTSTATS,0,5,IQMarketDataAdmin-2023-04-23T20:33:41.3085897-06:00,20230424 023332,,,0.11,33989.99,0.00,
S,CLIENTSTATS,1,9,_IQIF_SBL1_IQMarketDataDeriv-2023-04-23T20:33:41.3079511-06:00,20230424 023332,36,0,2.60,1009167.23,362.70,
S,CLIENTSTATS,1,7,IQMarketDataL1-2023-04-23T20:33:41.3091165-06:00,20230424 023331,36,0,1.32,35792.81,0.00,
S,CLIENTSTATS,1,5,TickIngestL12023-04-22T21:06:23.6683403-06:00,20230423 030623,36,0,29.16,1031126.10,0.00,

TO CLIENT Admin 404 1 2023-04-24 13:42:23 S,STATS,12.36.218.206,60004,500,36,7,0,0,1,Apr 23 2:57AM,Apr 24 09:42AM,Connected,6.2.0.25,465642,264440.68,50.31,2.11,2426464.41,210.23,19.39,

TO CLIENT Admin 404 4 2023-04-24 13:42:23 S,STATS,12.36.218.206,60004,500,36,7,0,0,1,Apr 23 2:57AM,Apr 24 09:42AM,Connected,6.2.0.25,465642,264440.68,50.31,2.11,2426464.41,210.23,19.39,

TO CLIENT Admin 404 5 2023-04-24 13:42:23 S,STATS,12.36.218.206,60004,500,36,7,0,0,1,Apr 23 2:57AM,Apr 24 09:42AM,Connected,6.2.0.25,465642,264440.68,50.31,2.11,2426464.41,210.23,19.39,



In this second section, you can see that some sockets disconnect while the _IQIF_SBL1 sockets remain connected.


TO CLIENT Admin 404 4 2023-04-24 13:43:04 S,CLIENTSTATS,3,37,_IQIF_SBLOOKUP_IQMarketDataDeriv-2023-04-23T20:33:41.3079511-06:00,20230424 023332,,,2.11,8.42,0.00,
S,CLIENTSTATS,0,1,,20230423 025714,,,0.00,16816.30,0.00,
S,CLIENTSTATS,0,4,TickIngestAdmin2023-04-22T21:06:23.6679584-06:00,20230423 030623,,,0.10,104519.58,0.00,
S,CLIENTSTATS,0,5,IQMarketDataAdmin-2023-04-23T20:33:41.3085897-06:00,20230424 023332,,,0.11,34025.14,0.00,
S,CLIENTSTATS,1,9,_IQIF_SBL1_IQMarketDataDeriv-2023-04-23T20:33:41.3079511-06:00,20230424 023332,36,0,2.60,1009167.23,9857.79,
S,CLIENTSTATS,1,7,IQMarketDataL1-2023-04-23T20:33:41.3091165-06:00,20230424 023331,36,0,1.32,36454.37,0.00,
S,CLIENTSTATS,1,5,TickIngestL12023-04-22T21:06:23.6683403-06:00,20230423 030623,36,0,29.16,1039695.57,0.00,

TO CLIENT Admin 404 5 2023-04-24 13:43:04 S,CLIENTSTATS,3,37,_IQIF_SBLOOKUP_IQMarketDataDeriv-2023-04-23T20:33:41.3079511-06:00,20230424 023332,,,2.11,8.42,0.00,
S,CLIENTSTATS,0,1,,20230423 025714,,,0.00,16816.30,0.00,
S,CLIENTSTATS,0,4,TickIngestAdmin2023-04-22T21:06:23.6679584-06:00,20230423 030623,,,0.10,104519.58,0.00,
S,CLIENTSTATS,0,5,IQMarketDataAdmin-2023-04-23T20:33:41.3085897-06:00,20230424 023332,,,0.11,34025.14,0.00,
S,CLIENTSTATS,1,9,_IQIF_SBL1_IQMarketDataDeriv-2023-04-23T20:33:41.3079511-06:00,20230424 023332,36,0,2.60,1009167.23,9857.79,
S,CLIENTSTATS,1,7,IQMarketDataL1-2023-04-23T20:33:41.3091165-06:00,20230424 023331,36,0,1.32,36454.37,0.00,
S,CLIENTSTATS,1,5,TickIngestL12023-04-22T21:06:23.6683403-06:00,20230423 030623,36,0,29.16,1039695.57,0.00,

TO CLIENT Admin 404 1 2023-04-24 13:43:04 S,STATS,12.36.218.206,60004,500,36,7,0,0,1,Apr 23 2:57AM,Apr 24 09:43AM,Connected,6.2.0.25,465642,266633.01,86.73,2.13,2435771.50,394.70,19.46,

TO CLIENT Admin 404 4 2023-04-24 13:43:04 S,STATS,12.36.218.206,60004,500,36,7,0,0,1,Apr 23 2:57AM,Apr 24 09:43AM,Connected,6.2.0.25,465642,266633.01,86.73,2.13,2435771.50,394.70,19.46,

TO CLIENT Admin 404 5 2023-04-24 13:43:04 S,STATS,12.36.218.206,60004,500,36,7,0,0,1,Apr 23 2:57AM,Apr 24 09:43AM,Connected,6.2.0.25,465642,266633.01,86.73,2.13,2435771.50,394.70,19.46,

TO CLIENT Admin 404 4 2023-04-24 13:43:05 S,CLIENTSTATS,3,37,_IQIF_SBLOOKUP_IQMarketDataDeriv-2023-04-23T20:33:41.3079511-06:00,20230424 023332,,,2.11,8.42,0.00,
S,CLIENTSTATS,0,1,,20230423 025714,,,0.00,16816.44,0.00,
S,CLIENTSTATS,0,4,TickIngestAdmin2023-04-22T21:06:23.6679584-06:00,20230423 030623,,,0.10,104520.44,0.00,
S,CLIENTSTATS,0,5,IQMarketDataAdmin-2023-04-23T20:33:41.3085897-06:00,20230424 023332,,,0.11,34026.00,0.00,
S,CLIENTSTATS,1,9,_IQIF_SBL1_IQMarketDataDeriv-2023-04-23T20:33:41.3079511-06:00,20230424 023332,36,0,2.60,1009167.23,10052.97,
S,CLIENTSTATS,1,7,IQMarketDataL1-2023-04-23T20:33:41.3091165-06:00,20230424 023331,36,0,1.32,36467.51,0.00,
S,CLIENTSTATS,1,5,TickIngestL12023-04-22T21:06:23.6683403-06:00,20230423 030623,36,0,29.16,1039871.30,0.00,

TO CLIENT Admin 404 5 2023-04-24 13:43:05 S,CLIENTSTATS,3,37,_IQIF_SBLOOKUP_IQMarketDataDeriv-2023-04-23T20:33:41.3079511-06:00,20230424 023332,,,2.11,8.42,0.00,
S,CLIENTSTATS,0,1,,20230423 025714,,,0.00,16816.44,0.00,
S,CLIENTSTATS,0,4,TickIngestAdmin2023-04-22T21:06:23.6679584-06:00,20230423 030623,,,0.10,104520.44,0.00,
S,CLIENTSTATS,0,5,IQMarketDataAdmin-2023-04-23T20:33:41.3085897-06:00,20230424 023332,,,0.11,34026.00,0.00,
S,CLIENTSTATS,1,9,_IQIF_SBL1_IQMarketDataDeriv-2023-04-23T20:33:41.3079511-06:00,20230424 023332,36,0,2.60,1009167.23,10052.97,
S,CLIENTSTATS,1,7,IQMarketDataL1-2023-04-23T20:33:41.3091165-06:00,20230424 023331,36,0,1.32,36467.51,0.00,
S,CLIENTSTATS,1,5,TickIngestL12023-04-22T21:06:23.6683403-06:00,20230423 030623,36,0,29.16,1039871.30,0.00,

TO CLIENT Admin 404 1 2023-04-24 13:43:05 S,STATS,12.36.218.206,60004,500,36,7,0,0,1,Apr 23 2:57AM,Apr 24 09:43AM,Connected,6.2.0.25,465642,266678.29,45.29,2.13,2435962.22,190.71,19.46,

TO CLIENT Admin 404 4 2023-04-24 13:43:05 S,STATS,12.36.218.206,60004,500,36,7,0,0,1,Apr 23 2:57AM,Apr 24 09:43AM,Connected,6.2.0.25,465642,266678.29,45.29,2.13,2435962.22,190.71,19.46,

TO CLIENT Admin 404 5 2023-04-24 13:43:05 S,STATS,12.36.218.206,60004,500,36,7,0,0,1,Apr 23 2:57AM,Apr 24 09:43AM,Connected,6.2.0.25,465642,266678.29,45.29,2.13,2435962.22,190.71,19.46,

This is where is have closed and disposed of all sockets connection (except #1). I have also started reconnecting. As you can see, the _IQIF_SBL1_IQMarketDataDeriv*** connections are not closed.
STATUS Information 404 7 2023-04-24 13:43:05 LEVEL1 SOCKET CLOSED - IQMarketDataL1-2023-04-23T20:33:41.3091165-06:00
STATUS Information 404 5 2023-04-24 13:43:05 ADMIN SOCKET CLOSED - IQMarketDataAdmin-2023-04-23T20:33:41.3085897-06:00
STATUS Information 404 11 2023-04-24 13:43:07 LEVEL1 SOCKET ACCEPTED -
STATUS Information 404 6 2023-04-24 13:43:07 ADMIN SOCKET ACCEPTED -
STATUS Information 404 5 2023-04-24 13:43:07 LEVEL1 SOCKET CLOSED - TickIngestL12023-04-22T21:06:23.6683403-06:00
STATUS Information 404 4 2023-04-24 13:43:07 ADMIN SOCKET CLOSED - TickIngestAdmin2023-04-22T21:06:23.6679584-06:00
STATUS Information 404 12 2023-04-24 13:43:08 LEVEL1 SOCKET ACCEPTED -
STATUS Information 404 11 2023-04-24 13:43:08 LEVEL1 SOCKET CLOSED -
STATUS Information 404 0 2023-04-24 13:43:08 LOOKUP SOCKET ACCEPTED 74 -
STATUS Information 404 7 2023-04-24 13:43:08 ADMIN SOCKET ACCEPTED -
STATUS Information 404 14 2023-04-24 13:43:09 LEVEL1 SOCKET ACCEPTED - iqconnect.exe
STATUS Information 404 12 2023-04-24 13:43:09 LEVEL1 SOCKET CLOSED -
STATUS Information 404 14 2023-04-24 13:43:09 LEVEL1 SOCKET CLOSED - _IQIF_SBL1
TO CLIENT Admin 404 6 2023-04-24 13:43:10 S,CURRENT PROTOCOL,6.2,

FROM CLIENT Admin 404 6 2023-04-24 13:43:10 S,SET PROTOCOL,6.2
TO CLIENT Admin 404 7 2023-04-24 13:43:10 S,CURRENT PROTOCOL,6.2,

FROM CLIENT Admin 404 7 2023-04-24 13:43:10 S,SET PROTOCOL,6.2
FROM CLIENT Admin 404 7 2023-04-24 13:43:10 S,SET CLIENT NAME,TickIngestAdmin2023-04-24T07:43:06.2700049-06:00
FROM CLIENT Admin 404 7 2023-04-24 13:43:10 S,CLIENTSTATS ON
TO CLIENT Admin 404 7 2023-04-24 13:43:10 S,CLIENTSTATS,3,37,_IQIF_SBLOOKUP_IQMarketDataDeriv-2023-04-23T20:33:41.3079511-06:00,20230424 023332,,,2.11,8.42,0.00,
S,CLIENTSTATS,3,74,_IQIF_SBLOOKUP,20230424 134308,,,0.05,0.02,0.00,
S,CLIENTSTATS,0,1,,20230423 025714,,,0.00,16816.58,0.00,
S,CLIENTSTATS,0,6,,20230424 134307,,,0.02,0.02,0.00,
S,CLIENTSTATS,0,7,TickIngestAdmin2023-04-24T07:43:06.2700049-06:00,20230424 134308,,,0.10,0.02,0.00,
S,CLIENTSTATS,1,14,_IQIF_SBL1,20230424 134308,0,0,1.57,1.05,0.00,
S,CLIENTSTATS,1,9,_IQIF_SBL1_IQMarketDataDeriv-2023-04-23T20:33:41.3079511-06:00,20230424 023332,36,0,2.60,1009167.23,10968.86,
S,CLIENTSTATS,1,12,TickIngestL12023-04-24T07:43:06.2701898-06:00,20230424 134307,36,0,2.98,705.04,0.00,
S,CLIENTSTATS,1,11,,20230424 134306,0,0,0.34,0.87,0.00,

TO CLIENT Admin 404 1 2023-04-24 13:43:10 S,STATS,12.36.218.206,60004,500,36,9,0,0,1,Apr 23 2:57AM,Apr 24 09:43AM,Connected,6.2.0.25,465642,266895.52,43.45,2.13,2436810.04,169.56,19.47,

TO CLIENT Admin 404 6 2023-04-24 13:43:10 S,STATS,12.36.218.206,60004,500,36,9,0,0,1,Apr 23 2:57AM,Apr 24 09:43AM,Connected,6.2.0.25,465642,266895.52,43.45,2.13,2436810.04,169.56,19.47,

TO CLIENT Admin 404 7 2023-04-24 13:43:10 S,STATS,12.36.218.206,60004,500,36,9,0,0,1,Apr 23 2:57AM,Apr 24 09:43AM,Connected,6.2.0.25,465642,266895.52,43.45,2.13,2436810.04,169.56,19.47,

STATUS Information 404 12 2023-04-24 13:43:10 LEVEL1 SOCKET CLOSED - TickIngestL12023-04-24T07:43:06.2701898-06:00
STATUS Information 404 7 2023-04-24 13:43:10 ADMIN SOCKET CLOSED - TickIngestAdmin2023-04-24T07:43:06.2700049-06:00
STATUS Information 404 17 2023-04-24 13:43:10 LEVEL1 SOCKET ACCEPTED -
STATUS Information 404 8 2023-04-24 13:43:11 ADMIN SOCKET ACCEPTED -
STATUS Information 404 17 2023-04-24 13:43:11 LEVEL1 SOCKET CLOSED -
TO CLIENT Admin 404 8 2023-04-24 13:43:11 S,CURRENT PROTOCOL,6.2,

FROM CLIENT Admin 404 8 2023-04-24 13:43:11 S,SET PROTOCOL,6.2
FROM CLIENT Admin 404 8 2023-04-24 13:43:11 S,SET CLIENT NAME,TickIngestAdmin2023-04-24T07:43:10.1673900-06:00
FROM CLIENT Admin 404 8 2023-04-24 13:43:11 S,CLIENTSTATS ON
TO CLIENT Admin 404 8 2023-04-24 13:43:11 S,CLIENTSTATS,3,37,_IQIF_SBLOOKUP_IQMarketDataDeriv-2023-04-23T20:33:41.3079511-06:00,20230424 023332,,,2.11,8.42,0.00,
S,CLIENTSTATS,3,74,_IQIF_SBLOOKUP,20230424 134308,,,0.05,0.02,0.00,
S,CLIENTSTATS,0,1,,20230423 025714,,,0.00,16816.72,0.00,
S,CLIENTSTATS,0,6,,20230424 134307,,,0.02,0.17,0.00,
S,CLIENTSTATS,0,8,TickIngestAdmin2023-04-24T07:43:10.1673900-06:00,20230424 134311,,,0.10,0.02,0.00,
S,CLIENTSTATS,1,14,_IQIF_SBL1,20230424 134308,0,0,1.57,1.07,0.00,
S,CLIENTSTATS,1,17,TickIngestL12023-04-24T07:43:10.1676316-06:00,20230424 134310,36,0,2.98,94.44,0.00,
S,CLIENTSTATS,1,9,_IQIF_SBL1_IQMarketDataDeriv-2023-04-23T20:33:41.3079511-06:00,20230424 023332,36,0,2.60,1009167.23,11084.92,
S,CLIENTSTATS,1,11,,20230424 134306,0,0,0.34,0.89,0.00,
MAC

IQFeed Datafeed Wish List » API to request historical option prices and symbols. Apr 26, 2021 11:58 AM (Total replies: 6)

Hi,

Any update on when the historical options file will be updated?

Thanks,
MAC

IQFeed Developer Support » Corrupted L1 data-in recent past on multiple products at 'random' times Feb 21, 2019 08:54 PM (Total replies: 28)

I just upgraded to IQConnect v6 and the bid is no longer frozen. I am still using 5.2 protocol. (see attached file)

@aQuant, which version of IQConnect are you running? If v6, I will assume the frozen bid I saw 2am on Feb 21 was version related.

I will continue to monitor with IQConnect v6.

I hope this finding will shed some light on the issue.
MAC

IQFeed Developer Support » Corrupted L1 data-in recent past on multiple products at 'random' times Feb 21, 2019 08:33 PM (Total replies: 28)

@quickTick,

It's interesting that you don't see this. The attached file is the raw text data from the socket. I've attached the full response to the HTT request run at 2018-02-21 21:20:00 ET. It's a bit longer but still shows the frozen bid.

I should note, I have yet to upgrade to IQConnect v6. I will do that now and see if it fixes the problem.

Below is the HTT command I issued.

HTT,@NQH19,20190221 015000,20190221 022000,0005000,000000,235959,1,REQ_HST_TCK0000000,0000500

Same result without including BeginFilterTime and EndFilterTime in the request

HTT,@NQH19,20190221 015000,20190221 022000,0005000,,,1,REQ_HST_TCK0000000,0000500
MAC

IQFeed Developer Support » Corrupted L1 data-in recent past on multiple products at 'random' times Feb 21, 2019 07:20 AM (Total replies: 28)

Attached is the HTT response for @NQH19.

Note that the bids do not change for 17 minutes.
MAC

IQFeed Developer Support » Corrupted L1 data-in recent past on multiple products at 'random' times Feb 21, 2019 06:18 AM (Total replies: 28)

aQuant,

I spoke too soon. I see the bid frozen from 2018-02-21 02:00:00 to 02:17:00 ET on all the products I'm monitoring except JY. It looks like my simple stats are able to catch this.

I haven't coded up the HT* command (request tick history). Will this show the same behavior? If so, surely IQFeed won't be able to refute the problem.

I will call them this morning.
MAC

IQFeed Developer Support » Corrupted L1 data-in recent past on multiple products at 'random' times Feb 20, 2019 08:14 PM (Total replies: 28)

aQuant,

I am now monitoring the following CME symbols (AD,BP,CD,JY,EC,NG,ES).

I didn't see anything out of the ordinary for trader date 2018-02-20. However, my monitoring is only gather stats and not saving the raw messages. There is a possibility I have a bug in my stats.

If it's not too much to ask, could you post here when you see one of these symbols get stuck or cross? I can then verify with my monitoring.

If I can verify, we can all push for a solution.

Thanks in advance.
MAC

IQFeed Developer Support » Latency of Q messages Feb 19, 2019 08:17 AM (Total replies: 3)

Thanks for the reply.

I have implemented the latest .net core 3.0 System.IO.Pipelines library. It "queues" and "dequeues" on separate threads while minimizing overallocation. With the help of the SequenceReader, you can access the buffer directly without having to copy to a string and then split. I have yet to test it with my mock server. I'll let you know if it performs better.

I also need to monitor KBQueued in code so I can log these infrequent latency events. Due to some logging changes, I was only able to see these events back in 2017. One of which coincided with an ECB announcement.

I will continue to monitor.
MAC

IQFeed Developer Support » Corrupted L1 data-in recent past on multiple products at 'random' times Feb 19, 2019 08:01 AM (Total replies: 28)

aQuant or quickTick,

Could you explain how you detect this? I want to start monitoring.

Is there a complete lack of Q messages arriving with 'b' or 'a' updates?

or

Are the Q messages coming in for 'b' or 'a' but the prices don't change?

I have disabled IQFeed from my production deployment and will run it in the background for monitoring only.

I was about to add NYMEX, COMEX, CBOT and EUREX to my subscriptions. I will wait until this has been resolved.
MAC

IQFeed Developer Support » Latency of Q messages Feb 11, 2019 08:00 PM (Total replies: 3)

Hi,

I've been trying to optimize my .net socket reader. First, I wrote my own server and pushed 1000 "Q," messages per second to help find any bottlenecks in my code. The mock server writes 1000 dummy Q messages in a loop. I've appended a high resolution time stamp to the messages (using interop GetSystemTimePreciseAsFileTime). I compare this time stamp on the reading side to see how fast I process messages. The writes of 1000 messages only takes a few milliseconds. I noticed that I was unable to keep up with this rate on the read side if I processed the message synchronously. I now push the messages to a BlockingCollection(ConcurrentQueue). Once I did this, my socket mostly reads one or two message per read.

I thought this would help reading the live feed. However, I still see many reads with numerous messages (>50). This is averaging 5, but sometime as many as 100+ messages are pulled in one read. Most of the time, these messages only differ in the Bid/Ask time stamp by a few milliseconds but sometimes are > 50ms (as high as 200 ms) from the first message to the last.

Is IQConnect batching the writes to the socket? Or are you batching the push from your servers? No matter what I do I cannot seem to reduce this.

I should note that I am collecting stats in an array and only dumping one line to the console every 10-15 second. I have timed the stat collection and display and I am confident that it is not skewing the results.

I need to sort this out not because I’m concerned about a few milliseconds but because I can get way behind (1-30 seconds!) in a fast market.

I am running on an AWS EC2 T2.Medium machine (2 cores).

My pings to the IQFeed server are constantly between 45-47 ms.

I am only watching CME products (@ES, @NQ, 5 major currencies)
MAC

IQFeed Developer Support » Corrupted L1 data-in recent past on multiple products at 'random' times Feb 7, 2019 11:38 PM (Total replies: 28)

Has this been resolved?
MAC

IQFeed Developer Support » Fast market probelm Jun 28, 2017 11:25 AM (Total replies: 3)

OK, my bad. I see 61,000 prints in the 8:17 minute.

That said, should a C# program be able to handle 1000+ prints per second? Is it possible?
MAC

IQFeed Developer Support » Fast market probelm Jun 28, 2017 10:43 AM (Total replies: 3)

I should note, the numbers quoted above (100 prints per second) were filtered prints. These were only logged if Bid or Ask price changed. I was not logging changes in size at the same price. It is likely much higher than 100 per second.

That said, I'm running a test right now that shows no latency issues with similar filtering. I'm getting fewer price changes since there isn't a fast market but only on the order of 30%-50%. I'm sampling every 10 seconds and see around 30-50 bid/ask price changes per second for ECU7. I am however subscribed to all 5 products. Average latencies are normal at around 75 milliseconds.

It's starting to look like my app can handle the throughput.

Any reported problems at IQFeed this morning?
MAC

IQFeed Developer Support » Fast market probelm Jun 28, 2017 09:44 AM (Total replies: 3)

I am relatively new to IQFeed and to socket programming in general. I’m coding in .Net. I’m basically doing the same socket reads as the example code, Level1Socket. I was naive to think I could keep up with the socket, I think.

I haven’t been monitoring the Admin socket so I don’t know how often I have queued data. I happened to be trading in a fast market at 08:17 - 08:18 ET June 28 when there was significant futures movement. I only watch 5 symbols; Globex currencies Euro, Japanese Yen, Australian Dollar, E-mini Nasdaq and AAPL. Euro currency had a very fast move at this time. I log level 1 update (bid/ask) latency during a trade. I was seeing 25-35 SECOND latencies.

After seeing this, I opened the Diagnostic program for market open at 9:30 ET to see if it reported any queued data. It showed none.

My machine clock is managed by a windows port of ntp.org software, so it is quite accurate (always within a few milliseconds of ntp time).

Is it possible for IQFeed to see if there were an abnormal amount of prints in this period relative to market open. I recorded around 6000 prints in the 08:18:00-08:18:59 minute for Euro Currency Sept 2017 (100 per second).

Unfortunately, having never seen this before, I didn’t spend too much effort optimizing my software. Every print, regardless of the product, fires an event with 5 subscribing class instances, one subscription per product. In the even handler, I check to see if the event args match the product I’m interest in. Maybe a better approach would be to open 5 different sockets, one per product. This could expand to 15 or 20 products in the future but not 100’s. This way I could read from the back of the socket? If a print is old, just throw it out. I just want the most recent info. If I had one socket per product, I could just grab the last update and throw the rest away. Also, this would enable me to fire an event with only one subscriber.

Also unfortunate, I don’t know where the bottleneck is.

Any thoughts?
MAC


Time: Wed April 24, 2024 9:09 PM CFBB v1.2.0 8 ms.
© AderSoftware 2002-2003