Join the 80,000 other DTN customers who enjoy the fastest, most reliable data available. There is no better value than DTN!

(Move your cursor to this area to pause scrolling)




"Interactive Brokers tick data was inconsistent, so I have switched to using DTN exclusively. It is great to no longer have to worry about my datafeed all day long." - Comment from Philippe
"Can I get another account from you? I am tired of ******* going down so often" - Comment from George
"Boy, probably spent a thousand hours trying to get ******* API to work right. And now two hours to have something running with IQFeed. Hmmm, guess I was pretty stupid to fight rather than switch all this time. And have gotten more customer service from you guys already than total from them… in five years." - Comment from Jim
"I'm satisfied with IQFeed. It's the most reliable and fastest quote feed I have ever used. Although I'm a resident in China, it's still very fast!" - Comment from Xiaofei
"I had always used ******* but for the past 2 weeks have been trying DTN IQFeed. Customer support has been extraordinary. They call just to make sure your problem hasn't recurred." - Comment from Public Forum
"I started a trial a few weeks back before the market went wild. DTN.IQ didn’t miss anything and beat my other provider. I decided to stay with you because of the great service through all the volatility." - Comment from Mike
"IQ feed is brilliant. The support is mind-bending. What service!" - Comment from Public Forum Post
"Everything is working great with the API. I love it." - Comment from Calvin
"You are much better than lawyers or the phone company because you answer the phone when I call! I just love your customer service." - Comment from Isreal
"I've been using IQFeed 4 in a multi-threaded situation for the last week or two on 2600 symbols or so with 100 simultaneous daily charts, and I have had 100% responsiveness." - Comment from Scott
Home  Search  Register  Login  Recent Posts

Information on DTN's Industries:
DTN Oil & Gas | DTN Trading | DTN Agriculture | DTN Weather
Follow DTNMarkets on Twitter
DTN.IQ/IQFeed on Twitter
DTN News and Analysis on Twitter
Viewing User Profile for: mkvalor
About Contact
Joined: Oct 6, 2020 09:26 PM
Last Post: Feb 23, 2024 01:58 AM
Last Visit: Feb 23, 2024 06:37 AM
Website:  
Location:
Occupation: Software Engineer
Interests: Guitar, Financial Market Speculation, Technology
Avatar:
Keep your tools sharp.
AIM:
ICQ:
MSN IM:
Yahoo IM:
Post Statistics
mkvalor has contributed to 24 posts out of 21185 total posts (0.11%) in 1,292 days (0.02 posts per day).

20 Most recent posts:
New IQFeed Forum » iqconnect Out of Memory Feb 23, 2024 01:58 AM (Total replies: 3)

I'm a user like yourself, but I've seen a few posts in the past dealing with memory issue problems.

One thing: do you happen to know if your app makes a separate connection to IQConnect.exe for each symbol? Especially using python, I could see this adding up to a lot of CPU and memory overhead. Even though this strategy can be more convenient (because each program only has to deal with data for a single symbol), it can be a lot more memory and CPU efficient to open multiple symbol watches (say, 50 at a time) on a single connection to IQConnect and then use logic in your applicatoin to send each line to the right place based on the ticker symbol at the beginning (such as to a separate data file, to a data stream queue like Kafka, or to some other logic handler to calculate moving averages, etc).

For example, you could use a python dict where the key is the IQF symbol name and the value is an open file handle for that symbol (if you don't find it in the dict, you create a new file and then add it there) -- then check the start of each line to look up the right file to write it to. This is more complex because of the logic I mentioned but also because you'd need to do something like read a config file per process with the list of symbols you want to watch per process. If you got fancy, you could have a single file with all symbols, one per line, and then start the program by adding command line params that tell it which line to start at in the symbols file and how many of them to read for that particular process.

Anyway -- enough advice based on guessing. If you're already reading multiple symbols per process, then it must be something else.



-Mark D. Valor
Edited by mkvalor on Feb 23, 2024 at 02:01 AM
Edited by mkvalor on Feb 23, 2024 at 02:03 AM

IQFeed API Questions » Connection refused Feb 10, 2024 07:29 PM (Total replies: 4)

I found a way to cover most of the edge cases. Recall that one may connect to several different ports on the local IQFeed exe server. One of those is the admin port. So firts, I try to connect to the admin port just as a test to see if IQFeed is already running. If that fails twice (after pausing between attempts), I call a shell script (a batch file on Windows) which runs IQFeed.exe. Then I try to connect 3 more times (after pauses). Since my rust code might looke like gobbledygook, I'll demonstrat this with pseudo code that resembles python (but is not actually python):

admin_conn_attempts = 0
conn_attempt_pause_seconds = 4
while True:
admin_conn = tcp_connect("127.0.0.1", 9300) #the admin socket
if admin_conn.is_connected():
break
else
sleep(conn_attempt_pause_seconds)
admin_conn_attempts += 1
if admin_conn_attempts == 2:
# run external program to start IQFeed.exe
run.program(IQFeed.exe)
else if admin_conn_attempts > 5:
print("ERROR: Unable to connect to IQFeed")
exit -1


If i get this far in the code and it didn't exit, that means I'm connected.

I'm aware IQFeed should just start automatically if it's not running but I'm running under Wine on Linux and I have not observed that behavior. I call a shell script to run it (instead of the executable itself) because I set some additional environment things up before running -- such as pinning IQFeed to a certain set of cores on my multi-core processor.


-Mark D. Valor
Edited by mkvalor on Feb 10, 2024 at 07:31 PM


Quote: Would it be possible to get an example command for "watching" only the two top depth levels for level 2 SPX option data?
Is it as simple as this?:
WPL,SPXoptionSymbol,2<CR><CR><LF>


I don't have a subscription for options. However, when I tried your example during off hours for a futures contract, the summary lines returned comprised only two levels of bid and two levels of ask, as expected.

T,20240210 17:58:29
S,CLEAR DEPTH,@ES#,B,
S,CLEAR DEPTH,@ES#,A,
7,@ES#,B,5041.50,51,6,2,16:59:02.811840,2024-02-09,
7,@ES#,B,5041.25,22,3,2,16:59:00.820439,2024-02-09,
7,@ES#,A,5042.00,9,3,2,16:59:58.043198,2024-02-09,
7,@ES#,A,5042.25,20,3,2,16:59:03.275683,2024-02-09,

Looks like you might need to subscribe to data or find someone who has options data to be completely sure about those instruments.

-mkvalor

-Mark D. Valor


Quote: If 'Most Recent Trade Conditions' contains HEX 40 (Closing Prints) and also 'Message Contents' contains 'C' or 'O' - field 'Last' contains exactly today's Close price.


This is really great info LeeFox. Thanks for sharing it. Do any of the IQFeed moderators have any comment on using this method?

-Mark D. Valor

Miscellaneous Messages » Support for Linux API for the datafeed? Oct 14, 2023 07:13 PM (Total replies: 33)

Glad your way is working for you, but I'm not sure what problems you encountered using Xvfb. I run Fedora with Wine and Xvfb and this configuration has been rock stable for me for nearly 3 years. So this comment is mostly for the benefit of others who might be dissuaded from trying Xvfb by your message above.

Once Wine and Xvfb are set up, you simply prepend the other commands you want to run in the virtual framebuffer with a couple of wrapper commands and be sure to pass in flags to the program which allow non-interactive gui operation. For example, to run the iqfeed installer:

$ xvfb-run wine $HOME/Downloads/iqfeed_client_6_2_0_25.exe /S


and then to connect to IQFeed such that your separate program passes in the password via the admin port shortly after:

$xvfb-run wine IQConnect.exe -product [PRODUCT_ID] -version [MADEUP_VERSION_STR] -login [LOGIN_ID]


Once you see your program running fine in terminal blocking mode, you can set up cron or systemd timer units to start and stop it like this in the background, so you don't need to keep a terminal window open.




-Mark D. Valor
Edited by mkvalor on Oct 14, 2023 at 07:14 PM
Edited by mkvalor on Oct 14, 2023 at 07:16 PM
Edited by mkvalor on Oct 14, 2023 at 07:18 PM


Great info; thanks Gary. Yes, I was talking about L2 Order ID in Q 3. Hmm, yes, I feel that data type indicates it would not be formatted with leading zeros.

I've definitely seen that, for CME ES, Tick ID typically repeats between trades.

I can certainly test pulling data at different times for CME futures to try to discover the daily(?) volume reset. Just thought I'd ask in case this was well known. If you learn the answer please do post it here.

-Mark D. Valor


Context: the point of my question is to figure out which of these fields is either fixed-length or at least reasonably unlikely to roll over to a higher number of digits or shrink to a lower number of digits across most trading days.

I have my IQFeed client set to start watching symbols shortly before market open, so for futures contract ES (for example) the first lines of my captured L1 feed will show values like this for tick id and total volume (respectively below):

Q,@ES#,271429934,...,276430,

and the L2 MBO feed for ES will show order ids like this (final summary line and first delete, add and update lines):

6,@ES#,6412236089210,...,
5,@ES#,6412236980317,...,
3,@ES#,6412236980495,...,
5,@ES#,6412236980188,...,
5,@ES#,6412236980184,...,
4,@ES#,6412236980185,...,

1) I imagine the L1 total volume resets to zero when ES off-hours trading starts at 6:00 pm ET the night before each trading day. Is that accurate?

1A) When it resets, does the field length shrink to the number of digits needed (such as "1" vs "000001")?

2) The L1 tick id, which is enormous in comparison to the daily total volume (since it includes bid/ask) doesn't look like it contains only a single day's L1 transactions. Does this only increase (starting from some fixed time in the past) or does it reset to zero (for example on the first trading session of each year)?

2A) If it resets, does the field length shrink to the number of digits needed (such as "1" vs "000000001")?

3) L2 field values are similarly orders of magnitude above the L1 tick id values (due to the much broader limit order activity). Does this only increase (starting from some fixed time in the past) or does it reset to zero (for example on the first trading session of each year)?

3A) If it resets, does the field length shrink to the number of digits needed (such as "1" vs "0000000000001")?

Thanks in advance.


-Mark D. Valor
Edited by mkvalor on Aug 21, 2023 at 10:51 PM


Well, I hope you will agree with me this is not clear from the docs, since the page for Fundamental Message Format lists the Origin of Data of this field as, "Calculated by DTN".

-Mark D. Valor


Ugh, nevermind -- I was fooled by the trailing comma at the end of the Quote lines to imagine I was missing the Total Volume field (empty value).

I can see now that the final field (which includes a trailing comma) is indeed the Total Volume field and that this value does change on the line with the "v" Message Type.

Thanks for your time.

-Mark D. Valor


EDIT: Nevermind -- see the first reply below. Got fooled by the trailing comma in the Q lines.

--- Original Message -----

In the L1 feed for a futures contract below, notice that the 9th message from the bottom has Message Type 'v'. Yet I don't see a change in anything "volume-like". The bid size is the same, the ask size is the same, the most recent trade size is the same. In what sense did a "volume change" occur which could be detected in sofware consuming an L1 feed for a futures contract?

Separate issue I'll ask in a separate new topic -- no idea why I don't receive Total Volume (final field) in my L1 feed. You can see that I request it in the custom dynamic field response a the top. The documentation for the field says, "Calculated by IQConnect when not sent from servers (or provided by the exchange)". None of my L1 feeds for futures contract receive it (and I'm not currently requesting an L1 feeds for non-futures instruments to see a difference there).

--- begin sample ---
S,CURRENT UPDATE FIELDNAMES,Symbol,TickID,Message Contents,Bid Time,Bid,Bid Size,Ask Time,Ask,Ask Size,Most Recent Trade Conditions,Most Recent Trade Time,Most Recent Trade,Most Recent Trade Size,Most Recent Trade Aggressor,Tick,Total Volume
...
Q,@MNQ#,234172322,a,09:23:09.529415,15836.50,8,09:23:09.529631,15836.75,4,01,09:23:09.528964,15836.75,1,2,,227513,
Q,@MNQ#,234172322,b,09:23:09.529708,15836.50,7,09:23:09.529631,15836.75,4,01,09:23:09.528964,15836.75,1,2,,227513,
Q,@MNQ#,234172322,b,09:23:09.529709,15836.50,6,09:23:09.529631,15836.75,4,01,09:23:09.528964,15836.75,1,2,,227513,
Q,@MNQ#,234172322,b,09:23:09.529817,15836.50,5,09:23:09.529631,15836.75,4,01,09:23:09.528964,15836.75,1,2,,227513,
Q,@MNQ#,234172328,C,09:23:09.529817,15836.50,5,09:23:09.529631,15836.75,4,01,09:23:09.529877,15836.50,1,2,175,227514,
Q,@MNQ#,234172328,b,09:23:09.529877,15836.50,4,09:23:09.529631,15836.75,4,01,09:23:09.529877,15836.50,1,2,,227514,
Q,@MNQ#,234172329,C,09:23:09.529877,15836.50,4,09:23:09.529631,15836.75,4,01,09:23:09.530059,15836.50,1,2,183,227515,
Q,@MNQ#,234172329,b,09:23:09.530059,15836.50,3,09:23:09.529631,15836.75,4,01,09:23:09.530059,15836.50,1,2,,227515,
Q,@MNQ#,234172330,C,09:23:09.530059,15836.50,3,09:23:09.529631,15836.75,4,01,09:23:09.530111,15836.50,3,2,183,227518,
Q,@MNQ#,234172330,b,09:23:09.530111,15836.25,5,09:23:09.529631,15836.75,4,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,a,09:23:09.530111,15836.25,5,09:23:09.530460,15836.50,1,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,a,09:23:09.530111,15836.25,5,09:23:09.531483,15836.50,2,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,a,09:23:09.530111,15836.25,5,09:23:09.531646,15836.50,3,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,a,09:23:09.530111,15836.25,5,09:23:09.531784,15836.50,4,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,b,09:23:09.531862,15836.25,4,09:23:09.531784,15836.50,4,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,a,09:23:09.531862,15836.25,4,09:23:09.531882,15836.50,5,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,a,09:23:09.531862,15836.25,4,09:23:09.531932,15836.50,6,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,b,09:23:09.532203,15836.25,3,09:23:09.531932,15836.50,6,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,a,09:23:09.532203,15836.25,3,09:23:09.532288,15836.50,5,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,a,09:23:09.532203,15836.25,3,09:23:09.532441,15836.50,4,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,b,09:23:09.532722,15836.25,2,09:23:09.532441,15836.50,4,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,a,09:23:09.532722,15836.25,2,09:23:09.532722,15836.50,5,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,a,09:23:09.532722,15836.25,2,09:23:09.532733,15836.50,6,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,a,09:23:09.532722,15836.25,2,09:23:09.532735,15836.50,7,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,a,09:23:09.532722,15836.25,2,09:23:09.532771,15836.50,6,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,a,09:23:09.532722,15836.25,2,09:23:09.536253,15836.50,5,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,b,09:23:09.536273,15836.25,3,09:23:09.536253,15836.50,5,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,a,09:23:09.536273,15836.25,3,09:23:09.536360,15836.50,4,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,b,09:23:09.540764,15836.25,2,09:23:09.536360,15836.50,4,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,a,09:23:09.540764,15836.25,2,09:23:09.540764,15836.50,5,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,a,09:23:09.540764,15836.25,2,09:23:09.586880,15836.50,4,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,b,09:23:09.586880,15836.25,3,09:23:09.586880,15836.50,4,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,b,09:23:09.586997,15836.25,4,09:23:09.586880,15836.50,4,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,a,09:23:09.586997,15836.25,4,09:23:09.587477,15836.50,3,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,b,09:23:09.587488,15836.25,5,09:23:09.587477,15836.50,3,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,a,09:23:09.587488,15836.25,5,09:23:09.587595,15836.50,2,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,b,09:23:09.587599,15836.25,6,09:23:09.587595,15836.50,2,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,a,09:23:09.587599,15836.25,6,09:23:09.587609,15836.50,1,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,b,09:23:09.587762,15836.25,5,09:23:09.587609,15836.50,1,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,a,09:23:09.587762,15836.25,5,09:23:09.587763,15836.75,6,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,b,09:23:09.587812,15836.25,4,09:23:09.587763,15836.75,6,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,b,09:23:09.587814,15836.25,3,09:23:09.587763,15836.75,6,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,a,09:23:09.587814,15836.25,3,09:23:09.587816,15836.50,1,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,a,09:23:09.587814,15836.25,3,09:23:09.587862,15836.50,2,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,a,09:23:09.587814,15836.25,3,09:23:09.587916,15836.50,3,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,b,09:23:09.587998,15836.25,4,09:23:09.587916,15836.50,3,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,a,09:23:09.587998,15836.25,4,09:23:09.587999,15836.50,2,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,b,09:23:09.587999,15836.25,5,09:23:09.587999,15836.50,2,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,a,09:23:09.587999,15836.25,5,09:23:09.588119,15836.50,1,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,a,09:23:09.587999,15836.25,5,09:23:09.588120,15836.75,7,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,b,09:23:09.588161,15836.25,4,09:23:09.588120,15836.75,7,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,b,09:23:09.588162,15836.25,3,09:23:09.588120,15836.75,7,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,b,09:23:09.588335,15836.25,4,09:23:09.588120,15836.75,7,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,b,09:23:09.588336,15836.25,5,09:23:09.588120,15836.75,7,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,b,09:23:09.595870,15836.25,4,09:23:09.588120,15836.75,7,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,a,09:23:09.595870,15836.25,4,09:23:09.595981,15836.50,1,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,b,09:23:09.596018,15836.25,3,09:23:09.595981,15836.50,1,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,b,09:23:09.596158,15836.25,4,09:23:09.595981,15836.50,1,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,a,09:23:09.596158,15836.25,4,09:23:09.596285,15836.75,7,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,b,09:23:09.596285,15836.25,5,09:23:09.596285,15836.75,7,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,b,09:23:09.596620,15836.25,6,09:23:09.596285,15836.75,7,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,b,09:23:09.597092,15836.25,7,09:23:09.596285,15836.75,7,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,b,09:23:09.597191,15836.25,8,09:23:09.596285,15836.75,7,01,09:23:09.530111,15836.50,3,2,,227518,
Q,@MNQ#,234172330,v,09:23:09.597191,15836.25,8,09:23:09.596285,15836.75,7,01,09:23:09.530111,15836.50,3,2,,227519,
Q,@MNQ#,234172330,a,09:23:09.597191,15836.25,8,09:23:09.800264,15836.75,6,01,09:23:09.530111,15836.50,3,2,,227519,
Q,@MNQ#,234172330,a,09:23:09.597191,15836.25,8,09:23:09.800280,15836.75,5,01,09:23:09.530111,15836.50,3,2,,227519,
Q,@MNQ#,234172330,a,09:23:09.597191,15836.25,8,09:23:09.800281,15836.75,4,01,09:23:09.530111,15836.50,3,2,,227519,
Q,@MNQ#,234172330,b,09:23:09.800495,15836.50,1,09:23:09.800281,15836.75,4,01,09:23:09.530111,15836.50,3,2,,227519,
Q,@MNQ#,234172330,b,09:23:09.800528,15836.50,2,09:23:09.800281,15836.75,4,01,09:23:09.530111,15836.50,3,2,,227519,
Q,@MNQ#,234172330,a,09:23:09.800528,15836.50,2,09:23:09.800528,15836.75,3,01,09:23:09.530111,15836.50,3,2,,227519,
Q,@MNQ#,234172330,a,09:23:09.800528,15836.50,2,09:23:09.800543,15836.75,2,01,09:23:09.530111,15836.50,3,2,,227519,
Q,@MNQ#,234172334,C,09:23:09.800528,15836.50,2,09:23:09.800543,15836.75,2,01,09:23:09.800628,15836.75,1,1,173,227520,
--- end sample ---


-Mark D. Valor
Edited by mkvalor on Aug 4, 2023 at 09:30 AM


Thanks. I've been issuing a fundamentals query and then cutting off my watches a few seconds after market close. And once I realized I wasn't (yet) seeing the current day's activity reflected, I understood I could just start delaying when I ended my watches. But I figured I'd ask, in case there was specific guidance about it.

As a follow up -- would any particular fields of the L1 F "fundamentals" message get updated at the same time? For example, if the instrument hit a new 52-Week High during that day, would that get updated in the Fundamentals message around the same time that P "Summary" message gets updated? (or if not, when would such an update occur?)

-Mark D. Valor


Here is a real-time L1 'F' fundamental message for the CME ES futures contract. I requested the watch about 15 minutes before open a few days ago using the @ES# symbol which stands in for the currently active contract (in case that makes a difference):

F,@ES#,22,,,4609.25,3683.50,,,,,,,,,,,,E-MINI S&P 500 SEPTEMBER 2023,,,,,,,,,,,,12,2,,12.42,8,43,07/19/2023,10/11/2022,,,,,,09/15/2023,,,ES,,,18:00:00,17:00:00,USD,50,--H--M--U--Z,0.250000000,,,,

Even though volume is recorded through the day in L1 updates and is available in the historical data, I notice that column 5 here for Average Volume is empty. I wondery, why?

-Mark D. Valor


I noticed in this thread:
https://forums.iqfeed.net/index.cfm?page=topic&topicID=6953
that Market Summary reports are compiled at about 5:30 pm each trading day. Would that be around the time that L1 "P" summary lines get updated with data for session end (such as close) for that day as well? And is there any difference in the time when this fresh data becomes available for US futures vs equities?

-Mark D. Valor


tl;dr: If you use L2 Market by Order messages for CME/ICE futures, do you feel your systtem greatly benefits from the broader range of updated price points and/or the ability to correlate the OrderId with the L1 messages?

I'm asking because I coded my system under an older protocol so I only have L2 Market by Price. It works fine. But I'm trying to gague whether it's worth it to go back and update my custom software system (a fairly invasive change).




---- Original message below ----
I'm an IQFeed API programmer who mostly analyzes CME/ICE futures and I joined DTN well after the older forum posts which initially requested something like the L2 MBO data. Yet I also joined before protocol 6 was out of beta. As such, I originally coded my apps using protocol 5.2 to pull in the L2 MBP data and 'it wasn't broke so I didn't fix it'. I did upgrade my IQFeed client and protocol once 6.x came out and my apps do use the latest of each (though still only pull MBP data).

Lately, out of curiosity about whether my algorithms might beneifit from more/different data, I've been wondering what the main benefits of MBO data are over MBP. Here's what I believe I understand about the differences:
  • MBO data doesn't have the same limitation on number of levels per side (buy, sell)

  • As a consequence of the first point, MBO summary messages may be of high volume while the initial state of all(?) prices are transmitted (with a consideration in the documentation to perhaps throttle these requests to avoid saturating the network connection and perhaps stalling other watches).

  • There's an OrderId with MBO messages that lets you correlate(?) these messages with the OrderIds of L1 messages

  • MBP updates replace the previous message you received at that price with a new aggregated total

  • MBO updates are only deltas and you must maintain the structure of the price levels yourself and mathmatically apply the deltas as they come in

  • Equity L2 (NASDAQ) comes through as MBO messages and includes MMID but not Order Id

If I made any errors or significant omissions above, please comment to correct them. So , focusing on futures -- I think the main benefits of MBO are the expanded price levels and using the OrderID to correlate with the L1 messages. Yet the cost would be recoding my apps to maintian the structure of the L2 data by applying the updates mathematically as well as potentially ingesting a bunch of price points far(?) from the current price activity. There is a potential fragility if MBO updates are somehow lost.

Please comment with your thoughts about switching from MBP levels to MBO updates (for markets that offer it). Do you make much use of the Order Id correlation and the broader price range (which may include prices far from the current price action)? Does an app analyzing L2 MBO data suddenly start consuming 3X the memory (vs MBP) to hold all the new price levels? If you switched, do you feel it was worth it? etc.


-Mark D. Valor
Edited by mkvalor on Jul 20, 2023 at 10:54 PM


A long time ago, someone stated in this Data and Contet Support topic: https://forums.iqfeed.net/index.cfm?page=topic&topicID=2402 that market indexes and stats are updated frequently, perhaps as often as tick-by-tick. I could swear I've read somewhere on the web that many of the famous indexes (such as SPX) have their vlaues updated only as often as every 15 seconds. But anyway...

For those with realtime subscriptions who write their own code with the TCP IQFeed API, what is the best way to consume indexes and market stats, if we want the quickest available updates? Obviously the Market Summary snapshot call wouldn't be ideal if, for instance, the Russell "exchange" summary file for security type Index is updated on DTN servers every 5 minutes.

1. Can indexes and market stats actually be consumed via an L1 Watch API call? (clearly most of the dynamic fields such as those related to bid and ask would not apply, which is why I ask)

1a. If an L1 feed is the way to go, are there any special values for any of the dynamic fields (such as "Message Contents") which woudl have any specific or unusual meanings for indexes or market stats?

2. Do people instead request streaming interval bars for these with an interval set as low as applicable (with a floor of 1 second)? If the old thread I referenced is correct about tick-by-tick frequency, this wouldn't take advantage of the "realtime" subscription.

2a. Many market stats such as "TRIN" lend themselves to line charts rather than bars... In this caseI guess one might simply use "last" from the OHLC values and just discard the other 3 "price" fields? (moot if streaming bars wouldn't be used for live index/market stat requests)

2a. I can grab historical bars off-hours, so I'm not worried about the "8/180" day restriction for second intervals which aren't a multiple of 20; I'd mainly be consuming only the current day's "trades". But, as a good citizen in the DTN ecosystm, is there an upper limit to the number of 1-second streaming bars calls I can make (currrent day only) or is this simply included in the (base) 500-symbol limit? (moot if streaming bars wouldn't be used for live index/market stat requests)

OR

3. If there is a better API call to stream real time data for indexes and/or market stats not covered above, just mention that and ignore the other numberd questions




-Mark D. Valor
Edited by mkvalor on Dec 1, 2022 at 12:53 AM
Edited by mkvalor on Dec 1, 2022 at 12:54 AM
Edited by mkvalor on Dec 1, 2022 at 01:04 AM

Forum Suggestions » https:// ???? Nov 18, 2022 04:20 PM (Total replies: 7)

Great news! Thanks for taking care of this.

-Mark D. Valor


Ah, I hadn't thought of the wider spread situation since I mostly trade markets where that is rare (during the hours when I am active). Thank you for your explanation.

-Mark D. Valor

New IQFeed Forum » Count of DTN Market Statistics Nov 14, 2022 11:13 AM (Total replies: 3)

Thanks for the reply. User error caused the low count on the web page. I had accidentally typed a single character into the search box which I hadn't noticed.

-Mark D. Valor

New IQFeed Forum » Count of DTN Market Statistics Nov 14, 2022 10:25 AM (Total replies: 3)

This is my first time exploring the Market Statistics available from IQFeed. Finally figured out the way to get them via API was to request a 5-minute snapshot (this is neither explained in the docs that I could see nor intuitive to someone new to the API, by the way).

Now, when I use the website search page, I find there are 752 Market Statistic codes results returned (and all of them under "Exchange" DTN). Yet, when I make a 5-minute snapshot request via API v6.2 for the security type code for Market Statistics and the exchange code for DTN, my system receives a file with 3383 lines returned. Are there actually additional types of Market Statistics available via the API? Or what else explains the difference between these two counts?

-Mark D. Valor


Quote: I have been running it for years from Sunday night to Friday night, i.e. a full trading week. No issues. The client has a very small memory footprint.
Great qQuant, just the type of reply I was looking for, thanks!

I noticed this recent reply from DTN_Steve_S in another forum thread mentioning that Saturdays are considered maintenance days [1]:
Quote: Hello all, Saturdays are our routine maintenance days. This involves server reboots and software restarts and, on occasion, can cause requests to fail. We typically try to keep it limited to the morning on Saturdays but sometimes it runs longer. Trying to avoid Saturday mornings for data downloads would be recommended if possible and keeping in mind that request failures should be expected during this time if you are working with our API.

So putting it all together, it seems that allowing IQConnect to shut down Fridays at the close of US trading hours and restarting it on Sundays (to grab historical data and begin off-hours quotes) may be ideal.

[1] http://forums.dtn.com/index.cfm?page=topic&topicID=5776

-Mark D. Valor


Time: Fri April 19, 2024 4:10 PM CFBB v1.2.0 10 ms.
© AderSoftware 2002-2003