||Mar 26, 2008 09:40 AM
||Jun 5, 2023 05:40 PM
||Jun 6, 2023 05:15 AM
Roberts has contributed to 44 posts out of 20985 total posts
(0.21%) in 5,555 days (0.01 posts per day).
20 Most recent posts:
Daily data is incorrect for stocks whose High or Low price for the day was hit in the first few minutes of the session.
Examples (just 2 of hundreds) -
RCL - Low was 85.86, but the Daily bar Low is 86.00
RE - High was 346.58, but the Daily bar High is 346.26
The doc here: http://www.iqfeed.net/dev/api/docs/OptionChainsviaTCPIP.cfm indicates that the response to any of the requests for option chains will have "a field containing the letters LC". We expect to see LC as the first field unless a RequestID is specified, and in that case LC would be the second token.
Nonetheless in response to the request, LC is nowhere to be found in the response.
I don't care if LC is there or not, but I just want to know that it will never be there. Will it?
Any chance that these issues will be fixed?
Edited by Roberts on Dec 16, 2022 at 08:02 AM
Finally, another image attached highlighting that the first 1048 bar occasionally is wrong, completely missing the first 78 minutes of data and instead appears to insert data for the 1-min of premarket trading prior to 0930. The data shown were part of the response to this request:
HIT,ABBV,4680,20211003 000000,20221216 000000,,093000,160000,1,7,,s,0
ABBV 78 Minute bars returned yesterday (12/13/2022) OHLCV
12/13/2022 10:48 166.92 167.16 165.62 166.06 417761
12/13/2022 12:06 166.05 166.06 164.71 164.985 394991
12/13/2022 13:24 164.985 165.55 164.69 165.085 253186
12/13/2022 14:42 165.09 165.94 164.83 164.86 323191
12/13/2022 16:00 164.89 165.65 164.67 164.92 644521
ABBV 78 Minute bars returned today (12/14/2022) OHLCV
12/13/2022 10:48 166.95 167.5 165.76 166.41 542785
12/13/2022 12:06 166.39 166.39 164.75 165.31 445391
12/13/2022 13:24 165.29 165.37 164.69 165.365 281420
12/13/2022 14:42 165.39 165.62 164.9 165.54 275937
12/13/2022 16:00 165.5491 165.94 164.67 164.92 786909
It's only 2AM EST, but the data returned for the same request has changed (and is now correct).
For that request, is it misleading that IQFeed timestamps the bars in the way that [appears to] use a 0930 start? If the (ending) timestamp of the 78-minute bar is 10:48, then that implies that it's the 78-minutes of data from 0930 to 1048 - yet that is not the case (today).
I had a hunch that paid off. I rescaled IQFeed 1-minute data and compared the result to the 78-minute bars returned from IQFeed. As it turns out, using the request above, IQFeed uses the 0930 filter time as a start reference. The problem is that it doesn't do that for the current day!
See the attached image that makes this comparison. Except for occasional - but significant - differences on [usually] the 1048 bar, the bars are identical prior to "today". My guess is that when I request the ABBV 78 minute bars tomorrow, they'll be different than they are today. That's not good.
When requesting odd-intervals, like 13 or 78 minute bars (both of which divide evenly into a 390-minute day), what is the correct way to identify the starting time for the data compression?
For example, if you start at midnight, 78 minute bar end-of-bar timestamps for bars that contain data for the U.S. market regular session would be (in bold):
However, I need bar data aligned with the 09:30 market open whose end-of-bar times are:
Specifying 0930 and 1600 for the HIT filter times returns bars with the correct timestamps. e.g.,
HIT,ABBV,4680,20221213 000000,20221215 000000,,093000,160000,1,3,,s,0
20221213 10:48 166.92 167.16 165.62 166.06 417,761
20221213 12:06 166.05 166.06 164.71 164.99 394,991
HOWEVER, inspecting the 1-minute data (always referring to end-of-bar timestamps), the OPEN for ABBV today's 09:31 bar was 166.95 - not 166.92 as indicated in the 78-minute bar data.
More strikingly, the 1-min bar close at 10:48 is 166.41 - not 166.06. Note that the bar ending at 10:17 closed at 166.06 - the only 1 minute bar to close at that price before 10:48.
What's going on with this data compression?
What's IQFeed's guidance on how to do this correctly?
Unfortunately, the data guys got the ex-date wrong. It's today 04/23/2021, not 04/22/2021. Can you get them to adjust that, please?
F,VXX,12,,58583000,45.0300,9.7300,21.7600,9.7300,0.0000,,,,,,,,,IPATH SER B S&P 500 VIX SHORT-TERM FUTURES ETN,VXX1,,1.00,,11570.0,,,,101585,4.00 04/22/2021,,14,4,,52.88,1,18,06/12/2020,04/16/2021,01/29/2021,04/16/2021,16.7900,,,,,0,,,,,,USD,,,,,BBG00JQ5JWB5,3,
Disregard the split in 2017. That one was for the previous VXX series, which was delisted.
I'm looking at both! You should know me by now ;)
I understand that only Daily is adjusted by IQFeed. We use the Fundamental message to adjust our intraday data too.
VXX reverse split 1:4 today, 04/23/2021
And, the Fundamental string isn't returned the penultimate split either - 1:4 on 08/23/2017:
F,VXX,12,,58583000,45.0300,9.7300,21.7600,9.7300,0.0000,,,,,,,,,IPATH SER B S&P 500 VIX SHORT-TERM FUTURES ETN,VXX1,,1.00,,11570.0,,,,101585,,,14,4,,52.88,1,18,06/12/2020,04/16/2021,01/29/2021,04/16/2021,16.7900,,,,,0,,,,,,USD,,,,,BBG00JQ5JWB5,3,
A customer is having trouble with streaming data for Indices, like VIX.XO and VIX3M.XO. From the description, there appear to be no updates, which are expected at least every 15 seconds for these instruments.
Is it possible that these are updated only as a "P" Summary message? (Our Dev feed doesn't have perms for these indices.)
Yes! Getting it now, thanks.
FNGU is still missing its recent 10:1 split event on 2/12/2021. Can you add that to the Fundamental message, please?
(The Daily data are adjusted, but we need that Split event to adjust the intraday properly.)
Edited by Roberts on Apr 15, 2021 at 04:56 AM
Edited by Roberts on Apr 15, 2021 at 04:56 AM
Newman! (said like Jerry Seinfeld)
Thanks for spamming my topic with your completely unrelated opinion.
But since we're doing that, let's do some more shameless promotion and make sure to visit https://www.wealth-lab.com after March 1st, 2021 to get your all-new Wealth-Lab Version 7. It will most certainly "help to improve and improve the efficiency of trading" (sic.) with auto-trading connections to stock, futures, and crypto-currency brokers, multi-core optimizations, intraday strategy monitoring of complete watchlists, and much, much more.
If it's turned on, you'll get an incomplete BU message whenever a new tick has a trade in it.
.. but you'll get that message only after the Update Interval. Sure, we can put that at 1 second and have at most a 1 second delay, which is acceptable, but it certainly complicates the adapter. Instead of just passing along the final result when it arrives, we'll have store the last BU message, and, if the BC message doesn't come in before 1 second after the interval, we use the last BU data, and, later have to ignore the next BC message. Not joy, but it is what it is.
Thanks for the link. Now I understand BU much better, because that was unclear too. (If I had requested 15 second updates, I was expecting updates at :00, :15, :30 and :45 seconds every minute, which is not how that works either.)
Quoting Tim Walter from your first link, "The answer, due to the impossibility of syncing all clocks involved and the random latencies between connections, was to say that a bar complete message is only sent when a new message, with a time that is outside of the current bar, is received. So that is why BC messages are not timely."
It's understandable that DTN wants the data to match perfectly, but it sure doesn't seem like a good design decision based on the tradeoffs that require the consumer to use a work-around that is even less perfect and that still doesn't provide a timely bar without building one ourselves using updates. IQFeed already provides a TimeMessage that we use as a heartbeat to "close" intervals precisely on time so that strategies can be executed. We've used this design for 20 years in streaming charts and sure seems like an easy solution to implement for a "BC" message too. If the 99% of the user base expects this, what's the problem?
Our solution may be to toss the whole concept and simply subscribe to hundreds of streams instead. That's worse for the user's cpu and has gotta be worse for the DTN backend.
I actually think I see what's happening. TTD doesn't print it's complete interval until a full lot trade occurs in the next interval. There's got to be a way around that, right?
Oh my gosh, thank you. You hit the nail on the head!
In these initial tests, I'm watching just 2 symbols, AAPL and TTD, at 1 minute intervals. Very commonly, I'll see AAPL update 1 or 2 seconds after the end of interval (good), but TTD's update for the same bar interval will come maybe to 60 seconds later (not good). I'm just looking at the Debug prints from the callback, so I'm not sure how my program could create that delayed effect. Does that sound like something to expect?
I've got 2 different implementations in 2 applications performing Bar Watches (BW) without any problem. But I'm having trouble with a new, 3rd implementation, and I'm looking for clues.
1. I'm connecting to the Derivative port 9400.
2. IQFeed returns "S,SERVER CONNECTED" in the socket's callback
3. Example: "BW,IBM,60,,,,,,req1,,,0
... and then nothing else is returned in the callback. No errors, nothing.
I've played around with the optional fields in the BW message, but nothing changes. Any ideas?