Join the 80,000 other DTN customers who enjoy the fastest, most reliable data available. There is no better value than DTN!

(Move your cursor to this area to pause scrolling)




"I started a trial a few weeks back before the market went wild. DTN.IQ didn’t miss anything and beat my other provider. I decided to stay with you because of the great service through all the volatility." - Comment from Mike
"I've been using IQFeed 4 in a multi-threaded situation for the last week or two on 2600 symbols or so with 100 simultaneous daily charts, and I have had 100% responsiveness." - Comment from Scott
"DTN has never given me problems. It is incredibly stable. In fact I've occasionally lost the data feed from Interactive Brokers, but still been able to trade because I'm getting good data from DTN." - Comment from Leighton
"Thank God for your Data Feed as the only Zippers I see are on my pants (LOL), and no more 200 pip spikes to mess up charts." - Comment from Spiro via Email
"I've never had DTN go out on me since switching. ******* would go down a couple times every month when I was using them." - Comment from Bryce in AL.
"Excellent datafeed !!!" - Comment from Arely
"This beats the pants off CQG, I am definitely switching to the ProphetX 3.0!" - Comment from Stephen
"I had always used ******* but for the past 2 weeks have been trying DTN IQFeed. Customer support has been extraordinary. They call just to make sure your problem hasn't recurred." - Comment from Public Forum
"I am enjoying the feed very much - so superior to the broker provided feed I was previously using." - Comment from George
"My broker in Davenport suggested I give you a try as he uses your service and says its the best." - Comment from Bill via RT Chat
Home  Search  Register  Login  Recent Posts

Information on DTN's Industries:
DTN Oil & Gas | DTN Trading | DTN Agriculture | DTN Weather
Follow DTNMarkets on Twitter
DTN.IQ/IQFeed on Twitter
DTN News and Analysis on Twitter
»Forums Index »Product Support »Data and Content Support »Same Timestamp with different TickID
Author Topic: Same Timestamp with different TickID (13 messages, Page 1 of 1)

Pierre845
-Interested User-
Posts: 5
Joined: Dec 18, 2018


Posted: May 30, 2020 11:28 PM          Msg. 1 of 13
Hi All,

While in the process of analysing data for efficient storage purpose (eliminating redundant or unecessary data) I encountered occurences where there is a serie of Timestamps of same value (to the microsecond).
Within it there is the same TickID to a point (also same Bid/Ask and same agressor).
Then the TickID changes, Bid/Ask as well, and also agressor, ss follows:

Seconds | Last | LastSize | Bid | Ask | TickID | Agressor
7.013979 | 3055.5 | 1 | 3055.50 | 3055.75 | 272584026 | 2
7.013979 | 3055.5 | 1 | 3055.50 | 3055.75 | 272584026 | 2
7.013979 | 3055.5 | 1 | 3055.50 | 3055.75 | 272584026 | 2
7.013979 | 3055.5 | 1 | 3055.25 | 3055.50 | 272584027 | 1 <---------

1. So Bid is hit 3 times - and I believe it's unbundled data hence the same TickID,
but how come at the same timestamp the bid/ask goes lower 0.25 pt, and a new buyer comes to hit the offer?

2. I was thinking .. is it the same agressor who was hitting the bid, but CME/IQfeed classify it as an offer agressor because the bid/Ask went down, and they calculate the agressor flag as 'last price at bid' or 'last price at ask' (and in this case the agressor field is useless as we can calculate it easily (redundant for data storage purpose).

3. Can it be explained otherwise if we had nanoseconds? as the timestamp might actually be different to the nanosecond?

DTN Todd
-Interested User-
Posts: 55
Joined: Mar 24, 2010


Posted: May 31, 2020 10:55 PM          Msg. 2 of 13
Hello Pierre845

We will look into your inquiry, we should have an answer for you within 24 hours

Thanks
Todd

DTN_Stephen
-DTN Guru-
Posts: 286
Joined: Aug 22, 2014


Posted: Jun 1, 2020 06:32 AM          Msg. 3 of 13
May we get a date and a more specific timestamp and symbol so that we can research this?

Stephen Shockey
Senior Customer Support Representative and Product Support Specialist

DTN
800-779-7299
support@iqfeed.net

Pierre845
-Interested User-
Posts: 5
Joined: Dec 18, 2018


Posted: Jun 1, 2020 06:44 AM          Msg. 4 of 13
Sure, please see the attached.
It's rounded (3055.8 is actually 3055.75)



File Attached: Timestamps_TickID.png (downloaded 142 times)

DTN_Stephen
-DTN Guru-
Posts: 286
Joined: Aug 22, 2014


Posted: Jun 1, 2020 06:46 AM          Msg. 5 of 13
May I get the symbol please?

Stephen Shockey
Senior Customer Support Representative and Product Support Specialist

DTN
800-779-7299
support@iqfeed.net

Pierre845
-Interested User-
Posts: 5
Joined: Dec 18, 2018


Posted: Jun 1, 2020 07:01 AM          Msg. 6 of 13
yes, it's in column 'Symbol' of the file I uploaded: @ESH20

DTN_Stephen
-DTN Guru-
Posts: 286
Joined: Aug 22, 2014


Posted: Jun 1, 2020 07:21 AM          Msg. 7 of 13
I am sorry I did not see the uploaded file. I appreciate all the information. I will post a comment as soon as our research is complete.

Stephen Shockey
Senior Customer Support Representative and Product Support Specialist

DTN
800-779-7299
support@iqfeed.net

DTN_Stephen
-DTN Guru-
Posts: 286
Joined: Aug 22, 2014


Posted: Jun 1, 2020 08:02 AM          Msg. 8 of 13
We unbundle all incoming trades from the CME, thus the multiple trades with the same Tick ID and timestamp.

The other 2 trades with separate Tick ID's and Aggressor flags with the same time stamp did occur as seen as we use the timestamp sent from the exchange and do not assign one.

The aggressor flag shows if the trade occurred at the bid or at the ask, that is what we can discern from the aggressor.

Stephen Shockey
Senior Customer Support Representative and Product Support Specialist

DTN
800-779-7299
support@iqfeed.net

Pierre845
-Interested User-
Posts: 5
Joined: Dec 18, 2018


Posted: Jun 1, 2020 08:46 AM          Msg. 9 of 13
Thanks Stephen, so do you mean that in essence the Agressor Field is useless? indeed in that case we would just need to comapre Bid/Ask/Last Price and work it out without the need of another data field.

Nevertheless, the fact that Trade TickID 272584027 occurs at the same price AND with the same timestamp, without being bundled by CME (I understand you unbundle, which creates same TickIDs), makes me wonder if this is an actual new trade (which would make the timestamp different if we had nanoseconds - could you please check that with CME maybe?),
OR
that this is the way the CME reports trades in this case, when the Bid/Ask moves but the trade is executed at Ask in this case instead of Bid (market Makers / HFT actions?), meaning the agressor code should stay to 2 and not 1.

This happens a few times during the day (>20 times), and I find that the likelihood of two traders, one bid agressor, and one ask agressor, to place their trade at the same time (to the microsecond) WHEN the Bid/Ask moves, is very low.

DTN_Gary_Stephen
-DTN Evangelist-
Posts: 141
Joined: Jul 3, 2019


Posted: Jun 2, 2020 06:30 AM          Msg. 10 of 13
I would like to add a couple other comments about how TickID works:

- TickIDs are unique by ID, but also symbol and traded market. For example, there may exist a TickID 123456789 on NYSE and a TickID 123456789 on NASDAQ. It's also possible for two different symbols to have the same TickID. If you want to identify trades uniquely, you have to include the traded market and symbol as well. (However, see next item.)

- Furthermore, TickIDs are not necessarily unique by trade. Futures exchanges and OPRA can have the same TickID applied to multiple trades. This is because those exchanges "bundle" trades, meaning, they can send several trades at once. All bundled trades will have the same TickID. The IQFeed API "unbundles" them, so each trade has its own record, but they do have the same TickID. This is by design, so you can "rebundle" the trade data yourself if desired.

- TickIDs are not necessarily incremental, a certain number of digits, or anything like that.

- TickID is a 32-bit unsigned integer. It can be as large as 4,294,967,295 (commas added for clarity). Allow for this when creating a variable or database column to store them in.

Sincerely,
Gary Stephen
DTN IQFeed Implementation Support Specialist

altmany
-Interested User-
Posts: 50
Joined: Jul 30, 2018

IQML - IQFeed-MATLAB connector


Posted: Jun 2, 2020 06:47 AM          Msg. 11 of 13
I think that it would be useful to include a unique sequence ID with each tick that *would indeed* be unique (an incremental long integer if possible) across the board.

I'd be surprised if you don't already have such an ID in your databases, since all databases have such IDs which are often used as index keys. I assume that adding it to the reported tick fields should not be too difficult.

A unique sequence ID will help end-users differentiate between different ticks in their programs. It will also help users when they report a potential data problem to refer to a unique sequence id that you could immediately find in your database.

Yair Altman
IQML - IQFeed-MATLAB connector
https://UndocumentedMatlab.com/IQML

I am not a DTN employee; my post reflects my personal opinion

Pierre845
-Interested User-
Posts: 5
Joined: Dec 18, 2018


Posted: Jun 2, 2020 08:04 AM          Msg. 12 of 13
Thanks Gary_Stephen for you input on TickIDs, a refresher on how they work is always useful, even though in this case it doesn't address what I was talking about.

Indeed in the case I explained, it would actually be the opposite, as it looks as if the two trades across the Bid/Ask change should be the same one (perhaps only nanoseconds could clarify); it looks like the only reason they're different is because the Bid/Ask moved (which made the agressor code change, which I believe should stay at 2 instead of 1). As you've been working / communicating with CME, they're likely to give you an explanation.

Unless it's market making action, placing two different opposite trades (at the same time to the microsecond, but with a time hierarchy to the nanosecond), first one to get the Bid/Ask down, second to hit the Ask .. but I don't see the point.
Edited by Pierre845 on Jun 2, 2020 at 08:06 AM

DTN_Gary_Stephen
-DTN Evangelist-
Posts: 141
Joined: Jul 3, 2019


Posted: Jun 10, 2020 10:20 AM          Msg. 13 of 13
Quote: I think that it would be useful to include a unique sequence ID with each tick that *would indeed* be unique (an incremental long integer if possible) across the board.

I'd be surprised if you don't already have such an ID in your databases, since all databases have such IDs which are often used as index keys. I assume that adding it to the reported tick fields should not be too difficult.

A unique sequence ID will help end-users differentiate between different ticks in their programs. It will also help users when they report a potential data problem to refer to a unique sequence id that you could immediately find in your database.

--- Original message by altmany on Jun 2, 2020 06:47 AM
Yair,

Prepare to be surprised :) - there is no unique identifier anywhere in the IQFeed system. Because of the massive volume of data DTN processes, and the number of sources it comes from, a unique identifier was never implemented. It would certainly be useful to have one, and it's been discussed many times, but adding one is not in any short-term plans.

TickID was never intended to be a unique identifier. Its purpose is to link DTN's data back to the exchange data, for verification purposes. If someone says to our support team "hey, the price of ABCD at 11:35:20 looks a bit off," we can easily look up that price via the TickID, and make sure our data matches what the exchange reported. A unique, DTN-created identifier would not help, because the exchanges wouldn't know it. Similarly, third-party software wouldn't know these IDs either, or would have to put in the development hours to implement them. So having a custom unique ID for each trade is not as useful as it may seem.

Sincerely,
Gary Stephen
DTN IQFeed Implementation Support Specialist
 

 

Time: Tue September 29, 2020 8:55 AM CFBB v1.2.0 32 ms.
© AderSoftware 2002-2003