Hi All,

While in the process of analysing data for efficient storage purpose (eliminating redundant or unecessary data) I encountered occurences where there is a serie of Timestamps of same value (to the microsecond).
Within it there is the same TickID to a point (also same Bid/Ask and same agressor).
Then the TickID changes, Bid/Ask as well, and also agressor, ss follows:

Seconds | Last | LastSize | Bid | Ask | TickID | Agressor
7.013979 | 3055.5 | 1 | 3055.50 | 3055.75 | 272584026 | 2
7.013979 | 3055.5 | 1 | 3055.50 | 3055.75 | 272584026 | 2
7.013979 | 3055.5 | 1 | 3055.50 | 3055.75 | 272584026 | 2
7.013979 | 3055.5 | 1 | 3055.25 | 3055.50 | 272584027 | 1 <---------

1. So Bid is hit 3 times - and I believe it's unbundled data hence the same TickID,
but how come at the same timestamp the bid/ask goes lower 0.25 pt, and a new buyer comes to hit the offer?

2. I was thinking .. is it the same agressor who was hitting the bid, but CME/IQfeed classify it as an offer agressor because the bid/Ask went down, and they calculate the agressor flag as 'last price at bid' or 'last price at ask' (and in this case the agressor field is useless as we can calculate it easily (redundant for data storage purpose).

3. Can it be explained otherwise if we had nanoseconds? as the timestamp might actually be different to the nanosecond?

Hello Pierre845

We will look into your inquiry, we should have an answer for you within 24 hours

Thanks
Todd

May we get a date and a more specific timestamp and symbol so that we can research this?

---

Stephen Shockey
Senior Customer Support Representative and Product Support Specialist

DTN
800-779-7299
support@iqfeed.net

Sure, please see the attached.
It's rounded (3055.8 is actually 3055.75)

May I get the symbol please?

---

Stephen Shockey
Senior Customer Support Representative and Product Support Specialist

DTN
800-779-7299
support@iqfeed.net

yes, it's in column 'Symbol' of the file I uploaded: @ESH20

I am sorry I did not see the uploaded file. I appreciate all the information. I will post a comment as soon as our research is complete.

---

Stephen Shockey
Senior Customer Support Representative and Product Support Specialist

DTN
800-779-7299
support@iqfeed.net

We unbundle all incoming trades from the CME, thus the multiple trades with the same Tick ID and timestamp.

The other 2 trades with separate Tick ID's and Aggressor flags with the same time stamp did occur as seen as we use the timestamp sent from the exchange and do not assign one.

The aggressor flag shows if the trade occurred at the bid or at the ask, that is what we can discern from the aggressor.

---

Stephen Shockey
Senior Customer Support Representative and Product Support Specialist

DTN
800-779-7299
support@iqfeed.net

Thanks Stephen, so do you mean that in essence the Agressor Field is useless? indeed in that case we would just need to comapre Bid/Ask/Last Price and work it out without the need of another data field.

Nevertheless, the fact that Trade TickID 272584027 occurs at the same price AND with the same timestamp, without being bundled by CME (I understand you unbundle, which creates same TickIDs), makes me wonder if this is an actual new trade (which would make the timestamp different if we had nanoseconds - could you please check that with CME maybe?),
OR
that this is the way the CME reports trades in this case, when the Bid/Ask moves but the trade is executed at Ask in this case instead of Bid (market Makers / HFT actions?), meaning the agressor code should stay to 2 and not 1.

This happens a few times during the day (>20 times), and I find that the likelihood of two traders, one bid agressor, and one ask agressor, to place their trade at the same time (to the microsecond) WHEN the Bid/Ask moves, is very low.

I would like to add a couple other comments about how TickID works:

- TickIDs are unique by ID, but also symbol and traded market. For example, there may exist a TickID 123456789 on NYSE and a TickID 123456789 on NASDAQ. It's also possible for two different symbols to have the same TickID. If you want to identify trades uniquely, you have to include the traded market and symbol as well. (However, see next item.)

- Furthermore, TickIDs are not necessarily unique by trade. Futures exchanges and OPRA can have the same TickID applied to multiple trades. This is because those exchanges "bundle" trades, meaning, they can send several trades at once. All bundled trades will have the same TickID. The IQFeed API "unbundles" them, so each trade has its own record, but they do have the same TickID. This is by design, so you can "rebundle" the trade data yourself if desired.

- TickIDs are not necessarily incremental, a certain number of digits, or anything like that.

- TickID is a 32-bit unsigned integer. It can be as large as 4,294,967,295 (commas added for clarity). Allow for this when creating a variable or database column to store them in.

---

Sincerely,
Gary Stephen
DTN IQFeed Implementation Support Specialist

I think that it would be useful to include a unique sequence ID with each tick that *would indeed* be unique (an incremental long integer if possible) across the board.

I'd be surprised if you don't already have such an ID in your databases, since all databases have such IDs which are often used as index keys. I assume that adding it to the reported tick fields should not be too difficult.

A unique sequence ID will help end-users differentiate between different ticks in their programs. It will also help users when they report a potential data problem to refer to a unique sequence id that you could immediately find in your database.

---

Yair Altman
IQML - IQFeed-MATLAB connector
https://UndocumentedMatlab.com/IQML

I am not a DTN employee; my post reflects my personal opinion

Thanks Gary_Stephen for you input on TickIDs, a refresher on how they work is always useful, even though in this case it doesn't address what I was talking about.

Indeed in the case I explained, it would actually be the opposite, as it looks as if the two trades across the Bid/Ask change should be the same one (perhaps only nanoseconds could clarify); it looks like the only reason they're different is because the Bid/Ask moved (which made the agressor code change, which I believe should stay at 2 instead of 1). As you've been working / communicating with CME, they're likely to give you an explanation.

Unless it's market making action, placing two different opposite trades (at the same time to the microsecond, but with a time hierarchy to the nanosecond), first one to get the Bid/Ask down, second to hit the Ask .. but I don't see the point.
Edited by Pierre845 on Jun 2, 2020 at 08:06 AM

Yair,

Prepare to be surprised :) - there is no unique identifier anywhere in the IQFeed system. Because of the massive volume of data DTN processes, and the number of sources it comes from, a unique identifier was never implemented. It would certainly be useful to have one, and it's been discussed many times, but adding one is not in any short-term plans.

TickID was never intended to be a unique identifier. Its purpose is to link DTN's data back to the exchange data, for verification purposes. If someone says to our support team "hey, the price of ABCD at 11:35:20 looks a bit off," we can easily look up that price via the TickID, and make sure our data matches what the exchange reported. A unique, DTN-created identifier would not help, because the exchanges wouldn't know it. Similarly, third-party software wouldn't know these IDs either, or would have to put in the development hours to implement them. So having a custom unique ID for each trade is not as useful as it may seem.

---

Sincerely,
Gary Stephen
DTN IQFeed Implementation Support Specialist