Join the 80,000 other DTN customers who enjoy the fastest, most reliable data available. There is no better value than DTN!

(Move your cursor to this area to pause scrolling)




"Everything is working great ! Very impressive client. The news refreshes better and is more pertinent than the ******* feed I paid $ 100/month for. I Also like the charts a lot." - Comment from Leon
"With HUGE volume on AAPL and RIMM for 2 days, everyone in a trading room was whining about freezes, crashes and lag with *******, RealTick, TS and Cyber. InvestorRT with IQFeed was rock solid. I mean SOLID!" - Comment from Public IRC Chat
"I was with ******* for 4 years at $230 a month, this is a huge savings for me, GOD BLESS YOU PEOPLE," - Comment from T.S. via Email
"Thanks for all of your help. Great customer service deserves to be recognized which one the reasons I've been a customer of DTN for over 10 years!" - Comment from Stuart
"If you want customer service that answers the phone, your best bet is IQFeed. I cannot stop praising them or their technical support. They are always there for you, and they are quick. I have used ****** too but the best value is IQFeed." - Comment from Public Forum
"I am enjoying the feed very much - so superior to the broker provided feed I was previously using." - Comment from George
"I started a trial a few weeks back before the market went wild. DTN.IQ didn’t miss anything and beat my other provider. I decided to stay with you because of the great service through all the volatility." - Comment from Mike
"It’s so nice to be working with real professionals!" - Comment from Len
"I've been using Neoticker RT with IQFeed for two months, and I'm very happy with both of the products (I've had IQFeed for two years with very few complaints). The service from both companies is exceptional." - Comment from Public Forum
"I just wanted to let you know how fast and easy I found it to integrate IQFeed into our existing Java code using your JNI client. In my experience, such things almost never go so smoothly - great job!" - Comment from Nate
Home  Search  Register  Login  Recent Posts

Information on DTN's Industries:
DTN Oil & Gas | DTN Trading | DTN Agriculture | DTN Weather
Follow DTNMarkets on Twitter
DTN.IQ/IQFeed on Twitter
DTN News and Analysis on Twitter
»Forums Index »Archive (2017 and earlier) »IQFeed Developer Support »Duplicate Q Messages
Author Topic: Duplicate Q Messages (1 messages, Page 1 of 1)

jwelch
-Interested User-
Posts: 1
Joined: Mar 15, 2018


Posted: Mar 15, 2018 10:38 PM          Msg. 1 of 1
TLDR; - Is it normal to expect the occasional duplicate Q message in the feed? Or are these meant to represent two genuine ticks that somehow end up with exactly the same data (including TickID?)

Details:
After doing some analysis on the feed data I receive, there are occasional duplicate tick messages (as in they have exactly the same crc64 value)

For Example, here is a summary of the duplicate lines found in a sample feed from 3/7/18.


Writing CRC Collisions to: IQFeed (3-7-2018).crc
Writing duplicate entries to: IQFeed (3-7-2018).dup
Processed 11734028 ticks
Found 2045 duplicate lines in the file
T: 0
Q: 1983
P: 2
S: 0
F: 60
CRC: 0 - No collisions

Ignoring the F and P duplicates as those might expected... here is a sample of some Q messages which had the same CRC64 value.


Duplicate Line Found on Hash [f612fb7a7fa49e97]
Line 1 [864989]: Q,HPQ,7,23.8700,-0.0800,-0.003340,366424,23.9506,23.7100,23.8600,23.8700,1100,1300,,0.2406,,23.7451,23.9500,0.0100,,,N,,,,-0.0800,0.0000,10.33,-0.970115,-0.0100,-0.0100,0.1249,1,0.010080,39086313,14,4,,9878149,,,,,1897,23.8356,877019989,N,,7,11,,200,09:44:32.512085,7,23.8700,200,09:44:32.512085,01,7,23.8700,200,09:44:32.512085,7,ba,09:44:32.595686,09:44:32.595686,03/07/2018,03/07/2018,03/07/2018,
Line 2 [864991]: Q,HPQ,7,23.8700,-0.0800,-0.003340,366424,23.9506,23.7100,23.8600,23.8700,1100,1300,,0.2406,,23.7451,23.9500,0.0100,,,N,,,,-0.0800,0.0000,10.33,-0.970115,-0.0100,-0.0100,0.1249,1,0.010080,39086313,14,4,,9878149,,,,,1897,23.8356,877019989,N,,7,11,,200,09:44:32.512085,7,23.8700,200,09:44:32.512085,01,7,23.8700,200,09:44:32.512085,7,ba,09:44:32.595686,09:44:32.595686,03/07/2018,03/07/2018,03/07/2018,
Duplicate Line Found on Hash [cada7e85403bc991]
Line 1 [866805]: Q,CTL,7,18.1000,0.0400,0.002215,501587,18.1600,17.8802,18.0900,18.1000,900,800,,0.2798,,17.9100,18.0600,0.0100,,,N,,,,0.0400,0.0000,8.19,-0.964172,-0.0100,-0.0100,0.1900,1,0.015459,19364502,14,4,,12264762,,,,,2470,18.0065,878202946,N,,7,7,,900,09:44:34.129734,19,18.1000,900,09:44:34.129734,01,19,18.1000,900,09:44:34.129734,19,ba,09:44:34.138719,09:44:34.138719,03/07/2018,03/07/2018,03/07/2018,
Line 2 [866807]: Q,CTL,7,18.1000,0.0400,0.002215,501587,18.1600,17.8802,18.0900,18.1000,900,800,,0.2798,,17.9100,18.0600,0.0100,,,N,,,,0.0400,0.0000,8.19,-0.964172,-0.0100,-0.0100,0.1900,1,0.015459,19364502,14,4,,12264762,,,,,2470,18.0065,878202946,N,,7,7,,900,09:44:34.129734,19,18.1000,900,09:44:34.129734,01,19,18.1000,900,09:44:34.129734,19,ba,09:44:34.138719,09:44:34.138719,03/07/2018,03/07/2018,03/07/2018,
Duplicate Line Found on Hash [3ee933e4adea4f37]
Line 1 [869514]: Q,T,7,36.7350,-0.1350,-0.003662,1214701,36.9800,36.5600,36.7400,36.7500,1000,1300,,0.4200,,36.5900,36.8700,0.0100,,,N,,,,-0.1350,0.0000,7.70,-0.952953,0.0100,0.0100,0.1450,1,0.011433,225609215,14,4,,17462057,,,,,5300,36.7725,878205011,N,,11,5,,100,09:44:35.459813,19,36.7350,100,09:44:35.459813,01,19,36.7350,100,09:44:35.459813,19,ba,09:44:37.923864,09:44:37.923864,03/07/2018,03/07/2018,03/07/2018,
Line 2 [869516]: Q,T,7,36.7350,-0.1350,-0.003662,1214701,36.9800,36.5600,36.7400,36.7500,1000,1300,,0.4200,,36.5900,36.8700,0.0100,,,N,,,,-0.1350,0.0000,7.70,-0.952953,0.0100,0.0100,0.1450,1,0.011433,225609215,14,4,,17462057,,,,,5300,36.7725,878205011,N,,11,5,,100,09:44:35.459813,19,36.7350,100,09:44:35.459813,01,19,36.7350,100,09:44:35.459813,19,ba,09:44:37.923864,09:44:37.923864,03/07/2018,03/07/2018,03/07/2018,


The number in square brackets e.g. 'Line 1 [#]' is a local message identifier. Clearly the messages are received very close to one another in time.

This is with the field parameters as follows:

S,CURRENT UPDATE FIELDNAMES,Symbol,Exchange ID,Last,Change,Percent Change,Total Volume,High,Low,Bid,Ask,Bid Size,Ask Size,Tick,Range,Open Interest,Open,Close,Spread,Settle,Delay,Restricted Code,Net Asset Value,Average Maturity,7 Day Yield,Extended Trading Change,Extended Trading Difference,Price-Earnings Ratio,Percent Off Average Volume,Bid Change,Ask Change,Change From Open,Market Open,Volatility,Market Capitalization,Fraction Display Code,Decimal Precision,Days to Expiration,Previous Day Volume,Open Range 1,Close Range 1,Open Range 2,Close Range 2,Number of Trades Today,VWAP,TickID,Financial Status Indicator,Settlement Date,Bid Market Center,Ask Market Center,Available Regions,Last Size,Last Time,Last Market Center,Most Recent Trade,Most Recent Trade Size,Most Recent Trade Time,Most Recent Trade Conditions,Most Recent Trade Market Center,Extended Trade,Extended Trade Size,Extended Trade Time,Extended Trade Market Center,Message Contents,Ask Time,Bid Time,Last Date,Extended Trade Date,Most Recent Trade Date


Note that TickID is included in the message, and is identical on lines which are found to have the same hash.

So, given the rather low occurrence of these duplicates (< 0.02%) this is probably not much of a concern for most analyses, but I was curious if this is expected behavior in the feed either way.

Thanks!
 

 

Time: Thu April 25, 2024 9:52 AM CFBB v1.2.0 8 ms.
© AderSoftware 2002-2003