Join the 80,000 other DTN customers who enjoy the fastest, most reliable data available. There is no better value than DTN!

(Move your cursor to this area to pause scrolling)




"I like you guys better than *******...much more stable and a whole lot fewer issues." - Comment from Philip
"I've been using Neoticker RT with IQFeed for two months, and I'm very happy with both of the products (I've had IQFeed for two years with very few complaints). The service from both companies is exceptional." - Comment from Public Forum
"Can I get another account from you? I am tired of ******* going down so often" - Comment from George
"IQ feed is brilliant. The support is mind-bending. What service!" - Comment from Public Forum Post
"I use IQ Feed, Great stuff as far as data analysis information, storage and retrieval is concerned." - Comment from Public Forum
"I am keeping IQFeed, much better reliabilty than *******. I may refer a few other people in the office to switch as well." - Comment from Don
"For anyone considering using DTN.IQ for a data feed, my experience with the quality of data and the tech support has been very positive." - Comment from Public Forum
"I just wanted to say how happy I am with your service. I was able to download the API docs last week and I was able to replicate Interactive Brokers historical bar queries and realtime bar queries over the weekend. That was about one of the fastest integrations that I've ever done and it works perfectly!!!!" - Comment from Jason via Email
"Version 4.0.0.2 has been working well for me and I appreciate that it is now a much tighter client to work with. I feel I can go to press with my own application and rely on a stable platform" - Comment from David in IA.
"IQFeed version 4 is a real screamer compared to anything else I have seen." - Comment from Tom
Home  Search  Register  Login  Recent Posts

Information on DTN's Industries:
DTN Oil & Gas | DTN Trading | DTN Agriculture | DTN Weather
Follow DTNMarkets on Twitter
DTN.IQ/IQFeed on Twitter
DTN News and Analysis on Twitter
»Forums Index »IQFeed Developer »IQFeed Developer Support »HD commands hang after 3988 symbols
Author Topic: HD commands hang after 3988 symbols (16 messages, Page 1 of 1)

sruff
-Interested User-
Posts: 5
Joined: May 12, 2005


Posted: Oct 4, 2005 11:57 AM          Msg. 1 of 16
I am finding that after about 3988 HD commands, I stop receiving data. It happens every time. It does not seem to matter which symbols they are. If I take the exact same code, and simply replace it with a HM command, it all works fine. Also the HD commands seem much slower that HM commands.

I am running the latest 4.0 release.

sruff

skunk
-DTN Evangelist-
Posts: 249
Joined: May 7, 2004


Posted: Oct 4, 2005 12:45 PM          Msg. 2 of 16
Does it always barf on the same symbol? Or does the 3988'th request always hang regardless of the symbol?

sruff
-Interested User-
Posts: 5
Joined: May 12, 2005


Posted: Oct 5, 2005 09:47 AM          Msg. 3 of 16
I don't believe it is symbol dependent. I did the test starting at an offset of 2000 in my symbol list and it still hung up at about the same time, but I will trace which symbols it hangs on and see if removing them helps.

Also I need to correct an earlier statement that the HM command worked fine. It worked longer but eventually hung at about 5000 symbols. If I let it just sit there after it hangs, IQConnect eventually crashes.

sruff

sd_cm
-Interested User-
Posts: 22
Joined: Aug 23, 2004


Posted: Oct 5, 2005 06:22 PM          Msg. 4 of 16
I have 4.00.2 hanging on many successive HT requests. The code was working perfectly with 2.3.0.1 for many months, and now after many symbol requests the socket hangs (I am using java to register the client and then TCP/IP). In fact, when this occurs I cannot access the network with any program on the IQConnect computer (I receive a "no buffer space available" network error from other programs when trying to access the network. If I restart IQConnect (by stopping it in the taskbar causing my program to re-niitiate a connection) everything is fine until several more symbols and it hangs again.

Please make this a high priority as historical downloading does not appear to work in 4.0.2 for more than a few symbols.

Thank You,
Scott

stargrazer
-DTN Guru-
Posts: 266
Joined: Jun 13, 2005

Right Here & Now


Posted: Oct 5, 2005 07:06 PM          Msg. 5 of 16
It would be helpful if you could identify how it crashes. Are you using a multi-threading model? Their new version can wreak havoc if you don't have multi-threading implemented properly. I had to do a bug-fix on my code to make it work properly.

You might also check your error code handling. I know they've made some changes in that regard. Look for messages like:

!ERROR! !NONE!
!ERROR! Invalid symbol.

I've been using 4.0.0.2 in a C# multi-threaded situation for the last week or two on 2600 symbols or so with 100 simultaneous HD's, and I have had 100% responsiveness with no crashes.

stargrazer
-DTN Guru-
Posts: 266
Joined: Jun 13, 2005

Right Here & Now


Posted: Oct 5, 2005 07:10 PM          Msg. 6 of 16
In addition, if you are using their mktsymbol.txt file as a seed, last time I looked, about 10 days ago, it has one problem line in it: there is a wrapped line that they don't seem to be fixing. If your symbol import doesn't catch that line, it may wreak havoc on the remainder of your symbol import/handling code.

sd_cm
-Interested User-
Posts: 22
Joined: Aug 23, 2004


Posted: Oct 6, 2005 01:45 PM          Msg. 7 of 16
Thanks for the help. I'll take another look at my code. This could possibly be related to http://forums.iqfeed.net/index.cfm?page=topic&topicID=656 because I had to put code in to "clean out " the socket after an invalid symbol request after migrating from 2.3.0.1.

Will post if the lockup still continues to appear in IQConnect. Thanks

DTN_Natalie_H
-DTN Evangelist-
Posts: 175
Joined: May 10, 2004

DTN Market Access, LLC.


Posted: Oct 7, 2005 10:05 AM          Msg. 8 of 16
Hi,

Please let me know if you are still seeing the lockup, and I can look into it. Thanks!

Natalie

Natalie Hannan DTN Market Access, LLC.

sruff
-Interested User-
Posts: 5
Joined: May 12, 2005


Posted: Oct 7, 2005 10:16 AM          Msg. 9 of 16
Natalie,

I am still getting the hang. For me it happens even earlier if I use only one thread. When it hangs up I need to kill IQConnect to even browse the internet. It is not symbol related because I filled my entire symbol vector with "INTC", and the problem still occurs.

However, I have not completely ruled out a problem in my code. My hope is to create a very simple, single threaded console app that repros the problem. I will then test that on a couple of machines, and a wired and wireless network. If I can repro everywhere with the simple app, I will send it to you.

Thanks for your reply!
Scott

sruff

DTN_Natalie_H
-DTN Evangelist-
Posts: 175
Joined: May 10, 2004

DTN Market Access, LLC.


Posted: Oct 7, 2005 10:42 AM          Msg. 10 of 16
Hi Scott,

That sounds good. If you can reporduce the issue, please send the app to the developer support email. Thanks!

Natalie

Natalie Hannan DTN Market Access, LLC.

sd_cm
-Interested User-
Posts: 22
Joined: Aug 23, 2004


Posted: Oct 10, 2005 09:31 AM          Msg. 11 of 16
>>scruff wrote: When it hangs up I need to kill IQConnect to even browse the internet

I've reviewed my code and still receive this exact behavior when downloading a large number of symbols with HT requests (works fine for small number of symbols < 500 or so, even if some requests are for bad or empty symbols). It is a single-threaded java and TCP/IP implementation. As I mentioned this code has worked well with 2.3.0.01 for many months. The only changes have been to accomodate new messages/ syntax on 4.0 and the synchronization issues referenced on another thread in this forum.

The program hangs in different places each time rendering all network access from that machine impossible until IQConnect is killed. Since others are having this problem I do believe the problem is with IQConnect. If scruff is unable to provide the demo mentioned I'll see what I can do to provide one. Thanks

sruff
-Interested User-
Posts: 5
Joined: May 12, 2005


Posted: Oct 10, 2005 12:41 PM          Msg. 12 of 16
I have completed my simple test app. I have tested on two machines on two different networks, both wired and wireless, and I still get the hang. My test program is a simple single threaded console app that sends the HD command for INTC asking for three days of data. It always asks for INTC to eliminate any symbol issues. It dumps to standard out the count of the request and all data read from the socket. It also should dump any errors sending or reading from the socket, but I'm not seeing any errors.

I will send a zip file containing the project to the support email.

sruff

sd_cm
-Interested User-
Posts: 22
Joined: Aug 23, 2004


Posted: Oct 12, 2005 05:27 PM          Msg. 13 of 16
Thanks for putting together the test app sruff. As I posted I am also having this problem and look forward to a DTN resolution as soon as possible.

stargrazer
-DTN Guru-
Posts: 266
Joined: Jun 13, 2005

Right Here & Now


Posted: Oct 23, 2005 07:28 PM          Msg. 14 of 16
I believe I am seeing the same thing. I'm not sure what my symbol count is, but it may be in the approx 3800 symbol area.

I run about 2500 symbols to get eod data. Because of what I think are some drop outs, I'll rerun it again. The run stops 'half way through'. I'd have to do some checking to figure out exactly how many, but 2500 plus 'half' would be around 3800.

When this happens, I simply exit my program and restart it. I don't have hard freezes like others in this thread indicate (I'm using C# with tcp/ip socket connections). Therefore, the lock ups appear to be session based.

What are the results of the sample ap that was submitted by someone earlier in this thread? Has it lead to any conclusions yet?

Based upon comments so far, and some of the errors I've seen in logging, I'm guessing there are still some subtle bugs in the new v4 code.

For the drop outs I mention earlier, I get error messags like:

CPN !ERROR! !NONE!
CSE !ERROR! Invalid symbol.

Each run servers up random errors like this. As far as I can tell, never the same symbol, hence multiple rungs to get full coverage.

These errors could be due to my reusing buffers when I shouldn't be, or it could be the v4 code not processing buffers properly. I seem to recall that my code worked back in the early days of v2, so I'm thinking it is a v4 problem (until I'm proven wrong!).

By the way, CPN and CSE are viable symbols, so I'm curious to know why I'm getting these errors.

stargrazer
-DTN Guru-
Posts: 266
Joined: Jun 13, 2005

Right Here & Now


Posted: Oct 23, 2005 07:39 PM          Msg. 15 of 16
Another comment: symbol data quality appears to get worse when I run DTN.IQ at the same time as I do my symbol traversal, and much worse when I have QuoteTracker running at the same time. What I mean, is that I get errors like the above !NONE!, etc, and that data will 'fizzle' out for some reason or another.

This would lead me to believe there are some threading/multi-access issues in the iqfeed code.

DTN_Natalie_H
-DTN Evangelist-
Posts: 175
Joined: May 10, 2004

DTN Market Access, LLC.


Posted: Oct 26, 2005 09:43 AM          Msg. 16 of 16
I believe that I have fixed this issue. I sent a test build to sruff, and it worked for him. I can email it to you for testing, if you'd like. It could be causing the invalid symbol errors that you are seeing, so why don't you take a look and let me know if it works for you. Thanks!

Natalie

Natalie Hannan DTN Market Access, LLC.
 

 

Time: Sat January 23, 2021 8:43 AM CFBB v1.2.0 31 ms.
© AderSoftware 2002-2003