Join the 80,000 other DTN customers who enjoy the fastest, most reliable data available. There is no better value than DTN!

(Move your cursor to this area to pause scrolling)




"I am keeping IQFeed, much better reliabilty than *******. I may refer a few other people in the office to switch as well." - Comment from Don
"I've never had DTN go out on me since switching. ******* would go down a couple times every month when I was using them." - Comment from Bryce in AL.
"I "bracket trade" all major news releases and I have not found one lag or glitch with DTN.IQ feed. I am very comfortable with their feed under all typical news conditions (Fed releases, employment numbers, etc)." - Comment from Public Forum
"I am very pleased with the DTNIQ system for quotes and news." - Comment from Larry
"I am enjoying the feed very much - so superior to the broker provided feed I was previously using." - Comment from George
"I'm satisfied with IQFeed. It's the most reliable and fastest quote feed I have ever used. Although I'm a resident in China, it's still very fast!" - Comment from Xiaofei
"After all the anxiety I had with my previous data provider it is a relief not to have to worry about data speed and integrity." - Comment from Eamonn
"I was on the phone with a friend who uses CQG and right after the Fed announcement, CQG was as much as 30 seconds behind DTN.IQ. Some quotes were off by as much as 15-18 cents. Your feed never missed a beat." - Comment from Roger
"I was with ******* for 4 years at $230 a month, this is a huge savings for me, GOD BLESS YOU PEOPLE," - Comment from T.S. via Email
"Everything is working great ! Very impressive client. The news refreshes better and is more pertinent than the ******* feed I paid $ 100/month for. I Also like the charts a lot." - Comment from Leon
Home  Search  Register  Login  Recent Posts

Information on DTN's Industries:
DTN Oil & Gas | DTN Trading | DTN Agriculture | DTN Weather
Follow DTNMarkets on Twitter
DTN.IQ/IQFeed on Twitter
DTN News and Analysis on Twitter
Viewing User Profile for: taa_dtn
About Contact
Joined: May 7, 2004 01:04 PM
Last Post: Sep 13, 2022 01:41 PM
Last Visit: Yesterday @ 11:40 AM
Website:  
Location:
Occupation:
Interests:
AIM:
ICQ:
MSN IM:
Yahoo IM:
Post Statistics
taa_dtn has contributed to 143 posts out of 20767 total posts (0.69%) in 6,781 days (0.02 posts per day).

20 Most recent posts:
IQFeed Developer Support » Unable to connect this morning Sep 13, 2022 01:41 PM (Total replies: 1)

False alarm. Turned out to be due to an update elsewhere in the system that cause network connectivity issues.

IQFeed Developer Support » Unable to connect this morning Sep 13, 2022 12:58 PM (Total replies: 1)

Using Wine 7.12 under Feodora 36. No updates to Wine or to my software in the past several weeks. No problems until this morning. Here's what's in the log. Any ideas?

=== IQConnect Log File Opened On Tue Sep 13 10:43:38 ===
Current Log Levels,Connectivity,Information,Admin
Current IQFeed Version,6.2.0.23
STATUS Connectivity 464 0 2022-09-13 10:43:38 Initializing the login thread
STATUS Connectivity 456 0 2022-09-13 10:43:38 Creating trader account verification thread. Status idle
STATUS Information 460 0 2022-09-13 10:43:38 Unable to start Authentication Server. Error (107): Connection refused
STATUS Connectivity 452 0 2022-09-13 10:43:42 Waiting for authentication threads. Result:102 Code:183.

Data and Content Support » Missing symbol guides? May 10, 2022 05:34 PM (Total replies: 3)

Thanks! It looks like that will work nicely.

Data and Content Support » Missing symbol guides? May 9, 2022 10:04 PM (Total replies: 3)

Any suggestions?

Data and Content Support » Missing symbol guides? May 6, 2022 03:20 PM (Total replies: 3)

Hi, folks. The lists of symbol additions/changes/deletions at http://iqfeed.net/symbolguide are no longer updating. Have those moved, or is this a glitch?

Trading Tips, Tricks and Ideas » Machine learning and tick-by-tick data Jan 31, 2022 11:10 AM (Total replies: 1)

Yes, I experimented with this a few years ago. Your questions are relevant and insightful, but I don't have much useful information to offer in reply.

I haven't seen many published papers on the subject in recent years. Take that with a grain of salt, though, because I'm not looking actively enough. Possibly if the technique has been applied successfully, it hasn't been discussed in public for the obvious reasons. Hopefully someone else will reply with better information.

In general, I hit the same roadblocks you did. It's hard to choose the right network architecture (financial data isn't statistically stationary, so I wasn't able to design either recurrent or convolutional networks that were consistently successful). Raw tick-by-tick data has so much variability along so many dimensions that I suspect feature engineering is necessary, but that's a major research project in its own right. Techniques currently being used for natural language processing are probably where I'd start if I were to look at this again today.

Possibly the most fundamental problem I ran into is that it doesn't seem workable to use a scalar value to measure outcomes, so anything based on simple gradient descent is problematic. I think a practical outcome measurement must be at least three-dimensional -- it needs to include return, risk, and capital management. Arguably more, but the need for those three is easy to understand.


Hmm. I haven't run into this problem on my system (Wine 6.21 on Fedora 34). However, I'm still having issues with iqconnect failing to shut down after the last client closes, so I'm killing it explicitly and restarting early each day. Do you have any unexpected iqconnect, explorer, etc. processes lying around?

IQFeed Developer Support » Possible problem under Wine? Nov 9, 2021 12:27 PM (Total replies: 3)

Thanks for the reply, Gary.

I've written a toy app that exhibits all the correct (expected) behavior, so I'll try to nail down what's different between that and my real apps. I don't have much time available this week, so my next update here might take a while longer.

Now that I have an example working, it's clear that the problem is unrelated to the protocol and client software upgrades.

For reference, I start iqconnect.exe using fork() and execl(), passing a path to the executable and all the appropriate arguments. I wait a few seconds for it to initialize and then I connect. This has worked well for years. CrossOver has never been necessary.

My concern is that iqconnect is still running, sending data to a socket that it thinks is still connected to a client even though the client process is long gone. This is very likely to cause problems somewhere down the line. At the very least, iqconnect's CPU demands are going to get progressively larger over time as clients connect and disconnect. At the worst, it could block and be unable to accept new connections. Plus, the widget tray fills up with multiple connection manager icons, some of which don't respond to mouse clicks, which is ugly and confusing.

IQFeed Developer Support » Possible problem under Wine? Nov 8, 2021 07:39 PM (Total replies: 3)

I still have no solution to this, though a few things I've observed suggest the problem might be related to the way I start iqconnect. I've tried several different methods, and they all misbehave, but some in different ways than others. The common behavior for all of them is that iqconnect thinks it's still connected to a client even after the client (a) closes the only socket it has open to iqconnect, or (b) exits entirely.

FWIW, this problem doesn't exist when using protocol 4.9, but does with 6.2.

So, what is the recommended way to start iqconnect under Wine?

IQFeed Developer Support » Possible problem under Wine? Oct 20, 2021 06:27 PM (Total replies: 3)

I'm in the middle of a long-overdue protocol upgrade (to 6.2) and I'm seeing some odd behavior that I wonder if anyone else has encountered.

The problem is that long after my app has exited, iqconnect.exe is still running. Diagnostics.exe client stats claims the app is still connected, and still receiving data on the Level 1 port. However, I've confirmed that my app is definitely not running.

ShutdownDelayLastClient is set to a reasonable value (10 seconds), but I doubt it applies, since iqconnect.exe thinks it's still talking to a client. I see that start.exe, conhost.exe, explorer.exe, and iqconnect.exe are all still running, though. Manually stopping the feed closes everything down as expected.

All this is on Fedora 34 Linux using Wine 6.18, so that might be a factor.

Any ideas?

IQFeed Developer Support » 4/12/21 Data Outage Status Updates Apr 14, 2021 06:39 PM (Total replies: 31)

Seconding BottomFeeder's request: Is there any estimate for when the historical data will be updated?


Just FYI, today I was seeing the response '!ERROR! Unknown Server Error code 0.' during history fetch from servers in the IP range 66.112.156.0/24. This occurred repeatably. I worked around it by adding a small delay between history requests.

IQFeed Developer Support » Failure under recent version of Wine Jan 10, 2021 04:57 PM (Total replies: 4)

Follow up: Fedora recently upgraded to Wine 6.0, and IQConnect seems to be working again. I'll let you know if the problem reappears.


Back to normal performance levels today.


Hi, Gary. Thanks for the update!

Just to clarify, is the rate limit per-connection, or per-IP-address (or some other key)? Since I have multiple threads issuing requests on separate connections, there are conditions under which more than 50 requests could be issued over the course of a second for all the connections in aggregate, though not for any individual connection.

I implemented a request-rate limiter last week, just in case. Maybe the observed slowdown was unrelated to the changes at your end. We'll see what happens today.


Following up with some data from my logs. Minor correction; I see there are 7 active threads, not 8; the code says I chose that to stay under half of IQFeed's suggested limit of 15.

On 2020-12-22, my history fetch showed mean latency (time from sending request to arrival of first data record) of 0.22 seconds. Min was about 0.1, median about 0.24, max about 0.6. Mean data transfer time (from arrival of first data record to arrival of last data record) was 0.09 seconds. Min was 0, median about 0.04, max about 10.2.

So, some equities generate a lot of data. For data transfer alone, the best average completion rate I could ever achieve would be about 80 requests per second. On the other hand, some equities trade very thinly, so their transfer time is essentially zero, and the peak request rate could be extremely high. This is what I mentioned in my earlier postings.

On 2020-12-23, the mean latency was 1.22 seconds (!); min 0.4, median 1.0, max 3.5. Mean transfer time was 1.9; min 0, median 0.04, max 118 (!!!).

Things got really slow on 2020-12-23 for both latency and throughput. Maybe it was just an odd coincidence that this happened the same day we received notice of the new rate limiting algorithm. I'm curious to see what happens tomorrow.


Hi, Mathieu. I don't object to showing you the code (other than embarrassment because some is old and crufty), but it's straightforward. This is a C++ socket-based app that has existed for more than a decade being run on a capable modern desktop machine with an AT&T gigabit fiber connection and no recent changes to the code or to its environment, so an overnight factor-of-six slowdown suggests to me that the new rate-limiting on the feed might be having more effect than intended.

Until the new rate-limiting started, I was averaging 12 completed requests per second, which seems pretty reasonable to me. Some loss of performance is caused by round-trip latency (for simplicity and error recovery, as well as compliance with previous IQFeed policy, each thread keeps only one request in flight at a time). But I suspect the main reason for the low average rate is that for some equities, an entire day worth of ticks takes a significant amount of time to prepare, transmit, and process. It's not just a matter of the maximum rate at which the app is allowed to issue history requests.


Hi, Gary. I'm confused about the purpose and implementation of the new limit.

Each day after the main trading session (at 7PM Eastern) I fetch tick history for all the symbols I'm screening. Currently there are 4090 of them.

My first reaction on hearing about the new rate limit was "That's fine; I'm OK with a lower limit, though I doubt I'm hitting 50 requests per second as it is." I run 8 threads, each of which performs history requests sequentially, so there are at most 8 requests in flight at any given time, and it takes a while for each one to complete and a new one to be issued. (The data has to be received, reformatted, compressed, and written to disk before a new request is issued.)

Yesterday, for example, the history fetch took 339 seconds, for an average of about 12 completed requests per second.

Today the history fetch took 2031 seconds, for an average of 2 completed requests per second. Wow; that's quite a change. It's slower than the days when I had a 6Mbps DSL line.

My best guess is that the majority of history requests finish quickly, so they're hitting the rate limit and are being delayed by as much as a factor of 6. However, a fair number of requests take a long time to finish, which drags down the average way below the nominal rate limit. High peak rate, low average rate.

So I'm curious as to whether your intent is to limit the rate at which requests are initiated, or the bandwidth demand on the servers. If the latter, the current approach may be too drastic; my connection is idle more than 80% of the time.

IQFeed Developer Support » Failure under recent version of Wine Dec 14, 2020 06:36 PM (Total replies: 4)

Thanks! If they need more information, I'll be happy to help debug.

IQFeed Developer Support » Failure under recent version of Wine Dec 10, 2020 06:44 PM (Total replies: 4)

FYI, I've downgraded Wine to version 5.5 (the most recent older version available in the Fedora 32 repositories) and everything works correctly. This will keep me going for a while, but eventually I'll need a more solid fix.


Time: Tue November 29, 2022 5:00 AM CFBB v1.2.0 15 ms.
© AderSoftware 2002-2003