Joined: Feb 23, 2009
Posted: Feb 23, 2009 08:48 AM
Msg. 1 of 2
I'd like to maintain a historical database of tick data for stocks.
Can anybody suggest a method?
I would be open to any method that might require programming, such as a Ninja Trader solution.
I would also be open to any pre-packaged software that will take over the task of populating a database with tick data, in realtime, 365 days a year, 24 hours per day (actually, within trading hours).
Joined: Oct 23, 2008
What color is an orange?
Posted: Mar 6, 2009 09:07 AM
Msg. 2 of 2
Since tick data is typically accessed sequentially (from start to end) instead of randomly (picking out a tick from the middle of at trading session) it makes sense to use a directory and file structure. One directory per trading instrument, and one file per trading session. To conserve space, we use a flyweight design pattern storing only the price and size offsets from an initial absolute price/size. So the tick file uses a "long" record that contains a timestamp, the LAST price and size, and volume, then subsequent ticks use a "short" record that contains a time offset from the previous record (in seconds), price offset (in "ticks" depending on the instrument) and a size offset. If the offsets are too large to fit into a "short" record, we write another "long" record. A day's worth of tick data is typically about 1Mb - small enough to be managed comfortably.