Originally Posted by davla  Interested in your database project, please keep up the great work.
How far have you gotten?
a few years ago.. opentick had such an offering... you might find some views on the net about them... morevoer you might find someone with the original opensource database..
however data from open tick has stopped sometime last year or 1.5 years ago..
They created a new firm called xasax Xasax Corporation and i am sure a phonecall to establish relationships might prove a great way to ocntinue their efforts on the side... maybe you as the premier opensource group to carry on their orignal opentick efforts ( which i guess failed as a comemrcial venue)..
however it might be a direction to take .. if there are a group of supporters for code and for sharing expenses of data..
if you aligned with a group of traders i am sure xasax might give you an account....
you might have to go as far as forming an LLc or something since there might be
(i guess ) stipulations that you do not re-distribute data..
however if a user is a member of the llc.. I think that would satisfy the re-distribution limitations,
Exchanges don't like people re-distributing data!!!!
They loose money.
A bit on me.. I am not a developer... however i am involved in quant trading in design and integration of intelligent systems for inst'l firms. i am always looking for a group that will promote great open source tools ofor real time and historical databases.
Rgds
Davla |
Davia...
You are SO under the wrong impression about me
Hehe.
* There is no need to form an LLC or something
I operate an IT company that just opens a trading branch, hopefully moving into that direction.
* Similar the cost of data for me is zero
I only collect data I want to have anyway
I strife into system development, and full tick data is a necessity. I basically need them for backtesting
And as I do not know what I Will trade next month.... I collect complete exchanges.
I have ot say I am not so interested in... hm.... open source per se. This is data collection for an enterprise. Being someone else open source arm would mean a commitment I am not sure I want to make at that point
Plus, seriously.... the code is more trivial. The main problem is handling the data
I am afraid I Will need a new server for the data - with another 24 discs or so
Next year
The real deal would be sharing the data - and that would not be free. Too much investment needed for that
This is a tremendous amount of stuff going through the system
Non-Redistribution is basically always for real time - and in that case... I will adhere. Point. An LLC would not protect against that anyway.... but I Think non-real-time is really valuable enough for many people for backtesting
My timeframe is starting with CME and possibly FOREX 1st of October. We have some performance issues with our data collector so far in the provider code that works great under intel, but breas down on AMD processors. I hope they fix it next week
It sucks if a CPU core (and their network code is inherently single threaded - which makes sense, as long as you only have one connection for one login) breaks down and uses way too much CPU. Totally no clue where that comes from - as I said, on Intel it works fine.
Once that is done, we will start collection on CME and - as I said - possibly FOREX, CBOT and the ICE are next on the list. I basically will add them as I see I can handle the data volume
And as I actually need them for my own operations.
Once the full feed scalability is fixed.... my collectors start working
Even if only in dump mode (write all out to a text file for later transversion). My programming focus is now on trade capture for internal reporting and getting the whole cleaning / accounting side into a database - so I can trade with another frontend and capture the trades with a software for internal reporting. Plus holding one's hand (new trader starting tomorrow after sim training - lets see how that goes) and looking for the first programmer to add full time to that project
Btw., I am 60% done with defining a binary coded market data stream format
Which WILL be open source
Incl. C# code to read it
Linear Mode
