Data Mining Bias

[zealousteedo 10]

As a newbie to algorithmic trading, and having designed my first working algorithmic trading system (with hope of many more), I'm looking for experiential advice on controlling data mining bias (overfitting as it were).

 

I'm not looking for textbook answers, but how do YOU as a system designer actually attempt to avoid data mining bias?

 

As for myself, to get the ball rolling, on my last system, I developed my system against several different types of securities, performed out of sample testing on 20 percent of my historical data, as well as employed a modified version of White's reality check (one should note that introducing negative-return models to the RC will hinder it's effectiveness, and therefore one must remove them from the final calculation to make the test effective) to determine the usefulness of my system.

 

Any experiences would be great!

[ScottB 10]

The easiest and best way is to ask yourself if the parameters make sense and are in any way connected to observed price patterns.

 

It is tough to find usable patterns that are not curve fitted but if you use some common sense relative to the results, it can be done.

 

You have probably heard this a dozen times but money/risk management and position management are key to algorithmic trading.

 

Best Regards,

Scott

[cbaer 10]

Hi zealousteedo,

 

Sounds good your basic methodology. I found that I have to experience the

trades to see if i can actually follow the system. So I would take some time

use new live data in demo mode do quite a lot of trades and record how I feel

taking them. Then maybe start with meaningless money to see if your psychology

changes and go from there.

 

Hope that helps,

 

cbaer

[equtrader 10]

As a newbie to algorithmic trading, and having designed my first working algorithmic trading system (with hope of many more), I'm looking for experiential advice on controlling data mining bias (overfitting as it were).

 

Data mining bias is not fitting necessarily. Data mining bias is a buzzword that sells. You cannot know if you have bias until you test your model. If you try to minimize the bias you may miss important information.

 

You referred to negative-return models hindering the effectiveness of reality check. I really wonder why any trader would include negative return models. It does not make any sense what you write.

 

Anyway, reality check is not really relevant to trading systems because there is always relevance in the rules. Optimization and curve-fitting is a more important issue.

[max8ion 10]

Data mining bias is not fitting necessarily. Data mining bias is a buzzword that sells. You cannot know if you have bias until you test your model. If you try to minimize the bias you may miss important information.

 

You referred to negative-return models hindering the effectiveness of reality check. I really wonder why any trader would include negative return models. It does not make any sense what you write.

 

Anyway, reality check is not really relevant to trading systems because there is always relevance in the rules. Optimization and curve-fitting is a more important issue.

 

I think that is really what it is, and this post gives some opinion and I personally agree with it.