Jump to content

Welcome to the new Traders Laboratory! Please bear with us as we finish the migration over the next few days. If you find any issues, want to leave feedback, get in touch with us, or offer suggestions please post to the Support forum here.

  • Welcome Guests

    Welcome. You are currently viewing the forum as a guest which does not give you access to all the great features at Traders Laboratory such as interacting with members, access to all forums, downloading attachments, and eligibility to win free giveaways. Registration is fast, simple and absolutely free. Create a FREE Traders Laboratory account here.

cowcool

Best Strategy Backtesting Software

Recommended Posts

And from the amibroker user group:

-----------------------------------------

Hello All,

 

I have just completed a few days of testing several modeling/backtesting

programs. I thought, perhaps, the other members of the list might find the

results useful; as I am new here, hopefully this can serve as my first

productive contribution.

 

I do a lot of testing & modeling on (a) daily bars (looking to execute intraday

trades) and (b) tick data (for short-term trades). In the case of the first, I

need to create signals on a daily series but execute orders against a 1M

intraday series. So, in order to test an idea over five years of data, I have to

do it over 5Y of 1M data. This gets into performance issues when you start

wanting to run a test on several issues and want to run multiple revisions of

the test. As for testing on tick data, I am sure you are familiar (at least

conceptually) with the performance issues there.

 

I took a look at NinjaTrader, TradeStation, OpenQuant & AmiBroker. I created an

ASCII file of 5Y of EURUSD, USDJPY, GBPUSD, AUDUSD & USDCAD 1M data. In each

program, I ran a simple EMA crossover test (10/50). It was an obnoxious test,

resulting in +300,000 trades, but it was easy to implement and was a good stress

test. What I wanted to do was see if I could get: a) reliable (e.g.

reproducible) results from a single-security test and b) a test of the five FX

pairs as a portfolio (again, in a reliable/reproducible manner).

 

1. Ninja: I use Ninja daily to scalp with and execute some short-term system

code. I had dim hopes for the backtesting since I am familiar with the program -

Ninja really is an execution platform first and an analytical platform second.

The results were more or less what I expected: I could get it to test one issue

with reproducible results at an okay speed (about 3:00M) but it would start to

go into fits when I ran it against all five at once. The results of the five

issue test would vary from instance to instance - it would usually show the

results for the first 3 issues correctly, but on the last 2 it would suffer some

kind of memory issue and give me numbers that were totally off. In one pass, it

even managed to corrupt itself and I had to reload all the data.

 

2. TradeStation: I have a lot of time invested in TradeStation and I was already

familar with it's problems - mainly, that over a large test set, TS will return

different test results. I have talked with TS support and posted on the message

board about this, but I never got anyone interested in what I found to be a

critical issue. The results of this test were as expected: I could not get two

results to match. Any time I would refresh data from the TS data servers and run

the test again, I would get a different result. Sometimes it was as much as test

1 being -$190K and test 2 being +$74K. I do not understand how anyone can use

this tool for successful modeling if they are testing over a large dataset;

just making up a number would have been as useful. I even exported and imported

the data to ensure that it wasn't an issue with the TS data servers. Same

inconsistency. I couldn't test all five pairs together since TS does not do

portfolio backtesting. As

for time of a single test, it is hard to tell with TS as to what, exactly, it

is waiting for/trying to do at any given moment, but I would say it would

usually take about 2/4M per test (although, I have no idea what it was doing in

that 2/4M since the results it returned seemed random at times).

 

3. OpenQuant: This platform looked interesting. I set it up, imported the data

and ran a test. After 20M I noticed that the first EURUSD 1M test was at 10%. I

closed it down and uninstalled it. The performance was just not going to work

for me.

 

4. AmiBroker: You can likely guess the results as I am here as a new member. AB

was able to test EURUSD in about 30 seconds and was able to do the portfolio of

all 5 pairs in about 2M. No matter how many times I ran the test, the numbers it

would return were the same. This fact is pretty crucial to my work. I understand

that data will have errors in it, but I at least need my modeling software to

consistently return results (which AB did). I even created new databases and

populated them with the same data - all test results returned were the same.

 

As you all know, AB was, by far, the cheapest option out of all of the above.

-----------------------------------------------------------

So, nothing to add, just stick to your software and leave the edge to me.

 

Regards

 

Traders37

Share this post


Link to post
Share on other sites
To the O.P....

You don't want to run a live auto strategy on a buggy platform (ninja) or one with bad data (TS).

 

Tradestation has bad data?

 

(I've been very happy with data quality and success of auto-strategies in Tradestation)

 

Thanks.

 

Uli

Edited by Uli Schmuli
Error in post

Share this post


Link to post
Share on other sites
general reading material
800Mb memory perhaps is not enough for NT. you measured computation speed of MC to the windows scratch disk speed really..

 

Tams, could you please tell - you mentioned TT is now supported - can I connect MC to the Guardian (I have XTrader) or do I need connection to FIX server (my broker has no FIX)?

Share this post


Link to post
Share on other sites
800Mb memory perhaps is not enough for NT. you measured computation speed of MC to the windows scratch disk speed really..

 

Tams, could you please tell - you mentioned TT is now supported - can I connect MC to the Guardian (I have XTrader) or do I need connection to FIX server (my broker has no FIX)?

 

 

you better post the question at the TSSupport forum,

they are the authority.

Share this post


Link to post
Share on other sites
Tradestation has bad data?

 

(I've been very happy with data quality and success of auto-strategies in Tradestation)

 

Thanks.

 

Uli

 

Hi Uli:

In backtesting with TradeStation, I have gotten vastly different results before and after deleting my cache. That was running a test on a stock symbol. As the veteran TS user "Goose" says: "The data is always suspect."

 

I don't know if other data sources are more reliable. Is there a thread on "most reliable data providers" for the different types of securities (stocks, futures, forex, options)? Reliable data is the foundation for everything else.

Share this post


Link to post
Share on other sites

I've looked at most packages over the last few years - did an in depth review 2 1/2 years ago for my employer at the time.

 

NeoTicker (the one I got into most) turned out to be a total dog - slow and cumbersome.

AB keeps crashing one me randomly.

Most software can't backtest scans which given that my first trading system was a scanning system I've pretty much given up on other peoples stuff and I'm writing my own stuff in Excel, VBA, VB6, C++ and possibly Smalltalk MT, and no doubt if I get statisically technical I'll use R and MatLab. (I'd probably use Ninja or OpenQuant as Bots if need be). I also use QCollector to download data almost-realtime.

 

At least writing the stuff myself I'm not constrained to other peoples ideas, and it does force me to understand my trading systems.

 

Anyone else using Excel as their main system?

 

If so does any one have a great way of pulling in history into Excel / code?

 

 

--DM

Share this post


Link to post
Share on other sites
I've looked at most packages over the last few years - did an in depth review 2 1/2 years ago for my employer at the time.

 

NeoTicker (the one I got into most) turned out to be a total dog - slow and cumbersome.

AB keeps crashing one me randomly.

Most software can't backtest scans which given that my first trading system was a scanning system I've pretty much given up on other peoples stuff and I'm writing my own stuff in Excel, VBA, VB6, C++ and possibly Smalltalk MT, and no doubt if I get statisically technical I'll use R and MatLab. (I'd probably use Ninja or OpenQuant as Bots if need be). I also use QCollector to download data almost-realtime.

 

At least writing the stuff myself I'm not constrained to other peoples ideas, and it does force me to understand my trading systems.

 

Anyone else using Excel as their main system?

 

If so does any one have a great way of pulling in history into Excel / code?

 

 

--DM

 

 

 

What software you use depends, to a large degree, is dictated by what instrument you trade and the frequencies of your trades...

 

Excel has its place in the game... their charting is actually quite good. It will be a surprise to most people that their charting is very capable and very sophisticated.

 

 

There is a 3rd party add-on that can easily pull EOD data into Excel... the name escapes me at the moment. I will post it if my Alzheimer improves.

Share this post


Link to post
Share on other sites

Dangermouse.

we built a quite intensive excel portfolio backtester. The problem was it takes a long time to run through (45minutes) but the advantage is that you can actually see behind the scenes and also being excel its completely flexible for the user.

The big disadvantage we found was in the data downloading every day became a pain.

Also when building a portfolio testing system you need to ensure the data is all matched up accurately.

We spent a lot of time getting that data as accurate as possible - using futures it was a pain due to the rolling.

What we found now that works far easier is to use the systems that handle data better - eg; esignal, multicharts etc. And run through the signals/alerts . then we get those trades into excel - from there we use excel to really analyse the trades.

Disclaimer - we are not fully automated - while i would like to be, we rather use the indicators to show us possible trades, and we test those for how often they work, what happens when they do and dont work, and then we apply context as a discretionary input.

I am finding Multicharts works pretty well for this - while it does have its issues - its very flexible and it seems the guys there are actively working to be able to link it to excel using more of a cut and paste basis. This would make using their scans great without needing a lot of programming.

Share this post


Link to post
Share on other sites

Hey...

 

When you say we - you work for a fund?

 

"problem...45 minutes" - I do most of my spreadsheet work in VBA or if that's not fast enough VB6 (compiled VBA) or if that's not fast enough C++, for that very reason.

 

Interesting about MultiCharts getting better at Excel linkage. Can it output trades and stops, indicator values at setup time etc for trade system analysis later on?

 

Also can MultiCharts backtest a scanning trading strategy? E.g. trade the best performer from a basket?

 

I'm curious about how people link to Excel. I can use YLoader (yloader.com) to download EOD quotes onto disk, as CSV. I have a COM DLL to load timeseries from disk into arrays in VBA (and into Excel if needed), which if there was demand I'm pondering making a version available as freeware - it does all the memory management so a large timeseries database on disk can be accessed easily without blowing up Excel's 1GB limit.

 

Do you (or does anyone) use intraday bars in Excel?

 

I'm thinking of Sierra charts (due to market profile functionality) - any comments anyone?

 

 

 

-- DM

Share this post


Link to post
Share on other sites
...it does all the memory management so a large timeseries database on disk can be accessed easily without blowing up Excel's 1GB limit.

...

-- DM

 

 

the newer Excel does not have the limit.

Share this post


Link to post
Share on other sites

"you can use EasyLanguage (PRINT) to export/extract anything/anytime to a csv file." - useful to know. I was analysising someone else's trade results and they said they couldn't get out the initial stop or the ATR - so I was restricted in the analysis.

 

the newer Excel does not have the limit - there's still the 2GB limit that 32-bit windows enforces (unless you play with the registry). My intraday database is much bigger than 2GB so having the memory management done for me by the DLL makes it much easier to use...

 

Is there any generic ways to read CSVs - other than the cumbersome import dialog - for let's say I have 100 results and want to summarise them in XL?

 

Cheers

 

-- DM

Share this post


Link to post
Share on other sites
...Is there any generic ways to read CSVs - other than the cumbersome import dialog - for let's say I have 100 results and want to summarise them in XL?

 

Cheers

 

-- DM

 

 

there is a 3rd party add-on that can do it easily... but I don't recall the name right now.

 

 

you can check this code to see if it helps:

 

http://www.traderslaboratory.com/forums/f46/dom-excel-ib-5900.html

10440d1241448157-dom-excel-ib-dom2.gif

89

Share this post


Link to post
Share on other sites

Hi DM -

"we" is myself, and a person i employed to make the coding for excel more efficient.

I know far more about trading than coding.

you asked....

Also can MultiCharts backtest a scanning trading strategy? E.g. trade the best performer from a basket?

Not that i can gather. Thats one of the reasons we used excel. it allowed us to rank instruments for our system/idea testing.

It sounds like you know a fair bit of programming, i would suggest trialling MC - there is a 30 day free trial. There is plenty of info about the language it uses on the web - there are some fantastic people on this and others forum that can help. MC is very flexible and a bit of a pain to learn at the start, but due to this flexibility it seems to be improving all the time.

Given what you have talked about I think you will find it a good system to tie in with your excel. Let it do all the data management, and excel do the sorting.

Share this post


Link to post
Share on other sites

I am on the TS platform and the back testing for the emini is deplorable. You can test at 100% and lose your shirt live. In addition, over night a great strategy will go from 1200.00 profit to 550.00 for two days testing on the weekend with no new data. Go figure.

Just my opinion

Share this post


Link to post
Share on other sites
I am on the TS platform and the back testing for the emini is deplorable. You can test at 100% and lose your shirt live. In addition, over night a great strategy will go from 1200.00 profit to 550.00 for two days testing on the weekend with no new data. Go figure.

Just my opinion

 

 

there is a method to everything...

Share this post


Link to post
Share on other sites
I am on the TS platform and the back testing for the emini is deplorable. You can test at 100% and lose your shirt live. In addition, over night a great strategy will go from 1200.00 profit to 550.00 for two days testing on the weekend with no new data. Go figure.

Just my opinion

 

More detail would be helpful if you are interested in learning how to make TS work for you, instead of just complaining about it. Your comment contains no information useful for troubleshooting your issues.

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Topics

  • Posts

    • This should really be very easy, but I can't find an article or video to walk me through it. I picked 20 ticker symbols where the stocks are in a tight trading range. I got them all into one list I call "Channel". I'd like to add several indicators that apply to all, such as MACD, volume, 3 moving averages. Then I'd like to scroll through the list, adding trendlines, or horizontal lines to mark the top & bottom of the price channel for each. Then set an alarm for a breakout in each direction that indicates a breakout. Could you point me to an article or video that walks me through how to do this? ...or give me the steps? Thank you, RichardV2, Experienced stock trader back before the Internet was invented.😁
    • The Economic Proscription of U.S. Farmers by China Maybe Forever   Similar to a black eye on the face, it’s placing an indelible imprint. The retaliatory levies by China over U.S. commodity producers, such as soybeans, which seem to be forever. The moment such happens for the market it becomes irreversible.   It’s a dread numerous farmers from North Dakota to Mississippi have recognized for as far back as last year. They worry that they’ve put millions in soybean development on account of China. Since Chinese focus is now transferred towards Brazil rather, that market might be gone forever.   Once the confidence merchants have in the U.S. declines as a steady provider because of the trade dispute, the more vital its important for them to support and further broaden other avenues.   The developing danger for American agribusiness presently is that a great part of the piece of the overall industry lost throughout the year will be hard or difficult to win back at any point shortly, the Boston Consulting Group said in a detailed analysis discharged on Wednesday.   This is for the most part because of long term contracts that are regularly recorded among purchasers and sellers, contingent upon the item. The lesson from the analysis shows that U.S. farmers need to turn out to be less reliant on China, and simply trust in the best concerning those customers organizing a rebound sooner or later.   For the time being, China is going to Australia, Brazil, New Zealand, Russia, and also for its domestic producers as an option in contrast to American developed crops and animal proteins.   From the detailed analysis: “The risk that U.S. agribusinesses may for all time lose foreign market share of the overall industry isn’t only hypothetical. In past trade disputes, for example, one with China including beef, the US has not recaptured its lost share. As a result of the increase of U.S. crops and food materials more costly than other choices, high duties bring down the price to merchants who plan to expand. Also, the fewer confidence merchants have in the US as a steady provider, in perspective on the potential for future trade disputes, the more important it progresses toward becoming for them to support and further expand. After some time, merchants could loosen up complex associations with suppliers from the U.S.”   China Receives Blames for the Pressure And this is so because China is important to American farmers. China purchased $19.5 billion in U.S. agricultural items as of 2017, representing 14% of exports of farm produce, in light of BCS analysis. In July 2018, China slammed a 25% levy on U.S. agricultural items.   Exports at that point declined by an incredible 53% for the year. While exports to China have declined also for this year, over past years free fall.   There is another motivation behind why some China customers may not come back to the U.S. China is extending its very own crop acreage, particularly for soybeans. After some time, China will turn out to be progressively independent. Except if request increases generously, China will purchase its very own soybeans, regulating export development and under control in any case.   “Individuals in the business were in a condition of cheerfulness, believing that a bargain would soon be reached,” says Michael McAdoo, associate, and related executive for BCS in Montreal. “Our analysis demonstrates that regardless of whether there is a bargain, there is worry that a similar volume won’t return. They need to try different markets,” he declared.   Source: https://learn2.trade 
    • Trade Dispute Responsible for China’s Overwhelming Gold Purchase Rate   China has included more than 100 tons of gold to its stores since it continued purchasing in December, fortifying its position as one of the significant authority collectors as national banks load up on the valuable metal.   The People’s Bank of China grabbed progressively gold a month ago, raising reserves to 62.64 million ounces in September from 62.45 million in August, as per information on its site. In tonnage terms, the most recent inflow sums 5.9 tons and comes in as an expansion of about 99.8 tons over the earlier nine months.   Bullion hit the most noteworthy in over six years in September as more slow development, the trade dispute and rate reductions prodded financial specialist request. National banks have been significant purchasers as well, particularly in developing markets. Administrative demands will probably proceed as protectionist strategies and geopolitical concerns add to the request, as forecasted by Suki Cooper, the valuable metals investigator at Standard Chartered Bank.   “With the stressed partnerships with the U.S., China requires support against its enormous possessions of the dollar, and gold serves that capacity,” said Howie Lee, a financial specialist at Singapore-based Oversea-Chinese Banking Corp. “As China turns into a superpower in its very own right, I anticipate progressively gold-purchases.”   China’s High Gold Appetite The PBOC’s continuos running of bullion-purchasing has come against the difficult setting of the trade dispute with the U.S. furthermore, a stamped lull in development at home. While high-level discussions are set to continue in Washington this week, Chinese authorities are flagging they’re progressively hesitant to consent to an expansive bargain.   Spot gold spiked to as much as 0.4% to $1,511.31 an ounce on Monday and exchanged at $1,505.84 in early London exchange. While the value declined 3.2% in September, they remain high at 17% this year. The PBOC information was discharged at the end of the week. Alongside China, Russia has additionally been including generous amounts of bullion. In the initial half-year, national banks overall got 374.1 tons, supporting the overall gold request to a three-year high, the World Gold Council declared.   While a tenth straight month of amassing, shows an unfaltering purchasing trend for the PBOC, China has in the past gone for significant stretches without uncovering moves for its gold possessions. At the point the national bank declared a 57% bounce in savings to 53.3 million ounces in mid-2015, that was the first update in quite a while.   Source: https://learn2.trade   
    • GBPJPY Reverses Its Sell-Off Around the Level at 130.75  OCTOBER 9, 2019  Azeez Mustapha  No Comments   GBPJPY Price Analysis – October 9 In the prior session, the pair closed lower for the second day in a row, but currently, the GBPJPY displays a weakness further downside of the pair while retaining its wider medium-term outlook by temporal reversal on the level at 130.75.   Key Levels Resistance Levels: 148.66, 137.80, 135.774 Support Levels: 130.75, 128.68, 126.54   GBPJPY Long term Trend: Bearish In the bigger picture, the GBPJPY consolidation structure is still forming from the technical support zone on the level at 126.54 low.   A further upward move may be recorded towards the level at 146.57 and 148.66 in an extension where its resistance is glaring before completing the structure. However, the overall trend remains bearish while displaying an intact downtrend in the medium and long-term.   GBPJPY Short term Trend: Bearish On the 4-hour time frame, its price is trading narrowly between the moving average 5 and 13 close to the key technical support level at 130.44.   As it is presently, the intraday bias in GBPJPY remains on the downside at this point where a corrective rebound from the level at 126.54 low should have completed. Meanwhile, its 4-hour RSI is bearish and pointing lower suggesting further weakness.   Source: https://learn2.trade 
    • USDCHF Breaks Below Its near Term Support Zone on the Level at 0.9926 but Recovers Abruptly USDCHF Price Analysis – October 8 The FX pair breaks below the horizontal zone on the level at 0.9926 but reverses again after recovering from its early selling pressure. The USDCHF was able to find buyers again around the level at 0.9908.   Key Levels   Resistance Levels: 1.0231, 1.0126, 1.0015   Support Levels: 0.9897, 0.9870, 0.9843   USDCHF Long term Trend: Ranging The price of the pair has moved back towards the moving average of 5 and 13 areas on the level at 0.9950. This area requires to be broken to give buyers more upside potential to move higher.   However, the decisive break of the level at 1.0231 is required to indicate bullish resumption. Meanwhile, the medium and longer-term may remain neutral first.   USDCHF Short term Trend: Bearish After trending downwards to about 50 pips lower after the open, the forex pair managed to reverse during the session as bulls took control and may exit the day above its opening price.   The USDCHF’s pull back from the level at 1.0015 extends lower today but stays well above the lower horizontal zone on the level at 0.9843 support. While still in a long-term uptrend, the short trends have turned bearish already.   Source: https://learn2.trade   
×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.