Jump to content

Welcome to the new Traders Laboratory! Please bear with us as we finish the migration over the next few days. If you find any issues, want to leave feedback, get in touch with us, or offer suggestions please post to the Support forum here.

  • Welcome Guests

    Welcome. You are currently viewing the forum as a guest which does not give you access to all the great features at Traders Laboratory such as interacting with members, access to all forums, downloading attachments, and eligibility to win free giveaways. Registration is fast, simple and absolutely free. Create a FREE Traders Laboratory account here.

cowcool

Best Strategy Backtesting Software

Recommended Posts

And from the amibroker user group:

-----------------------------------------

Hello All,

 

I have just completed a few days of testing several modeling/backtesting

programs. I thought, perhaps, the other members of the list might find the

results useful; as I am new here, hopefully this can serve as my first

productive contribution.

 

I do a lot of testing & modeling on (a) daily bars (looking to execute intraday

trades) and (b) tick data (for short-term trades). In the case of the first, I

need to create signals on a daily series but execute orders against a 1M

intraday series. So, in order to test an idea over five years of data, I have to

do it over 5Y of 1M data. This gets into performance issues when you start

wanting to run a test on several issues and want to run multiple revisions of

the test. As for testing on tick data, I am sure you are familiar (at least

conceptually) with the performance issues there.

 

I took a look at NinjaTrader, TradeStation, OpenQuant & AmiBroker. I created an

ASCII file of 5Y of EURUSD, USDJPY, GBPUSD, AUDUSD & USDCAD 1M data. In each

program, I ran a simple EMA crossover test (10/50). It was an obnoxious test,

resulting in +300,000 trades, but it was easy to implement and was a good stress

test. What I wanted to do was see if I could get: a) reliable (e.g.

reproducible) results from a single-security test and b) a test of the five FX

pairs as a portfolio (again, in a reliable/reproducible manner).

 

1. Ninja: I use Ninja daily to scalp with and execute some short-term system

code. I had dim hopes for the backtesting since I am familiar with the program -

Ninja really is an execution platform first and an analytical platform second.

The results were more or less what I expected: I could get it to test one issue

with reproducible results at an okay speed (about 3:00M) but it would start to

go into fits when I ran it against all five at once. The results of the five

issue test would vary from instance to instance - it would usually show the

results for the first 3 issues correctly, but on the last 2 it would suffer some

kind of memory issue and give me numbers that were totally off. In one pass, it

even managed to corrupt itself and I had to reload all the data.

 

2. TradeStation: I have a lot of time invested in TradeStation and I was already

familar with it's problems - mainly, that over a large test set, TS will return

different test results. I have talked with TS support and posted on the message

board about this, but I never got anyone interested in what I found to be a

critical issue. The results of this test were as expected: I could not get two

results to match. Any time I would refresh data from the TS data servers and run

the test again, I would get a different result. Sometimes it was as much as test

1 being -$190K and test 2 being +$74K. I do not understand how anyone can use

this tool for successful modeling if they are testing over a large dataset;

just making up a number would have been as useful. I even exported and imported

the data to ensure that it wasn't an issue with the TS data servers. Same

inconsistency. I couldn't test all five pairs together since TS does not do

portfolio backtesting. As

for time of a single test, it is hard to tell with TS as to what, exactly, it

is waiting for/trying to do at any given moment, but I would say it would

usually take about 2/4M per test (although, I have no idea what it was doing in

that 2/4M since the results it returned seemed random at times).

 

3. OpenQuant: This platform looked interesting. I set it up, imported the data

and ran a test. After 20M I noticed that the first EURUSD 1M test was at 10%. I

closed it down and uninstalled it. The performance was just not going to work

for me.

 

4. AmiBroker: You can likely guess the results as I am here as a new member. AB

was able to test EURUSD in about 30 seconds and was able to do the portfolio of

all 5 pairs in about 2M. No matter how many times I ran the test, the numbers it

would return were the same. This fact is pretty crucial to my work. I understand

that data will have errors in it, but I at least need my modeling software to

consistently return results (which AB did). I even created new databases and

populated them with the same data - all test results returned were the same.

 

As you all know, AB was, by far, the cheapest option out of all of the above.

-----------------------------------------------------------

So, nothing to add, just stick to your software and leave the edge to me.

 

Regards

 

Traders37

Share this post


Link to post
Share on other sites
To the O.P....

You don't want to run a live auto strategy on a buggy platform (ninja) or one with bad data (TS).

 

Tradestation has bad data?

 

(I've been very happy with data quality and success of auto-strategies in Tradestation)

 

Thanks.

 

Uli

Edited by Uli Schmuli
Error in post

Share this post


Link to post
Share on other sites
general reading material
800Mb memory perhaps is not enough for NT. you measured computation speed of MC to the windows scratch disk speed really..

 

Tams, could you please tell - you mentioned TT is now supported - can I connect MC to the Guardian (I have XTrader) or do I need connection to FIX server (my broker has no FIX)?

Share this post


Link to post
Share on other sites
800Mb memory perhaps is not enough for NT. you measured computation speed of MC to the windows scratch disk speed really..

 

Tams, could you please tell - you mentioned TT is now supported - can I connect MC to the Guardian (I have XTrader) or do I need connection to FIX server (my broker has no FIX)?

 

 

you better post the question at the TSSupport forum,

they are the authority.

Share this post


Link to post
Share on other sites
Tradestation has bad data?

 

(I've been very happy with data quality and success of auto-strategies in Tradestation)

 

Thanks.

 

Uli

 

Hi Uli:

In backtesting with TradeStation, I have gotten vastly different results before and after deleting my cache. That was running a test on a stock symbol. As the veteran TS user "Goose" says: "The data is always suspect."

 

I don't know if other data sources are more reliable. Is there a thread on "most reliable data providers" for the different types of securities (stocks, futures, forex, options)? Reliable data is the foundation for everything else.

Share this post


Link to post
Share on other sites

I've looked at most packages over the last few years - did an in depth review 2 1/2 years ago for my employer at the time.

 

NeoTicker (the one I got into most) turned out to be a total dog - slow and cumbersome.

AB keeps crashing one me randomly.

Most software can't backtest scans which given that my first trading system was a scanning system I've pretty much given up on other peoples stuff and I'm writing my own stuff in Excel, VBA, VB6, C++ and possibly Smalltalk MT, and no doubt if I get statisically technical I'll use R and MatLab. (I'd probably use Ninja or OpenQuant as Bots if need be). I also use QCollector to download data almost-realtime.

 

At least writing the stuff myself I'm not constrained to other peoples ideas, and it does force me to understand my trading systems.

 

Anyone else using Excel as their main system?

 

If so does any one have a great way of pulling in history into Excel / code?

 

 

--DM

Share this post


Link to post
Share on other sites
I've looked at most packages over the last few years - did an in depth review 2 1/2 years ago for my employer at the time.

 

NeoTicker (the one I got into most) turned out to be a total dog - slow and cumbersome.

AB keeps crashing one me randomly.

Most software can't backtest scans which given that my first trading system was a scanning system I've pretty much given up on other peoples stuff and I'm writing my own stuff in Excel, VBA, VB6, C++ and possibly Smalltalk MT, and no doubt if I get statisically technical I'll use R and MatLab. (I'd probably use Ninja or OpenQuant as Bots if need be). I also use QCollector to download data almost-realtime.

 

At least writing the stuff myself I'm not constrained to other peoples ideas, and it does force me to understand my trading systems.

 

Anyone else using Excel as their main system?

 

If so does any one have a great way of pulling in history into Excel / code?

 

 

--DM

 

 

 

What software you use depends, to a large degree, is dictated by what instrument you trade and the frequencies of your trades...

 

Excel has its place in the game... their charting is actually quite good. It will be a surprise to most people that their charting is very capable and very sophisticated.

 

 

There is a 3rd party add-on that can easily pull EOD data into Excel... the name escapes me at the moment. I will post it if my Alzheimer improves.

Share this post


Link to post
Share on other sites

Dangermouse.

we built a quite intensive excel portfolio backtester. The problem was it takes a long time to run through (45minutes) but the advantage is that you can actually see behind the scenes and also being excel its completely flexible for the user.

The big disadvantage we found was in the data downloading every day became a pain.

Also when building a portfolio testing system you need to ensure the data is all matched up accurately.

We spent a lot of time getting that data as accurate as possible - using futures it was a pain due to the rolling.

What we found now that works far easier is to use the systems that handle data better - eg; esignal, multicharts etc. And run through the signals/alerts . then we get those trades into excel - from there we use excel to really analyse the trades.

Disclaimer - we are not fully automated - while i would like to be, we rather use the indicators to show us possible trades, and we test those for how often they work, what happens when they do and dont work, and then we apply context as a discretionary input.

I am finding Multicharts works pretty well for this - while it does have its issues - its very flexible and it seems the guys there are actively working to be able to link it to excel using more of a cut and paste basis. This would make using their scans great without needing a lot of programming.

Share this post


Link to post
Share on other sites

Hey...

 

When you say we - you work for a fund?

 

"problem...45 minutes" - I do most of my spreadsheet work in VBA or if that's not fast enough VB6 (compiled VBA) or if that's not fast enough C++, for that very reason.

 

Interesting about MultiCharts getting better at Excel linkage. Can it output trades and stops, indicator values at setup time etc for trade system analysis later on?

 

Also can MultiCharts backtest a scanning trading strategy? E.g. trade the best performer from a basket?

 

I'm curious about how people link to Excel. I can use YLoader (yloader.com) to download EOD quotes onto disk, as CSV. I have a COM DLL to load timeseries from disk into arrays in VBA (and into Excel if needed), which if there was demand I'm pondering making a version available as freeware - it does all the memory management so a large timeseries database on disk can be accessed easily without blowing up Excel's 1GB limit.

 

Do you (or does anyone) use intraday bars in Excel?

 

I'm thinking of Sierra charts (due to market profile functionality) - any comments anyone?

 

 

 

-- DM

Share this post


Link to post
Share on other sites
...it does all the memory management so a large timeseries database on disk can be accessed easily without blowing up Excel's 1GB limit.

...

-- DM

 

 

the newer Excel does not have the limit.

Share this post


Link to post
Share on other sites

"you can use EasyLanguage (PRINT) to export/extract anything/anytime to a csv file." - useful to know. I was analysising someone else's trade results and they said they couldn't get out the initial stop or the ATR - so I was restricted in the analysis.

 

the newer Excel does not have the limit - there's still the 2GB limit that 32-bit windows enforces (unless you play with the registry). My intraday database is much bigger than 2GB so having the memory management done for me by the DLL makes it much easier to use...

 

Is there any generic ways to read CSVs - other than the cumbersome import dialog - for let's say I have 100 results and want to summarise them in XL?

 

Cheers

 

-- DM

Share this post


Link to post
Share on other sites
...Is there any generic ways to read CSVs - other than the cumbersome import dialog - for let's say I have 100 results and want to summarise them in XL?

 

Cheers

 

-- DM

 

 

there is a 3rd party add-on that can do it easily... but I don't recall the name right now.

 

 

you can check this code to see if it helps:

 

http://www.traderslaboratory.com/forums/f46/dom-excel-ib-5900.html

10440d1241448157-dom-excel-ib-dom2.gif

89

Share this post


Link to post
Share on other sites

Hi DM -

"we" is myself, and a person i employed to make the coding for excel more efficient.

I know far more about trading than coding.

you asked....

Also can MultiCharts backtest a scanning trading strategy? E.g. trade the best performer from a basket?

Not that i can gather. Thats one of the reasons we used excel. it allowed us to rank instruments for our system/idea testing.

It sounds like you know a fair bit of programming, i would suggest trialling MC - there is a 30 day free trial. There is plenty of info about the language it uses on the web - there are some fantastic people on this and others forum that can help. MC is very flexible and a bit of a pain to learn at the start, but due to this flexibility it seems to be improving all the time.

Given what you have talked about I think you will find it a good system to tie in with your excel. Let it do all the data management, and excel do the sorting.

Share this post


Link to post
Share on other sites

I am on the TS platform and the back testing for the emini is deplorable. You can test at 100% and lose your shirt live. In addition, over night a great strategy will go from 1200.00 profit to 550.00 for two days testing on the weekend with no new data. Go figure.

Just my opinion

Share this post


Link to post
Share on other sites
I am on the TS platform and the back testing for the emini is deplorable. You can test at 100% and lose your shirt live. In addition, over night a great strategy will go from 1200.00 profit to 550.00 for two days testing on the weekend with no new data. Go figure.

Just my opinion

 

 

there is a method to everything...

Share this post


Link to post
Share on other sites
I am on the TS platform and the back testing for the emini is deplorable. You can test at 100% and lose your shirt live. In addition, over night a great strategy will go from 1200.00 profit to 550.00 for two days testing on the weekend with no new data. Go figure.

Just my opinion

 

More detail would be helpful if you are interested in learning how to make TS work for you, instead of just complaining about it. Your comment contains no information useful for troubleshooting your issues.

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Topics

  • Posts

    • MNST Monster Beverage stock, top of range breakout above 60.45, from Stocks to Watch at https://stockconsultant.com/?MNST
    • there is no avoiding loses to be honest, its just how the market is. you win some and hopefully more, but u do lose some. 
    • Date: 11th July 2025.   Demand For Gold Rises As Trump Announces Tariffs!   Gold prices rose significantly throughout the week as investors took advantage of the 2.50% lower entry level. Investors also return to the safe-haven asset as the US trade policy continues to escalate. As a result, investors are taking a more dovish tone. The ‘risk-off’ appetite is also something which can be seen within the stock market. The NASDAQ on Thursday took a 0.90% dive within only 30 minutes.   Trade Tensions Escalate President Trump has been teasing with new tariffs throughout the week. However, the tariffs were confirmed on Thursday. A 35% tariff on Canadian imports starting August 1st, along with 50% tariffs on copper and goods from Brazil. Some experts are advising that Brazil has been specifically targeted due to its association with the BRICS.   However, the President has not directly associated the tariffs with BRICS yet. According to President Trump, Brazil is targeting US technology companies and carrying out a ‘witch hunt’against former Brazilian President Jair Bolsonaro, a close ally who is currently facing prosecution for allegedly attempting to overturn the 2022 Brazilian election.   Although Brazil is one of the largest and fastest-growing economies in the Americas, it is not the main concern for investors. Investors are more concerned about Tariffs on Canada. The White House said it will impose a 35% tariff on Canadian imports, effective August 1st, raised from the earlier 25% rate. This covers most goods, with exceptions under USMCA and exemptions for Canadian companies producing within the US.   It is also vital for investors to note that Canada is among the US;’s top 3 trading partners. The increase was justified by Trump citing issues like the trade deficit, Canada’s handling of fentanyl trafficking, and perceived unfair trade practices.   The President is also threatening new measures against the EU. These moves caused US and European stock futures to fall nearly 1%, while the Dollar rose and commodity prices saw small gains. However, the main benefactor was Silver and Gold, which are the two best-performing metals of the day.   How Will The Fed Impact Gold? The FOMC indicated that the number of members warming up to the idea of interest rate cuts is increasing. If the Fed takes a dovish tone, the price of Gold may further rise. In the meantime, the President pushing for a 3% rate cut sparked talk of a more dovish Fed nominee next year and raised worries about future inflation.   Meanwhile, jobless claims dropped for the fourth straight week, coming in better than expected and supporting the view that the labour market remains strong after last week’s solid payroll report. Markets still expect two rate cuts this year, but rate futures show most investors see no change at the next Fed meeting. Gold is expected to finish the week mostly flat.       Gold 15-Minute Chart     If the price of Gold increases above $3,337.50, buy signals are likely to materialise again. However, the price is currently retracing, meaning traders are likely to wait for regained momentum before entering further buy trades. According to HSBC, they expect an average price of $3,215 in 2025 (up from $3,015) and $3,125 in 2026, with projections showing a volatile range between $3,100 and $3,600   Key Takeaway Points: Gold Rises on Safe-Haven Demand. Gold gained as investors reacted to rising trade tensions and market volatility. Canada Tariffs Spark Concern. A 35% tariff on Canadian imports drew attention due to Canada’s key trade role. Fed Dovish Shift Supports Gold. Growing expectations of rate cuts and Trump’s push for a 3% cut boosted the gold outlook. Gold Eyes Breakout Above $3,337.5. Price is consolidating; a move above $3,337.50 could trigger new buy signals. Always trade with strict risk management. Your capital is the single most important aspect of your trading business.   Please note that times displayed based on local time zone and are from time of writing this report.   Click HERE to access the full HFM Economic calendar.   Want to learn to trade and analyse the markets? Join our webinars and get analysis and trading ideas combined with better understanding of how markets work. Click HERE to register for FREE!   Click HERE to READ more Market news.   Michalis Efthymiou HFMarkets   Disclaimer: This material is provided as a general marketing communication for information purposes only and does not constitute an independent investment research. Nothing in this communication contains, or should be considered as containing, an investment advice or an investment recommendation or a solicitation for the purpose of buying or selling of any financial instrument. All information provided is gathered from reputable sources and any information containing an indication of past performance is not a guarantee or reliable indicator of future performance. Users acknowledge that any investment in Leveraged Products is characterized by a certain degree of uncertainty and that any investment of this nature involves a high level of risk for which the users are solely responsible and liable. We assume no liability for any loss arising from any investment made based on the information provided in this communication. This communication must not be reproduced or further distributed without our prior written permission.
    • Back in the early 2000s, Netflix mailed DVDs to subscribers.   It wasn’t sexy—but it was smart. No late fees. No driving to Blockbuster.   People subscribed because they were lazy. Investors bought the stock because they realized everyone else is lazy too.   Those who saw the future in that red envelope? They could’ve caught a 10,000%+ move.   Another story…   Back in the mid-2000s, Amazon launched Prime.   It wasn’t flashy—but it was fast.   Free two-day shipping. No minimums. No hassle.   People subscribed because they were impatient. Investors bought the stock because they realized everyone hates waiting.   Those who saw the future in that speedy little yellow button? They could’ve caught another 10,000%+ move.   Finally…   Back in 2011, Bitcoin was trading under $10.   It wasn’t regulated—but it worked.   No bank. No middleman. Just wallet to wallet.   People used it to send money. Investors bought it because they saw the potential.   Those who saw something glimmering in that strange orange coin? They could’ve caught a 100,000%+ move.   The people who made those calls weren’t fortune tellers. They just noticed something simple before others did.   A better way. A quiet shift. A small edge. An asymmetric bet.   The red envelope fixed late fees. The yellow button fixed waiting. The orange coin gave billions a choice.   Of course, these types of gains are rare. And they happen only once in a blue moon. That’s exactly why it’s important to notice when the conditions start to look familiar.   Not after the move. Not once it's on CNBC. But in the quiet build-up— before the surface breaks.   Enter the Blue Button Please read more here: https://altucherconfidential.com/posts/netflix-amazon-bitcoin-blue  Profits from free accurate cryptos signals: https://www.predictmag.com/ 
    • What These Attacks Look Like There are several ways you could get hacked. And the threats compound by the day.   Here’s a quick rundown:   Phishing: Fake emails from your “bank.” Click the link, give your password—game over.   Ransomware: Malware that locks your files and demands crypto. Pay up, or it’s gone.   DDoS: Overwhelm a website with traffic until it crashes. Like 10,000 bots blocking the door. Often used by nations.   Man-in-the-Middle: Hackers intercept your messages on public WiFi and read or change them.   Social Engineering: Hackers pose as IT or drop infected USB drives labeled “Payroll.”   You don’t need to be “important” to be a target.   You just need to be online.   What You Can Do (Without Buying a Bunker) You don’t have to be tech-savvy.   You just need to stop being low-hanging fruit.   Here’s how:   Use a YubiKey (physical passkey device) or Authenticator app – Ditch text message 2FA. SIM swaps are real. Hackers often have people on the inside at telecom companies.   Use a password manager (with Yubikey) – One unique password per account. Stop using your dog’s name.   Update your devices – Those annoying updates patch real security holes. Use them.   Back up your files – If ransomware hits, you don’t want your important documents held hostage.   Avoid public WiFi for sensitive stuff – Or use a VPN.   Think before you click – Emails that feel “urgent” are often fake. Go to the websites manually for confirmation.   Consider Starlink in case the internet goes down – I think it’s time for me to make the leap. Don’t Panic. Prepare. (Then Invest.)   I spent an hour in that basement bar reading about cyberattacks—and watching real-world systems fall apart like dominos.   The internet going down used to be an inconvenience. Now, it’s a warning.   Cyberwar isn’t coming. It’s here.   And the next time your internet goes out, it might not just be your router.   Don’t panic. Prepare.   And maybe keep a backup plan in your back pocket. Like a local basement bar with good bourbon—and working WiFi.   As usual, we’re on the lookout for more opportunities in cybersecurity. Stay tuned.   Author: Chris Campbell (AltucherConfidential) Profits from free accurate cryptos signals: https://www.predictmag.com/   
×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.