Differences between backtests done by several software

[emiuqam]

Hi everyone,

Does software play a significant role in the quality of the backtest? MT4 is capable of doing backtests and it is free but I am wondering if other software like Amibroker or Multicharts offer additional quality and maybe other capabilities that I cannot think of. Am I missing something while using MT4? Any info on the topic is appreciated. Thank you.

[emullujogic]

When you say quality of the backtest, its only as good as the parameters or functionality of the software. I've performed manual backtests many times especially as alot of strategies are more subjective but I've tended to place more reliance on the software generated ones.
But it needs to be realistic. Ie a backtest that takes account of overall market trend is far better than an individual backtest on one instrument. One that can also "prioritise" based on a ranking, as its highly likely you may have more than buy signal on different instruments at the same time.
One that can backtest against a whole portfolio and one that can recognise different timeframes.
All of these are the qualities I look for, and lastly but also just as important is one that is easy to configure. Design your own exit strategies, partial exits and scale in or out of positions
They all play quite a large role I've found in assessing the quality of the strategy I intend to use

On the flip side, with backtesting software there is a tendency to add more and more parameter, to get the results better, leaning to optimisation
I never optimise, however curvefitting becomes inevitable the more you tweak something.

[entaituef]

Thank you for your reply malaguti.

I don't optimize any of the parameters I do backtest on to avoid curvefitting. I plan to do forward testing to my strategies as well to measure their performance better. I just wondered whether MT4 had inferior backtesting qualities because it is free. I think MT4 cannot backtest strategies that incorporate indicators from different timeframes, but I am not sure. Maybe that is the advantage of the paid backtesters.

Can you recommend any backtesters that are free? Any of them capable of walk-forward testing? I downloaded M1 data from a free source and would like to give my strategies a try in a free application. I am a student so anything free, I will take it http://static2.trade2win.com/boards/images/smilies/icon_smile.gif.

[ENURBCUNDACER]

ProRealTime is free and you can backtest. Not sure how difficult it is vs MT4, I've never got into metatrader. good luck though

[eoggjpah31]

Backtesting has limitations and is subjective.

If you use software there is often a variety of limitations and assumptions it makes in terms placing trades, how the trades are filled and the calculation of profits and losses.

I am not saying you should not back test but you must be aware that the results often don't replicate future performance.