PDA

View Full Version : Can a neural network trade the markets effectively?



Martavis
02-08-2016,
I thought I'd share with you a little experiment I'm performing to see if a neural network can be trained to successfully trade an index and/or forex.

Generally speaking, neural networks are trained by comparing the output it produces with an expected result and feeding the error back into the network so it can correct itself slightly. The next time the neural net is tested the output is slightly closer to the expect result.

But herein lies the problem with teaching a neural network how to trade - who's to say the way you are teaching it is either the best or correct way. You are teaching it with your own knowledge and therefore own misconceptions and mistakes and the neural net will repeat those mistakes after training. I've seen neural nets use moving averages, entry/exit rules and other such inputs to make trading decisions. I want to get away from this.

Wouldn't it be better to let the neural network figure out how to trade for itself?

So the aim of the experiment is to start off with a completely random network and let it trade and record the results. I am not going to tell it what results I am expecting, but just reward the best results. I am going to use Darwins theory of natural selection to let the network "evolve" itself into a better trader. The network will spawn off several copies of itself with small random variations, let them all trade and take the best few networks and discard the rest. Then repeat the process until consistent and reliable results are achieved and no more improvements can be found.

I also want to see what the optimum amount of data is necessary to produce the best network - e.g. for trading on daily candles do you need only 1 week of data or several months to make an accurate prediction.
That's the theory anyway. Currently I have no idea if it will work and it may take several neural net configurations before I have some consistent data. What will be interesting is to see what kind of trading approach evolves, I'm going to start trading with daily candlesticks and it'll be interesting to see whether it decides buy and hold is the best or whether it tries to trade short term swings.

Simple steps to start with, I am going to build a direction predictor. The output of the network will be a single neuron with 2 states - either BUY or SELL. I will input a range of candlestick data into the net and count how many pips are won/lost by the time the next candlestick is completed, then the net will evaluate BUY/SELL again.

For the first experiments, networks will be kept/disgarded purely on the basis of the total number of points they achieve over the historical data. However I will weigh the points according to the index level. Gaining 1 point when the S&P was at 100 is a lot more significant than gaining one point when it was at 1200.

I'm going to train the network on daily data from the S&P from 3-Jan-1962 to 31-Dec-2006. The S&P is relatively friendly to indicators given it's large number of constituents. I am keeping data from 1-Jan-2007 aside for forward testing, I think it will be interesting to see how any evolved network will cope with the credit crunch. I'm hoping by training on such a large range of data from '62 - '07 that the net will not become too optimised, however I can't guarantee this and I'm deliberately leaving out a pretty volatile period in history to see if the network can cope with this unseen data.

Afterwards I will increase the complexity of the outputs - stating trade size, stop size and limits. I will also see if the evolved network is capable of trading other more tricky indices like the Dow Jones Industral and the Nikkei without ever having seen the data before.

If any of this is successful I will move onto shorter timeframes in forex to see if a similar network can be built.


keep tuned more will follow in the coming days..
__________________
Hoggums

Mazinsa
02-10-2016,
articulate, interesting and challenging, as always.
will follow with interest
__________________
my Opinion is never "humble "

karl6666: 'Why doesnt anybody like you R_E, is it because you shout?'

MaryProorgo
02-10-2016,
You are in for a LOT of work. At one stage I spent quite a lot of time fiddling around with TradingSolutions. It's hard to build decent ANN models. What TS does by default is construct an "optimal analog trading signal" using future knowledge according to certain parameters (eg profit 5% per trade) and trains the NN to predict this signal. It also runs the predicted signal thru a genetic optimizer to adjust he zero offset. Even with all the bells and whistles of a reasonable software package such as TS it's still hard to make any real progress.

Even if you get a reasonable correlation between the optimal signal and predicted signal, the thing still does not necessarily trade profitably. In other words even with well chosen inputs, a suitable network and demonstrably some predictive ability you can still lose money. It really is difficult stuff.

Good luck with training over 40 years of S&P data, I found that over those sorts of periods the training just doesn't converge. You can have too much training data as well as too little. Probably because of the changing nature of the market.

One other possible application for ANN is not prediction, but classification and pattern recognition. One thing I have been thinking about is pattern recognition in order book behavior in futures markets. It's hard stuff, but probably no more difficult than time series forecasting.

Anyway, good luck. Will be very interested to hear if you have any success.

MatthewVore
02-11-2016,
"I thought I'd share with you a little experiment I'm performing..............." - not so little, and very interesting methinks!

I'd also like to know the physical details of how you are putting this into practice eg kit & software assistance etc. I suspect this thread will be very illuminating if you have the time (and the brain pain hasn't been too serious) to explain your progress. I wonder if at the end of it we will be reminded of how difficult and primitive AI (artificial intelligence) still is - compared to the mk1 human brain?

Good stuff - looking forward to more, and thanks for sharing.

__________________
0007 -
"A Gentleman should not be seen before mid-morning unless he is returning home from the night before"http://static2.trade2win.com/boards/images/smilies/icon_cool.gif

MellisaCan
02-11-2016,
My university 3rd year project was on pattern recognition in neural networks so I have some experience in this. It's an idea that's been sitting in the back of my mind for a while to try out. I've already written the program to do the testing which I'm running final tests and checks on at the moment to make sure it produces sensible results.

It's my own bespoke c++ program and I'm using high/low/close data from Yahoo to start with. I know there are doubts about the accuracy of the data from yahoo but this is more of a proof of concept than an attempt to write a tradable EA at this stage.

dcraig1 - one of the purposes of this test is not to impose specific trading conditions on the neural nets such as 5% profit and optimal signals - there are no optimal paths I am training this net to. Only nets which are more profitable than it's ancestor over the training period will be allowed to spawn, I am not influencing it's style of trading.

However when I start looking at position sizing and stop losses I will impose drawdown conditions so that any net that has too large total drawdown from peak to trough will not be allowed to spawn, but I will not be influencing the "correct" way to trade on any network.

I also have deliberately chosen a large range of trading data so that the network does not become too optimised on a specific period, the theory being that if it can cope with 40+ years of trading it should have found common patterns to trade against for all of those periods and therefore be more likely to be profitable in the future.

I agree - I don't think it's going to be easy - but part of what I'm doing is also to dispell the myth that neural networks and artificial intelligence are somehow super advanced technologies that will solve all the worlds problems. Neural networks are designed to mimic brain function - so why should they be any better than a human brain. However neural networks are not affected by mood swings and emotions.

Training a large and complex network takes a long time - especially as I don't have a server farm to play with. I'll be starting with smallish networks to begin with for that reason. __________________

mhfruxbh60
02-12-2016,
The first experiment I'm doing is more of a simple test and proof of concept and will serve as a reference point for future tests. I'm going to start with a small network that is quick to train. I will only read in the last 2 days of Hi/Lo data plus the current market close and make a prediction of next days direction.

Attached shows a picture of the net - just 5 input neurons. I've added 3 hidden layers for no other reason than to be sure there are enough connections to ensure the net is capable of sufficient complexity. There is no real science behind the design.

In the uploaded spreadsheet I have uploaded test runs from 3 nets - the first is the net without any training, the 2nd is after 2 training runs and the third is the fully trained net.

The column accum Percent is the sum of all of the percent gains/losses from each daily trade. It serves as a better gauge than using pips/points only because percentages are scaled to the level the market is at. It is this result that the nets are trained against. If the net was trained against pips only it would have a bias towards recent years when the index and pip gains were larger.

Initially the net seems to sell everything - but after 2 generations it has already decided that buy and hold (with a few exceptions) is the best policy. After 7 generations the network cannot be improved upon which you can see produces only marginally better results than from generation 2. Bear in mind the net can only see the last 2 days of market data.

This net will be retained for forward and cross index testing later.

Coming next is training a net with last five days of data to see if using this extra data it can improve the results.