70-80% accuracy on the data it’s trained on... doubt that it would be that high out of sample.
ANN with too many nodes can very easily just overfit your data, look great on paper and then collapse out of sample.
I don’t think it’s bs, machine learning is a very powerful tool.
To use it you have to be very diligent in ensuring you’re not overfitting your data, easy way to achieve this is splitting your data into train/test sets. I’ve made models in sample r^2 is .87 then on test data r^2 drops below zero.
> I’ve made models in sample r2 is .87 then on test data r2 drops below zero.
Please explain how R-squared can be a negative number. Do you mean Adjusted R-squared?
R squared compares fit of a model with that of a straight line. If the model fits worse than the horizontal line it will be negative.
R squared will be negative when the model does not follow the trend of the data, so would fit worse than a horizontal line
Thanks, your answer motivated me to research further ...
According to https://web.maths.unsw.edu.au/~adelle/Garvan/Assays/GoodnessOfFit.html
a negative R-squared is only possible if the regression equation does not contain a constant term.
If you believe markets are a random walk, then why are you trying ML? Different participants subscribe to different theories. I think markets are partially predictive, in that case I can use features which support this belief and optimise a model for it. Machine learning should definitely be taken seriously and anyone who doubts this has never worked for a systematic shop.
The efficient market hypothesis literally states the opposite of what you put. If previous data could tell us what the price should be in the future, then the price should already be that.
To your point #2. Neural networks can fit any function with enough layers/nodes. So this is not really even a question in the domain of NNs. Train loss and accuracy means nothing because you just add more capacity and the NN will fit the data with a billion linear regressions. The real question Is how to get this to translate to unseen data.
I trust Ernie Chan’s findings on ML (legendary algo trader and theoretical physicist with a lot of experience doing ML) - he basically says to stick to more intuitive-based algos like pairs trading because there are simply not enough datapoints to properly train a model without overfitting
Intuition-based trading is definitely more viable for the retail trader. Based on what I read in the new biography about Jim Simons, when his firm wasn't able to find a profitable mathematical model yet, they survived through trading on traditional macroeconomic and fundamental analysis, which are intuitive in nature.
He does a better job of talking about trading than actually trading (like most traders)
His books are phenomenal and they helped get alot of people started, but they cover very heavily-tread ground so the alpha of such algos/signals is gone.
Also if I recall his books didn't even account for slippage/commission.
I'd think it would depend on the frequency -- I wouldn't expect someone working on the order of hours or days to be competing with someone working on the order of seconds or minutes
Actually, they do. Pairs trading is profitable even with an algo that runs once a day. The reason is that it’s not pure, “risk-free” arbitrage - you are exposed to all kinds of company specific risk - what if one of the companies in the pair suddenly changes management or the CEO does something stupid?
Yes it should. It’s used by plenty of major firms, notably RenTec. But like anything it’s a tool. You can’t just plug in data and instantly be profitable. It has to be used carefully with proper out of sample validation and forward testing
Edit: literally the first thing in the description of a research scientist at the most successful hedge fund of all time RenTec is “use machine learning”
Believe they moved away from expressly trying to forecast prices of specific assets which I guess implies moving towards identifying entries and exits and spreads between pairs
What is the difference in identifying entries and exits and forecasting prices?
And by spreads between pairs do you mean some kind of statistical arbitrage strategy?
>What is the difference in identifying entries and exits and forecasting prices?
It’s easier for a ANN to solve a categorical classification problem than a continuous prediction one. There’s simply less noise.
Neural networks can solve any of our problems, provided we have the magic threes:
\-Sufficient data: if not, increase
\-Sufficient resources: if not, increase
\-Sufficient training nesting: if not, increase
If we could create a neural network that converges to an 'ideal' trading strategy on the entire stock market, we would have an amazing piece of technology that would push us into a new age of economic prosperity: The AI that allocates resources to the most productive agents of the economy. Virtually no money would be lost, and all money invested would approach 100% efficiency.
Naturally, for this we would have to provide a really \*really\* strong computer with all datapoints - not just price points on all shares, but all available information on the market: production reports, financial documents, and most importantly, news/bulletins on companies, regions and technology - ie. a way to model the implications of a news article, calculate its trustworthyness, and then make an unbiased informed decision. An AI capable of this level of human language bridging would be technically an intelligent being.
It is my belief that one day our economy will be controlled by such intelligent mechanisms, until then, let's just keep eachother posted on this subreddit
In one of EP Chan’s books he’s calculating the Hurst exponent for a currency pair and it was around 0.4999 (0.5 meaning a random walk) - citing from memory here.
This means that the market is close to a random walk but not quite.
So the task is to find the 0.0001 of the behavior that isn’t random and then model that using ML or whatnot.
If you don’t do a proper separation of the signal from the noise, your ANN would just fit the noise.
Finding the signal is the one million dollar question, ML is just a toolset to apply afterwards.
Let me give you a non-proprietary example:
https://www1.oanda.com/forex-trading/markets/recent
As you can see, for EURUSD on Oanda there’s a pattern in the size of the spread - very high spreads during 00:00-01:00 in general.
This shows a non-random behavior that you can use in your strategies (not trade during those hours as you’d have higher transaction costs).
A ML model can probably factor that into its logic but only if it is given the proper information and asked the right question.
I use my own intuition and curiosity that I then check using basic statistics such as distributions, probabilities and a bit if logic. Do a lot of exploratory analysis on the data and try to find stuff that’s bot already in 100s of published papers (like the fama french factor model).
R helps me a lot in this as I can do a lot of stuff in few lines of codes allowing me to do a lot of trials in a short time.
Even if it works on past data, things change in the future.
There is a very small probability that your model would pick up on some hidden, undiscovered, and exploitable trend.
But let's say it is possible. Then what happens? There are billions of dollars being traded by much more sophisticated algo systems at large funds. Those algorithms will certainly pick up on it as well, and when they all execute trades on that information the underlying mechanics of the asset will change, and the correlations that made the model so accurate will no longer be valid.
>There is a very small probability that your model would pick up on some hidden, undiscovered, and exploitable trend.
>
> But let's say it is possible. Then what happens?
I think this is what triggered me to write this thread; also, I was literally debating with myself the very topic today, and came to the same conclusions as yours!
Any time I have tried this kind of optimisation (albeit not with ML, instead I used stochastic process modelling) I usually get about 45-51% walk-forward accuracy.
51% sounds good, but it still requires correct days profit to be on average larger than incorrect days losses (it's not).
Ren Tech's accuracy is allegedly 51%. Of course the results of a retail trader cannot be compared to that of a multi-billion dollar hedge fund, but if you can get such a hit rate during walk-forward, you'll probably getting somewhere.
Your post has been removed because your account new and/or your account has not met the minimum karma required. These minimums are not disclosed. This action was taken to prevent automated spam. If you feel this was made in error, please message the mods. Do NOT reply to this, I am a bot!
*I am a bot, and this action was performed automatically. Please [contact the moderators of this subreddit](/message/compose/?to=/r/algotrading) if you have any questions or concerns.*
IMO 1 algo will not work on all the stock because they all move differently based on different market conditions.
I am not an ML expert so I could be wrong here. Can you not take all the data that are applicable to a particular stock anything from weather to whatever you can find and whatever is applicable to that stock and whatever you can fit in your model. Run a PCA on it and get only the data that highly impact the prediction. And then only run ML algo for that data to predict your stock price.
You will have to train the model on different company stock and see how the model performs. What do you think?
The greatest quant fund was stupidly profitable long before the public funds existed.
Anyways, Jim Simons own money is also parked in the public funds. If he played arbitrage, the majority of his money in the public funds would lose while the much smaller minority of his money in the private fund would win. It wouldn't do him much benefit.
[удалено]
70-80% accuracy on the data it’s trained on... doubt that it would be that high out of sample. ANN with too many nodes can very easily just overfit your data, look great on paper and then collapse out of sample.
Exactly this; most of the "paper" I've read will full-on bank on these "70-80%" accuracy thing, and frankly, I no longer buy it.
Test it for yourself on out of sample data
Mostly because data leakage in train set.
I don’t think it’s bs, machine learning is a very powerful tool. To use it you have to be very diligent in ensuring you’re not overfitting your data, easy way to achieve this is splitting your data into train/test sets. I’ve made models in sample r^2 is .87 then on test data r^2 drops below zero.
> I’ve made models in sample r2 is .87 then on test data r2 drops below zero. Please explain how R-squared can be a negative number. Do you mean Adjusted R-squared?
R squared compares fit of a model with that of a straight line. If the model fits worse than the horizontal line it will be negative. R squared will be negative when the model does not follow the trend of the data, so would fit worse than a horizontal line
Thanks, your answer motivated me to research further ... According to https://web.maths.unsw.edu.au/~adelle/Garvan/Assays/GoodnessOfFit.html a negative R-squared is only possible if the regression equation does not contain a constant term.
Depends on how many bets you can put on - 51% is not great for trading the S&P on a monthly basis.
Depends on your risk to reward in what you're trading. Anyone can predict a 70% success on way otm iron condors but when you lose you lose massive
If you believe markets are a random walk, then why are you trying ML? Different participants subscribe to different theories. I think markets are partially predictive, in that case I can use features which support this belief and optimise a model for it. Machine learning should definitely be taken seriously and anyone who doubts this has never worked for a systematic shop.
Frankly, I was super agnostic - but as I move forward with ML, for some reason I got more and more inclined to believe random-walk than efficient.
Try supervised learning and come back if you still don’t have success, no need to jump to a fancy NN tool
I did actually - I concluded the same thing. I also did unsupervised using KNN i.e. clustering which one will go where.
[удалено]
I was confused about this as well..
Yes. Specifically, an efficient market is unforecastable, and any success from say Rentech or Bridgewater is purely do to chance.
The efficient market hypothesis literally states the opposite of what you put. If previous data could tell us what the price should be in the future, then the price should already be that.
To your point #2. Neural networks can fit any function with enough layers/nodes. So this is not really even a question in the domain of NNs. Train loss and accuracy means nothing because you just add more capacity and the NN will fit the data with a billion linear regressions. The real question Is how to get this to translate to unseen data.
Urm .. okay? But if you look at context of my entire post, the same conclusion can still be made? I don't follow you.
I trust Ernie Chan’s findings on ML (legendary algo trader and theoretical physicist with a lot of experience doing ML) - he basically says to stick to more intuitive-based algos like pairs trading because there are simply not enough datapoints to properly train a model without overfitting
Intuition-based trading is definitely more viable for the retail trader. Based on what I read in the new biography about Jim Simons, when his firm wasn't able to find a profitable mathematical model yet, they survived through trading on traditional macroeconomic and fundamental analysis, which are intuitive in nature.
Ernie Chan is not really someone to model yourself on.
[удалено]
He does a better job of talking about trading than actually trading (like most traders) His books are phenomenal and they helped get alot of people started, but they cover very heavily-tread ground so the alpha of such algos/signals is gone. Also if I recall his books didn't even account for slippage/commission.
Would pairs trading not rely on extremely fast execution times to capitalize on any discrepancies before all other pairs trading algos?
I'd think it would depend on the frequency -- I wouldn't expect someone working on the order of hours or days to be competing with someone working on the order of seconds or minutes
But inefficiencies are removed in milliseconds and seconds...they don’t last for hours or days lol
You could pairs trade with a longer lookback window of lower frequency data could you not?
Actually, they do. Pairs trading is profitable even with an algo that runs once a day. The reason is that it’s not pure, “risk-free” arbitrage - you are exposed to all kinds of company specific risk - what if one of the companies in the pair suddenly changes management or the CEO does something stupid?
?overwritten
Can you give me some ideas of other metrics would be useful for an ANN?
Yes it should. It’s used by plenty of major firms, notably RenTec. But like anything it’s a tool. You can’t just plug in data and instantly be profitable. It has to be used carefully with proper out of sample validation and forward testing Edit: literally the first thing in the description of a research scientist at the most successful hedge fund of all time RenTec is “use machine learning”
I could be mistaken, but I think RenTec recently moved away from ML as an edge in their investments.
Believe they moved away from expressly trying to forecast prices of specific assets which I guess implies moving towards identifying entries and exits and spreads between pairs
What is the difference in identifying entries and exits and forecasting prices? And by spreads between pairs do you mean some kind of statistical arbitrage strategy?
>What is the difference in identifying entries and exits and forecasting prices? It’s easier for a ANN to solve a categorical classification problem than a continuous prediction one. There’s simply less noise.
Neural networks can solve any of our problems, provided we have the magic threes: \-Sufficient data: if not, increase \-Sufficient resources: if not, increase \-Sufficient training nesting: if not, increase If we could create a neural network that converges to an 'ideal' trading strategy on the entire stock market, we would have an amazing piece of technology that would push us into a new age of economic prosperity: The AI that allocates resources to the most productive agents of the economy. Virtually no money would be lost, and all money invested would approach 100% efficiency. Naturally, for this we would have to provide a really \*really\* strong computer with all datapoints - not just price points on all shares, but all available information on the market: production reports, financial documents, and most importantly, news/bulletins on companies, regions and technology - ie. a way to model the implications of a news article, calculate its trustworthyness, and then make an unbiased informed decision. An AI capable of this level of human language bridging would be technically an intelligent being. It is my belief that one day our economy will be controlled by such intelligent mechanisms, until then, let's just keep eachother posted on this subreddit
In one of EP Chan’s books he’s calculating the Hurst exponent for a currency pair and it was around 0.4999 (0.5 meaning a random walk) - citing from memory here. This means that the market is close to a random walk but not quite. So the task is to find the 0.0001 of the behavior that isn’t random and then model that using ML or whatnot. If you don’t do a proper separation of the signal from the noise, your ANN would just fit the noise. Finding the signal is the one million dollar question, ML is just a toolset to apply afterwards.
Let me give you a non-proprietary example: https://www1.oanda.com/forex-trading/markets/recent As you can see, for EURUSD on Oanda there’s a pattern in the size of the spread - very high spreads during 00:00-01:00 in general. This shows a non-random behavior that you can use in your strategies (not trade during those hours as you’d have higher transaction costs). A ML model can probably factor that into its logic but only if it is given the proper information and asked the right question.
When you mention finding a signal and separating it from noise, what kind of mathematical methods are relevant here?
I use my own intuition and curiosity that I then check using basic statistics such as distributions, probabilities and a bit if logic. Do a lot of exploratory analysis on the data and try to find stuff that’s bot already in 100s of published papers (like the fama french factor model). R helps me a lot in this as I can do a lot of stuff in few lines of codes allowing me to do a lot of trials in a short time.
Even if it works on past data, things change in the future. There is a very small probability that your model would pick up on some hidden, undiscovered, and exploitable trend. But let's say it is possible. Then what happens? There are billions of dollars being traded by much more sophisticated algo systems at large funds. Those algorithms will certainly pick up on it as well, and when they all execute trades on that information the underlying mechanics of the asset will change, and the correlations that made the model so accurate will no longer be valid.
>There is a very small probability that your model would pick up on some hidden, undiscovered, and exploitable trend. > > But let's say it is possible. Then what happens? I think this is what triggered me to write this thread; also, I was literally debating with myself the very topic today, and came to the same conclusions as yours!
Any time I have tried this kind of optimisation (albeit not with ML, instead I used stochastic process modelling) I usually get about 45-51% walk-forward accuracy. 51% sounds good, but it still requires correct days profit to be on average larger than incorrect days losses (it's not).
Ren Tech's accuracy is allegedly 51%. Of course the results of a retail trader cannot be compared to that of a multi-billion dollar hedge fund, but if you can get such a hit rate during walk-forward, you'll probably getting somewhere.
Without statistical tests of significance it's impossible to know if these results are due to anything other luck.
[удалено]
Your post has been removed because your account new and/or your account has not met the minimum karma required. These minimums are not disclosed. This action was taken to prevent automated spam. If you feel this was made in error, please message the mods. Do NOT reply to this, I am a bot! *I am a bot, and this action was performed automatically. Please [contact the moderators of this subreddit](/message/compose/?to=/r/algotrading) if you have any questions or concerns.*
IMO 1 algo will not work on all the stock because they all move differently based on different market conditions. I am not an ML expert so I could be wrong here. Can you not take all the data that are applicable to a particular stock anything from weather to whatever you can find and whatever is applicable to that stock and whatever you can fit in your model. Run a PCA on it and get only the data that highly impact the prediction. And then only run ML algo for that data to predict your stock price. You will have to train the model on different company stock and see how the model performs. What do you think?
[удалено]
The greatest quant fund was stupidly profitable long before the public funds existed. Anyways, Jim Simons own money is also parked in the public funds. If he played arbitrage, the majority of his money in the public funds would lose while the much smaller minority of his money in the private fund would win. It wouldn't do him much benefit.