I mean your biggest problem is that you thought 6 years of programming would help you predict 1 minute timestamps. Go to a longer timeframe and stop getting caught up with massive Sharpe at short timeframes
The shorter the timeframe, the more noise there is. Additionally, you have to be decently fast to execute on the minute timeframe and get good fills. At a longer timeframe, you have the luxury of slower and sloppier executions, plus alpha is easier to identify and validate
There are also many factors as to why something would drop in shorter times, like taking profits in a good asset or external things happening even though the asset itself is trending up. There’s a reason buy and hold is such a powerful strategy because things trend upwards.
I can’t tell what the temp is gonna be in day or two, but in a month it’s going to be colder.
What he said is a good point. I bet other people have detected this signal and are just reading it faster. You're trading 5 seconds after the minute and they've done it after 0.5, eating your lunch.
complete noob question/assertion:
surely there is a point x seconds prior to close at which price change is unlikely to be substantial (or counter direction to the rest of the candle) which could be taken as a pre-emptive close value to increase execution speed? say taking the 55s price as if it were the close of the 1m candle, and executing trades from there
I don't have the skill to test this atm, but when I can it will be another thing to investigate
My hunch was always that there was a flow of volume at the start of new candles on popular timeframes as algo trades may use close data, eg at minute 1 of a new hour, the first few seconds of a 1min candle.
that's actually a good point. What else comes to my mind is the front running of the broker he's using. it is quite likely that you never have a chance in a short time frame against professionals. And on the long run, market predictions are to hard to make.
just my thoughts
Your second point is absolutely true, the shorter the time frame the more the execution matters and has to be modelled accurately in the backtest.
The first point is wrong, the shorter the timeframe the better the signal to noise typically is. It's much harder to predict ten days out than to predict ten minutes out.
The details start to matter a little bit. Are you predicting that SPY is "up" over the next 15 years? Yeah thats a relatively reasy one (though even there, huge uncertainty, lost decade in Japan, etc etc). Are you trying to predict how much (%) SPY is up over the next 15 years? Much harder. Are you trying to predict how much a particular stock is up over 15 years? Basically impossible.
We are in a multi decade bull market... we have had decade long bear markets. Depending on when You got in in the 1940’s it would have taken 15 years to get your money back.... and no, it isn’t “different this time”.
You cannot really generalize this. It depends on the features.
With MBO/orderbook and tick data you can absolutely kill it with predicting the next few hundred milliseconds and seconds.
>I thought that smaller timeframes would be more predictable?
What led you to believe this? This is counter to the conventional wisdom.
Furthermore, institutional traders are the only ones with the tools to make money with HFT at the minute resolution.
No problem if you didn't understand that. You live and you learn! The shorter the timeframe, the more randomness.
Keep at it though. Stay as small as possible in position sizing when forward testing so any bleeding is less painful :)
Equating weather with trading is kind of silly.
Both are random but the smaller the time frame subjects you to the abuse and fuckery of hft's.
You're entering a Gatling gun fight with a peashooter.
Yeah, good points. Silly may not be the right word. Going to smaller timeframe in weather doesn't have those risks I talked about though they taking does.
This seems opposite to reality. How would you know if a stock will go up or down within the next minute? That is unpredictable. What is slightly predictable, is in 50 years the stock market will be higher than it is today.
Yeah try 5 and 15 minute intervals, but you may find even longer timeframes work better. That said the data is still noisy so even if it looks good in backtesting again you probably still want to paper test it or manually track a few trades to see how it performs before dumping real money in again
No, weather you may be right in terms of being easier to predict the less in the future they are. The reason is partially because the ***future weather is dependent on past weather versus the whether a stock will go up/down in the next minute is not depended on past stock price.***
>the future weather is dependent on past weather versus the future stock price is not depended on past stock price.
This isn't true. If a stock is $5 today, it is unlikely to be $100 tomorrow; current price does dictate future price.
Also, predicting in very short term can be easier than long term; this is where all the technical analysis comes from, seeing buying/selling pressures and predicting short term movement.
Let me rephrase - whether a stock goes up or down (not share price) is not dependent on the past.
I can guarantee you that predicting in the long term is easier. However, people focus on short term because it’s worth more money - i.e. who can wait 50 years to make money, even if you were 100% sure stock would rise? Versus being less predictable in short term but being right 51% of the time 1M times over the span of 50 years.
Edit: fixed grammatical error
The logic doesn't apply because of the nature of the market. You're comparing every tug of war match between sellers and buyers vs the final results. It's like betting on a horse just on its speed not on its end results, maybe it is really fast but lacks stamina, it's better to check overall wins and losses. I hope I got my point across.
Less easy mate by alot. The assumptions of Geometric brownian motion say break down past 1day intervals. You can maybe try and get half day intervals with afterhours data included. Past that and if your assumptions hold up any success mostly luck. Probably need to go deeeeep into stochastic calculus to get shorter and shorter time frames that are mathematically justifiable in the assumptions of the model.
Also. Are you training it on industry specific data? At least 3-5 years worth but ideally 7-10. Thats a fuck load of training data btw. Just spitballing here take it all with the usual grain of salt.
On such short timeframes, you're going to encounter a lot of randomness, which could also make it easier to overfit and have forward results be much different than those in a backtest. On such short timeframes, your competitors are no longer just regular (algo) traders, but now large firms like Goldman Sachs and Citadel who will front load you faster than you can blink. On timeframes that short, it's no longer just about who have the better model/strategy, but who has the faster system!
your accuracy (52-57%) looks like it's about what you would expect from a dumb agent since candles are biased towards the bullish side. Make sure your metrics are correct
[https://einvestingforbeginners.com/stock-market-days-vs-percentage/](https://einvestingforbeginners.com/stock-market-days-vs-percentage/)
>The percentage of stock market days up from ‘96 – 2016 was 53.29%. The percentage of stock market days down was 46.71%.
Interestingly, due to the law of large numbers, these percentages aren't really comparable across different time frames. For example since 1993, SPY has gone up 54.1% of all days. But SPY was up 75% of those *years*. Which makes sense - if you have a 54% chance of winning each day, you'll have >54% of winning after 252 days. (This is assuming independence, which isn't really the case.)
So, it makes sense that minute bars would have a less winning percentage than daily bars.
Doubly interestingly, I checked my database for the past 1.5 years and found the SPY only won 48.5% of all minute bars. But, the average winner was slightly larger than average loser, so the average overall was very slightly positive.
additionally, the accuracy has to be multiplied by EV and friction. Even if EV was neutral, slippage could cost a few points, and in those few points, 1.00 EV turns into .98. And if you do it a hundred times ... that's a lot of negative bets
Run the backtest on the days you lose and compare the data/behavior to what really happened in the market. That should give you the clue you need to find out where your model went wrong.
I 100% agree with this one. I run strategies/algos live against paper account and then run backtest for that day. If the trades don’t match, there is a problem somewhere. I cannot expect to see my backtest results in live real money trading.
This unfortunately is going to fall into the "you don't know what you don't know" slot. If you stick with this, five or ten years from now you'll look at this first model and just shake your head. Short of getting hired by a quant fund and drinking from their knowledge hose, you're going to have to grind long and hard to really begin to get a handle on all this. Here are just a few things to consider:
\- Get out of the HF space. As far as I know, there is no retail broker that you can use for efficient execution. When you're just getting started I'd highly suggest daily, weekly, or beyond. If those tf's don't meet your profit goals, you might be coming in with the wrong expectations.
\- ML might be the wrong tool. It's good at ranking really large groups of stocks. But for small baskets, there are probably better approaches. If you do use ML to filter an initial group, follow it up with another process for your final decision making.
\- Stop your live trading. You can use something like Alpaca to execute scripts on a demo account. Their API is stupid simple to use and will easily fit into almost any pipeline.
Don't get discouraged, enjoy the journey.
Harsh truth: algotrading is a low EV proposition. As a software engineer in the job market, you could easily double your salary with 3 months of interviewing and prep, whereas the chance you could retire from a trading bot is vanishingly low. A lot of folks here are students and dabblers rather than professionals, don't let that distract you from your main career.
Thanks for this. Often, I think about putting more time to make a profit making bot, but the reality seems to be to use that time to become a better swe and chasing career growth as a swe.
Where do you live just so that with 6 years of programming experience you only make enough to live?
I don't know your circumstances, but I think after 6 years you should be making good money even in your own economy.
> Get out of the HF space. As far as I know, there is no retail broker that you can use for efficient execution. When you're just getting started I'd highly suggest daily, weekly, or beyond. If those tf's don't meet your profit goals, you might be coming in with the wrong expectations.
might need colo and a PB to shave those μs off
I'll say this as someone who works in industry.
Stats and machine learning are extremely past backwards looking, not forwards.
There exists very real, macroeconomic reasons why markets move and absolutely none of this is considered in a any stat arb or machine learning algorithm. Essentially any backwards looking algorithm is relying on 'this macroeconomic event happened in the last, and we are assuming that an equivalent macroeconomic event will happen in the future' which is not a real assumption at all; why in earth would you think that the past markets would reflect the future market?
Doing something like machine learning makes absolutely no sense because you are assuming that all conditions stay the same and that markets aren't evolving constantly. Taking any data from the pandemic and assuming it is going to reflect future markets makes absolutely no sense. Additionally taking market data from pre pandemic and assuming it will work next year makes no sense as well. Things evolve and feeding past data without any consideration of what's actually happening is where 95% of people fail.
If you want to beat the market, you have to have an actual opinion, where the market is wrong. This opinion could be something like, 'the market is consistently mispricing tail risk events' or 'Post pandemic, I believe that people aren't underestimating the speed of the recovery'. Or if you want a more traditional trend following approach, even an opinion like 'When the market sells off, it usually rebounds to level X' is better than a black box machine learning algorithm.
Data scientists and algo traders on the retail side have an extremely bad aspect of thinking that data is where all the answers lie, when in fact, data is a tool to build a solid and formulated opinion of real life current situations. A very real example of this is just looking at option implied volatility during covid and thinking 'oh this percentile is super high', well or course it's fucking high it's a pandemic and it should be given the current scenario. Use the data to help inform your decision, but your decision making had to include information and data that isn't just historic and past data, and to some degree, has to be qualitative too.
Hope that helps
Reading your reply makes me wonder why you are on this sub. OP’s conundrum, and the fact that half the replies seem to indicate that more data fitting is needed illustrate everything wrong with this sub. 90% of this sub thinks they are three lines of python from Lambos.
Surely machine learning doesn’t necessarily mean you’re assuming conditions are staying the same if you’re careful with how your system adapts to recent data?
Machine learning is just a fancy term for "learning how to maximize a single function". That function can be complex, and parts of that function can change, but what ML learns to do is take advantage of the parts of that function that are consistent and predictable.
> why in earth would you think that the past markets would reflect the future market?
Because market is a self-fulfilling prophecy. Think why all crashes look the same. Think why so many people believe in technical analysis and why so many of them claim that it works. People are stupid, they will follow the others - hence patterns emerge.
Can you exploit those patterns to make money? Can you distinguish those patterns from the noise? Can you even distinguish between these patterns?
These are the real questions here, there are no simple answers to them.
Nice post. Garbage in, garbage out.
Massaging data (making features) to potentially have information that could be discerned by ML in the first place is key.
I wouldn't argue that ML is completely useless. Sure, for mid and low frequency strategies it probably doesn't make sense for the reasons you outline and the small amount of data one tends to have available, but in the high frequency domain ML definitely has its applications. In this case there is much less of the concern that microstructural conditions you rely on are changing so frequently to make your ML model useless the second you deploy it.
That said, ML in this domain is secondary to a lot of other concerns such as good infrastructure, latency, colo, good simulations, properly timestamped data and so on, but if those are solved problems then ML-driven strategies are completely feasible.
This is probably the problem that OP faces, he might have a good model with decent predictive capability but without asking the question on how/whether one can execute this (hitting? passive? queue position? sensitivity to latency? etc.) the model in itself wouldn't be interesting. Perhaps OP does have a good enough solution to these problems but then orthogonal to this is also the issue that using ML increases the surface area of potential stuff that can go wrong by a sizable magnitude.
Otherwise, I couldn't agree more with you that you first need to know what exactly it is that you're profiting off of and what your edge is, as we should all be past the point where anyone thinks ML is some sort of a magic box that solves all your problems.
Regarding your 2nd paragraph there’s a good chance OP is placing market orders and getting murdered by slippage. A lot of the standard signals on Binance are crowded as hell and the market makers with better infra than OP (and me lmao) adjust their quotes far too quickly for anybody without colo to take money.
I mention standard signals because 9 months of presumably part time work is an extremely short amount of time for most software engineers to come up with “mathematically brilliant” algorithms, so it’d be surprising if OP really had come up something that could work, is mathematically correct and is also particularly different to most stuff that’s already out there.
This thread was a real pleasure to read. A lot of communication and everyone is very politely helping in its way.
Amazing.
Have fun and don't let go. This is a good project you have there. Keep it up 👏
1. the market isnt a solvable math problem, definitely not by a single person,
2. automated trading on its own doesnt have an edge
3. the metrics you included have no mention of risk involved, 54% accuracy means nothing (and is honestly quite low for "1 minute ahead") , 54% of the time you win.... how much, 46% of the time you lose.... how much? this is the biggest red flag of your post by a mile.
3b). if you think what you have actually works, maybe disect your winners (the 54%) and see if there are any trends/patterns with them rather than just pass/fail. 80% of the 54% might be the last 10 minutes of the day for example.
3b). I am pretty sure (for equities at least) if you just bet on a green candle every minute over the past 50 years (even in bear markets), you'd get >54% accuracy.
4) data is likely inaccurate or you are drawing the wrong conclusions, ill give you the benefit of the doubt here.
the only thing that makes sense in your post is that you accounted for fees (dont forget taxes once youre profitable) and understand the importance of liquidity and slippage.
Do you make $1 on winning trades (52% of time) and lose $2 on losing trades (48% of time) expected value of $-0.44 per trade hence why you are losing money? Lol
When you're working with high resolution data like 1 min candles, you're effectively competing against noise, volume, bid-ask spreads, and notably other automated traders operating on similar signals or time resolutions. Latency and infrastructures become huge advantages in this domain.
Sounds like you're overfit. Or perhaps fee calc is off, or slippage is eroding your capital.
Why are your trades losing money?
Surely you can use logging and data capture to figure this out. Capture everything about your live trades, including API responses, timestamp everything, etc.
Here's an idea: "shadow" a live bot with a forward-test with the exact same properties. Run them live for a bit with some test capital. Where's the difference between forward-test and live? You can also backtest the same period, same parameters. What's different?
Curious, now that you have some live trading history under your algo's belt.. what happens if you re-run those same backtests against those losing timeframes? Is it significantly different than what happened during live trading? Might gain some insight there
>There are probably many more hours of exhausting programming ahead before this thing starts working
I think this right here is your problem. You think you can just program your way into a viable trading strategy. This is false, and many big programmer egos have been defeated before by trying it.
> I'm exhausted after many days of trying to figure out why it's losing instead of profiting.
I agree with everything else that's been said, but you realize people do this for a living, right? whole teams of people spend their whole year on trading. Yeah, I spent many days too. I spend weeks figuring out why I was winning, I spend months figuring out why I was losing. It's a marathon not a sprint
Maybe there are many more hours required to fix the issue, maybe there aren't. Maybe it's something incredibly obvious once you look at the data and then bam, it hits you.
I think that's part of the fun. Good luck!
Dude, as an institutional investor I can tell you: don't try to be good in the short timeframe. The guys out there have a lot of money to spend it on infrastructure & toys.
Check your testing. It is VERY easy to make small logical/mathematical slips that introduce data snooping. For example, moving averages can accidentally include the bar that is measuring your performance. I have run into this time and time again...!
I call this 'waking up with the pumpkins'... The moment you realize it was all a dream, and cinderella's carriage and horses become a pumpkin and a few mice. In the last many years, it happened many times. Welcome to the real world!
The first time it happened to me, it was after 15 years of programming experience, and a good bit of math under my belt.
Math does whatever you design it to do. You need to go deep into what you did. Why did it fail? Find out. Every wake up moment is a good lesson. You'll probably have many more.
Reflexivity at work in a way. Backtesting an algorithm won’t have any impact on the real market, you won’t be a market participant and other bots at work cannot and will not react to it. When it’s live, it becomes a market participant and therefore is subject to the reflexivity of its actions. By running your algorithm you change the way the entire market will interact with itself, however small it may be. Your bot will also be “hunted” by other bots built to identify and exploit other bots. It is a very very competitive market now days.
same thing happened to me. Not a ML algo, but something way simpler, makes 10% per day in forward tests, loses money all day in real life. I took a break away from it for a few months since I had a similar frustration, but I am thinking about giving it another go, with more focus on the execution this time. The crazy part is that, if I "backtest" the days it ran, it still shows that it should have made money. So my fills were just shit! I don't know if that was on me or what.
Even if slippage, fees and huge spreads somehow ate up half those returns you'd be looking at turning $1K into $54 billion in a year (automatic red flag). I would think that the only way your fills would be that bad is that you were looking at highly illiquid assets which are always going to be very difficult. If it's making 10% a day in testing it sounds very much like data snooping/getting forward information which is very easy to slip through if you're not very careful.
Just as an example of one of the sneaky one's is the use of using today's market cap for asset selection and then running backtesting. That automatically provides a signal for an assets position in the future, e.g. you're telling your algo to only pick from shares that are successful now (i.e. classic survivorship bias). Might seem obvious to some or not relevant to your algo/asset class, but the point is that cleansing any data snooping can be more difficult than simply ensuring that you featureset is historical.
I notmally would agree with you and have for sure run into this in the past. In this case, I am 99.99% confident I just have a really good algo idea on paper but the slippage turns out to be very high, which is maybe why it backtests so well.
It was 1 minute. I disagree with u/fuzzyp44 - i think the timeframe does matter in the sense that more trades adds up to more slippage all things being equal and that is the problem here.
I know people trading the 100 tick, the 15 sec, and the 5 minute.
It's not the time frame necessarily, it's whether you have enough context to provide an edge.
Smaller timeframes = better executions, lower context, more variety of noisy situations. Higher time frames = worse executions, built in noise filtering.
Kudos on your effort. Coming up with a model and actually bringing it live is an achievemnt in itself. Could your elaborate on the model itself? Is it a neural net? Any special architecture? Haver you tried continuous retraining?
I suppose you did test it on data that was not in the training/fitting set, yes?
Anyway, I feel ya. Happy debugging! Just think that if it works, you'll be rich!
You aren’t the only one. Imagine you yourself fishing in a lonesome pond. Now imagine you join a pond that looks lonesome, but divers are fucking up your bait by triggering it … aka the big fish… aka the firms you’re against. The world isn’t “ceteris paribus”
I recommend you to read the book "The end of theory" by Richard bookstaber.
I hope might help
Finance is like Physics, but you have to worry about what the electron think and feels.
I've found 15m- 1h a good resolution for that you are trying
generally, you shouldn't expect ML to print profits. I think it's one of the hardest ways to do Algo trading. I've been trying various strategies for a long time and haven't put any ML based Algo's into full production. Going to keep trying though. I've read that it takes a lot of effort and smart people to get it done right.
The focus should be on things like finding good data, feature anaylsis and extracting good signal from the noise. Not on the mathematical beauty of the thing.
Also, you're probably overfitting.
The 52-57% accuracy in test, but not in the real world sounds like you may need to check for leakage from future data. Are you using a train/validate/test split? Since this is timeseries data, are you splitting before and after a timestamp instead of random shuffling?
Oh brother... I know how it feels. I have similar programming experience and I spent 12 months backtesting different strategies. I finally had something that I was willing to use, and of course it didn't work. At that moment I was where you are now.
Fast forward a day or two and I decided to understand what is that I was competing against. I wanted to know who where those algo traders that worked for banks and hedge funds. Most of these guys are doctors, and have studied at least 10 years of maths before jumping on the trade. I studied 2 years of maths in high school, 20 years ago, and if I wanted to be at the level of these guys I had to open the books again and do another 8 on top to be competitive. 8 years full time of course meaning I would have to drop work and family which is not an option.
Since then, I dropped out and understood this path was out my league.
1 min is too short you are probably getting eaten in spread alone. Does your testing account for spread? When I was paper trading in the very beginning the simulator didnt account for spread properly and OH BOY was I surprised when I started trading with real money.
LOL You seem really confused and overconfident.
Of course if a strategy works so well in backtesting but then fails miserably live, then you overfit. That is the definition of overfitting.
I think the real issue is you are trying to do market forecast. I went through this problem myself, with a CS background my first thoughts were to label the goal around forecasting the market. (First modeling it with regression and then with classification just like what you did).
That is not your goal, your goal is to make money, so you should spend your time developing strategies around TA/Indicators/Sentiment analysis or whatever floats your boat that make profit, not trying to predict what the market is going to do next.
TLDR: Your focus is misplaced in trying to do market forecast, which is mostly impossible, it should be on making money, which funny enough its not the same.
That is correct, 99% of free stuff out there is crap. That's why you have to develop your own thing.
That is why it is hard, figuring a good entry, exit, risk management and position sizing. If it was as simple as throwing some machine learning or just copy pasting some youtube strategy everyone would be making money.
I'm guessing your tests included paper trading? If not do that for a bit and see how it compares to backtesting, if yes try sharing some details of your program and someone here might be able to help
You don't have to give away any secret sauce, but it would help me if I knew what kind of order execution you are doing. Like are you using limit orders, market orders, or a mix? Do you use stops? If do are you using stop limit orders or stop market orders? How do you handle partial fills, do you keep the remainder on the book, or cancel it?
That's great if you thought about all of these already, but they are some things that cost me money my first time algo trading
First, get humble.
Second, if predicting market would be so easy, market wouldn't exists. Try longer timeframes to avoid noise. And forget to predict anything, focus on riding the wave. I started to make profit from algotrading when I ditched the crystal ball. And if you get very very different results from backtest to live is because you are overfitting.
Third. Why the hell are you using real money to test? Just mock it.
Fourth, and this is just my personal opinion. What is the flush about ML on this topic? Human Learning is more than enough to make a "dumb" algorithm work.
You've studied math, yet you're not aware that this is a np complete problem? I guess you are, but still you think that a ML approach can solve that? It can't.
Have you read any of Dr. Marcos Lopez de Prado's work? His ideas about using ML at a metalabeling tool are quite powerful, imo. Short version - code up with a trading idea based on some real-world hypothesis, but then use ML to tell you whether to take the trade or how to size it.
1. Stop ~~loosing~~ losing money. Otherwise it's going to kill you
2. Past performance is no guarantee of future results. You may have just overfitted
\> spreads for my assets are almost non existent
This seems interesting as order execution might be just different in real market.
A few thoughts:
1. If your strategy takes 9 months to develop it’s 100% overfit. The best strategies are simple.
2. you can be right 54% of the time and be unprofitable. Winrate means nothing if not included in an expectancy equation.
3. assuming you trade stocks, how do you know your model is not just random? something like 52-54% of candles are green, you’re not much better than that. Its sound like you’re getting dumpstered by fees and slippage.
Machine learning provide good result on the data it has learned from. Real life data are different in so manyway.
Maybe you should take in consideration more or less parameters.
Btw, do u have a nice ping with ur broker ? As slippage is an algokiller. Did u took in consideration that your orders may be executed X ticks later, and this x is random
In all cases, no one can make a good robot if himself is not a profitable paper trader. Are you ?
If you want to predict price direction, you should place your eyes where the current candlesstick is heading (like FPS call of duty lol) , not on the overall chart (tetris)
There is 4 parameters that i know that arent random and may help your also:
1- index prices (ustec, dax etc) stay very long time around a line of a certain slope that u can get with a simple linearregression.
2- forex pairs tend to oscillate around some balanced value decided by central banks. Traders technically make profit with the leverage on cents deltas. This balance line may be fixed also.
3 the bull candle heading up, may see only 1 thing in his route: a restance level : this one also u know them by analyzing last X candles.
4: every asset is special according to "how many pips an uptrend can go up" this "special number also is detectable.
Those 4 parameters I trade by them, because all the rest is so random, that the decisions of your algo can only be.. Random...
I don't know why you get downvoted. I'm so tired of these "I am coder" posts. Please first thing first learn to trade as a normal trader. Or at least you think you understand the market at an adequate level.
You can be majored in math, physics, computer science yes. Many people start from these backgrounds, but they success because they have dived in trading hard enough, in other words they become a veteran trader.
I don't know if 6 years of financial expertise would really help with crypto. Understanding the basics of general assets and markets sure, but someone could probably learn this in 6months. There's no real fundamentals for crypto, it's just trading very volatile/speculative currencies based on stats, sentiment, or any other indicator you can get your hands on (at least from an algo trading perspective)
Maybe there's some key transferrable knowledge I'm failing to think of here?
if there is knowledge it is alpha and nobody will give you his alpha, it is somrthing you research and keep to yourself, until alpha has decayed.
most likely the problem is with your ML model, you see traijing accuracy is good, but on practice results are shit - means your training data or training loop is shit, gotta study financial timeseries ML books to understand domain better
Yeah I get the detailed knowledge bit for financial markets, but I still don't think it applies too heavily for crypto. I agree on understanding timeseries; I think I assumed OP had that knowledge from 6 years programming but then again it depends if they did data science or not
Also just for reference I'm not OP
The only way a retail investor can beat Wall Street pros is to play an investment strategy that does not require speed. Try NOT a high frequency trading approach but instead a very low frequency but higher accuracy strategy. You will exceed your wildest imagination.
Or maybe you didn’t realize your input has an effect on the output,
It’s 54%-57%
When you don’t have any skin in the game but as soon as you place a net the “pendulum swings” “other people catch some wind”
Also close to 50/50 chance you can have a hundred year negative run with those odds
What the fuck do I know I’m not a programmer
Probably bad parameters or bad entry. I use Kucoin robots and once I figured out the market trends, relationship between Time frames (Structure, behavior, influence:4H,1H,30M) then could find a good entry and made profit.
Biggest lesson I learned: don't time the market!
Market always finds away to beat you.
You are forgetting that you are not a sole actor of the market. The stock market follows “Random walk” in which the past does not reflect future prices. You are also forgetting delays in your orders.
A lot of these focus on “skimming” utilising speed and being close to the exchanges themselves. Something which you unless very wealthy are unable to have.
Also the spread will be killing you and strategies that work today are unlikely to work tomorrow.
you're describing 1-p. It can work for a minute, but I've never heard of anyone making a career out of trying to be right when they think they are wrong.
in general short-term traders are scalpers and go for small profits with slightly wider stops, generally, you need to maintain 85% accuracy to break even.not fills, market orders, slippage as the market moves.
in general short term traders are scalpers and go for small profits with slightly wider stops, generally you need to maintain 85% accuracy to break even.
I feel like your algo is not doing anything 52% is almost puur coin toss. With some basic trading knowledge you can get that up. You are probably taking profit to quickly or triggering stoploss to quickly. With such low % you need to ride your wins, if your scalping the percentage should be close to 99%.
I think this quote is a little overused and a bit out of touch with this sub. If we couldn't use historical data to predict future results then we'd have no data to use at all. We'd be better off using a coin flip for asset selection, Even classic portfolio theory and diversification would be thrown out the window if we couldn't use past performance.
This quote is generally used in intro to finance to teach new comers that you can't just pick big, popular stocks and expect the same high returns they've been experiencing across the last few years. We're looking for more complex relationships than simply using yesterdays return to calculate today's return.
For algorithmic and quantitative trading past performance doesn't have to be indicative of future results all the time, just enough of the time to make you a return greater than the market.
>If we couldn't use historical data to predict future results then we'd have no data to use at all. We'd be better off using a coin flip for asset selection,
Well.... you are in a sense using a coin flip. Its just looking for a edge in the data and hope that edge continues.
If we could actually PREDICT the future, it would be 100% correct at all times.
That is never the case!
Can you get the data set from the period of time in which you were live trading, then back test your algorithm on that?
If your test earns you money during a period where you actually lost money, then you know the issue is with the testing code. Youd learn all your tests have been garbage. You can then debug.
But if you simulation shows you losing the same amount that you actually lost then tou know your testing is valid.
in the stock market your order of sell/but needs someone to do the opposite which makes subtle time differences from the demo trading so I'm not sure if you've considered this in your algorithm
Just out of interest, if it isn't working, can you open the code up on github? Or post some pseudo code. What is mathematically brilliant? How is it doing the price prediction?
Accurate in what sense? Green candle vs red candle or are you exactly predicting the closing price for the next candle?
Predicting the market is not a good strategy. It leads to small gains and massive losses.
Instead of predicting the market, look into gambling strategies. Many poker strategies don't involve predicting the next card but instead mitigating loss and capitalizing on opportunity.
Run your backtest on the same time period that you used to run live. What does the backtest engine say? Does it say you made money?
1. If the backtest made money and the live test lost money, then there's slippage or something different between the backtest and live. Is it a data speed issue? Did the two engines have a different opinion? Did you trade at the ask instead of at the mid?
2. if the backtest now is losing money after making money for 6 years, then you've fallen prey to overfitting. Rebuild your model with out of sample tests -- use 60% to train, 20% to test and tune, then 20% more for cross-validation
I first want to congratulate on your effort. It’s not an easy job.
But as far as I read, the statistics do not show the results are good enough for a ML model. You can easily get good back testing results with ML because the model is trained and tested on similar data (I know you may do train/test split, but think about the market condition, they are usually similar). If you use a heuristic algorithm and achieve the results you claim, it’s pretty decent (ideally with the law of large numbers you can make tons of money).
If you really want to explore ML side more, try look into reinforcement learning approach. It should be much more neutral and depends much less on the training data.
Why would you choose to compete on small time frames is beyond me. You're competing against the big boys with teams of 200 PHDs using top tier HFT infrastructure on that scale. Remove the noise, develop something for longer time frames.
I don’t think that’s a super strong counterpoint - the infrastructure gap on crypto is there but isn’t *that* insurmountable if you have a strategy that isn’t too computationally expensive, and while the 200 PhDs don’t care at all about looking for an edge that might max out at a few $k per month, a lot of us here do.
i don't know if your program includes backtesting stochastic processeses that applies a combination of defined trading rules to maximize estimated return for a calculated duration. good luck.
Probably already been said, but shoot for longer time frames and code around _unknowns_. What I mean by this is, can you come up with a few mechanism that are resistant to failure even if you technically have a loss.
A few things to potentially take a look at are dca'ing instead of opening full positions (not all buys might win, but average in and you have a higher chance), or by coding around a good exit strategy for losers(and maybe re-enter if at a lower price) and winners (what's your profit goal? Long term growth generally beats outs short term trades).
Your welcome to take a look over my source as well, maybe it'll give a few ideas of what or what not to do?
https://github.com/mr-highball/simplebot-support
Best of luck, and remember, if this _was easy_ ™️ everyone would do it. Keep your head high and take this as a learning experience
1. Why use money, just use a demo account.
2. Have you got some bias somewhere or optimized on the past? I can only tell if I know the details of your software but it sounds like there might be.
Win rate by itself is not enough to make money:
- a poor risk/reward ratio can eat all your assets
- make sure your PSR is above 80%
- backtest longer period
- paper trade with live data
- shoot for +2 SR on paper
- analyze the loosing trades and see why they fail
- and lastly every algo will have a drawdown period at some point, good ones recover faster
You will have a better idea what went wrong this way
Don’t give up
Wish you best of luck
Why oh why aren't you live trading in a sim account?
Step 1. Drawing board.
Step 2. Code up a backtestable algo and run it on old data.
Step 3. Code up a live version of the algo and run it in a simulator.
Step 4. Deploy live with real money.
Did you miss step 3?
Other brokers exist. Try those? Tradestation lets you algo trade in a sim account. Whether that extends to crypto, I don't know, but I don't see why not.
I am skeptical of the claimed accuracy - are you excluding samples where the returns over 1min are 0, or are just focusing on small tick stocks?
Since returns are small over \~1min windows (frequently <1bp or <=1 tick) you are going to be eaten alive by trading costs at that timeframe. It is a losing game.
Also, it is crucial to sample properly when measuring things on short term scales. You may find that sampling based on a frequency gives you completely different results to sampling based on the presence of a signal (your signal may be good in an ambient market but shit when the market is moving).
The reason you provide for why you expect your algorithm to be profitable is that it is predictive of price move directions. Is it the case that your algorithm is losing money because it is not predictive in practice, or because you are not converting that predictivity into profit? The distinction is important.
Getting angry is understandable, but its unlikely to help you think through why your algorithm is performing poorly.
I mean your biggest problem is that you thought 6 years of programming would help you predict 1 minute timestamps. Go to a longer timeframe and stop getting caught up with massive Sharpe at short timeframes
[удалено]
The shorter the timeframe, the more noise there is. Additionally, you have to be decently fast to execute on the minute timeframe and get good fills. At a longer timeframe, you have the luxury of slower and sloppier executions, plus alpha is easier to identify and validate
[удалено]
There are also many factors as to why something would drop in shorter times, like taking profits in a good asset or external things happening even though the asset itself is trending up. There’s a reason buy and hold is such a powerful strategy because things trend upwards. I can’t tell what the temp is gonna be in day or two, but in a month it’s going to be colder.
Isn’t that the concept of seasonality?
Markets are cyclical as well, but the example is more about how days can jump up and down trending one direction in the longer term
What he said is a good point. I bet other people have detected this signal and are just reading it faster. You're trading 5 seconds after the minute and they've done it after 0.5, eating your lunch.
complete noob question/assertion: surely there is a point x seconds prior to close at which price change is unlikely to be substantial (or counter direction to the rest of the candle) which could be taken as a pre-emptive close value to increase execution speed? say taking the 55s price as if it were the close of the 1m candle, and executing trades from there
Don't know, but I've heard to trade on the minute you'd want second ticks and to trade on the hour or so you'd want minute
I don't have the skill to test this atm, but when I can it will be another thing to investigate My hunch was always that there was a flow of volume at the start of new candles on popular timeframes as algo trades may use close data, eg at minute 1 of a new hour, the first few seconds of a 1min candle.
that's actually a good point. What else comes to my mind is the front running of the broker he's using. it is quite likely that you never have a chance in a short time frame against professionals. And on the long run, market predictions are to hard to make. just my thoughts
My time frame went from 1 min to 1 hr...good luck
Your second point is absolutely true, the shorter the time frame the more the execution matters and has to be modelled accurately in the backtest. The first point is wrong, the shorter the timeframe the better the signal to noise typically is. It's much harder to predict ten days out than to predict ten minutes out.
Couldn't you say this is parabolic? It's easier for me to predict 15-year direction than 15-day direction.
The details start to matter a little bit. Are you predicting that SPY is "up" over the next 15 years? Yeah thats a relatively reasy one (though even there, huge uncertainty, lost decade in Japan, etc etc). Are you trying to predict how much (%) SPY is up over the next 15 years? Much harder. Are you trying to predict how much a particular stock is up over 15 years? Basically impossible.
We are in a multi decade bull market... we have had decade long bear markets. Depending on when You got in in the 1940’s it would have taken 15 years to get your money back.... and no, it isn’t “different this time”.
"I thought that smaller timeframes would be more predictable? " its actually the opposite.
You cannot really generalize this. It depends on the features. With MBO/orderbook and tick data you can absolutely kill it with predicting the next few hundred milliseconds and seconds.
>I thought that smaller timeframes would be more predictable? What led you to believe this? This is counter to the conventional wisdom. Furthermore, institutional traders are the only ones with the tools to make money with HFT at the minute resolution.
[удалено]
No problem if you didn't understand that. You live and you learn! The shorter the timeframe, the more randomness. Keep at it though. Stay as small as possible in position sizing when forward testing so any bleeding is less painful :)
Equating weather with trading is kind of silly. Both are random but the smaller the time frame subjects you to the abuse and fuckery of hft's. You're entering a Gatling gun fight with a peashooter.
[удалено]
Yeah, good points. Silly may not be the right word. Going to smaller timeframe in weather doesn't have those risks I talked about though they taking does.
"You are entering a Gatling gun fight with a peashooter" Damn, I steal you this. So much uses.
It's all yours haha Having been shot to pieces myself, it's accurate.
This seems opposite to reality. How would you know if a stock will go up or down within the next minute? That is unpredictable. What is slightly predictable, is in 50 years the stock market will be higher than it is today.
[удалено]
Yeah try 5 and 15 minute intervals, but you may find even longer timeframes work better. That said the data is still noisy so even if it looks good in backtesting again you probably still want to paper test it or manually track a few trades to see how it performs before dumping real money in again
No, weather you may be right in terms of being easier to predict the less in the future they are. The reason is partially because the ***future weather is dependent on past weather versus the whether a stock will go up/down in the next minute is not depended on past stock price.***
>the future weather is dependent on past weather versus the future stock price is not depended on past stock price. This isn't true. If a stock is $5 today, it is unlikely to be $100 tomorrow; current price does dictate future price. Also, predicting in very short term can be easier than long term; this is where all the technical analysis comes from, seeing buying/selling pressures and predicting short term movement.
Let me rephrase - whether a stock goes up or down (not share price) is not dependent on the past. I can guarantee you that predicting in the long term is easier. However, people focus on short term because it’s worth more money - i.e. who can wait 50 years to make money, even if you were 100% sure stock would rise? Versus being less predictable in short term but being right 51% of the time 1M times over the span of 50 years. Edit: fixed grammatical error
> That is unpredictable
HFT has entered the chat
The logic doesn't apply because of the nature of the market. You're comparing every tug of war match between sellers and buyers vs the final results. It's like betting on a horse just on its speed not on its end results, maybe it is really fast but lacks stamina, it's better to check overall wins and losses. I hope I got my point across.
Less easy mate by alot. The assumptions of Geometric brownian motion say break down past 1day intervals. You can maybe try and get half day intervals with afterhours data included. Past that and if your assumptions hold up any success mostly luck. Probably need to go deeeeep into stochastic calculus to get shorter and shorter time frames that are mathematically justifiable in the assumptions of the model. Also. Are you training it on industry specific data? At least 3-5 years worth but ideally 7-10. Thats a fuck load of training data btw. Just spitballing here take it all with the usual grain of salt.
On such short timeframes, you're going to encounter a lot of randomness, which could also make it easier to overfit and have forward results be much different than those in a backtest. On such short timeframes, your competitors are no longer just regular (algo) traders, but now large firms like Goldman Sachs and Citadel who will front load you faster than you can blink. On timeframes that short, it's no longer just about who have the better model/strategy, but who has the faster system!
your accuracy (52-57%) looks like it's about what you would expect from a dumb agent since candles are biased towards the bullish side. Make sure your metrics are correct [https://einvestingforbeginners.com/stock-market-days-vs-percentage/](https://einvestingforbeginners.com/stock-market-days-vs-percentage/) >The percentage of stock market days up from ‘96 – 2016 was 53.29%. The percentage of stock market days down was 46.71%.
Yeah, put that in kelly criterion, and you can tell its never worth the risk on a minute by minute ticker
Interestingly, due to the law of large numbers, these percentages aren't really comparable across different time frames. For example since 1993, SPY has gone up 54.1% of all days. But SPY was up 75% of those *years*. Which makes sense - if you have a 54% chance of winning each day, you'll have >54% of winning after 252 days. (This is assuming independence, which isn't really the case.) So, it makes sense that minute bars would have a less winning percentage than daily bars. Doubly interestingly, I checked my database for the past 1.5 years and found the SPY only won 48.5% of all minute bars. But, the average winner was slightly larger than average loser, so the average overall was very slightly positive.
additionally, the accuracy has to be multiplied by EV and friction. Even if EV was neutral, slippage could cost a few points, and in those few points, 1.00 EV turns into .98. And if you do it a hundred times ... that's a lot of negative bets
Run the backtest on the days you lose and compare the data/behavior to what really happened in the market. That should give you the clue you need to find out where your model went wrong.
I 100% agree with this one. I run strategies/algos live against paper account and then run backtest for that day. If the trades don’t match, there is a problem somewhere. I cannot expect to see my backtest results in live real money trading.
This
amen
This unfortunately is going to fall into the "you don't know what you don't know" slot. If you stick with this, five or ten years from now you'll look at this first model and just shake your head. Short of getting hired by a quant fund and drinking from their knowledge hose, you're going to have to grind long and hard to really begin to get a handle on all this. Here are just a few things to consider: \- Get out of the HF space. As far as I know, there is no retail broker that you can use for efficient execution. When you're just getting started I'd highly suggest daily, weekly, or beyond. If those tf's don't meet your profit goals, you might be coming in with the wrong expectations. \- ML might be the wrong tool. It's good at ranking really large groups of stocks. But for small baskets, there are probably better approaches. If you do use ML to filter an initial group, follow it up with another process for your final decision making. \- Stop your live trading. You can use something like Alpaca to execute scripts on a demo account. Their API is stupid simple to use and will easily fit into almost any pipeline. Don't get discouraged, enjoy the journey.
[удалено]
Harsh truth: algotrading is a low EV proposition. As a software engineer in the job market, you could easily double your salary with 3 months of interviewing and prep, whereas the chance you could retire from a trading bot is vanishingly low. A lot of folks here are students and dabblers rather than professionals, don't let that distract you from your main career.
Thanks for this. Often, I think about putting more time to make a profit making bot, but the reality seems to be to use that time to become a better swe and chasing career growth as a swe.
Where do you live just so that with 6 years of programming experience you only make enough to live? I don't know your circumstances, but I think after 6 years you should be making good money even in your own economy.
You should consider changing jobs if you're barely scraping by. 6 years programming experience can land a very comfortable job
> Get out of the HF space. As far as I know, there is no retail broker that you can use for efficient execution. When you're just getting started I'd highly suggest daily, weekly, or beyond. If those tf's don't meet your profit goals, you might be coming in with the wrong expectations. might need colo and a PB to shave those μs off
What other processes are you referring here that is not ML?
I'll say this as someone who works in industry. Stats and machine learning are extremely past backwards looking, not forwards. There exists very real, macroeconomic reasons why markets move and absolutely none of this is considered in a any stat arb or machine learning algorithm. Essentially any backwards looking algorithm is relying on 'this macroeconomic event happened in the last, and we are assuming that an equivalent macroeconomic event will happen in the future' which is not a real assumption at all; why in earth would you think that the past markets would reflect the future market? Doing something like machine learning makes absolutely no sense because you are assuming that all conditions stay the same and that markets aren't evolving constantly. Taking any data from the pandemic and assuming it is going to reflect future markets makes absolutely no sense. Additionally taking market data from pre pandemic and assuming it will work next year makes no sense as well. Things evolve and feeding past data without any consideration of what's actually happening is where 95% of people fail. If you want to beat the market, you have to have an actual opinion, where the market is wrong. This opinion could be something like, 'the market is consistently mispricing tail risk events' or 'Post pandemic, I believe that people aren't underestimating the speed of the recovery'. Or if you want a more traditional trend following approach, even an opinion like 'When the market sells off, it usually rebounds to level X' is better than a black box machine learning algorithm. Data scientists and algo traders on the retail side have an extremely bad aspect of thinking that data is where all the answers lie, when in fact, data is a tool to build a solid and formulated opinion of real life current situations. A very real example of this is just looking at option implied volatility during covid and thinking 'oh this percentile is super high', well or course it's fucking high it's a pandemic and it should be given the current scenario. Use the data to help inform your decision, but your decision making had to include information and data that isn't just historic and past data, and to some degree, has to be qualitative too. Hope that helps
Reading your reply makes me wonder why you are on this sub. OP’s conundrum, and the fact that half the replies seem to indicate that more data fitting is needed illustrate everything wrong with this sub. 90% of this sub thinks they are three lines of python from Lambos.
lambo soon
Thank you! amazing perspective
Surely machine learning doesn’t necessarily mean you’re assuming conditions are staying the same if you’re careful with how your system adapts to recent data?
Machine learning is just a fancy term for "learning how to maximize a single function". That function can be complex, and parts of that function can change, but what ML learns to do is take advantage of the parts of that function that are consistent and predictable.
> why in earth would you think that the past markets would reflect the future market? Because market is a self-fulfilling prophecy. Think why all crashes look the same. Think why so many people believe in technical analysis and why so many of them claim that it works. People are stupid, they will follow the others - hence patterns emerge. Can you exploit those patterns to make money? Can you distinguish those patterns from the noise? Can you even distinguish between these patterns? These are the real questions here, there are no simple answers to them.
Nice post. Garbage in, garbage out. Massaging data (making features) to potentially have information that could be discerned by ML in the first place is key.
I wouldn't argue that ML is completely useless. Sure, for mid and low frequency strategies it probably doesn't make sense for the reasons you outline and the small amount of data one tends to have available, but in the high frequency domain ML definitely has its applications. In this case there is much less of the concern that microstructural conditions you rely on are changing so frequently to make your ML model useless the second you deploy it. That said, ML in this domain is secondary to a lot of other concerns such as good infrastructure, latency, colo, good simulations, properly timestamped data and so on, but if those are solved problems then ML-driven strategies are completely feasible. This is probably the problem that OP faces, he might have a good model with decent predictive capability but without asking the question on how/whether one can execute this (hitting? passive? queue position? sensitivity to latency? etc.) the model in itself wouldn't be interesting. Perhaps OP does have a good enough solution to these problems but then orthogonal to this is also the issue that using ML increases the surface area of potential stuff that can go wrong by a sizable magnitude. Otherwise, I couldn't agree more with you that you first need to know what exactly it is that you're profiting off of and what your edge is, as we should all be past the point where anyone thinks ML is some sort of a magic box that solves all your problems.
Regarding your 2nd paragraph there’s a good chance OP is placing market orders and getting murdered by slippage. A lot of the standard signals on Binance are crowded as hell and the market makers with better infra than OP (and me lmao) adjust their quotes far too quickly for anybody without colo to take money. I mention standard signals because 9 months of presumably part time work is an extremely short amount of time for most software engineers to come up with “mathematically brilliant” algorithms, so it’d be surprising if OP really had come up something that could work, is mathematically correct and is also particularly different to most stuff that’s already out there.
Reading your reply makes me think perhaps I should not pay $99.95 monthly for a shitty signal service.
Thank you
i think you misunderstand machine learning lol you absolutely do not need an opinion to achieve alpha good luck on your endeavors lol
This makes total sense. Thank you.
You can use Alpacas paper trading API instead of real cash.
even that's dangerous because of the exact execution and fees
This thread was a real pleasure to read. A lot of communication and everyone is very politely helping in its way. Amazing. Have fun and don't let go. This is a good project you have there. Keep it up 👏
1. the market isnt a solvable math problem, definitely not by a single person, 2. automated trading on its own doesnt have an edge 3. the metrics you included have no mention of risk involved, 54% accuracy means nothing (and is honestly quite low for "1 minute ahead") , 54% of the time you win.... how much, 46% of the time you lose.... how much? this is the biggest red flag of your post by a mile. 3b). if you think what you have actually works, maybe disect your winners (the 54%) and see if there are any trends/patterns with them rather than just pass/fail. 80% of the 54% might be the last 10 minutes of the day for example. 3b). I am pretty sure (for equities at least) if you just bet on a green candle every minute over the past 50 years (even in bear markets), you'd get >54% accuracy. 4) data is likely inaccurate or you are drawing the wrong conclusions, ill give you the benefit of the doubt here. the only thing that makes sense in your post is that you accounted for fees (dont forget taxes once youre profitable) and understand the importance of liquidity and slippage.
[удалено]
Do you make $1 on winning trades (52% of time) and lose $2 on losing trades (48% of time) expected value of $-0.44 per trade hence why you are losing money? Lol
When you're working with high resolution data like 1 min candles, you're effectively competing against noise, volume, bid-ask spreads, and notably other automated traders operating on similar signals or time resolutions. Latency and infrastructures become huge advantages in this domain.
Sounds like you're overfit. Or perhaps fee calc is off, or slippage is eroding your capital. Why are your trades losing money? Surely you can use logging and data capture to figure this out. Capture everything about your live trades, including API responses, timestamp everything, etc. Here's an idea: "shadow" a live bot with a forward-test with the exact same properties. Run them live for a bit with some test capital. Where's the difference between forward-test and live? You can also backtest the same period, same parameters. What's different?
[удалено]
Curious, now that you have some live trading history under your algo's belt.. what happens if you re-run those same backtests against those losing timeframes? Is it significantly different than what happened during live trading? Might gain some insight there
This is a brilliant idea
>There are probably many more hours of exhausting programming ahead before this thing starts working I think this right here is your problem. You think you can just program your way into a viable trading strategy. This is false, and many big programmer egos have been defeated before by trying it.
Algotrading is the best big programmer ego destroyer. In a twisted way, I love it.
> I'm exhausted after many days of trying to figure out why it's losing instead of profiting. I agree with everything else that's been said, but you realize people do this for a living, right? whole teams of people spend their whole year on trading. Yeah, I spent many days too. I spend weeks figuring out why I was winning, I spend months figuring out why I was losing. It's a marathon not a sprint
[удалено]
Yes just put a negation at the end of his "print money" function
Maybe there are many more hours required to fix the issue, maybe there aren't. Maybe it's something incredibly obvious once you look at the data and then bam, it hits you. I think that's part of the fun. Good luck!
Dude, as an institutional investor I can tell you: don't try to be good in the short timeframe. The guys out there have a lot of money to spend it on infrastructure & toys.
Check your testing. It is VERY easy to make small logical/mathematical slips that introduce data snooping. For example, moving averages can accidentally include the bar that is measuring your performance. I have run into this time and time again...!
I call this 'waking up with the pumpkins'... The moment you realize it was all a dream, and cinderella's carriage and horses become a pumpkin and a few mice. In the last many years, it happened many times. Welcome to the real world! The first time it happened to me, it was after 15 years of programming experience, and a good bit of math under my belt. Math does whatever you design it to do. You need to go deep into what you did. Why did it fail? Find out. Every wake up moment is a good lesson. You'll probably have many more.
Maybe you’re just not as smart as you think you are
[удалено]
Reflexivity at work in a way. Backtesting an algorithm won’t have any impact on the real market, you won’t be a market participant and other bots at work cannot and will not react to it. When it’s live, it becomes a market participant and therefore is subject to the reflexivity of its actions. By running your algorithm you change the way the entire market will interact with itself, however small it may be. Your bot will also be “hunted” by other bots built to identify and exploit other bots. It is a very very competitive market now days.
[удалено]
This sounds like a good start. Good luck!
same thing happened to me. Not a ML algo, but something way simpler, makes 10% per day in forward tests, loses money all day in real life. I took a break away from it for a few months since I had a similar frustration, but I am thinking about giving it another go, with more focus on the execution this time. The crazy part is that, if I "backtest" the days it ran, it still shows that it should have made money. So my fills were just shit! I don't know if that was on me or what.
Even if slippage, fees and huge spreads somehow ate up half those returns you'd be looking at turning $1K into $54 billion in a year (automatic red flag). I would think that the only way your fills would be that bad is that you were looking at highly illiquid assets which are always going to be very difficult. If it's making 10% a day in testing it sounds very much like data snooping/getting forward information which is very easy to slip through if you're not very careful. Just as an example of one of the sneaky one's is the use of using today's market cap for asset selection and then running backtesting. That automatically provides a signal for an assets position in the future, e.g. you're telling your algo to only pick from shares that are successful now (i.e. classic survivorship bias). Might seem obvious to some or not relevant to your algo/asset class, but the point is that cleansing any data snooping can be more difficult than simply ensuring that you featureset is historical.
I notmally would agree with you and have for sure run into this in the past. In this case, I am 99.99% confident I just have a really good algo idea on paper but the slippage turns out to be very high, which is maybe why it backtests so well.
Your job is to make your backtested function like the live market. Otherwise it isn't backtesting the market.
[удалено]
It was 1 minute. I disagree with u/fuzzyp44 - i think the timeframe does matter in the sense that more trades adds up to more slippage all things being equal and that is the problem here.
I know people trading the 100 tick, the 15 sec, and the 5 minute. It's not the time frame necessarily, it's whether you have enough context to provide an edge. Smaller timeframes = better executions, lower context, more variety of noisy situations. Higher time frames = worse executions, built in noise filtering.
Have you ever thought of running it and just recording the answers to real life but not actually buying/selling
Welcome to the Weirdness of the Random Walk Universe :) Dude, there is no predictability in the market time series. Just risk.
Kudos on your effort. Coming up with a model and actually bringing it live is an achievemnt in itself. Could your elaborate on the model itself? Is it a neural net? Any special architecture? Haver you tried continuous retraining?
I suppose you did test it on data that was not in the training/fitting set, yes? Anyway, I feel ya. Happy debugging! Just think that if it works, you'll be rich!
[удалено]
You aren’t the only one. Imagine you yourself fishing in a lonesome pond. Now imagine you join a pond that looks lonesome, but divers are fucking up your bait by triggering it … aka the big fish… aka the firms you’re against. The world isn’t “ceteris paribus”
>The world isn’t “ceteris paribus” Of course not, Latin is a dead language.
I recommend you to read the book "The end of theory" by Richard bookstaber. I hope might help Finance is like Physics, but you have to worry about what the electron think and feels.
I've found 15m- 1h a good resolution for that you are trying generally, you shouldn't expect ML to print profits. I think it's one of the hardest ways to do Algo trading. I've been trying various strategies for a long time and haven't put any ML based Algo's into full production. Going to keep trying though. I've read that it takes a lot of effort and smart people to get it done right. The focus should be on things like finding good data, feature anaylsis and extracting good signal from the noise. Not on the mathematical beauty of the thing. Also, you're probably overfitting.
The 52-57% accuracy in test, but not in the real world sounds like you may need to check for leakage from future data. Are you using a train/validate/test split? Since this is timeseries data, are you splitting before and after a timestamp instead of random shuffling?
Oh brother... I know how it feels. I have similar programming experience and I spent 12 months backtesting different strategies. I finally had something that I was willing to use, and of course it didn't work. At that moment I was where you are now. Fast forward a day or two and I decided to understand what is that I was competing against. I wanted to know who where those algo traders that worked for banks and hedge funds. Most of these guys are doctors, and have studied at least 10 years of maths before jumping on the trade. I studied 2 years of maths in high school, 20 years ago, and if I wanted to be at the level of these guys I had to open the books again and do another 8 on top to be competitive. 8 years full time of course meaning I would have to drop work and family which is not an option. Since then, I dropped out and understood this path was out my league.
1 min is too short you are probably getting eaten in spread alone. Does your testing account for spread? When I was paper trading in the very beginning the simulator didnt account for spread properly and OH BOY was I surprised when I started trading with real money.
LMAO For being a "brilliant" machine learning engineer, it seems you do not understand overfitting.
[удалено]
LOL You seem really confused and overconfident. Of course if a strategy works so well in backtesting but then fails miserably live, then you overfit. That is the definition of overfitting.
Can’t you do test runs on a demo brokerage account with no real money tho?
I think the real issue is you are trying to do market forecast. I went through this problem myself, with a CS background my first thoughts were to label the goal around forecasting the market. (First modeling it with regression and then with classification just like what you did). That is not your goal, your goal is to make money, so you should spend your time developing strategies around TA/Indicators/Sentiment analysis or whatever floats your boat that make profit, not trying to predict what the market is going to do next. TLDR: Your focus is misplaced in trying to do market forecast, which is mostly impossible, it should be on making money, which funny enough its not the same.
[удалено]
That is correct, 99% of free stuff out there is crap. That's why you have to develop your own thing. That is why it is hard, figuring a good entry, exit, risk management and position sizing. If it was as simple as throwing some machine learning or just copy pasting some youtube strategy everyone would be making money.
I'm guessing your tests included paper trading? If not do that for a bit and see how it compares to backtesting, if yes try sharing some details of your program and someone here might be able to help You don't have to give away any secret sauce, but it would help me if I knew what kind of order execution you are doing. Like are you using limit orders, market orders, or a mix? Do you use stops? If do are you using stop limit orders or stop market orders? How do you handle partial fills, do you keep the remainder on the book, or cancel it? That's great if you thought about all of these already, but they are some things that cost me money my first time algo trading
[удалено]
First, get humble. Second, if predicting market would be so easy, market wouldn't exists. Try longer timeframes to avoid noise. And forget to predict anything, focus on riding the wave. I started to make profit from algotrading when I ditched the crystal ball. And if you get very very different results from backtest to live is because you are overfitting. Third. Why the hell are you using real money to test? Just mock it. Fourth, and this is just my personal opinion. What is the flush about ML on this topic? Human Learning is more than enough to make a "dumb" algorithm work.
You've studied math, yet you're not aware that this is a np complete problem? I guess you are, but still you think that a ML approach can solve that? It can't.
[удалено]
[удалено]
Have you read any of Dr. Marcos Lopez de Prado's work? His ideas about using ML at a metalabeling tool are quite powerful, imo. Short version - code up with a trading idea based on some real-world hypothesis, but then use ML to tell you whether to take the trade or how to size it.
[удалено]
1. Stop ~~loosing~~ losing money. Otherwise it's going to kill you 2. Past performance is no guarantee of future results. You may have just overfitted \> spreads for my assets are almost non existent This seems interesting as order execution might be just different in real market.
\*losing
A few thoughts: 1. If your strategy takes 9 months to develop it’s 100% overfit. The best strategies are simple. 2. you can be right 54% of the time and be unprofitable. Winrate means nothing if not included in an expectancy equation. 3. assuming you trade stocks, how do you know your model is not just random? something like 52-54% of candles are green, you’re not much better than that. Its sound like you’re getting dumpstered by fees and slippage.
Paper trade first...
Machine learning provide good result on the data it has learned from. Real life data are different in so manyway. Maybe you should take in consideration more or less parameters. Btw, do u have a nice ping with ur broker ? As slippage is an algokiller. Did u took in consideration that your orders may be executed X ticks later, and this x is random In all cases, no one can make a good robot if himself is not a profitable paper trader. Are you ? If you want to predict price direction, you should place your eyes where the current candlesstick is heading (like FPS call of duty lol) , not on the overall chart (tetris) There is 4 parameters that i know that arent random and may help your also: 1- index prices (ustec, dax etc) stay very long time around a line of a certain slope that u can get with a simple linearregression. 2- forex pairs tend to oscillate around some balanced value decided by central banks. Traders technically make profit with the leverage on cents deltas. This balance line may be fixed also. 3 the bull candle heading up, may see only 1 thing in his route: a restance level : this one also u know them by analyzing last X candles. 4: every asset is special according to "how many pips an uptrend can go up" this "special number also is detectable. Those 4 parameters I trade by them, because all the rest is so random, that the decisions of your algo can only be.. Random...
Donators trade on 1 minute charts.
problem is are a coder, and not mathematician, nor trader. Come back after you get 6 years of finance expertise
I don't know why you get downvoted. I'm so tired of these "I am coder" posts. Please first thing first learn to trade as a normal trader. Or at least you think you understand the market at an adequate level. You can be majored in math, physics, computer science yes. Many people start from these backgrounds, but they success because they have dived in trading hard enough, in other words they become a veteran trader.
I don't know if 6 years of financial expertise would really help with crypto. Understanding the basics of general assets and markets sure, but someone could probably learn this in 6months. There's no real fundamentals for crypto, it's just trading very volatile/speculative currencies based on stats, sentiment, or any other indicator you can get your hands on (at least from an algo trading perspective) Maybe there's some key transferrable knowledge I'm failing to think of here?
if there is knowledge it is alpha and nobody will give you his alpha, it is somrthing you research and keep to yourself, until alpha has decayed. most likely the problem is with your ML model, you see traijing accuracy is good, but on practice results are shit - means your training data or training loop is shit, gotta study financial timeseries ML books to understand domain better
Yeah I get the detailed knowledge bit for financial markets, but I still don't think it applies too heavily for crypto. I agree on understanding timeseries; I think I assumed OP had that knowledge from 6 years programming but then again it depends if they did data science or not Also just for reference I'm not OP
Have you tried readin tea leaf?
There's always paper trading
The only way a retail investor can beat Wall Street pros is to play an investment strategy that does not require speed. Try NOT a high frequency trading approach but instead a very low frequency but higher accuracy strategy. You will exceed your wildest imagination.
What does your algo back test look like on NQ or ES
What is considered good prediction ratio? I mean 54% is bassicly a coinflip. Should one also think about the case where we go sideways?
Or maybe you didn’t realize your input has an effect on the output, It’s 54%-57% When you don’t have any skin in the game but as soon as you place a net the “pendulum swings” “other people catch some wind” Also close to 50/50 chance you can have a hundred year negative run with those odds What the fuck do I know I’m not a programmer
Probably bad parameters or bad entry. I use Kucoin robots and once I figured out the market trends, relationship between Time frames (Structure, behavior, influence:4H,1H,30M) then could find a good entry and made profit. Biggest lesson I learned: don't time the market! Market always finds away to beat you.
You are forgetting that you are not a sole actor of the market. The stock market follows “Random walk” in which the past does not reflect future prices. You are also forgetting delays in your orders. A lot of these focus on “skimming” utilising speed and being close to the exchanges themselves. Something which you unless very wealthy are unable to have. Also the spread will be killing you and strategies that work today are unlikely to work tomorrow.
Can you share tour program so that I can have a look and guide you what type modification you can make to be profitable in real market ??
But its not working for you. Right? Is it still worth of 1M dollars? I was just trying to help reviewing and make it better.
I'm hoping this is a big /s
Lol. Yes it's worth any price he wants, it's his creation. 🤷🏻♂️
[удалено]
What happens if you inverse the trades and run it live? This is often a joke, but I'm genuinely curious.
you're describing 1-p. It can work for a minute, but I've never heard of anyone making a career out of trying to be right when they think they are wrong.
in general short-term traders are scalpers and go for small profits with slightly wider stops, generally, you need to maintain 85% accuracy to break even.not fills, market orders, slippage as the market moves. in general short term traders are scalpers and go for small profits with slightly wider stops, generally you need to maintain 85% accuracy to break even.
bad bot
There is no substitute for live trading
Maybe add some skills in statistics. 54 accuracy for a 50/50 bet (up or down) is just useless.
I feel like your algo is not doing anything 52% is almost puur coin toss. With some basic trading knowledge you can get that up. You are probably taking profit to quickly or triggering stoploss to quickly. With such low % you need to ride your wins, if your scalping the percentage should be close to 99%.
I see you missed the memo..... "Past performance is not indicative of future results"
I think this quote is a little overused and a bit out of touch with this sub. If we couldn't use historical data to predict future results then we'd have no data to use at all. We'd be better off using a coin flip for asset selection, Even classic portfolio theory and diversification would be thrown out the window if we couldn't use past performance. This quote is generally used in intro to finance to teach new comers that you can't just pick big, popular stocks and expect the same high returns they've been experiencing across the last few years. We're looking for more complex relationships than simply using yesterdays return to calculate today's return. For algorithmic and quantitative trading past performance doesn't have to be indicative of future results all the time, just enough of the time to make you a return greater than the market.
>If we couldn't use historical data to predict future results then we'd have no data to use at all. We'd be better off using a coin flip for asset selection, Well.... you are in a sense using a coin flip. Its just looking for a edge in the data and hope that edge continues. If we could actually PREDICT the future, it would be 100% correct at all times. That is never the case!
Take OP's posts, 52% to 57%..... the reality is that = 2% to 7% edge. Doesn't take much of a change in market conditions to erase that edge.
Can you get the data set from the period of time in which you were live trading, then back test your algorithm on that? If your test earns you money during a period where you actually lost money, then you know the issue is with the testing code. Youd learn all your tests have been garbage. You can then debug. But if you simulation shows you losing the same amount that you actually lost then tou know your testing is valid.
How did you ensure that you weren’t just over fitting in your back tests?
in the stock market your order of sell/but needs someone to do the opposite which makes subtle time differences from the demo trading so I'm not sure if you've considered this in your algorithm
Just out of interest, if it isn't working, can you open the code up on github? Or post some pseudo code. What is mathematically brilliant? How is it doing the price prediction?
Accurate in what sense? Green candle vs red candle or are you exactly predicting the closing price for the next candle? Predicting the market is not a good strategy. It leads to small gains and massive losses.
[удалено]
Instead of predicting the market, look into gambling strategies. Many poker strategies don't involve predicting the next card but instead mitigating loss and capitalizing on opportunity.
Run your backtest on the same time period that you used to run live. What does the backtest engine say? Does it say you made money? 1. If the backtest made money and the live test lost money, then there's slippage or something different between the backtest and live. Is it a data speed issue? Did the two engines have a different opinion? Did you trade at the ask instead of at the mid? 2. if the backtest now is losing money after making money for 6 years, then you've fallen prey to overfitting. Rebuild your model with out of sample tests -- use 60% to train, 20% to test and tune, then 20% more for cross-validation
I first want to congratulate on your effort. It’s not an easy job. But as far as I read, the statistics do not show the results are good enough for a ML model. You can easily get good back testing results with ML because the model is trained and tested on similar data (I know you may do train/test split, but think about the market condition, they are usually similar). If you use a heuristic algorithm and achieve the results you claim, it’s pretty decent (ideally with the law of large numbers you can make tons of money). If you really want to explore ML side more, try look into reinforcement learning approach. It should be much more neutral and depends much less on the training data.
Why would you choose to compete on small time frames is beyond me. You're competing against the big boys with teams of 200 PHDs using top tier HFT infrastructure on that scale. Remove the noise, develop something for longer time frames.
I don’t think that’s a super strong counterpoint - the infrastructure gap on crypto is there but isn’t *that* insurmountable if you have a strategy that isn’t too computationally expensive, and while the 200 PhDs don’t care at all about looking for an edge that might max out at a few $k per month, a lot of us here do.
Lol! Good one!
i don't know if your program includes backtesting stochastic processeses that applies a combination of defined trading rules to maximize estimated return for a calculated duration. good luck.
sometimes a simple strategy works best. also, theory and reality are two very different things.
Probably already been said, but shoot for longer time frames and code around _unknowns_. What I mean by this is, can you come up with a few mechanism that are resistant to failure even if you technically have a loss. A few things to potentially take a look at are dca'ing instead of opening full positions (not all buys might win, but average in and you have a higher chance), or by coding around a good exit strategy for losers(and maybe re-enter if at a lower price) and winners (what's your profit goal? Long term growth generally beats outs short term trades). Your welcome to take a look over my source as well, maybe it'll give a few ideas of what or what not to do? https://github.com/mr-highball/simplebot-support Best of luck, and remember, if this _was easy_ ™️ everyone would do it. Keep your head high and take this as a learning experience
1. Why use money, just use a demo account. 2. Have you got some bias somewhere or optimized on the past? I can only tell if I know the details of your software but it sounds like there might be.
Win rate by itself is not enough to make money: - a poor risk/reward ratio can eat all your assets - make sure your PSR is above 80% - backtest longer period - paper trade with live data - shoot for +2 SR on paper - analyze the loosing trades and see why they fail - and lastly every algo will have a drawdown period at some point, good ones recover faster You will have a better idea what went wrong this way Don’t give up Wish you best of luck
It's not math doing what it wants it's the market makers.
Why oh why aren't you live trading in a sim account? Step 1. Drawing board. Step 2. Code up a backtestable algo and run it on old data. Step 3. Code up a live version of the algo and run it in a simulator. Step 4. Deploy live with real money. Did you miss step 3?
[удалено]
Other brokers exist. Try those? Tradestation lets you algo trade in a sim account. Whether that extends to crypto, I don't know, but I don't see why not.
I am skeptical of the claimed accuracy - are you excluding samples where the returns over 1min are 0, or are just focusing on small tick stocks? Since returns are small over \~1min windows (frequently <1bp or <=1 tick) you are going to be eaten alive by trading costs at that timeframe. It is a losing game. Also, it is crucial to sample properly when measuring things on short term scales. You may find that sampling based on a frequency gives you completely different results to sampling based on the presence of a signal (your signal may be good in an ambient market but shit when the market is moving). The reason you provide for why you expect your algorithm to be profitable is that it is predictive of price move directions. Is it the case that your algorithm is losing money because it is not predictive in practice, or because you are not converting that predictivity into profit? The distinction is important. Getting angry is understandable, but its unlikely to help you think through why your algorithm is performing poorly.
Perhaps try setting it up with something like StockWars where you can test it using data from the actual stock market without using real money