So, let's make the data series longer, much longer (30,000 bars). All three charts below were generated using the same method as for the first chart. The first 40 bars of chart 1 originate from chart 4.

Note that for each chart up to 5 factors were extracted: \(x^1 \) to \(x^5 \). By adding factors, it increased the coefficient of determination: \(R^2 \). It is understandable since each factor in the equations explained more and more of the price series.

We could consider the 3 charts as part of a long-term chart that has been sliced in multiple parts of which only the first 3 charts are shown below.

Fig. 4 \( \; \) XYZ Price Series, First 3,000 Bars \( \, \) (in sample)

Fig. 5 \( \; \) XYZ Price Series, Next 3,000 Bars \( \, \) (out of sample)

Fig. 6 \( \; \) XYZ Price Series, Next 3,000 Bars \( \, \) (walk forward)

The So What Clause

There is a point to all this. Charts 1 to 3 were all unpredictable, even if they had the same opening price series. So are charts 4, 5 and 6. They are all linked to the same price series. Chart 4 is totally unpredictable. Since charts 5 and 6 are continuations of this randomly generated price series, they too are unpredictable.

The same would go for the next 21,000 bars (not shown) of this randomly generated price series. It would remain short-term, mid-term and long-term unpredictable.

No matter what kind of factors you would want to extract, none of them might carry forward. As if saying that your “predictions” are in fact only guesses. Even if no one is stopping you from making those guesses, it does not make them right. It makes them only guesses, nothing more. The reason is simple: there is nothing more than luck to be had.

Partitioning Data

Even if you declared chart 4 as an “in sample” training period, it would not change the unpredictability of that chart or the ones that follow it. Even if you wanted to extend the factor equations to include \(x^6 \) to \(x^{11} \), it would not change the unpredictability you would find at the right edge of the charts as shown in chart 5 (the continuation of chart 4), or the right edge of chart 5 or 6, for that matter. Each bar of this 30,000-bar series would remain unpredictable by construction.

There is nothing exceptional in all these charts. They do resemble actual stock prices that have been detrended, meaning that their respective regression lines have been removed, and yet, they still retain one on the random-like price behavior. Even on a Wiener process, you can extract multi-factor equations with as many factors as you want.

Even so you can extract factors from charts 4, 5, and 6, none of them could be of use in implementing any type of worthwhile trading strategy where you could say, in advance, that it would win the game. And not be something entirely due to luck. Just pressing F9 on this Excel spreadsheet would generate a totally new price series with new factor equations.

A Trading Strategy Expectancy

A trading strategy is the sum of the outcome of all trades. This is easily expressed: \( \sum_1^n ( q_i \cdot \Delta ^i p_i) \) where all trades are sequentially number from \( i = 1, \dots, n \) and then added to make the total profit or loss. On a long-term randomly generated price series, the payoff should be expected to tend to zero: \( \mathbf{E}[ \sum_1^n ( q_i \cdot \Delta ^i p_i)] \to 0 \). The same outcome as if playing a heads or tails game. You could literally make thousands and thousands of trades and still have a long-term zero expectancy. Not the most desired outcome for all the efforts put in the game.

There is so much one could say about the above charts, but also so much no one wants to hear.

For instance, designing trading strategies around randomly generated data series is a lot more demanding that one might think. From the start, you will be told that the expectancy is zero, and yet, some will still forge ahead. For what reasons, I do not know. Or they will come up with so many different ways of looking at the data and declare they can see patterns, Fibonacci sequences, or what have you. All of it will just be gibberish, and for cause. Those charts were randomly generated after all.

Notwithstanding, there are patterns in randomly generated prices. You will find flags, pennants, head and should stuff, wedges and all. But, overall, they will not be useful in helping you make trading decisions that will be better than luck. How could they? From any one bar in that 30,000-bar price chart, there is no assurance that if you started a trade right there that there would be a profit, none whatsoever.

We should understand that nothing will give you an edge except pure coincidence which translates to you made a guess and you got it right or you got it wrong, the same as if you had flipped a coin. There is no machine learning program either or deep learning one for that matter that has shown they could outperformed the game of heads or tails with any consistency. All they could rely on is that sometimes they had some luck, and at other times they did not.

The Long-Term Trend

Over the short term, everything above prevails. The more it resembles a random-walk.

It is by putting the left-out trend \( \mu dt \) back in that you can give these trading strategies something. In a way declaring that the game is not a martingale, that stock prices do have memory (long-term trends), that you can design profitable trading strategies with a much higher expectancy than zero. In fact, you could design strategies with an “almost surely” mathematical positive expectancy, as long as you played the game with a long-term vision of the game.

What you appear to be saying is that the long term trend in stocks is upwards. What you should perhaps have added is "over the past couple of hundred years". It is not guaranteed that the trend will persist for the next couple of hundred. We may descend into a new Dark Ages for all we know.

Glance up at the Milky Way on a dark autumn evening. The universe is 15 bn years old. The earth is 4bn has has seen a few changes. For one thing the pulling apart of Pangaea means you and I are separated by the Atlantic Ocean. Industrial society has existed for a mere 200 years. It may behove us to assume our society and economy will continue "as is" but somehow a glance at the night sky has a way of cautioning me that my prediction may be fragile.

Also you must account for selection bias of the United States. Many people think that the stock market goes up over time. if they are just looking at the United States, then yes that is the case, however our economy has also been growing at the same rate. What happens when we start to slow down? Will the stock market still give positive returns in the long term? My point is that you cannot assume that the stock market goes up over time just because the United States has done so since the Great Depression. Inevitably there will be a slow down where instead of looking a value investments, you will look at value shorts in stocks that are slowing down. Look at the Nikkei 225 Benchmark. If you had invested in 1990 you would just be breaking even now. Thats 25+ years of making no returns! That doesn't take into account inflation either in which would probably break many years from now even if it kept going up. Just my 2 cents on this topic.

I would personally have to disagree with this.

To be politically correct, using past data, they could make a lot of assumptions and put all sorts of lines on these charts, but on future data, they could never outguess randomly generated price series such as these except by luck, almost by definition.

What you can do with this data is analyse the distribution of the returns (assume Gaussian, but if you don't want to there are other distributions) and hence determine where it is most likely to end up. Of course the method by which it gets there is completely random, but the eventual outcome mean is predictable. Therefore you should be able to make consistent returns over the long run by betting on stocks whose distribution of returns ends up with a mean at time t greater than the price at time 0.

Extrapolating, a month is made up of many days, a day is made up of many hours, a hour is made up of many minutes and a minute is made up of many seconds. Surely you can do exactly the same thing for second data in order to predict the next minute that you just did to predict the next month using daily data.

Of course you're not going to be able to divine what the price will be based on trend lines and abstract technical analysis, but what's to stop you realising based on past data that this stock is more likely to go up than down (or vice versa) in the monthly time period and acting based on that?

A university acquaintance of mine was the Duke of Northumberland; we were in the same college at Oxford. His family have remained rich for almost 1,000 years thanks to their holdings in land. Land does not disappear in a puff of smoke. Most stocks do. Even land can disappear in a puff of smoke - ask the Imperial Russians. Or the inhabitants of Santorini.

Its all about diversification. Even then this moldy little planet could get nuked by man or an asteroid.

The only durable is "information".

I have come to wonder why those with science degrees and skills in maths and statistics waste their time with markets. Persistence of success over time in financial markets is pretty well zero. Particularly with hedge funds. They are all one trick ponies.

@Quant Trader, by construction, the expectation of a randomly generated price series using a Wiener process is zero: \( \mathsf{E}{[W_t}] = 0\), see Wikipedia if need be: https://en.wikipedia.org/wiki/Wiener_process. The expected mean of such a series is predictably zero. The correlation coefficient of such a curve is also zero. And its expected R-squared tends close to zero as well \( \mathsf{R}^2 \to 0\).

Therefore, you could NOT make consistent returns over the long run except by luck alone.

You can only guess where your random-like series will end up and you could not give your bets a higher probability than 50/50, this by design. No one beats the game of heads or tails, consistently. Even if their short-term “forecasts” are expected to win about half the time, it does not change the long-term expectancy for them to be right only about half the time.

A trading strategy playing randomly generated price series have a profit expectancy of ZERO. It is facing a mathematical martingale. Even if one did a million trades, it won't change the expectancy.

Just looking at these two groups (bull series or bear series) even though each group is fundamentally random, the eventual outcome is highly probable. Therefore even though you'd be wasting your time trading the series, it wouldn't be a waste of time being long the 'bull' series and short the 'bear' series.

Random doesn't necessarily mean that the expected outcome is zero, nor that the possibility of consistent returns over the long run is based on luck alone. It just means that the more 'fundamental' methods of trading patterns are fairly useless.

The other option is that the past 'randomness' does not describe the future 'randomness' in which case there must be something dictating how the market will move, making consistent returns possible over the long run if someone can work this out.

Obviously, a normal distribution doesn't adequately describe the markets, other distributions can though.

@Quant Trader, you are making my point with your NB.

It is sufficient to remove the slight upside bias you introduced in the bull series and the picture will change. Make it:

series_bull[i] = np.random.normal(1.0, 0.01, 1000)  

As said, it is by adding back (\( \mu dt \)) to the stochastic equation that you can give your trading strategies an edge. Which is what you did with your line of code:

series_bull[i] = np.random.normal(1.001, 0.01, 1000)  

Try to win on the attached NB.

Oh no, you obviously can't in that Notebook, but that's assuming that all stocks are random in the exact same manner is it not? Stocks like AMZN, AAPL or KO for example cannot be compared with a stock like DRYS, KHC or similar.

I would argue that even on a relatively small amount of data, the distribution from which the data is drawn can be derived and hence an 'edge' can be found. Say for example AAPL has a mean of +5% and KO has a mean of +1%, you can get a market neutral 4% (on average) if a trader can work that out.

You would be hard pushed to find two assets (excluding etf's which are designed to mirror, SPXL to SPXS etc) that are random in the exact same manner. Implying that they're driven by a different set of rules, the rules can be found out from a small(ish) dataset.

I'm just going to put a permanent stipulation in here to save me having to write it again later, I'm writing like I assume the market is Gaussian, I don't believe that. The terms are just far easier to use (for me at least) so just automatically translate it in your brain.

@Quant Trader, the six charts I presented in the first two posts were generated under the same principle as the last attached notebook based on your design. It should be evident from that notebook that there is no edge available from the construction of randomly generated price series.

What's in the notebook is based on the presented stochastic equation: \( p(t) = \mu dt + \sigma dW - \mu dt = \sigma dW \). Removing the trend \( \mu dt \), you were left with only the random component which is what all the charts showed. It should be sufficient evidence to show that stocks are not totally random but more random-like with trends. And I stress on the words: random-like with trends.

Do real life stocks operate as if under the above equation? It is a fair representation, and it has been an acceptable model for more than half a century.

The difference is in \( \mu dt \), which is what you added in your notebook. Note that \( \mu dt \) could also be negative. The equation for a stock going bankrupt is: \( \mu dt + \sigma dW < - f(t) \).

All stocks are different in real life. We know their past but we certainly do have a hard time outguessing where they are going on a daily basis. They each have their own price series: \( p_i(t) = \mu_i dt + \sigma_i dW_i \). Each will behave differently with some similarities with others (correlations).

The question should be: what are the odds that AAPL or AMZN go up tomorrow and by how much? You want to know now the size of the bet to make now prior to tomorrow's outcome.

Will you try to outguess \( \sigma_i dW_i \), \(\mu_i dt\) or both? And from there, what is the probability of each? Can you do this every day for 1,000 stocks for 20 or 30+ years? Here is the equation to solve every day for \( i = 1, \dots , 1000 \) :

$$P[f(t+1)] = B_i \cdot (q_{i+1} \cdot \mu_{i+1} dt + (1-q_{i+1})\cdot \sigma_{i+1} dW_{i+1})$$

where \(q_{i+1}\) is the probability of being right on \(\mu_{i+1} dt\) for stock i and \(B_{i}\) is the current bet on tomorrow's expected outcome.

If you watched TLRY today, it went from 150 to 300 and back, had 4 halts, traded half its outstanding shares, twice its available float, made over 4 50 dollar moves during the day with two moves exceeding 100 points. How could you outguess that?

First of all, thanks for pointing out TLRY, I'm always interested in weird stocks, let's me re-evaluate some of the models I have.

Secondly, I must have missed the phrase 'random-like with trends' in the original passage you wrote, that was basically what my argument was so it seems like most of this was based on a miscommunication.

Just to clarify though, you don't have to outguess $\mu_idt$ or $\sigma_idW_i$. You just need to determine whether the average result will be favourable versus another stock. This should be easily determinable based on how it's performed. I know for a fact I've managed to get distributions to fit data up to 10 years out of sample with a KS test statistic of <0.015 and a p-value of greater than 0.99. Implying to me that there is some consistency in how these things work.

Obviously, you're not going to outguess what TLRY has done (too little data available) but you could have seen that the possibility of it occurring was not non-zero with enough data.

I'm not sure how to use maths in the forums, so I'm just going to leave it like that for now until I work it out.

Randomly Trading Stocks and Winning

A previous post concluded with it would be possible to design trading strategies that could “almost surely” win, in aggregate, almost all the time, given time. It was also said that simply adding back the removed trend line to the presented stochastic equation would be sufficient. I would like to substantiate it so that it is not just a claim or an opinion, but something that can be translated into facts.

We all know that randomly trading stocks is not destined to be that productive over the long term. It has been demonstrated again and again. However, by changing one's trading procedures, even in the face of uncertainty, one can have such trading strategies perform with a positive edge provided that these series, on average, show some upside biases. All that is needed is to show that it can be done.

I have done such tests some years back. My intention here is to elaborate more and give a better understanding of the methodology. In my 2012 article: Changing the Game II, the case was made that a portfolio of randomly generated prices having an aggregate average upward drift was sufficient to generate profits even if using a randomly generated trading strategy to do so. This might have been written some 6 years ago, but nothing has changed. All of it still holds today.

You had this portfolio of random-like price series with fat tails and random drifts to which a randomly generated trading strategy was applied, and you won almost all the time. Each time you pressed F9 in this Excel spreadsheet, it would generate a totally new set of price series with jumps, and totally new trading strategies. The new price series and strategies had no memory of whatever prior occurrences. Yet, again and again, you would win. It would show a positive payoff matrix almost every time (>98%). The price generation formula used was: \( p_i(t) = \mu_i dt + \sigma_i dW_i + \Sigma_i j_i \). The mean returns were randomly set and outliers were also random with random amplitudes.

You could not know how the price series would behave, what would be the next price variation added or subtracted, nor could you know when or if a trade would be taken or exited. You could not predict any of the outliers either.

The strategy would at times buy tops and sell bottoms, just as it could do the opposite. In fact, it would buy and sell anywhere on those curves due to the random-like properties of the trading strategies. And yet, the sum of all profits and losses from all the executed trades for each of these strategies would result in profits.

These tests were summarized in the following paper: A Changing Game.

One might not be able to predict where prices are going, but can still play the game and win. Overall, I say the same thing as Mr. Buffett has said numerous times and for a long time about the US stock markets: investing in stocks “is a bet on America”. And over the past 50+ years, he has been right.

There is much to learn from A Changing Game. It said that trading procedures could dominate the outcome of a trading strategy. For me, it demonstrated the trading mechanics by which alpha could be powered up. It led to designing better trading strategies. So, I hope it can help you by presenting this “different” point of view.

The reason this randomly generated stock trading strategy wins is very simple. You could use the same principles and apply your skills to trade better than randomly.

Go for it then.

Yes. A straight bet on the US. Or Weimar Germany. Or imperial Russia. But if you wish to make such a bet why waste time traditing? Simply buy and hold. In time your upward trend will be broken and you will lose you money - Rome was over- run by Barbarians. But if you must invest in stocks at least diversify geographically.