Welcome Santo,

I cloned your algorithm and ran it starting in 2010. The advantage over the S&P 500 is considerably less compared to your backtest going back to 2008. I saw the same effect in a recently posted algorithm (https://www.quantopian.com/posts/beat-spy-by-buying-and-selling-spy). So, I think to declare victory, you need to see a consistent rate of return of your portfolio that exceeds the S&P 500, independent of start date.

Grant

Did you choose the top 9 stocks as they are now, or as they were in 2008? If you extended that rule back in time (top 9 stocks excluding financials at each year), how do the results look then?

The goal is to avoid look-ahead bias - S&P 500 is market cap weighted, so the top stocks have already done very well...

I'm also wondering how the "top" stocks were chosen. For a backtest starting in 2008 the stocks must be picked based on 2007 information (to avoid look-ahead bias).

Also what constitutes "top" in this case? Performance? Capitalization?

It would be nice to have a time series with market cap.

Here's my backtest with the top 9 S&P 500 stocks of 01/01/2008 excluding financials (XOM, GE, MSFT, T, PG, CVX, JNJ, AAPL, CSCO) -- BAC was excluded.

It's worth pointing out, also, that "excluding financials" is itself a look-ahead bias, considering that we are still sort-of in the "financial crisis".

Where this sort of backtest would get interesting would be if set_universe could screen and sort on fundamental factors, like EBIT/EV -- then you can really start to quantify the little book, or even the system outlined in Quantitative Value.

BTW I picked to 10 based on market cap, not sure how you picked your list, but it included WMT, IBM and GOOG which were not in the top 10 at the end of 2007.

Thanks for all the feedback. I knew I should have done more work before posting but I was too proud having coded my first algorithm not to post so sorry for all the confusion. I tested a bunch of different versions, stocks I like, stocks with low p/e ratios etc. all from S&P lists I found dating back to 2007 or earlier. I suspect you can select any group of stocks based on your favorite criteria and you'd get reasonable results. The reason I take out financial stocks is mainly bcs of Joel Greenblatt's book. His key criteria on universe selection is return on investment. He looks at investments companies make (to build something new, factory, R&D etc) and the return it generates to screen companies. For that reason, he leaves specific sectors out like financials and mining as it is not clear what they 'build'. He wrote that book many years ago, it seems even wiser now than ever. My next task is to build a tool to select from a random universe based on your favorite criteria.

Shouldn't the algo be tested with random stocks? or at least random stocks that meet a certain criteria?

Hello Santo,

Note that the Quantopian set_universe function selects securities based on dollar volume (see the help page). And you can only "analyze" a relatively small fraction of the universe, ranked by dollar volume. Additionally, Quantopian is not a general-purpose securities analysis tool, but rather a backtesting and paper/live trading platform. Quantopian has talked about eventually offering such tools, but live trading has been their focus so far. So, my caution is that Quantopian may not be the right tool for the job if you are looking to develop a flexible tool for analyses and screening (but it could be great for verification and validation).

In any case, I've attached an example of an algorithm that uses set_universe to define a list of sids and then extracts a trailing window of data for those sids. It's an extreme example, since it updates the window every minute, with a trailing window of 390 minutes (1 day's worth of tics). It's kinda slow, typically taking several hundred milliseconds to get the data (which adds up, when performed 390 times per day).

Would the Google Finance screener have the tools that you need?

Grant

Thanks Grant - just getting back to this after a long work week. I think your approach is good. I will try to see if I can come up with an algorithm to randomize the inputs and see what I get. I think it is important to stick with the SIDs for the long term and I have no clue what the randomization will do. I will publish the results when I am more confident as to what's going on.

Thanks again