Hello Sébastien,

Do you have offline access to the daily OHLCV data for the 500 securities in your universe (e.g. via Yahoo or similar)? If so, you should be able to do the scoring offline, and then use fetcher to upload the trading decision for the day.

Grant

Hello Grant,

Ok that seems possible for me. Is this possible to apply it later for live trading ? Will I have a way to load this external csv (generated on my server) during handle_data, to choose the trades of the day ?

Thanks :)

I don't have experience, but my understanding is that you can update a file for fetcher on a daily basis, to be uploaded before the market opens. You might have a look at https://www.quantopian.com/posts/define-custom-universe-via-fetcher-using-the-new-universe-func-callback, which is a just-released update to fetcher. My thinking is that you would change your universe every night along with the allocations. I'd have to dig into the details to sort out all of the mechanics (if you figure it out, just do us all a favor and post your findings here).

--Grant

Ok will do. I am still in the discovering quantopian, so I was just asking the question to know how I could implement my algorithm in a sustainable way.

Thank you very much for your answers :)

Hello Sebastien,

Just curious...where will you get the data for 500 securities? And will you have to do any clean-up/pre-processing before you can use it in your algorithm?

Grant

Sébastien, Grant,

Please have a look at our newest feature, setting a custom universe with fetcher: https://www.quantopian.com/posts/define-custom-universe-via-fetcher-using-the-new-universe-func-callback
There are still rough edges to the feature, but it enables evaluating as many securities as you have in your external file on a daily basis to reset the universe of tradeable securities.

thanks,
fawce

@Grant: until now I used QTSK to backtest, where getting the stock data for s&p stocks is quite easy. I think I will simply load a csv file containing signals for stocks in my quantopian code. All the processing (and the csv generation process), as I see it, could be done on either my shared hosting (ovh) - in this case there will be some work required to get the stock data and generate the signals using PHP (it might be possible to use python), but nothing spectacular - or on my EC2 unit on amazon - the advantage in this second solution is that I would still be able to use python, numpy, zipline, TA-lib, QTSK... On either case, there will be some extra work to get daily stock data in a reliable way (I will probably use yahoo / quandl), but nothing compared to having to deal with interactive brokers directly...

@John: great news, I will definitely try this. I have two questions though ;
- when I use fetch_csv to retrieve my signal, if I change the csv file on my server each day, will this csv file be reloaded during live trading ? If so, when ?
- I am currently using my algorithm on a minute basis, but it is a bit of a shame because I only call my algorithm once a day (few minutes before the market closes). This is partly because of the history function - but since I will use signals, it should not be a problem any more. But during live trading, if I use the daily mode, when will the trades be executed ? I am still new at this, but is there any way of telling quantopian to execute handle_data each day but at a precise time (few minutes after beginning / before close of the market for instance) ? If not, I think it would be a nice feature since it would be simpler and improve the performance of the backtester (sometimes a bit slow on minute mode)... And if the csv file can be reloaded at the same time, it would be awesome.

Anyway I think Quantopian is a great product and it is nice to regularly see new features coming out.

Thank you very much :)

Hello Fawce,

Perhaps you could clarify the number of securities supported by the new feature. Above, you say "it enables evaluating as many securities as you have in your external file" but on https://www.quantopian.com/posts/creating-a-universe-of-all-of-the-stocks-in-a-single-index, Jess comments:

If you're working with minutely backtest data we cap you at 200 securities I believe, so you need to pull in the SP500 and then pass only a subset on to handle_data. In the example I posted I winnowed the universe down by selecting just a single sector (financials).

How does the new feature work with respect to limits on the number of securities?

Grant

I confirmed and Jess was correct in her post, you can import up to 200 securities using Fetcher's custom universe!

@Sebastien - just noticed your last round of questions hadn't been answered (at least not on thread), here are some answers inline:

• when I use fetch_csv to retrieve my signal, if I change the csv file on my server each day, will this csv file be reloaded during live trading ? If so, when ?

In live trading fetcher data is reloaded when the algorithm is launched in the early morning hours before each trading day. Based on our current design you need to maintain persistent historical data in your .csv file, only appending new data as it becomes available. Any new results posted and timestamped by midnight prior to the trading day should be available to your algorithm that day. Down the line we'd like to support fetcher calls during the trading day - but I don't have a timeline on that at this time.

• I am currently using my algorithm on a minute basis, but it is a bit of a shame because I only call my algorithm once a day (few minutes before the market closes). This is partly because of the history function - but since I will use signals, it should not be a problem any more. But during live trading, if I use the daily mode, when will the trades be executed ? I am still new at this, but is there any way of telling quantopian to execute handle_data each day but at a precise time (few minutes after beginning / before close of the market for instance) ? If not, I think it would be a nice feature since it would be simpler and improve the performance of the backtester (sometimes a bit slow on minute mode)... And if the csv file can be reloaded at the same time, it would be awesome.

We actually only support live trading in minute mode. However, it's no problem if your algorithm 'sleeps' all day and trades close to market close (or even once a week or month just prior to close) as long as your logic specifies this. We have a few ideas in the queue to make it simpler to implement 'daily' trading within our minute framework, but at the moment you do need to code your algo to be explicit about when to place trades. There are some good examples in the forums of this, and you can take a look at our commonly shared live trading algorithm as well, this version trades once every 21 days.