Your CSV can have any number of stocks, but you can only receive data for (and trade upon) 500 securities at a time. The file limit is 100MB, but the larger the file, the longer it takes to read it. The algo has 6 minutes to fully ingest the data, and if its not done, a timeout error is thrown.

You're on the right track with universe_func :) You can use it to create a dynamic changing universe and whittle down your list. If you need an example, take a look at this thread.

Enjoy working on your algo!

We're also making it easier to process large trading universes. Karen announced the kickoff of that here, and you can watch her webinar to see examples. We've been working on it this summer, and a lot of progress has been made.

Thanks for the prompt replies & attention.

Seriously!

"You can use [universe_func] to create a dynamic changing universe"

I still don't understand. Can I call universe_func multiple times?

Here is how I understand the lifecycle of a quantopian algo:

(Program starts)
[initialize function is called]
   [[inside initialize, fetch_csv is called]]  
   [[the universe_func callback for fetch_csv then gets a list of fundamental data for the current day and whittles down my list]]  
[the day's trading begins]
[it's the next day.  how do I re-adjust my universe based on the dynamic, changing fundamentals?]

I think the answer I'm looking for, the design pattern, is inside my initial (and only) universe_func(context, fetcher_data) callback invocation, I should store fetcher_data on context. Then in my before_trading_start() function, I can look at context.fetcher_data and re-adjust my Universe/set

Hi Zach,
I just spent some time trying to find something that will work for you. I did come up with something, it's not perfect, but it works.

When an algo starts up, the order of the function calls is initialize -> before_trading_start -> universe_func, so you are not able to access the fetcher symbols from within before_trading_start. However, you can access the before_trading_start data within the universe func.

The trick is that you are not allowed to have > 500 sid objects in your algo, but you can create a list of their N ticker symbols using sid.symbol. You can do your fundamentals screen first and save the set of symbol strings that passed your screen as a context variable, then you can then filter the symbols in your fetcher file by removing the ones that did not pass the fundamentals screen.

Here is a basic example of what I'm talking about. It's a bit hacky, but it works, I hope this helps you get going again.

Best,
David

Thanks David.

I solved it using a similar solution.

It took me a while to realize that there is no limit on the number of securities that your fetch_csv could get data for, AND that the data is automatically slurped into the "data" parameter to handle_data.

:-)

Right now I'm trying to figure out why my Backtests are getting stuck at 0%..

http://imgur.com/Yw1khIA

It's probably taking a while to crunch the large amount of data. Try kicking off a full backtest and coming back to it (you can close the tab, the full backtests run in the cloud and the data gets saved).