Hi Delaney -
Thanks for this opportunity. Some feedback:
Functional block diagrams - A good example was provided by your former Chief Investment Officer, Jonathan Larkin, in his blog post, A Professional Quant Equity Workflow. This type of high-level architectural diagram is effective in putting detailed efforts into context. For example, in his diagram, it is immediately clear that Alpha Combination is a separate function from Portfolio Construction, what inputs and outputs are required for each function, and generally what role each plays in the whole system. Under the heading of "functional block diagram" I would include all forms of informal and formal diagrams, depending on the context. I'd recommend considering how you could support diagrams and more generally, editable graphical representations of information (e.g. a "virtual white board"). This would include your research notebooks, algos, and the forum.
Math - There is a tendency to jump too quickly into commented Python code, rather than fleshing out details in the more approachable universal language of mathematics. Be sure to cover the math thoroughly first (with some diagrams) before showing the specific implementation in Python and the Quantopian API (in fact, skip these, if they don't add to the discussion). If code is used, it should tie directly and transparently to the math.
Simple, readable code - You have a bunch of Pythonistas, but it can lead to all manner of fanciness in your examples. There is just too much of a mental load and time-consuming effort in unraveling "elegant" Pythonic code that I'm sure is world-class, but is not helpful for non-experts. Generally, this is a huge barrier, I think, for your success. You require a rare bird--some background and interest in trading, math and statistics expertise (possibly mixed with data science and machine learning), Python coding expertise, and a good dose of patience and persistence, plus spare time. So, you just need to be sure to lighten the load wherever possible, even if the code examples are less-than-Pythonic. There also seem to be multiple styles/frameworks within Quantopian, which adds to the mental load. If you are publishing code, and the intent is to educate, I'd make sure that it is reviewed and edited for read-ability and meets some sort of consistent style that a beginner/advanced beginner can digest.
Industry veterans - I think it would go a long way if you could get an experienced quant trader to be on the front lines of support and training. Frankly, Quantopian from the inception has felt like the blind leading the blind. Jonathan Larkin was a good addition and he did interact to some extent with "the crowd" but unfortunately he departed, and has not been replaced. Having one or more people who've actually worked at hedge funds and can provide specific, real-world advice would be great (of course, it would take a special kind of patience to field questions from inexperienced knuckleheads like me, but that's your business, right?). Along these same lines, as mentioned above, hearing directly from folks like Jess Stauth, who (as I understand...you guys are very secretive) leads the algo selection team for the 1337 Street Fund would be nice. Also, you have other folks on your leadership team who I'm sure have a wealth of knowledge that could be shared, beyond the nitty-gritty of researching and writing algos. For example, a Q&A on the legal/regulatory issues of setting up and running a hedge fund would be fascinating.
Base algo templates - You need to get serious about publishing and maintaining standard algo templates that conform to the workflow. This is very simple, so don't over-think it. Develop and release a modular multi-factor/feature template on Github, and maintain it regularly. Open it up to users, so that they can provide feature requests and report bugs. The algo should run on the
QTradableStocksUS
back to at least 2005 without glitches and conform to the contest rules and fund needs. A user should be able to research a handful of Pipeline factors, plug them in, and be able to submit the algo to the contest--presto!. One nice option would be to select a basic ML algo for the alpha combination step, without having to do any coding--just grab the module and plug-and-play.
Forum style - For moderated threads, it would be very helpful if the moderator/presenter kept a running Q&A at the top of the thread, versus doing lots of individual @user replies scattered about. I wouldn't necessarily change the forum format (e.g. something akin to Slack, which almost has too many features), but if you are trying to educate on a specific set of topics, capturing them at the top would be very helpful.
Specific, actionable feedback - I would echo Stephen's comments above that one would think that getting specific feedback on an algo on its potential suitability for the 1337 Street Fund would be a standard part of your process. I have algos in the contest, and I'm doing o.k. (can buy more than a sandwich with my winnings, which was my initial goal), but I have no clue what I should be doing to bring a given algo to the point where I would definitely get an allocation. What I'd like is some definite measure of the probability of getting an allocation for a given algo, and specific guidance for improving that probability.
Public issue tracker - Simple. Don't over-think it. Just do it. Zipline and other open-source projects don't count. I'm talking about your entire platform.
Open-source code - Another no-brainer. For example, you've held off publishing the Optimize API code. And my understanding is that the risk model and QTradableStocksUS
code will not be published. Not really sensible, in my opinion, as I understand your business, but maybe I'm missing something.
Business model & culture - I (and I suspect other users) would be glad to provide guidance on how you've set up the business, the culture, and other fundamental issues that impact your efforts in attracting, training, and engaging quants. I realize that this is far afield from your request, but I do believe you've missed the mark as a truly crowd-sourced effort, and I have a sense why (I have been advised that you are not likely to change, but I'm still holding out hope). Fawce and the management team could do an open Q & A, for example. Not in their DNA, I suspect, but I thought I'd throw it out there for consideration.