Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
Son's stock club yields different simulation results

Suggestions welcome on this simple algorithm that takes my son's stock club picks and uses them within the Quantopian simulation. Yes, they are play-trading and the platform is here: http://www.democrat-chroniclesms.com/

What They Wanted
Well, they just wanted to see the graphs mainly over time, but they were also interested in how they might be rated by financial metrics which Quantopian generates during backtests. They loved seeing the graphs over the 12 weeks session.

What I Wanted
Sneakily, I wanted to engage them a bit and introduce them to algorithms. Gently, they eventually deduced how the decisions they were making could be modeled in an algorithm.

What We Found
The results puzzled me as my naive assumption was that this method would give the same results of the simulator. However, they were very different. The cause could be related to their market trading platform or Quantopian's simulation... or very possibly the algorithmic approach. Perhaps the slippage model is incorrect?. The algorithm is attached for comments. It basically sets limit orders at the start of the trading day for each stock traded that day.

Questions
1. Why is the cost basis greater than the limit order price?
2. Why are the overall results different than the market simulator? For instance, their max return was ~15% and their final return was ~3%.

Conclusions
Well they liked the concept of algorithmic trading and had lots of questions about the program and whether it was really possible to make real trades with algorithms. And I get a chance to learn a bit more about comparing simulators.

7 responses

That's a fine, informative logging example and I'd encourage everyone to clone/run and take a look.
Your algo on Q (Quantopian) ends at 14.7% Returns as we can see, however, there is a gotcha that might be affecting you. Q currently ignores negative cash in that calculation. Your algo leveraged (borrowed) $108391 (the graph does smoothing and maybe that's why it doesn't show).
The contest allows what they call a 50% margin. "Margin Requirements – The margin requirement for the simulation is 50%. Initially, this means that since you are starting with $100,000, you may borrow another $100,000, making your total buying power $200,000." So I trust they inform you the actual amount borrowed and that might be one difference.
You are set to a high commission. Q's broker Interactive Brokers might be the most attractive in the industry. On the other hand that contest charges $25/trade, sheesh. Although I don't know whether that nor slippage are affecting this all that much, these are your settings and then some others you might want to try.

set_commission(commission.PerTrade(cost=10))  
set_slippage(slippage.VolumeShareSlippage(volume_limit=0.25, price_impact=0.0))

#set_commission(commission.PerShare(cost=0.005, min_trade_cost=1.00)) # Realistic?  
#set_slippage(slippage.FixedSlippage(spread=0.00)) # Zero slippage  
#set_slippage(slippage.VolumeShareSlippage(volume_limit=0.25, price_impact=0.1)) # Default

Slippage relates to partial fills in part.
Hope others chime in, my comments are not an answer. And I'd like to know how that all turns out.

Some dropped-in code for what it's worth ...

Charlie,
Way to go finding a way to get the kids thinking algorithmically! The cost of commission and slippage are factored into the cost basis for each position, however, in live trading, that value is swapped out for the value that IB reports. I have the commission set to 0 here, and a fixed slippage at zero, you can see that the cost bases roughly equal the limit prices. The fixed slippage is the simplest slippage model, it charges .5*(bid/ask spread), I would use that while investigating other issues because it remains constant throughout the backtest (no volume related partial fill stuff going on). I also added a function to cancel any open orders at the end of the day because that's what happens in live trading.

David

Thanks for the great responses! Updated alg to move URBN purchase to prior day, print open orders as an error.

  • Unfilled Orders: Nice, that was an algorithmic issue, thanks. There was a single unfilled order that Quantopian couldn't fill while the other simulator could. It occurred with an order for URBN that was placed at 9:33am according to democrat-n-chronicle (DnC). Moving that order to the previous day got it filled, maybe 9:33am is too early and DnC really traded earlier somehow. Swapping the entry for the 2 URBN orders that day had the same result, so there isn't anything really obviously at error except it just didn't trade. Updated the algorithm to give an error when a trade doesn't occur, but keeps it open in the hopes it may eventually trade at the limit price and keep the portfolio consistent. Agree that this isn't how International Brokers work.

  • Basis contains trade cost? The $10/trade is what democrat-n-chronicle use. But if I divide $10 by the # of shares... it isn't even close to the differences reported by Quantopian basis on these limit orders. Maybe something else is going on under the hood at Quantopian w.r.t. limit orders and cost basis. Any other thoughts on basis, could it be dividends? There were several dividend payouts shown in the DnC simulator. Could these be added in somehow? If so, why immediately instead of at ex-dividend.

  • It turns out the largest difference between the simulator and Quantopian is ending cash. Quantopian has ~$10,000 more cash (-$35k Quant, -$45k Dnc)... still hunting for this which appears to be the difference.

Updated with open orders error log and move of URBN purchase to prior day

Hey, all set... silly really. The limit order filled properly at lower prices which makes sense. I would have to issue the order closer to the transaction time to get more accurate.

Thanks.

Glad you got it figured out, as for the cash discrepancy, my guess is that it's related to dividends. Quantopian distributes dividends as cash, there might be some difference if the other simulator handles them differently.