John Jay Buchtel asked about Profit by Symbol Traded, and Days Held by Symbol, that's trivial now with this analysis. See the updated NB.

Tom,

Fantastic work. We will put this to immediate and good use.

Have you gotten the Bayesian Sharpe Ratio going yet?

Thank you!

Glad it's useful John. The Bayesian tear sheet should work fine, here's an example. Let me know if you have any other questions.

Awesome!!

On the Profit by Symbol Traded, first of all, fantastic, it is exactly what I wanted to see, and secondly I'm seeing a problem with (same-day) reversals whereby a reversal from long to short produces a profit about equal to - the position size.

I'm definitely seeing total profits not equal total profits in the back test, so one idea would be to put a line in somewhere that is like:
if calculated_profits != backtest.profits:
print_in_bold_letters('Profit Error SomeWhere')

Looking at the code, grouby_consecutive aggregates tnx so that, even if you start with:

Day Qty
1 +100
2 -100
2 -100

it ends up looking like:

Day Qty
1 +100
2 -200

Then when you close the trade you are netting the full $ amount of the day 2 tnx against the full $ amount of the day 1 tnx which probably messes things up going forward so a following tnx of:

Day Qty
3 +100

would open a new trade instead of closing the last one.

Maybe one way to fix it, would be to force a new tnx when there is a reversals at the end of groupby, something like:

prev_shares = 0
for tnx in tnx.sorted_by_symbol_and_date()
total_shares += tnx.shares
if(prev_shares > 0 and total_shares < 0 or prev_shares > 0 and total_shares < 0) then
....split reversal tnx into 2 pieces, -prev_shares in 1, and the rest in the other

So it ends up looking like:

Day Qty Sequence
1 +100 1
2 -100 2
2 -100 3
3 +100 4

Then, the rest of it might work fine with a small change incorporate the sequence

Indeed!

orders:
+100 @ 10
-200 @ 15
+100 @ 5

ought to be two trades:

trades:
long 100 @ 10 -> 15
short 100 @ 15 -> 5

Need to handle within-order position zero-crossing. Ditto deleveraging trades from an average cost basis:

orders
+100 @ 10
+100 @ 12
-100 @ 13
-100 @ 14

trades:
long 100 @ 11 -> 13
long 100 @ 11 -> 14

Hopefully soon!

Simon.

Thanks for the careful analysis. I think the case of 0-crossing you are describing should already be handled correctly. Specifically:

                for price in indiv_prices:  
                    if len(sym_stack) != 0:  
                        # Retrieve last dt, stock-price pair from stack  
                        prev_dt, prev_price = sym_stack.pop()

                        pnl += -(price + prev_price)  
                        cur_durations.append(dt - prev_dt)  
                        cur_open_dts.append(prev_dt)  
                        invested += np.abs(prev_price)

                    else:  
                        # Push additional stock-prices onto stack  
                        sym_stack.append((dt, price))  

Thus, your short on day 2 of 200 shares would first accumulate 100 PnLs until sym_stack is empty and then keep filling up the sym_stack again with the left-over 100 shorts which would not be counted towards the pnl of that trade. Correct me if I'm missing something.

Actually, reading again through that logic I see a bug where it would only work correctly for the first left-over share which would make len(sym_stack) != 0again. It would then flip-flop between these two branches. Adding a check for the sign like in the first if-branch should fix that.

OK, I think I fixed the issue. I also added the example we talked about to show that it's doing the correct thing if there's a 0-crossing.

Awesome, thanks for double-checking.

OK, this has a couple of improvements:
* The aggregation function is more robust now.
* Uses vwap to compute prices.
* Compute true returns.
* Better stats (summary, PnL, returns, duration, symbol-wise).

It will be nice to see in this trade book commission and slippage expanses.

The Order object has a float commission field that stores the amount in USD charged on this order. Quantopian help incorrectly states that its name is Commission and type Integer. See https://www.quantopian.com/help, click "Order properties" on the left margin.

Does anyone know how to retrieve the price (or amount) actually charged on an order, or order part if filled in several chunks? Is it something like data[security].price or data[security].vwap(1) at the time the order or chunk has just been filled?

This is a great tool. Thanks for the hard work on it. One issue that I've run into, however, is that fractional shares are dropped. When the number of shares are converted to an abs_amount with:
abs_amount = int(abs(t.amount))

I realize you do this to be able to multiply a list of prices by the number of shares, giving you a list with that many members. Can anyone see a way to retain and deal with the fractional amounts?

OK, so I've got a first-cut at fractional share support, but it was basically a rewrite to do the math rather than the previous approach. I get different durations because I'm using FIFO ordering (let me know if that's not common practice). Anyway, please provide feedback as I'm not sure I'm doing the dates and durations correctly. It's super tedious, ugly code, so a code review would be awesome.

updated (cleaned up) notebook attached with fractional share support.

Thanks Thomas, Travis, and whoever else contributed to this -- it looks extremely useful. However, when I use Travis' last notebook with a backtest of my long-only algo it shows a large proportion of short trades in the 'Summary trade stats' section. Is there anything I should check?

Stephen,

If you can provide a reference to your bt data, I'll be happy to track down the problem. I'm sure it's a bug.

Backtest ID: 58a62c085de6215e040afce0

I'm getting this...

bt = get_backtest('58a62c085de6215e040afce0')  
Share  
  0% ETA:  --:--:--|                                                          |

NoSuchAlgorithmTraceback (most recent call last)  
<ipython-input-1-e22635b1e622> in <module>()  
----> 1 bt = get_backtest('58a62c085de6215e040afce0')

/build/src/qexec_repo/qexec/research/api.py in get_backtest(backtest_id)
    260         client.get_sqlbacktest(backtest_id),  
    261         progress_bar,  
--> 262         backtest_id,  
    263     )  
    264 

/build/src/qexec_repo/qexec/research/results.py in from_stream(cls, result_iterator, progress_bar, algo_id)
    536         progress_bar.start()  
    537  
--> 538         for msg in result_iterator:  
    539             prefix, payload = msg['prefix'], msg['payload']  
    540 

/build/src/qexec_repo/qexec/research/web/client.pyc in get_sqlbacktest(self, backtest_id)
    360         Returns a generator of perf messages for the backtest.  
    361         """  
--> 362         resp = self._make_get_sqlbacktest_request(backtest_id)  
    363         with closing(resp):  
    364             for msg in resp.iter_lines():

/build/src/qexec_repo/qexec/research/web/client.pyc in _make_get_sqlbacktest_request(self, backtest_id)
    350         # TODO: Should this just plug into the handle_json_errors machinery?  
    351         if resp.status_code == 403:  
--> 352             raise NoSuchAlgorithm(backtest_id)  
    353  
    354         return resp

NoSuchAlgorithm: Algorithm with id '58a62c085de6215e040afce0' was not found.```  

That ID works for me... do I need to do something special to share it? I'd prefer to not share the entire algo and its source code.

Stephen,

I'm new to all this stuff, but I did find this from an old post:
~~~~~~~~~~~ Hello Daniel,

I'm sorry it was confusing. When you're on the "Full Backtest" result page, upper right, there is a "share backtest" button. Super long version:

Go to My Algorithms
Click the algo you want to share
Click "build" to test the algo
Click the blue "full backtest" button to run a full backtest
Click the blue "share backtest" button to share it here with the commmunity

Stephen,

I was able to pull your backtest data. I've corrected a bug in the long/short flag (I had it a bit backwards), but your long-only strategy looks like it's giving some short positions. Please see the attached notebook to check me on this.

Best,
Travis

Thanks Travis. I'm still checking my algo but so far don't see any way it could produce short positions. For one thing, this is in initialize():
set_long_only()
And as far as I can see with the logging I included order_target_percent() is always called with positive numbers.

Stephen,

I've never written an algo, but I think that must be the source of the problem (the code that is trading the algo). The cumulative shares for a particular symbol should never dip below zero on a long_only algo, and they most certainly do in that list of transactions. Wish I could be of more help, but perhaps someone with algo experience could chime in?

Best,

Travis

Travis, I looked through the Daily Positions and Gains section of the backtest results and don't see any negative (short) positions. Because of that I think the round-trip analysis is flawed.

While it's highly likely that the round-trip analysis is flawed, I'm still confused about how your cumulative amounts in your transactions can swing negative unless there is also a problem with how the algo is being traded by the system. Maybe I'm just looking at the wrong things, but there's a weird 'floor' to some of your symbol positions. Please take a look at the attached notebook with the chart of the cumulative amounts of QQQ and XIV. QQQ looks right -- XIV, not so much.

The algorithm is adapted from one shared by Elsid Aliaj on 2-2-2017:
https://www.quantopian.com/posts/the-spy-who-loved-wvf-or-just-another-volatility-spy-strategy#5893fd88acdeab61094afc59

For further verification of the long-only nature of the algorithm I added logging to all the places where trades are made. Only order_target_percent() is used for trading and the target percentage parameter is always positive. Here's an example:

for i,stock in enumerate(context.stocks):  
    order_target_percent(stock, allocation[i] * 0.5 * context.leverage)  
    if allocation[i] < 0:  
        log.info("SHORT: " + stock.symbol + ": " + str(allocation[i]))

I always give context.leverage a positive value. The 0.5 is there because this purchase should only be 50% of the portfolio. The test in the last two lines above results in no "SHORT" entries in the log, confirming that the algo stays on the long side.

The only other place XIV is traded is to exit the position with a target percentage 0.0:
order_target_percent(xiv, 0.00)

I also use record() to show the portfolio positions on a daily basis. The lines never dip below zero. For example:

record(XIV=(context.portfolio.positions[symbol('XIV')].amount * context.portfolio.positions[symbol('XIV')].last_sale_price)/context.portfolio.positions_value)  

Just to be clear, the data plotted above are just your transaction data from the backtest you provided. This has nothing to do with the round-trip code. When I plot that data--your data--you clearly have an amount that sums below zero shares. It happens when you've accumulated a total of 7 shares of XIV in all the transactions through 6/28/2011 and then, on that date, you sell 52 shares, giving you a total of -45 shares. Beyond that, there are several other trades that give you a total below 0. There is a floor at -63 shares, so I think there's a bug somewhere.

I understand that you are looking at the output of your record() and the math on the algo ... I'm just looking at the actual transactions from your backtest (that's what the round-trip analysis would have to use). Can anyone else chime in as to why the transaction cumulative amounts on a backtest would be negative, while the positions recorded never dip below zero?

I think you're right, we need some help with this. As I posted above I've searched for any possible way for my algo to end up with short positions and it doesn't seem possible. I hope the Q backtest engine doesn't have a bug.

• Using set_long_only()
  
• Using only order_target_percent() for trades with target_percent parameter always positive or zero
  

Something else that may be of interest is the attached tearsheet for that backtest. I don't see any indication of short positions in the 'Exposure' and 'Portfolio allocation over time' sections near the bottom.

Travis wrote:

Just to be clear, the data plotted above are just your transaction
data from the backtest you provided. This has nothing to do with the
round-trip code. When I plot that data--your data--you clearly have an
amount that sums below zero shares. It happens when you've accumulated
a total of 7 shares of XIV in all the transactions through 6/28/2011
and then, on that date, you sell 52 shares, giving you a total of -45
shares. Beyond that, there are several other trades that give you a
total below 0. There is a floor at -63 shares, so I think there's a
bug somewhere.

I think I've found what's causing the discrepancy: on 6/27/2011 there was a 10:1 split for XIV. As far as I can tell the backtest handles it correctly with regard to the final results. However, that split may be causing the trades to be recorded incorrectly, which in turn throws off the round-trip analysis.