Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
'portfolio.pnl' Calculation Error? Example provided...

This is a 2800 short sell on AAPL at the indicated date and time based on 1 minute bars. I ran this example several times and stepped through code to confirm the result error. Why doesn't 'order_info' indicate a fill price? What's going on with 'portfolio.pnl' calculation?
Thanks,
Mark

c.order_info
Order: -2800 shares of AAPL (Limit $NaN.N Stop $NaN.N, Open)
amount: -2800
 commission: NoneType  created: 2014-01-06 14:33:00  dt: 2014-01-06 14:33:00 filled: 0
id: 0e0e0cbd79974e109294305d0369ea9f
 limit: NoneType limit_reached: False
 reason: NoneType  sid: AAPL (24) status: Open (0)
 stop: NoneType

(same bar as order)

Open[0]
76.780107
High[0]
76.93011
Low[0]
76.57296
Close[0]
76.6458186
c.portfolio.pnl
-38.0133027553

(next 1min bar after order - it appears that order was filled at prior close based on 'pnl' calculation below)

Open[0]
76.6572474
High[0]
76.6929624
Low[0]
76.2200958
Close[0]
76.2200958
c.portfolio.pnl
-7.01832843106
(This negative $7 pnl doesn't make sense. The 'pnl' it appears to calculate the 2800 share short sell order against the 76.657 open bar price assuming the order was 'filled' at the previous 76.645 bar close. I would assume a 'new' order would fill at the next 1 min bar open price. The context.order_info shown above doesn't indicate a fill price, and subsequent context.order_info information does not update the order fill info)

(2nd next 1 min bar after order)

Open[0]
76.3215264
High[0]
76.4686722
Low[0]
76.2429534
Close[0]
76.2429534
c.portfolio.pnl
-446.021462755 (Where did this negative value come from? All bar values are lower!)

7 responses

Here is the code example that I mentioned in the post title. Set the backtest parameter to minute and start on January 3, 2014.

The backtest results are...

2014-01-06PRINT('0933', 'Prf 0', 'Open 76.772', 'High 76.781', 'Low 76.573', 'Close 76.573')
2014-01-06PRINTEvent({'status': 0, 'limit_reached': False, 'created': Timestamp('2014-01-06 14:33:00+0000', tz='UTC'), 'stop': None, 'reason': None, 'stop_reached': False, 'commission': None, 'amount': -2800, 'limit': None, 'sid': Security(24, symbol='AAPL', security_name='APPLE INC', exchange='NASDAQ GLOBAL SELECT MARKET', start_date=Timestamp('1993-01-04 00:00:00+0000', tz='UTC'), end_date=Timestamp('2015-02-23 00:00:00+0000', tz='UTC'), first_traded=None), 'id': 'c36d7781e2cc4215af1d1d622dc19df3', 'dt': Timestamp('2014-01-06 14:33:00+0000', tz='UTC'), 'filled': 0})
2014-01-06PRINT('0934', 'Prf -7', 'Open 76.657', 'High 76.693', 'Low 76.220', 'Close 76.220')
2014-01-06PRINT('0935', 'Prf -415', 'Open 76.322', 'High 76.469', 'Low 76.243', 'Close 76.243')
2014-01-06PRINT('0936', 'Prf -671', 'Open 76.453', 'High 76.559', 'Low 76.417', 'Close 76.417')
2014-01-06PRINT('0937', 'Prf -439', 'Open 76.559', 'High 76.559', 'Low 76.413', 'Close 76.413')

import pandas as pd
from datetime import datetime

def initialize(context):

c = context  

columns = ['index', 'open', 'high', 'low', 'close', 'volume']  
c.s1 = pd.DataFrame(columns=columns)  
c.order = ""  
c.order_info = ""  
c.raw_time = ""  
c.PosProf = 0  
c.beg_time = "0933"  
c.end_time = "1557"

set_slippage(slippage.VolumeShareSlippage(volume_limit=0.25,price_impact=0.05))  
set_commission(commission.PerShare(cost=0.0025))

def handle_data(context, data):

c = context

# Eastern time  
c.raw_time = get_datetime('US/Eastern')  
Time = int(c.raw_time.strftime('%H%M'))  
Date = int(c.raw_time.strftime('%y%m%d'))  

# check for stop loss during trading session beyond 233 bars  
if Date == 140106 and Time >= 933 and Time < 945:  

    open = history(1, '1m', 'open_price')  
    high = history(1, '1m', 'high')  
    low = history(1, '1m', 'low')  
    close = history(1, '1m', 'close_price')

    c.s1 = pd.DataFrame({'index' : close[sid(24)].index,  
                                'open' : open[sid(24)].ix[0],  
                                'high' : high[sid(24)].ix[0],  
                                'low' : low[sid(24)].ix[0],  
                                'close' : low[sid(24)].ix[0]})  
    c.s1 = c.s1[['index', 'open', 'high', 'low', 'close']]  

    Open = c.s1['open'].values  
    High = c.s1['high'].values  
    Low = c.s1['low'].values  
    Close = c.s1['close'].values  

    if Time == 933 :  
        print(c.raw_time.strftime('%H%M'),'Prf %4.0f' % c.PosProf,  
            'Open %2.3f' % Open[0], 'High %2.3f' % High[0], 'Low %2.3f' % Low[0],'Close %2.3f' % Close[0])  
        c.order = order(symbol('AAPL'), -2800)  
        c.order_info = get_order(c.order)  
        print(c.order_info)

    # check positions value  
    c.PosProf = c.portfolio.pnl

    if Time > 933 :  
        print(c.raw_time.strftime('%H%M'),'Prf %4.0f' % c.PosProf,  
            'Open %2.3f' % Open[0], 'High %2.3f' % High[0], 'Low %2.3f' % Low[0],'Close %2.3f' % Close[0])  

The API doesn't describe how the order is placed, 'next bar', open price? I am assuming the market order is placed as the next bar open price. The portfolio.pnl position value calculation is not obvious. What is the basis for the 'pnl' order value and calculations from bar to bar? If the short sell negative amount is specified as a positive long amount, portfolio.pnl values become positive.

Mark, orders are filled at the next bar's closing price I believe. As to the details of the pnl calculation, see here for the source code: https://github.com/quantopian/zipline/blob/master/zipline/finance/performance/period.py#L284

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Thanks for the portfolio code reference. Since position transaction entry and exit sequence is essential during 1 min bar action, is there any source code to specify the transaction sequence? As I mentioned above, when a 'market' order is placed, during back test entry should be filled at the open price on the next bar, not the close price, and then subsequent position value should be calculated on each position at each bar close price versus the initial fill open price. From my example code results "Timestamp('2014-01-06 14:33:00+0000', tz='UTC'), 'filled': 0))" - The order object never identifies that order is 'filled:0' or provides a fill price to confirm the fill, even when checking the 'order_info' during later bars.

On the example I provided, I can't calculate any reasonable combination of bar prices that provide the erroneous position value supplied by portfolio.pnl. Example short sell position values should be positive, not negative expected applying a 'next bar' open price fill.

I’ve added a ‘stop loss’ component to the example to highlight the order fill and portfolio calculation problem…

The maximum negative profit value is at 0941 bar. Back-calculating the ‘loss’ -> -$868/2800 = -$0.31/share difference from the unidentified fill price
Estimating the fill price from the 0941 bar High[0] -> 76.644 -0.31 = $76.334 estimated order fill price.
Estimating the fill price from the 0941 bar Close[0] -> 76.466 -0.31 = $76.156 estimated order fill price.

Since the 0934 bar indicates ‘0’ value, it appears that the order was somehow filled at one of the 0934 bar Open/High/Low/Close prices. None of the 0934 bar prices are near the $76.334 or $76.156 prices. The Low and Close are both $76.220 and the Open and High are $76.657 and $76.693 respectively. Even taking an ‘average’ between the Open and Close is $76.439.

Additionally, if a stop price is explicitly specified $0.15 above the ‘estimated’ fill price $76.334, or a stop price of $76.534, a stop loss is reached at the 0936 bar where the High alone is above the designated $76.534 stop loss price. This confirms that a stop loss is triggered from exceeding the High price.

That still doesn’t help setting a proper stop price since the 0934 bar prices range from $0.11 (Close) below to $0.32 (Open) above the above estimated fill price. How is someone supposed to ‘guess’ at an appropriate stop price no reference order fill price is supplied?

Another confusing issue is IF the order was filled at the above estimated $76.334 price, and portfolio.pnl values are determined at each bar’s Close, how does portfolio.pnl calculate a -$408 loss for the 0935 bar? That represents a fill price $0.146 higher than the 0935 bar Close, or an estimated fill price of $76.243 (Close) + $0.146 = $76.389. This is yet another 0934 bar fill price estimate that doesn’t match any 0934 bar price!

What is going on here?

It appears that Zipline financial trade transactions are designed to fill orders with an end of next bar close fill price following a trade order, instead of the normal professional trading platform practice of filling a trade order at the beginning next bar opening fill price https://github.com/quantopian/zipline/blob/master/zipline/finance/slippage.py . According to my comments above, there appears to be a problem assigning the order fill close price, substituting an almost random fill price, that almost negates the effect of specifying an 'optimized' stop loss price. Since the main criteria for waiting to the end of the next bar is simply to calculate the order volume versus trading volume, why not apply the trade order to the end of the current bar close price? This practice would more normally coincide with the professional trading platform practice of filling market orders with the next bar open price. Anyone agree?

Thanks for your support in the suggestion,

Mark

Two updates, first I found a coding error in my example code that created the impression of an error in the portfolio.pnl value calculation. I had accidently substituted 'low' instead of 'close' in the dateframe creation list. The corrected 'close' pricing validates the portfolio.pnl value calculation. I appreciate the community responses and comments by everyone, including comments by the Quantopian support team.

Second, Quantopian suggested a Quantopian Community discussion regarding using the next bar Open price as a custom slippage model https://www.quantopian.com/posts/trade-at-the-open-slippage-model. Only the default slippage model is allowed during the current algorithm competition.

Sorry for the confusion on the Close price issue,

Mark