Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
Custom P&L measurement does not reflect backtest results
Anony Mole
edited Jan 19, 2014

I'm doing some vetting of strategy efficiency and coded up some simple P&L metrics. When I run and tabulate the running P&L from my code I get vastly different results than what I get from the backtest engine.

I extracted the P&L measurements from the log statements and dropped them into excel and matched the recorded metric in the Quantopian chart. Now, I realize that I can't technically get the true P&L for a trade as I'm not privy to the actual closing price of the trade. But I should be able to get close. And I'm not. At least, I don't think so.

If you run this strat, you'll see the PandL line. This comes from keeping a running PnL value and incrementing (decrementing) from it on the close of every trade. Again, I won't be completely accurate - but what I'm seeing is nowhere close.

So, what's going on here?

Simple strat logic:
produce custom metric
smooth it twice.
when fast ma moves above slow ma,
- close open position
- buy
when fast ma moves below slow ma,
- close open position
- sell
if unrealized position P&L > 100
- close trade

Here's how I'm retrieving the P&L at the time of the trade:

def GetPositionPandL(context, data, securityID):  
    openQty = context.portfolio.positions[securityID].amount  
    if (openQty == 0):  
        return 0  
    entryPrice = context.portfolio.positions[securityID].cost_basis  
    currentPrice = data[securityID].close_price  
    entryNotional = entryPrice * openQty  
    currentNotional = currentPrice * openQty  
    if ( openQty > 0):  
        return currentNotional - entryNotional  
    else:  
        return entryNotional - currentNotional
6 responses
Dan Dunn
Jan 21, 2014

Anony, got your update tonight. I'll dig into it and get back to you.

Sorry the reply wasn't as fast as we usually are, but today was a market holiday. Most of us took some time off. Tomorrow we'll be back at full speed.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Alisa Deychman
Jan 23, 2014

Thanks for being patient with us! I started digging into this tonight and I want to make sure that I understand your concern. You're worried that your running P&L doesn't match the overall algorithm return, the blue line? Is that what you were trying to replicate?

I have some initial thoughts but need to do some research and I'll circle back to you.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Alisa Deychman
Jan 23, 2014

That's great to see! Happy that it worked out for you

Dan Dunn
Jan 23, 2014

I'm glad that there isn't an issue. If you find something you'd like us to look at, we will.

I dug into this with Alisa today. I think that the running PandL in the algorithm was trying to match the algo backtest results. The thing is that the backtester is actually pretty complex, and the simple PandL was missing some important nuances. Dividends, commissions, and slippage were things that I was going to look at next.

Also, I wanted to make sure you understood how the backtester works filling orders in daily mode. The backtester places an order in one bar, and then it is filled in the next bar. The close price of the bar is used for both calculations. That means in daily mode, if your algo places an order on a day, it is filled the next day - at the close price, not the open.

Dan Dunn
Jan 23, 2014

When you're trading in daily mode, yes, it would look exactly like that.

What you really should do is change your algorithm to minute mode. Then, in your chart, it would read "Minute1 Minute2. . . " Of course, you don't have to trade every minute. You can make it Buy ABC at the first minute, sell ABC at noon, by DEF at 12:01, sell DEF at 3:45, etc. It's up to you how you want that to play out.

Trading View does have some very pretty graphs. Definitely some good stuff there.

Russell Hanson
Apr 18, 2014

I think this function needs an abs() around the number of shares for short positions to be calculated properly. 

def GetPositionPandL(context, data, securityID):  
    openQty = context.portfolio.positions[securityID].amount  
    if (openQty == 0):  
        return 0  
    entryPrice = context.portfolio.positions[securityID].cost_basis  
    currentPrice = data[securityID].close_price  
    entryNotional = entryPrice * openQty  
    currentNotional = currentPrice * openQty  
    if ( openQty > 0):  
        return currentNotional - entryNotional  
    else:  
        return entryNotional - currentNotional
About Quantopian
Careers
Community
Events
Documentation
Academia
Lectures
Status
Facebook
Twitter
LinkedIn
YouTube
Terms of Use
Privacy Policy

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian.

In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.