Same code as above using a GUI value of one dollar instead of one thousand. (GUI stands for Graphical User Interface)

To get around that issue and get useful metrics and a reasonable comparison with the benchmark graph, I've been computing my shares to purchase based upon the account value as was done in the examples. Then if making final preparation for live trading, replace the bogus share computation with the actual number of shares to trade. Still, it would be good if that info was documented.

This time the initial capital is the default $1,000,000 (just has some code cleaned up) where Returns are reported as 0%.
Notice the real Returns value if you run it is actually still 846%. $24.41 was spent to make $230.96. 100 * ($230.96 - $24.41) / $24.41

Changing the initial capital should not cause large changes in results, instead, the calculation should be based on capital used.
https://www.quantopian.com/posts/cash-vs-returns also has some discussion on this.

The nomenclature of the site is correct: if you declare a value for capital, then do not deploy any of it, your "return on capital" is zero. If you'd like to calculate your own metric and track it, awesome, but the site is not wrong.

All opinions welcome.

My perspective: The results equation is meant to represent output per actual input, and replacing starting_cash with a drawdown model will remedy many issues all at once:
1. People will have a sound and certain reflection of their code performance immediately without needing to worry about adjusting initial capital for different results followed by re-run, they will know right off the top how well their code is really doing.
2. Suddenly Alpha, Beta, Sharpe, Sortino, Volatility, Drawdown, Information Ratio and Returns percentage as well as the chart will be trustworthy as to the true performance of their algorithm with certainty no matter what the initial capital value is set to.
3. Evaluations for the hedge fund Quantopian Managers Program will be deterministic vs variable based on initial capital.
4. Negative cash will no longer be ignored as if it were irrelevant (it is currently cast aside in the results calculation, yet it is a part of real input).
5. Unused cash will no longer be included in the calculation (it was not used and therefore is not part of real input).
6. I think it is a safe bet that one of the first things many people consider when encountering a backtest (in the community) is the right side of the chart. That is taken to be an indication of how well the algo does, so they will be able to trust that result instead of the current situation where they would have to observe the initial capital and analyze the algorithm contents and run the algo with additional code to find out how much capital was actually used and whether the code ventured into negative cash (and/or to what degree) or did not use much to be able to determine where that chart result falls in the spectrum between good and bad.
7. There can no longer be charts that look good while the code is actually bad or charts that look bad while the code is actually good.
8. The Quantopian brand will be improved for a great 2015 and beyond.

Those are my opinions, formed in part by having added Run Summary to probably more than 100 algorithms of others while testing that and observing the results. (I have struggled over this same issue in that code, stubbornly came around to this conclusion, and candidly I think it currently has a bug wrt to this by the way, need to muster up the energy to make it righter).

Q could draw up a version with the change, make available around the office and round up some thoughts, and if all goes well, then to backtests, eventually live.
A start could be changing these two lines in performance/period.py, need to bring in max_drawdown from the rval dictionary I think, and then try 'er out and see how it looks.
An additional condition will be necessary to handle short selling.

    def calculate_performance(self):  
        self.ending_value = self.calculate_positions_value()  
        total_at_start    = self.starting_cash + self.starting_value  
        self.ending_cash  = self.starting_cash + self.period_cash_flow  
        total_at_end      = self.ending_cash + self.ending_value  
        max_drawdown      = self.max_drawdown  
        #self.pnl         = total_at_end - total_at_start

        if max_drawdown != 0:  
            self.returns = (total_at_end - max_drawdown) / max_drawdown  
            #self.returns = self.pnl / total_at_start  
        else:  
            self.returns = 0.0  

I do agree with you Gary, but also Adam is right. What we need is basically a metric that expresses the return on capital usage and the "max_drawdown" would express that. In the examples above...although the return with a million is low in percentage the risk is also almost nil, currency risk aside. In my algorithms I deploy I create on purpose a cushion that will cater for the settlement delay of my account (Ozzies cannot have a margin account, they are legally forced to take a cash account) to prevent drawing into negative and therefore not executing my algorithms properly. That means that on fast-trading algorithms I can only use 50% of my deployable capital which in the end would make my Quantopian return low but my capitial at risk return would be equal to a fully invested account.

The return calculation is missing a super important thing (I mentioned this to you earlier Gary): Of all buy and sells I did what % of my transactions were profitable and of all transactions what was the average profit when there was a profit and what was the average loss when there was a loss. I tend to optimise for those %'s and absolute figures to determine improvement of my algorithm.

Of course this needs some extensive coding around the edge scenario's (shorts, scaling in, taking partial profits and what's left in holding at the end) but I think it would increase the tinkering a lot. I have this on my todo list as an extension on the RunSummary code but time seems to slip away in life ;).

@PB I remember, and those ideas sound really useful, similarly I tried to code a rebalancing that would reward winners with more weight and visa versa, got stuck, shelved for now. Lately I've been wrestling with trying to calculate returns on individual securities, really tough.

Alright, especially with the paper trading $100,000 Quantopian Open prize/challenge, which is connected to later live trading, I'd like to invite us all to consider a scenario:
Two people each run the same algorithm.
One starts with $10,000 initial capital.
The second starts with $1 million.
After some time, the total drawdown for each is $9,500.

Here, an algo bought some shares once a month during 2014 and was run six times where the only change was the Initial Capital. The more money in the account, the worse the results.

So currently, it would seem there are two ways to achieve a good looking result real/money or paper:
1. Try to keep only enough money in the account to be able to buy whatever amount the algo might need.
2. Establish a margin account and be borrowing on it (the IDE will not reflect reality).
This also means you cannot really use the metrics to gage how well your algo is doing, at least not very easily (you would have to set it to however much your algo uses, however some code will trade differently based on the starting cash). If you change the initial capital, it will change the results in this way in the case of a basic algo.

Below are screenshots of four of the six runs and the one at the bottom includes the custom chart of calculated Returns.
The key point is: For each run, the custom chart Returns using (profit / drawdown) were exactly the same as the one you see in the custom chart, initial capital becomes no more significant than cash in one's own bank account at home.
Since Alpha, Beta, Sharpe ... are calculated from Returns, those also would have all been consistent across the board in all six runs had Returns been calculated by (profit / drawdown) instead of initial capital.

Suppose you're an analyst working for a hedge fund, and, your boss the portfolio manager says, "Here's $1,000,000 of the fund's money for you to invest as you see fit."

You decide that "as you see fit" is to buy $10,000 worth of stock, and leave the other $990,000 of cash sitting in your account, untouched.

A year later, the $10,000 worth of stock you purchased doubles in value, to $20,000.

You go to your boss and say, "Look, I made a 100% return on investment!"

Your boss will fire you. You did not make a 100% ROI, you made 1%.

Quantopian's metrics are your boss at the hedge fund. If your algorithm has a capital base of $1,000,000, then your algorithm is expected to maximize the value it can extract from that $1,000,000. If your algorithm decides to only invest $10,000 of that $1,000,000, then your algorithm is making poor decisions and is likely to perform poorly.

There may be another way to think about this return business which makes sense to everyone:

A single Quantopian trading strategy does NOT represent your full unit of measure when it comes to trading one's "account."

Imagine if one's retirement account contained $1M (I wish). You would never invest all of that into a single strategy. Never. Yes, each of these Quantopian trading strategies is pinned to and controls a single IB account -- that's true. But you would never just dump your whole $1M into that single IB account and have one strategy work that entire financial resource. The truth is much more complex, but can probably be simplified. The simple solution is to cut your $1M into smaller chunks and trade each independently. The complex is touched upon below.

Undoubtedly one would take a portion of that $1M and attribute it to a single strategy (no matter how many instruments it traded). The amount you devote to a single strategy would be your risk tolerance level for this strategy. Say the strategy trades insurance + home improvement companies during hurricane season. Maybe it's pretty risky; 30% drawdowns but 50% returns in heavy weather years. Would you put your whole $1M into that strategy? Hell no. Maybe 10% of your total account. More likely 5% or so. But, and here's the point about returns... you would expect that strategy to leverage ALL of that $50k right? You're managing total account risk outside of this strategy. Concerning your full account, maybe you're 50% invested in equity trading strategies. 30% in fixed income 10% cash and 10% precious metals. That's you managing risk outside of any one strategy. This particular strategy, however, does not know this, nor does it care. This hurricane strategy has one task, take $50k and work it, fully.

So $50k for this one strategy is about right, for it and it alone. Just one chunk of your $1M. (Or $500k for equity trading as stipulated above.)

You then have other strategies you would devote a portion of your account into. Each of which would assume that it had complete control of its total balance and that it should leverage, to the maximum level permissible, its account resource.

The complex part is what does strategy apportioning risk mean when it comes to a strategy which has its own internal trade risk metrics and measurements. Does a safe, well diversified strategy merit 10%, 20% of your total account? Regardless of the portion of your total account you attribute to any one strategy you would expect the strategy to be able to trade its sub-account balance fully, or not as it sees fit. You have risk within risk now. This complicates the sub-account portioning. But a simple way to look at this is never risk more than the standard 2% of your total wealth in any one strategy. If a strategy has a yearly drawdown of 20% then you could invest a maximum of $100k in it. (20% of 100k = 20k which equals 2% of $1M). That's the complex part.

I know IB provides for the creation of sub or "client" accounts to which some portion of a master account can be attributed. The Q supports this ability and so this is the right way to approach division of risk among multiple strategies. Each of which retains full control over its own portion allotted to it. (Thanks Jonathan K.)

The basic concept here is that a single Quantopian strategy does not represent your whole investing scheme. Its one slice of a bigger account pie. It should, however, be able to leverage that whole slice fully. Not doing so defeats the strategy's singular purpose of producing a maximum return for the funds given it.

@Market Tech, yes, that's exactly right.

To answer your question, as I understand it, not only do we support the use of dedicated per-algorithm IB sub-accounts, we encourage it, and in fact we only allow one algorithm to trade against any particular IB account number at any given time.

Well, my take is that if, for example, you end up holding a portion of the portfolio in cash, this is just another allocation decision that impacts returns. If there is "money on the sidelines" it counts. I'm not sure that Quantopian needs to conjure up another metric, since one can always set the cash to zero in your algorithm, if you want to understand the "all in" return. --Grant

"since one can always set the cash to zero in your algorithm". Please explain.

There are fine algos where the quant is led to believe they are awful by everything shown in the IDE, and frequently algos are posted where the strategy is bad yet the IDE makes them appear to be great. The current condition allows a lot of room for confusion.

[6/2016 removed a less important comment about drawdown]

Another point I wanted to add, returns in the industry have been thought of in more than one way:
http://en.wikipedia.org/wiki/Rate_of_return
(they don't discuss maximum on the first so not entirely clear to me)

@JK Your 100% gain per year is fantastic code, all the quant would have to do is adjust cash usage.
The way you have it set up, the quant will be fooled into thinking it is garbage and will throw it away.
Can you see why that's bad?
You're talking End Zone, third party investor scenarios.
What about the other 100 yards on the field? What about all of the preparation? Quants working on strategies. Don't you want them to see what's really going on?
Why do you ignore negative cash borrowing as if it never happened?
Why do you count unused cash in the returns calculation? It's no different than the million buried in one's basement.
The word "Returns" is "Returns" for a reason, it is return on what was invested/activated.

Gary,

For example, if you have a long-only algo, and you use order_target_percent, with the total stock allocation 100% in the market, then the cash will be approximately zero. There is no setting, but the code can be written so that you allocate 100% of the capital to the market, rather than leaving some in a cash buffer.

Grant

@GK Impossible. As anyone with experience knows, as soon as you make some changes, different stocks are opened/closed at different times and especially with the $10M contest dollars, partial fills are entirely unpredictable so that argument is invalid.

[Edit: Some extraneous wordiness removed later by author and the above changed 10/2016]

It took me some 11 months last year to wake up to what was happening.

Ok, so what now. This post has comments from only 7 people while over 1100 views and very few have cloned the algos. Please clone/run one of the shared backtests in this post, modify the initial capital a few times and notice the results, seems to me that would help grasp what's happening (I suspect many do not really comprehend it) and why it ought to matter to most of us.

There must be some who agree with me. You're out there, I could use some back up.

Two children are given $100 and told to buy trading cards - for "investment" purposes.

Child one goes and buys 100 cards at $1 a piece - "the market looks good" she says.
Child two buys 1 card and retains $99 of the cash - "the market looks bad" he says.

A year goes by.

Child one sells all of her cards which increased a penny apiece, net return $1.
Child two sells his 1 card but his card doubled in price, net return $1.

Are you saying that child one made 1% in profit while child two made 100% profit?

Let's say we repeat the experiment restarting with $100 apiece.
Both children buy identical numbers of cards as they did before, 100 cards for child one, 1 card for child two.

Another year goes by.

This time all cards for all children have doubled in value.
Both children cash out.
Child one now has $200, net return $100.
Child two has $101, net return $1.

Are you going to say that both children earned 100% profit on their capital?

Both children had $100 to invest. That was their capital. What was the return on capital during their investment? Return on allotted capital is the measurement here. Whether you buy 1 share or 100 is up to you. But the returns are measured against allotted capital. That capital is unavailable for other investment. It has been dedicated to this investment strategy and cannot be used for another purpose. If the strategy does not invest the capital then it must remain in the account, untouched. Yet it still forms the basis for the return on capital.

You may be thinking that untouched capital is somehow available for other strategies to use. It is not. This is not to say that all capital must remain invested at all times. But the strategy must have available to it (and it alone) all the funds allotted to it so when the time comes the cash is there to invest.

Child three bought $500 worth of cards on that $100, was never informed of the fact that borrowing had occurred, made a huge return, said wow look at me, my investment strategy is pure genius, showed an investor the impressive graph from the IDE and exclaimed, see there! My initial capital was only $100 and now look at all this profit on $100!

Leverage is a considerable concern with regards to looking at a simple P&L curve -- true. Without knowing that leverage was employed child three would indeed be crowned an investing genius. So perhaps a metric that stated "MaxLeverage" would be worth knowing. And indeed if you add the context.account.leverage field to a record() statement you can show, at the will of the quant of course, how much cash (over and above initial capital) was actually used. I would be of a mind to dismiss any strategy that used a leverage > 1.0. All profits (or losses) can be multiplied by leverage. But knowing the presence of leverage is the first step. And you're right, this is a buried factor that should be surfaced.

I've read this topic with interest, and had several offline conversations with participants and readers. For what it's worth, this is my thinking:

• We need to do better at helping people manage their leverage, cash, borrowing, whatever you want to call it. We need to make the use of leverage more obvious in shared algorithms too. I readily agree! It's one of the many things that's been on our list for a while. I can give you a list of reasons we haven't gotten to it yet, but it would be a boring exercise. Adding leverage as a first-world risk variable earlier this month was a good step in this direction, and there will be more later this year I'm sure.
  
• There are a number of algo writers who prefer a returns calculation that is based on money used, not money held. I understand why, and I would like to provide it at some point. This feature request isn't quite as high on the list simply because anyone who wants it can calculate it themselves. But, we want to make it easy, so we will at some point.
  
• I'm not persuaded there is a such a thing as "true returns." There are a lot of ways to calculate returns; in different use cases some methods are better than others, and no one method is always the best method.

This discussion is valuable. I'm regularly delighted at how much this community's wisdom is driving Quantopian's development. Thank you all.

Thanks for your input.

I want to throw in another thing relevant to this discussion: Algorithms should be tradeable. One thing mentioned above is the leverage. If you have a leverage higher than 1.3 you are probably not a retail investor.
Another thing is settlements of funds. Some accounts are marked as cash accounts. This means that any strategy that rebalances by selling something on the "signal day" and buying something else in the same bar is not going to be an executable strategy (I found this out through live trading in Quantopian). So to make the strategy work and produce returns in backtesting is something completely different than doing so in real life, as settlement delays and variable leverage allowance is going to influence the Real Returns. So if we could add the type of account to get_environment(), the environment variables of Quantopian (IB has several defined accounts, cash, port.margin, RegT.margin, 401, etc, etc) and we could use those params as a start, then tested returns and real returns might be more alike.

@DanDunn: is that on the roadmap?

This is a good discussion, here's my thought process on the subject.

Cash should be treated the same as any other position in your portfolio. If you have cash in your account, you have a long position on US dollars. Cash just has a very low return (roughly 0% interest) and a very low risk profile.

If you have a negative cash balance, you have a short position on the dollar. Shorting dollars works the same way as shorting any other asset, you borrow the asset and trade it at its current going rate (face value) for some other asset. Later, you can cover your short dollar position by trading the asset for dollars again, if the asset appreciated relative to the dollar (net of interest and fees), then you've made a few bucks on the trade.

As for return on initial capital, if the capital exists, you must account for it, the form the capital is currently in is irrelevant. Suppose I'm your money manager and you give me $100K to invest on your behalf. One day you call me and ask what your current returns are and I tell you that I've made 20%, how much do you assume your account is currently worth?

Being in cash has risk associated, and an opportunity cost. It should certainly be considered when considering return on capital, just like any other asset. I don't see a compelling reason to make an exception, and if anything I'd rather see a more dynamic model -- options, multiple currency classes, FX exchange rates etc. -- rather than a simplification of tracking.

@Dave E. You reminded me of a post I once made on the TWSAPI list that effectively stated that all trades, and even the act of not trading, are decisions to go both long and short in two (or more) entities. When you earn $100 and stuff that into your mattress you are in essence long USD and short everything else (in the world). Removing that $100 and buying shares in some stock you are now long the stock and short the dollar. Removing that $100 and buying anything, silver, wine, groceries, firewood, shoes, anything -- you are, maybe unbeknownst to you, shorting the dollar and buying long whatever it is you purchased. Your post should remind everyone that money in the bank (or in your pocket) is a long bet on the dollar. What should be a strategic long bet on the dollar. Yes with opportunity costs taken into consideration. But who'd of thought that by keeping that $1 in your pocket and NOT buying that Swiss chocolate candy bar you just earned 10%!

Both of Calvin's algos have no leverage--er, margin. (thanks fawce for the correction below)

Here's a realistic example posted by David Edwards where the algo shows only 3.7% however it is actually almost four times higher at 14.4%. That better code quality is not reflected in the metrics because he didn't utilize all of the capital, very common.

Quantopian Tutorial with Sample Momentum Algorithm - Lesson 1: The basics of the IDE
https://www.quantopian.com/posts/quantopian-tutorial-with-sample-momentum-algorithm-lesson-1-the-basics-of-the-ide#54b79bcea35e88ffb4000151

The charts and metrics we see on Quantopian currently (2015-01) are only correct in algos that use 100% of starting capital at some point without going over.

Gary, the issue here seems to be what the purpose of the returns metric is. As I see it, it is not meant to track how well the money which got into the market performs, but how effectively the algo manages funds allocated by capital base.

As in your example, if that algo returns 3.7% rather than 14.4%, but that is only because it is not using it's full potential and investing, say, 1/4 of the available capital, that's a code problem that I want to know about. Further if your idea were implemented, you may see that 14.4% and get no indication that you have some bug keeping your capital on the sidelines for no reason. So, when you say "in terms of code quality" that's misleading, because if your code isn't effectively using the funds which it has under management, that's a code problem.

One solution is to log the portion of capital which gets invested, on average, and/or the peak. However, since my ultimate goal is to make a return on capital allocated irrespective of what actually gets into the market, I am more interested in performance relative to that number.

Sounds like many want numbers locked into imaginary use of 100% of capital, I get that.
We need a way to serve both schools of thought. Who has an idea on how to add other figures? Optionally?

For this notion, forget image-creation in Javascript on the client side. HTML and CSS? Meh, perhaps:

Low quality image, sorry.
Btw obviously one can't have both borrowing and unused cash on the same run, that is for illustrative purposes only.

Thinking out loud. I already understand it is a big undertaking. ...
There could be a dropdown/option for returns style (plus alpha, beta, sharpe etc), I don't really like that.
A setting in one's profile? No.
Possibly at least just a bar along the bottom of the chart to indicate unused cash.
Or could lightly shade the chart background in areas where cash goes negative. Then what about unused cash?
Maybe a vertical side bar.
There's a lot of clear real estate above the chart next to the buttons for added content.
Your turn.

Everyone expect return to be calculated the way it is, changing it would make it confusing.
Gary has a good point that a metric to measure Algorithm Efficiency would be nice but it is
so hard to define a standard one that it is better to provide all raw information and each user
combine the metrics into their own efficiency value.

I don't think True Return or return on allocated money can be used without combining with some
method to count the risk.

I'd encourage a broader view here. One idea I've had in mind awhile is the concept of a virtual advisor/assistant/technical support rep. The basic concept is that a new user could configure their backtest session by answering a series of questions, with the goal of setting up for a realistic simulation for that particular user. It could be a kind of interactive, guided Q & A, tutorial session, that would result in a custom settings file to be imported:

import my_backtest_settings  

Then, custom feedback could be provided relative to the settings. In analogy to simulation software used in science and engineering, unrealistic events (i.e. not realizable vis-a-vis the settings) would generate advisories, warnings, and errors (particularly for time-consuming minutely backtests, it would make sense to stop the algo for errors). Imagine using an electrical circuit simulation program that doesn't, by default, adhere to the laws of physics; the same principle applies to financial simulations.

Quantopian is already going down this path with trading guards (https://www.quantopian.com/help#ide-trading-guards), but eventually it all needs to pulled together into a set of custom settings that ensure that the backtest is a realistic simulation for a given user.

To Gary's point, one of the settings could be some measure of the efficiency of capital application, so that an advisory could be issued (I wouldn't call it a "warning" since anybody who got out of the market into cash prior to 2008-2009 is a genius). His Calvin example above would result in an advisory, "Potential inefficient use of capital..."

Grant

I understand the point of view about wanting to have a metric based on the capital allocated, but a standardization problem arises when you start creating new metrics. Something that is intuitive for one person may not be intuitive for another, so Quantopian's methodology is to strictly adhere to what has been widely accepted by the academic community.

After taking an initial look, I think this kind of metric is only valid for long only strategies. I tossed a short sale of 1 share of SPY into this algo, and your print-run summary shows a 160% return, which is not true (the value printed in the logs, not the recorded variable). In order to justify using something it has to be 100% consistent, and work in 100% of possible scenarios. Decades of academic groundwork has been laid down and is widely accepted, I don't see a way to justify adding something the academic community wouldn't agree with.

Unless something is an accepted truth in academia, we cannot include it in the API because it creates more confusion than it fixes. However, you will always have the ability to create and record whatever metrics you find most useful within the API.

David

Hi,

When I saw Gary's gif (really got me to laugh, in a good way), I've been waiting to for a bit of quiet time to reply. First, I'd like to clarify the behavior that Gary is depicting in the gif. In both cases, there is leverage. I thought it would be useful to explain that point a bit, and define some common terms.

• Leverage is just a way of explaining how much market exposure you have relative to the value of your portfolio. Leverage is always defined for a portfolio as a ratio between the total value of your stock positions (excluding cash) and the net liquidating value of your portfolio (stock + cash).
  
• Net Liquidating Value of a portfolio is the cash you would have if you sold all your positions. Net Liquidating Value is the denominator for the leverage ratio. Say you have a $100 in cash, and a single share of ACME worth $100. The net liquidation value of your portfolio is $200. The leverage of that same portfolio is $100 / $200 = 0.5.
  
• Margin is the name for credit extended to you to invest. Leverage doesn't mean to borrow, but there is a colloquialism in finance to say "you are using leverage". That actually means that you are buying and selling on margin -- borrowing in order to invest. For stocks, you can borrow cash to buy stock, and you can borrow stock to sell it (short selling, which adds a wrinkle to leverage calculations that I'm ignoring here). Margin is the real missing piece for Quantopian's simulation -- we don't limit borrowing at all.

Leverage is always an informative value, regardless of whether you are using margin. One way to think about it is how much will changes in your stock position be magnified in your portfolio. Let's go through three examples: one where you have exactly enough cash to buy 1 share of stock, a second where you have much more cash, and a third where you need to borrow cash to buy the stock.

Case 1: Just the right amount of cash
If you start with $100 in cash, and buy 1 share of ACME for $100, your total stock value will be $100. Your net liquidating value will also be $100. And your leverage would be 1.0. You would have borrowed no cash or stock. If ACME goes to $110, you would have a 10% return. That's a tidy example that satisfies everyone's intuition about returns.

Case 2: Much more cash to start
Now, imagine you start with $1,000,000. You then buy 1 share of ACME for $100. Your total stock value would be $100. Your net liquidating value would be $100 in stock plus $999,900 in cash, or $1,000,000. Your leverage = $100 / $1,000,000 = 0.0001. If ACME goes to $110, you would have a 0.001% return. If you ignore your leverage, this example is frustrating. In this case, the leverage figure is minuscule -- it means you have a tiny amount of exposure.

Case 3: Very little cash to start
Finally, imagine you start with $1. You borrow $99 and buy 1 share of ACME for $100. Your total stock value is $100, like before. But your net liquidating value will have to account for the $99 loan you received -- $100 in stock - $99 loan = $1 net liquidating value. Your leverage is 100x. When ACME goes to $110, you will have 10 times your original money, for returns of 1,000%.

While all three cases have very different returns, they also have very different leverage, and you can make sense of the portfolio returns versus the returns of ACME based on the leverage.

Here are a few other facts that can help build your intuition for leverage:

• Changes in the value of your stock positions will change your leverage. Above I calculated the leverage after the share of ACME is purchased.
  
• Changes in the amount of cash will change your leverage. Borrowing cash, receiving dividends, shorting stocks -- all change your leverage.
  
• A portfolio with leverage > 1 will tend to increase in leverage as the portfolio value falls, and decrease in leverage as the portfolio value rises.
  
• A portfolio with leverage < 1 will tend to decrease in leverage as the portfolio value falls, and increase in leverage as the portfolio value rises.

Lastly, I wanted to comment on Gary's request for the calculation of returns on the capital actually deployed in your positions. I think of the returns figure that Gary is describing as being per position. In a portfolio, you look at the returns for the portfolio as a whole, but you'll also want to see the returns for each individual position. If we had per position returns, all three examples above would show a 10% return on the position in ACME. That's very useful information, and I think it is a complement to the existing returns calculation.

thanks,
fawce

Haha, Awesome Gary! Here is a couple resources from IB about how they handle margin accounts. Specifically Reg-T margin accounts.
IB Margin overview

IB Margin examples

Thanks DE. Incidentally the algo you posted says right there in two places that it wasn't geared for shorting so your critique was not applicable. I like your earlier point about cash as another position, just that during code development it isn't productive to need to be constrained to have to worry about that in my opinion. The quant says: How does this change affect the output? Trying to maintain 100% usage of cash can be like chasing one's tail.

Here, several anti folks used third-party investor hypotheticals. Third-party will be relevant in the hedge fund. And yet, it's like saying, when we arrive at the party we have to be all dressed up in suit and tie at our best, and that's ok, however, getting there is a whole different can of worms, fording rivers and fixing flats on dusty roads. It's our work-clothes environment I'm addressing. Software development. Preparation. Understanding. Clarity. Certainty. Programming. Not the actual investing phase, not yet.

Here's an illustration of the problem:
I had been working with my latest code for the last couple of days and just could not for the life of me achieve much beyond the benchmark without a lot of volatility.
Then I swapped these two lines:

    TakeProfit(context, sec)  
    #order_target(sec, 0)  

And bam! Suddenly the output leapt from 68% to 153%. Awesome looking graph. Great! Right? Right??

Wrong. It only appeared to be an improvement because ending cash went from positive to negative.
Most people don't realize it is even happening.

The real result if I had used exactly 100% of initial capital was worse. It was down from 68% to 55%. Think about that. Don't just brush that aside.
The UI said it was more than twice as good as before. Many people would then continue building on that now inferior foundation.
There could be entries in the contest right now doing surprisingly worse than they expected because of this.
There could be real live money at stake, algos that looked great and then poof.

It is impossible to guarantee using 100% of initial capital always, while making changes, even with the simplest of algorithms.
Frankly I don't see how anyone can argue that the current situation is healthy in the least unless maybe they like that people are being mis-led by the charts and metrics often and therefore being held back. Or maybe people would just rather have a tooth pulled than admit I'm making perfect sense? Or they'll say, you're right but--. Hey, no buts. Heh.

I'm suggesting that it can be worthwhile to consider ways to add this metric, IF I were 100% in ... then what would my result be. That's perfect. That's critical to efficient development. Maybe an option, a tab, something, we can use our heads and creativity, we're smart people (theoretically). And thanks fawce for the info on leverage vs margin, first time Calvin and I understood that. :D

Gary,
I think Fawce nailed it when he said that this could be achieved by tracking returns on a per position basis in addition to the portfolio level where you're only concerned with the sum of their effect. Right now the daily PNL is recorded for each position, but the returns on a per position basis would be very useful and could/should be added sometime as well. It would be great to have the ability to plot PNL curves for each security in your portfolio, something like viewing the growth of $100 in just the stock XYZ leg of the portfolio.

Now you switched over to talking about the full backtest environment rather than the IDE chart and metrics under discussion. Ok I think I'll take a breather (head starting to hurt, tough wall).

Feb 8, 2016 (originally). This marks one year since that last message.

PvR (Profit vs Risk) is a tool I developed to work around this problem. Using PvR within the last week there have been dramatic improvements to two particular algorithms in profit per dollar activated/spent/risked:

1. 58.3% ==> 200% For Robinhood trading
   
2. 108% ==> 344% minimum variance w/ constraint

There was an overview of this issue posted Aug 23, 2015

[quote]

I would just like to highlight what I think are some truly important take-home messages in this thread. I think Gary’s posts here, here, and within this thread reveal serious deficiencies in the Quantopian backtester. And it looks to me like Gary’s assertions are irrefutable. In many cases, what is displayed in Q’s cumulative returns plot is inaccurate - therefore misleading and untrustworthy.
Here’s what I derive from Gary’s work:

1. Negative cash flows are frequently obscure and are not accounted for in the returns value and metrics.
2. The Quantopian returns value in the IDE uses initial cash (instead of drawdown) and ignores any negative cash.
3. Backtests are using money they don’t have.
4. The backtest returns and metrics will only be accurate to the degree the algorithm spends 100% of initial capital at some point and no more.
5. Negative cash can increase in magnitude while leverage remains around 1. (The leverage value can’t be trusted either.)
6. For many algorithms, true performance will be worse than what appears in the GUI; for others, the user’s code would actually outperform what the chart indicates. That the UI can reflect returns much better than live trading would produce should concern every developer.
7. The ability to follow the algorithm’s use of capital and monitor unused cash is critical.
8. The “MaxSpentReturns” value in the custom chart is the one we should rely on while developing code, not the value in the backtester.
9. All algorithms should take margin into account.

For simulations, trading profits and the trading decision model tend to capture most of our attention. But experimental results - in terms of real portfolio end values - must accurately account for margin, drawdown, and leverage. Quantopian's backtester apparently doesn't.

This is not a small issue.

[/quote]

Based on the positive results from this metric, it is obvious that it would be highly beneficial to Quantopian and its software developer members.
Rather than maybe, some day, this ought to be priority 1, severity 1.
Could possibly be a third trace on the main chart.

John Fawcett made this very clear.

John Fawcett made this very clear.

Except that leverage is not part of the returns calculation and to figure how well one's stocks are performing, injecting leverage into the equation can't get there. It can only be done by keeping track of amount risked, put into play.