Today, we are releasing a small change to the Optimize API that improves the reliability of exiting positions when using the TargetWeights objective. When you use TargetWeights to exit a position (either by not passing an asset you currently hold or by passing the asset and giving it a target weight of 0), it is now much more likely that your resulting portfolio will not include the undesired position.
What problem is this fixing?
Previously, when you used TargetWeights to place orders with the Optimize API, it was possible for your portfolio to include small positions in assets that you did not intend to hold. Specifically, if you held a position in an asset at the start of an optimization and you attempted to exit that position — either by not passing the asset to TargetWeights or by passing the asset with a target weight of 0 — it was possible for your resulting portfolio to still include a small holding in the undesired asset. To understand why this happened, we need to review some basic Optimize API concepts.
When you construct a portfolio using the Optimize API, you provide two pieces of information: an objective, and a list of constraints. Your objective tells the Optimize API what you want from your new portfolio and your constraints give a set of conditions that your new portfolio must satisfy. When you ask the Optimize API to find a portfolio, it finds the portfolio that minimizes (or maximizes) the function defined by your objective, but the optimizer ensures that the resulting portfolio satisfies all the conditions defined by your constraints.
In the case of the TargetWeights objective, the Optimize API tries to find a portfolio that’s as close as possible to your target, ensuring that the result satisfies your constraints. This is easy when your target doesn’t conflict with any constraints: the optimal portfolio is simply your target. Things get more interesting if your target does violate one or more constraints. When that happens, the optimizer has to figure out how to “adjust” your target portfolio as little as possible while ensuring that all your constraints are satisfied.
When the optimizer searches for target positions to adjust, it considers assets referenced by your objective as well as assets in which your algorithm currently holds a position. This means that if you try to exit a position using TargetWeights, the optimizer may determine that the portfolio that minimizes the TargetWeights objective function is a portfolio that still contains a position you were trying to exit. In particular, because of the distance metric used by TargetWeights, a common outcome for certain constraint violations was that the optimizer would choose a portfolio with many small (e.g., 1-5 shares) holdings for positions that were supposed to be exited.
What is the new behavior?
In this release, we’ve tweaked the optimization function used by TargetWeights to penalize holding positions that your algorithm is attempting to exit. This affects positions for which you passed a target weight of 0, as well as currently-held positions for which you didn’t provide a target at all. The result of this change is that TargetWeights will prioritize zeroing positions that your algorithm is trying to exit when it considers which target positions to adjust.
This change should help amend issues like the ones seen in the following forum posts:
- Stat Arb - 11 year backtest
- How am I NOT trading the tradeable universe when using QTradableStocksUS() ?
In both of these cases, the optimizer previously had difficulty maintaining a portfolio within the QTU. With this change in effect, non-QTU positions will be more effectively zeroed-out because the optimizer now has a strong preference against holding these positions where a weight is undesired. This will allow more of these strategies’ holdings to be in the QTU.
In short, this means that the optimizer behaves more intuitively when using TargetWeights — positions for which no weight is desired are much more likely to be dropped from the portfolio.
Note: This change improves the behavior of TargetWeights when adjusting portfolios after constraint violations, but TargetWeights still generally works best when your target portfolio satisfies or nearly satisfies your constraints. It’s usually not a good idea to use TargetWeights with a target portfolio that you expect the optimizer to significantly transform. For example, if you apply a DollarNeutral constraint in your optimization, you should ensure that your target portfolio contains an equal mix of long and short positions.
