Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
Update to TargetWeights/Optimize bug?

Hi Q Team,

In a number of previous posts, users have reported a problem where optimize doesn't close out non-QTU positions when used in conjunction with TargetWeights. This makes entering the contest without the use of MaximizeAlpha fairly difficult. Ex thread here: https://www.quantopian.com/posts/the-contest-is-really-hard-dot-dot-dot

Jamie McCorriston said it was non trivial to fix but it was on the engineering team's radar (albeit low priority).

Is there any update or timeline for a permanent fix? From the perspective of a new user, this bug creates a lot of friction to join the platform. Also, there seems to be some ongoing confusion as to what MaximizeAlpha does under the hood, so that doesn't seem like the best alternative for individuals that want to control all the levers in their algorithm. Additionally, other users have pointed out that previous workarounds are not closed-end solutions. James Villa, author of the primary workaround, still receives errors in long backtests (e.g. "InfeasibleConstraints")

-Spencer

2 responses

Side note, Thomas Wiecki advises here: https://www.quantopian.com/posts/how-to-get-an-allocation-in-2019 that "code-wise, you should not use MaximizeAlpha and instead [use] TargetWeights"

I will take his word for that. He also says, "The advice contained within here is still intact for 2020"

So to reiterate my confusion, what is the permanent fix for the TargetWeights bug if Quantopian now views MaximizeAlpha as an inferior function?

There currently aren't any known 'bugs' in the TargetWeights objective. A fix was made in May 2019 which addressed the issue Jamie McCorriston alluded to in the post above. See this post for the update. The TargetWeights objective now closes out all zero weight, or non-weighted, positions.

One can use either TargetWeights or MaximizeAlpha as objectives. Both work fine. The choice depends upon what your algorithm is trying to accomplish. The TargetWeights objective tries to find an array of portfolio weights that minimizes the distance from the target weights (as measured by Euclidean Distance). The MaximizeAlpha objective tries to find an array of portfolio weights that maximizes the sum of each asset's weight times the target alpha value. Under the hood, the optimize methods rely upon cvxpy. The Quantopian methods are simply wrappers to make it a bit easier to use. There is a bit of background in this post and of course the documentation.

With all that said, optimization can sometimes lead to some non-intuitive results. Consider the following case

# Create target weights of 50% for two securities  
weights = pd.Series(data=.5, index=symbols(['AAPL', 'IBM']))  
target_portfolio = opt.TargetPortfolioWeights(weights)

# Constrain portfolio to be 'Dollar Neutral' (ie equal long and short)  
dollar_neutral = opt.DollarNeutral()  
optimized_portfolio = opt.order_optimal_portfolio(  
        objective=target_portfolio,  
        constraints=[dollar_neutral])

The above code will end up not ordering anything. We wanted a portfolio with AAPL and IBM but ended up with no orders. How can that be? Very simple. Optimize essentially starts with the target weights of .5 and moves them a little bit until the DollarNeutral constraint is met. The first solution it finds is a weight of 0 for each security. This isn't what we expected but it's technically true. The sum of the longs and shorts is 0. Constraints often lead to unexpected results.

The complete advice that Thomas Wiecki gave in the post referenced above is

Code-wise, you should not use MaximizeAlpha and instead TargetWeights.
This is a good place to start:
order_optimal_portfolio(opt.TargetWeights(weights), constraints=[])

The emphasis is meant to be on not using constraints as a starting point (note constraints=[]). It wasn't meant to say don't use MaximizeAlpha. The advice is to start by programmatically defining target weights in one's algo and then using the order_optimal_portfolio to simply execute the orders. By using the TargetWeights objective without any constraints, the weighting logic is moved out of the optimize method and into one's logic. It's generally better to explicitly define one's weighting rules than relying on optimize to implicitly, and sometimes unexpectedly, implement them.

Hope that clears up any confusion. The requirement to use order_optimal_portfolio shouldn't be a reason not to join the contest or the platform.

Good luck!

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.