If you're not above the minimum leverage constraint, forcing it up would necessarily violate the position concentration constraint or some other constraint for contest entry, barring a change to the algorithm itself.

I was afraid of that. Makes sense that the model has at least one unconstrained variable. It would be nice though to at least be able to put it in the objective function as an optimizable variable; a soft constraint.

@Jonathon Walker. What objective (ie TargetWeights or MaximizeAlpha) and constraints are you using? Does the series you pass to the objective have both positive and negative values? Also, do you know ahead of time which securities to long and short or do you leave it up to the optimizer to figure that out? There are different approaches for different scenarios.

Thanks for following up Dan,

So after my pipeline uses an alpha metric to return a filtered pipeline of ranked longs and shorts, I use the basic set up from the "Getting Started" tutorial (https://www.quantopian.com/tutorials/getting-started#lesson7) and let the great Quantopian tools do what they were designed to do, as follows:

def rebalance(context, data):  
    # Retrieve alpha from pipeline output  
    alpha = context.pipeline_data.alpha_signal

    if not alpha.empty:  
        # Create MaximizeAlpha objective  
        objective = opt.MaximizeAlpha(alpha)

        # Create position size constraint  
        constrain_pos_size = opt.PositionConcentration.with_equal_bounds(  
            -context.max_pos_size,  
            context.max_pos_size  
        )

        # Constrain target portfolio's leverage  
        leverage_constraint = opt.MaxGrossExposure(context.max_leverage)  
        #leverage_constraint = opt.NetExposure(context.min_leverage, context.max_leverage)

        # Ensure long and short books  
        # are roughly the same size  
        dollar_neutral = opt.DollarNeutral()

        # Constrain portfolio turnover  
        max_turnover = opt.MaxTurnover(context.max_turnover)

        # Constrain target portfolio's risk exposure  
        # By default, max sector exposure is set at  
        # 0.2, and max style exposure is set at 0.4  
        factor_risk_constraints = opt.experimental.RiskModelExposure(  
            context.risk_factor_betas,  
            version=opt.Newest  
        )

        # Rebalance portfolio using objective  
        # and list of constraints  
        algo.order_optimal_portfolio(  
            objective=objective,  
            constraints=[  
                constrain_pos_size,  
                leverage_constraint,  
                dollar_neutral,  
                max_turnover,  
                factor_risk_constraints,  
            ]  
        )

My initialize function sets up the following constraint limits and rebalance schedules:

def initialize(context):  
    # Constraint parameters  
    context.max_leverage = 1.1  
    context.max_pos_size = 0.008  
    context.max_turnover = 0.95

    # Attach data pipelines  
    algo.attach_pipeline(  
        make_pipeline(),  
        'data_pipe'  
    )  
    algo.attach_pipeline(  
        risk_loading_pipeline(),  
        'risk_pipe'  
    )

    # Schedule rebalance function  
    algo.schedule_function(  
        rebalance,  
        algo.date_rules.week_start(),  
        algo.time_rules.market_open(),  
    )  

I'm thinking that part of the problem is that I just don't have enough shorts to use up all the available margin and since it is dollar neutral, that means both longs and shorts; there just not enough qualified trades to act on. So, one thing I am trying tonight is to cast a bigger net. I'm running a mean reversion technique that relies on the t-statistic of the alpha signal crossing a t-statistic critical value at a particular confidence interval. I realized that it's just taking a snapshot once a week of all the stocks that crossed that line in the sand on that day. That seems like it would miss a lot of good trades, so I set up a few more "tripwires" for it; four linearly spanned critical value crossover filters above the original critical value and 4 below it -- 9 critical value crossover filters in total with OR logic (e.g. np.any()), any signal that meets one of those gets traded. Hopefully that gives me enough trades to meet the minimum leverage.

I also noted that there were a lot of instances where it could not fill all the orders, so I started to reduce the max position size a bit until it started to interfere with the return. Shaving off any more on this won't be helpful since it appears that it's already hit the sweet spot and and restricting it more diministhes returns due to an already limited population of trades.

One idea I'd like to pursue, but haven't figured out how to implement yet, is to multiply the alpha signal by the crossectional market cap weight (e.g. market_cap_i,day/sum(market_cap_i,day)) prior to ranking the pipeline alpha signal. I feel like the algorithm is finding a high scoring small companies with lower stock prices that require the algorithm to fill more shares than available, resulting in inefficient cancelled orders. I'm thinking that if I could give more weight to the larger companies with larger stock prices and and fewer shares to fill per order for the same dollar amount would hopefully be easier to fill and at higher prices that use up that available margin faster.

I can get the daily market cap for each stock, but how do you calculate the crossectional sum of all market caps of all the pipeline stocks for each day from within the pipeline in order to calculate a market cap weight and multiply it to the alpha factor before the alpha factor is ranked?

Time series custom factors are easy but I haven't seen any info on how to create a crossectional custom factors for the other axis. I haven't had much luck with "axis = 1" or anything on the forum. I'm surprised there isn't a built in factor for this already. I'll likely post this question in a separate post as well so others can benefit from the answer.

Any other ideas ? All ideas and hints are welcome and appreciated.

I guess the market cap summation can only be done post processing?

import quantopian.pipeline.factors as Factors

def make_pipeline():  
    pipe = Pipeline()  
    # Get Asset MarketCaps  
    market_cap = Factors.MarketCap(mask=universe)  
    return Pipeline(  
        columns={  
            'market_cap':market_cap,  
        },  
        screen= longs_filter | shorts_filter  
    )  
    return pipe

pipeline_output['market_weight'] = np.array(market_cap)/sum(np.array(market_cap))  

Is there any way to do this within the pipeline so that I can do statistical transformations of the alpha * market_cap_weight product?

I'm trying to do something like this from within the pipeline:

class Market_Cap_Weight(CustomFactor):  
    # Default inputs  
    window_length = 1  
    window_safe = True  
    def compute(self, today, asset_ids, out, values):  
        values = np.nan_to_num(values)  
        weights = values[-1]/np.nansum(values, axis=1)  
        out[:] = weights  

But I keep getting the error:

ValueError: MarketCap does not have multiple outputs.

Is there a way to structure this custom factor to calculate the crossectional sum of the market caps for each stock, ignoring nans?