Research notebook attached.

And the example algo using the "TrendyEPS" factor

Great, expecially the work you do to "clean" the data from NaNs,zeros etc. I wonder if Q could maybe in the future insert an option when we import data to have it already more "cleaned" somehow

Good post. I re ran it, and see specific returns are very low or negative. Is the risk model considering trendyeps as value, hence subtracting it?

I added constraints in order_optimal_portfolio, but got into a Timeout error. Any idea how to fix it?

Thanks Kyle for sharing

Good Stuff! I hope these 'Learn from the Experts' threads will earn their own folder
soon.

I'm also curious why this backtests so slowly. I tried added a DollarNeutral constraint and it times out.

I don't really know why adding constraints lead to slow backtests. Perhaps someone at Q might know? Otherwise, I don't see any explicitly slow code anywhere in the algo...

That issue should be fixed now, please let me know if that's not the case.

@Kyle

Do you have as well a zipline version?
Just found a zipline command in the tear sheet.
Would be nice to use it on zipline

Thankx
Carsten

Wow, this is extremely educational. Thank you so much for sharing the video, notebook, and algorithm. I have been having a hard time connecting the report from Alphalens and the result of the full backtest and this interview answers that question.

I generalized the EPSGrowth class a little bit:

class FactorGrowth(CustomFactor):  
    inputs = [None, None, None]  
    window_length = 1  
    window_safe = True

    def compute(self, today, assets, out, prev_q, curr_q, sectors):  
        # Calculate surprise  
        prev_q = prev_q[-1, :]  
        curr_q = curr_q[-1, :]  
        surprise = (curr_q - prev_q) / np.abs(prev_q)

        # Replace inf w/ NaN  
        surprise[np.isinf(surprise)] = np.nan

        # Standardize the data  
        surprise = standardize(surprise, sectors=sectors.as_string_array()[-1, :])

        # Normalize the data (NOTE: only include if looking at factor individually)  
        if PIPE_NORMALIZE:  
            surprise = normalize(surprise)

        out[:] = surprise  

So the make_factors() function can be written as:

def make_factors():  
    factors = {}  
    # Quarterly  
    qn1_q_eps = Actuals.slice('EPS', 'qf', -1).actual_value  
    q0_q_eps = Actuals.slice('EPS', 'qf', 0).actual_value  
    # Annual  
    qn1_a_eps = Actuals.slice('EPS', 'af', -1).actual_value  
    q0_a_eps = Actuals.slice('EPS', 'af', 0).actual_value  
    focus = RBICSFocus().l2_name

    factors['EPSGrowth'] = FactorGrowth(inputs=[qn1_q_eps, q0_q_eps, focus], mask=universe)  
    factors['EPSGrowthYr'] = FactorGrowth(inputs=[qn1_a_eps, q0_a_eps, focus], mask=universe)  
    factors['TrendyEPS'] = TrendyEPS(mask=universe)

    return factors  

The pro is I can experiment with additional factors a bit faster as I do not need to write a CustomFactor for each of them, but the con is I can't just change the operations to a specific factor I experimented with, also this is probably an unneeded abstraction as it makes the code a little harder to read.

If I wanted to do some changes to a specific factor I'd have to make a copy of the FactorGrowth class and rename it for that specific factor.

May the Alpha be with you.

Thank You very much Kyle and Thomas for this very educational and very helpful video. I have a question on this. If someone could please help on this. In the following code, how would one go about combining two factors?

def make_factors():  
    factors = {}

    factors['EPSGrowth'] = EPSGrowth  
    factors['EPSGrowthYr'] = EPSGrowthYr  
    factors['CombinedFactor'] = EPSGrowth*EPSGrowthYr

    return factors

# Define the universe  
universe = QTradableStocksUS()

def factor_pipeline(universe):  
    all_factors = make_factors()  
    factors = {a: all_factors[a]() for a in all_factors}  
    pipe = Pipeline(columns=factors, screen=universe)  
    return pipe  

I am trying above code for combining two factors, but it is giving me following error WHILE running pipeline -

unsupported operand type(s) for *: 'ABCMeta' and 'ABCMeta'

Could someone please post a code which will work?

Nadeem, I typically do the factor combination outside of the pipeline. In other words, I'd run the pipeline as normal, and in the results just multiply the two pandas columns. It's also possible that the factors have a .latest or some other attribute that change them from ABCMeta classes to BoundColumns, although I'm not sure.

Hi,

Thanks Kyle for the useful research notebook. For one of the factor, I am trying to have multiple outputs, but i encountered a RecarrayField error. The idea is to see the various intermediate variables for the computation of each factor in the class, which i thought will be useful when we want to reconstruct our own Fundamental variable (e.g. PE ratio). Under make_factors(), im stuck with the error. How do i go about doing this?

Sorry for the newbie qn, just started learning coding a month ago.

Thanks.

Thanks Kyle!! Your video is very useful!!
Do you have as well a zipline version?

@Thomas Wiecki

is there a way to get the factor loadings and returns for zipline.
How are they designed? I'm running zipline with fundamentals, I could construct them by myself if I knew the formulas you are using.
Should be similar to Farmans and French?

Load risk factor loadings and returns

factor_loadings = get_factor_loadings(assets, start_date, new_end_date)
factor_returns = get_factor_returns(start_date, new_end_date)

@Carsten: The model is only available on Quantopian. You can find the details here: https://www.quantopian.com/posts/risk-model-white-paper-released-and-available-for-your-reading

How long did it take to make all the resarch? from zero to the end?

@Alejandro, I start with a hypothesis. Ex: I think that companies who beat earnings repeatedly are good buys. Then, I code a bunch of different factors that attempt to capture that hypothesis. Next, I plug them into my notebook and run it. I iterate over this for a few hours until I've found something statistically significant. Only then do I run a backtest to confirm that the factor translates well to a trading environment.

All in all, I've spent many hours in research. It really is up to you.

Hi @Kyle
This is an excellent demo who helped me allot to learn how to analyze factors.
It might be a rookie question but I am having trouble understand how to research my hypothesis...

I try to take current ratio as a factor.
I want to have a current ratio of 1-5 as long, and <1 as short

1. I am not sure if the math I did actually correct (full notebook is provided):
   match = match - 1 # current ratio > 1 is welcomed

class Current(CustomFactor):  
    factor = mf.current_ratio  
    inputs = [factor, RBICSFocus().l2_name]  
    window_length = 1  
    window_safe = True

    def compute(self, today, assets, out, factor, sectors):  
        # Calculate surprise  
        factor = factor[-1, :]  
        match = factor  
        match[factor > 5] = np.nan  
        match = match - 1 # current ratio > 1 is welcomed

        # Replace inf w/ NaN  
        match[np.isinf(match)] = np.nan

        # Standardize the data  
        match = standardize(match, sectors=sectors.as_string_array()[-1, :])

        # Normalize the data (NOTE: only include if looking at factor individually)  
        if PIPE_NORMALIZE:  
            match = normalize(match)

        out[:] = match

1. In the tear sheet the specific returns is positive but the total is negative.
   What does this behavior means?

I would really appreciate the help here to continue
Thanks!
Idan

Hi @Kyle, many thanks for this very good educational video and sources shared. Question on factors combination - several posts earlier you mentioned that in order to analyze the combined factor you do the multiplication of two columns in pandas with separate factors. Why do you multiply but not make the linear combination of these?

Hi Vyacheslav,

I typically do linear combination as you're describing. However, there may be cases where you actually want to scale one factor's impact based on another's. For example, maybe your strategy succeeds when there is both positive sentiment as well as good momentum. In that case, multiplicative combination would allow you to take varying positions weights in many companies as opposed to a binary cutoff with fewer companies and thus slightly larger weights.

Your research can confirm whether or not this multiplicative aggregation is useful or not. Again, to clarify, I typically just do linear combination.

@Kyle, thanks!

Hi Kyle,

In the video at 4:40 you were talking about a notebook with a bunch of basic fundamental factors, but I cannot find that notebook. Can you send me the link of the notebook that you were talking about? Thank you in advance!

Hi Elwin,

Here is what I was referencing. The factors are a bit dated, but the idea is there.

https://www.quantopian.com/posts/alpha-combination-via-clustering