Great Work! I was also working on the same topic but then stopped because of some Quantopian Limitation for Fundamentals data.
Where did you find that get_fundamentals supports a date also in the backtest environment and not only the research notebook? Is it an undocumented feature?

Here some links to my work on QVAL:
Campbell, Hilscher, Szilagyi (CHS) Model - Probability of corporate failure
https://www.quantopian.com/posts/campbell-hilscher-szilagyi-chs-model-probability-of-corporate-failure

Piotroski score:
https://www.quantopian.com/posts/error-in-fundamental-data#56fbfe939426be39440000a4

Beneish score:
https://www.quantopian.com/posts/beneish-score-accounting-manipulation-as-pipeline-factor-memoryerror#56f2bc685fe2be6837000355

Hey Costantino,

You've just found a major bug in my implementation, get_fundamentals indeed does not take a date argument in the backtest. So I've been playing with an extremely partial implementation of the quantitative value algorithm, managing to squeeze out good performance from the few factors that are correctly implemented... This is beyond embarrassing, and quite frankly also really impressive that the broken algorithm produced this performance.

So the options I have now are to either save up state in the context object, which will not translate to the live trading environment, or to use pipeline, which at least used to perform horribly with fundamental data, to the point where the algorithm had to be broken up into phases that ran across multiple days. Ugh. I honestly don't understand some of the design choices that the quantopian platform has made.

Well, back to the drawing board.

Sunil

Hi Sunil,

I wanted also to check, because I suspected that... anyway your work is a very good basis for the QVAL algorithm... it only needs to be updated, when Quantopian will make easier to work with historical fundamental data.

I'm complaining about the lack of this feature for a long time... if you query for my username, you will find many post about this subject.
There is a workaround, storing the data in a Panda Panel, but then is not possible to trade live (you have to wait a year o more before the panel is filled with data).

Q2 remains a great platform and maybe if we unite our forces, we can convince the Q-Team to add this feature as soon as possible ;-)... a get_fundamentalswith date like in the Research environment would be great!

Well, I'm slowly converting the algorithm over to use pipelines. So far I've only converted STA and SNOA computations, I suspect I'll soon run into timeout issues. Especially when I start implementing the franchise power and margin factors. The pipeline API as set up doesn't tell us where it is spending compute cycles, so any sort of optimization is virtually impossible.

I considered storing data in the context object, but as you said, that just doesn't work for live trading.

Another alternative to adding dates to get_fundamentals would be to allow specification of specific times for which we want to retrieve data for custom factors. Right now you have a very coarse window_size argument, would be good to be able to generalize that, even with something as simple as taking a vector of indices rather than a single window size.

The pipeline API is a really nice feature, but it is currently very inefficient especially when you want pipeline outputs infrequently (rather than daily), and does not come with a particularly rich set of operations that can be run on pipeline factors.

Sunil

You're right! Pipeline API is nice but very memory consuming, when it comes to the many datapoints required for historical fundamental data.

I posted a feature request exactly as you're now proposing (vector of indices rather than a single window size):
https://www.quantopian.com/posts/pipeline-api-feature-request-retrive-single-data-points-instead-of-a-full-array-of-data-window-length
but until now it remained a dead letter.

A kindly request to the Quantopian Team: get_fundamentals already supports timestamp and timeframe in the Research environment. It would be great, to enable the same also in the Backtesting!

I am for now giving up on fixing my implementation. I have posted a last cry for help here...

https://www.quantopian.com/posts/fundamentals-fundamentally-broken

I've also been working on a QV screen algo, and running into the same issues as Sunil and Constantino when trying to run backtests.

However, I have put together a notebook (using pipeline) showing my working and would love thoughts / inputs from others. I already know there are a few things that need working on, for example:

• some of Constantino's examples above seem to have more accurate ways of determining window sizes and indexing for fundamentals - I need to look at that
  
• I still have to implement the 'probability of manipulation' part of the forensic screen
  
• my franchise power factors only go back a maximum of 2 years, as opposed to 8 - notebook might be able to handle more than 2 years, I need to look at that

If you spot any other bugs or have any other suggestions for improvement, please let me know. Otherwise I'm just going to be waiting for Constantino's feature request above to be implemented :)

After working with the QV algorithm for a few months, here's what I've learned:

1. The algorithm has been developed and optimized for long term investing with annual rebalancing. The outcome is quite tax efficient for an individual investor. It isn't necessarily the kind of thing that fits into the Quantopian hedge fund. However, you can improve performance a bit if you add in momentum and technical factors. I've used regression lines with some success.
   
2. Large parts of the algorithm can't be implemented using Quantopian's current pipeline implementation. Getting over an year of data (that's two data points per factor) is infeasible. You end up consuming too much memory, it kills the notebook.
   
3. There are large holes in the morningstar data. Many factors are defined for surprisingly few companies. EBIT for instance is defined for about 1000 fewer companies than EBITDA. Look at the fundamental data you're using carefully. I've developed notebooks to get a sense for the data cardinality. I'd post a link for the notebook, but the forum refuses to attach them to a message. I'll see if I can share them via bitbucket.
   
4. On a related note, most fundamental factors seem to be updated on a quarterly basis. The QV text I believe prefers annual numbers. Quantopian's infrastructure can't accommodate annual aggregation particularly well due to memory issues noted above.
   
5. There are outright errors lurking in the fundamental data. These can be really hard to spot.

I might try running some experiments again in the future if Quantopian fixes issues surrounding fundamental data access.

Sunil

thanks Sunil

Thank Sunil, yeah I found many NaN values in the fundamental data too. I think this algorithm may be better used as a screener rather than a trading algorithm, at least with the current data availability.

Some updates:
- Use today's date.
- Remove stocks with NaN total_assets (i.e. fundamental data of some companies like CVE, UBS, etc. do not get listed in Quantopian dataset as they filed as foreign issuers).
- Get all companies instead of only ones with high AverageDollarVolume.
- Use 8-year factors for growth. The code looks incorrect but I cannot confirm at the moment. Will look more into this later.

Jay Teguh Wijaya - amazing notebook!!! thanks... you saved me weeks of work

Hello Friends, I am currently trying to implement the allocation system from Wes Gray's and Tobias Carlisle's "Quantitativ Value". Thank you for the amazing notebooks you provided! You also saved me weeks of work. We are truly standing on the shoulders of giants :-)

I noticed that something appears to be off with the STA score. This is what I found:
If I understood the forensic screen correctly, STA is supposed to take YoY changes of net income and operating cash-flow as inputs,
but "morningstar.cash_flow_statement.operating_cash_flow" seems to give us operating cash-flow from the most recent quarter.
Does anybody know how we can maybe sum up the last 4 quarters?

For net income we are using "morningstar.cash_flow_statement.net_income". For many companies I get very strange values.
I compared these numbers with a few 10-Qs and 10-Ks and could no find where they are coming from.
I'm not sure. Maybe I just accidentally picked companies with faulty datasets.
But if I use "Fundamentals.net_income_income_statement.latest" instead, I do get at least a number that matches the last quarter net income from the 10-Qs.

In addition to that, if I want to check "Fundamentals.net_income_income_statement_asof_date.latest" I do get the correct date of the most recent quarter like "2018-09-30". Unfortunately, if I do the same with "morningstar.cash_flow_statement.net_income_as_of.latest" I only get ghibberish like "1969-12-31 23:59:54.033000000".

Hope somebody has an idea how this can be improved.

Hi @Sebastian,

That Wes Gray & Tobias Carlisle book is absolutely one of my favorites (out of a large trading library of hundreds of books) and has also been a great inspiration to me. As you say, shoulders of giants; indeed we are fortunate.

I have also had some strange results from time to time with regard to cash flow vs income statement data in the Morningstar DB, although i have not done any sort of thorough investigation like checking back with 10-K's & -Q's as you have. I am about to start looking at some of these specific items more closely soon, and i will be happy to communicate & share with you regarding whatever i/you find. Have you made any comparisons between Morningstar and the new FactSet DB items?
Cheers, best regards, TonyM.

Hello Tony,

I'd be happy to work on this thing with a fellow enthusiast.

After a little experimenting I think this should work for building a 12-month trailing net income.:

class NetIncome12M(CustomFactor):  
    inputs = [ Fundamentals.net_income_income_statement ]  
    window_length = 252  
    def compute(self, today, assets, out, nico):  
        out[:] = sum([nico[-1], nico[-94], nico[-157], nico[-220]])  

I looked at the values for the four lookback periods and it does seem to match
the filing nicely. With ADRs it's more complicated. Some seem to work. Some have the
exact same value in each of the 4 periods.

Cheers,
Sebastian

Hi Sebastian,
What sort of look-back periods are you currently working with or interested in working with?

At the moment i'm putting most of my efforts into finding what factors seem to work best even BEFORE doing a careful job of cleaning up the data nicely.
Some of "the usual suspect" factors as per the book do indeed work fine, others not so well (although of course that might be because of bad data and may improve later after cleanup, as you are focusing on), and also i'm finding some very interesting new ideas that are not in the book. Cheers, Tony.

In addition to Morningstar, you might like to take a look at FactSet | Income Statement | Net Income items, specifically the following annual items:

net_inc_af = Net Income
net_inc_aft_xord_af = Net Income After Extraordinary Items
net_inc_basic_af = Net Income Available to Common, Basic
net_inc_basic_aft_xord_af = Net Income Available to Common, Basic, After Extraordinary Items
net_inc_basic_beft_xord_af = Net Income Before Extraordinary Items
net_inc_dil_af = Net Income Available to Common, Fully Diluted
net_inc_dil_aft_xord_af = Net Income Available to Common, Fully Diluted, After Extraordinaries

So far i'm finding this to be a good DB. I think you may find that some of the "cleanup" work required with Morningstar is already done, as well as giving a wider data coverage without unnecessary additional calculation. Potentially may save quite a bit of work! Cheers, Tony

Hello Tony,
This might be an excellent idea. I will put the FactSet values side by side with the morningstar/Fundamentals items and see how they compare.

Cheers,
Sebastian

About your first post:
I must admit that I am not good at backtesting yet. Probably have to watch a few more lectures. Still I feel confident to apply the metrics that Wes and Tobias (and Jack Vogel) used in their books. After all they tested their hypotheses over a much longer timeframe than we have available here.

I am positive that there are other factors that carry alpha and can be integrated into a quantitive value system. I am thinking about signals from insider dealings or short interest but the most promising approach is probably to find a way to combine value and momentum.

Alpha architect wrote that combining a simple value screen with a momentum metric can improve returns significantly. The only reason why they did not add it to quantitative value was that their value system in combination with their multistep "quality on steroids" factors still provided a little bit bettern returns than value plus momentum.

You might find this blog post interesting:
https://alphaarchitect.com/2015/03/26/the-best-way-to-combine-value-and-momentum-investing-strategies/

By the way, I would love to be able to also apply the quantitative screener on stocks from outside the US. There is no logical
reason why it should not work. Therefore I am trying to keep at least as many ADRs inside the screen as possible (in contrast to Wes and Tobias).
And I can probably consider stocks well below the 40th market cap percentile on NYSE without the transaction costs killing my returns.
At least that's one of the advantages of not belonging to the top 0.1% yet ;-)

Cheers,
Sebastian

Hi @Sebastian,

Thanks for the alpha-architect link. I did look at it a few years ago but had forgotten about it.

"I would love to be able to also apply the quantitative screener on stocks from outside the US. There is no logical reason why it should not work".
I have been trading stocks outside the US for many years and came up with ideas similar to those of Wes & Tobias long before i read the book, but it certainly helped me to consolidate some of those ideas. I have been using them very successfully in my own US and also Non-US stock selection & trading outside of Quantopian, and i can assure you that the ideas definitely DO work in other markets. In fact in some cases they work even better in other markets outside of the US, in other countries where these ideas are less well-known & less widely applied.

ADRs are a slightly different beast, with a mixture of characteristics of both the original foreign stocks and of US stocks, but again the ideas do generally work there too.

As to Market Cap, although i certainly use it in some ratio calculations, it actually not something i consider very much in trading. Generally of more importance from a practical trading perspective is liquidity, which can reasonably be defined as price multiplied by trading volume. As a trader, what one does NOT want is to get caught holding illiquid (i.e. un-sellable) stocks when the market starts to turn down, because if they don't have decent trading volume then maybe you just can't get out and end up with big losses. (Yeah, i did learn that lesson the hard way years ago). Often Liquidity & MarketCap are closely related, but not necessarily. Good trading stocks do generally need Volume, and this is closely related to transaction costs. Especially if you are considering options trading outside of the context of Q.

With me it's the same. Back at university I read about return anomalies, the value-premium, the small-cap-premium and things like that and I know it should be possible the utilize that. I could punch myself that I did not dig more into quantitative investing back then. We had Thomson Financial Datastream terminals at our disposal.

Later I read a lot about behavioral finance and the more I read, the more convinced I became that I should have no discretion whatsoever about when to buy and when to sell a stock. As humans we are hardwired to buy and sell at the worst possible time. This is the reason why I want to develop a system with a statistical edge that makes the investment decision on my behalf. I did some swing trading when I was younger but now I am more interested in holding a portfolio of equities over a period of 6 or 12 months. Maybe that's still called trading ;-)

Like Wes and Tobias I also stumbled upon Joel Greenblatt's 'Magic Formula' and fell in love with the idea of ranking a universe and staying invested the whole time instead of trying to time the market based on some absolute metric of cheapness.

I think what you say about liquidity is very true. There are some high market cap stocks with a minuscule free float on the other hand there are quite a number of 80 Mio. USD market cap stocks that are rather liquid. If I look at the average daily trading volume over the last 30 days and my order would be 1% of this daily volume or less, I do not worry about moving the price too much, regardless of the actual market cap.

By the way, I have a challenge in cleaning up the data. maybe you have an idea. When I look at

Fundamentals.net_income_income_statement_asof_date.latest  

...I get some data from outdated filings. 2007, 2010, 2014 and so on. So far I haven't found the correct synthax to exclude equities with a filing date before YYYY.MM.DD.
Maybe you have an idea.

Cheers,
Sebastian

Behavioral Finance ideas make a lot of sense than the EMH crap that i got dished out!! ;-))
" ... the more convinced I became that I should have no discretion whatsoever about when to buy and when to sell a stock".

My own take on this is that "discretion" is generally something best used when OUT of the market and DESIGNING a system, rather than when IN the market trading it.

"... interested in holding a portfolio of equities over a period of 6 or 12 months...." I have done a lot of trading systems design (before and still currently outside of Q) and with regard to equities, for the most part i don't even try to design the timeframe, i just let the logical rules (partly fundamental, partly technical) tell me when to hold & when not to for each stock, and the holding periods can turn out to be anything from a few days days right up to more-or-less indefinite in some cases.

Staying invested the whole time, but changing the investments as the rules advise, that makes good sense to me too, although with the caveat that although we haven't seen a bear market for > 10 years doesn't mean one can't happen. What do you do then? Get out or try holding inverse equity ETFs?

Yes, Greenblatt's ideas are good too, but the more you look at fundamentals, the more you can find other ideas that work but no-one has really written about. That's mostly what i am playing with right now here on Q.

Regarding illiquid stocks (e.g. speculative mining, biotech, etc) they can sometimes be fantastic "investments", but just don't want to be holding them if/when the market as a whole goes down. Yes, i agree with the "my order size < 1%% of av daily volume" type of idea too.

Excluding stocks with old (i.e. no recent) data:
My analysis for my own trading is not here in Q and honestly i am not a good python programmer, but i will think about this and, if i can't figure it out, then i will try asking around a bit. Certainly the idea makes good sense and worth doing.

All the best, Tony.

Hello Tony,

You definitely have a point about the possibility of a >10 year bear market.
I will tell you more about my personal asset allocation strategy py PM.

I'm not mistaken, there seems to be a major flaw in the STA-calculation even if I manage to use trailing 12 month net income and operating cash flow instead of quarterly values.

This is the code for the custom factor:

class STA(CustomFactor):  
    inputs = [morningstar.cash_flow_statement.operating_cash_flow,  
              morningstar.cash_flow_statement.net_income,  
              morningstar.balance_sheet.total_assets]  
    window_length = 1  
    def compute(self, today, assets, out, ocf, ni, ta):  
        ta = np.where(np.isnan(ta), 0, ta)  
        ocf = np.where(np.isnan(ocf), 0, ocf)  
        ni = np.where(np.isnan(ni), 0, ni)  
        out[:] = abs(ni[-1] - ocf[-1])/ ta[-1]  

If we look at the absolut difference between net income and operating cash flow, doesn't it mean we are also punishing companies with high cash flows in comparison to net earnings? Basically this means a company with 15 billion in net income and 5 billion in ocf is just as bad as a company with 15 billion in ocf and 5 billion in net earnings. This doesn't seem right. Maybe I missed something but I thing it should just be:

 out[:] = (ni[-1] - ocf[-1])/ ta[-1]  

All the best,
Sebastian

[email protected]

Finally got time to tinker around with the quantitave vaue screen a little more.
That's how you can check the filings for "freshness" of the filing. I just looked at net income from the most recent income statement to keep things simple.

#Has to be importet first  
from quantopian.pipeline.factors import BusinessDaysSincePreviousEvent

#That's what I use for freshness when I filter the universe  
ni1current = (BusinessDaysSincePreviousEvent(inputs=[Fundamentals.net_income_income_statement_asof_date]) <= 190)  

Cheers,
Sebastian

Hi Sebastian, that "BusinessDaysSince..." looks like just the thing. Cheers.

Hello Tony,
It took me some time but i dug deeper into the algo. One important takeaway:
Pay attention what you are looking at!
Many of the Morningstar metrics mean something different than you would expect.

If I am not totally mistaken, things like Gross Margin or ROA at a certain point in time
are actually calculated using only the income statement information from
the most recent quarter - not the last 4 quarters and not even the most recent fiscal year end.
This means yesterday's ROA is actually net income from the most recent quarter divided by most recent total assets.
That's no problem as long as your fundamental only contains balance sheet items
like the YoY long term debt growth inside the F-Score but as soon as you need to analyze anything from the
income or cashflow statement, you are probably looking at only one quarter worth of data.

Cheers,
Sebastian

Small correction:
I noticed there was still something off when I compared the ROA with the actual 10Qs.

morningstar.operation_ratios.roa.latest  

... is actually the most recent quarter's net income but NOT divided by the most recent quarter's total assets but by the average of the most recent quarter's and the previous quarter's total assets.

Cheers