Notebook

Portfolio Analysis using pyfolio

There are many ways to evaluate and analyze an algorithm. While we already provide you with some of these measures like a cumulative returns plot in the Quantopian backtester, you may want to dive deeper into what your algorithm is doing. For example, you might want to look at how your portfolio allocation changes over time, or what your exposure to certain risk-factors is.

At Quantopian, we built and open-sourced pyfolio for exactly that purpose. In this notebook you will learn how you can use this library from within the Quantopian research environment (you can also use this library independently, see the pyfolio website for more information on that).

At the core of pyfolio, we have tear sheets that summarize information about a backtest. Each tear sheet returns a number of plots, as well as other information, about a given topic. There are five main ones:

  • Cumulative returns tear sheet
  • Shock event returns tear sheet
  • Positional tear sheet
  • Transactional tear sheet
  • Bayesian tear sheet

We have added an interface to the object returned by get_backtest() to create these various tear sheets. To generate all tear sheets at once, it's as simple as generating a backtest object and calling create_full_tear_sheet on it:

In [1]:
# Get backtest object
bt = get_backtest('56cf0f6d92576d0f277f6d0e')

# Create all tear sheets
bt.create_full_tear_sheet()

100% Time: 0:00:04|###########################################################|
Entire data start date: 2011-06-01
Entire data end date: 2016-02-23


Backtest Months: 56
                   Backtest
annual_return          0.50
annual_volatility      0.42
sharpe_ratio           1.18
calmar_ratio           1.74
stability              0.93
max_drawdown          -0.29
omega_ratio            1.24
sortino_ratio          1.71
skewness              -0.25
kurtosis               3.55
information_ratio      0.06
alpha                  0.46
beta                   0.38

Worst Drawdown Periods
   net drawdown in %  peak date valley date recovery date duration
1              28.62 2014-01-22  2014-02-05    2014-05-28       91
2              27.44 2013-04-12  2013-10-08    2013-11-15      156
4              26.58 2012-03-26  2012-08-30    2012-10-17      148
0              24.58 2014-07-03  2014-08-07    2015-01-30      152
3              15.14 2015-01-30  2015-03-06    2015-04-10       51


2-sigma returns daily    -0.051
2-sigma returns weekly   -0.094
dtype: float64

Stress Events
                                    mean    min    max
US downgrade/European Debt Crisis  0.006 -0.081  0.062
EZB IR Event                       0.002 -0.078  0.092
Apr14                              0.002 -0.018  0.016
Oct14                              0.000 -0.014  0.020
Fall2015                          -0.000 -0.031  0.024
Recovery                           0.002 -0.100  0.104
New Normal                         0.002 -0.138  0.121


Top 10 long positions of all time (and max%)
[u'EDV' u'XIV']
[ 1.008  1.005]


Top 10 short positions of all time (and max%)
[]
[]


Top 10 positions of all time (and max%)
[u'EDV' u'XIV']
[ 1.008  1.005]


All positions ever held
[u'EDV' u'XIV']
[ 1.008  1.005]

Interpreting the output

There are many metrics being reported in all the tear sheets above. At the top, there are tables that tell you about summary performance statistics like the Sharpe ratio, Sortino ratio, and worst drawdown periods. The following plots are hopefully pretty self-explanatory, but more information can be found on the pyfolio website.

More fine-grained access

As the name suggests, create_full_tear_sheet() creates all tear sheets available (except for the Bayesian one, see below). You can also create individual tear sheets. For example, lets create one that only uses the returns of your strategy.

In addition, we will pass in a keyword argument called live_start_date. The use-case for this feature is that you might have deployed this algorithm and want to see how the out-of-sample period measures up to your backtest. Although it currently is not possible to access returns from live-traded algorithms in research, you could still note the date when you deployed it and run a new backtest over the full time period. This date can be passed with live_start_date. Lets pretend that we developed and deployed this algorithm on 2014-1-1. As I had access to 10 years of historical data, I could have easily overfit my algorithm to only work well on that time period. In fact, it is very difficult not to overfit, so comparing in-sample and out-of-sample (OOS) data is a good way to look at that.

This time, we will create just the returns tear sheet on the same backtest object from above:

In [2]:
bt.create_returns_tear_sheet(live_start_date='2014-1-1')

Entire data start date: 2011-06-01
Entire data end date: 2016-02-23


Out-of-Sample Months: 25
Backtest Months: 31
                   Backtest  Out_of_Sample  All_History
annual_return          0.65           0.33         0.50
annual_volatility      0.48           0.32         0.42
sharpe_ratio           1.28           1.05         1.18
calmar_ratio           2.37           1.15         1.74
stability              0.88           0.87         0.93
max_drawdown          -0.27          -0.29        -0.29
omega_ratio            1.26           1.21         1.24
sortino_ratio          1.88           1.48         1.71
skewness              -0.21          -0.50        -0.25
kurtosis               2.53           4.26         3.55
information_ratio      0.06           0.06         0.06
alpha                  0.56           0.33         0.46
beta                   0.45           0.25         0.38

Worst Drawdown Periods
   net drawdown in %  peak date valley date recovery date duration
1              28.62 2014-01-22  2014-02-05    2014-05-28       91
2              27.44 2013-04-12  2013-10-08    2013-11-15      156
4              26.58 2012-03-26  2012-08-30    2012-10-17      148
0              24.58 2014-07-03  2014-08-07    2015-01-30      152
3              15.14 2015-01-30  2015-03-06    2015-04-10       51


2-sigma returns daily    -0.051
2-sigma returns weekly   -0.094
dtype: float64

There are a few differences in the returns tear sheet that was created. Note for example that the performance table at the top now has 3 columns: Backtest, Out_of_Sample, and All_History.

The cumulative returns plot also differentiates between in-sample and OOS time periods. In addition, there is a cone that gives you an indiciation of how your algorithm is performing OOS compared to in it's backtest.

At the bottom we also see 3 distribution plots comparing the in-sample and OOS returns distributions. The first one standardizes both distributions to have the same mean and standard deviation of 1. The other two plots relax this standardization.

Bayesian analysis

There are also a few more advanced (and still experimental) analysis methods in pyfolio based on Bayesian statistics.

The main benefit of these methods is uncertainty quantification. All the values you saw above, like the Sharpe ratio, are just single numbers. These estimates are noisy because they have been computed over a limited number of data points. So how much can you trust these numbers? You don't know because there is no sense of uncertainty. That is where Bayesian statistics helps as instead of single values, we are dealing with probability distributions that assign degrees of belief to all possible parameter values.

Lets create the Bayesian tear sheet. Under the hood this is running MCMC sampling in PyMC3 to estimate the posteriors which can take quite a while (that's the reason why we don't generate this by default in create_full_tear_sheet()).

In [3]:
bt.create_bayesian_tear_sheet(live_start_date='2014-1-1')

Running T model
===============================
===============================
collect2: fatal error: cannot find 'ld'
compilation terminated.


00001	#include <Python.h>
00002	#include <iostream>
00003	#include "theano_mod_helper.h"
00004	#include <math.h>
00005	#include <numpy/arrayobject.h>
00006	#include <numpy/arrayscalars.h>
00007	#include <vector>
00008	#include <algorithm>
00009	//////////////////////
00010	////  Support Code
00011	//////////////////////
00012	
00013	
00014	    namespace {
00015	    struct __struct_compiled_op_8251ea1b4836b28fc8365dcfb7aefbef {
00016	        PyObject* __ERROR;
00017	
00018	        PyObject* storage_V3;
00019	PyObject* storage_V5;
00020	PyObject* storage_V1;
00021	        
00022	
00023	        __struct_compiled_op_8251ea1b4836b28fc8365dcfb7aefbef() {
00024	            // This is only somewhat safe because we:
00025	            //  1) Are not a virtual class
00026	            //  2) Do not use any virtual classes in the members
00027	            //  3) Deal with mostly POD and pointers
00028	
00029	            // If this changes, we would have to revise this, but for
00030	            // now I am tired of chasing segfaults because
00031	            // initialization code had an error and some pointer has
00032	            // a junk value.
00033	            memset(this, 0, sizeof(*this));
00034	        }
00035	        ~__struct_compiled_op_8251ea1b4836b28fc8365dcfb7aefbef(void) {
00036	            cleanup();
00037	        }
00038	
00039	        int init(PyObject* __ERROR, PyObject* storage_V3, PyObject* storage_V5, PyObject* storage_V1) {
00040	            Py_XINCREF(storage_V3);
00041	Py_XINCREF(storage_V5);
00042	Py_XINCREF(storage_V1);
00043	            this->storage_V3 = storage_V3;
00044	this->storage_V5 = storage_V5;
00045	this->storage_V1 = storage_V1;
00046	            
00047	
00048	
00049	
00050	
00051	            this->__ERROR = __ERROR;
00052	            return 0;
00053	        }
00054	        void cleanup(void) {
00055	            __label_1:
00056	
00057	double __DUMMY_1;
00058	__label_3:
00059	
00060	double __DUMMY_3;
00061	__label_5:
00062	
00063	double __DUMMY_5;
00064	__label_8:
00065	
00066	double __DUMMY_8;
00067	
00068	            Py_XDECREF(this->storage_V3);
00069	Py_XDECREF(this->storage_V5);
00070	Py_XDECREF(this->storage_V1);
00071	        }
00072	        int run(void) {
00073	            int __failure = 0;
00074	            
00075	    PyObject* py_V1;
00076	    
00077	        PyArrayObject* V1;
00078	        
00079	            typedef npy_float64 dtype_V1;
00080	            
00081	    PyObject* py_V3;
00082	    
00083	        PyArrayObject* V3;
00084	        
00085	            typedef npy_float64 dtype_V3;
00086	            
00087	    PyObject* py_V5;
00088	    
00089	        PyArrayObject* V5;
00090	        
00091	            typedef npy_int8 dtype_V5;
00092	            
00093	{
00094	
00095	    py_V1 = PyList_GET_ITEM(storage_V1, 0);
00096	    {Py_XINCREF(py_V1);}
00097	    
00098	        if (py_V1 == Py_None)
00099	        {
00100	            
00101	        V1 = NULL;
00102	        
00103	        }
00104	        else
00105	        {
00106	            
00107	            V1 = NULL;
00108	            if (py_V1 == Py_None) {
00109	                // We can either fail here or set V1 to NULL and rely on Ops
00110	                // using tensors to handle the NULL case, but if they fail to do so
00111	                // they'll end up with nasty segfaults, so this is public service.
00112	                PyErr_SetString(PyExc_ValueError, "expected an ndarray, not None");
00113	                {
00114	        __failure = 2;
00115	        if (!PyErr_Occurred()) {
00116	            PyErr_SetString(PyExc_RuntimeError,
00117	                "Unexpected error in an Op's C code. "
00118	                "No Python exception was set.");
00119	            }
00120	        goto __label_2;}
00121	            }
00122	            if (!PyArray_Check(py_V1)) {
00123	                PyErr_SetString(PyExc_ValueError, "expected an ndarray");
00124	                {
00125	        __failure = 2;
00126	        if (!PyErr_Occurred()) {
00127	            PyErr_SetString(PyExc_RuntimeError,
00128	                "Unexpected error in an Op's C code. "
00129	                "No Python exception was set.");
00130	            }
00131	        goto __label_2;}
00132	            }
00133	            // We expect NPY_FLOAT64
00134	            if (!PyArray_ISALIGNED((PyArrayObject*) py_V1)) {
00135	                PyArrayObject * tmp = (PyArrayObject*) py_V1;
00136	                PyErr_Format(PyExc_NotImplementedError,
00137	                             "expected an aligned array of type %ld "
00138	                             "(NPY_FLOAT64), got non-aligned array of type %ld"
00139	                             " with %ld dimensions, with 3 last dims "
00140	                             "%ld, %ld, %ld"
00141	                             " and 3 last strides %ld %ld, %ld.",
00142	                             (long int) NPY_FLOAT64,
00143	                             (long int) PyArray_TYPE((PyArrayObject*) py_V1),
00144	                             (long int) PyArray_NDIM(tmp),
00145	                             (long int) PyArray_NDIM(tmp) >= 3 ?
00146	            PyArray_DIMS(tmp)[PyArray_NDIM(tmp)-3] : -1,
00147	                             (long int) PyArray_NDIM(tmp) >= 2 ?
00148	            PyArray_DIMS(tmp)[PyArray_NDIM(tmp)-2] : -1,
00149	                             (long int) PyArray_NDIM(tmp) >= 1 ?
00150	            PyArray_DIMS(tmp)[PyArray_NDIM(tmp)-1] : -1,
00151	                             (long int) PyArray_NDIM(tmp) >= 3 ?
00152	            PyArray_STRIDES(tmp)[PyArray_NDIM(tmp)-3] : -1,
00153	                             (long int) PyArray_NDIM(tmp) >= 2 ?
00154	            PyArray_STRIDES(tmp)[PyArray_NDIM(tmp)-2] : -1,
00155	                             (long int) PyArray_NDIM(tmp) >= 1 ?
00156	            PyArray_STRIDES(tmp)[PyArray_NDIM(tmp)-1] : -1
00157	            );
00158	                {
00159	        __failure = 2;
00160	        if (!PyErr_Occurred()) {
00161	            PyErr_SetString(PyExc_RuntimeError,
00162	                "Unexpected error in an Op's C code. "
00163	                "No Python exception was set.");
00164	            }
00165	        goto __label_2;}
00166	            }
00167	            // This is a TypeError to be consistent with DEBUG_MODE
00168	            // Note: DEBUG_MODE also tells the name of the container
00169	            if (PyArray_TYPE((PyArrayObject*) py_V1) != NPY_FLOAT64) {
00170	                PyErr_Format(PyExc_TypeError,
00171	                             "expected type_num %d (NPY_FLOAT64) got %d",
00172	                             NPY_FLOAT64, PyArray_TYPE((PyArrayObject*) py_V1));
00173	                {
00174	        __failure = 2;
00175	        if (!PyErr_Occurred()) {
00176	            PyErr_SetString(PyExc_RuntimeError,
00177	                "Unexpected error in an Op's C code. "
00178	                "No Python exception was set.");
00179	            }
00180	        goto __label_2;}
00181	            }
00182	            
00183	        V1 = (PyArrayObject*)(py_V1);
00184	        Py_XINCREF(V1);
00185	        
00186	        }
00187	        
00188	{
00189	
00190	    py_V3 = PyList_GET_ITEM(storage_V3, 0);
00191	    {Py_XINCREF(py_V3);}
00192	    
00193	            V3 = NULL;
00194	            if (py_V3 == Py_None) {
00195	                // We can either fail here or set V3 to NULL and rely on Ops
00196	                // using tensors to handle the NULL case, but if they fail to do so
00197	                // they'll end up with nasty segfaults, so this is public service.
00198	                PyErr_SetString(PyExc_ValueError, "expected an ndarray, not None");
00199	                {
00200	        __failure = 4;
00201	        if (!PyErr_Occurred()) {
00202	            PyErr_SetString(PyExc_RuntimeError,
00203	                "Unexpected error in an Op's C code. "
00204	                "No Python exception was set.");
00205	            }
00206	        goto __label_4;}
00207	            }
00208	            if (!PyArray_Check(py_V3)) {
00209	                PyErr_SetString(PyExc_ValueError, "expected an ndarray");
00210	                {
00211	        __failure = 4;
00212	        if (!PyErr_Occurred()) {
00213	            PyErr_SetString(PyExc_RuntimeError,
00214	                "Unexpected error in an Op's C code. "
00215	                "No Python exception was set.");
00216	            }
00217	        goto __label_4;}
00218	            }
00219	            // We expect NPY_FLOAT64
00220	            if (!PyArray_ISALIGNED((PyArrayObject*) py_V3)) {
00221	                PyArrayObject * tmp = (PyArrayObject*) py_V3;
00222	                PyErr_Format(PyExc_NotImplementedError,
00223	                             "expected an aligned array of type %ld "
00224	                             "(NPY_FLOAT64), got non-aligned array of type %ld"
00225	                             " with %ld dimensions, with 3 last dims "
00226	                             "%ld, %ld, %ld"
00227	                             " and 3 last strides %ld %ld, %ld.",
00228	                             (long int) NPY_FLOAT64,
00229	                             (long int) PyArray_TYPE((PyArrayObject*) py_V3),
00230	                             (long int) PyArray_NDIM(tmp),
00231	                             (long int) PyArray_NDIM(tmp) >= 3 ?
00232	            PyArray_DIMS(tmp)[PyArray_NDIM(tmp)-3] : -1,
00233	                             (long int) PyArray_NDIM(tmp) >= 2 ?
00234	            PyArray_DIMS(tmp)[PyArray_NDIM(tmp)-2] : -1,
00235	                             (long int) PyArray_NDIM(tmp) >= 1 ?
00236	            PyArray_DIMS(tmp)[PyArray_NDIM(tmp)-1] : -1,
00237	                             (long int) PyArray_NDIM(tmp) >= 3 ?
00238	            PyArray_STRIDES(tmp)[PyArray_NDIM(tmp)-3] : -1,
00239	                             (long int) PyArray_NDIM(tmp) >= 2 ?
00240	            PyArray_STRIDES(tmp)[PyArray_NDIM(tmp)-2] : -1,
00241	                             (long int) PyArray_NDIM(tmp) >= 1 ?
00242	            PyArray_STRIDES(tmp)[PyArray_NDIM(tmp)-1] : -1
00243	            );
00244	                {
00245	        __failure = 4;
00246	        if (!PyErr_Occurred()) {
00247	            PyErr_SetString(PyExc_RuntimeError,
00248	                "Unexpected error in an Op's C code. "
00249	                "No Python exception was set.");
00250	            }
00251	        goto __label_4;}
00252	            }
00253	            // This is a TypeError to be consistent with DEBUG_MODE
00254	            // Note: DEBUG_MODE also tells the name of the container
00255	            if (PyArray_TYPE((PyArrayObject*) py_V3) != NPY_FLOAT64) {
00256	                PyErr_Format(PyExc_TypeError,
00257	                             "expected type_num %d (NPY_FLOAT64) got %d",
00258	                             NPY_FLOAT64, PyArray_TYPE((PyArrayObject*) py_V3));
00259	                {
00260	        __failure = 4;
00261	        if (!PyErr_Occurred()) {
00262	            PyErr_SetString(PyExc_RuntimeError,
00263	                "Unexpected error in an Op's C code. "
00264	                "No Python exception was set.");
00265	            }
00266	        goto __label_4;}
00267	            }
00268	            
00269	        V3 = (PyArrayObject*)(py_V3);
00270	        Py_XINCREF(V3);
00271	        
00272	{
00273	
00274	    py_V5 = PyList_GET_ITEM(storage_V5, 0);
00275	    {Py_XINCREF(py_V5);}
00276	    
00277	            V5 = NULL;
00278	            if (py_V5 == Py_None) {
00279	                // We can either fail here or set V5 to NULL and rely on Ops
00280	                // using tensors to handle the NULL case, but if they fail to do so
00281	                // they'll end up with nasty segfaults, so this is public service.
00282	                PyErr_SetString(PyExc_ValueError, "expected an ndarray, not None");
00283	                {
00284	        __failure = 6;
00285	        if (!PyErr_Occurred()) {
00286	            PyErr_SetString(PyExc_RuntimeError,
00287	                "Unexpected error in an Op's C code. "
00288	                "No Python exception was set.");
00289	            }
00290	        goto __label_6;}
00291	            }
00292	            if (!PyArray_Check(py_V5)) {
00293	                PyErr_SetString(PyExc_ValueError, "expected an ndarray");
00294	                {
00295	        __failure = 6;
00296	        if (!PyErr_Occurred()) {
00297	            PyErr_SetString(PyExc_RuntimeError,
00298	                "Unexpected error in an Op's C code. "
00299	                "No Python exception was set.");
00300	            }
00301	        goto __label_6;}
00302	            }
00303	            // We expect NPY_INT8
00304	            if (!PyArray_ISALIGNED((PyArrayObject*) py_V5)) {
00305	                PyArrayObject * tmp = (PyArrayObject*) py_V5;
00306	                PyErr_Format(PyExc_NotImplementedError,
00307	                             "expected an aligned array of type %ld "
00308	                             "(NPY_INT8), got non-aligned array of type %ld"
00309	                             " with %ld dimensions, with 3 last dims "
00310	                             "%ld, %ld, %ld"
00311	                             " and 3 last strides %ld %ld, %ld.",
00312	                             (long int) NPY_INT8,
00313	                             (long int) PyArray_TYPE((PyArrayObject*) py_V5),
00314	                             (long int) PyArray_NDIM(tmp),
00315	                             (long int) PyArray_NDIM(tmp) >= 3 ?
00316	            PyArray_DIMS(tmp)[PyArray_NDIM(tmp)-3] : -1,
00317	                             (long int) PyArray_NDIM(tmp) >= 2 ?
00318	            PyArray_DIMS(tmp)[PyArray_NDIM(tmp)-2] : -1,
00319	                             (long int) PyArray_NDIM(tmp) >= 1 ?
00320	            PyArray_DIMS(tmp)[PyArray_NDIM(tmp)-1] : -1,
00321	                             (long int) PyArray_NDIM(tmp) >= 3 ?
00322	            PyArray_STRIDES(tmp)[PyArray_NDIM(tmp)-3] : -1,
00323	                             (long int) PyArray_NDIM(tmp) >= 2 ?
00324	            PyArray_STRIDES(tmp)[PyArray_NDIM(tmp)-2] : -1,
00325	                             (long int) PyArray_NDIM(tmp) >= 1 ?
00326	            PyArray_STRIDES(tmp)[PyArray_NDIM(tmp)-1] : -1
00327	            );
00328	                {
00329	        __failure = 6;
00330	        if (!PyErr_Occurred()) {
00331	            PyErr_SetString(PyExc_RuntimeError,
00332	                "Unexpected error in an Op's C code. "
00333	                "No Python exception was set.");
00334	            }
00335	        goto __label_6;}
00336	            }
00337	            // This is a TypeError to be consistent with DEBUG_MODE
00338	            // Note: DEBUG_MODE also tells the name of the container
00339	            if (PyArray_TYPE((PyArrayObject*) py_V5) != NPY_INT8) {
00340	                PyErr_Format(PyExc_TypeError,
00341	                             "expected type_num %d (NPY_INT8) got %d",
00342	                             NPY_INT8, PyArray_TYPE((PyArrayObject*) py_V5));
00343	                {
00344	        __failure = 6;
00345	        if (!PyErr_Occurred()) {
00346	            PyErr_SetString(PyExc_RuntimeError,
00347	                "Unexpected error in an Op's C code. "
00348	                "No Python exception was set.");
00349	            }
00350	        goto __label_6;}
00351	            }
00352	            
00353	        V5 = (PyArrayObject*)(py_V5);
00354	        Py_XINCREF(V5);
00355	        
00356	{
00357	// Op class Elemwise
00358	
00359	        npy_float64* V3_iter;
00360	        
00361	        npy_int8* V5_iter;
00362	        
00363	
00364	
00365	        npy_float64* V1_iter;
00366	        
00367	    {
00368	        npy_intp dims[0];
00369	        //npy_intp* dims = (npy_intp*)malloc(0 * sizeof(npy_intp));
00370	        
00371	        if (!V1) {
00372	            V1 = (PyArrayObject*)PyArray_EMPTY(0, dims,
00373	                                                    NPY_FLOAT64,
00374	                                                    0);
00375	        }
00376	        else {
00377	            PyArray_Dims new_dims;
00378	            new_dims.len = 0;
00379	            new_dims.ptr = dims;
00380	            PyObject* success = PyArray_Resize(V1, &new_dims, 0, NPY_CORDER);
00381	            if (!success) {
00382	                // If we can't resize the ndarray we have we can allocate a new one.
00383	                PyErr_Clear();
00384	                Py_XDECREF(V1);
00385	                V1 = (PyArrayObject*)PyArray_EMPTY(0, dims, NPY_FLOAT64, 0);
00386	            }
00387	        }
00388	        if (!V1) {
00389	            {
00390	        __failure = 7;
00391	        if (!PyErr_Occurred()) {
00392	            PyErr_SetString(PyExc_RuntimeError,
00393	                "Unexpected error in an Op's C code. "
00394	                "No Python exception was set.");
00395	            }
00396	        goto __label_7;}
00397	        }
00398	    }
00399	    
00400	
00401	                {
00402	                  
00403	                  V3_iter = (npy_float64*)(PyArray_DATA(V3));
00404	V5_iter = (npy_int8*)(PyArray_DATA(V5));
00405	V1_iter = (npy_float64*)(PyArray_DATA(V1));
00406	
00407	                  npy_float64& V3_i = *V3_iter;
00408	npy_int8& V5_i = *V5_iter;
00409	npy_float64& V1_i = *V1_iter;
00410	
00411	                  V1_i = V3_i - V5_i;
00412	                  
00413	                }
00414	                __label_7:
00415	
00416	double __DUMMY_7;
00417	
00418	}
00419	__label_6:
00420	
00421	        if (V5) {
00422	            Py_XDECREF(V5);
00423	        }
00424	        
00425	    {Py_XDECREF(py_V5);}
00426	    
00427	double __DUMMY_6;
00428	
00429	}
00430	__label_4:
00431	
00432	        if (V3) {
00433	            Py_XDECREF(V3);
00434	        }
00435	        
00436	    {Py_XDECREF(py_V3);}
00437	    
00438	double __DUMMY_4;
00439	
00440	}
00441	__label_2:
00442	
00443	    if (!__failure) {
00444	      
00445	        {Py_XDECREF(py_V1);}
00446	        if (!V1) {
00447	            Py_INCREF(Py_None);
00448	            py_V1 = Py_None;
00449	        }
00450	        else if ((void*)py_V1 != (void*)V1) {
00451	            py_V1 = (PyObject*)V1;
00452	        }
00453	
00454	        {Py_XINCREF(py_V1);}
00455	
00456	        if (V1 && !PyArray_ISALIGNED((PyArrayObject*) py_V1)) {
00457	            PyErr_Format(PyExc_NotImplementedError,
00458	                         "c_sync: expected an aligned array, got non-aligned array of type %ld"
00459	                         " with %ld dimensions, with 3 last dims "
00460	                         "%ld, %ld, %ld"
00461	                         " and 3 last strides %ld %ld, %ld.",
00462	                         (long int) PyArray_TYPE((PyArrayObject*) py_V1),
00463	                         (long int) PyArray_NDIM(V1),
00464	                         (long int) PyArray_NDIM(V1) >= 3 ?
00465	        PyArray_DIMS(V1)[PyArray_NDIM(V1)-3] : -1,
00466	                         (long int) PyArray_NDIM(V1) >= 2 ?
00467	        PyArray_DIMS(V1)[PyArray_NDIM(V1)-2] : -1,
00468	                         (long int) PyArray_NDIM(V1) >= 1 ?
00469	        PyArray_DIMS(V1)[PyArray_NDIM(V1)-1] : -1,
00470	                         (long int) PyArray_NDIM(V1) >= 3 ?
00471	        PyArray_STRIDES(V1)[PyArray_NDIM(V1)-3] : -1,
00472	                         (long int) PyArray_NDIM(V1) >= 2 ?
00473	        PyArray_STRIDES(V1)[PyArray_NDIM(V1)-2] : -1,
00474	                         (long int) PyArray_NDIM(V1) >= 1 ?
00475	        PyArray_STRIDES(V1)[PyArray_NDIM(V1)-1] : -1
00476	        );
00477	            {
00478	        __failure = 2;
00479	        if (!PyErr_Occurred()) {
00480	            PyErr_SetString(PyExc_RuntimeError,
00481	                "Unexpected error in an Op's C code. "
00482	                "No Python exception was set.");
00483	            }
00484	        goto __label_2;}
00485	        }
00486	        
00487	      PyObject* old = PyList_GET_ITEM(storage_V1, 0);
00488	      {Py_XINCREF(py_V1);}
00489	      PyList_SET_ITEM(storage_V1, 0, py_V1);
00490	      {Py_XDECREF(old);}
00491	    }
00492	    
00493	        if (V1) {
00494	            Py_XDECREF(V1);
00495	        }
00496	        
00497	    {Py_XDECREF(py_V1);}
00498	    
00499	double __DUMMY_2;
00500	
00501	}
00502	
00503	            
00504	        if (__failure) {
00505	            // When there is a failure, this code puts the exception
00506	            // in __ERROR.
00507	            PyObject* err_type = NULL;
00508	            PyObject* err_msg = NULL;
00509	            PyObject* err_traceback = NULL;
00510	            PyErr_Fetch(&err_type, &err_msg, &err_traceback);
00511	            if (!err_type) {err_type = Py_None;Py_INCREF(Py_None);}
00512	            if (!err_msg) {err_msg = Py_None; Py_INCREF(Py_None);}
00513	            if (!err_traceback) {err_traceback = Py_None; Py_INCREF(Py_None);}
00514	            PyObject* old_err_type = PyList_GET_ITEM(__ERROR, 0);
00515	            PyObject* old_err_msg = PyList_GET_ITEM(__ERROR, 1);
00516	            PyObject* old_err_traceback = PyList_GET_ITEM(__ERROR, 2);
00517	            PyList_SET_ITEM(__ERROR, 0, err_type);
00518	            PyList_SET_ITEM(__ERROR, 1, err_msg);
00519	            PyList_SET_ITEM(__ERROR, 2, err_traceback);
00520	            {Py_XDECREF(old_err_type);}
00521	            {Py_XDECREF(old_err_msg);}
00522	            {Py_XDECREF(old_err_traceback);}
00523	        }
00524	        // The failure code is returned to index what code block failed.
00525	        return __failure;
00526	        
00527	        }
00528	    };
00529	    }
00530	    
00531	
00532	        static int __struct_compiled_op_8251ea1b4836b28fc8365dcfb7aefbef_executor(__struct_compiled_op_8251ea1b4836b28fc8365dcfb7aefbef* self) {
00533	            return self->run();
00534	        }
00535	
00536	        static void __struct_compiled_op_8251ea1b4836b28fc8365dcfb7aefbef_destructor(void* executor, void* self) {
00537	            delete ((__struct_compiled_op_8251ea1b4836b28fc8365dcfb7aefbef*)self);
00538	        }
00539	        
00540	//////////////////////
00541	////  Functions
00542	//////////////////////
00543	static PyObject * instantiate(PyObject * self, PyObject *argtuple) {
00544	  assert(PyTuple_Check(argtuple));
00545	  if (4 != PyTuple_Size(argtuple)){ 
00546	     PyErr_Format(PyExc_TypeError, "Wrong number of arguments, expected 4, got %i", (int)PyTuple_Size(argtuple));
00547	     return NULL;
00548	  }
00549	  __struct_compiled_op_8251ea1b4836b28fc8365dcfb7aefbef* struct_ptr = new __struct_compiled_op_8251ea1b4836b28fc8365dcfb7aefbef();
00550	  if (struct_ptr->init( PyTuple_GET_ITEM(argtuple, 0),PyTuple_GET_ITEM(argtuple, 1),PyTuple_GET_ITEM(argtuple, 2),PyTuple_GET_ITEM(argtuple, 3) ) != 0) {
00551	    delete struct_ptr;
00552	    return NULL;
00553	  }
00554	  PyObject* thunk = PyCObject_FromVoidPtrAndDesc((void*)(&__struct_compiled_op_8251ea1b4836b28fc8365dcfb7aefbef_executor), struct_ptr, __struct_compiled_op_8251ea1b4836b28fc8365dcfb7aefbef_destructor);
00555	  return thunk; }
00556	
00557	//////////////////////
00558	////  Module init
00559	//////////////////////
00560	static PyMethodDef MyMethods[] = {
00561		{"instantiate", instantiate, METH_VARARGS, "undocumented"} ,
00562		{NULL, NULL, 0, NULL}
00563	};
00564	PyMODINIT_FUNC init8251ea1b4836b28fc8365dcfb7aefbef(void){
00565	   import_array();
00566	   (void) Py_InitModule("8251ea1b4836b28fc8365dcfb7aefbef", MyMethods);
00567	}
00568	
Problem occurred during compilation with the command line below:
/usr/bin/g++ -shared -g -O3 -fno-math-errno -Wno-unused-label -Wno-unused-variable -Wno-write-strings -march=core-avx-i -mcx16 -msahf -mno-movbe -maes -mpclmul -mpopcnt -mno-abm -mno-lwp -mno-fma -mno-fma4 -mno-xop -mno-bmi -mno-bmi2 -mno-tbm -mavx -mno-avx2 -msse4.2 -msse4.1 -mno-lzcnt -mno-rtm -mno-hle -mrdrnd -mf16c -mfsgsbase -mno-rdseed -mno-prfchw -mno-adx -mfxsr -mxsave -mxsaveopt --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=25600 -mtune=core-avx-i -D NPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -m64 -fPIC -I/usr/local/lib/python2.7/dist-packages/numpy/core/include -I/usr/include/python2.7 -I/usr/local/lib/python2.7/dist-packages/theano/gof -fvisibility=hidden -o /home/qexec-restricted/.theano/compiledir_Linux-4.2--generic-x86_64-with-Ubuntu-14.04-trusty-x86_64-2.7.11-64/tmpuYhxjw/8251ea1b4836b28fc8365dcfb7aefbef.so /home/qexec-restricted/.theano/compiledir_Linux-4.2--generic-x86_64-with-Ubuntu-14.04-trusty-x86_64-2.7.11-64/tmpuYhxjw/mod.cpp -L/usr/lib -lpython2.7

---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
<ipython-input-3-5b96393d1a54> in <module>()
----> 1 bt.create_bayesian_tear_sheet(live_start_date='2014-1-1')

/build/src/qexec_repo/qexec/research/results.py in create_bayesian_tear_sheet(self, benchmark_rets, live_start_date, return_fig)
    941             benchmark_rets=benchmark_rets,
    942             live_start_date=live_start_date,
--> 943             return_fig=return_fig
    944         )
    945 

/usr/local/lib/python2.7/dist-packages/pyfolio/plotting.pyc in call_w_context(*args, **kwargs)
     44         if set_context:
     45             with context():
---> 46                 return func(*args, **kwargs)
     47         else:
     48             return func(*args, **kwargs)

/usr/local/lib/python2.7/dist-packages/pyfolio/tears.pyc in create_bayesian_tear_sheet(returns, benchmark_rets, live_start_date, samples, return_fig, stoch_vol)
    744     trace_t, ppc_t = bayesian.run_model('t', df_train,
    745                                         returns_test=df_test,
--> 746                                         samples=samples, ppc=True)
    747     previous_time = timer("T model", previous_time)
    748 

/usr/local/lib/python2.7/dist-packages/pyfolio/bayesian.pyc in run_model(model, returns_train, returns_test, bmark, samples, ppc)
    576                                                   bmark, samples)
    577     elif model == 't':
--> 578         model, trace = model_returns_t(returns_train, samples)
    579     elif model == 'normal':
    580         model, trace = model_returns_normal(returns_train, samples)

/usr/local/lib/python2.7/dist-packages/pyfolio/bayesian.pyc in model_returns_t(data, samples)
    165 
    166     with pm.Model() as model:
--> 167         mu = pm.Normal('mean returns', mu=0, sd=.01, testval=data.mean())
    168         sigma = pm.HalfCauchy('volatility', beta=1, testval=data.std())
    169         nu = pm.Exponential('nu_minus_two', 1. / 10., testval=3.)

/usr/local/lib/python2.7/dist-packages/pymc3/distributions/distribution.pyc in __new__(cls, name, *args, **kwargs)
     22             data = kwargs.pop('observed', None)
     23             dist = cls.dist(*args, **kwargs)
---> 24             return model.Var(name, dist, data)
     25         elif name is None:
     26             return object.__new__(cls) #for pickle

/usr/local/lib/python2.7/dist-packages/pymc3/model.pyc in Var(self, name, dist, data)
    224         if data is None:
    225             if getattr(dist, "transform", None) is None:
--> 226                 var = FreeRV(name=name, distribution=dist, model=self)
    227                 self.free_RVs.append(var)
    228             else:

/usr/local/lib/python2.7/dist-packages/pymc3/model.pyc in __init__(self, type, owner, index, name, distribution, model)
    436             self.tag.test_value = np.ones(
    437                 distribution.shape, distribution.dtype) * distribution.default()
--> 438             self.logp_elemwiset = distribution.logp(self)
    439             self.model = model
    440 

/usr/local/lib/python2.7/dist-packages/pymc3/distributions/continuous.pyc in logp(self, value)
    171 
    172         return bound(
--> 173             (-tau * (value - mu) ** 2 + log(tau / pi / 2.)) / 2.,
    174             tau > 0,
    175             sd > 0

/usr/local/lib/python2.7/dist-packages/theano/tensor/var.pyc in __sub__(self, other)
    151         # and the return value in that case
    152         try:
--> 153             return theano.tensor.basic.sub(self, other)
    154         except (NotImplementedError, AsTensorError):
    155             return NotImplemented

/usr/local/lib/python2.7/dist-packages/theano/gof/op.pyc in __call__(self, *inputs, **kwargs)
    645                 # compute output value once with test inputs to validate graph
    646                 thunk = node.op.make_thunk(node, storage_map, compute_map,
--> 647                                            no_recycling=[])
    648                 thunk.inputs = [storage_map[v] for v in node.inputs]
    649                 thunk.outputs = [storage_map[v] for v in node.outputs]

/usr/local/lib/python2.7/dist-packages/theano/tensor/elemwise.pyc in make_thunk(self, node, storage_map, compute_map, no_recycling)
    813 
    814         return super(Elemwise, node_.op).make_thunk(node_, storage_map,
--> 815                                                     compute_map, no_recycling)
    816 
    817     def perform(self, node, inputs, output_storage):

/usr/local/lib/python2.7/dist-packages/theano/gof/op.pyc in make_thunk(self, node, storage_map, compute_map, no_recycling)
   1166         self.update_self_openmp()
   1167         return super(OpenMPOp, self).make_thunk(node, storage_map,
-> 1168                                                 compute_map, no_recycling)
   1169 
   1170 

/usr/local/lib/python2.7/dist-packages/theano/gof/op.pyc in make_thunk(self, node, storage_map, compute_map, no_recycling)
    930             try:
    931                 return self.make_c_thunk(node, storage_map, compute_map,
--> 932                                          no_recycling)
    933             except (NotImplementedError, utils.MethodNotDefined):
    934                 logger.debug('Falling back on perform')

/usr/local/lib/python2.7/dist-packages/theano/gof/op.pyc in make_c_thunk(self, node, storage_map, compute_map, no_recycling)
    848         logger.debug('Trying CLinker.make_thunk')
    849         outputs = cl.make_thunk(input_storage=node_input_storage,
--> 850                                 output_storage=node_output_storage)
    851         fill_storage, node_input_filters, node_output_filters = outputs
    852 

/usr/local/lib/python2.7/dist-packages/theano/gof/cc.pyc in make_thunk(self, input_storage, output_storage, storage_map, keep_lock)
   1205         cthunk, in_storage, out_storage, error_storage = self.__compile__(
   1206             input_storage, output_storage, storage_map,
-> 1207             keep_lock=keep_lock)
   1208 
   1209         res = _CThunk(cthunk, init_tasks, tasks, error_storage)

/usr/local/lib/python2.7/dist-packages/theano/gof/cc.pyc in __compile__(self, input_storage, output_storage, storage_map, keep_lock)
   1150                                     output_storage,
   1151                                     storage_map,
-> 1152                                     keep_lock=keep_lock)
   1153         return (thunk,
   1154                 [link.Container(input, storage) for input, storage in

/usr/local/lib/python2.7/dist-packages/theano/gof/cc.pyc in cthunk_factory(self, error_storage, in_storage, out_storage, storage_map, keep_lock)
   1600         else:
   1601             module = get_module_cache().module_from_key(
-> 1602                 key=key, lnk=self, keep_lock=keep_lock)
   1603 
   1604         vars = self.inputs + self.outputs + self.orphans

/usr/local/lib/python2.7/dist-packages/theano/gof/cmodule.pyc in module_from_key(self, key, lnk, keep_lock)
   1172             try:
   1173                 location = dlimport_workdir(self.dirname)
-> 1174                 module = lnk.compile_cmodule(location)
   1175                 name = module.__file__
   1176                 assert name.startswith(location)

/usr/local/lib/python2.7/dist-packages/theano/gof/cc.pyc in compile_cmodule(self, location)
   1511                 lib_dirs=self.lib_dirs(),
   1512                 libs=libs,
-> 1513                 preargs=preargs)
   1514         except Exception as e:
   1515             e.args += (str(self.fgraph),)

/usr/local/lib/python2.7/dist-packages/theano/gof/cmodule.pyc in compile_str(module_name, src_code, location, include_dirs, lib_dirs, libs, preargs, py_module, hide_symbols)
   2185             # difficult to read.
   2186             raise Exception('Compilation failed (return status=%s): %s' %
-> 2187                             (status, compile_stderr.replace('\n', '. ')))
   2188         elif config.cmodule.compilation_warning and compile_stderr:
   2189             # Print errors just below the command line.

Exception: ("Compilation failed (return status=1): collect2: fatal error: cannot find 'ld'. compilation terminated.. ", '[Elemwise{sub,no_inplace}(mean returns, TensorConstant{0})]')

Lets go through these row by row:

  • The first one is the Bayesian cone plot that is the result of a summer internship project of Sepideh Sadeghi here at Quantopian. It's similar to the cone plot you already saw at in the tear sheet above but has two critical additions: (i) it takes uncertainty into account (i.e. a short backtest length will result in a wider cone), and (ii) it does not assume normality of returns but instead uses a Student-T distribution with heavier tails.
  • The next row is comparing mean returns of the in-sample (backest) and OOS (forward) period. As you can see, mean returns are not a single number but a (posterior) distribution that gives us an indication of how certain we can be in our estimates. As you can see, the green distribution on the left side is much wider representing our increased uncertainty due to having less OOS data. We can then calculate the difference between these two distributions as shown on the right side. The grey lines denote the 2.5% and 97.5% percentiles. Intuitively, if the right grey line is lower than 0 you can say that with probability > 97.5% the OOS mean returns are below what is suggested by the backtest. The model used here is called BEST and was developed by John Kruschke.
  • The next couple of rows follow the same pattern but are an estimate of annual volatility, Sharpe ratio and their respective differences.
  • The 5th row shows the effect size or the difference of means normalized by the standard deviation and gives you a general sense how far apart the two distributions are. Intuitively, even if the means are significantly different, it may not be very meaningful if the standard deviation is huge amounting to a tiny difference of the two returns distributions.
  • The 6th row shows predicted returns (based on the backtest) for tomorrow, and 5 days from now. The blue line indicates the probability of losing more than 5% of your portfolio value and can be interpeted as a Bayesian VaR estimate.
  • Lastly, a Bayesian estimate of annual alpha and beta. In addition to uncertainty estimates, this model, like all above ones, assumes returns to be T-distributed which leads to more robust estimates than a standard linear regression would.

For more information on Bayesian statistics, check out these resources:

Using pyfolio directly

Above, we saw how we can easily create a variety of tear sheets. These are all created using a thin wrapper on top of the pyfolio OSS library. You might also want more fine-grained access over the functionality provided by this library. For this, you can import pyfolio and use it directly.

In [ ]:
import pyfolio as pf
In [ ]:
returns = bt.daily_performance.returns
pf.timeseries.cum_returns(returns).plot();

For more information on the usage of the library, check out the pyfolio website or our GitHub repo.

Contributing

pyfolio is still a very new project — there will be bugs and there are many rough edges. Your help is greatly appreciated.

If you find bugs or have other questions, please report them to our issue tracker. We also appreciate any contributions. For some ideas on where to start, see the 'help wanted' tag.