New Feature - Record and Plot Variables

Back to Community

edited Mar 5, 2013

We shipped a feature today that I think you are all going to love. You can now graph arbitrary values with your backtests!

You can read the full help description in our API help document, but here is the quick-and-dirty. Within your handle_data function, you can identify up to five data series to be recorded and plotted. Each series consists of a name of the series and the corresponding values to be plotted. Your code might look like this:

    record(short_mavg = data[context.sid].mavg(20),  
        long_mavg = data[context.sid].mavg(60),  
        goog_price = data[context.sid].price)

Those time series are then plotted in your backtest. You can click on a data series in the legend to remove a series, and then click again to add it back.

There are a number of different ways to use this new feature. We think the most common way is going to display the buy/sell signals that you are computing, like the example below. It's very illuminating to see when the different-length moving averages cross each other. You can also code it such that it shows stock position, buy/sell events, just about anything that you want to see displayed.

There is at least one bug out there - the y-axis is sometimes drawn without numerical labels. Please let us know if you find any others.

All of our features come from member feedback, but this one in particular was driven by your suggestions. Thank you all for weighing in, particularly here and here.

One note about data exploration: We get a lot of feature requests for data exploration tools. We hear them, loud and clear, and we don't think that this feature is powerful enough to be a true data exploration tool; it just scratches the surface. Most days we're working hard on providing live trading so that you can actually trade your algorithms through your broker. Once live trading is available we will have more time to work on data exploration.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

14 responses

Thanks! --Grant

Here's a quick example comparing:

context.stocks = [sid(8554),sid(32268)]  # SPY & SH

Note the peak in NCD in early 2009.

--Grant

Dan Dunn

Mar 6, 2013

That's some fascinating stuff, Grant. You can also plot the price signal stuff you built with zlib and see what insight you get.

Dan

Disclaimer

Tim Meggs

Mar 6, 2013

Dan - this new feature is great and will be super for helping users get an intuitive feel for the interaction between the various parameters within their algo. Being able to plot variables/indicators like this helps with both idea generation and when 'debugging' the performance of a strategy. Great work Quantopian dev team!

John Fawcett

Mar 7, 2013

Anony Mole,

Thanks for the questions. We regularly debate using today's stock symbols versus using unique and time-invariant identifiers for the securities. Your comments here have us debating again :).

On the one hand, over a 10 year history, stock symbols are very problematic - symbols change, symbols are passed between companies, symbols are even re-used for as many as 36 different stock series for the same company. This is all because symbols are designed to uniquely identify stocks on an exchange today, which of course doesn't work well when running a simulation over a decade of data.

But, on the other hand, it is far easier to look up stocks using today's symbols. The IDE provides auto-complete that includes a name/symbol search for this reason. For the closely related issue of reading the code later, we provide a mouse-over tip on the security identifier (sid) that displays the company information. Even still, most people end up defining variables named for the security symbol: context.UTH = sid(21704)

Today, we got to thinking that maybe we should simply provide both. sid("UTH") could use today's exchange symbols to choose a security to test. That would let users identify securities naturally, and users would just have to be aware that the code may be affected by symbol changes. If a user is concerned about symbol changes, they can choose to use the numeric: sid(21704).

I'd love to hear what folks think about this design point.

thanks again,
fawce

Disclaimer

Jonathan Kamens

Mar 7, 2013

Since backtests should be idempotent, i.e., aside from any bugs we fix in the backtester, if you run the same algorithm against the same stocks over the same time range twice it should return the same results each time, I'm not sure it's a good idea to allow a lookup like sid("UTH") which could return different results depending on when it is run.

However, how about allowing time-bound by-symbol lookups, e.g., add a second argument to the sid() call to indicate, "I want the stock which was represented by this symbol as of this date."

To make this easier to use, we could make the IDE automatically insert the second argument as the current date when the user types the start of a SID Call with a symbol instead of a number; then the user can edit the date if he actually wants to refer to the stock as of a date other than today.

Disclaimer

Grant Kiehne

Mar 8, 2013

Fawce & Jonathan,

Any thoughts on a more sophisticated security selection tool (e.g. https://www.google.com/finance/stockscreener)? A specialized, customizable screening tool could be integrated with the algorithm code editor. The user would click an "Add Securities" button and a GUI would open, do his screening, and then click an "Insert into Algorithm" button with the result:

# Security symbols on 3/8/2013:  
# SPY, SH  
# Additional relevant information  
context.stocks = [sid(8554),sid(32268)]

If you enable comment blocks that can be collapsed, lots of additional information can be automatically populated into the algorithm without cluttering the code.

By the way, I am surprised that you haven't provided more guidance and ideas to users based on screening routines that you could run on the data (or perhaps your license excludes this). For example, could you write a screen in an effort the identify another trading pair like GLD & GDX, and then provide candidates to users for backtesting.

Grant

Dan Dunn

Mar 8, 2013

Hi Grant,

We're definitely interested in better stock picking tools. To me, that's a specific type of data exploration. You've heard our overall vision before on this one: algo writing is data exploration, followed by backtesting, followed by live trading. We've built a backtesting tool, we're doing live trading now, and we plan on doing data exploration in the future.

Part of stock picking is adding additional data sources, like company fundamentals. We think that is an integral element. Again, something on our plate for future addition.

Pair trading remains one of the interesting options, doesn't it? set_universe makes it possible to check wider swaths of stocks for pairs that was possible before. We're continuing to improve the backtester, and that should open up even more.

Dan

Disclaimer

Grant Kiehne

Mar 9, 2013

Hello Dan,

Good point regarding the new set_universe feature...I'll have to get a better feel for how it works.

Grant

Pallas Sallap

Mar 22, 2013

fawce,

Sorry to necro-post but your symbol-lookup selections are spot on. Idempotency of symbol lookup is better than no symbol lookup, but it is surprising and can be relaexd: if one sees "T" and runs a backtest for "T", that's "T" when the backtest was run (news articles about "T" from 1980 are not in significant danger of being misinterpreted). For idempotency, use the sid or possibly a named parameter to the sid lookup function that takes a date.

Eugeny Kaminsky

Dec 3, 2015

This is fantastic feature. How I can create much more one graph window?
It's possible?
Also, can I place line (support,resistance) to graph?

ccbttn

Sep 20, 2016

also curious about how to create more than one graph window

ryan clous

Mar 15, 2017

I agree with Emma, a second graph window would be phenomenal. I personally think only 5 records is sort of limiting especially for a live trading algo. I would prefer at least 10. Although, I really do appreciate what I have and appreciate the hard work you all have put into this. Only trying to provide feedback.

Dave od

Jun 18, 2020

My biggest time-frame is 2 or 3 days.
I use minutely talib indicators but record funtion only allows me to plot 1 value a day.

Can I use record method for 1 hour or minute data?
If not, can I export the data in some way?

Thanks in advance.

You've successfully submitted a support ticket.

Our support team will be in touch soon.