Hello Grant,

I think you are closing orders "> 15" hundred hours i.e. at 16:00. This means they are filled at 09:31 if they are not cancelled overnight by Quantopian. It's not clear to me what the status of cancelling overnight orders is now and in the future. See: https://www.quantopian.com/posts/intraday-trades

P.

Orders placed in the last minute of the day (or open orders hanging around from anytime prior) are cancelled at EOD both in simulation and in broker-backed live trading. The last opportunity for an algo to place a trade is at 3:59pm (on days w/normal market hours of course).

If you try to create orders overnight in some way I don't think that will work, but I'd have to look at an example to be sure.

Hello Jess,

I think in Grant's algo above an order is placed at 16:00 and is filled at the 09:31 bar price. If this is the case when will the backtester be brought in line with paper/live trading? (Dan's comment at https://www.quantopian.com/posts/intraday-trades suggests the reverse is the case.)

Also, are any Stop, Limit and Stop Limit orders cancelled overnight or just unfulfilled Market orders?

P.

Thanks Peter,

Yeah, that's an algorithm bug. I'd intended to close out before EOD. But it appears that you've uncovered a backtester bug (or at least inconsistency with live trading). Indeed, when I look at the Transaction Details, all of the action is at 9:30ish in the morning, so the close-out orders are being carried over to the next morning.

Grant

Hi Peter,

I just went over this with Dan again to make sure: my answer was correct for live trading, where we cancel all orders EOD. But you're right - in the backtest we do not cancel any orders, they can hang around and get executed the next day.

We will look at bringing these two into line I think, as this seems like one of the few places where we do have an inconsistency in behavior between backtested and simulated execution.

What I would suggest in the meantime to maintain as close as possible parity with live trading is to add logic that cancels all open orders at 4:00pm and not to place new orders at 4pm. Not a great answer I'll grant you - but for now that should work I think.

Best regards, Jess

Here's the algo, with the close-out orders at 3 pm or later:

def intradingwindow_check(context):  
    # Converts all time-zones into US EST to avoid confusion  
    loc_dt = get_datetime().astimezone(timezone('US/Eastern'))  
    # if loc_dt.hour > 12 and loc_dt.hour < 15:  
    # if loc_dt.hour == 10 and loc_dt.minute == 0:  
    if loc_dt.hour > 14:  
        return True  
    else:  
        return False  

For some reason, 2/25 results in the close-out ordering going off the rails. Will need to sort that out. --Grant

To remedy the close-out ordering problem mentioned above, I used:

if context.portfolio.positions_value != 0.0 and intradingwindow_check(context):  
        for stock in context.stocks:  
            if not bool(get_open_orders(stock)):  
                amount = context.portfolio.positions[stock].amount  
                if amount != 0:  
                    order(stock,-amount)  

I think the problem was that without checking for open orders, excess orders were submitted for a thinly traded security.

Here's a version w/ set_universe. Not sure if I violate typical margin limits here or not...perhaps someone can point me to some code to assess this (probably an opportunity for a standardized Quantopian helper function). --Grant

Hello Grant,

With your $1M starting cash you simultaneously go long $1M and short $1M which I don't think is allowed. As the Quantopian search feature only returns 25 results it's hard to find this https://www.quantopian.com/posts/template-limiting-leverage-by-brandon-ogle-and-dan-sandberg now but this should be the starting point. (Search for "margin call".)

P.

Thanks Peter,

I'll have to learn more about this margin thingy (although I don't even know how to trade with my own money, let alone borrowed). Presumably, in real-world trading, Interactive Brokers (IB) would just cancel (or hold?) orders that exceed limits, and send a notification via e-mail (automated phone call?), which brings up a good question for the Quantopian folks. Will the algorithm receive any communication from IB regarding how much capital can be allocated and in what fashion (long/short/borrowed)? In other words, will the algorithm have access to info. from IB giving the current bounds to avoid a margin call (perhaps including any pending orders)?

Grant

Hello Grant,

IB use real-time margining from what I read so they reject any orders that would lead to a margin call. Available margin would be available to view in TWS but it would be useful (essential?) if this could be available to the algo.

P.

Peter and Jess,

Here's an example of an order (TZV) that got carried over to the next day, even though I tried to get it filled before EOD:

2013-02-21 09:36:00     TZV     BUY     28097   $37.74  $1,060,380.78  
2013-02-25 09:31:00     TZV     SELL    -28097  $37.95  ($1,066,281.15)  
2013-02-25 09:32:00     RNN     BUY     3219538     $0.33   $1,068,886.62  
2013-02-25 15:48:00     RNN     SELL    -3219538    $0.33   ($1,052,788.93)  

The code is set up to close out before EOD (e.g. RNN), but sometimes, orders carry over (e.g TZV). The problem I see is that if open orders are automatically cancelled at EOD, then the code will have to compensate by closing out as early as possible the next morning (which I think my code does). I've posted a separate thread to address the question (see https://www.quantopian.com/posts/why-cancel-orders-at-end-of-day), but I thought I'd give you some insight into why I thought to ask the question in the first place. It basically means that every algorithm has to have code to potentially re-submit any orders that got automatically cancelled (but perhaps this is standard coding practice to be regularly checking to see that the portfolio is in the desired state at any given time).

Grant

Hello Jess,

Just a thought, but is the EOD order cancellation mechanism for live trading already coded into zipline, and I just need to use it for backtesting? Or is it not part of the open source code?

Grant

apologies for this python/pandas related question but what exactly is happening with the syntax myPandasDataFrame[1:,:]?
I don't know what the comma in there is doing, and I can't find reference to using this syntax anywhere.

but it seems like you're taking the 2nd to the end of the list for opens and the start to the penultimate item for closes? Could that be correct? If so, why do this?

    opens = history(30,'1d','open_price',ffill=False).as_matrix(context.stocks)[1:,:]  
    closes = history(30,'1d','close_price',ffill=False).as_matrix(context.stocks)[0:-1,:]

Hello P.,

I don't understand that syntax either. I'm going to guess that it's meant to discard the minute value value from 'history'?

To answer the second part of the question:

Open    100    104    103    110

Close   102    106    109    108

Gap       ?     +2     -3     +1  

P.

Hi Pumplerod,

Let's say that you have a dataframe of stocks on the rows and OHLC values on the columns. Now, when you call df.ix[1:, :] this says for all the stock in the rows, except for the first stock, return all the OHLC values. This will return another dataframe. In this syntax the comma is distinguishing between the rows and columns of the dataframe.

Now let's push the example a little bit further. Let's say that you have the same dataframe as before, but you only want the open price. You can say df.ix[1:, 0] which will return all the stocks except the first one and their open prices.

Also, note that you need the ".ix" notation when you're slicing into the dataframe since you have both integers and letters.

Cheers,
Alisa

Hmm. Then in grants example (Which doesn't use the ".ix" notation) is it selecting all open values for all but the first stock and all closing values for all but the last stock? I must be missing something pretty basic. I can see some logic behind finding the gap between yesterday's close and today's open, which Peter seems to pointing out.

Hello Peter and Pumplerod,

The code I think you are trying to sort out is:

    opens = history(30,'1d','open_price',ffill=False).as_matrix(context.stocks)[1:,:]  
    closes = history(30,'1d','close_price',ffill=False).as_matrix(context.stocks)[0:-1,:]  
    gap = pd.DataFrame(opens - closes).as_matrix()  

The idea is to subtract the closing prices from the opening ones. So, the slicing [1:,:] of the opens eliminates the first row, which corresponds to the oldest open. And the slicing [0:-1,:] of the closes eliminates the last row, which corresponds to the most recent close. So, when the closes are subtracted from the opens, elementwise, the resulting matrix is the gap between the open, and the prior close, for each day. So, for example, the last row in gap will correspond to the opening price for the day minus the closing price for the prior day.

By the way, this will work, too:

gap = opens - closes  

I think that originally I'd used a pandas function to filter out NaN's, but as it turned out, I dropped the approach.

Overall, the code I posted was kinda hacked together, and surely can be improved, including perhaps using the ".ix" notation Alisa refers to.

Hope this helps.

Grant

Yes, it does grant, thanks. I need to learn more about working with arrays. I'm always looping over each element, where it seems much more elegant to use these built in methods.

Here's an update. I added the function cancel_orders_EOD to cancel open orders at EOD. Seems to work, since before adding it, I got:

2013-02-21 09:36:00     TZV     BUY     28097   $37.74  $1,060,380.78  
2013-02-25 09:31:00     TZV     SELL    -28097  $37.95  ($1,066,281.15)  
2013-02-25 09:32:00     RNN     BUY     3219538     $0.33   $1,068,886.62  
2013-02-25 15:48:00     RNN     SELL    -3219538    $0.33   ($1,052,788.93)  

Now, the result is:

2013-02-21 09:36:00     TZV     BUY     27473   $37.74  $1,036,831.02  
2013-02-25 10:05:00     TZV     SELL    -27473  $38.16  ($1,048,342.21)  

Still more work to do, but the EOD cancellation seems to be consistent with the anticipated live trading behavior.

Grant

Another update. I added an error function to compute the percent to put into the security of the day:

    a = 0.25  
    pct = math.erf(-a*min_gap_z)  
    record(pct=pct)  
    order_target_percent(context.stocks[k_min],pct)  

Seems to smooth out the returns a bit. I also made a few other changes, such as (typically) buying near the open, and selling at 10 am (some securities are very thinly traded, so buy/sell doesn't always go through as directed).

--Grant

thanks grant. I get the feeling i should move to intraday by default, my current work is all using interday, so I am guessing the extra aftermarket volitility may impact my algorithm designs.

Please take note that the code I posted is a messy work-in-progress (I can hardly follow it), which I'm trying to understand how to structure. One thought is to use some form of a finite state machine architecture (see the nice example posted by Ed Bartosh on https://www.quantopian.com/posts/finite-state-machine-in-python). Using low dollar-volume stocks seems like a good way to go in developing code, to mimic a worst-case real-world trading scenario (e.g. I've turned off the slippage model, and still see that after buying near the open, sometimes the algorithm can't sell until the next day).

Grant

Grant, I'm liking this strategy. Thanks for keeping us up to date on it.

fyi i tried running a backtest of this from 2003 and the model doesn't perform well. please be careful of overfitting! this is the first intraday backtest I tried... it ran for about 4 hours before timing out at 50% (tried 2003 to present)

Hello Jason,

Are you sure it timed out? I use Chrome and sometimes it is necessary to refresh the full backtest page. Also the end time for the algo often isn't shown until the page is refreshed.

(A few months ago I felt Chrome was better suited to Quantopian than Firefox but I haven't revisited this lately.)

P.

@Peter: I just checked the backtests and it says "Exception" so yeah, i guess it tanked. i'll try attaching it here:

i guess it got through 2009 before crashing.

Jason,

Yes, in many ways, this hacked together algorithm is a work-in-progress. It has piqued my interest since there does seem to be some "money left on the table" for low dollar-volume/thinly traded securities. To really sort out what's going on, I think that an R&D platform would be required, to look at statistics versus time. Backtesting is kinda the last verification step, with validation being paper and finally real-money trading at IB. I've been scratching my head how to do the R&D part, so if anyone has a recommendation, I'm interested.

Grant

Here's a run w/:

    price = data[context.stocks[k_min]].price  
    shares = cash/price  
    order(context.stocks[k_min],shares,limit_price=price,stop_price=price)  

Oddly, it just stops trading mid-June, 2012. Gotta sort out why.

[EDIT] I suspect that the problem is due to my blanket checking for open orders prior to selling. With the limit/stop in, perhaps an order is hanging around that prevents the algorithm from executing properly.

Grant

Grant,

Just curious, what was the reason behind setting your universe to the 20-22nd percentile vs any other range?

Mike,

Well, I just started playing around with set_universe. I recall that for higher dollar-volume securities the performance was poor. And the other day, I tried very low dollar-volume, which was bad, as well. So, presently, I can continue to develop the algorithm with the 20-22nd percentile range, but it is not sacred.

My intuition (based on no real-world trading experience) is that low dollar-volume stocks might be the way to go. High dollar-volume ones are going to be very efficient incorporating overnight/weekend information, and there will be less over-reaction/panicky price swings. But this is just an uneducated hunch at this point.

Grant

I have noticed that when I attempt to use securities from the lower end of the dollar/Volume universe that there is trouble executing trades. Even when I try something as simple as buy/sell just 100 shares, there is often carryover into the next day. When I use securities in the 98-100% range this does not occur. For me, this makes looking at the results a bit confusing. Are others finding this to be true as well? And is this expected behavior?

Yes, I've seen the same thing, but I think that it comes with the territory (i.e. in the real world, for thinly traded securities, buying and selling can't be done on-demand). I do believe it is the expected behavior, since my understanding is that the backtester relies on historical trade events (rolled up minute bars from real trade data) to trigger buying/selling (although I can't lay out the exact mechanics -- Quantopian folks, can you elaborate?). --Grant

Grant you're exactly right - this is the expected behavior comes with the territory for thinly traded securities. For illiquid stocks, sometimes there are not enough trades on the market to buy/sell even 100 shares in one day. So the execution will carry over several days until the order is filled. This is contrary to a liquid stock, like Apple, which will be fully filled in the next bar because of the high availability.

So, it seems like an intraday trading method for low dollar/volume stocks would be difficult to get any reliability from.

Pumplerod,

Again, I ain't no expert, but there should be ways of mitigate the liquidity risk. And I figure there must be a lot less competition, so maybe the reward is greater?

One important question is how well can the Quantopian backtester model thinly traded stocks? Or does one have to go to a paper trading scenario at IB to really understand the situation?

Grant

The backtest above got stuck on SNK (http://finance.yahoo.com/q?s=SNK). Any idea what happened?

@Grant, I'm using quantopian for my R&D, currently working on a strategy based around stochastic oscillators. generally I do development/tuning against a benchmark security (currently SPY because i'm interested in high volume etf's) and then I run verification pass on other similar etfs. finally, i try running a test on the 99.5 to 100 percent universe.

however my current algorithm goes crazy when using the universe, and no debug support means it's hard to track down the issue. plus, I hate the 3 minute wait just for running a interday backtest (for a single security) so I might have to try zipline, especially once I try to move my algorithms to something that better simulates a real-world market.

Hello Jason,

For your bug issue, you could try posting some stripped-down code to the forum, removing anything you need to keep private. You might also elaborate on what you mean by the algorithm going crazy.

Regarding the time it takes to run a backtest, you can run multiple backtests in parallel (unless something has changed, there is no limit). It is a manual process to launch each backtest, but to keep track of parameters that you are varying , you can write them out to the log. A trick I've used is to use a random number generator to sample a parameter space (however, as I recall, it is important to re-seed the generator, otherwise it won't be random--I'll dig up the info. and post it separately to the forum).

Note that if you are planning on paper/real money trading with Quantopian/IB, you'll need to write your algos to run on minute bars.

Grant

Update on SNK:

http://www.nyse.com/pdfs/TraderUpdateSusp%28SKN&SVW%29.pdf

Per the document, SNK is the symbol for Bank of America SPX Capped Leveraged Index Return
Notes (CUSIP 06052K281), with a maturity of 6/29/2012.

So, the algorithm bought into SNK, and then couldn't sell (likely due to the lack of SNK events in the database). However, presumably, anyone holding the notes after de-listing would receive a cash payment, right?

How to handle this (in a general sense, not by putting in logic to exclude SNK from context.stocks)?

Grant

I excluded SNK, and got the attached result. Note that the slippage model is disabled. Also, I need to add some checks that there is no excessive borrowing, or other unreal-isms.

For a comparison, I'm also running with:

set_slippage(slippage.VolumeShareSlippage(volume_limit=0.25, price_impact=0.1))  

The return basically mirrors the market about the horizontal axis--very negative, but not yet clear why.

Grant

Here's the result w/:

set_slippage(slippage.VolumeShareSlippage(volume_limit=0.25, price_impact=0.1))  

I made a slight variation of your algo Grant, where I wanted to try and spread out the risk over a set of securities rather than just pick the security with the min z-score.

@Grant I setup a framework to help manage things, like ensuring the pool of invested securities doesn't exceed my capital, etc. I'd share it but

1) it is my first-try so is a bit of a mess
2) doesn't work with intraday so kind of pointless to design algorithms around this heavy framework
3) potential disaster bugs: as I described, when using the universe from 2003 to present, my algo goes crazy: starts exceeds the capital limits I put in place and then a month or two later the whole thing crashes to -100% I'm guessing it's hitting either a security that fails spectacularly, or one that gets delisted. I'll try hard-coding the securities and A/B test them until I find the cause.
4) no offline support: so hard to debug, and slow to backtest (I've spent a good 100 hours on this over the last 2 weeks)

so when I resolve those issues I'll post the framework up on github no problem. If someone wants my framework now just lemmi know, though I don't think it'd be very useful to anyone but myself at the moment.

Thanks Pumplerod,

I had the same thought in the back of my mind. I think that my next consideration will be to understand how to apply an accurate slippage model (which, in the end, would require validation via paper trading at IB). If the whole idea falls apart with realistic slippage, then it's just an exercise in writing code.

Grant

one thing I notice about my stochastic algo that may (or may not) be applicable to you, but as my algo accumulates state over time, it's effectivively non-deterministic. that means if I change the start date slightly the decisions my algo makes over time will be different. I'm thinking about changing it to look at fixed windows of history (recomputing the state each timestep) so it becomes deterministic.