Hi Kyu,

To answer your questions:

1). You're correct, we use minute price data. In your example, I assume that you're looking to BUY, so I will tweak your numbers a little bit - in a buy limit order, you're waiting for the price to drop low enough to trigger your order, not the other way around.

Say you want to buy AAPL at 500.50. At 10:00:00 the price is 500.55 and at 10:01:00 the price is 500.45. Your order will be filled in the second minute bar at 10:01:00 according to the slippage model. The default slippage model will allow you to only execute for 25% of the market volume in the minute bar. If the order is not completely filled, they will be filled in the next bar that meets your limit order criteria. This may happen in minute 3, minute 16, or may never be met.

2). Yes, we have several ways to track the order status. You can use get_open_orders() to see your list of open orders or you can call get_order() to see a specific transaction. If you want to see additional messages, you can use logs in your algo to track other behavior.

3). You're right, that is a very difficult problem - we've done a lot of work to model this as well as we reasonably can. I think you'll like our solution. If you place an order in minute 0, it will get filled in the next bar at minute 1. Each minute bar has an open, low, high, and close - the price for the bar will be the close price. This fill system is intended to prevent look-ahead bias - you cannot have forward knowledge of the price. Also, we use slippage (as I mentioned earlier) to model the impact of your order in the market. You can modify the slippage to any amount in your code.

4). When you're doing live trading, we'll execute your algorithm every minute as the price data becomes available. And depending on the complexity of your algo, it will take 1-2 seconds to send off your order to the market.

Best,
Alisa

Thanks Alisa,

Just few more questions:

Say you want to buy AAPL at 500.50. At 10:00:00 the price is 500.55 and at 10:01:00 the price is 500.45. Your order will be filled in the second minute bar at 10:01:00 according to the slippage model. The default slippage model will allow you to only execute for 25% of the market volume in the minute bar.

1). So in this example, instead of 500.50 (limit order price), 500.45 (close price at 10:01:00) will be the one that my limit order get filled at, right?

If you place an order in minute 0, it will get filled in the next bar at minute 1. Each minute bar has an open, low, high, and close - the price for the bar will be the close price. This fill system is intended to prevent look-ahead bias - you cannot have forward knowledge of the price.

2). So this means that all the orders are based on the close price of each bar (in this example, 1-min. bar). No inside bar prices (e.g. high, low, or tick data) affects filling my orders, right?

For example, at 10:00:00, I want to buy AAPL with limit order at 500.50.
At 10:00:01, the price is 500.55, at 10:00:10, the price(close) is 500.45, at 10:00:30, 500.30, at 10:00:40, 500.50, and at 10:00:50, 500.60, and at 10:01:00, the close price is ** 500.55**. So at this bar of 10:01:00, OHLC was 500.55, 500.60, 500.30, and 500.55 respectively.

Then, my order is not going to get filled, right? After then, if AAPL goes straight up to 501.00, and up until 505.00. Then, even though my limit order to buy was 500.50 and the limit price was inside bar of 10:01:00, am I going to miss this?

In real market, the story will be different. What do you think?

I just want to figure out how much discrepancy I will get between result from this back testing model and real money trading.

Thank you.

Hi Kyu,

1). Yes, since you are waiting for the price to drop below 500.50 for your order to be filled. This is achieved at 10:01 in the example at 500.45, which will be the price when your order gets filled.

2). You're exactly right - all orders are based on the close price of each bar. We do not provide tick-level data; only minute and daily bars, which we find is sufficient for most intraday strategies. Otherwise, we would be headed down the path of high frequency trading. In your specific example, your order will not get filled because the close price did not meet your limit order. This backtest simulation is similar to live trading because Interactive Brokers provides minute-level price data.

Cheers,
Alisa

Thanks, Alisa

Glad to help!

Hi Alisa,

I have a few more questions:
Basically, I want to get the exact price filled after sending my orders. It is shown in Transaction Details after running a full backtest. But I want to know the exact price that my orders got filled at ***inside my algo***. The reason why I want this is that I want to run a stop-loss and profit target features based on the cost basis.

1). I tried the object, cost_basis but log says 0.0. I don't know why I get 0 instead of cost price.
That's why I used open_price as below.

 short_stop_price = data[context.stock].open_price * context.short_stop_pct

2). I added a stop-loss for short position only as price times 1.005 (meaning if price goes back up by 0.5% of cost).
See below:

context.short_stop_pct = 1.005

qty = context.portfolio.positions[context.stock].amount

short_stop_price = data[context.stock].open_price * context.short_stop_pct

*** stop loss short position ***
elif qty < 0 and price > short_stop_price and context.short_position_exited == 0:
context.short_position_entered = 0
context.short_position_exited = 1
order_target_percent(context.stock, 0)
log.info('stop loss buytocover: ' + str(qty) + 'shares')

See below what the logs showed:
2013-01-14handle_data:85INFOSellshort -985 shares of Security(24 [AAPL]), price, 500.69
2013-01-14handle_data:91INFOstop loss buytocover: -985shares

And see what the Transaction Details said:
2013-01-14, 09:32:00, AAPL, SELL, -985, $500.12, ($492,620.17)
2013-01-14, 09:37:00, AAPL, BUY, 985, $503.80, $496,242.01

4). The price (in the logs), 500.69 is different from 500.12 (in the Transaction Details). It had a slippage of 0.57, which I think is hugh.

5). And my stop-loss (0.5% back up for short) price, 503.80 is not the price that 0.5% backed up from 500.12. It was actually 0.736% instead of 0.5%. This discrepancy is also hugh.

So in this single transaction, I lost a lot more than I was supposed to.
Am I doing something wrong here?
Please help me here.

Thank you.

@Kyu: let me investigate and I'll get back to you with some answers.

@Anony: Yes, the market orders are filled at the close of the next bar. If they were filled at the open of next bar, this would be one tick later. We don't support tick-by-tick updates and didn't design for HFT. Tracking the price from close-to-close is more conservative, and if you're trying to see updates per second, we're not the right platform. Handle_data() is called once per minute and will place the orders at that frequency. This setup can be useful for many intraday strategies!

AAPL

01/17/2014

Time, Open, High, Low, Close
09:31, 551.20, 551.60, 550.25, 551.20,
09:32, 551.19, 552.08, 551.00, 551.05,
09:33, 551.05, 551.07, 549.51, 549.58,
09:34, 549.53, 550.19, 548.63, 548.96,
09:35, 549.05, 549.12, 547.80, 547.94.
.......

15:56, 540.62, 540.66, 540.22, 540.27.
.......

The above data (1-min.) were imported from TradeStation for this experiment.

Hi Alisa and all,

I did some experiment regarding orders for the period of 1/17/2014 to 1/23/2014 (4 trading days).
I set my algo as simple as possible as below.

1). Limit Order

LONG
if price >= 553.00 and notional < context.max_notional and context.long_position_entered == 0 and context.long_position_exited == 0:
context.long_position_entered = 1
order_target_value(context.stock,500000, limit_price=553.00 )

SHORT
elif price <= 552.00 and notional > context.min_notional and context.short_position_entered == 0 and context.short_position_exited==0:
context.short_position_entered = 1
order_target_value(context.stock, -500000, limit_price=551.10)

..............

When limit_price was set to 551.10, it was NOT filled.
Although at 09:31, the close price was 551.20, I think this system does not count the price at 09:31.
Is this right?

When limi_price was set to 551.00, my order got filled at 551.07 (close of 09:32 bar).

2). Market Order
LONG
order_target_value(context.stock,500000)

SHORT
order_target_value(context.stock,-500000)

When orders were set as market order, my order got filled at 09:32 (instead of at 09:31).
I guess here too the price at 09:31 is not counted.
Is this right?

2014-01-17, 09:32:00, AAPL, SELL, -907, $551.07, ($499,819.58)

3). Exit at the end of day
2014-01-17, 15:56:00, AAPL, BUY, 907, $540.28, $490,029.43

I set the time of exit to 15:55 (EST).
As you can see above, the exit order(market) got filled at 15:56.

4). Daily
I did exactly same experiment with daily data. However I got a strange result as below.

The below data were imported from Yahoo finance.
Date, Open, High, Low, Close, Adj Close
Jan 23, 2014 549.94 556.50 544.81 556.18 556.18
Jan 22, 2014 550.91 557.29 547.81 551.51 551.51
Jan 21, 2014 540.99 550.07 540.42 549.07 549.07
Jan 17, 2014 551.48 552.07 539.90 540.67 540.67

Transaction Details showed as below:
2014-01-20 19:00:00 AAPL SELL -924 $548.99 ($507,268.61)

I don't know about this. Can you explain about this?

Thank you.

Hi Anony,

The backtest logic is always that the order gets placed in the first bar and gets filled in the second bar. And the price is always the close price. If you're running a backtest in minute mode, this means the order gets placed in minute 0 and filled at the end of minute 1. If you're running in daily mode, your order is placed on day 0 and filled at the end of day 1.

Daily mode is generally a nice way to see the overall behavior of your algo, but I'd strongly recommend you to further test it in minute mode. This way, you'll get to see more granular details of the market. This is the highest order frequency we offer and when you connect to the broker, IB orders get placed in minute mode, so it would model real world transactions. If you like your algo in minute mode, try paper trading it with Quantopian or IB since its not real money. This way you'll get to see it against current market data.

Alisa

Hey Kyu,

I will try to answer your questions one at a time.

1). I tried the object, cost_basis but log says 0.0. I don't know why I get 0 instead of cost price.

The cost basis represents the volume-weighted average price paid per share in the position. The algo can only place the order in the first bar, and it is not filled until a later bar. Since there are no orders filled in the first bar, there is no cost basis for the first bar. Note that the cost basis for a given bar is not necessarily the same as the price at which an order is filled on a given bar. The cost basis is a cumulative measure that takes into account all shares in the position, not just the order filled in the current or previous bar. See the attached backtest and notice the difference between the log output and Transaction Details in the backtest.

2). I added a stop-loss for short position only as price times 1.005 (meaning if price goes back up by 0.5% of cost).

4). The price (in the logs), 500.69 is different from 500.12 (in the Transaction Details). It had a slippage of 0.57, which I think is hugh.

The computation of the cost basis may be accounting for the differences in price you are seeing in the logs versus the Transaction Details. I only see part of your code listed. Could you post more of it so we can dig further into the investment logic?

Kyu, I haven't gone into the details of it as Ryan has. I did want to make a general comment about our slippage model.

We have a default slippage model. We think it's a pretty good default. It's a little on the conservative side for highly liquid stocks, or stocks with big bid/ask books. By conservative, I mean we probably model a bigger market impact than would be ideal. For thinly traded stocks it probably has the opposite problem, and it probably under-estimates the market impact.

So, if we have this model with some known deficiencies, how do we handle it? By making it modifiable. If you know you are going to only be trading in highly liquid stocks, then go to a lower multiplier in the VolumeShareSlippage. If you know you are going to only be doing small orders in thinly traded stocks, then set a good spread in FixedSlippage. If you're going to be doing a mix, then write yourself a relevant CustomSlippage model. You can make the slippage perform the way you think it should. The first two options are trivially easy; the third one requires more effort.

Our slippage model, like all slippage models, is an estimation. You can never know for sure what would really happen when you place your order in a market.

All that said - if you think we should change our default slippage model, we're always interested in hearing about how to improve our product.

Hey Kyu, I'll give a few line by line details to further illustrate how the cost basis is computed. The first line of the logs is

2014-01-06handle_data:8DEBUG2014-01-06 14:31:00+00:00 cost basis is: 0.0

An order is placed but not filled in the first bar. The second line is:
2014-01-06handle_data:8DEBUG2014-01-06 14:32:00+00:00 cost basis is: 537.29

The first line of the Transaction Details is:
2014-01-06 09:32:00 AAPL BUY 10 $537.29 $5,372.90

Since this order is the only one that was placed, it represents the only assets in the portfolio and the price paid is equal to the cost basis. The third line of the log is 2014-01-06handle_data:8DEBUG2014-01-06 14:33:00+00:00 cost basis is: 536.9, while the second line of the Transaction Details is 2014-01-06 09:33:00 AAPL BUY 10 $536.51 $5,365.10. The cost basis is equal to the volume-weighted average price paid for each share in the position. 536.9 is the average of 537.29 and 536.51.

Hi all,

Thank you for your explanation.
Finally, I understood how Quantopian order generation system works.

Thank you Ryan. Now I understood how to get the cost_basis.
Dan, thank you for explaining slippage models.

So far what I understand is that all the orders start being sent when your logic meets the specific price (either O,H,L,C) at the time of the specific bar (bar 0). Then your orders get filled at the close price of next bar (both for minute bar and daily bar) (bar 1).

One more question:
Regarding daily based trading, for more realistic trades, can we build our strategy based on daily data but run with minute based system? Is this possible? For example, all the computations are based on daily data. So my orders are sent after close of day 0. But I want to buy stocks during the market session of day 1 vs. close price of the next day (day 1). (e.g. I want to buy AAPL at 09:31 on the next day (day1) rather than wait until 16:00 of day 1).

You can never know for sure what would really happen when you place your order in a market.

What I agree AND concern most is exactly what you said as above.

With Quantopian's system, I will have almost no idea what price I will get after sending my orders. Same goes with real market too.
But what I want is to get some realistic idea with Quantopian back testing tool and systematic trading tool.
On one hand, I totally agree with this system that sends orders based on OHLC of bar 0 at time 00:00. But I still want to question why orders get filled at the close price of bar 1 at 01:00. Why not getting filled at open price of right next bar (=bar 1) at 00:10 (or 00:30...)? We never know high, low, close price information of bar 1 until the time goes to 01:00 (or 00:59?).

But we definately know open price of bar 1 right after 00:01. And also this open price is very close to the close price of previous bar. So why don't Quantopian systems work like this? We may get totally odd price due to this order fill system OR we may get some advantage from this system. The thing is that we never know what is going to happen.

I understand that Quantopian and IB use 1-minute base data for now. And Alisa said that handle_data is called every 1 minute. If I may say, why don't you change it to 30-second calling system. The reason why I said so is that although we all know that slippage models and order filling models are assumption and estimation, we basically need to get some pictures of our logics in our algos as close as possible.

If you and all the members of this forum think it is reasonable, my suggestion is that Quantopian give us two options: One is that orders get filled at open price of bar 1 at time 00:30 (30 seconds after bar 0). Second is the current system. So we can choose either of two options. For more realistic results, we can set slippage model that Quantopian built, which is totally depending on our choice.

I am learning Python and Quantopian everyday. Also I am excited about experience here with Quantopian. I'm sure I'm going to stick with Quantopian for long time. And I will have a lot more questions too.
Hope Quantopian systems support us with much more and better features than any others. (I know you're already doing great great job.).

Thank you again.

Regarding daily based trading, for more realistic trades, can we build our strategy based on daily data but run with minute based system? Is this possible?

This is most certainly possible. Of course, you have to run the simulation at a minute level. But you can use the data any way you like, accumulating daily bars of data, deriving your signal, and then acting on that signal as you see fit - minutely, hourly, daily, whatever. It's all possible within the code. We have a lot of convenience functions and shared code to help with this.

Our backtester is aimed at providing a good test, a good guideline of understanding how your algorithm will work in the real market. Whether it's 1-minute bars, or 30-second pars, I don't think will matter too much. One of the best things you can do is get an idea that works for you in our backtester, then do forward testing with live market data. That's an incredibly powerful tool that completely removes the question of open v. close prices for fill times. Your order is going to the broker, exactly like a real order, and you're getting your performance information immediately.

I'm very glad you're enjoying it.