I think this might be a bug:

line 103:
(tmpCompareData[x].returns < tmpCompareData[y].returns)

"returns" is a function. I am not sure why you need to compare the function handles. If you meant to compare the return values, you probably should change the line to:

(tmpCompareData[x].returns() < tmpCompareData[y].returns())

@Henry, I agree that is a bug (and fairly easy one to make since no error is raised).

Incidentally fixing it increases the returns!

Thanks, that is a bug. What I was trying to do is sort stocks in portfolio before picking one to trade that day. I could not find a way to send the data[x] to the python sort function. Initial version of the code used random x in stocks. Do you know how could one sort securities in data array by some criteria, say by highest 6 day standard deviation?

Saving data[x] into tmp variable above does not work all that well. One can access the .something but not call functions as x.something(), at least I got runtime error when trying to do that. In Perl I would figure it out in no time, but needless to say this is my first attempt at Python, a fact made painfully obvious when reading the code.

@Vlatko, the runtime error was probably because you have turned off date range checking (not all the listed securities have overlapping dates). So the security is not present in every data[] event. Look at my modified backtest (above) for an easy way to check if the security is in the data[] object.

if not x in data:  
    return 0  

Funny, I wan under impression once could not call functions once the data[] is moved into tmp variable. I was wrong. Thanks for spotting and fixing it guys. Yes, the idea was to sort and pick the stock(s) with highest potential to continue performing. Glad it got improved.

@vlatko I just wrote the following codes to sort the stocks by return. You can use them to replace your lines 100-107. I got the same performance results as Dennis did.

Btw it seems this strategy performs quite well!

def handle_data(context, data):

    sid_returns = [(sid_, data[sid_].returns())  
                   for sid_ in context.etfs + context.stocks  
                   if sid_ in data]  
    sid_returns = sorted(sid_returns,  
                         key = lambda(pair) : pair[1],  
                         reverse=True)  
    for x, _ in sid_returns:  

Thank you both guys. As expected, both fixes give the very same, improved result. I have attached the back test of Henry's code change, confirming result Dennis showed above.

The indicatorTrendPct computation can be moved outside of the for loop to avoid computing the same thing multiple times.

I'm not sure if it was intentional with the return statement after the buy order. By doing this you might also be skipping sell orders for SIDs in the for loop that come after the one that had a buy order. For example the first SID might have a buy order sent. SIDs after that may have reached the maximum holding period or trendPct limit but no sell order will be sent because return had already been called in handle_data.

The return statement was there to arbitrarily slow down the trading. It can be removed, probably should be by now. It's not a bug but a feature, albeit a very stupid feature that I should get rid of.

Now that I think about it, there is a reason for such seemingly arbitrary return. The algorithm was supposed to stay all long and borrow no money. Once I put a buy order in, it takes a while for it to get processed so if I just continue to next security it may result in multiple buys sending cash into negative territory. There is a better way to do it, which I should implement.

Hey so I had a question about sell volume being at 9000 vs the buy volume being at 5000. Not sure why you're sale volume is higher; I assumed it was to clear you completely out of a position but I changed the volume back to 5000 and got some interesting results but I am not sure if they are of any value.

Thanks for the improvement, it is quite a nice one!

The idea was to sell quickly in the downtrend. Looks like what the strategy does is amplifies the good times, it also amplifies the bad times. Overall it comes on top only because there are more good days/years than bad ones. I was originally trying to keep the winning days while eliminating as much as i could the losing streaks. Every tweak points to algorithm being quite capable of exploiting the upside, yet offering no protection in downtrends.

I wish I knew how to add breaks when the cart is going downhill.

Trying to use set_universe() instead of a predetermined set of stocks. It works for a day or two of back tests, longer intervals cause a runtime error. Perhaps there is a way to avoid the bad data when processing.

        # This check was not enough to prevent runtime error  
        if not x or not x in data or not data[x]:  
            continue  

What is the runtime error? I don't get it when I clone your algo.

Also you should probably remove all the sids except context.indicator = sid(8554) or that sort of defeats the purpose of using set_universe.

There was a runtime error. See the more detailed error below. If you need help, please send us feedback.  
OrderNonSecurity:  
File test_algorithm_sycheck.py:243, in handle_data  
File algoproxy.py:1191, in guarded_order  

I ran into a problem in another algo where the usual checks for "if x in data" weren't enough.

In the end I had to change the way I passed the sid to order()

order( data[x]['sid'], amount )  

For some reason that worked when other things failed.

That indeed worked. Thanks!

Now that I am using set_universe instead of pre-picked set of stocks a problem arises. Algorithm is designed to pick quickly rising stocks in order to go along for the ride, a momentum trade. That works reasonably well for large cap socks, but not for highly volatile stocks of .com style bubbles and small cap stocks. If the stocks are picked form a very large set, one would wish to pick those that have a high market capitalization before turning this algorithm on, otherwise it will tank trying to keep up the super high volatility stocks - whiplash pattern.

Just some ideas ...

Since it's a momentum strategy you may want to use a stop-loss. Or maybe model the volatility and adjust your order size to reduce your risk.

Another option would be to model the volatility of each stock and either trade it as a momentum stock or a reversion stock.

Both points make sense. The 60 day trading interval is something I need to observe for reasons unrelated to this discussion, thus the stop loss is not a good option. I would like to use volatility as you suggested. Do you have an idea at which level the volatility indicated that reversal to the mean is more likely to be profitable than trend following prediction?

In the next iteration I am relying on get_open_orders() to tell me of any orders that are out still open so I do not order too many securities. The question is, how long does the order keep open if it's not fulfilled? In the test run, it creates one order that just stays in the active list never going away.

# Order object returned by get_open_orders():  
{20061: [Event({
    'status': 0,  
    'created': datetime.datetime(2002, 12, 12, 0, 0, tzinfo=<UTC>),  
    'limit_reached': False,  
    'stop': None,  
    'stop_reached': False,  
    'amount': 15,  
    'limit': None,  
    'sid': 20061,  
    'dt': datetime.datetime(2002, 12, 12, 0, 0, tzinfo=<UTC>),  
    'id': '47d2f10b113c46fea125f20e604bb164',  
    'filled': 0  
    })]}  

Filled is 0, presumably this order is not filled. But it should not stay active forever. It is an open order, thus it should be filled or rejected/expired.

On most exchanges all orders expire at the end of the day, to the best of my knowledge. If one wants to have a multiple day order, broker simply re-enters the same order before the market opens the next day. As far as the exchange is concerned, the order entered on one day is fulfilled, rejected or expired on that same day. I may be wrong about this but that was my expectation. If indeed an order can stay open over multiple days, I should adjust my algorithm to cancel orders not fulfilled, so they do not linger for days or years.

It may be a good idea to describe the order object in the API, so one does not need to print it out in a debug statement to get the keys in the dictionary. Also, status and filled are presumably enumerations, worth explaining if someone wishes to use them in a meaningful manner.