A couple of thoughts...
First, are you aware that Q doesn't support brokerage integrations anymore? Not sure how you planned on trading the algorithm yourself? See https://www.quantopian.com/posts/phasing-out-brokerage-integrations . You could consider trading with Robinhood and pay NO commissions. Problem solved.
That said, your observation on trading cost is valid. Commissions can quickly turn a great algorithm into a less than stellar one. It's often very illuminating to test an algo with, and then without, commissions to get direction on what parts of an algo to focus on. Moreover, it's also just as illuminating to see how an algo performs with different capital levels using commissions
By the fact the algo performs well with larger capital implies the per share costs aren't the problem (they scale equally across capital levels). The minimum $1 per trade is the culprit (which disproportionately impacts smaller volume trades) .
A few things to look at then to keep the $1/trade from kicking in:
- Try to trade at least 200 shares at a time (under that level the $1/trade minimum kicks in). Maybe try simply putting in a condition if trade qty < 200 don't do it. If your algo makes a lot of small trades over a short time, can those be aggregated?
- Try to trade in lower costs stocks. Use a filter maybe like if price > $40 don't do it. 200 shares of AAPL would cost around $35k and might be half one portfolio. 200 shares of Comcast would only be $7000.
- Limit the number of holdings. Rather than holding perhaps 50 stocks, limit that number to 10 or 20. This increases the number of shares of each stock. Often one doesn't see appreciable diversification benefit above 20 holdings anyway.
- Trade all or none of ones holdings. Does the algorithm make small adjustments by trading only a portion of ones holdings? Maybe modify the algorithm to trade all or none. If it makes sense to sell half ones shares maybe it make sense to sell it all.
Just some, ideas.