Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
Ability to pull all columns in a dataset into a pipineline

Hi Team,

been trying to find a way to systematically return all the columns in a dataset into my pipe to do some exploratory analysis.

I tried a few solutions found on Stack overflow which suggest using getattr and other builtin functions but these are all protected functions which you can't use in quantopian.
Main idea being to store the column names as a list and , as they are also the same name as the call, iterate through the list to create a list of codes. This would replace twt.code in the below:

columns = [ "twt." + i for i in twt.fields]  

https://stackoverflow.com/questions/26421734/call-functions-based-on-their-names-as-a-strings-in-a-list

If we take the tweets database for instance, i'd like to be able to write something that says, in pipeline:

from quantopian.interactive.data.psychsignal import aggregated_twitter_withretweets_stocktwits as twt  
mydict={}

for i,v in zip(twt.fields, twt.codes): #can swap twt.codes for columns lists define above.  
        mydict[i]=v

Pipeline(  
        columns=mydict,  
        screen=QTradableStocksUS()  
        ),  
        start_date='2019-01-04',  
        end_date='2019-01-10')  
         )  

would someone know of a way using the infrastructure on quantopian, found a similar thread here https://www.quantopian.com/posts/pipeline-help but the can't seem to get past the following error with the code:

pipp.add(twt.source,"source")
"zipline.pipeline.pipeline.validate_column() expected a value of type zipline.pipeline.term.Term for argument 'term', but got str instead."

pipp.add(twt.source(),"source") yields :
TypeErrorTraceback (most recent call last)
in ()
----> 1 pipp.add(twt.source(),"source")

TypeError: 'Field' object is not callable

Any help would be appreciated,

let me know if i've missed any helpful information.

best,