Is there a way to loop through a large list of variables in order to add columns to a pipeline processed dataframe?
In other words, I'd like to avoid adding 1540 financial statement items manually with lines like
Acctg_pipe.add(balance_sheet.accounts_payable_as_of.latest, 'accounts_payable_as_of')
Acctg_pipe.add(balance_sheet.accounts_receivable_as_of.latest, 'accounts_receivable_as_of')
Acctg_pipe.add(balance_sheet.accrued_interest_receivable_as_of.latest, 'accrued_interest_receivable_as_of')
Is there some way to feed the following numpy arrays into a custom function to achieve the same outcome?
CFS_items = np.array(dir(morningstar.balance_sheet))
IS_items = np.array(dir(morningstar.income_statement))
BS_items = np.array(dir(morningstar.cash_flow_statement))
I've tried writing custom functions with python's (*) splat operator; e.g.
function(*CFS_items). But, I get an error with everything I've tried so far.
Another thing I tried was to create lists of strings containing python code that I was planning to execute with eval() or exec(), but the system won't let me use these functions due to security concerns.
'''
creates list of strings containing python code
'''
BS = morningstar.balance_sheet
IS = morningstar.income_statement
CFS = morningstar.cash_flow_statement
CFS_items = np.array(dir(CFS))
IS_items = np.array(dir(IS))
BS_items = np.array(dir(BS))
# retrieve list of accounting items by financial statement
BS_asof_idx = np.array([item.endswith('_as_of') for item in BS_items])
IS_asof_idx = np.array([item.endswith('_as_of') for item in IS_items])
CFS_asof_idx = np.array([item.endswith('_as_of') for item in CFS_items])
# retrieve financial statement items "as of" dates
BS_asof_items = BS_items[BS_asofDates]
IS_asof_items = IS_items[IS_asofDates]
CFS_asof_items = CFS_items[CFS_asofDates]
BS_f_items = [] # financial statement items
IS_f_items = []
CFS_f_items = []
BS_asof_pipes = [] # pipelines to create dataframe columns of as_of dates
IS_asof_pipes = []
CFS_asof_pipes = []
BS_f_pipes = [] # pipelines to create dataframe columns of financial statement items
IS_f_pipes = [] # corresponding to as_of date pipeline above
CFS_f_pipes = []
for idx, item in enumerate(BS_asof_items):
BS_f_items.append(item.replace('_as_of',""))
BS_asof_pipes.append(''.join(['pipe.add(BS.', item,".latest, '", item,"')" ]))
BS_f_pipes.append(''.join(['pipe.add(BS.', BS_f_items[idx],".latest, '", BS_f_items[idx],"')" ]))
for idx, item in enumerate(IS_asof_items):
IS_f_items.append(item.replace('_as_of',""))
IS_asof_pipes.append(''.join(['pipe.add(IS.', item,".latest, '", item,"')" ]))
IS_f_pipes.append(''.join(['pipe.add(IS.', IS_f_items[idx],".latest, '", IS_f_items[idx],"')" ]))
for idx, item in enumerate(CFS_asof_items):
CFS_f_items.append(item.replace('_as_of',""))
CFS_asof_pipes.append(''.join(['pipe.add(CFS.', item,".latest, '", item,"')" ]))
CFS_f_pipes.append(''.join(['pipe.add(CFS.', CFS_f_items[idx],".latest, '", CFS_f_items[idx],"')" ]))
Sample output of one of the lists created above:
["pipe.add(BS.accounts_payable_as_of.latest, 'accounts_payable_as_of')",
"pipe.add(BS.accounts_receivable_as_of.latest, 'accounts_receivable_as_of')",
"pipe.add(BS.accrued_interest_receivable_as_of.latest, 'accrued_interest_receivable_as_of')",
"pipe.add(BS.accrued_investment_income_as_of.latest, 'accrued_investment_income_as_of')"]