hi Grant,
in the unlikely event (but quite likely esp for fundamental data) that there is an outlier, say outlier=70000 (unit), and the true mean is 1(unit)..... "a = np.nan_to_num((a-np.nanmean(a))) " might result in np.nanmean(a) =say 25 (unit) as opposed to 1(unit, being the true mean), then "a-25" pushes most of the values to say -24 (unit). This is fine so far (as we will scale things again at the end), however, we then replace NaN with 0 by method of np.nan_to_num, so the NaN names become good alpha names (right of the distribution). These NaN names are more likely than not going to stay as good alpha numbers in the remaining of the preprocessing, despite the subsequent winzorization and final scaling.
i guess ideally the winsorization should be done before the first normalization, so that the first normalised numbers are more likely centred around zero, then it is fine to assign zero to NaN names. Do note, that scipy's winsorize method doesn't work well with NaNs in the array, so your current method of replacing NaN with zero is right, in a sense that it ensures the next line "winsorize" will work.
actually the best way to do the whole processing is to ignore NaN, then winsorize, then normalize, then throw the NaN back into the distribution as zero.
