I created the following notebook to get a feel for how the violation of the t-test independence assumption impacted the resulting p-values of the test. I created two simulated AR(1) time series (which you could imagine as two factor return streams). My thought is that the auto-correlation would violate the independence assumption. I then performed a paired 1-sample randomization test and a paired 1-sample t-test (see the notebook for my sources of information on the topic, as I was not really familiar with randomization tests until yesterday).
My thought going into the experiment was that the t-test would overstate the confidence (understate the p-value). However, it appears that both tests generate roughly the same p-value on average.
Any thoughts from the community?
