Interpretability and efficiency in hypothesis tests

Abstract

Hypothesis testing is a very common and widespread statistical tool. Unfortunately, such methodology still presents several challenges to statisticians. This project aims at developing hypothesis tests by filling several existing gaps.More precisely, the follows issues will be addressed: (1) Agnostic Tests. There is a disagreement about the interpretation of results from a hypothesis test: while some understand that a hypothesis test is able to either reject or accept the null hypothesis $H_0$, others believe its outcomes should be interpreted as either reject or not reject $H_0$. This often lead practitioners to have difficulties in understanding the conclusions from significance tests. In particular, the second (and most common) perspective is deeply linked to the development of non-inferiority tests used in clinical trials. Here, we propose an alternative formulation to hypothesis tests in which, besidesthe decisions “accept $H_0$“ and “reject $H_0$“, there is a third decision, namely the “no conclusion“ decision, which we call the agnostic decision. (2) Bayesian Nonparametric Tests. Because of the large volume of data available today in several applications, nonparametric methods have been gaining a lot of attention as they allow one to make less assumption about the data generating process. Unfortunately, there is almost no literature on Bayesian nonparametric tests, even though the Bayesian paradigm is widespread today. Here, we investigate new tests that try to overcome such gap. In particular, we investigate Bayesian nonparametric two-sample tests. (3) FBST in High Dimensions. Another challenge that exits in several applications is the issue of high dimensionality: in many problems, the number of covariates is very large; many times larger than the sample size. This makes sevaral standard methods fail. In particular, it has been observed that the Full Bayesian Significance Test has difficulties dealing with such situation. We will propose improvements in such method so that it is able to overcome the issue of high dimensionality, and we will investigate their theoretical properties. As a part of this project, we will also develop R packages that implementthe methods developed here. (AU)