To determine whether each coefficient is equal to zero or not, usual $t$-tests are a popular choice (among others) in linear regression to practitioners because all statistical packages provide the statistics and their corresponding $p$-values. Under smaller samples (especially with non-normal errors) the tests often fail to correctly detect statistical significance. We propose a permutation approach by adopting a sufficient dimension reduction methodology to overcome this deficit. Numerical studies confirm that the proposed method has potential advantages over the t-tests. In addition, data analysis is also presented.