There are many situations in which the well-known tests such as log-rank test and Gehan-Wilcoxon test fail to detect the survival differences. Assuming large samples, these tests are developed asymptotically normal properties. Thus, they shall be called asymptotic tests in this paper, Several asymptotic tests sensitive to some specific types of survival differences have been recently proposed. This paper compares by simulations the test levels and the powers of the conventional asymptotic tests and their random permutation versions. Simulation studies show that the random permutation tests possess competitive powers compared to the corresponding asymptotic tests, keeping exact test levels even in the small sample case. It also provides the guidelines for choosing the valid and most powerful test under the given situation.