In this paper, a new technique is proposed for obtaining the error probabilities of the VSB(vestigial sideband modulation) signal in the presence of the cochannel interference and frequency-selective fading channel. For the receivers, a suboptimal matched filter receiver and the MLSE(maximum likelihood sequence estimation) receiver, which is known to be optimal on the fading channel, are considered. First, for the matched filter receiver, the distributions of the random variables, which determine the SER(symbol error rate) are obtained by decomposing the multi-path fading channel into Rayleigh distributed main path and Gaussian distributed remained path channels. the random variables mean the energy of the main path and subpath respecitively, and SER can be calculated from the distribution of them. Next, for the case of the MLSE receover, it is found that the random variables are expressed as a function of integrals. In order to obtain the distribution for the random variables, we expanded each element of integrals with the KL(Karhunen-Loeve) transformation. And it is derived that the distributions for the transformed random variables are given by a sum of chi-square distributions. Finally, we calculated the error rate derived formula on the two-ray fading channel, which is one of widely used models for the frequency-selective fading channel. From the numerical results, it is found that for the matched filer receiver, performance degradation is significant, while the performance degradation at the MLSE receiver is insignificant on the frequency-selective fading channel. However, in case of cochannel interference environment, the error rateis found to increase significantly both at the matched filter and at the MLSE receiver.