Recently demands for the speech technology-based products targeted for the mobile terminals such as cellular phones and PDA are rapidly increasing. And voting-based speaker identification algorithm is known to have a good performance in the mobile environment, since it works well with small amount of speaker training data. In this paper, we proposed a method to improve the performance of this voting based speaker identification system by using the observation confidence value which is derived from the function of SNR each frame. The proposed method is evaluated with ETRI cellular phone DB which is made for the speaker recognition task. The experimental results show that the proposed method has better performance of 2-3% identification rate than the conventional GMM method.