The H.264/ AVC video coding standard supports the intra prediction with various block sizes for luma component and a 8x8 block size for chroma components. This new feature of H.264/AVC offers a considerably higher improvement in coding efficiency compared to previous compression standards. In order to achieve this, H.264/AVC uses the Rate-distortion optimization (RDO) technique to select the best intra prediction mode for each block size, and it brings about the drastic increase of the computation complexity of H.264 encoder. In this paper, a fast block size decision algorithm is proposed to reduce the computation complexity of the intra prediction in H.264/AVC. The proposed algorithm computes the smoothness based on AC and DC coefficient energy for macroblocks and compares with the nonparametric criteria which is determined by considering information on neighbor blocks already reconstructed, so that deciding the best probable block size for the intra prediction. Also, the use of non-parametric criteria makes the performance of intra-coding not be dependent on types of video sequences. The experimental results show that the proposed algorithm is able to reduce up to 30% of the whole encoding time with a negligible loss in PSNR and bitrates and provides the stable performance regardless types of video sequences.