This paper proposes a plate number extraction method which uses the information of both the colors and the arrangement of numbers in a vehicle image with complex background. In a number plate, color of the numbers is usually black or white, and numbers are arranged in a row. At first, a raw image is partitioned into the plate number candidate regions and non-interest region. The number candidate regions are thresholded in mean binarization. After eliminating the illegal candidate regions using the aspect ratio of the plate number, the plate number region is finally extracted by using the arrangement information among the numbers. To evaluate the proposed mothed, 292 images are taken in various places and at different times. The experimental results show that the rate of the proposed number regions extraction is about 89.8%, 95.5% for the plate of green and white backgrounds, respectively.