This study was conducted to develop the vision system of a robotic transplanter for plug-seedling. A color image processing algorithm was developed to identify and locate empty cells and bad plants in the seedling tray. The image of pepper and tomato seedling tray was segmented into regions of plants, frame and soil using threshold technique which utilized Q of YIQ for finding leaves and H of HSI for finding frame of tray in the color coordinate system. The recognition system was able to successfully identify empty cells and bad seeding and locate their two-dimensional locations. The overall success rate of the recognition system was about 99%.