In this paper, an algorithm is presented to recognize road based on unpaved test courses image. The road images obtained by a video camera undergoes a pre-processing that includes filtering, gray level slicing, masking and identification of unpaved test courses. After this pre-processing, a part of image is grouped into 27 sub-windows and fed into a three-layer feed-forward neural network. The neural network is trained to indicate the road direction. The proposed algorithm has been tested with the images different from the training images, and demonstrated its efficacy for recognizing unpaved road. Based on the test results, it can be said that the algorithm successfully combines the traditional image processing and the neural network principles towards a simpler and more efficient driver warning or assistance system.