In this paper, an algorithm is presented to recognize lane and obstacles based on highway road image. The road images obtained by a video camera undergoes a pre-processing that includes filtering, edge detection, and identification of lanes. After this pre-processing, a part of image is grouped into 27 sub-windows and fed into a three-layer feed-forward neural network. The neural network is trained to indicate the road direction and the presence of absence of an obstacle. The proposed algorithm has been tested with the images different from the training images, and demonstrated its efficacy for recognizing lane and obstacles. Based on the test results, it can be said that the algorithm successfully combines the traditional image processing and the neural network principles towards a simpler and more efficient driver warning of assistance system