A new control method far vision-based autonomous vehicle is proposed to determine navigation direction by analyzing lane information from a camera and to navigate a vehicle. In this paper, characteristic featured data points are extracted from lane images using a lane recognition algorithm. Then the vehicle is controlled using new Levenberg-Marquardt neural network algorithm. To verify the usefulness of the algorithm, another algorithm, which utilizes the geometric relation of a camera and vehicle, is introduced. The second one involves transformation from an image coordinate to a vehicle coordinate, then steering is determined from Ackermann angle. The steering scheme using Ackermann angle is heavily depends on the correct geometric data of a vehicle and a camera. Meanwhile, the proposed neural network algorithm does not need geometric relations and it depends on the driving style of human driver. The proposed method is superior than other referenced neural network algorithms such as conjugate gradient method or gradient decent one in autonomous lateral control .