In this paper, a novel vision-based nighttime vehicle detection approach is presented, combining state machines and downhill simplex optimization. In the proposed approach, vehicle detection is modeled as a sequential state transition problem; that is, vehicle arrival, moving, and departure at a chosen detection area. More specifically, the number of bright pixels and their differences, in a chosen area of interest, are calculated and fed into the proposed state machine to detect vehicles. After a vehicle is detected, the location of the headlights is determined using the downhill simplex method. In the proposed optimization process, various headlights were evaluated for possible headlight positions on the detected vehicles; allowing for an optimal headlight position to be located. Simulation results were provided to show the robustness of the proposed approach for nighttime vehicle and headlight detection.