We present an eye tracking system using a low cost 3D CMOS imager for 3D displays that ensures a correct auto stereoscopic view of position- dependent stereoscopic 3D images. The tracker is capable of segmenting the foreground objects (viewer) from background objects using their relative distance from the camera. The tracker is a novel 3D CMOS Image Sensor based on Time of Flight (TOF) principle using innovating photon gating techniques. The basic feature incorporates real time depth imaging by capturing the shape of a light-pulse front as it is reflected from a three dimensional object. The basic architecture and main building blocks of a real time depth CMOS pixel are described. For this application, we use a stereoscopic type of display using parallax barrier elements that is described as well.