This paper presents a phase-only modulation scheme for a three-dimensional (3-D) image matching system to improve optical efficiency of the system. The 3-D image matching system is based on the two mask heterodyne scanning. A hologram of the 3-D reference object is first created and then the phase of the hologram is extracted. The phase of the hologram is represented as one mask with the other mask being a plane wave. The superposition of each beam modulated by the two masks generated a scanning beam pattern. This beam pattern scans the 3-D target object to be recognized. The output of the scanning system gives out the correlation of the phase-only hologram of the reference object and the complex hologram of the target object. Since a hologram contains 3-D information of an object as a form of fringe pattern, the correlation of holograms matches whole 3-D aspect of the objects. Computer simulations are performed with additive gaussian noise and without noise for the complex hologram modulation scheme and the phase-only hologram modulation scheme. The computer simulation results show that the phase-only hologram modulation scheme improves the optical efficiency. Thus the system with the phase-only hologram modulation scheme is more robust than the system with the complex hologram modulation scheme.