In this paper, we describe the modeling for the 3D robot laser scanning system consisting of a laser stripe projector, camera, and 5-DOF robot and propose its calibration method. Nonlinear radial distortion in the camera model is considered for improving the calibration accuracy. The 3D range data is calculated using the optical triangulation principle which uses the geometrical relationship between the camera and the laser stripe plane. For optimal estimation of the system model parameters, real-coded genetic algorithm is applied in the calibration process. Experimental results show that the constructed system is able to measure the 3D position within about 1mm error. The proposed scheme could be applied to the kinematically dissimilar robot system without losing the generality and has a potential for recognition for the unknown environment.