A phase different method to evaluate the instrument error of roundness measuring instrument and the form error of specimens for the calibration of the instrument is used. An instrument with a rotary table supported by an air bearing was calibrated by using the standard balls as a standard. The calibration was carried out repeatedly by setting the same ball in 12 phase angles(per 30.deg.) on the table and by recording their roundness errors with a magnification of 100,000 times. As a result of data analysis of all the observations, readout at each of 144 orientations(per 2.5.deg.) from recorded data file, the error of performance of the instrument and the specimens are separated. In the particular instrument used in the present experiment, the error of the instrument was determined with the accuracy of 0.0164 (.mu.m) and the form error of the specimens was determined with the accuracy of 0.0264,0.0172(.mu.m), respectively. If the instrument was calibrated by using the above specimens, then the accuracy of the measurement of roundness error can be improved to about 0.017 (.mu.m).