The noise and vibration of a rotary compressor, a type of multi-input, single output system, are generally studied through frequency analysis. Although this method is effective in analyzing frequency components, using this method to identify the specific source of the noise (4 kHz to 6 kHz) is difficult. Hence, noise source should be studied systematically. In this study, a coherence analysis method based on systems analysis is used to identify the compressor noise source. Compressor noise source is identified through the coherence between the vibration signals on the shell of the compressor and the noise signal at one point near the compressor (1 m away from the compressor). A one-third octave band is employed for frequency analysis. The design of experiment is conducted to identify possible noise factors, such as volume, size, and neck area of the resonator in the compressor cylinder. Analysis showed that noise was generated from the cavity of the cylinder and the muffler inside the rotary compressor. A new type of muffler was applied to the rotary compressor to verify this finding. Noise was dramatically reduced.