Nonlinear behavior analysis was used to verify whether a one-way clutch is effective for reducing the torsional vibration of a paired spur gear system under periodic excitation. The dynamic responses were studied over a wide frequency range by speed sweeping to check the nonlinear behavior using numerical integration. The gear system with a one-way clutch showed typical nonlinear behavior. The oscillating component of the dynamic transmission error was reduced over the entire frequency range compared to a system without a one-way clutch. The one-way clutch also eliminated unsteady continuous jump phenomena over multiple solution bands, and prevented double-side contact, even with very small backlash. Installing a one-way clutch on both sides of the gear system was more effective at mitigating the negative effects of external periodic excitation and various parameter changes than a conventional gear system without a one-way clutch.