Ultrasonic motors are newly developed motors which are expected to be useful as actuators in many practical systems such as robot arms or manipulators because of several advantages against the electromagnetic motors. However, the precise control of the ultrasonic motor is generally difficult due to the absence of appropriate and rigorous mathematical model. Furthermore, owing to heavy nonlinearity, the position control of a pendulum system driven by the ultrasonic motor has a problem that control method using multiple neural network controllers based on a fuzzy inference system that can determine the initial position of the pendulum in the beginning of control operation. In addition, and appropriate neural network controller that has been learned to operate well at the corresponding initial position is adopted by switching schemes. The effectiveness of the proposed method was verified and evaluated from real experiments.