To improve the limitation of fuzzy PI controller when is applied to systems of order higher than one, a fuzzy PI controller that fuzzily resets or amplifies the accumulated control input according to fuzzy rules defined on (error, change of error) space is proposed. The proposed controller structure was motivated by the characteristics of fuzzy PI controller that it generally gives unevitable large overshoot in trial of reducing rise time of response especially when a system of order higher than one is considered. Based on the observation that the undesirable characteristics of the fuzzy PI controller is caused by integrating control input excessively, even though the integrator is introduced to overcome steady state error, we propose a controller that clear out or doubles integrated control input in a fuzzy manner according to the situation to reduce rise time as well as overshoot. To show the usefulness of the proposed controller, it is applied to the systems that are difficult to stabilize or difficult to get satisfactory response by conventional fuzzy PI controllers.