A new parameter adaptation scheme for RBF Neural Network (NN) has been developed in this paper. Even though the RBF Neural Network (NN) based controllers are robust against both un-modeled dynamics and external disturbances, the performance is not satisfactory for a fast and precise mobile robot. To improve the tracking performance as well as robustness, all the parameters of RBF NN are updated in real time. The stability of this control law is rigorously proved by following the Lyapunov stability theory and shown by the experimental simulations. The fact that all of the weighting factors, width and center of RBF NN have been updated implies that this scheme utilizes all the possibilities in RBF NN to make the controller robust and precise while the mobile robot is following un-known trajectories. The performance of this new algorithm has been compared to the conventional RBF NN controller where some of the parameters are adjusted for robustness.