In order to evaluate the small signal stability of power system, eigenvalue analysis is an important method to implement. However, there are some disadvantages for example it cannot analyze in on-line condition and is effective for calculating in limited operation conditions. This paper aims to investigate the performance of online oscillation mode estimation techniques for large power systems. Autoregressive and moving average (ARMA model) with least-square and kalman filter algorithms are employed and compared. The performance of both algorithms is tested in term of calculation time and mean square error. The results present ARMA model with both algorithms can provide the satisfied performance. However, ARMA model with kalman filter algorithm has more advantages such as faster calculation time and more accurate calculation results. Thus, this algorithm can be utilized in online oscillation mode estimation and can be used in design the control system for enhancing the damping in power system.