A semi-active mass damping (SMD) system with magnetorheological (MR) dampers focusing on low- and mid-rise buildings is proposed in this paper. The main purpose of this study is to integrate the reliable characteristics of the traditional tuned mass damper (TMD) and the superior performance of the active mass damper (AMD) to the new system. In addition, the commonly seen solution of deploying dense seismic dampers throughout the structure nowadays to protect the main structure is also expected to switch to the developed SMD system on the roof with a similar reduction performance. In order to demonstrate this concept, a full-size three-story steel building representing a typical mid-rise building was used as the benchmark structure to verify its performance in real life. A numerical model with the interpolation technique integrated was first established to accurately predict the behavior of the MR dampers. The success of the method was proven through a performance test of the designated MR damper used in this research. With the support of the MR damper model, a specific control algorithm using a continuous-optimal control concept was then developed to protect the main structure while the response of the semi-active mass damper is discarded. The theoretical analysis and the experimental verification from a shaking table test both demonstrated the superior mitigation ability of the method. The proposed SMD system has been demonstrated to be readily implemented in practice.