In this research, an internal model based method is proposed to estimate the structural displacements and velocities under ambient excitation using only acceleration measurements. The structural response is assumed to be within the linear range. The excitation is assumed to be with zero mean and relatively broad bandwidth such that at least one of the fundamental modes of the structure is excited and dominates in the response. Using the structural modal parameters and partial knowledge of the bandwidth of the excitation, the internal models of the structure and the excitation can be respectively established, which can be used to form an autonomous state-space representation of the system. It is shown that structural displacements, velocities, and accelerations are the states of such a system, and it is fully observable when the measured output contains structural accelerations only. Reliable estimates of structural displacements and velocities are obtained using the standard Kalman filtering technique. The effectiveness and robustness of the proposed method has been demonstrated and evaluated via numerical simulations on an eight-story lumped mass model and experimental data of a three-story frame excited by the ground accelerations of actual earthquake records.