The high dielectric constant of insulators currently investigated as alternatives to SiO$_2$ in metal-oxide-semiconductor structures is due to their large ionic polarizability. This is usually accompanied by the presence of soft potical phonons. We show that the long-range dipole field associated with the interface excitations resulting from small in the case of SiO$_2$ for most high-k materials causes a reduction of the effective electron mobility in the inversion layer of the Si substrate. We study the dispersion of the interfacial coupled phonon-plasmon modes, their electron-scattering strength, and their effect on the electron mobility for Si-gate structures employing films of SiO$_2$, Al$_2$O$_3$, AIN, ZrO$_2$, HfO$_2$ and ZrSiO$_4$ for ″SiO$_2$-equivalent″ thickness ranging from 5 to 0.5nm