The correlation between black hole mass and stellar velocity dispersion provides an important clue on the black hole growth and galaxy evolution. In the case of AGN, however, it is extremely difficult to measure stellar velocity dispersions in the optical since AGN continuum dilutes stellar absorption features. In contrast, stellar velocity dispersions of active galaxies can be measured in the near-IR, where AGN-to-star flux ratio is much smaller. Expecting that more stellar velocity dispersion measurements will be available using future near-IR facilities, it is crucial to test whether the stellar velocity dispersions measured from the near-IR spectra are consistent with those measured from the optical spectra. For a sample of 35 nearby galaxies, for which optical stellar velocity dispersion measurements and dynamical black hole masses are available, we obtained high quality H-band spectra, using the TripleSpec at the Palomar 5-m Telescope, in order to calibrate the stellar velocity dispersions and define the $M_{BH}-sigma_*$ relation in the near-IR. Based on the spatially resolved kinematics, we correct for the rotation component and determine the luminosity-weighted stellar velocity dispersion of the spheroid component in each galaxy. In this presentation, we will show the comparison between optical and near-IR stellar velocity dispersion measurements and define the $M_{BH}-sigma_*$ relation based on uniformly measured stellar velocity dispersion in the near-IR.