Let f be a diffeomorphism of a compact $C^{infty}$ manifold, and let p be a hyperbolic periodic point of f. In this paper we introduce the notion of $C^1$-stable inverse shadowing for a closed f-invariant set, and prove that (i) the chain recurrent set $cal{R}(f)$ of f has $C^1$-stable inverse shadowing property if and only if f satisfies both Axiom A and no-cycle condition, (ii) $C^1$-generically, the chain component $C_f(p)$ of f associated to p is hyperbolic if and only if $C_f(p)$ has the $C^1$-stable inverse shadowing property.