Partially doped Aurivillius phases SrBi2N$b_{2-x}M_xO_9$ (M=Cr and Mo) were successfully synthesized and characterized. The extent of the substitution was limited at ~20 mole % because of the size differences between $Nb^{5+}$ and $Cr^{6+}$, and between $Nb^{5+}$ and $Mo^{6+}$. When the amount of substitution exceeded ~20 mole%, the phases began to collapse and the second phases were made. The dielectric constants of substituted compounds were enlarged nevertheless Cr or Mo is substituted. The increment is bigger in the Mo substituted compound than in the Cr doped one although the Nb(Cr)$O_6$ octahedra could be more strongly distorted than the Nb(Mo)$O_6$ octahedra since the ionic size difference between $Nb^{5+}$ and of $Cr^{6+}$ is much bigger than that between $Nb^{5+}$ and $Mo^{6+}$. Consequently, the dielectric constant of the substituted Aurivillius phase $Bi_2$A_{n-1}B_{n-x}M_xO_{3n+1}$$</TEX> depends on the extent of distortion of the B$O_6$ octahedra and more strongly on the polarizability of the metal.