The electrochemical properties of Mg-doped $LiFe_{0.48}Mn_{0.48}Mg_{0.04}PO_4$ and pure $LiFe_{0.5}Mn_{0.5}PO_4$ olivine cathodes are examined and the lattice parameters are refined by Rietveld analysis. The calculated atomic parameters from the refinement show that $Mg^{2+}$ doping has a significant effect in the olivine $LiFeMnPO_4$ structure. The unit cell volume is 297.053(2) ${AA}^3$ for pure $LiFe_{0.5}Mn_{0.5}PO_4$ and is decreased to 296.177(1) ${AA}^3$ for Mg-doped $LiFe_{0.48}Mn_{0.48}Mg_{0.04}PO_4$ sample. The doping of $Mg^{2+}$ cation with atomic radius smaller than $Mn^{2+}$ and $Fe^{2+}$ ion induces longer Li-O bond length in $LiO_6$ octahedra of the olivine structure. The larger interstitial sites in $LiO_6$ octahedra facilitate the lithium ion migration and also enhance the diffusion kinetics of olivine cathode material. The $LiFe_{0.48}Mn_{0.48}Mg_{0.04}PO_4$ sample with larger Li-O bond length delivers higher discharge capacities and also notably increases the rate capability of the electrode.