Oxy-fuel combustion exhibits combustion and heat transfer characteristics different from air-fuel combustion due to high concentrations of $CO_2$ and $H_2O$. This study evaluated the effect of gas and particle emissions on radiative heat transfer in oxy-fuel combustion of coal. For a hexahedral furnace, prescribed gas compositions based on combustion calculation were used to simplify the combustion reactions. The values of radiative heat fluxes ($q_{rad}$) were compared for different combustion modes, flue gas recirculation (FGR) methods, particle concentrations, furnace sizes and $O_2$ concentrations in the oxidizer. The radiation was calculated by the discrete ordinate method with gaseous emission predicted by the weighted sum of gray gases models (WSGGMs). The results showed that employing an optimized WSGGM is essential for the accurate prediction of $q_{rad}$ in oxy-fuel combustion for gaseous fuels. The conventional WSGGM showed large errors for larger furnace volumes or under dry FGR conditions. With higher particle concentrations such as in pulverized coal combustion, however, $q_{rad}$ was dominated by emission of particles. The effect of gas emissivity was not critical in the furnace with a mean beam length of 8.3m. Oxy-fuel combustion with wet FGR had higher $q_{rad}$ than dry FGR. The $O_2$ concentration in the oxidizer was a key parameter for oxy-fuel combustion since increasing its value linearly increased $q_{rad}$.