Species competition among the toxic dinoflagellates Alexandrium tamarense and Gymnodinium catenatum and the diatom Skeletonema costatum was simulated using a mathematical model. Prior to the model simulation competition experiments using the three species were conducted to obtain data for validation by the simulation model. S. costatum dominated at a density of ${sim}10^{4};cells/mL$ compared to the other species in the medium with dissolved inorganic phosphorus (DIP). The growth of S. costatum was also stimulated by the addition of dissolved organic phosphorus (DOP), such as uridine-5-monophosphate (UMP) or glycerophosphate (Glycero-P), although this species is unable to take up DOP. This implies that the growth of S. costatum may be supported by DIP, which is hydrolyzed by alkaline phosphatase produced from A. tamarense and G. catenatum. The species competition model was run assuming the environmental conditions of northern Hiroshima Bay, Japan, during spring and summer. G. catenatum increased in cell density and neared the level of S. costatum at the end of the calculation. In the sensitivity analyses by means of doubling and halving parameters, depleted DIP had little effect on the cell density of G. catenatum. However the growth of A. tamarense and S. costatum was significantly affected by changes in the parameter values. These results indicate that if DIP depletion is ongoing, species that have a large phosphate pool in their cells, such as G. catenatum, will predominate in the community.