The purpose of this study is to investigate the degree of anxiety factors for learning mathematics and look into the reason of anxiety of mathematics and show the method of eliminating negative and anxious factors in elementary school, secondary school and high school. To have learner take an interest in learning mathematics, develop the positive attitude about learning mathematics and maximize the effect of learning of mathematics, three kinds of hypotheses are established as follows: 1. There would be difference in anxiety factor of learning mathematics according to elementary school, secondary school and high school. 2. There would be difference in anxiety factor of learning mathematics according to sex. 3. There be difference in anxiety factor of learning mathematics according to the level of achievement. The results of this study are as follows: 1. Hypotheses I was testified. The anxiety factor between mathematics learning and mathematics teacher was significant difference according to elementary school, secondary school and high school. 2. Hypotheses II was testified. The result of hypotheses II was significant difference only in high school. The anxiety factor of mathematics teacher was significant difference according to boy student and girl student. 3. Hypotheses III was testified. The anxiety factor of mathematics, teacher and test was significant difference only in elementary school and high school.