We report refinement of crystalline boron by an attrition ball milling system and the superconducting properties of the $MgB_2$ pellets prepared from the refined boron. In this work, we have conducted the ball milling with only crystalline boron powder, in order to improve homogeneity and control the grain size of the $MgB_2$ that is formed from it. We observed that the crystalline responses in the ball-milled boron became broader and weaker when the ball-milling time was further increased. On the other hand, the $B_{2}O_{3}$ peak became stronger in the powders, resulting in an increase in the amount of MgO within the $MgB_2$ volume. The main reason for this is a greater oxygen uptake. From the perspective of the superconducting properties, however, the sample prepared from boron that was ball milled for 5 hours showed an improvement of critical current density ($J_c$), even with increased MgO phase, under an external magnetic field at 5 and 20 K.