Scanning Thermal Microscope (SThM) is the tool that can map out temperature or the thermal property distribution with the highest spatial resolution. Since the local temperature or the thermal property of samples is measured from the extremely small heat transferred through the nanoscale tip-sample contact, improving the sensitivity of SThM probe has always been the key issue. In this study, we develop a new design and fabrication process of SThM probe to improve the sensitivity. The fabrication process is optimized so that cantilevers and tips are made of thermally grown silicon dioxide, which has the lowest thermal conductivity among the materials used in MEMS. The new design allows much higher tip so that heat transfer through the air gap between the sample-probe is reduced further. The position of a reflector is located as far away as possible to minimize the thermal perturbation due to the laser. These full $SiO_2$ SThM probes have much higher sensitivity than that of previous ones.