There has been steady needs for the PAN based carbon fibers, which have high thermal insulation properties and high mechanical properties. Thus, in this study, low thermal conductive PAN carbon fibers were prepared by using zirconia nanofibers, which were prepared from the electrospinning and sol-gel synthesis of zirconium oxide. Wet spun PAN fibers containing the zirconia nanofibers were stabilized and carbonized in order to prepare the composite carbon fibers. Based on the characterization results using scanning electron microscope (SEM), FT-IR, thermo-gravimetric analysis (TGA), and thermal conductivity measurement, the optimal conditions of composite carbon fibers were established. In addition, the thermal conductivity of prepared composite carbon fibers was reduced considerably by the effects of zirconia nanofiber inclusions.