A new hybrid ON/OFF method is presented for the fast solution of electromagnetic inverse problems in high frequency domains. The proposed method utilizes both topological sensitivity (TS) and material sensitivity (MS) to update material properties in unit design cells. MS provides smooth design space and stable convergence, while TS enables sudden changes of material distribution when MS slows down. This combination of two sensitivities enables a reduction in total computation time. The TS and MS analyses are based on a variational approach and an adjoint variable method (AVM), which permits direct calculation of both sensitivity values from field solutions of the primary and adjoint systems. Investigation of the formulations of TS and MS reveals that they have similar forms, and implementation of the hybrid ON/OFF method that uses both sensitivities can be achieved by one optimization module. The proposed method is applied to dielectric material reconstruction problems, and the results show the feasibility and effectiveness of the method.