This paper presents a new self-adjoint material sensitivity formulation for optimal designs and inverse problems in the high frequency domain. The proposed method is based on the continuum approach using the augmented Lagrangian method. Using the self-adjoint formulation, there is no need to solve the adjoint system additionally when the goal function is a function of the S-parameter. In addition, the algorithm is more general than most previous approaches because it is independent of specific analysis methods or gridding techniques, thereby enabling the use of commercial EM simulators and various custom solvers. For verification, the method was applied to the several numerical examples of dielectric material reconstruction problems in the high frequency domain, and the results were compared with those calculated using the conventional method.