This paper presents structural topology optimization that is being applied for the design of electromagnetic vibration energy harvester. The design goal is to maximize the root-mean-square value of output voltage generated by external vibration leading structures. To calculate the output voltage, the magnetic field analysis is performed by using the finite element method, and the obtained magnetic flux linkage is interpolated by using Lagrange polynomials. To achieve the design goal, permanent magnet is designed by using topology optimization. The analytical design sensitivity is derived from the adjoint variable method, and the formulated optimization problem is solved through the method of moving asymptotes (MMA). As optimization results, the optimal location and shape of the permanent magnet are provided when the magnetization direction is fixed. In addition, the optimization results including the design of magnetization direction are provided.