A Pd-based hydrogen membranes for hydrogen purification and separation need high hydrogen perm-selectivity. The surface roughness of the support is important to coat the pinholes free and thin-film membrane over it. Also, The pinholes drastically decreased the hydrogen perm-selectivity of the Pd-based composite membrane. In order to remove the pinholes, we introduced various surface pre-treatment such as alumina powder packing, nickel electro-plating and micro-polishing pre-treatment. Especially, the micro-polishing pretreatment was very effective in roughness leveling off the surface of the porous nickel support, and it almost completely plugged the pores. Fine Ni particles filled surface pinholes with could form open structure at the interface of Pd alloy coating and Ni support by their diffusion to the membrane and resintering. In this study, a $4{mu}m$ surface pore-free Pd-Cu-Ni ternary alloy membrane on a porous nickel substrate was successfully prepared by micro-polishing, high temperature sputtering and Cu-reflow process. And $H_2$ permeation and $N_2$ leak tests showed that the Pd-Cu-Ni ternary alloy hydrogen membrane achieved both high permeability of $13.2ml{cdot}cm^{-2}{cdot}min^{-1}{cdot}atm^{-1}$ permation flux and infinite selectivity.