This paper presents an elliptical vibration machining (EVM) method, which is utilized to establish a micro-dimple or groove pattern for surface texturing. The EVM has been explored extensively by the precision community to significantly improve machining performance through lower cutting force, reducing tool wear, burr suppression, extended tool life and lower surface roughness. A new design of a tool holder has been developed, yielding two-directional motions of the tool tip of about $10{mu}m$ and $30{mu}m$ in the axial and transverse directions, respectively. The tool holder design uses a four-bar flexure hinge, of which the flexure body has a first natural frequency of about 4.6 kHz according to an impact test. In this research, two piezo actuators have been driven in open-loop conditions by energizing two sinusoidal voltages from a function generator with the developed tool holder attached to a conventional computer numerical controlled (CNC) 3-axis milling machine. The micro-dimples and grooves have been successfully built using a vibration frequency of 100 Hz with various phases between $15^{circ}$ and $60^{circ}$ to verify the cutting performance of the developed tool holder.