In this paper, we propose an AR-based tangible interaction using a finger fixture for virtual evaluation of digital handheld products. To realize tangible interaction between a user and a product in a computer-vision based AR environment, we uses two types of tangible objects: a product-type object and a finger fixture. The product-type object is used to acquire the position and orientation of the product, and the finger fixture is used to recognize the position of a finger tip. The two objects are fabricated by RP technology and AR markers are attached to them. The finger fixture is designed to satisfy various requirements with an ultimate goal that the user holding the finger fixture in his or her index finger can create HMI events by touching specified regions (buttons or sliders) of the product-type object with the finger tip. By assessing the accuracy of the proposed interaction, we have found that it can be applied to a wide variety of digital handheld products whose button size is not less than 6 mm. After performing the design evaluation of several handheld products using the proposed AR-based tangible interaction, we received highly encouraging feedback from users since the proposed interaction is intuitive and tangible enough to provide a feeling like manipulating products with human hands.