As a basic research for the development of the automatic grading and sorting system for dried oak mushrooms, the device to acquire both cap and gill side images of mushroom has been developed and neural network based side recognition and quality grading has been proposed via inputting both side images. 20 quality grades have been selected considering the requirement of grade classifications imposed by the mushroom company. Developed DC motor driven‘V’type reversing device for the image acquisition of both side images of mushroom showed more than 95% success. Most error was caused by very small size mushrooms with a radius of around 1cm. However, it required a further research to reduce the reversing time. Grading and side recognition were performed via inputting normalized size factors and average gray levels of $8{ imes}8$ grids converted from the raw images of both surfaces to the multi-layer back propagation(BP) network. Accuracy of the grading showed about 88.5% and the total grading time including reversing operation was around 2 seconds.