The rivet joint has typical structural feature that can be initiation site for the fatigue crack due to the combination of local stress concentration around rivet hole and the moisture trapping. From a viewpoint of structural assurance, it is crucial to evaluate the size of crack around the rivet holes by appropriate nondestructive evaluation techniques. Lamb wave that is one of guided waves, offers a more efficient tool for nondestructive inspection of plates. The neural network that is considered to be the most suitable for pattern recognition has been used by researchers in NDE field to classify different types of flaws and flaw sizes. In this study, clack size evaluation around the rivet hole using the neural network based on the back-propagation algorithm has been tarried out by extracting some features from the ultrasonic Lamb wave for A12024-T3 skin panel of aircraft. Special attention was paid to reduce the coupling effect between the transducer and the specimen by extracting some features related to time md frequency component data in ultrasonic waveform. It was demonstrated clearly that features extracted from the time and frequency domain data of Lamb wave signal were very useful to determine crack size initiated from rivet hole through neural network.