Testing of ultrasonic pulse velocity (UPV) is one of the most popular and actual non-destructive techniques used in the estimation of the concrete properties in structures. In this paper, artificial neural network (ANN) approach has been proposed for the evaluation of relationship between concrete compressive strength, UPV, and density values by using the experimental data obtained from many cores taken from different reinforced concrete structures with different ages and unknown ratios of concrete mixtures. The presented approach enables to find practically concrete strengths in the reinforced concrete structures, whose records of concrete mixture ratios are not yet available. Thus, researchers can easily evaluate the compressive strength of concrete specimens by using UPV values. The method can be used in conditions including too many numbers of the structures and examinations to be done in restricted time duration. This method also contributes to a remarkable reduction of the computational time without any significant loss of accuracy. Statistic measures are used to evaluate the performance of the models. The comparison of the results clearly shows that the ANN approach can be used effectively to predict the compressive strength of concrete by using UPV and density data. In addition, the model architecture can be used as a non-destructive procedure for health monitoring of structural elements.