Before deciding if structures exposed to high temperature are to be repaired or demolished, their final state should be carefully examined. Destructive and non-destructive testing methods are generally applied for this purpose. Compressive strength and color change in mortars are observed as a result of the effects of high temperature. In this study, ordinary and pozzolan-added mortar samples were produced using different aggregates, and exposed to 100, 200, 300, 600, 900 and $1200^{circ}C$. The samples were divided into two groups and cooled to room temperature in water and air separately. Compression tests were carried out on these samples, and the color change was evaluated by the Munsell Color System. The relationships between the change in compressive strength and color of mortars were determined by using a multi-layered feed-forward Neural Network model trained with the back-propagation algorithm. The results showed that providing accurate estimates of compressive strength by using the color components and ultrasonic pulse velocity design parameters were possible using the approach adopted in this study.