Ancient masonry towers are regarded as among the most important historical heritage structures of the world. These slender structures typically have orthogonal and circular geometry in plane. These structural forms are commonly installed with adjacent structures. Because of their geometrical shapes and structural constraints, ancient masonry towers are more vulnerable to earthquake damage. The main goal of the paper is to investigate the seismic behavior of Erzurum Clock Tower under earthquake loading and to determine the contribution of the castle walls to the seismic performance of the tower. In this study, four three-dimensional finite element models of the Erzurum Clock Tower were developed and the seismic responses of the models were investigated. Time history analyses were performed using the earthquakes that took place in Turkey in 1983 near Erzurum and in 1992 near Erzincan. In the first model, the clock tower was modeled without the adjacent walls; in the second model, the clock tower was modeled with a castle wall on the south side; in the third model, the clock tower was modeled with a castle wall on the north side; and in the last model, the clock tower was modeled with two castle walls on both the north and south sides. Results of the analyses show that the adjacent walls do not allow lateral movements and the horizontal displacements decreases. It is concluded that the adjacent structures should be taken into consideration when modeling seismic performance in order to get accurate and realistic results.