This paper proposes a building recognition algorithm using watershed image segmentation algorithm and integrated region matching (IRM). To recognize a building, a preprocessing algorithm which is using Gaussian filter to remove noise and using canny edge extraction algorithm to extract edges is applied to input building image. First, images are segmented by watershed algorithm. Next, a region adjacency graph (RAG) based on the information of segmented regions is created. And then similar and small regions are merged. Second, a color distribution feature of each region is extracted. Finally, similar building images are obtained and ranked. The building recognition algorithm was evaluated by experiment. It is verified that the result from the proposed method is superior to color histogram matching based results.