This paper describes an approach to analyze geometrical information of building images for understanding outdoor environment of autonomous navigation robot. Line segments and color information are used to classily a building with the other objects such as sky, trees, and roads. The line segments and their two neighboring regions are extracted from detected edges in image. The model of line segment (MLS) consists of color information of neighbor regions. This model rules out the line segments of non-building face. A building face converges into dominant vanishing points (DVPs) which include one vertical point and one of five horizontal points in maximum. The intersection of vertical and horizontal lines creates a facet of building. The geometrical characteristics such as the center coordinates, area, aspect ratio and aligned coexistence are used for extracting the windows in the building facet. In experiments, 150 building faces and 1607 windows were detected from the database of outdoor environment. We found that this result shows 94.46% detection rate. These experimental images were all taken in Ulsan metropolitan city in Korea under difference of viewpoints, daytime, camera system and weather condition.