In the intelligent video surveillance system, moving objects generally are detected by calculating difference between background and input image. However formation of reliable background is known to be still challenging task because it is hard to cope with the complicated background. In this paper we propose an adaptive background formation algorithm considering stationary object. At first, the initial background is formed by averaging the initial N frames. Object detection is performed by comparing the current input image and background. If the object is at a stop for a long time, we consider the object as stationary object and background is replaced with the stationary object. On the other hand, if the object is a moving object, the pixels in the object are not reflected for background modification. Because the proposed algorithm considers gradual illuminance change, slow moving object and stationary object, we can form background adaptively and robustly which has been shown by experimental results.