A moving object detection algorithm for surveillance video is here proposed which employs background initialization based on two-stage median filtering and a background updating method based on state transition diagram. In the background initialization, the spatiotemporal similarity is measured in the subinterval. From the accumulated difference between the base frame and the other frames in a subinterval, the regions affected by moving objects are located. The median is applied over the subsequence in the subinterval in which regions share similarity. The outputs from each subinterval are filtered by a two-stage median filter. The background of every frame is updated by the suggested state transition diagram The object is detected by the difference between the current frame and the updated background. The proposed method showed good results even for busy, crowded sequences which included moving objects from the first frame.