Edges are a robust feature for object detection. In this paper, we present an edge-based background modeling method for the detection of moving objects. The edges in the image frames were mapped using robust Canny edge detector. Two edge maps were created and combined to calculate the ultimate moving-edge map. By selecting all the edge pixels of the current frame above the defined threshold of the ultimate moving edges, a temporary background-edge map was created. If the frequencies of the temporary background edge pixels for several frames were above the threshold, then those edge pixels were treated as background edge pixels. We conducted a performance comparison with previous works. The existing edge-based moving-object detection algorithms pose some difficulty due to the changes in background motion, object shape, illumination variation, and noises. The result of the performance evaluation shows that the proposed algorithm can detect moving objects efficiently in real-world scenarios.