In this paper, we popose a neural network that detects edges of moving objects in an image using a neural network of hierarchical spatial filter with orientation selectivity. We modify the temporal difference network by adding a self loop to each neuraon cell to reduce the problems of phantom edge detected by the neural network proposed by kwon yool et al.. The modified neural network alleviates the phantom edges of moving objects, and also can detect edges of miving objects even for the noisy input. By computer simulation with real images, the proposed neural netowrk can extract edges of different orientation efficiently and also can reduce the phantom edges of moving objects.