In this paper, an elderly assistance system is developed based on Xenomai, a real-time development framework cooperating with the Linux kernel. A Kinect sensor is used to recognize the behavior of the elderly and A-star search algorithm is implemented to find the shortest path to the person. The mobile robot also generates a trajectory using a digital convolution operator which is based on a Bezier curve for smooth driving. In order to follow the generated trajectory within the control period, we developed real-time tasks and compared the performance of the tracking trajectory with that of non real-time tasks. The real-time task has a better result on following the trajectory within the physical constraints which means that it is more appropriate to apply to an elderly assistant system.