Many embedded multimedia systems employ special hardware blocks to co-process with the main processor. Even though an efficient handling of such hardware blocks is critical on the overall performance of real-time multimedia systems, traditional real-time scheduling techniques cannot afford to guarantee a high quality of multimedia playbacks with neither delay nor jerking. This paper presents a hardware-aware rate monotonic scheduling (HA-RMS) algorithm to manage hardware tasks efficiently and handle special hardware blocks in the embedded multimedia system. The HA-RMS prioritizes the hardware tasks over software tasks not only to increase the hardware utilization of the system but also to reduce the output jitter of multimedia applications, which results in reducing the overall response time.