H.264 decoders usually have pipeline architecture by a macroblock or a 4 ${ imes}$ 4 sub-block. The period of the pipeline is usually fixed to guarantee the operation in the worst case which results in many idle cycles and higher data bandwidth. Adaptive pipeline architecture for H.264 decoders has been proposed for efficient decoding and lower the requirement of the bandwidth for the memory bus. However, it requires a controller for the adaptive priority control to utilize the advantage. We propose a smart bus arbiter that replaces the controller. It is introduced to adjust the priority adaptively the QoS (Quality of Service) control of the decoding process. The smart arbiter can be integrated the arbiter of bus systems and it works when certain conditions are met so that it does not affect the original functions of the arbiter. An H.264 decoder using the proposed architecture is designed and implemented to verify the operation using an FPGA.