In this paper, a hierarchical link allocation algorithm between mobile stations (MSs) and the corresponding base station (BS) by an optimal utilization of relay stations (RSs) is proposed to improve throughput of RS systems. In the proposed hierarchical algorithm, each RS operates cognitive radio functions to sense the degree of satisfaction in the quality of services (QoSs) and then selects the candidate set of MSs to have links with the RS. Such information is reported to the BS, where an auction process is performed to get an optimal allocation of communication links between the MSs and the BS. To maximize system throughput, the proposed auction algorithm is conducted upon bidding prices of communication links, considering both the co-channel interference (CCI) information shared among adjacent cells and the QoS enhancement information for each MS collected from RSs. The BS then switches the communication links of the auction winner MSs through the corresponding RS. The computer simulation shows that the proposed algorithm enhances the user QoS more than the conventional algorithm, especially for RS systems with more users requiring higher QoS. The proposed algorithm has also been proved to have more robust performance than the conventional one when the traffic load is higher and the CCI becomes stronger.