In the complex real-time multi-agent system such as game environment, dynamic task allocations are repeatedly performed to achieve a goal in terms of system efficiency. In this research, we present a task allocation scheme suitable for the real-time multi-agent environment. The scheme is to optimize the task allocation by complementing existing coordination agent with $A^*$ algorithm. The coordination agent creates a status graph that consists of nodes which represent the combinations of tasks and agents, and refines the graph to remove nodes of non-execution tasks and agents. The coordination agent performs the selective utilization of the $A^*$ algorithm method and the greedy method for real-time re-allocation. Then it finds some paths of the minimum cost as optimized results by using $A^*$ algorithm. Our experiments show that the coordination agent with $A^*$ algorithm improves a task allocation efficiency about 25% highly than the coordination agent only with greedy algorithm.