As the electrical power industry is restructured, the electrical power exchange is becoming extended. One of the key information used to determine how much power can be transferred through the network is known as available transfer capability (ATC). To calculate ATC, traditional deterministic approach is based on the severest case, but the approach has the complexity of procedure. Therefore, novel approach for ATC calculation is proposed using cost-optimization method, well-being method and risk-benefit method in this paper. This paper proposes the optimal transfer capability of HVDC system between mainland and a separated island in Korea through these three methods. These methods will consider production cost, wheeling charge through HVDC system and outage cost with one depth (N-1 contingency).