Data compression at the transport layer could both reduce transmitted bytes over network links and increase the transmitted application data (TCP PDU) in one RTT at the same network conditions. Therefore, it is able to improve transmission efficiency on Internet, especially on the networks with limited bandwidth or long delay links. In this paper, we propose an on-the-fly TCP data compression scheme, i.e., the TCPComp, to enhance TCP performance. This scheme is primarily composed of the compression decision mechanism and the compression ratio estimation algorithm. When the application data arrives at the transport layer, the compression decision mechanism is applied to determine which data block could be compressed. The compression ratio estimation algorithm is employed to predict compression ratios of upcoming application data for determining the proper size of the next data block so as to maximize compression efficiency. Furthermore, the assessment criteria for TCP data compression scheme are systematically developed. Experimental results show that the scheme can effectively reduce transmitted TCP segments and bytes, leading to greater transmission efficiency compared with the standard TCP and other TCP compression schemes.