A wavelet-based image compression system allowing both lossless and lossy image compression is proposed in this paper. The proposed algorithm consists of the two stages. The first stage uses the wavelet packet transform and the quad-tree coding scheme for the lossy compression. In the second stage, the residue image taken between the original image and the lossy reconstruction image is coded for the lossless image compression by using the integer wavelet transform and the context based predictive technique with feedback error. The proposed wavelet-based algorithm, allowing an optional lossless reconstruction of a given image, transmits progressively image materials and chooses an appropriate wavelet filter in each stage. The lossy compression result of the proposed algorithm improves up to the maximum 1 dB PSNR performance of the high frequency image, compared to that of JPEG-2000 algorithm and that of S+P algorithm. In addition, the lossless compression result of the proposed algorithm improves up to the maximum 0.39 compression rates of the high frequency image, compared to that of the existing algorithm.