In this paper, we propose a simple Double random phase encryption (DRPE)-based orthogonal encoding technique for color image encryption. In the proposed orthogonal encoding technique, a color image is decomposed into red, green, and blue components before encryption, and the three components are independently encrypted with DRPE using the same key in order to decrease the complexity of encryption and decryption. Then, the encrypted data are encoded with a Hadamard matrix that has the orthogonal property. The purpose of the proposed orthogonal encoding technique is to improve the security of DRPE using the same key at the cost of a little complexity. The proposed orthogonal encoder consists of simple linear operations, so that it is easy to implement. We also provide the simulation results in order to show the effects of the proposed orthogonal encoding technique.