The shortage of fossil fuel drives researchers to find a new way to increases energy efficiency. In this paper, we propose a method to control the direction and speed of an air conditioner using a thermal image and fuzzy controlling method, which results in the increase of energy efficiency. The thermal image is first converted into a color temperature image which represents the temperature range from $24.0^{circ}C$ to $27.0^{circ}C$. The temperature image is divided into 5 columns and the distribution of them is used to analyze room temperature and control an air conditioner. The proposed method was applied to 300 by 400 thermal images. When the performance of the proposed method was compared to existing systems in energy efficiency, the proposed method was better than existing methods, which is clear from experimental results.