This paper proposes a simulation algorithm to assess the gradual yellowing vision of the elderly, which refers to the predominance of yellowness in their vision due to aging of the ocular optic media. This algorithm employed the spectral transmittance property of a yellow filter to represent the color appearance perceived by elderly people with yellow vision, and modeled the changes in the color space through a spectrum change in light using the yellow filter effect. The spectral reflectivity data of 1269 Munsell matte color chips were used as reference data. Under the standard conditions of a D65 illuminant and a $10^{circ}$ observer of 1964 CIE, the spectrum of the 1269 Munsell colors were processed through the yellow filter effect to simulate yellow vision. Various degrees of yellow vision were modeled according to the transmittance percentage of the yellow filter. The color differences before and after the yellow filter effect were calculated using the DE2000 formula, and the color pairs were selected based on the color difference function. These color pairs are distinguishable through normal vision, but the color difference diminishes as the degree of yellow vision increases. Assuming 80% of yellow vision effect, 17 color pairs out of $(1269{ imes}1268)/2$ pairs were selected, and for the 90% of yellow vision effect, only 3 color pairs were selected. The result of this study can be utilized for the diagnosis system of gradual yellow vision, making various types of test charts with selected color pairs.