This research investigates the influence of color as an important factor of the visual image created by clothes. First, the factor analysis of the adjectives describing the images of clothes shows that the images of clothes are classified into 4 factors that include attraction, brightness, femininity, and the figure type (of which the attraction factor and brightness factor were found to be important dimensions). Second, as for the images of feminine style clothes colors, violet appears more refined and attractive than other colors in all 3 tones. Red appears as a brilliant and glowing image in a vivid tone. Yellow in a vivid tone and pale tone, and red in deep tone appear as a warm image, while blue appears as a cold image in all 3 tones. Blue and violet appear as a tall and slim image in all 3 tones. As for the images of mannish style clothes colors, yellow in vivid tone, violet in pale tone and red in deep tone appear as the most refined and attractive image, while green in all the tones appears as a rustic and unattractive image. Red in vivid tone, yellow in pale tone and violet in deep tone appear as a very brilliant and glowing image. Red in pale tone and deep tone appear as a warm and feminine image. Third, yellow in all the tones is evaluated to be attractive in the mannish style in the comparison of the image of feminine and mannish style clothes color, while blue in a pale tone in feminine style and in deep tone in mannish style earned high points. Red and violet did not show any significant differences between the two styles.