This study investigates the gaps between ideal and actual corporate social responsibility (CSR) by comparing levels of CSR perceptions and examining the relationship differences between CSR perception, perceived quality, and affective trust from a consumer perspective. Data from 315 adult samples were collected through a nationwide online survey. In order to compare ideal and actual CSR, the study employed the approach of latent means analysis (LMA) and multi-group comparisons within the framework of a structural equation modeling (SEM). The result of latent means comparison showed significant differences of perceptions on fashion CSR across ideal and actual contexts. Only actual economic CSR was evaluated higher than in the ideal context and the effect size of inter-management CSR was the largest among seven domains. Multi-group SEM revealed a significant discrepancy between ideal and actual CSR structural models that exhibited relationships of CSR perception, perceived quality, and affective trust. The paths from internal management and ethical CSR to perceived quality and from economic and philanthropy CSR to affective trust were different across ideal and actual contexts. Implying the understanding of consumer CSR perceptions and the mediating role of perceived quality, this study suggests that marketers narrow the gap between ideal and actual CSR.