Objective: Alcohol drinking can cause various psychiatric and medical diseases. Although women generally consume less alcohol than men, they may be at a greater risk for alcohol-related psychological distress. The aim of the current study is to evaluate whether the association between alcohol consumption and psychological distress, including stress, depressed mood, and suicidal ideation and the risks of psychological distress differ based on gender. Methods: The cross-sectional study included 31,657 participants (17,915 women and 13,742 men) from the 2010–2013 and 2015 Korea National Health and Nutrition Examination Survey. Alcohol drinking and Alcohol Use Disorders Identification Test (AUDIT) levels were assessed for evaluating the amount of alcohol intake and alcohol-related problems. Self-perception of stress, depressed mood, and suicidal ideation were assessed for evaluating psychological distress. Odds ratio and 95% confidence intervals for psychological distress were calculated using multiple logistic regression models. Results: The risks of psychological distress were not significantly associated with drinking level in both sexes. However, the risks of psychological distress were associated with an increase in AUDIT levels, and there were gender differences in the psychological consequences of alcohol-related problems. The association was more drastic in women, and women showed a significant association even though the severity of drinking problem was low with the exception of stress perception. Conclusion: The risks of psychological distress were associated with the severity of alcohol-related problems and women were more likely to be susceptible. Therefore, it is recommended that women even at low-risk for problematic drinking should be screened for psychological distress.