This study aims to develop measures to turn former users and non-users of public libraries back into users by understanding the differences between the user, former user and non-user groups in terms of their perceptions to public libraries, as well as the differences between these groups in their perceptions regarding the factors to non-use of public libraries. A literature review and statistical analysis of the results from a questionnaire survey were conducted to identify the differences. The results from the statistical analysis showed significant differences between the user, former user and non-user groups in terms of their perceptions to public libraries and the non-use factors. From the results, we could categorize the factors into common non-use factors, non-use-only factors, and use interrupt factors, and we develop suggestions to improve the factors in order to move non-users and former users to frequent users of the public libraries.