The purpose of this study was to identify the effective physical factors in socio-economic labeling of residents from the perspective of two groups of users and designers. In order to achieve these goals, a set of physical factors in the sources was identified using systematic data base method and was developed in the form of housing physical factors questionnaire. Snowballs were placed and polls were conducted. Identification of the effective factors was performed by calculating the mean score obtained for each category (as a cut-off point) and deletion of factors with a score below the designated score and categorized into 4 groups through exploratory factor analysis. Independent t-test was used to investigate the difference between the two groups. The results showed that there was a significant difference between the two groups regarding the location, site and interior dimensions of the eligible housing. Users also found that internal dimension factors were the most important factor in determining socioeconomic status of residents, but designers identified the location factor set as the most effective category. Therefore, designers need to design according to user views and purposeful use of elements and factors at the custom level; this process reduces the induction of negative labeling to low-income housing by transferring socially accepted meanings.