The purpose of the study was to investigate the performance of the auditory and visual memory with different anxiety levels. Seventy-eight undergraduate students in Shiraz University participated in the study. The subjects first answered the Beck Anxiety Questionnaire and, based on their anxiety scores, were categorized in four groups; namely, non-anxiety, low anxiety, moderate anxiety, and high anxiety students. All subjects were exposed to a self-threat program. Then they were given two auditory and visual memory tasks. For data analysis, ANOVA was applied. The results indicated that anxiety affected auditory memory only and that the difference was observed between moderate and high anxiety groups. The findings show that the negative effects of anxiety on the performance of tasks with phonological processes involving working memory are more compared with those in visual tasks. Since the moderate anxiety group had the best performance on auditory tasks, it can be argued that an optimum level of arousal could help with a better performance.