The relationship between self-esteem (SE), type of stressor, and fluctuations in heart rate was assessed in a sample of 59 college students (40 females, 19 males; with a mean age of 23.98 years (SEM = 1.0)). SE was measured using the Rosenberg Self-Esteem Scale. The study assessed whether SE buffers the cardiovascular response to stress by comparing responses to two types of stressor: mental arithmetic and verbal memory. As predicted, an SE x stressor interaction was found (p = 0.039). High-SE participants found both stressors moderately stressful but low-SE participants found the mental arithmetic task particularly stressful. This is consistent with the view that mental arithmetic elicits a specific fear that exceeds that associated with other domains of performance. The present study suggests that such fear affects low-SE participants more strongly than high-SE participants. The interaction was statistically independent of potential physiological contaminants such as gender, age, smoking, and caffeine consumption.