Although cell-to-cell variability has been recognized as an unavoidable consequence of stochasticity in gene expression, it may also serve a functional role for tuning physiological responses within a cell population. In the immune system, remarkably large variability in the expression of cytokine genes has been observed in homogeneous populations of lymphocytes, but the underlying molecular mechanisms are incompletely understood. Here, we study the interleukin-4 gene (il4) in T-helper lymphocytes, combining mathematical modeling with the experimental quantification of expression variability and critical parameters. We show that a stochastic rate-limiting step upstream of transcription initiation, but acting at the level of an individual allele, controls il4 expression. Only a fraction of cells reaches an active, transcription-competent state in the transient time window determined by antigen stimulation. We support this finding by experimental evidence of a previously unknown short-term memory that was predicted by the model to arise from the long lifetime of the active state. Our analysis shows how a stochastic mechanism acting at the chromatin level can be integrated with transcriptional regulation to quantitatively control cell-to-cell variability.