The Sternberg Memory Scanning (SMS) task provides a measure of processing speed (PS) and working memory retrieval speed (WMS). In this task, participants are presented with sets of stimuli that vary in size. After a delay, one item is presented, and participants indicate whether or not the item was part of the set. Performance is assessed by speed and accuracy for both the positive (item is part of the set) and the negative trials (items is not part of the set). To examine the causes of variation in PS and WMS, 623 adult twins and their siblings completed the SMS task. A non-linear growth curve (nLGC) model best described the increase in reaction time with increasing set size. Genetic analyses showed that WMS (modeled as the Slope in the nLGC model) has a relatively small variance which is not due to genetic variation while PS (modeled as the Intercept in the nLGC model) showed large individual differences, part of which could be attributed to additive genetic factors. Heritability was 38% for positive and 32% for negative trials. Additional multivariate analyses showed that the genetic effects on PS for positive and negative trials were completely shared. We conclude that genetic influences on working memory performance are more likely to act upon basic processing speed and (pre)motoric processes than on the speed with which an item is retrieved from short term memory. © 2009 The Author(s).