Dutch listeners are known to employ positional stress for word recognition, while Mandarin listeners use pitch contour contrasts. The present study investigated the influence of pitch dimensions, namely, pitch level, pitch contour, together with pitch position in non-word learning by Dutch and Mandarin listeners. Both groups learned to identify disyllabic pseudo-words differing only in pitch dimensions in a picture selection task. Language specific perceptual patterns were found. Mandarin listeners were found to be able to encode non-native pitch contrasts for word identification, regardless of pitch position. They showed a preference for contour contrasts to level contrasts. Compared to Mandarin listeners, Dutch listeners encountered difficulties. Still, they showed a better performance when pitch contrasts occurred word-finally than -initially, which could be due to recency effect. The findings suggest the influence of phonological representations in the native language on mapping sound to meaning.