Abstract Over the last two decades, the educational use of digital devices, including digital assessments, has become a regular feature of teaching in primary education in the Netherlands. However, researchers have not reached a consensus about the so‐called “mode effect,” which refers to the possible impact of using computer‐based tests (CBT) instead of paper‐and‐pencil‐based tests (PBT) to assess student performance. Some researchers suggest that the occurrence of a mode effect might be related to the type of device used, the subject being assessed and the characteristics of both the test and the students taking the test. The international TIMSS 2019 Equivalence Study offered the opportunity to explore possible performance differences between a PBT and a tablet assessment in mathematics and science among Dutch primary school students. In the spring of 2017, the TIMSS PBT and tablet test were administered to 532 grade‐four Dutch students. Item response theory was used to explore potential mode effects. This exploration revealed no significant differences in the student ability scales between the paper and the tablet tests for mathematics and science. Also, no systematic mode effects were found for the items with high reading demand. A marginal difference was found for girls outperforming boys on the TIMSS tablet test, although no gender differences in achievement were found for the PBT. Practitioner Notes Previous knowledge about this topic Many studies have investigated the test mode differences between CBT and PBT, but their results are divergent and inconclusive. Factor familiarity with the test device has an important impact on test performance. The difference between reading on paper or on a tablet is an important factor. Many studies demonstrate that when reading on a screen, text comprehension is perceived to be more difficult. What this paper contributes This equivalence study shows that CBT is highly comparable to PBT, indicating that both test modes are suitable for testing the conceptual knowledge of ...