Given that burnout is a major problem in many societies and that employers are legally obliged to act in preventing job stress, there is a need of validated and reliable short self-report instruments. The Burnout Assessment Tool (BAT) is developed to measure burnout as a syndrome with four core components (exhaustion, mental distance, cognitive and emotional impairment). So far, the BAT was tested in over 40 studies with encouraging results. Although a short, 12-item version of the BAT exists, there is need for an ultra-short version with even less items. The overall aim is to develop an ultra-short 4-item version of the BAT (BAT4) and to evaluate its construct validity using Rasch analysis in samples from various countries along with its measurement invariance regarding country, age and gender. The BAT4 was developed using mixed methods, i.e. combining the results from a Rasch analysis, a subject matter analysis and expert judgements. Construct validity was tested on data from national representative samples from eight countries (the Netherlands, Belgium (Flanders), Austria, Czech Republic, Finland, Germany, Ireland, and Japan) and in a pooled sample combing the data from all eight countries. Differential item functioning regarding age, gender and country was investigated. The BAT4 fulfilled all the criteria required by the Rasch measurement model to constitute a valid measure in the pooled and country specific samples, except Austria and Japan. In the pooled sample, measurement invariance between the eight countries as well as between gender and age was found. Analyses within different countries showed occasional gender and age DIF for some items. The results were promising regarding BAT4’s construct validity and measurement invariance. Although the BAT4 includes only four items, its content coverage is acceptable. The BAT4 can be used as a short screening instrument for burnout complaints at the group or organisational level.