These R-markdown files can be used to calculate the inter-rater reliability of ELAN annotations. Two approaches are available here: a frame-based approach and an event-based approach. The event-based approach is not yet discussed in published work; the frame-based approach is discussed in: Oomen, Marloes, Tobias de Ronde, Lyke Esselink & Floris Roelofsen. In press. Towards a procedure for annotating non-manual markers in sign languages and multi-modal communication. In Proceedings of NELS 53 . The input is a .csv file with annotations extracted from ELAN. The output consists of confusion matrices of overlapping categories, Cohen's Kappa agreement scores, and a .csv file with overlapping annotations. Some notes: - These functions only work with two annotators. - Each video has a "Question" tier, which denotes the video that is annotated. Our file structure was organised as such that multiple videos could be annotated in one ELAN file. - Our annotators both annotated the same videos, in separate files. To ...