Efficient monitoring of tree canopy mortality requires data that cover large areas and capture changes over time while being precise enough to detect changes at the canopy level. In the development of automated approaches, aerial images represent an under-exploited scale between high-resolution drone images and satellite data. Our aim herein was to use a deep learning model to automatically detect canopy mortality from high-resolution aerial images after severe drought events in the summers 2018–2020 in Luxembourg. We analysed canopy mortality for the years 2017–2020 using the EfficientUNet++, a state-of-the-art convolutional neural network. Training data were acquired for the years 2017 and 2019 only, in order to test the robustness of the model for years with no reference data. We found a severe increase in canopy mortality from 0.64 km2 in 2017 to 7.49 km2 in 2020, with conifers being affected at a much higher rate than broadleaf trees. The model was able to classify canopy mortality with an F1-score of 66%–71% and we found that for years without training data, we were able to transfer the model trained on other years to predict canopy mortality, if illumination conditions did not deviate severely. We conclude that aerial images hold much potential for automated regular monitoring of canopy mortality over large areas at canopy level when analysed with deep learning approaches. We consider the suggested approach a cost-efficient and -effective alternative to drone and field-based sampling.