Temporally Sorting Images from Real-World Events

Abstract

As smartphones become ubiquitous in modern life, every major event — from musical concerts to terrorist attempts — is massively captured by multiple devices and instantly uploaded to the Internet. Once shared through social media, the chronological order between available media pieces cannot be reliably recovered, hindering the understanding and reconstruction of that event. In this work, we propose data-driven methods for temporally sorting images originated from heterogeneous sources and captured from distinct angles, viewpoints, and moments. We model the chronological sorting task as an ensemble of binary classifiers whose answers are combined hierarchically to estimate an image’s temporal position within the duration of the event. We evaluate our method on images from the Notre-Dame Catedral fire and the Grenfell Tower fire events and discuss research challenges for analyzing data from real-world forensic events. Finally, we employ visualization techniques to understand what our models have learned, offering additional insights to the problem.

@article{padilha21temporally,
    author = {Rafael Padilha and Fernanda A. Andaló and Bahram Lavi and Luís A.M. Pereira and Anderson Rocha},
    title = {Temporally sorting images from real-world events},
    journal = {Pattern Recognition Letters},
    volume = {147},
    pages = {212-219},
    year = {2021},
    doi = {https://doi.org/10.1016/j.patrec.2021.04.027},
}