Learning Visual Clues of the Passage of Time

Abstract

Understanding the passage of time is a natural human ability. Even though time flows in a single direction, its passage can be perceived in many ways. With the popularization of social media and improvements in cameras and smartphones in recent years, every event — from social gatherings to terrorist attacks — is massively captured and instantaneously shared online. Once these images leave their originating devices, they lose most ties to the real world, making it difficult to assess the exact moment they were taken. This temporal information, which is often lost in the uploading and sharing process, is crucial for a forensic expert to understand and fact-check the event. In this research seminar, I will present my Ph.D. research, in which I proposed Deep Learning and Computer Vision methods that leverage the visual content of photographs to mine temporal knowledge. With the backdrop of forensic event analysis, the obtained knowledge can be used to verify if the timestamp of a picture has been manipulated, chronologically sort images within an event, and date historical photographs. The proposed approaches were evaluated in data collected from social media and originated from recent events, such as the Notre-Dame Cathedral Fire (France, 2019) and the Grenfell Tower Fire (UK, 2017). Finally, I will briefly discuss my past experiences with machine learning applied to the medical domain and future research interests in this area.