Master thesis: Fact-checking des contenus multimédias: Exploitation des modèles de fusion multimodale et del’intelligence artificielle explicable

The proliferation of misinformation, exacerbated by generative AI technologies and sophisticated deepfakes, poses a significant threat to informational integrity in modern societies. Traditional fact-checking methods, primarily text-based, struggle to keep pace with the volume and complexity of multimodal content, where text and images are increasingly combined and manipulated. This thesis addresses these limitations by proposing an innovative multimodal fact-checking approach. It aims to develop a robust and transparent system for detecting fake news by effectively integrating visual and textual modalities, thereby enhancing the trustworthiness of automated verification processes. This research underscores the critical need for advanced AI solutions to combat the rapid spread of fabricated information.

Researchers in Artificial Intelligence, Natural Language Processing, Computer Vision, and Explainable AI; professionals in fact-checking, content moderation, and digital journalism; and academics studying misinformation and multimodal data analysis.

This study introduces an innovative multimodal fact-checking system designed to counter the escalating challenge of misinformation, especially that amplified by generative AI and deepfakes. Recognizing the limitations of traditional, text-centric verification methods, the research proposes a hybrid approach leveraging the strengths of computer vision (CV), natural language processing (NLP), and explainable artificial intelligence (XAI). The core methodology involves integrating CLIP for visual analysis and BERT for semantic textual processing, employing a late modality fusion strategy with cross-attention mechanisms to detect semantic inconsistencies between image and text. The system's effectiveness was rigorously evaluated on the MiRAGeNews dataset, demonstrating an impressive overall accuracy of 97% and an AUC of 0.99. A key contribution is the transparent decision-making facilitated by XAI techniques, specifically GradCAM++ for visual explanations and LIME for textual interpretation, which reveal the models' attention zones and discriminant elements. This transparency, coupled with an intelligent uncertainty management based on confidence distributions, ensures user trust and operational utility. By providing a robust, transparent, and explainable framework for detection de fausses informations, this work offers significant advancements for professional fact-checkers and social platforms, setting new benchmarks in automated content verification.