Snap-Seek: Turning Questions Into Timelines

Abstract

Snap-Seek is an AI-powered video assistant designed to help users navigate and extract information from video content efficiently. Our project integrates state-of-the-art artificial intelligence models for speech recognition, natural language processing, and information retrieval to enable features such as automatic transcription of video audio, abstractive summarization of transcripts, and an interactive question-answering system for video content. The assistant processes a given video (from a file or YouTube link) by first transcribing its audio using a robust speech recognition model (OpenAI’s Whisper ), then generating a concise summary of the video using a pre-trained abstractive summarization model (Facebook’s BART ). It further allows users to ask questions about the video; the system finds relevant segments in the transcript and provides answers using a transformer based QA model, supplemented by a semantic search mechanism and summary for context. Snap-Seek supports multilingual queries by translating questions and answers when the video or query language is non-English, leveraging cross-lingual transformer models and translation APIs. Our thesis presents the design, implementation, and evaluation of Snap-Seek. We discuss the underlying algorithms and models, system architecture, and experimental results demonstrating the system’s ability to summarize videos and answer detailed questions about their content accurately. The outcome of our project is an interactive AI assistant that enhances video accessibility and knowledge extraction, showcasing the application of modern AI techniques in multimedia data analysis.