TalkTrace: Secure Automated Transcription and Summary Generation

Abstract

Although the importance of accurate documentation of meetings is paramount in modern organizations, the majority of the
state-of-the-art transcription services utilize third-party cloud-based AI services, leading to severe concerns over data privacy, security, and user control. This paper proposes a secure and privacy-focused meeting transcription system called TalkTrace, which utilizes automated bot-based audio capture and server-side speech processing by the provider through a locally deployed speech-to-text model. By providing a link to the meeting through the web interface, such as a Zoom or Google Meet link, the automated bot joins the meeting andexits the meeting after the completion of the meeting or manually by the user. The recorded audio is encrypted in real time by employing a hybrid RSA-AES encryption technique to ensure the confidentiality
of the recorded data. On a rented server from a hosting provider, speech transcription and speaker diarization take place, with the Whisper model running locally on the server in inference mode, ensuring that audio data is not transmitted to any external AI services or used for training any model. Speaker identification is done using guided verbal introductions that match with the diarized speech using timestamp-based matching methods. To secure the integrity of the transcripts, the final output of the speech-to-text process is hashed using SHA-256 before secure storage. TalkTrace provides an alternative to traditional cloud-based meeting transcription services by using strong encryption, automated meeting integration, and the
use of hosted local inference using AI services.