LiveAnon

Improving Privacy and Utility in Streaming Speaker Anonymization with Codec Language Models

Yejin Kim^*, Kyungsu Kim^*, Seungu Han^*, Kyogu Lee^*†‡

^* Music and Audio Research Group (MARG)

^† Department of Intelligence and Information, Seoul National University

^‡ Interdisciplinary Program in Artificial Intelligence, Seoul National University

allaboutmy@snu.ac.kr

Abstract

Streaming speaker anonymization must jointly preserve privacy, utility, and real-time feasibility, yet existing systems often lose paralinguistic information through constrained content representations or require additional emotion-specific modeling. We propose LiveAnon, a streaming codec language model that uses continuous ASR encoder features for source content and codec prompt continuation for target identity, while a capped RVQ delay pattern reduces multi-codebook generation latency. Under the VoicePrivacy Challenge 2024 protocol, LiveAnon improves over the matched fixed-target SVA setting across all metrics, increasing emotion UAR from 39.71% to 44.09% and semi-informed EER from 15.92% to 32.28%. LiveAnon also achieves competitive privacy--utility performance against the offline VPC26 baselines, demonstrating that strong anonymization and paralinguistic utility can be maintained in a streaming setting.

System Overview

Overview of the LiveAnon training and inference architecture — **Figure. Overview of LiveAnon.** LiveAnon performs streaming anonymization through continuous source-content conditioning and acoustic prompt continuation. A frozen streaming ASR encoder provides continuous content features, while codec tokens from a reference utterance serve as the acoustic prompt. The causal Transformer continues the prompt according to the source content, and capped RVQ decoding partially parallelizes later codebooks to reduce generation delay.

Audio Quality Comparison

We compare LiveAnon with SVA, a strong recent streaming speaker anonymization system. The examples below are selected from LibriTTS dev-clean and test-clean and present the original source speech, SVA output, and LiveAnon output side by side for direct listening comparison.

Transcript	Original (Src)	SVA	LiveAnon (Ours)
Example 1 “But here they were-the Yankees-a battle had to be fought. We were ordered forward.”
Example 2 “They would think that they might do so too, and that would make you a great deal of trouble.”
Example 3 “The Squierses are a selfish, hard fisted lot, and the old lady, especially, is a well known virago. But they could not have proven a case against Lucy, if she was innocent, and all their threats of arresting her were probably mere bluff.”
Example 4 “The Saxons had now been for four centuries in Britain, the older inhabitants of which-Celts like the Gauls and Irish-they had cruelly harassed, just as the Milesian Irish oppressed their Belgic predecessors, and as the Normans, in turn, will be found oppressing both Celt and Saxon in England and Ireland.”