Hi, I'm Sonal Kumar

We introduce TAC, a system for generating audio captions with precise timestamps, enabling fine-grained alignment between audio events and their textual descriptions. TAC advances the state of the art in temporal audio understanding and supports downstream applications like audio retrieval and audio question answering.

Audio Flamingo 3 presents a fully open-source suite of large audio language models, advancing audio intelligence through improved training recipes, open model weights, and competitive performance across diverse audio understanding and reasoning benchmarks.

MMAU-Pro extends the original MMAU benchmark with significantly more challenging questions requiring deeper expert-level knowledge and multi-step reasoning across speech, music, and environmental sound. It provides a harder evaluation frontier for the next generation of audio general intelligence systems.

We systematically investigate the robustness of audio-language models to linguistic variations in text queries — including paraphrasing, negation, and syntactic transformations — revealing significant brittleness in current ALMs and proposing evaluation protocols to benchmark this understudied dimension of model robustness.

SILA introduces a data augmentation framework for text-to-audio generation that bridges the gap between signal-level audio properties and language descriptions. By automatically generating rich, controllable textual descriptions from audio signals, SILA enables more faithful and controllable audio synthesis from text prompts.

MultiVox is a comprehensive benchmark for evaluating voice assistants in real-world multimodal interaction scenarios that combine speech, visual, and audio signals. It exposes limitations of current voice assistants when handling complex, concurrent modalities.

We introduce a multi-domain audio question answering challenge spanning environmental sounds, music, and speech. The task requires models to reason about acoustic content across diverse domains, pushing the frontier of audio general intelligence beyond category classification.

ReClap improves zero-shot audio classification by augmenting class labels with rich natural language descriptions of the corresponding sounds. By training CLAP-style models with these enriched descriptions, ReClap significantly improves zero-shot transfer across diverse audio classification benchmarks.

LipGER is a visually-conditioned generative approach to ASR error correction that uses lip movements as an additional conditioning signal to disambiguate acoustically confusing speech segments. By incorporating visual speech information, LipGER substantially reduces word error rates in noisy acoustic environments.

RECAP introduces retrieval-augmented generation for automated audio captioning. By retrieving semantically similar audio captions at inference time and conditioning caption generation on them, RECAP produces more accurate, diverse, and grounded audio descriptions compared to standard encoder-decoder baselines.

AV-RIR leverages both audio and visual cues to estimate room impulse responses — the acoustic fingerprint of a space. By jointly modeling visual scene geometry and captured audio, AV-RIR achieves significantly more accurate RIR estimation than audio-only methods, enabling better downstream spatial audio applications.