Tencent Open-Sources WeMM-Video for Omni-Modal Video Understanding
Tencent open-sourced WeMM-Video under Apache 2.0: 2B to 35B-A3B, reading video with its soundtrack and answering in Chinese or English.
Not a generator.
Tencent's WeChat Vision team released open weights for WeMM-Video: dense 2B, 4B and 9B checkpoints plus a 35B-A3B mixture of experts, all under Apache 2.0, with third-party components remaining subject to their original terms. The models take text, images, audio and video and generate Chinese or English text. On a clip with a soundtrack, they can use the frames and the original audio together to describe events, transcribe speech and reason about what is seen and heard. This is an understanding model. It does not generate video. The earliest post we found on X was a HuggingPapers item at 13:05 Beijing time today, with a follow-up at 13:32 highlighting the 35B-A3B checkpoint. Tencent's official X accounts had no matching original post in this window. Empty GitHub and Hugging Face repos were created on November 17; the BF16 weights landed this morning and midday, at about 09:41 for 2B, 11:57 for 35B-A3B, 12:08 for 4B and 13:08 for 9B.
Do not collapse it into two near-homonyms. WeMM-Embedding is a separate multimodal embedding family from the same group: it does not take audio, and it returns vectors, not dialogue. The Hunyuan flagship Hy4, open-weighted earlier today, is a 770B-class model under the Tencent Hy License, not this line. The WeMM-Video technical report is dated November 19, 2026, and titled WeMM-Video: Pushing the Frontiers of Open-Source Omni-Modal Video Understanding.
The size names follow the language-model backbones: Qwen3.5-2B, Qwen3.5-4B and Qwen3.5-9B for the dense models, and Qwen3.6-35B-A3B for the MoE, which activates 3B parameters per token. Once the vision and audio modules are included, Hugging Face lists the four checkpoints at about 3B, 6B, 10B and 36B total parameters. Official BF16 weight files are 6.12, 10.45, 19.54 and 67.39 GiB, with FP8 Block128, AWQ W4A16, GGUF Q4_K_M and NVFP4 W4A16 quantizations published alongside.
It listens to the original track, not a separate narration
The architecture largely follows video-SALMONN 2. Vision uses each backbone's native encoder and projection. Audio goes through a Whisper-based encoder at 16 kHz, in 30-second chunks, then a window-level Q-Former that maps every 0.1-second window to one audio token—10 audio tokens per second. For video with a soundtrack, visual and audio tokens are interleaved in time and given explicit timestamps. In the API, frames only are video; frames plus the original soundtrack are video_audio. Instruct mode is the default. Setting enable_thinking=True turns on thinking mode, which the authors say is for longer cross-modal reasoning and for reducing hallucination.
The training corpus is about 1.2 trillion multimodal tokens. In the official breakdown, joint audio-visual data is the largest slice at 45.4%, and captioning is 37.7% of the task mix. After pre-training comes a three-stage post-training pipeline: multi-teacher on-policy distillation (OPD), anti-repetition MrDPO, and specialist reinforcement learning. The paper also says the models are already used in WeChat services, including live-stream commerce, short-form drama analysis and interactive video. That is the authors' own deployment claim, not a third-party audit.
On the official table it is close to several proprietary models, not ahead of all of them
The report calls 35B-A3B the strongest open-source model at a comparable scale, says it narrows the gap with Gemini-3.0-Flash, and says it outperforms Qwen3.5-Omni-Flash and Seed2.0-Mini-0428. That is the authors' summary; the fine-grained table is more mixed. On Table 1's joint audio-visual results, all official numbers: on Video-MME with audio, Gemini-3.0-Flash scores 88.6, WeMM-Video-35B-A3B Thinking 86.4 and Instruct 84.2, ahead of Seed2.0-Mini-0428 at 83.6 and Qwen3.5-Omni-Flash at 79.3. On the harder Video-MME-v2 with audio, Gemini remains at 63.2, against 54.9 for Thinking. On Daily-Omni, Thinking scores 83.4, just above Seed at 82.5 and Gemini at 82.4. On the audio-visual hallucination set AVHBench Judgement, Instruct is 80.3 and Thinking 80.0, above Gemini at 78.5. Thinking is not a uniform lift: on FutureOmni, Instruct is 58.4 and Thinking 56.2; on JointAVBench the pair is 67.4 versus 65.9. These figures are author-reported. Other readings on the main official figure should be taken from the figure itself; we are not estimating points from pixels.
It runs, but not on stock vLLM out of the box
Serving depends on the project's patched runtimes. The official paths are Transformers for BF16, patched vLLM for BF16, FP8, AWQ and NVFP4, patched SGLang for BF16, and a native runtime for GGUF Q4_K_M on CPU or CUDA. The sample commands set the context length to 8,192 tokens and cap each request at four images, two videos and two audio items. That is the configuration in the official examples; the cards do not state a single overall maximum context window. The setup is specific: Python 3.12, an NVIDIA GPU compatible with CUDA 12.9, FFmpeg, and transformers==5.2.0. Loading custom code requires trust_remote_code=True, and the README tells you to review that code first. The vLLM example for 35B-A3B also sets --tensor-parallel-size 2.
Against the Hunyuan Hy4 weights released earlier today, the practical contrast is sharp: this family is an order of magnitude smaller, it hears the original soundtrack, and the license is Apache 2.0, without the Hy License MaaS revenue and MAU conditions. Decide first whether you need understanding or generation, WeMM-Video or WeMM-Embedding, and then pick 2B, 9B or 35B-A3B for the memory you actually have.