[Tencent 2025] HY-WU: Unleashing Functional Memory to Solve the "Conflict" in Model Adaptation

TL;DR

Tencent’s HY Team has introduced HY-WU (Weight Unleashing), a paradigm shift in how we adapt foundation models. By moving away from static, shared weights to on-the-fly parameter generation, HY-WU creates a "functional memory" that specializes for every single input. In the high-stakes arena of text-guided image editing, it outperformed GPT Image 1.5 and Seedream 4.5, proving that routing in parameter space is the key to solving catastrophic interference.

Problem & Motivation: The "Single Point" Trap

Most current adaptation methods (LoRA, SFT) try to find a single "golden" parameter vector $heta+\Delta het{a}_{static}$ that satisfies all users and tasks. The authors argue this is mathematically brittle.

In a heterogeneous world, different objectives (e.g., "make this image look older" vs. "restore this old photo") induce separated feasible regions in parameter space. When you force a model to learn both via a single static weight update:

1. Infeasible Sharing: The model finds a "compromise" that does neither task well.
2. Over-specialization: If you train separate adapters for every task, you lose generalization and face a management nightmare.

HY-WU’s insight is that memory should be functional: instead of storing facts, store the ability to generate the right transformation operator for the specific instance.

Methodology: The Neural Network Transformer (NNT)

HY-WU treats weight generation as a first-class modality. The core of the system is a Neural Network Transformer (NNT) that maps a hybrid condition (image + text) directly into LoRA weights.

1. Rank-Anchored 2D Tokenization

Generating millions of parameters is computationally expensive. HY-WU solves this by exploiting the structure of LoRA. Since the rank ($r$) is typically constant across layers, they use it as an "anchor" to organize weights into manageable $rimesd$ tokens. This preserves architectural locality while allowing the transformer to process weights as a sequence.

2. On-the-Fly End-to-End Training

Unlike previous hypernetworks that required "pre-training" on a collection of existing checkpoints, HY-WU is trained end-to-end. The gradient flows from the image-editing loss, through the generated weights, and back into the generator.

Experiments: Crushing the Baselines

The researchers put HY-WU through a "stress test": text-guided image editing where instructions are often mutually exclusive.

SOTA Performance

HY-WU achieved the #1 rank on GEdit-Bench among open-source models and even surpassed closed-source titans.

The "Routing" Discovery

A fascinating ablation study showed that if you "shuffle" the conditions (giving the model the right capacity but the wrong instruction), performance collapses. This proves the gain isn't from having more parameters—it's from the correct alignment between the instance and the generated operator.

Deep Insight: The Parameter Manifold

By analyzing the generated weights using t-SNE, the authors found that HY-WU implicitly learns a semantically structured weight manifold.

• Edits involving "content removal" cluster in one area of parameter space.
• Edits involving "pose modification" cluster in another.
• Parameters generated for similar images/prompts are "closer" in Euclidean space.

This confirms that the generator isn't just outputting noise; it's navigating a continuous landscape of specialized operators.

Critical Analysis & Future Outlook

Limitations: While the training is "on-the-fly," the initial NNT itself is large (8B parameters). This adds an inference overhead, although the authors mitigated this with kernel optimizations like FlashInfer.

Takeaway: HY-WU marks the transition from "Retrieval-Augmented" models (which fetch data) to "Generation-Augmented" models (which synthesize their own brain). By allocating capacity to structured, routable functional memory, we can finally scale models to handle the infinite edge cases of the real world without "corrupting" the base knowledge.

Keep an eye on this series—Part II is expected to extend this to long-horizon agents and video personas.