Vision Banana: Why Your Favorite Image Generator is Secretly a Vision Expert

TL;DR

Is the ability to create art the same as the ability to understand geometry? According to Google's latest research, yes. Vision Banana is a generalist model built on the Nano Banana Pro image generator. By simply "asking" the model to generate visualizations of depth or segmentation masks via instruction-tuning, it beats specialized, task-specific SOTA models like SAM 3 and Depth Anything 3. We are witnessing a paradigm shift: Image Generation is the new "pretraining" for all of computer vision.

Context: The Emergence of Visual Intelligence

For years, we treated Computer Vision (CV) and Image Generation as two separate disciplines. CV models were discriminative (identifying what is there), while generators were creative (dreaming up what isn't). However, just as Large Language Models (LLMs) developed reasoning through text prediction, image generators seem to be developing an "internalized physics" of the visual world.

The authors argue that to generate a realistic cat, a model must understand its shape (segmentation), its distance from the camera (depth), and how light hits its fur (surface normals).

Methodology: Perception as a Generation Task

The core insight of Vision Banana is the "Universal Interface." Instead of adding new heads to the network, the authors treat every task as an image-to-image generation problem.

1. Instruction Tuning

They took Nano Banana Pro and performed lightweight instruction-tuning. The model learns to follow prompts like "Segment the skateboard category in pure yellow" or "Generate a metric depth heatmap."

2. The Invertible RGB Trick

To make this scientifically rigorous, the outputs must be decodable. For depth estimation, they use a Power Transform to map infinite depth values into a bounded RGB cube.

• Formula for Depth Mapping: $f(d,\lambda ,c)=1-(1-d/\lambda c{)}^{\lambda +1}$
• This ensures that the generated "artistic" heatmap can be mathematically reversed to calculate the exact distance in meters.

Figure 1: Vision Banana uses text instructions to format internal representations into precise, per-pixel labels.

Experimental Performance: Beating the Specialists

The most shocking result is that this "generalist" model, which was never explicitly built for depth or segmentation, outperforms the world's best specialists.

2D Segmentation

On the Cityscapes dataset, Vision Banana achieved a 0.699 mIoU, significantly higher than SAM 3 (0.652). It excels particularly in Reasoning Segmentation, where it must understand complex queries like "the object casting a shadow on the wall."

3D Metric Depth

While most depth models require camera intrinsic parameters (focal length, etc.), Vision Banana predicts absolute metric depth from visual cues alone.

• δ1 Accuracy: 0.929 (Vision Banana) vs. 0.918 (Depth Anything 3)
• Takeaway: The model has a better "internal sense" of object sizes than models trained specifically for depth.

Table 1: A summary of Vision Banana crushing benchmarks across the board.

Visual Evidence

The qualitative results are stunning. In Figure 8, we see the surface normal estimation compared against Lotus-2. Vision Banana captures fine-grained details (like the texture of curtains or the sharp edges of furniture) that the specialist model misses or blurs.

Figure 8: High-fidelity surface normals generated by Vision Banana vs. Lotus-2.

Conclusion: A Paradigm Shift

The success of Vision Banana suggests two major takeaways for the industry:

1. Image Generators are Foundation Models: We should stop training separate backbones for every vision task. Large-scale generative pretraining is the "GPT moment" for pixels.
2. Generative Robustness: Generative models handle ambiguity better. If a prompt is vague, a generator explores the distribution, whereas discriminative models often "average" the result into a blurry mess.

Limitations: The primary hurdle is computational. Running a full diffusion-based generator to get a segmentation mask is significantly more "expensive" than running a lightweight U-Net. The next frontier? Making these generalist vision learners as fast as their specialist counterparts.