kerasformers/janus_pro_7b

See our collection for all versions of Janus-Pro.

Run Janus-Pro with Keras 3: JAX, PyTorch, or TensorFlow

kerasformers/janus_pro_7b

Paper: Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling (arXiv:2501.17811) · HF Papers

Janus-Pro is a multimodal model (SigLIP tower + GELU aligner + Llama decoder). This KerasFormers port covers the understanding path only (image + text → text). Multi-image conversations are supported; VQ image generation is not ported.

For more details on the model, please go to the upstream model card.

Pure-Keras 3 conversion of deepseek-ai/Janus-Pro-7B for kerasformers. One implementation runs unmodified on TensorFlow / Torch / JAX.

This is a vision-language checkpoint (JanusGenerate, 7B).

✨ Quick start

import os
os.environ["KERAS_BACKEND"] = "torch"  # or "jax" / "tensorflow"

from PIL import Image
from kerasformers.models.janus import JanusGenerate, JanusProcessor

model = JanusGenerate.from_weights("kerasformers/janus_pro_7b")
processor = JanusProcessor.from_weights("kerasformers/janus_pro_7b")

image = Image.open("your_image.jpg")
inputs = processor(
    conversation=[
        {
            "role": "user",
            "content": [
                {"type": "image", "image": image},
                {"type": "text", "text": "Describe this image in one sentence."},
            ],
        }
    ]
)
outputs = model.generate(**inputs, max_new_tokens=64)
print(processor.decode(outputs[0]))

Load any Janus-Pro variant the same way with from_weights("kerasformers/<variant>"):

Variant	Hub
janus_pro_1b	kerasformers/janus_pro_1b
janus_pro_7b	kerasformers/janus_pro_7b

Tips

• Set KERAS_BACKEND before importing Keras / kerasformers.
• Prefer JanusProcessor.from_weights(...) so image size and tokenizer match.
• Add multiple {"type": "image", ...} items for multi-image chats.
• See Janus-Pro docs and Loading Weights.
• Community / upstream safetensors still work via the hf: prefix, e.g. JanusGenerate.from_weights("hf:deepseek-ai/Janus-Pro-7B").

Special Thanks

A huge thank you to the DeepSeek Janus-Pro authors for creating and releasing these models.

License: MIT.