briaai/Fibo-mlx-4bit

FIBO Guidance Distillation LoRA ⚡

Accelerate FIBO inference by 2x with a distillation LoRA.
This LoRA distills classifier-free guidance into the model, allowing you to generate images with 2x faster inference. As with most distillation approaches, there may be a slight degradation in quality compared to the full model.

---

🚀 Overview

This is a Guidance Distillation LoRA for the FIBO text-to-image model.

By distilling the knowledge of the teacher model (typically running at high guidance scales) into this lightweight adapter, you can run inference with Guidance Scale (CFG) = 1.0. This skips the negative prompt pass entirely, effectively doubling the inference speed compared to standard generation. As this is a distillation LoRA, there is a slight quality degradation compared to the full model at CFG=5, though the speed benefits often outweigh this tradeoff for many use cases.

✨ What's New

• Nov 2025: Initial release of FIBO Guidance Distillation LoRA
• 2x inference speedup with maintained quality
• Compatible with standard diffusers pipeline

🔑 Key Benefits

• 2x Faster Inference: Running at guidance_scale=1 means calculating the noise prediction only once per step instead of twice.
• Quality Tradeoff: As a distillation LoRA, there is a slight quality degradation compared to the full model at CFG=5, but the speed gains make it ideal for rapid iteration and production workflows where speed is prioritized.
• Drop-in Replacement: Works seamlessly with existing FIBO workflows—just set guidance_scale=1.0.
• Memory Efficient: Minimal additional GPU memory overhead.

---

📊 Comparison & Examples

Left: Regular FIBO (Base Model, CFG=5)  |  Right: FIBO Lite (Distilled LoRA, CFG=1)

🖼️ More Example Outputs from FIBO Distilled LoRA:

Feature	Base FIBO	FIBO + Distill LoRA
Guidance Scale	5.0 (Typical)	1.0 (Distilled)
Compute per Step	2x (Cond + Uncond)	1x (Cond Only)
Speed	Baseline	~2x Faster
Quality	Full Quality	Slight Degradation

---

🛠️ Usage

Requirements

pip install git+https://github.com/huggingface/diffusers torch torchvision  boltons ujson sentencepiece accelerate transformers

for using gemini api, you need to install google-genai as well

Quick Start

import os

import torch
from diffusers import BriaFiboPipeline
from diffusers.modular_pipelines import ModularPipeline

# -------------------------------
# Load the VLM pipeline
# -------------------------------
torch.set_grad_enabled(False)

# Using local VLM
vlm_pipe = ModularPipeline.from_pretrained("briaai/FIBO-VLM-prompt-to-JSON", trust_remote_code=True)


# Using Gemini API, requires GOOGLE_API_KEY environment variable
# assert os.getenv("GOOGLE_API_KEY") is not None, "GOOGLE_API_KEY environment variable is not set"
# vlm_pipe = ModularPipeline.from_pretrained("briaai/FIBO-gemini-prompt-to-JSON", trust_remote_code=True)



# Load the FIBO pipeline
pipe = BriaFiboPipeline.from_pretrained(
    "briaai/Fibo-lite",
    torch_dtype=torch.bfloat16,
)
pipe.to("cuda")

# Generate with guidance_scale=1.0 (2x faster!)
output = vlm_pipe(
    prompt="A hyper-detailed, ultra-fluffy owl sitting in the trees at night, looking directly at the camera with wide, adorable, expressive eyes. Its feathers are soft and voluminous, catching the cool moonlight with subtle silver highlights. The owl's gaze is curious and full of charm, giving it a whimsical, storybook-like personality."
)
json_prompt_generate = output.values["json_prompt"]

def get_default_negative_prompt(existing_json: dict) -> str:
    negative_prompt = ""
    style_medium = existing_json.get("style_medium", "").lower()
    if style_medium in ["photograph", "photography", "photo"]:
        negative_prompt = """{'style_medium':'digital illustration','artistic_style':'non-realistic'}"""
    return negative_prompt


negative_prompt = get_default_negative_prompt(json.loads(json_prompt_generate))
results_generate = pipe(
    prompt=json_prompt_generate, num_inference_steps=50, guidance_scale=1, negative_prompt=negative_prompt
)
results_generate.images[0].save("image_generate.png")
with open("image_generate_json_prompt.json", "w") as f:
    f.write(json_prompt_generate)

Key Parameters

• guidance_scale=1.0: This is the magic setting! With the LoRA loaded, you get the quality of CFG=5 at the speed of CFG=1.
• num_inference_steps=50: Standard for FIBO. Adjust based on your quality/speed tradeoff.
• No negative prompt needed: The distillation handles this internally.

---

🧠 Training Details

• Method: Guidance Distillation (Distilling the CFG effect into the model weights via LoRA).
• Base Model: briaai/FIBO
• Trainable Parameters: 471,472,128
• Precision: bfloat16
• Teacher Configuration: CFG=5.0 (standard FIBO setting)
• Student Configuration: CFG=1.0 (target deployment)
• Training Objective: Minimize KL divergence between teacher and student outputs

Training Process

This model was trained to minimize the difference between:

• Teacher Model: Base FIBO running at guidance_scale=5.0
• Student Model: Base FIBO + LoRA running at guidance_scale=1.0

The training process effectively "bakes in" the stylistic and structural benefits of classifier-free guidance without the computational cost at inference time.

---

🤝 Community & Support

• GitHub: FIBO Repository
• Hugging Face: FIBO Model Card
• Commercial Licensing: Contact Bria AI

---

📚 Citation

If you use this model in your research or project, please cite:

@misc{gutflaish2025generating,
  title         = {Generating an Image From 1,000 Words: Enhancing Text-to-Image With Structured Captions},
  author        = {Gutflaish, Eyal and Kachlon, Eliran and Zisman, Hezi and Hacham, Tal and Sarid, Nimrod and Visheratin, Alexander and Huberman, Saar and Davidi, Gal and Bukchin, Guy and Goldberg, Kfir and Mokady, Ron},
  year          = {2025},
  eprint        = {2511.06876},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV},
  doi           = {10.48550/arXiv.2511.06876},
  url           = {https://arxiv.org/abs/2511.06876}
}

---

📄 License

Source-Code & Weights

• The model is open source for non-commercial use with this license
• For commercial use Click here.