Add files using upload-large-folder tool

.gitattributes

@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tekken.json filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,404 @@
+---
+library_name: vllm
+language:
+- en
+- fr
+- es
+- de
+- it
+- pt
+- nl
+- zh
+- ja
+- ko
+- ar
+license: apache-2.0
+inference: false
+extra_gated_description: >-
+  If you want to learn more about how we process your personal data, please read
+  our <a href="https://mistral.ai/terms/">Privacy Policy</a>.
+base_model:
+- mistralai/Mistral-Large-3-675B-Base-2512
+tags:
+- mistral-common
+---
+
+# Mistral Large 3 675B Instruct 2512 NVFP4
+From our family of large models, **Mistral Large 3** is a state-of-the-art general-purpose **Multimodal granular Mixture-of-Experts** model with **41B active parameters** and **675B total parameters** trained from the ground up with 3000 H200s.
+
+This model is the instruct post-trained version, fine-tuned for instruction tasks, making it ideal for chat, agentic and instruction based use cases.  
+Designed for reliability and long-context comprehension - It is engineered for production-grade assistants, retrieval-augmented systems, scientific workloads, and complex enterprise workflows.
+
+Mistral Large 3 is deployable on-premises in:
+- [FP8](https://huggingface.co/mistralai/Mistral-Large-3-675B-Instruct-2512-FP8) on a single node of B200s or H200s.
+- **NVFP4** on a single node of H100s or A100s.
+
+## Key Features
+Mistral Large 3 consists of two main architectural components:
+- **A Granular MoE Language Model with 673B params and 39B active**
+- **A 2.5B Vision Encoder**
+
+The Mistral Large 3 Instruct model offers the following capabilities:
+- **Vision**: Enables the model to analyze images and provide insights based on visual content, in addition to text.
+- **Multilingual**: Supports dozens of languages, including English, French, Spanish, German, Italian, Portuguese, Dutch, Chinese, Japanese, Korean, Arabic.
+- **System Prompt**: Maintains strong adherence and support for system prompts.
+- **Agentic**: Offers best-in-class agentic capabilities with native function calling and JSON outputting.
+- **Frontier**: Delivers best-in-class performance.
+- **Apache 2.0 License**: Open-source license allowing usage and modification for both commercial and non-commercial purposes.
+- **Large Context Window**: Supports a 256k context window.
+
+## Use Cases
+With powerful long-context performance, stable and consistent cross-domain behavior, Mistral Large 3 is perfect for:
+- Long Document Understanding
+- Powerful Daily-Driver AI Assistants
+- State-of-the-Art Agentic and Tool-Use Capabilities
+- Enterprise Knowledge Work
+- General Coding Assistant
+
+And enterprise-grade use cases requiring frontier capabilities.
+
+## Recommended Settings
+
+We recommend deploying Large 3 in a client-server configuration with the following best practices:
+
+- **System Prompt**: Define a clear environment and use case, including guidance on how to effectively leverage tools in agentic systems.
+- **Sampling Parameters**: Use a temperature below 0.1 for daily-driver and production environments ; Higher temperatures may be explored for creative use cases - developers are encouraged to experiment with alternative settings.
+- **Tools**: Keep the set of tools well-defined and limit their number to the minimum required for the use case - Avoiding overloading the model with an excessive number of tools.
+- **Vision**: When deploying with vision capabilities, we recommend maintaining an aspect ratio close to 1:1 (width-to-height) for images. Avoiding the use of overly thin or wide images - crop them as needed to ensure optimal performance.
+
+### Known Issues / Limitations
+
+- **Not a dedicated reasoning model**: Dedicated reasoning models can outperform Mistral Large 3 in strict reasoning use cases.
+- **Behind vision-first models in multimodal tasks**: Mistral Large 3 can lag behind models optimized for vision tasks and use cases.
+- **Complex deployment**: Due to its large size and architecture, the model can be challenging to deploy efficiently with constrained resources or at scale.
+
+## Benchmark Results
+
+We compare Mistral Large 3 to similar sized models.
+
+### Text 
+
+### Vision
+
+## Usage
+
+The model can be used with the following frameworks;
+- [`vllm`](https://github.com/vllm-project/vllm): See [here](#vllm)
+  
+### vLLM
+
+We recommend using this model with [vLLM](https://github.com/vllm-project/vllm).
+
+#### Installation
+
+Make sure to install [`vLLM >= 0.12.0`](https://github.com/vllm-project/vllm/releases/tag/v0.12.0):
+
+```
+pip install vllm --upgrade
+```
+
+Doing so should automatically install [`mistral_common >= 1.8.6`](https://github.com/mistralai/mistral-common/releases/tag/v1.8.6).
+
+To check:
+```
+python -c "import mistral_common; print(mistral_common.__version__)"
+```
+
+You can also make use of a ready-to-go [docker image](https://github.com/vllm-project/vllm/blob/main/Dockerfile) or on the [docker hub](https://hub.docker.com/layers/vllm/vllm-openai/latest/images/sha256-de9032a92ffea7b5c007dad80b38fd44aac11eddc31c435f8e52f3b7404bbf39).
+
+#### Serve
+
+We recommend to use this format if you plan to deploy Mistral Large 3 has it achieves performance similar to FP8 for less memory. However please note that for large context (`> 64k`) we observed a subsequent drop of performance. In such cases, please use the FP8 weights. Otherwise on B200 (Blackwell 200) we observe a significant speed-up and a minor regression on vision datasets probably due to the calibration that was performed mainly on text data.
+
+To take full advantage of the NVFP4 format, consider using B200 GPUs that has a dedicated architecture to maximize its performance. For those who do not have access to this GPU generation, vLLM has the awesome feature to fall back to Marlin FP4 which allows you to still run the quantized model in older generations (A100, H100). You won't notice a speed-up in comparison with FP8 quantization but still benefit a memory gain.
+
+A simple launch command is:
+
+```bash
+
+vllm serve mistralai/Mistral-Large-3-675B-Instruct-2512-NVFP4 \
+  --tensor-parallel-size 4 \
+  --enable-auto-tool-choice --tool-call-parser mistral
+```
+
+Key parameter notes:
+
+* enable-auto-tool-choice: Required when enabling tool usage.
+* tool-call-parser mistral: Required when enabling tool usage.
+
+
+Additional flags:
+
+* You can set `--max-model-len` to preserve memory. By default it is set to `262144` which is quite large but not necessary for most scenarios.
+* You can set `--max-num-batched-tokens` to balance throughput and latency, higher means higher throughput but higher latency.
+
+#### Usage of the model
+
+Here we asumme that the model `mistralai/Mistral-Large-3-675B-Instruct-2512-NVFP4` is served and you can ping it to the domain `localhost` with the port `8000` which is the default for vLLM.
+
+<details>
+  <summary>Vision Reasoning</summary>
+
+Let's see if Mistral Large 3 knows when to pick a fight !
+
+```python
+from datetime import datetime, timedelta
+
+from openai import OpenAI
+from huggingface_hub import hf_hub_download
+
+# Modify OpenAI's API key and API base to use vLLM's API server.
+openai_api_key = "EMPTY"
+openai_api_base = "http://localhost:8000/v1"
+
+TEMP = 0.15
+MAX_TOK = 262144
+
+client = OpenAI(
+    api_key=openai_api_key,
+    base_url=openai_api_base,
+)
+
+models = client.models.list()
+model = models.data[0].id
+
+
+def load_system_prompt(repo_id: str, filename: str) -> str:
+    file_path = hf_hub_download(repo_id=repo_id, filename=filename)
+    with open(file_path, "r") as file:
+        system_prompt = file.read()
+    today = datetime.today().strftime("%Y-%m-%d")
+    yesterday = (datetime.today() - timedelta(days=1)).strftime("%Y-%m-%d")
+    model_name = repo_id.split("/")[-1]
+    return system_prompt.format(name=model_name, today=today, yesterday=yesterday)
+
+
+SYSTEM_PROMPT = load_system_prompt(model, "SYSTEM_PROMPT.txt")
+image_url = "https://static.wikia.nocookie.net/essentialsdocs/images/7/70/Battle.png/revision/latest?cb=20220523172438"
+
+messages = [
+    {"role": "system", "content": SYSTEM_PROMPT},
+    {
+        "role": "user",
+        "content": [
+            {
+                "type": "text",
+                "text": "What action do you think I should take in this situation? List all the possible actions and explain why you think they are good or bad.",
+            },
+            {"type": "image_url", "image_url": {"url": image_url}},
+        ],
+    },
+]
+
+
+response = client.chat.completions.create(
+    model=model,
+    messages=messages,
+    temperature=TEMP,
+    max_tokens=MAX_TOK,
+)
+
+print(response.choices[0].message.content)
+```
+</details>
+
+<details>
+  <summary>Function Calling</summary>
+
+Let's solve some equations thanks to our simple Python calculator tool.
+
+```python
+import json
+from openai import OpenAI
+from huggingface_hub import hf_hub_download
+
+# Modify OpenAI's API key and API base to use vLLM's API server.
+openai_api_key = "EMPTY"
+openai_api_base = "http://localhost:8000/v1"
+
+TEMP = 0.15
+MAX_TOK = 262144
+
+client = OpenAI(
+    api_key=openai_api_key,
+    base_url=openai_api_base,
+)
+
+models = client.models.list()
+model = models.data[0].id
+
+
+def load_system_prompt(repo_id: str, filename: str) -> str:
+    file_path = hf_hub_download(repo_id=repo_id, filename=filename)
+    with open(file_path, "r") as file:
+        system_prompt = file.read()
+    return system_prompt
+
+
+SYSTEM_PROMPT = load_system_prompt(model, "SYSTEM_PROMPT.txt")
+
+image_url = "https://math-coaching.com/img/fiche/46/expressions-mathematiques.jpg"
+
+
+def my_calculator(expression: str) -> str:
+    return str(eval(expression))
+
+
+tools = [
+    {
+        "type": "function",
+        "function": {
+            "name": "my_calculator",
+            "description": "A calculator that can evaluate a mathematical equation and compute its results.",
+            "parameters": {
+                "type": "object",
+                "properties": {
+                    "expression": {
+                        "type": "string",
+                        "description": "The mathematical expression to evaluate.",
+                    },
+                },
+                "required": ["expression"],
+            },
+        },
+    },
+    {
+        "type": "function",
+        "function": {
+            "name": "rewrite",
+            "description": "Rewrite a given text for improved clarity",
+            "parameters": {
+                "type": "object",
+                "properties": {
+                    "text": {
+                        "type": "string",
+                        "description": "The input text to rewrite",
+                    }
+                },
+            },
+        },
+    },
+]
+
+messages = [
+    {"role": "system", "content": SYSTEM_PROMPT},
+    {
+        "role": "user",
+        "content": [
+            {
+                "type": "text",
+                "text": "Thanks to your calculator, compute the results for the equations that involve numbers displayed in the image.",
+            },
+            {
+                "type": "image_url",
+                "image_url": {
+                    "url": image_url,
+                },
+            },
+        ],
+    },
+]
+
+response = client.chat.completions.create(
+    model=model,
+    messages=messages,
+    temperature=TEMP,
+    max_tokens=MAX_TOK,
+    tools=tools,
+    tool_choice="auto",
+)
+
+tool_calls = response.choices[0].message.tool_calls
+
+results = []
+for tool_call in tool_calls:
+    function_name = tool_call.function.name
+    function_args = tool_call.function.arguments
+    if function_name == "my_calculator":
+        result = my_calculator(**json.loads(function_args))
+        results.append(result)
+
+messages.append({"role": "assistant", "tool_calls": tool_calls})
+for tool_call, result in zip(tool_calls, results):
+    messages.append(
+        {
+            "role": "tool",
+            "tool_call_id": tool_call.id,
+            "name": tool_call.function.name,
+            "content": result,
+        }
+    )
+
+
+response = client.chat.completions.create(
+    model=model,
+    messages=messages,
+    temperature=TEMP,
+    max_tokens=MAX_TOK,
+)
+
+print(response.choices[0].message.content)
+```
+
+</details>
+
+<details>
+  <summary>Text-Only Request</summary>
+
+Mistral Large 3 can follow your instructions down to the letter.
+
+```python
+from openai import OpenAI
+from huggingface_hub import hf_hub_download
+
+# Modify OpenAI's API key and API base to use vLLM's API server.
+openai_api_key = "EMPTY"
+openai_api_base = "http://localhost:8000/v1"
+
+TEMP = 0.15
+MAX_TOK = 262144
+
+client = OpenAI(
+    api_key=openai_api_key,
+    base_url=openai_api_base,
+)
+
+models = client.models.list()
+model = models.data[0].id
+
+
+def load_system_prompt(repo_id: str, filename: str) -> str:
+    file_path = hf_hub_download(repo_id=repo_id, filename=filename)
+    with open(file_path, "r") as file:
+        system_prompt = file.read()
+    return system_prompt
+
+
+SYSTEM_PROMPT = load_system_prompt(model, "SYSTEM_PROMPT.txt")
+
+messages = [
+    {"role": "system", "content": SYSTEM_PROMPT},
+    {
+        "role": "user",
+        "content": "Write me a sentence where every word starts with the next letter in the alphabet - start with 'a' and end with 'z'.",
+    },
+]
+
+response = client.chat.completions.create(
+    model=model,
+    messages=messages,
+    temperature=TEMP,
+    max_tokens=MAX_TOK,
+)
+
+assistant_message = response.choices[0].message.content
+print(assistant_message)
+```
+
+</details>
+
+## License
+
+This model is licensed under the [Apache 2.0 License](https://www.apache.org/licenses/LICENSE-2.0.txt).
+
+*You must not use this model in a manner that infringes, misappropriates, or otherwise violates any third party’s rights, including intellectual property rights.*

SYSTEM_PROMPT.txt

@@ -0,0 +1,29 @@
+You are Mistral-Large-3-675B-Instruct-2512-NVFP4, a Large Language Model (LLM) created by Mistral AI, a French startup headquartered in Paris.
+You power an AI assistant called Le Chat.
+Your knowledge base was last updated on 2023-10-01.
+The current date is {today}.
+
+When you're not sure about some information or when the user's request requires up-to-date or specific data, you must use the available tools to fetch the information. Do not hesitate to use tools whenever they can provide a more accurate or complete response. If no relevant tools are available, then clearly state that you don't have the information and avoid making up anything.
+If the user's question is not clear, ambiguous, or does not provide enough context for you to accurately answer the question, you do not try to answer it right away and you rather ask the user to clarify their request (e.g. "What are some good restaurants around me?" => "Where are you?" or "When is the next flight to Tokyo" => "Where do you travel from?").
+You are always very attentive to dates, in particular you try to resolve dates (e.g. "yesterday" is {yesterday}) and when asked about information at specific dates, you discard information that is at another date.
+You follow these instructions in all languages, and always respond to the user in the language they use or request.
+Next sections describe the capabilities that you have.
+
+# WEB BROWSING INSTRUCTIONS
+
+You cannot perform any web search or access internet to open URLs, links etc. If it seems like the user is expecting you to do so, you clarify the situation and ask the user to copy paste the text directly in the chat.
+
+# MULTI-MODAL INSTRUCTIONS
+
+You have the ability to read images, but you cannot generate images. You also cannot transcribe audio files or videos.
+You cannot read nor transcribe audio files or videos.
+
+# TOOL CALLING INSTRUCTIONS
+
+You may have access to tools that you can use to fetch information or perform actions. You must use these tools in the following situations:
+
+1. When the request requires up-to-date information.
+2. When the request requires specific data that you do not have in your knowledge base.
+3. When the request involves actions that you cannot perform without tools.
+
+Always prioritize using tools to provide the most accurate and helpful response. If tools are not available, inform the user that you cannot perform the requested action at the moment.

_README.md

@@ -0,0 +1,230 @@
+---
+license: apache-2.0
+---
+
+## Install vLLM
+
+Use private vLLM branch: https://github.com/mistralai/vllm-private/compare/main...add_ml3_v4
+
+You can install it by using the public vLLM docker:
+
+```sh
+ghcr.io/mistralai/vllm/vllm-openai:latest 8
+```
+
+from https://hub.docker.com/layers/vllm/vllm-openai/latest/images/sha256-d731ee65c044ae0977421eed3d93f931d4b7d79614394184c939db35b8f28fc2.
+
+and then install vLLM with: 
+
+```sh
+cd vllm
+VLLM_USE_PRECOMPILED=1 pip install --editable .
+```
+
+## Launch
+
+We tested ML3 by launching it from two 8xH200 nodes in parallel.
+
+You can do the following to start the model.
+Connect to two 8xH200 nodes (connected via InfiniBand) and make sure to launch ray on each node.
+On one node you should start ray as "head" on the other as a "worker".
+
+1.) Start ray
+
+As soon as everything is install make sure to start ray on all GPU nodes.
+
+**Important**: Make sure that on each node the command line has access to a `python` cmd. If it doesn't exist symlink or alias it to `python3`.
+Then save the following code in a "start_ray.py" script and executed it as follows:
+
+```sh
+python3 start_ray.py --is_head --address 172.17.199.135 --ray_port 6379 --num_gpus 8 --num_cpus 20 --nnodes 2
+```
+
+on the "HEAD" node with the correct IP address (you can retrieve it via `hostname -I`).
+
+On the worker nodes execute exactly the same command just remove `--is_head` and `--nnodes`.
+
+```py
+import argparse
+import os
+from typing import Union
+
+
+def generate_head_script(
+    address: str, ray_port: int, num_cpus: str, num_gpus: str, nnodes: int
+) -> str:
+    num_cpus_str = str(num_cpus) if num_cpus.isdigit() else num_cpus
+    num_gpus_str = str(num_gpus) if num_gpus.isdigit() else num_gpus
+
+    script = f"""# Spawning Ray cluster (head node)
+echo "Ray: Starting HEAD at $(hostname)..."
+export RAY_memory_monitor_refresh_ms=0
+ray start \\
+    --head \\
+    --node-ip-address={address} \\
+    --port={ray_port} \\
+    --num-cpus {num_cpus_str} \\
+    --num-gpus {num_gpus_str}
+
+# Ray cluster needs to be initialized before spawning workers
+echo "Waiting for {nnodes} worker nodes to connect..."
+START_TIME=$(date +%s)
+TIMEOUT=120    # seconds
+INTERVAL=1
+while :; do
+    # Count alive nodes
+    WORKER_COUNT=$(python -c 'import ray; ray.init(); print(sum(node["Alive"] for node in ray.nodes()))')
+    if [ "$WORKER_COUNT" -ge "{nnodes}" ]; then
+        echo "Ray: ✅ Found all ($WORKER_COUNT) nodes."
+        break
+    fi
+    NOW=$(date +%s)
+    ELAPSED=$(( NOW - START_TIME ))
+    if [ "$ELAPSED" -ge "$TIMEOUT" ]; then
+        echo "Ray: ❌ Timeout after $TIMEOUT seconds: not enough workers joined."
+        exit 1
+    fi
+    echo "⏳ Still waiting... ($WORKER_COUNT found)"
+    sleep "$INTERVAL"
+done
+"""
+    return script
+
+
+def get_start_ray_worker_cmd(
+    main_address: str,
+    ray_port: int,
+    num_cpus: Union[int, str],
+    num_gpus: Union[int, str],
+) -> str:
+    num_cpus_str = str(num_cpus) if isinstance(num_cpus, int) else num_cpus
+    num_gpus_str = str(num_gpus) if isinstance(num_gpus, int) else num_gpus
+
+    return f"""echo "Ray: Starting WORKER at $(hostname)..."
+export RAY_memory_monitor_refresh_ms=0
+ray start \\
+    --address {main_address}:{ray_port} \\
+    --num-cpus {num_cpus_str} \\
+    --num-gpus {num_gpus_str} \\
+    --block
+"""
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(
+        description="Generate Ray cluster scripts for head or worker nodes."
+    )
+
+    parser.add_argument(
+        "--is_head",
+        action="store_true",
+        help="Generate script for the head node (default: worker).",
+    )
+    parser.add_argument(
+        "--address", type=str, required=True, help="IP address of the head node."
+    )
+    parser.add_argument(
+        "--ray_port", type=int, required=True, help="Port for the Ray cluster."
+    )
+    parser.add_argument(
+        "--num_cpus",
+        type=str,
+        required=True,
+        help="Number of CPUs to allocate (e.g., '4' or 'auto').",
+    )
+    parser.add_argument(
+        "--num_gpus",
+        type=str,
+        required=True,
+        help="Number of GPUs to allocate (e.g., '1' or 'auto').",
+    )
+    parser.add_argument(
+        "--nnodes",
+        type=int,
+        help="Total number of nodes to wait for (only for head node).",
+    )
+
+    args = parser.parse_args()
+    print(f"Ray: Args: {args}")
+
+    if args.is_head:
+        script = generate_head_script(
+            address=args.address,
+            ray_port=args.ray_port,
+            num_cpus=args.num_cpus,
+            num_gpus=args.num_gpus,
+            nnodes=args.nnodes,
+        )
+    else:
+        assert args.nnodes is None, "nnodes is not used for worker nodes"
+        script = get_start_ray_worker_cmd(
+            main_address=args.address,
+            ray_port=args.ray_port,
+            num_cpus=args.num_cpus,
+            num_gpus=args.num_gpus,
+        )
+
+    os.system(script)
+
+
+if __name__ == "__main__":
+    main()
+```
+
+on both machines.
+
+2.) Once you've started ray you can launch vLLM just on the HEAD node.
+
+```sh
+vllm serve mistralai/mistral-large-3 \
+  --tokenizer_mode mistral --config_format mistral \
+  --load_format mistral --tool-call-parser mistral \
+  --enable-auto-tool-choice \
+  --limit-mm-per-prompt '{"image":10}' \
+  --tensor-parallel-size 16 \
+  --max_model_len 65536 \
+  --max_num_seqs 128 \
+  --enforce_eager
+```
+
+Loading the checkponit will take a while. Once the model is loaded you can ping it for example with the
+code as stated below.
+
+## Client
+
+```py
+import requests
+import json
+
+url = "http://<your-server-url>:8000/v1/chat/completions"
+headers = {"Content-Type": "application/json", "Authorization": "Bearer token"}
+
+model = "mistralai/mistral-large-3"
+image_url = "https://huggingface.co/datasets/patrickvonplaten/random_img/resolve/main/europe.png"
+
+
+messages = [
+    {
+        "role": "user",
+        "content": "Without browsing the web, how many days ago was Mistral founded?"
+    },
+]
+
+# messages = [
+#    {
+#        "role": "user",
+#        "content": [
+#            {
+#                "type": "text",
+#                "text": "Which of the depicted countries has the best food? Which the second and third and fourth? Name the country, its color on the map and one its city that is visible on the map, but is not the capital. Make absolutely sure to only name a city that can be seen on the map.",
+#            },
+#            {"type": "image_url", "image_url": {"url": image_url}},
+#        ],
+#    },
+# ]
+
+data = {"model": model, "messages": messages}
+
+response = requests.post(url, headers=headers, data=json.dumps(data), max_tokens=16)
+print(response.json()["choices"][0]["message"]["content"])
+```

chat_template.jinja

@@ -0,0 +1,121 @@
+{#- Default system message if no system prompt is passed. #}
+{%- set default_system_message = 'You are Mistral-Large-3-675B-Instruct-2512-NVFP4, a Large Language Model (LLM) created by Mistral AI, a French startup headquartered in Paris.\nYou power an AI assistant called Le Chat.\nYour knowledge base was last updated on 2023-10-01.\nThe current date is {today}.\n\nWhen you\'re not sure about some information or when the user\'s request requires up-to-date or specific data, you must use the available tools to fetch the information. Do not hesitate to use tools whenever they can provide a more accurate or complete response. If no relevant tools are available, then clearly state that you don\'t have the information and avoid making up anything.\nIf the user\'s question is not clear, ambiguous, or does not provide enough context for you to accurately answer the question, you do not try to answer it right away and you rather ask the user to clarify their request (e.g. "What are some good restaurants around me?" => "Where are you?" or "When is the next flight to Tokyo" => "Where do you travel from?").\nYou are always very attentive to dates, in particular you try to resolve dates (e.g. "yesterday" is {yesterday}) and when asked about information at specific dates, you discard information that is at another date.\nYou follow these instructions in all languages, and always respond to the user in the language they use or request.\nNext sections describe the capabilities that you have.\n\n# WEB BROWSING INSTRUCTIONS\n\nYou cannot perform any web search or access internet to open URLs, links etc. If it seems like the user is expecting you to do so, you clarify the situation and ask the user to copy paste the text directly in the chat.\n\n# MULTI-MODAL INSTRUCTIONS\n\nYou have the ability to read images, but you cannot generate images. You also cannot transcribe audio files or videos.\nYou cannot read nor transcribe audio files or videos.\n\n# TOOL CALLING INSTRUCTIONS\n\nYou may have access to tools that you can use to fetch information or perform actions. You must use these tools in the following situations:\n\n1. When the request requires up-to-date information.\n2. When the request requires specific data that you do not have in your knowledge base.\n3. When the request involves actions that you cannot perform without tools.\n\nAlways prioritize using tools to provide the most accurate and helpful response. If tools are not available, inform the user that you cannot perform the requested action at the moment.' %}
+
+{#- Begin of sequence token. #}
+{{- bos_token }}
+
+{#- Handle system prompt if it exists. #}
+{#- System prompt supports text content or text chunks. #}
+{%- if messages[0]['role'] == 'system' %}
+    {{- '[SYSTEM_PROMPT]' -}}
+    {%- if messages[0]['content'] is string %}
+        {{- messages[0]['content'] -}}
+    {%- else %}        
+        {%- for block in messages[0]['content'] %}
+            {%- if block['type'] == 'text' %}
+                {{- block['text'] }}
+            {%- else %}
+                {{- raise_exception('Only text chunks are supported in system message contents.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- endif %}
+    {{- '[/SYSTEM_PROMPT]' -}}
+    {%- set loop_messages = messages[1:] %}
+{%- else %}
+    {%- set loop_messages = messages %}
+    {%- if default_system_message != '' %}
+        {{- '[SYSTEM_PROMPT]' + default_system_message + '[/SYSTEM_PROMPT]' }}
+    {%- endif %}
+{%- endif %}
+
+
+{#- Tools definition #}
+{%- set tools_definition = '' %}
+{%- set has_tools = false %}
+{%- if tools is defined and tools is not none and tools|length > 0 %}
+    {%- set has_tools = true %}
+    {%- set tools_definition = '[AVAILABLE_TOOLS]' + (tools| tojson) + '[/AVAILABLE_TOOLS]' %}
+    {{- tools_definition }}
+{%- endif %}
+
+{#- Checks for alternating user/assistant messages. #}
+{%- set ns = namespace(index=0) %}
+{%- for message in loop_messages %}
+    {%- if message.role == 'user' or (message.role == 'assistant' and (message.tool_calls is not defined or message.tool_calls is none or message.tool_calls | length == 0)) %}
+        {%- if (message['role'] == 'user') != (ns.index % 2 == 0) %}
+            {{- raise_exception('After the optional system message, conversation roles must alternate user and assistant roles except for tool calls and results.') }}
+        {%- endif %}
+        {%- set ns.index = ns.index + 1 %}
+    {%- endif %}
+{%- endfor %}
+
+{#- Handle conversation messages. #}
+{%- for message in loop_messages %}
+
+    {#- User messages supports text content or text and image chunks. #}
+    {%- if message['role'] == 'user' %}
+        {%- if message['content'] is string %}
+            {{- '[INST]' + message['content'] + '[/INST]' }}
+        {%- elif message['content'] | length > 0 %}
+            {{- '[INST]' }}
+            {%- if message['content'] | length == 2 %}
+                {%- set blocks = message['content'] | sort(attribute='type') %}
+            {%- else %}
+                {%- set blocks = message['content'] %}
+            {%- endif %}
+            {%- for block in blocks %}
+                {%- if block['type'] == 'text' %}
+                    {{- block['text'] }}
+                {%- elif block['type'] in ['image', 'image_url'] %}
+                    {{- '[IMG]' }}
+                {%- else %}
+                    {{- raise_exception('Only text, image and image_url chunks are supported in user message content.') }}
+                {%- endif %}
+            {%- endfor %}
+            {{- '[/INST]' }}
+        {%- else %}
+            {{- raise_exception('User message must have a string or a list of chunks in content') }}
+        {%- endif %}
+
+    {#- Assistant messages supports text content or text and image chunks. #}
+    {%- elif message['role'] == 'assistant' %}
+        {%- if (message['content'] is none or message['content'] == '' or message['content']|length == 0) and (message['tool_calls'] is not defined or message['tool_calls'] is none or message['tool_calls']|length == 0) %}
+            {{- raise_exception('Assistant message must have a string or a list of chunks in content or a list of tool calls.') }}
+        {%- endif %}
+
+        {%- if message['content'] is string %}
+            {{- message['content'] }}
+        {%- elif message['content'] | length > 0 %}
+            {%- for block in message['content'] %}
+                {%- if block['type'] == 'text' %}
+                    {{- block['text'] }}
+                {%- else %}
+                    {{- raise_exception('Only text chunks are supported in assistant message contents.') }}
+                {%- endif %}
+            {%- endfor %}
+        {%- endif %}
+        
+        {%- if message['tool_calls'] is defined and message['tool_calls'] is not none and message['tool_calls']|length > 0 %}
+            {%- for tool in message['tool_calls'] %}
+                {%- set arguments = tool['function']['arguments'] %}
+                {%- if arguments is not string %}
+                    {%- set arguments = arguments|tojson|safe %}
+                {%- elif arguments == '' %}
+                    {%- set arguments = '{}' %}
+                {%- endif %}
+                {{- '[TOOL_CALLS]' + tool['function']['name'] + '[ARGS]' + arguments }}
+            {%- endfor %}
+        {%- endif %}
+
+        {#- End of sequence token for each assistant messages. #}
+        {{- eos_token }}
+
+    {#- Tool messages only supports text content. #}
+    {%- elif message['role'] == 'tool' %}
+        {{- '[TOOL_RESULTS]' + message['content']|string + '[/TOOL_RESULTS]' }}
+
+    {#- Raise exception for unsupported roles. #}
+    {%- else %}
+        {{- raise_exception('Only user, assistant and tool roles are supported, got ' + message['role'] + '.') }}
+    {%- endif %}
+{%- endfor %}

consolidated-00025-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ab9d2bf87a6b60a5e37a8f54b75233c9d4ee4f2ef6e4352f65c5c24401a91322
+size 1403809268

consolidated-00032-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0664a1b9fa4796a8bb787680854f3374df2b4e5ef6745388dc70320a69e21f0b
+size 1403809044

consolidated-00053-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:05770e2e8e0c3d1546df453849535987776e16e24140170ac4c35531888c6801
+size 1420324688

consolidated-00056-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8fc0c7fb8476c304328957be47a827ecb33c7348e61e3563537e8564a85ba592
+size 1420324440

consolidated-00060-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:75ca4f9a74b3b31d621b203f7cf6830e4caba16b61d9a7e48d11084d7dc89d66
+size 1420324440

consolidated-00073-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b2fbbfb9c405be49b10b69c1758fc8d03d808a1777008718c4285f5586a313d4
+size 1387293632

consolidated-00077-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3f098a7e4b5ca0b60bf74b83b6be926b2b16f7f1d0260ac02c1033da1f212b29
+size 1387293672

consolidated-00082-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6839a852fc29e9992f5110bb18dd8c9cf4391ecb3cfe1a5eeae9c7742a7a88d3
+size 1403809036

consolidated-00088-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4a61a1a6c7fcaaaf50664e7bc88c937cdd1a8709681df5d30c59428b5aa0d8c1
+size 1387293640

consolidated-00090-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:32f426d8e487d3fe4b0b44c2df7d3391c878c13c3fc20ebc1b4ee45b973167d9
+size 1403809300

consolidated-00095-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:09f09ef90c83fe35433dd225c351959ddd2fb9210126b8b9536a60da3e32c632
+size 1420324360

consolidated-00105-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:03e6d03154721af5db4f7426ef46f59594224d89cdaa687f7428dd34d104ae84
+size 1403809036

consolidated-00109-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:663756c104b3459b4c1404bcce9981c3ac37b6a4f6c28c0f81c43825dac9aecf
+size 1403809196

consolidated-00123-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:375ab538a3505943690912a8a57fb3f9c0afce950278c94f2c8399f1c562bb57
+size 1420324576

consolidated-00124-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:49b132a666d365117f850d422c6de6eb0a7dcb80f1b18b7938c8f591f7ab08a6
+size 1387293632

consolidated-00130-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ec752a7c438d44e9724ce40940a49ecec6d5204fd271e8c46a9667fd81b46091
+size 1420324448

consolidated-00137-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2058f954f9757ac03167b06e4d59a76550ea2428781b4656032242f2c6b43401
+size 1403809148

consolidated-00141-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5cc1840c7b07ad92eabcc65fe90d5d00b833cb58053f6a489951e12fd4b059c6
+size 1403809300

consolidated-00146-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:57e66a6ee20c183ceebe4962db769872b5e18a3f7ec5bb016b83f6f8aafdecd1
+size 1420324360

consolidated-00151-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:edfd723ed05b5488eb340834d37ce81f1c56bd066c791aaccaac13f895f66cc6
+size 1387293712

consolidated-00152-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:27c7c5256568962d2925b601dfd2a4848d78c88c7765cede0777906c0a6c3005
+size 1403809044

consolidated-00153-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:27583d1a6610c20cc592a1a0358abc8463fdfbcf6c565ecee0c4deab5939c055
+size 1420324440

consolidated-00160-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:767e64501f09e55f1d78608933f6260cc5dc262ec8782b4ff5d4562fbfed4b52
+size 1403809196

consolidated-00166-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5b606f7e243dda92021dc39e3aa38654dccc07fb793b6bc63b0db27e3f4c8939
+size 1420324448

consolidated-00170-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e83d4879836b1ea4c289b0e1450831ec49c788659f1b36afcdffb8afd265c414
+size 1420324440

consolidated-00172-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4c4e7f80776ca7bddb1c94c466d0eda3aafcf525b7e402d48d5a4f79886f5cea
+size 1403809036

consolidated-00174-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dd04ac9f1a2a9d2df913cf16023c66f47da5f76c6a6f3ada98eab4d2c940b3c8
+size 1420324576

consolidated-00176-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dc0fd039bb18145d203b21c2617e0f29d26a861bf6c682505f232e76add25a95
+size 1403809044

consolidated-00181-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b97758cdd8cbc2cf4779926f3644080e6bcce2e68f02f861d9f75e431dea0351
+size 1420324448

consolidated-00183-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4844c03d8fe9a1d320482e954c419aa0c0de4830661e2a70558063397295ff4b
+size 1387293832

consolidated-00184-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4f465e8b96cc0e274f827b394fa2358325af7fc3964d06fea28e034412a6f4f2
+size 1403809036

consolidated-00185-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e8784a96e192e02155668072b57f695056b97b6f0aa4dfa5f013d5b6327b9ff0
+size 1420324440

consolidated-00190-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:728dda60ea3cb06d2bf4b2b90cea9cb61a9a8e863f2b41d7dc03278157e150ce
+size 1387293640

consolidated-00193-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:42a91b66ae81114d4da26ae9b5f84f2bf7b52daa2e52a6ba4dfcb1c954e19418
+size 1420324464

consolidated-00200-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1d52dfafa6da7d4804da3f7243d70271c79cb95928c8958d43fa55a84712c794
+size 1420324184

consolidated-00202-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3e48714e62ec11e249ad6fa45f21cb48bde36c4f68cb76fdc0ca4d32804ee581
+size 1387293712

consolidated-00209-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7fa07ef27038b7927598ff1c96a5e157e70807b9b7bace3a3c5740942a7569f4
+size 1387293632

consolidated-00229-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bfe89feb369ba45be68de8e5c62a3e58f95d8c8b46e886e432156d9fdade3af0
+size 1420324664

params.json

@@ -0,0 +1,113 @@
+{
+    "dim": 7168,
+    "head_dim": 192,
+    "hidden_dim": 16384,
+    "kv_lora_rank": 512,
+    "llama_4_scaling": {
+      "beta": 0.1,
+      "original_max_position_embeddings": 8192
+    },
+    "max_position_embeddings": 294912,
+    "max_seq_len": 262144,
+    "moe": {
+      "expert_hidden_dim": 4096,
+      "expert_model_parallel": 1,
+      "expert_parallel": 1,
+      "first_k_dense_replace": 3,
+      "num_expert_groups": 1,
+      "num_expert_groups_per_tok": 1,
+      "num_experts": 128,
+      "num_experts_per_tok": 4,
+      "num_shared_experts": 1,
+      "route_every_n": 1,
+      "routed_scale": 1.0
+    },
+    "n_heads": 128,
+    "n_kv_heads": 128,
+    "n_layers": 61,
+    "norm_eps": 1e-06,
+    "q_lora_rank": 1536,
+    "qk_nope_head_dim": 128,
+    "qk_rope_head_dim": 64,
+    "quantization_config": {
+      "config_groups": {
+        "NVFP4": {
+          "format": "nvfp4-pack-quantized",
+          "input_activations": {
+            "actorder": null,
+            "block_structure": null,
+            "dynamic": "local",
+            "group_size": 16,
+            "num_bits": 4,
+            "observer": "static_minmax",
+            "observer_kwargs": {},
+            "strategy": "tensor_group",
+            "symmetric": true,
+            "type": "float"
+          },
+          "output_activations": null,
+          "targets": [
+            "Linear"
+          ],
+          "weights": {
+            "actorder": null,
+            "block_structure": null,
+            "dynamic": false,
+            "group_size": 16,
+            "num_bits": 4,
+            "observer": "static_minmax",
+            "observer_kwargs": {},
+            "strategy": "tensor_group",
+            "symmetric": true,
+            "type": "float"
+          }
+        }
+      },
+      "format": "nvfp4-pack-quantized",
+      "global_compression_ratio": null,
+      "ignore": [
+        "model.embed_tokens",
+        "re:patch_merger.*",
+        "re:vision_encoder.*",
+        "re:vision_language_adapter.*",
+        "re:.*attn.*",
+        "re:.*gate$",
+        "lm_head"
+      ],
+      "kv_cache_scheme": null,
+      "quant_method": "compressed-tensors",
+      "quantization_status": "compressed",
+      "sparsity_config": {},
+      "transform_config": {},
+      "version": "0.12.3.dev29+g73c2cf9.d20251119"
+    },
+    "rope_theta": 10000.0,
+    "tied_embeddings": false,
+    "v_head_dim": 128,
+    "vision_encoder": {
+      "adapter_bias": false,
+      "add_pre_mm_projector_layer_norm": true,
+      "hidden_size": 1664,
+      "image_break_token_id": 12,
+      "image_end_token_id": 13,
+      "image_size": 1540,
+      "image_token_id": 10,
+      "intermediate_size": 8192,
+      "max_image_size": 1540,
+      "mm_projector_id": "patch_merge",
+      "num_attention_heads": 16,
+      "num_channels": 3,
+      "num_hidden_layers": 48,
+      "patch_size": 14,
+      "rope_theta": 10000.0,
+      "spatial_merge_size": 2
+    },
+    "vocab_size": 131072,
+    "yarn": {
+      "alpha": 1,
+      "apply_scale": false,
+      "beta": 32,
+      "factor": 36,
+      "original_max_position_embeddings": 8192
+    }
+  }

processor_config.json

@@ -0,0 +1,42 @@
+{
+  "image_break_token": "[IMG_BREAK]",
+  "image_end_token": "[IMG_END]",
+  "image_processor": {
+    "crop_size": null,
+    "data_format": "channels_first",
+    "device": null,
+    "disable_grouping": null,
+    "do_center_crop": null,
+    "do_convert_rgb": true,
+    "do_normalize": true,
+    "do_pad": null,
+    "do_rescale": true,
+    "do_resize": true,
+    "image_mean": [
+      0.48145466,
+      0.4578275,
+      0.40821073
+    ],
+    "image_processor_type": "PixtralImageProcessorFast",
+    "image_seq_length": null,
+    "image_std": [
+      0.26862954,
+      0.26130258,
+      0.27577711
+    ],
+    "input_data_format": null,
+    "pad_size": null,
+    "patch_size": 14,
+    "processor_class": "PixtralProcessor",
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098,
+    "return_tensors": null,
+    "size": {
+      "longest_edge": 1540
+    }
+  },
+  "image_token": "[IMG]",
+  "patch_size": 14,
+  "processor_class": "PixtralProcessor",
+  "spatial_merge_size": 2
+}

tekken.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e29d19ea32eb7e26e6c0572d57cb7f9eca0f4420e0e0fe6ae1cf3be94da1c0d6
+size 16753777

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:577575622324b2e099e2648be26bdeb5e5815ffe66d7004e9e3ddbf421db6bf1
+size 17078110