Running Mistral offline: the tokenizer is on disk, but the library can't find it
An inference server cut off from the Internet. A dedicated disk for models. Mistral's tokenizer downloaded in advance, sitting in its folder. And at startup: FileNotFoundError: No local files found for the repo ID mistralai/Mistral-7B-v0.1 and revision None. The file is there.
This article discusses a bug in Mistral AI's open-source library, mistral-common, that prevents the tokenizer from being found when running Mistral models offline. The issue arises when the tokenizer is stored in a custom folder, and the library fails to locate it, even though the file is present.
What happened
A bug in Mistral AI's open-source library, mistral-common, caused the tokenizer to be inaccessible when running Mistral models offline. The library failed to locate the tokenizer, even though it was stored in a custom folder. The issue was resolved by ensuring both steps of the tokenizer loading process read the same folder.
Why it matters
This bug is critical for teams deploying Mistral models in air-gapped environments, where internet access is restricted. It highlights the importance of testing deployment scenarios, not just development setups, to catch such issues before they affect production.
- Always test deployment scenarios, not just development environments, to catch issues like this.
- Ensure that all functions in a library receive the same parameters, especially when dealing with offline operations.
- Custom model storage in air-gapped networks requires careful configuration to avoid such bugs.
- Contributing back to open-source projects can help improve tools for the broader community.
- Maintain backward compatibility when making changes to avoid disrupting existing workflows.
The bigger picture
The bug could lead to deployment failures in environments without internet access, such as enterprise setups. It emphasizes the need for thorough testing in production-like conditions to avoid such issues.
This article highlights a critical bug in Mistral AI's library that affects offline deployments. The fix ensures consistency in tokenizer loading, emphasizing the importance of testing in production-like environments and maintaining backward compatibility.
Technical details
- The tokenizer loading process involves two functions: list_local_hf_repo_files and hf_hub_download.
- list_local_hf_repo_files always looked in the default cache, ignoring a custom folder.
- hf_hub_download received a cache_dir parameter and looked in the custom folder.
- The fix ensures both functions read the same folder, either custom or default.
- The fix was implemented by aligning the behavior with the Hugging Face library.
FAQ
What is the cause of the tokenizer not being found in offline mode?
The tokenizer loading process involves two functions that read different folders, leading to the error.
How was the bug fixed?
The fix aligned both functions to read the same folder, either custom or default, following the same rule as Hugging Face.
Who is affected by this bug?
Teams running Mistral models offline with a custom model folder, especially in air-gapped environments.
What should users do if they cannot upgrade mistral-common?
Point HF_HUB_CACHE to the same folder as the cache_dir to ensure both steps read the same place.
Why is testing in production-like environments important?
This bug only appears in offline deployments, highlighting the need to test real-world scenarios to catch such issues.
This summary and analysis were generated with AI from the original article and may contain errors. Read the original for the full story.
Read the full story
Originally published by Dev.to AI on Oct 5, 2026.
Comments
Sign in to join the discussion.