Granite 4.0 Nano: Just how small can you go?
•
121
AI & ML interests
LLMs for language and code + Time series and geospatial foundation models
Recent Activity
View all activity
Articles
frreiss
published
a
model
about 7 hours ago
frreiss
updated
a
model
about 7 hours ago
danielhanchen
posted
an
update
about 13 hours ago
Post
155
Mistral's new SOTA coding models Devstral 2 can now be Run locally! (25GB RAM) 🐱
We fixed the chat template, so performance should be much better now!
24B: unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF
123B: unsloth/Devstral-2-123B-Instruct-2512-GGUF
🧡Step-by-step Guide: https://docs.unsloth.ai/models/devstral-2
We fixed the chat template, so performance should be much better now!
24B: unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF
123B: unsloth/Devstral-2-123B-Instruct-2512-GGUF
🧡Step-by-step Guide: https://docs.unsloth.ai/models/devstral-2
alozowski
authored
a
paper
3 days ago
Add run Ollama script
2
#32 opened 3 days ago
by
kndtran
Import files
#1 opened 3 days ago
by
frreiss
danielhanchen
posted
an
update
8 days ago
Post
3372
Mistral's new Ministral 3 models can now be Run & Fine-tuned locally! (16GB RAM)
Ministral 3 have vision support and the best-in-class performance for their sizes.
14B Instruct GGUF: unsloth/Ministral-3-14B-Instruct-2512-GGUF
14B Reasoning GGUF: unsloth/Ministral-3-14B-Reasoning-2512-GGUF
🐱 Step-by-step Guide: https://docs.unsloth.ai/new/ministral-3
All GGUFs, BnB, FP8 etc. variants uploads: https://huggingface.co/collections/unsloth/ministral-3
Ministral 3 have vision support and the best-in-class performance for their sizes.
14B Instruct GGUF: unsloth/Ministral-3-14B-Instruct-2512-GGUF
14B Reasoning GGUF: unsloth/Ministral-3-14B-Reasoning-2512-GGUF
🐱 Step-by-step Guide: https://docs.unsloth.ai/new/ministral-3
All GGUFs, BnB, FP8 etc. variants uploads: https://huggingface.co/collections/unsloth/ministral-3
-
- 3 replies
Upload config.yaml
#14 opened 16 days ago
by
helen-tw
grahamwhiteuk
updated
a
Space
9 days ago
Upload config.yaml
#12 opened 15 days ago
by
helen-tw
Plan to support in Ollama
1
#13 opened 16 days ago
by
lordofthejars
danielhanchen
posted
an
update
13 days ago
Post
8294
Qwen3-Next can now be Run locally! (30GB RAM)
Instruct GGUF: unsloth/Qwen3-Next-80B-A3B-Instruct-GGUF
The models come in Thinking and Instruct versions and utilize a new architecture, allowing it to have ~10x faster inference than Qwen32B.
💜 Step-by-step Guide: https://docs.unsloth.ai/models/qwen3-next
Thinking GGUF: unsloth/Qwen3-Next-80B-A3B-Thinking-GGUF
Instruct GGUF: unsloth/Qwen3-Next-80B-A3B-Instruct-GGUF
The models come in Thinking and Instruct versions and utilize a new architecture, allowing it to have ~10x faster inference than Qwen32B.
💜 Step-by-step Guide: https://docs.unsloth.ai/models/qwen3-next
Thinking GGUF: unsloth/Qwen3-Next-80B-A3B-Thinking-GGUF
Update granite_geospatial_uki_flood_detection_v1.ckpt
#11 opened 16 days ago
by
rosielickorish
Update granite_geospatial_uki_flood_detection_v1.ckpt
#10 opened 16 days ago
by
rosielickorish
Update granite_geospatial_uki_flood_detection_v2.ckpt
#13 opened 16 days ago
by
rosielickorish
Upload granite_geospatial_uki_flood_detection_v2.ckpt
#12 opened 16 days ago
by
romeokienzler
Upload granite_geospatial_uki_flood_detection_v1.ckpt
1
#9 opened 17 days ago
by
helen-tw
danielhanchen
posted
an
update
about 1 month ago
Post
4236
You can now run Kimi K2 Thinking locally with our Dynamic 1-bit GGUFs:
unsloth/Kimi-K2-Thinking-GGUF
We shrank the 1T model to 245GB (-62%) & retained ~85% of accuracy on Aider Polyglot. Run on >247GB RAM for fast inference.
We also collaborated with the Moonshot AI Kimi team on a system prompt fix! 🥰
Guide + fix details: https://docs.unsloth.ai/models/kimi-k2-thinking-how-to-run-locally
We shrank the 1T model to 245GB (-62%) & retained ~85% of accuracy on Aider Polyglot. Run on >247GB RAM for fast inference.
We also collaborated with the Moonshot AI Kimi team on a system prompt fix! 🥰
Guide + fix details: https://docs.unsloth.ai/models/kimi-k2-thinking-how-to-run-locally
ariG23498
authored
a
paper
about 2 months ago
ashish23
authored
a
paper
2 months ago