What is GGUF?
GGUF (GPT-Generated Unified Format) is an optimized model format created for llama.cpp to enable fast local inference of large language models.
Frequently Asked Questions
Do I need Python or CLI knowledge?
No, you don't need Python or command line knowledge. GGUF Loader provides a user-friendly graphical interface.
Can I run AI models completely offline?
Yes, GGUF models can run entirely offline on your local machine without requiring internet connectivity.
What are the system requirements?
GGUF models can run on standard hardware. Smaller models work on systems with 8GB RAM, while larger models may require 16GB or more.
How do I get started?
Getting started is simple: install GGUF Loader, download a GGUF model, load it in the app, and start chatting with AI locally.
Can I create custom addons?
Yes! GGUF Loader has a powerful addon system that lets you create custom functionality with Python. Build your own AI tools and integrations.
What is the Floating Chat?
The Floating Chat is GGUF Loader's built-in addon - a Messenger-style floating button that stays on top of all windows. Click it to open a chat window connected to your loaded model.
Where can I find GGUF models?
You can download GGUF models from Hugging Face (especially TheBloke's optimized models), or convert your own models to GGUF format.