Installation Guide

Complete guide to installing GGUF Loader on Windows, macOS, and Linux systems

Beginner 5 minutes

This guide will help you install GGUF Loader 2.0.0 on your system. Want to see what GGUF Loader can do first? Explore the features on our homepage.

🚀 Quick Installation

Using pip (Recommended)

pip install ggufloader

That’s it! You can now run GGUF Loader with:

ggufloader

📋 System Requirements

Minimum Requirements

• Python: 3.8 or higher
• RAM: 4GB (8GB+ recommended for larger models)
• Storage: 2GB free space for models
• OS: Windows 10/11, macOS 10.14+, or Linux

Recommended Requirements

• Python: 3.10 or higher
• RAM: 16GB or more
• GPU: NVIDIA GPU with CUDA support (optional but recommended)
• Storage: 10GB+ free space for multiple models

🔧 Detailed Installation

Step 1: Install Python

If you don’t have Python installed:

Windows

1. Download Python from python.org
2. Run the installer and check “Add Python to PATH”
3. Verify installation: python --version

macOS

# Using Homebrew (recommended)
brew install python

# Or download from python.org

Linux (Ubuntu/Debian)

sudo apt update
sudo apt install python3 python3-pip

Step 2: Create Virtual Environment (Recommended)

# Create virtual environment
python -m venv ggufloader-env

# Activate it
# Windows:
ggufloader-env\Scripts\activate
# macOS/Linux:
source ggufloader-env/bin/activate

Step 3: Install GGUF Loader

pip install ggufloader

Step 4: Verify Installation

ggufloader --version

🎮 First Run

Launch GGUF Loader

ggufloader

This will open the GGUF Loader application with the Smart Floating Assistant addon already loaded. You can see this feature demonstrated on the homepage features section.

Load Your First Model

1. Download a GGUF model (e.g., from Hugging Face) - Browse recommended models on our homepage
2. Click “Select GGUF Model” in the application
3. Choose your model file
4. Wait for loading (may take a few minutes)
5. Start chatting!

💡 See it in Action: Check out the model loading demonstration on our homepage to see this process in action.

🔧 GPU Acceleration (Optional)

For better performance with larger models, you can enable GPU acceleration:

NVIDIA GPU (CUDA)

# Uninstall CPU version
pip uninstall llama-cpp-python

# Install GPU version
pip install llama-cpp-python --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu121

Apple Silicon (Metal)

# Uninstall CPU version
pip uninstall llama-cpp-python

# Install Metal version
CMAKE_ARGS="-DLLAMA_METAL=on" pip install llama-cpp-python

🛠️ Advanced Installation

Development Installation

If you want to contribute or modify GGUF Loader, or develop custom addons:

# Clone the repository
git clone https://github.com/gguf-loader/gguf-loader.git
cd gguf-loader

# Install in development mode
pip install -e .

# Install development dependencies
pip install -e .[dev]

For addon development, see our Addon Development Guide and API Reference.

Custom Installation Location

# Install to specific directory
pip install --target /path/to/directory ggufloader

# Add to Python path
export PYTHONPATH="/path/to/directory:$PYTHONPATH"

🐛 Troubleshooting Installation

Common Issues

Issue: “pip not found”

# Windows
python -m pip install ggufloader

# macOS/Linux
python3 -m pip install ggufloader

Issue: “Permission denied”

# Use --user flag
pip install --user ggufloader

# Or use virtual environment (recommended)
python -m venv venv
source venv/bin/activate  # Linux/macOS
# or
venv\Scripts\activate     # Windows
pip install ggufloader

Issue: “Package not found”

# Update pip first
pip install --upgrade pip

# Then install
pip install ggufloader

Issue: “SSL Certificate error”

# Use trusted hosts
pip install --trusted-host pypi.org --trusted-host pypi.python.org ggufloader

Platform-Specific Issues

Windows

• Issue: “Microsoft Visual C++ 14.0 is required”
  • Solution: Install Microsoft C++ Build Tools

macOS

• Issue: “Command line tools not found”
  • Solution: xcode-select --install

Linux

• Issue: Missing system dependencies

  # Ubuntu/Debian
  sudo apt install build-essential python3-dev
    
  # CentOS/RHEL
  sudo yum groupinstall "Development Tools"
  sudo yum install python3-devel
  

🔄 Updating GGUF Loader

Update to Latest Version

pip install --upgrade ggufloader

Check Current Version

ggufloader --version

Downgrade if Needed

pip install ggufloader==1.0.0  # Replace with desired version

🗑️ Uninstallation

Remove GGUF Loader

pip uninstall ggufloader

Clean Up Dependencies (Optional)

# List installed packages
pip list

# Remove specific dependencies if not needed elsewhere
pip uninstall llama-cpp-python PySide6 pyautogui pyperclip

Remove Configuration Files

# Windows
rmdir /s "%APPDATA%\ggufloader"

# macOS/Linux
rm -rf ~/.config/ggufloader
rm -rf ~/.local/share/ggufloader

📦 Alternative Installation Methods

Using conda

# Create conda environment
conda create -n ggufloader python=3.10
conda activate ggufloader

# Install via pip (no conda package yet)
pip install ggufloader

Using pipx (Isolated Installation)

# Install pipx if not already installed
pip install pipx

# Install ggufloader in isolated environment
pipx install ggufloader

# Run
ggufloader

From Source

# Download source
wget https://github.com/gguf-loader/gguf-loader/archive/v2.0.0.tar.gz
tar -xzf v2.0.0.tar.gz
cd gguf-loader-2.0.0

# Install
pip install .

🎯 Next Steps

After installation:

1. Read the Quick Start Guide to get up and running
2. Explore Addon Development to create custom addons
3. Learn about the Smart Floating Assistant built-in addon
4. Join our community for support and discussions

🏠 Explore More on Homepage

• See GGUF Loader in Action - Watch live demos of key features
• Download for Your Platform - Get the right version for your system
• Browse Model Collection - Find the perfect model for your needs
• Read User Stories - See how others use GGUF Loader

💡 Need Help?

• 📖 Check the Troubleshooting Guide
• 🐛 Report installation issues
• 💬 Ask for help in discussions
• 📧 Contact us: support@ggufloader.com

Related Topics

Before proceeding, you might want to understand:

• GGUF Loader Package Structure - How GGUF Loader is organized
• System Requirements and Compatibility - Ensure your system is compatible

---

🎉 Congratulations!

You’ve successfully installed GGUF Loader! Here’s what you can do next:

Immediate Next Steps

1. Start the Quick Start Guide - Load your first model and start chatting
2. Browse the Model Library - Discover curated models for different use cases
3. Explore Features - Learn about the Smart Floating Assistant and other powerful features

Ready to Go Deeper?

• Try the Interactive Demos - Experience GGUF Loader’s features live on our homepage
• Join the Community - Share your experience and get help from other users
• Read Success Stories - Learn how others are using GGUF Loader

Welcome to GGUF Loader! 🎉

Related Documentation

• Get up and running with gguf loader 2.0.0 in just a few minutes! new to gguf loader? [check out the homepage](/ "discover gguf loader's features and capabilities") to see what makes it special. ## 🚀 installation ```bash pip install ggufloader ``` need detailed installation instructions? check out our [installation guide](/docs/installation/ "complete installation guide for all platforms") for platform-specific instructions and troubleshooting. ## 🎯 launch gguf loader ```bash ggufloader ``` this opens gguf loader with the smart floating assistant addon already loaded. ## 📥 load your first model ### step 1: get a gguf model download a gguf model from one of these sources: #### popular model sources: - **hugging face**: [https://huggingface.co/models?library=gguf](https://huggingface.co/models?library=gguf) - **thebloke's models**: search for "thebloke" on hugging face - **local models**: convert your own models to gguf format #### recommended starter models: - **small (2-4gb)**: `llama-2-7b-chat.q4_k_m.gguf` - **medium (4-8gb)**: `mistral-7b-instruct-v0.1.q5_k_m.gguf` - **large (8gb+)**: `llama-2-13b-chat.q4_k_m.gguf` 💡 **see model comparisons**: visit our [homepage model section](/#models "compare models and see performance benchmarks") to see detailed comparisons and performance benchmarks for different models. ### step 2: load the model 1. **click "select gguf model"** in the main window 2. **browse and select** your downloaded `.gguf` file 3. **wait for loading** (this may take 1-5 minutes depending on model size) 4. **look for "model ready!"** message ## 💬 test basic chat ### try your first chat 1. **type a message** in the chat input box 2. **press enter** or click "send" 3. **watch the ai respond** in real-time 4. **continue the conversation!** #### example conversations: ``` you: hello! how are you today? ai: hello! i'm doing well, thank you for asking. i'm here and ready to help you with any questions or tasks you might have. how can i assist you today? you: can you explain what gguf models are? ai: gguf (gpt-generated unified format) is a file format designed for storing large language models... ``` ## ✨ use the smart floating assistant the smart floating assistant is gguf loader's killer feature - it works across all applications! [see it in action on our homepage](/#features "watch the smart floating assistant demo") to understand its full potential. ### step 1: select text anywhere 1. **open any application** (browser, word, notepad, etc.) 2. **select some text** by highlighting it with your mouse 3. **look for the ✨ floating button** that appears near your cursor ### step 2: process the text 1. **click the ✨ button** 2. **choose an action**: - **summarize**: get a concise summary - **comment**: generate an insightful comment 3. **wait for ai processing** 4. **copy the result** and use it anywhere! #### example workflow: ``` 1. reading a long article in your browser 2. select a complex paragraph 3. click ✨ → "summarize" 4. get: "this paragraph explains that..." 5. copy and paste the summary into your notes ``` ## 🎛️ basic settings ### model settings - **temperature**: controls creativity (0.1 = focused, 1.0 = creative) - **max tokens**: maximum response length - **context size**: how much conversation history to remember ### smart assistant settings - **enable/disable**: toggle the floating assistant - **response speed**: adjust processing speed vs quality - **auto-copy**: automatically copy results to clipboard ## 🔧 common tasks ### task 1: summarize articles 1. **open an article** in your browser 2. **select the main content** 3. **click ✨ → "summarize"** 4. **get a concise summary** ### task 2: generate comments 1. **select a social media post** or forum discussion 2. **click ✨ → "comment"** 3. **get a thoughtful response** 4. **edit and post** your ai-assisted comment ### task 3: process code 1. **select code** in your ide or github 2. **click ✨ → "comment"** 3. **get code explanation** or suggestions ### task 4: email assistance 1. **select email content** 2. **click ✨ → "summarize"** for long emails 3. **click ✨ → "comment"** to draft responses ## 🎨 customization ### addon management 1. **click the addon sidebar** (left panel) 2. **view loaded addons** 3. **click addon names** to open their interfaces 4. **manage addon settings** ### themes and ui - **dark/light mode**: available in settings - **font size**: adjustable for better readability - **window layout**: resizable panels ## 🐛 quick troubleshooting ### model won't load - **check file format**: must be `.gguf` - **check file size**: ensure enough ram - **try smaller model**: start with 4gb or less ### floating assistant not working - **check model**: must have a model loaded first - **try different text**: select at least 5+ characters - **restart application**: sometimes helps with initialization ### performance issues - **close other apps**: free up ram - **use smaller model**: try q4 quantization - **enable gpu**: if you have compatible hardware ## 📚 next steps ### learn more - **[addon development](/docs/addon-development/ "create custom gguf loader addons")**: create custom addons - **[api reference](/docs/addon-api/ "complete api documentation")**: technical documentation - **[package structure](/docs/package-structure/ "understanding gguf loader's architecture")**: technical architecture ### get involved - **[github](https://github.com/gguf-loader/gguf-loader "source code and issue tracker")**: source code and issues - **[discussions](https://github.com/gguf-loader/gguf-loader/discussions "community support forum")**: community support - **[smart floater example](/docs/smart-floater-example/ "learn from the built-in addon")**: study the built-in addon ### 🏠 explore homepage features - **[interactive demos](/#features "try gguf loader features live")** - experience features before diving deeper - **[community showcase](/#community "see what others are building")** - get inspired by community projects - **[download options](/#download "get gguf loader for your platform")** - find the right version for your system ## 💡 pro tips ### efficiency tips 1. **use keyboard shortcuts**: learn the hotkeys for faster workflow 2. **bookmark good models**: keep a list of your favorite gguf models 3. **organize your workflow**: use the floating assistant for specific tasks 4. **experiment with prompts**: different phrasings get different results ### model selection tips 1. **start small**: begin with 7b parameter models 2. **match your hardware**: don't exceed your ram capacity (see [installation guide](/docs/installation/#system-requirements "system requirements and hardware recommendations")) 3. **try different quantizations**: q4_k_m is a good balance 4. **read model cards**: check hugging face for model details for more technical details about how models are loaded and managed, see the [package structure documentation](/docs/package-structure/ "technical architecture and model loading details"). ### smart assistant tips 1. **select quality text**: better input = better output 2. **use specific actions**: "summarize" vs "comment" give different results 3. **edit the results**: ai output is a starting point, not final copy 4. **practice regularly**: the more you use it, the more useful it becomes ## 🎉 you're ready! congratulations! you now know the basics of gguf loader 2.0.0. the smart floating assistant will transform how you work with text across all your applications. ### what you've learned: - ✅ how to install and launch gguf loader - ✅ how to load and use gguf models - ✅ how to use the smart floating assistant - ✅ basic troubleshooting and customization ### ready for more? - 🛠️ create addons with the [development guide](/docs/addon-development/ "complete addon development tutorial") - 📚 study the [smart floater example](/docs/smart-floater-example/ "real-world addon example with full source code") - 🔧 explore the [api reference](/docs/addon-api/ "detailed api documentation for developers") - 🤝 join our [community discussions](https://github.com/gguf-loader/gguf-loader/discussions "community support and addon sharing") --- ## 🎊 you're now a gguf loader user! congratulations on completing the quick start guide! you now have the foundation to use gguf loader effectively. ### what you've accomplished - ✅ installed and launched gguf loader - ✅ loaded your first gguf model - ✅ used the smart floating assistant - ✅ learned basic troubleshooting ### ready for the next level? now that you're comfortable with the basics, here are some exciting paths to explore: #### 🛠️ **become a power user** - **[create custom addons](/docs/addon-development/ "build your own extensions")** - extend gguf loader with your own features - **[study the smart floater code](/docs/smart-floater-example/ "learn from real examples")** - understand how the built-in addon works - **[explore the api](/docs/addon-api/ "technical documentation")** - dive into the technical details #### 🌟 **join the community** - **[share your experience](/#community "community showcase")** - tell others about your gguf loader journey - **[browse community projects](/#community "see what others built")** - get inspired by what others have created - **[contribute ideas](/#faq "feedback and suggestions")** - help shape the future of gguf loader #### 🚀 **explore advanced features** - **[try different models](/#models "model comparison")** - experiment with various models for different tasks - **[watch feature demos](/#features "interactive demonstrations")** - see advanced features in action - **[read advanced tutorials](/#how-to "step-by-step guides")** - master complex workflows **happy ai-powered text processing! 🚀** need help? contact us at support@ggufloader.com or visit our [support page](/docs/troubleshooting/).
• Troubleshooting
• User-guide

🔗 Related Features

📥

Download Center Get GGUF Loader for your platform

🤖

Model Library Find the perfect model for your needs

🎯 What's Next?

You've completed this guide! Here are some suggested next steps to continue your GGUF Loader journey:

🚀

Start Using GGUF Loader

Now that you have GGUF Loader installed, follow our Quick Start guide to load your first model and start chatting.

Quick Start Guide →

🤖

Browse Models

Explore our curated collection of GGUF models to find the perfect one for your needs.

Browse Models →

✨

Explore Features

Discover all the powerful features that make GGUF Loader special.

See Features →

🌟 Explore More Features

Interactive Demos Model Library Community Projects User Stories

🏠 Back to Homepage

✨

Explore Features See what GGUF Loader can do

📥

Download Now Get GGUF Loader for your system

🤖

Browse Models Find the perfect model for your needs

👥

Join Community Connect with other users