Related Documentation

• addon development guide

  learn to create custom addons for gguf loader with examples and best practices

  advanced 15 minutes

  gguf loader’s addon system lets you extend the app without touching its source. addons are python packages dropped into the addons/ folder; the app discovers, loads, and manages them automatically.

  📦 what an addon looks like

  addons/
  └── my_addon/
      ├── __init__.py      # must expose register()
      ├── main.py          # your logic (any structure you like)
      └── ...              # widgets, resources, etc.
  

  minimum viable addon

  # addons/my_addon/__init__.py
  from pyside6.qtwidgets import qlabel
  
  def register(parent=none):
      """called by gguf loader when the addon loads."""
      return qlabel("hello from my addon!")
  

  that’s it — drop the folder in addons/, restart (or addons → refresh addons), and your widget appears in the addons menu.

  🔍 how loading works

  addonmanager (addon_manager.py):

  1. scan_addons() — lists subdirectories of the addons folder that contain __init__.py.
  2. load_addon(name, path) — dynamically imports the module (importlib.util.spec_from_file_location) and requires a register attribute.
  3. load_all_addons() — loads every addon and records success/failure in the console log.
  4. get_addon_widget(name, parent) — calls register(parent) and returns the widget.
  5. open_addon_dialog(name, parent) — shows the addon widget in a non-modal dialog from the addons menu.

  the main window calls get_addon_widget for each loaded addon at startup, so register() runs even before the user opens the menu — this is how background addons (like the floating chat button) start automatically.

  📡 talking to the app

  your register(parent) receives the parent widget. the main window exposes an addon-facing api (see the addon api reference). the robust pattern used by the built-in addon:

  def register(parent=none):
      gguf_app = none
      # 1. parent might already be the main window
      if parent and hasattr(parent, 'model') and hasattr(parent, 'model_loaded'):
          gguf_app = parent
      else:
          # 2. walk up the parent chain
          current = parent
          while current is not none:
              if hasattr(current, 'model') and hasattr(current, 'model_loaded'):
                  gguf_app = current
                  break
              current = current.parent()
          # 3. fall back to scanning top-level widgets
          if gguf_app is none:
              app = qapplication.instance()
              for widget in app.toplevelwidgets():
                  if hasattr(widget, 'model') and hasattr(widget, 'model_loaded'):
                      gguf_app = widget
                      break
      ...
  

  key api members: model (callable backend), model_loaded / model_unloaded / generation_finished / generation_error signals, addon_manager, and _floating_chat_addon.

  🤖 calling the model

  # streaming
  for token in gguf_app.model(prompt, stream=true, max_tokens=512):
      print(token["choices"][0]["text"])
  
  # or use the backend directly
  response = gguf_app.model.generate(prompt, max_tokens=512, temperature=0.7)
  

  model is none until the user loads a model — always check, and subscribe to model_loaded to react.

  🧩 widget vs. background addons

  • return a widget from register() → appears in the addons menu, opens in a dialog.
  • return none (or also keep your own top-level windows) → runs in the background. the floating chat addon returns a small status widget but drives its own always-on-top windows.

  💡 best practices

  1. guard everything — wrap registration in try/except and log failures; a broken addon must never crash the app.
  2. clean up state — if you keep references on the main window (e.g. gguf_app._my_addon = ...), stop them on app close (mainwindow.closeevent stops _floating_chat_addon — follow that pattern).
  3. use qsettings for persistence — qsettings("ggufloader", "youraddon").
  4. prefer signals — let your widgets emit signals; don’t reach into other widgets’ internals.
  5. qt-free logic — if you have non-ui logic (parsing, computation), put it in a module with no qt imports so it’s unit-testable.

  ✅ debugging

  addon load errors print to the console with a traceback (failed to load addon <name>: ...). run the app from a terminal to see them:

  python main.py
  

  📚 next steps

  • addon api reference — every hook and member
  • floating chat example — full walkthrough of the built-in addon

  related documentation

  • addon api reference

    complete api reference for developing gguf loader addons

    advanced 20 minutes

    complete reference for the hooks, signals, and objects available to gguf loader addons.

    🪝 addon contract

    an addon is a folder under addons/ with an __init__.py exposing:

    register(parent=none) -> qwidget | none

    called by addonmanager when the addon loads. returns an optional widget shown in the addons menu (opened in a non-modal dialog), or none for background-only addons.

    parameters — parent: the widget the addon is being attached to (may be the main window, a dialog, or another widget).

    return — a widget, or none.

    def register(parent=none):
        return qlabel("my addon")   # widget addon
    

    🖥️ main window api

    addons receive a reference to the main window (via parent or by scanning qapplication.instance().toplevelwidgets()). it exposes:

    properties

    | member | type | description | |—|—|—| | model | modelbackend \| none | the loaded model backend; callable. none until a model is loaded. | | chat_generator | none | legacy hook; always none. addons should call model instead. | | addon_manager | addonmanager | the addon manager instance. | | _floating_chat_addon | floatingchataddon \| none | the built-in floating chat addon (if running). |

    signals

    | signal | signature | description | |—|—|—| | model_loaded | signal(object) | emitted with the modelbackend after a model loads. | | model_unloaded | signal() | emitted after the model is released. | | generation_finished | signal() | emitted when a chat generation completes. | | generation_error | signal(str) | emitted with an error message when generation fails. | | theme_changed | signal(bool) | emitted when dark mode toggles (true = dark). |

    gguf_app.model_loaded.connect(self._on_model_loaded)
    gguf_app.generation_finished.connect(self._on_finished)
    

    🧠 modelbackend

    gguf_app.model is a core.llm.model_backend.modelbackend. it is callable and thread-safe (all access serialized through a lock).

    calling convention

    # raw llama-cpp shape (addon-compatible)
    result = gguf_app.model(prompt, stream=false, max_tokens=512, temperature=0.7)
    text = result["choices"][0]["text"]
    
    # streaming
    for token_data in gguf_app.model(prompt, stream=true, max_tokens=512):
        text = token_data["choices"][0]["text"]
    

    methods & attributes

    | member | description | |—|—| | model_path | path of the loaded gguf file. | | use_gpu | whether gpu acceleration is active. | | n_ctx | context length. | | is_loaded | true once the runtime exists. | | load() | create the underlying llama runtime (raises on failure). | | unload() | release the model and free memory. | | generate(prompt, **kwargs) | complete non-streamed response string. | | generate_stream(prompt, **kwargs) | iterator of token strings. | | __call__(prompt, **kwargs) | llama-cpp-compatible callable (stream returns token-dict generator). |

    kwargs are passed straight to llama_cpp: max_tokens, temperature, top_p, top_k, repeat_penalty, stop, etc.

    🧰 agent tools (for agent-capable addons)

    the agent engine (core/agent/) exposes sandboxed tools through toolregistry. tools are not qt-bound and are safe to call from any thread.

    tool	params	result
    list_directory	path (default .)	{status, result: [{name, type, size}]}
    read_file	path, max_size, encoding	{status, result: content, lines, encoding}
    write_file	path, content	{status, path, bytes_written}
    edit_file	path, operation (replace/insert_line/delete_line), find, replace, line_number, content	{status, operation, changes_made}
    search_files	pattern/query, path	{status, result: [paths], total_matches}

    every tool result includes status ("success" / "error") and tool_name. all paths are sandboxed to the workspace — escaping paths raise.

    from core.agent.tool_registry import create_default_registry
    registry = create_default_registry("./workspace")
    result = registry.execute("list_directory", {"path": "."})
    

    🧩 addonmanager

    method	description
    scan_addons()	{name: path} of addons with __init__.py.
    load_addon(name, path)	import + validate register; true/false.
    load_all_addons()	load all; {name: success}.
    get_addon_widget(name, parent)	widget from register(parent).
    open_addon_dialog(name, parent)	show addon in a non-modal dialog.
    get_loaded_addons()	list of successfully loaded names.

    💾 persistence

    use qt’s qsettings with a scoped org/app key to avoid collisions:

    from pyside6.qtcore import qsettings
    settings = qsettings("ggufloader", "myaddon")
    settings.setvalue("position", point)
    pos = settings.value("position", qpoint(100, 100))
    

    ⚡ lifecycle

    • register() is called at startup and on addons → refresh addons.
    • the main window’s closeevent stops the floating chat addon; follow that pattern to stop your own background objects:

      if hasattr(gguf_app, '_my_addon') and gguf_app._my_addon:
          gguf_app._my_addon.stop()
      

    📚 see also

    • addon development guide — tutorial
    • floating chat example — full real-world addon

    related documentation

    • gguf loader's addon system lets you extend the app without touching its source. addons are python packages dropped into the `addons/` folder; the app discovers, loads, and manages them automatically. ## 📦 what an addon looks like ``` addons/ └── my_addon/ ├── __init__.py # must expose register() ├── main.py # your logic (any structure you like) └── ... # widgets, resources, etc. ``` ### minimum viable addon ```python # addons/my_addon/__init__.py from pyside6.qtwidgets import qlabel def register(parent=none): """called by gguf loader when the addon loads.""" return qlabel("hello from my addon!") ``` that's it — drop the folder in `addons/`, restart (or **addons → refresh addons**), and your widget appears in the **addons** menu. ## 🔍 how loading works `addonmanager` (`addon_manager.py`): 1. `scan_addons()` — lists subdirectories of the addons folder that contain `__init__.py`. 2. `load_addon(name, path)` — dynamically imports the module (`importlib.util.spec_from_file_location`) and requires a `register` attribute. 3. `load_all_addons()` — loads every addon and records success/failure in the console log. 4. `get_addon_widget(name, parent)` — calls `register(parent)` and returns the widget. 5. `open_addon_dialog(name, parent)` — shows the addon widget in a non-modal dialog from the addons menu. the main window calls `get_addon_widget` for each loaded addon at startup, so `register()` runs even before the user opens the menu — this is how background addons (like the floating chat button) start automatically. ## 📡 talking to the app your `register(parent)` receives the parent widget. the **main window** exposes an addon-facing api (see the [addon api reference](/docs/addon-api/)). the robust pattern used by the built-in addon: ```python def register(parent=none): gguf_app = none # 1. parent might already be the main window if parent and hasattr(parent, 'model') and hasattr(parent, 'model_loaded'): gguf_app = parent else: # 2. walk up the parent chain current = parent while current is not none: if hasattr(current, 'model') and hasattr(current, 'model_loaded'): gguf_app = current break current = current.parent() # 3. fall back to scanning top-level widgets if gguf_app is none: app = qapplication.instance() for widget in app.toplevelwidgets(): if hasattr(widget, 'model') and hasattr(widget, 'model_loaded'): gguf_app = widget break ... ``` key api members: `model` (callable backend), `model_loaded` / `model_unloaded` / `generation_finished` / `generation_error` signals, `addon_manager`, and `_floating_chat_addon`. ## 🤖 calling the model ```python # streaming for token in gguf_app.model(prompt, stream=true, max_tokens=512): print(token["choices"][0]["text"]) # or use the backend directly response = gguf_app.model.generate(prompt, max_tokens=512, temperature=0.7) ``` `model` is `none` until the user loads a model — always check, and subscribe to `model_loaded` to react. ## 🧩 widget vs. background addons - **return a widget** from `register()` → appears in the addons menu, opens in a dialog. - **return `none`** (or also keep your own top-level windows) → runs in the background. the floating chat addon returns a small status widget but drives its own always-on-top windows. ## 💡 best practices 1. **guard everything** — wrap registration in `try/except` and log failures; a broken addon must never crash the app. 2. **clean up state** — if you keep references on the main window (e.g. `gguf_app._my_addon = ...`), stop them on app close (`mainwindow.closeevent` stops `_floating_chat_addon` — follow that pattern). 3. **use qsettings for persistence** — `qsettings("ggufloader", "youraddon")`. 4. **prefer signals** — let your widgets emit signals; don't reach into other widgets' internals. 5. **qt-free logic** — if you have non-ui logic (parsing, computation), put it in a module with no qt imports so it's unit-testable. ## ✅ debugging addon load errors print to the console with a traceback (`failed to load addon : ...`). run the app from a terminal to see them: ```bash python main.py ``` ## 📚 next steps - [addon api reference](/docs/addon-api/) — every hook and member - [floating chat example](/docs/floating-chat-example/) — full walkthrough of the built-in addon
    • this page walks through **floating_chat**, gguf loader's built-in addon — a facebook messenger-style floating button that opens a chat window connected to the loaded model. it's the best real-world reference for writing a background addon with its own windows. ``` addons/floating_chat/ ├── __init__.py # exposes register() ├── main.py # floatingchataddon (qobject) + register() ├── floating_button.py # floatingchatbutton - draggable always-on-top button ├── chat_window.py # floatingchatwindow - the chat ui └── status_widget.py # small sidebar widget returned by register() ``` ## 🧠 the design the addon is a `qobject` (`floatingchataddon`) that owns two windows: - **`floatingchatbutton`** — a frameless, always-on-top tool window that can be dragged anywhere and remembers its position via `qsettings`. - **`floatingchatwindow`** — a regular top-level window (`400×600`) positioned next to the button, with a message list, streaming replies, and copy all / clear buttons. ## 🪝 registration `__init__.py` simply re-exports `register`: ```python from .main import register __all__ = ['register'] ``` `register()` finds the main window (parent → parent chain → `toplevelwidgets()` scan), stops any previous instance, starts the addon, and returns a **status widget** for the sidebar: ```python def register(parent=none): gguf_app = _find_main_window(parent) # model + model_loaded check if gguf_app is none: return none if getattr(gguf_app, '_floating_chat_addon', none): gguf_app._floating_chat_addon.stop() addon = floatingchataddon(gguf_app) if addon.start(): gguf_app._floating_chat_addon = addon return floatingchatstatuswidget(addon) return none ``` the returned `floatingchatstatuswidget` shows up in the **addons** menu, and because `register()` runs at startup, the button appears automatically — no user action needed. ## 📡 connecting to the app `floatingchataddon.__init__` subscribes to model lifecycle signals: ```python if hasattr(self.gguf_app, 'model_loaded'): self.gguf_app.model_loaded.connect(self._on_model_loaded) if hasattr(self.gguf_app, 'generation_finished'): self.gguf_app.generation_finished.connect(self._on_generation_finished) if hasattr(self.gguf_app, 'generation_error'): self.gguf_app.generation_error.connect(self._on_generation_error) ``` `_on_model_loaded` flips the chat window's status to **🟢 model: ready**; generation events update the streaming bubble / error state. ## 🏃 lifecycle: start() / stop() ```python def start(self): self._floating_button = floatingchatbutton() self._floating_button.clicked.connect(self._on_button_clicked) self._load_button_position() # qsettings restore, clamped to screen self._floating_button.show() self.addon_started.emit() return true def stop(self): self._save_button_position() # persist for next session if self._chat_window: self._chat_window.close() if self._floating_button: self._floating_button.close() self.addon_stopped.emit() ``` `stop()` is idempotent and called on app close via `mainwindow.closeevent`, so there's never a dangling button. ## 🖱️ button click → window toggle the click handler distinguishes three states (a minimized window is "visible" to qt but off-screen, so it must be restored, not hidden): ```python def _on_button_clicked(self): chat = self._chat_window if chat and chat.isvisible(): if chat.isminimized(): chat.shownormal(); chat.raise_(); chat.activatewindow() else: chat.hide() else: self._show_chat_window() ``` `_show_chat_window` creates the window lazily, positions it next to the button (flipping to the left / up when it would overflow the screen's **available geometry**, which excludes taskbars and docks), then `shownormal()` + `raise_()` + `activatewindow()`. ## 🪟 window flags (the platform subtleties) ```python flags = (qt.windowtype.framelesswindowhint | qt.windowtype.windowstaysontophint | qt.windowtype.x11bypasswindowmanagerhint) # linux/x11 if sys.platform != "darwin": flags |= qt.windowtype.tool # keeps it out of the taskbar ``` - **windows/linux**: `tool` keeps the button out of the taskbar. `x11bypasswindowmanagerhint` gives true always-on-top under x11. - **macos**: the `tool` flag would turn the button into a utility window that **auto-hides when the app loses focus** — so it's deliberately dropped. trade-off: the button appears in mission control. - **wayland**: compositors don't allow floating above *other* apps' windows; the button is confined to the app window (documented, and x11 is recommended for the full experience). the chat window drops the minimize button entirely — a minimized companion window is a trap state that can't be reliably restored. ## 📌 resiliancy touches - `changeevent` bounces the button straight back to normal if a system shortcut (e.g. win+d) minimizes *every* window including the tool: ```python if event.type() == qevent.type.windowstatechange and self.isminimized(): qtimer.singleshot(0, self.shownormal) ``` - dragging clamps to `screen.availablegeometry()` (not `(0,0)`), so the button can't slide under the macos menu bar or a windows taskbar. - saved positions are clamped the same way on restore. ## 🧩 what you can reuse - the **window-finding pattern** in `register()` — works for any addon. - the **qsettings position persistence** with on-screen clamping. - the **platform-conditional window flags** — copy it for any always-on-top addon window. - the **show/hide/restore toggle** logic. ## 📚 see also - [addon development guide](/docs/addon-development/) - [addon api reference](/docs/addon-api/)
    • this page documents how the gguf loader codebase is organized. the app is a pyside6 desktop application; a single `main.py` bootstraps everything, and logic is split into `core/` (pure logic), `services/` (qt threading bridges), and `ui/` + `widgets/` (presentation). ## 🗂️ top-level layout ``` gguf-loader/ ├── main.py # entry point: logging, dll paths, qapplication, mainwindow ├── config.py # central configuration constants ├── resource_manager.py # resource/path discovery (dev, package, or frozen) ├── addon_manager.py # loads and manages addons ├── requirements.txt # python dependencies ├── build_exe.spec # pyinstaller spec used for windows & linux builds │ ├── core/ # pure, testable logic (no qt) │ ├── llm/ │ │ ├── model_backend.py # thread-safe llama-cpp-python wrapper │ │ └── prompt_builder.py # system-prompt & conversation assembly │ └── agent/ │ ├── agent_engine.py # tool-use agent loop (no qt, no llama_cpp) │ └── tool_registry.py # sandboxed workspace tools │ ├── services/ # qobject bridges that run work on threads │ ├── model_service.py # load/unload models on a qthread │ ├── chat_service.py # streaming generation on a qthread │ ├── agent_service.py # runs agentengine on a worker thread │ ├── environment_service.py # venv/dependency checks & pip tasks │ └── launcher_service.py # launches scripts/ utilities │ ├── ui/ # main window & panels │ ├── main_window.py # composition root + addon-facing api │ ├── chat_panel.py # chat display, input, agent controls │ ├── sidebar_panel.py # model settings sidebar │ └── theme.py # dark/light qss token system │ ├── widgets/ # reusable widgets │ ├── chat_bubble.py # chatgpt-style bubble │ └── feedback_dialog.py # feedback form dialog │ ├── addons/ # addon packages (each has __init__.py with register()) │ └── floating_chat/ # built-in floating chat addon │ ├── scripts/ # utility & release scripts │ ├── capture_screenshots.py # regenerates readme/site screenshots │ ├── install_linux.sh # linux installer/uninstaller │ ├── package_linux.sh # builds the linux .tar.gz release │ └── ... # gpu install/monitor helpers │ └── build_hooks/ # pyinstaller hook modules ``` ## 🔁 key design rules - **`core/` never imports qt or llama_cpp.** it receives plain callables, so it can be unit-tested in isolation. - **`services/` are the only place qt threads are created.** ui never spins up threads directly. - **`ui/` widgets are "dumb"** — they render state and emit signals; `mainwindow` owns all logic. - **`resource_manager.py`** makes paths work identically in dev, as an installed package, and in a pyinstaller bundle (`sys._meipass`). ## 🧵 threading model ``` ui thread (mainwindow) worker thread │ │ │── modelservice.load() ─────→│ qthread: llama_cpp loads model │←────── loaded(modelbackend) │ │── chatservice.generate() ──→│ qthread: streams tokens │←────── token_received(text) │ │── agentservice.process() ──→│ qthread: agentengine tool loop │←────── status/tool/response │ ``` a fresh `qthread` + worker is created per request (the professional qt pattern — `qthread` is never subclassed). ## 🚀 entry point flow `main.py` → `setup_library_path()` (finds bundled llama.cpp libs) → `qapplication` → `mainwindow()` → builds ui, wires services, checks environment, loads addons → `app.exec()`. ## 📦 packaging - **windows**: `build_exe.bat` / `build_exe.spec` → `ggufloader_vx.y.z.exe` - **linux**: build the spec inside a linux environment, then `scripts/package_linux.sh` wraps the binary + installer + icon into a `.tar.gz` - github actions publishes both to every release automatically. see the [architecture overview](/docs/architecture/) for deeper design rationale.

    🔗 related features

    👥

    community hub connect with other developers

    🛠️

    addon showcase see community creations

    🎯 what's next?

    you've completed this guide! here are some suggested next steps to continue your gguf loader journey:

    🚀

    share your addon

    built something awesome? share it with the community and get featured on our homepage.

    share with community →

    📖

    study examples

    learn from real-world addon examples and see what's possible.

    addon showcase →

    🤝

    get support

    need help with your addon? our community is here to support you.

    get help →

    🌟 explore more features

    interactive demos model library community projects user stories

    🏠 back to homepage

    ✨

    explore features see what gguf loader can do

    📥

    download now get gguf loader for your system

    🤖

    browse models find the perfect model for your needs

    👥

    join community connect with other users
  • this page walks through **floating_chat**, gguf loader's built-in addon — a facebook messenger-style floating button that opens a chat window connected to the loaded model. it's the best real-world reference for writing a background addon with its own windows. ``` addons/floating_chat/ ├── __init__.py # exposes register() ├── main.py # floatingchataddon (qobject) + register() ├── floating_button.py # floatingchatbutton - draggable always-on-top button ├── chat_window.py # floatingchatwindow - the chat ui └── status_widget.py # small sidebar widget returned by register() ``` ## 🧠 the design the addon is a `qobject` (`floatingchataddon`) that owns two windows: - **`floatingchatbutton`** — a frameless, always-on-top tool window that can be dragged anywhere and remembers its position via `qsettings`. - **`floatingchatwindow`** — a regular top-level window (`400×600`) positioned next to the button, with a message list, streaming replies, and copy all / clear buttons. ## 🪝 registration `__init__.py` simply re-exports `register`: ```python from .main import register __all__ = ['register'] ``` `register()` finds the main window (parent → parent chain → `toplevelwidgets()` scan), stops any previous instance, starts the addon, and returns a **status widget** for the sidebar: ```python def register(parent=none): gguf_app = _find_main_window(parent) # model + model_loaded check if gguf_app is none: return none if getattr(gguf_app, '_floating_chat_addon', none): gguf_app._floating_chat_addon.stop() addon = floatingchataddon(gguf_app) if addon.start(): gguf_app._floating_chat_addon = addon return floatingchatstatuswidget(addon) return none ``` the returned `floatingchatstatuswidget` shows up in the **addons** menu, and because `register()` runs at startup, the button appears automatically — no user action needed. ## 📡 connecting to the app `floatingchataddon.__init__` subscribes to model lifecycle signals: ```python if hasattr(self.gguf_app, 'model_loaded'): self.gguf_app.model_loaded.connect(self._on_model_loaded) if hasattr(self.gguf_app, 'generation_finished'): self.gguf_app.generation_finished.connect(self._on_generation_finished) if hasattr(self.gguf_app, 'generation_error'): self.gguf_app.generation_error.connect(self._on_generation_error) ``` `_on_model_loaded` flips the chat window's status to **🟢 model: ready**; generation events update the streaming bubble / error state. ## 🏃 lifecycle: start() / stop() ```python def start(self): self._floating_button = floatingchatbutton() self._floating_button.clicked.connect(self._on_button_clicked) self._load_button_position() # qsettings restore, clamped to screen self._floating_button.show() self.addon_started.emit() return true def stop(self): self._save_button_position() # persist for next session if self._chat_window: self._chat_window.close() if self._floating_button: self._floating_button.close() self.addon_stopped.emit() ``` `stop()` is idempotent and called on app close via `mainwindow.closeevent`, so there's never a dangling button. ## 🖱️ button click → window toggle the click handler distinguishes three states (a minimized window is "visible" to qt but off-screen, so it must be restored, not hidden): ```python def _on_button_clicked(self): chat = self._chat_window if chat and chat.isvisible(): if chat.isminimized(): chat.shownormal(); chat.raise_(); chat.activatewindow() else: chat.hide() else: self._show_chat_window() ``` `_show_chat_window` creates the window lazily, positions it next to the button (flipping to the left / up when it would overflow the screen's **available geometry**, which excludes taskbars and docks), then `shownormal()` + `raise_()` + `activatewindow()`. ## 🪟 window flags (the platform subtleties) ```python flags = (qt.windowtype.framelesswindowhint | qt.windowtype.windowstaysontophint | qt.windowtype.x11bypasswindowmanagerhint) # linux/x11 if sys.platform != "darwin": flags |= qt.windowtype.tool # keeps it out of the taskbar ``` - **windows/linux**: `tool` keeps the button out of the taskbar. `x11bypasswindowmanagerhint` gives true always-on-top under x11. - **macos**: the `tool` flag would turn the button into a utility window that **auto-hides when the app loses focus** — so it's deliberately dropped. trade-off: the button appears in mission control. - **wayland**: compositors don't allow floating above *other* apps' windows; the button is confined to the app window (documented, and x11 is recommended for the full experience). the chat window drops the minimize button entirely — a minimized companion window is a trap state that can't be reliably restored. ## 📌 resiliancy touches - `changeevent` bounces the button straight back to normal if a system shortcut (e.g. win+d) minimizes *every* window including the tool: ```python if event.type() == qevent.type.windowstatechange and self.isminimized(): qtimer.singleshot(0, self.shownormal) ``` - dragging clamps to `screen.availablegeometry()` (not `(0,0)`), so the button can't slide under the macos menu bar or a windows taskbar. - saved positions are clamped the same way on restore. ## 🧩 what you can reuse - the **window-finding pattern** in `register()` — works for any addon. - the **qsettings position persistence** with on-screen clamping. - the **platform-conditional window flags** — copy it for any always-on-top addon window. - the **show/hide/restore toggle** logic. ## 📚 see also - [addon development guide](/docs/addon-development/) - [addon api reference](/docs/addon-api/)
  • this page documents how the gguf loader codebase is organized. the app is a pyside6 desktop application; a single `main.py` bootstraps everything, and logic is split into `core/` (pure logic), `services/` (qt threading bridges), and `ui/` + `widgets/` (presentation). ## 🗂️ top-level layout ``` gguf-loader/ ├── main.py # entry point: logging, dll paths, qapplication, mainwindow ├── config.py # central configuration constants ├── resource_manager.py # resource/path discovery (dev, package, or frozen) ├── addon_manager.py # loads and manages addons ├── requirements.txt # python dependencies ├── build_exe.spec # pyinstaller spec used for windows & linux builds │ ├── core/ # pure, testable logic (no qt) │ ├── llm/ │ │ ├── model_backend.py # thread-safe llama-cpp-python wrapper │ │ └── prompt_builder.py # system-prompt & conversation assembly │ └── agent/ │ ├── agent_engine.py # tool-use agent loop (no qt, no llama_cpp) │ └── tool_registry.py # sandboxed workspace tools │ ├── services/ # qobject bridges that run work on threads │ ├── model_service.py # load/unload models on a qthread │ ├── chat_service.py # streaming generation on a qthread │ ├── agent_service.py # runs agentengine on a worker thread │ ├── environment_service.py # venv/dependency checks & pip tasks │ └── launcher_service.py # launches scripts/ utilities │ ├── ui/ # main window & panels │ ├── main_window.py # composition root + addon-facing api │ ├── chat_panel.py # chat display, input, agent controls │ ├── sidebar_panel.py # model settings sidebar │ └── theme.py # dark/light qss token system │ ├── widgets/ # reusable widgets │ ├── chat_bubble.py # chatgpt-style bubble │ └── feedback_dialog.py # feedback form dialog │ ├── addons/ # addon packages (each has __init__.py with register()) │ └── floating_chat/ # built-in floating chat addon │ ├── scripts/ # utility & release scripts │ ├── capture_screenshots.py # regenerates readme/site screenshots │ ├── install_linux.sh # linux installer/uninstaller │ ├── package_linux.sh # builds the linux .tar.gz release │ └── ... # gpu install/monitor helpers │ └── build_hooks/ # pyinstaller hook modules ``` ## 🔁 key design rules - **`core/` never imports qt or llama_cpp.** it receives plain callables, so it can be unit-tested in isolation. - **`services/` are the only place qt threads are created.** ui never spins up threads directly. - **`ui/` widgets are "dumb"** — they render state and emit signals; `mainwindow` owns all logic. - **`resource_manager.py`** makes paths work identically in dev, as an installed package, and in a pyinstaller bundle (`sys._meipass`). ## 🧵 threading model ``` ui thread (mainwindow) worker thread │ │ │── modelservice.load() ─────→│ qthread: llama_cpp loads model │←────── loaded(modelbackend) │ │── chatservice.generate() ──→│ qthread: streams tokens │←────── token_received(text) │ │── agentservice.process() ──→│ qthread: agentengine tool loop │←────── status/tool/response │ ``` a fresh `qthread` + worker is created per request (the professional qt pattern — `qthread` is never subclassed). ## 🚀 entry point flow `main.py` → `setup_library_path()` (finds bundled llama.cpp libs) → `qapplication` → `mainwindow()` → builds ui, wires services, checks environment, loads addons → `app.exec()`. ## 📦 packaging - **windows**: `build_exe.bat` / `build_exe.spec` → `ggufloader_vx.y.z.exe` - **linux**: build the spec inside a linux environment, then `scripts/package_linux.sh` wraps the binary + installer + icon into a `.tar.gz` - github actions publishes both to every release automatically. see the [architecture overview](/docs/architecture/) for deeper design rationale.

  🔗 related features

  👥

  community hub connect with other developers

  🛠️

  addon showcase see community creations

  🎯 what's next?

  you've completed this guide! here are some suggested next steps to continue your gguf loader journey:

  🚀

  share your addon

  built something awesome? share it with the community and get featured on our homepage.

  share with community →

  📖

  study examples

  learn from real-world addon examples and see what's possible.

  addon showcase →

  🤝

  get support

  need help with your addon? our community is here to support you.

  get help →

  🌟 explore more features

  interactive demos model library community projects user stories

  🏠 back to homepage

  ✨

  explore features see what gguf loader can do

  📥

  download now get gguf loader for your system

  🤖

  browse models find the perfect model for your needs

  👥

  join community connect with other users
• addon api reference

  complete api reference for developing gguf loader addons

  advanced 20 minutes

  complete reference for the hooks, signals, and objects available to gguf loader addons.

  🪝 addon contract

  an addon is a folder under addons/ with an __init__.py exposing:

  register(parent=none) -> qwidget | none

  called by addonmanager when the addon loads. returns an optional widget shown in the addons menu (opened in a non-modal dialog), or none for background-only addons.

  parameters — parent: the widget the addon is being attached to (may be the main window, a dialog, or another widget).

  return — a widget, or none.

  def register(parent=none):
      return qlabel("my addon")   # widget addon
  

  🖥️ main window api

  addons receive a reference to the main window (via parent or by scanning qapplication.instance().toplevelwidgets()). it exposes:

  properties

  | member | type | description | |—|—|—| | model | modelbackend \| none | the loaded model backend; callable. none until a model is loaded. | | chat_generator | none | legacy hook; always none. addons should call model instead. | | addon_manager | addonmanager | the addon manager instance. | | _floating_chat_addon | floatingchataddon \| none | the built-in floating chat addon (if running). |

  signals

  | signal | signature | description | |—|—|—| | model_loaded | signal(object) | emitted with the modelbackend after a model loads. | | model_unloaded | signal() | emitted after the model is released. | | generation_finished | signal() | emitted when a chat generation completes. | | generation_error | signal(str) | emitted with an error message when generation fails. | | theme_changed | signal(bool) | emitted when dark mode toggles (true = dark). |

  gguf_app.model_loaded.connect(self._on_model_loaded)
  gguf_app.generation_finished.connect(self._on_finished)
  

  🧠 modelbackend

  gguf_app.model is a core.llm.model_backend.modelbackend. it is callable and thread-safe (all access serialized through a lock).

  calling convention

  # raw llama-cpp shape (addon-compatible)
  result = gguf_app.model(prompt, stream=false, max_tokens=512, temperature=0.7)
  text = result["choices"][0]["text"]
  
  # streaming
  for token_data in gguf_app.model(prompt, stream=true, max_tokens=512):
      text = token_data["choices"][0]["text"]
  

  methods & attributes

  | member | description | |—|—| | model_path | path of the loaded gguf file. | | use_gpu | whether gpu acceleration is active. | | n_ctx | context length. | | is_loaded | true once the runtime exists. | | load() | create the underlying llama runtime (raises on failure). | | unload() | release the model and free memory. | | generate(prompt, **kwargs) | complete non-streamed response string. | | generate_stream(prompt, **kwargs) | iterator of token strings. | | __call__(prompt, **kwargs) | llama-cpp-compatible callable (stream returns token-dict generator). |

  kwargs are passed straight to llama_cpp: max_tokens, temperature, top_p, top_k, repeat_penalty, stop, etc.

  🧰 agent tools (for agent-capable addons)

  the agent engine (core/agent/) exposes sandboxed tools through toolregistry. tools are not qt-bound and are safe to call from any thread.

  tool	params	result
  list_directory	path (default .)	{status, result: [{name, type, size}]}
  read_file	path, max_size, encoding	{status, result: content, lines, encoding}
  write_file	path, content	{status, path, bytes_written}
  edit_file	path, operation (replace/insert_line/delete_line), find, replace, line_number, content	{status, operation, changes_made}
  search_files	pattern/query, path	{status, result: [paths], total_matches}

  every tool result includes status ("success" / "error") and tool_name. all paths are sandboxed to the workspace — escaping paths raise.

  from core.agent.tool_registry import create_default_registry
  registry = create_default_registry("./workspace")
  result = registry.execute("list_directory", {"path": "."})
  

  🧩 addonmanager

  method	description
  scan_addons()	{name: path} of addons with __init__.py.
  load_addon(name, path)	import + validate register; true/false.
  load_all_addons()	load all; {name: success}.
  get_addon_widget(name, parent)	widget from register(parent).
  open_addon_dialog(name, parent)	show addon in a non-modal dialog.
  get_loaded_addons()	list of successfully loaded names.

  💾 persistence

  use qt’s qsettings with a scoped org/app key to avoid collisions:

  from pyside6.qtcore import qsettings
  settings = qsettings("ggufloader", "myaddon")
  settings.setvalue("position", point)
  pos = settings.value("position", qpoint(100, 100))
  

  ⚡ lifecycle

  • register() is called at startup and on addons → refresh addons.
  • the main window’s closeevent stops the floating chat addon; follow that pattern to stop your own background objects:

    if hasattr(gguf_app, '_my_addon') and gguf_app._my_addon:
        gguf_app._my_addon.stop()
    

  📚 see also

  • addon development guide — tutorial
  • floating chat example — full real-world addon

  related documentation

  • gguf loader's addon system lets you extend the app without touching its source. addons are python packages dropped into the `addons/` folder; the app discovers, loads, and manages them automatically. ## 📦 what an addon looks like ``` addons/ └── my_addon/ ├── __init__.py # must expose register() ├── main.py # your logic (any structure you like) └── ... # widgets, resources, etc. ``` ### minimum viable addon ```python # addons/my_addon/__init__.py from pyside6.qtwidgets import qlabel def register(parent=none): """called by gguf loader when the addon loads.""" return qlabel("hello from my addon!") ``` that's it — drop the folder in `addons/`, restart (or **addons → refresh addons**), and your widget appears in the **addons** menu. ## 🔍 how loading works `addonmanager` (`addon_manager.py`): 1. `scan_addons()` — lists subdirectories of the addons folder that contain `__init__.py`. 2. `load_addon(name, path)` — dynamically imports the module (`importlib.util.spec_from_file_location`) and requires a `register` attribute. 3. `load_all_addons()` — loads every addon and records success/failure in the console log. 4. `get_addon_widget(name, parent)` — calls `register(parent)` and returns the widget. 5. `open_addon_dialog(name, parent)` — shows the addon widget in a non-modal dialog from the addons menu. the main window calls `get_addon_widget` for each loaded addon at startup, so `register()` runs even before the user opens the menu — this is how background addons (like the floating chat button) start automatically. ## 📡 talking to the app your `register(parent)` receives the parent widget. the **main window** exposes an addon-facing api (see the [addon api reference](/docs/addon-api/)). the robust pattern used by the built-in addon: ```python def register(parent=none): gguf_app = none # 1. parent might already be the main window if parent and hasattr(parent, 'model') and hasattr(parent, 'model_loaded'): gguf_app = parent else: # 2. walk up the parent chain current = parent while current is not none: if hasattr(current, 'model') and hasattr(current, 'model_loaded'): gguf_app = current break current = current.parent() # 3. fall back to scanning top-level widgets if gguf_app is none: app = qapplication.instance() for widget in app.toplevelwidgets(): if hasattr(widget, 'model') and hasattr(widget, 'model_loaded'): gguf_app = widget break ... ``` key api members: `model` (callable backend), `model_loaded` / `model_unloaded` / `generation_finished` / `generation_error` signals, `addon_manager`, and `_floating_chat_addon`. ## 🤖 calling the model ```python # streaming for token in gguf_app.model(prompt, stream=true, max_tokens=512): print(token["choices"][0]["text"]) # or use the backend directly response = gguf_app.model.generate(prompt, max_tokens=512, temperature=0.7) ``` `model` is `none` until the user loads a model — always check, and subscribe to `model_loaded` to react. ## 🧩 widget vs. background addons - **return a widget** from `register()` → appears in the addons menu, opens in a dialog. - **return `none`** (or also keep your own top-level windows) → runs in the background. the floating chat addon returns a small status widget but drives its own always-on-top windows. ## 💡 best practices 1. **guard everything** — wrap registration in `try/except` and log failures; a broken addon must never crash the app. 2. **clean up state** — if you keep references on the main window (e.g. `gguf_app._my_addon = ...`), stop them on app close (`mainwindow.closeevent` stops `_floating_chat_addon` — follow that pattern). 3. **use qsettings for persistence** — `qsettings("ggufloader", "youraddon")`. 4. **prefer signals** — let your widgets emit signals; don't reach into other widgets' internals. 5. **qt-free logic** — if you have non-ui logic (parsing, computation), put it in a module with no qt imports so it's unit-testable. ## ✅ debugging addon load errors print to the console with a traceback (`failed to load addon : ...`). run the app from a terminal to see them: ```bash python main.py ``` ## 📚 next steps - [addon api reference](/docs/addon-api/) — every hook and member - [floating chat example](/docs/floating-chat-example/) — full walkthrough of the built-in addon
  • this page walks through **floating_chat**, gguf loader's built-in addon — a facebook messenger-style floating button that opens a chat window connected to the loaded model. it's the best real-world reference for writing a background addon with its own windows. ``` addons/floating_chat/ ├── __init__.py # exposes register() ├── main.py # floatingchataddon (qobject) + register() ├── floating_button.py # floatingchatbutton - draggable always-on-top button ├── chat_window.py # floatingchatwindow - the chat ui └── status_widget.py # small sidebar widget returned by register() ``` ## 🧠 the design the addon is a `qobject` (`floatingchataddon`) that owns two windows: - **`floatingchatbutton`** — a frameless, always-on-top tool window that can be dragged anywhere and remembers its position via `qsettings`. - **`floatingchatwindow`** — a regular top-level window (`400×600`) positioned next to the button, with a message list, streaming replies, and copy all / clear buttons. ## 🪝 registration `__init__.py` simply re-exports `register`: ```python from .main import register __all__ = ['register'] ``` `register()` finds the main window (parent → parent chain → `toplevelwidgets()` scan), stops any previous instance, starts the addon, and returns a **status widget** for the sidebar: ```python def register(parent=none): gguf_app = _find_main_window(parent) # model + model_loaded check if gguf_app is none: return none if getattr(gguf_app, '_floating_chat_addon', none): gguf_app._floating_chat_addon.stop() addon = floatingchataddon(gguf_app) if addon.start(): gguf_app._floating_chat_addon = addon return floatingchatstatuswidget(addon) return none ``` the returned `floatingchatstatuswidget` shows up in the **addons** menu, and because `register()` runs at startup, the button appears automatically — no user action needed. ## 📡 connecting to the app `floatingchataddon.__init__` subscribes to model lifecycle signals: ```python if hasattr(self.gguf_app, 'model_loaded'): self.gguf_app.model_loaded.connect(self._on_model_loaded) if hasattr(self.gguf_app, 'generation_finished'): self.gguf_app.generation_finished.connect(self._on_generation_finished) if hasattr(self.gguf_app, 'generation_error'): self.gguf_app.generation_error.connect(self._on_generation_error) ``` `_on_model_loaded` flips the chat window's status to **🟢 model: ready**; generation events update the streaming bubble / error state. ## 🏃 lifecycle: start() / stop() ```python def start(self): self._floating_button = floatingchatbutton() self._floating_button.clicked.connect(self._on_button_clicked) self._load_button_position() # qsettings restore, clamped to screen self._floating_button.show() self.addon_started.emit() return true def stop(self): self._save_button_position() # persist for next session if self._chat_window: self._chat_window.close() if self._floating_button: self._floating_button.close() self.addon_stopped.emit() ``` `stop()` is idempotent and called on app close via `mainwindow.closeevent`, so there's never a dangling button. ## 🖱️ button click → window toggle the click handler distinguishes three states (a minimized window is "visible" to qt but off-screen, so it must be restored, not hidden): ```python def _on_button_clicked(self): chat = self._chat_window if chat and chat.isvisible(): if chat.isminimized(): chat.shownormal(); chat.raise_(); chat.activatewindow() else: chat.hide() else: self._show_chat_window() ``` `_show_chat_window` creates the window lazily, positions it next to the button (flipping to the left / up when it would overflow the screen's **available geometry**, which excludes taskbars and docks), then `shownormal()` + `raise_()` + `activatewindow()`. ## 🪟 window flags (the platform subtleties) ```python flags = (qt.windowtype.framelesswindowhint | qt.windowtype.windowstaysontophint | qt.windowtype.x11bypasswindowmanagerhint) # linux/x11 if sys.platform != "darwin": flags |= qt.windowtype.tool # keeps it out of the taskbar ``` - **windows/linux**: `tool` keeps the button out of the taskbar. `x11bypasswindowmanagerhint` gives true always-on-top under x11. - **macos**: the `tool` flag would turn the button into a utility window that **auto-hides when the app loses focus** — so it's deliberately dropped. trade-off: the button appears in mission control. - **wayland**: compositors don't allow floating above *other* apps' windows; the button is confined to the app window (documented, and x11 is recommended for the full experience). the chat window drops the minimize button entirely — a minimized companion window is a trap state that can't be reliably restored. ## 📌 resiliancy touches - `changeevent` bounces the button straight back to normal if a system shortcut (e.g. win+d) minimizes *every* window including the tool: ```python if event.type() == qevent.type.windowstatechange and self.isminimized(): qtimer.singleshot(0, self.shownormal) ``` - dragging clamps to `screen.availablegeometry()` (not `(0,0)`), so the button can't slide under the macos menu bar or a windows taskbar. - saved positions are clamped the same way on restore. ## 🧩 what you can reuse - the **window-finding pattern** in `register()` — works for any addon. - the **qsettings position persistence** with on-screen clamping. - the **platform-conditional window flags** — copy it for any always-on-top addon window. - the **show/hide/restore toggle** logic. ## 📚 see also - [addon development guide](/docs/addon-development/) - [addon api reference](/docs/addon-api/)
  • this page documents how the gguf loader codebase is organized. the app is a pyside6 desktop application; a single `main.py` bootstraps everything, and logic is split into `core/` (pure logic), `services/` (qt threading bridges), and `ui/` + `widgets/` (presentation). ## 🗂️ top-level layout ``` gguf-loader/ ├── main.py # entry point: logging, dll paths, qapplication, mainwindow ├── config.py # central configuration constants ├── resource_manager.py # resource/path discovery (dev, package, or frozen) ├── addon_manager.py # loads and manages addons ├── requirements.txt # python dependencies ├── build_exe.spec # pyinstaller spec used for windows & linux builds │ ├── core/ # pure, testable logic (no qt) │ ├── llm/ │ │ ├── model_backend.py # thread-safe llama-cpp-python wrapper │ │ └── prompt_builder.py # system-prompt & conversation assembly │ └── agent/ │ ├── agent_engine.py # tool-use agent loop (no qt, no llama_cpp) │ └── tool_registry.py # sandboxed workspace tools │ ├── services/ # qobject bridges that run work on threads │ ├── model_service.py # load/unload models on a qthread │ ├── chat_service.py # streaming generation on a qthread │ ├── agent_service.py # runs agentengine on a worker thread │ ├── environment_service.py # venv/dependency checks & pip tasks │ └── launcher_service.py # launches scripts/ utilities │ ├── ui/ # main window & panels │ ├── main_window.py # composition root + addon-facing api │ ├── chat_panel.py # chat display, input, agent controls │ ├── sidebar_panel.py # model settings sidebar │ └── theme.py # dark/light qss token system │ ├── widgets/ # reusable widgets │ ├── chat_bubble.py # chatgpt-style bubble │ └── feedback_dialog.py # feedback form dialog │ ├── addons/ # addon packages (each has __init__.py with register()) │ └── floating_chat/ # built-in floating chat addon │ ├── scripts/ # utility & release scripts │ ├── capture_screenshots.py # regenerates readme/site screenshots │ ├── install_linux.sh # linux installer/uninstaller │ ├── package_linux.sh # builds the linux .tar.gz release │ └── ... # gpu install/monitor helpers │ └── build_hooks/ # pyinstaller hook modules ``` ## 🔁 key design rules - **`core/` never imports qt or llama_cpp.** it receives plain callables, so it can be unit-tested in isolation. - **`services/` are the only place qt threads are created.** ui never spins up threads directly. - **`ui/` widgets are "dumb"** — they render state and emit signals; `mainwindow` owns all logic. - **`resource_manager.py`** makes paths work identically in dev, as an installed package, and in a pyinstaller bundle (`sys._meipass`). ## 🧵 threading model ``` ui thread (mainwindow) worker thread │ │ │── modelservice.load() ─────→│ qthread: llama_cpp loads model │←────── loaded(modelbackend) │ │── chatservice.generate() ──→│ qthread: streams tokens │←────── token_received(text) │ │── agentservice.process() ──→│ qthread: agentengine tool loop │←────── status/tool/response │ ``` a fresh `qthread` + worker is created per request (the professional qt pattern — `qthread` is never subclassed). ## 🚀 entry point flow `main.py` → `setup_library_path()` (finds bundled llama.cpp libs) → `qapplication` → `mainwindow()` → builds ui, wires services, checks environment, loads addons → `app.exec()`. ## 📦 packaging - **windows**: `build_exe.bat` / `build_exe.spec` → `ggufloader_vx.y.z.exe` - **linux**: build the spec inside a linux environment, then `scripts/package_linux.sh` wraps the binary + installer + icon into a `.tar.gz` - github actions publishes both to every release automatically. see the [architecture overview](/docs/architecture/) for deeper design rationale.

  🔗 related features

  👥

  community hub connect with other developers

  🛠️

  addon showcase see community creations

  🎯 what's next?

  you've completed this guide! here are some suggested next steps to continue your gguf loader journey:

  🚀

  share your addon

  built something awesome? share it with the community and get featured on our homepage.

  share with community →

  📖

  study examples

  learn from real-world addon examples and see what's possible.

  addon showcase →

  🤝

  get support

  need help with your addon? our community is here to support you.

  get help →

  🌟 explore more features

  interactive demos model library community projects user stories

  🏠 back to homepage

  ✨

  explore features see what gguf loader can do

  📥

  download now get gguf loader for your system

  🤖

  browse models find the perfect model for your needs

  👥

  join community connect with other users
• Get up and running with gguf loader 2.1.2 in just a few minutes! new to gguf loader? [check out the homepage](/) to see what makes it special. ## 🚀 step 1: install gguf loader follow the [installation guide](/docs/installation/) for your platform, or download directly: - **windows**: [ggufloader_v2.1.2.exe](https://github.com/ggufloader/gguf-loader/releases/download/v2.1.2/ggufloader_v2.1.2.exe) - **linux**: [ggufloader_v2.1.2_linux_x86_64.tar.gz](https://github.com/ggufloader/gguf-loader/releases/download/v2.1.2/ggufloader_v2.1.2_linux_x86_64.tar.gz) ## 📥 step 2: get a gguf model gguf loader runs any gguf-format model. good starter models on hugging face: - **small (fast)**: [llama-3-8b-instruct gguf (q4_0)](https://huggingface.co/thebloke/llama-3-8b-instruct-gguf) — ~4.7 gb, runs great on cpu - **medium**: [mistral-7b-instruct gguf (q4_k_m)](https://huggingface.co/thebloke/mistral-7b-instruct-v0.2-gguf) — ~4.1 gb - **larger**: [phi-3.5-mini-instruct gguf](https://huggingface.co/microsoft/phi-3.5-mini-instruct-gguf) or any qwen/llama gguf you like download the `.gguf` file and remember where you saved it. ## 🖥️ step 3: load the model 1. launch gguf loader. 2. in the **model settings** sidebar, choose **processing** mode — `cpu only` (default, works everywhere) or `gpu accelerated` (nvidia cuda). 3. set a **context length** (8192 is a safe default for most models; 32768 uses more ram). 4. click **load gguf model** and select your `.gguf` file. 5. wait for the status to show **"model ready!"** — the header chip turns green with the model name. ## 💬 step 4: chat type a message in the input box at the bottom and press **enter** (shift+enter inserts a newline). responses stream in as chatgpt-style bubbles — your messages on the right (amber), the ai's on the left. use the **view → text size** menu to adjust bubble font size. ## 🤖 step 5 (optional): try agent mode click **🤖 agent mode: off** in the input area to toggle it on: 1. pick a **workspace folder** (defaults to `./agent_workspace`). 2. ask the agent to do file work — e.g. *"create a file called hello.py that prints 'hi'"*. 3. the agent plans tool calls, executes them (read/write/edit/search files inside the workspace only), and reports back with live status updates. ## ✅ next steps - [user guide](/docs/user-guide/) — everything the app can do - [addon development](/docs/addon-development/) — extend gguf loader with addons