User Guide#

This guide covers all major user features in InferencePort AI.

Everyday Workflow#

  1. Open Chat.

  2. Select a Host and Model.

  3. Send prompts and review streamed output.

  4. Reopen older sessions from the sidebar when needed.

Chat Basics#

  1. Go to Chat.

  2. Choose your host (local or remote).

  3. Choose a model.

  4. Type a prompt and press Enter.

Helpful chat controls:

  • + New chat starts a fresh session.

  • Stop aborts an in-progress response.

  • Reset clears chat state for the current run.

  • Right-click a session for rename/delete actions.

Lightning Mode#

Chat includes a Lightning toggle in the sidebar.

  • Turn Lightning on to use Lightning model flow.

  • Turn it off to use your normal host/model selections.

Sessions#

  • Sessions are saved locally by default.

  • You can continue, rename, favorite, or delete sessions.

  • Delete All Sessions is available from the session context menu.

Marketplace Overview#

Marketplace includes:

  • Local Chat (Ollama models)

  • Hugging Face chatbot discovery/import

  • Import Model for .gguf/Modelfile

  • Hugging Face Spaces demos

  • Websites cards

  • Import .import File for supported cards/imports

Local Models (Ollama)#

  1. Open Marketplace -> Local Chat.

  2. Browse recommended or all available models.

  3. Download a model.

  4. Open Chat and select the model.

You can also:

  • Search installed and available models.

  • Remove models you no longer need.

  • Add/manage remote hosts from host controls.

Import GGUF / Modelfile#

  1. Open Marketplace.

  2. Select Import Model.

  3. Drop .gguf and/or Modelfile.

  4. Start import and wait for completion.

If your Modelfile references a GGUF, provide both files together.

Installed Page#

The Installed page helps you manage what you already added:

  • Installed chat models

  • Saved Hugging Face Spaces

  • Saved website cards

Use launch/delete actions per card as needed.

Spaces and Websites#

  • Launch Spaces from Marketplace to use hosted demos.

  • Save useful Spaces/websites for quick access.

  • Remove saved cards when you do not need them.

  • Some sites may open externally if embedding is blocked.

Account (Optional)#

You can use the app without an account.

If needed, sign in from Auth using:

  • Email + password

  • GitHub

  • Google

Settings includes account actions such as username change, account deletion, subscription management, and Lightning API key generation.

Sync (Optional)#

  1. Sign in.

  2. Open Settings.

  3. Enable chat sync.

Notes:

  • Sync is off by default.

  • Local usage works without sync.

Remote Host and Hosting (Advanced)#

Two separate advanced features are available:

  • Remote host: connect this app to a model host running elsewhere.

  • Host (Proxy) in Settings: run a local proxy server for allowed users.

For hosting/proxy mode you can:

  • Set port and allowed emails

  • Start/stop hosting

  • View server logs

Troubleshooting#

If you hit issues with install, model pulls, streaming, auth, or sync, use Troubleshooting.