How Can I Self Host the AI Starter Kit Without Docker to Enable GPU Support?

Below is a summary of what we currently know:

Official Documentation Status
The Self-hosted AI Starter Kit documentation is focused on a Docker Compose–based deployment. At this time, there isn’t an official, documented guide or set of detailed notes for implementing the kit without Docker.

Running Without Docker & GPU Considerations
If you want to take advantage of a GPU rather than running everything on the CPU (as the Docker method does by default), you would need to set up and configure each component of the kit manually. This means installing the following on your host:

• n8n: Install using the standard installation method (e.g. via npm or as a native service) rather than using the Docker image.
• Ollama: Install the native version that supports your local GPU acceleration.
• Qdrant: Install and configure as a standalone service.
• PostgreSQL: Set up and configure locally.

For GPU usage, you must ensure that the AI component(s) (such as the LLMs running via Ollama) are installed in a way that can access the GPU. This generally means confirming that:

1. The proper drivers for your GPU are installed.
2. Any required libraries or toolkits (for example, CUDA or similar, depending on the component) are installed and configured.
3. The AI software you’re using is built and/or configured to leverage GPU acceleration.

Community Notes
In the community thread you referenced (from the n8n forum), @ThatPrivacyGuy mentioned having done a non-Docker implementation, but unfortunately no detailed procedure was provided in that discussion. If you’re looking for additional details, you might consider following up on that thread or asking for more specifics directly in the forum.

Next Steps

• Manual Deployment: Investigate the native installation documentation for each component. For n8n, refer to the official installation guides (e.g. for a global installation). For other components like Ollama, consult their respective documentation regarding GPU support.
• Community Engagement: Since someone in the community has experience with this setup, additional insights might eventually be shared. Keeping an eye on or replying in the community thread could prove beneficial.

If you need guidance on the native installation of any of the kit’s components, that documentation is available separately. Let me know if you’d like more details on a specific component’s installation process.