Hermes Agent: From LLM to AI Agent

Before we install Hermes, we create a dedicated virtual machine for it. This way the agent runs separately from our Proxmox host and the existing LXC containers. We can also manage the VM independently of the rest of the local AI system and rebuild it at any time if needed.

Download the Debian 13 ISO

For our VM we first need the Debian 13 installation medium. We download the ISO directly to the Proxmox server.
First we open the official Debian download page and copy the download link of the desired Debian 13 ISO to the clipboard.
In the Proxmox web interface, we select the storage local (pve) on the left, where the ISO should be stored, and open:

ISO Images → Download from URL

There we enter the previously copied download address of the Debian 13 ISO, give the file a name and then start the download.
The ISO is stored directly on the Proxmox server and is then available as installation medium when creating the VM.
Alternatively, the ISO can also be downloaded directly via the terminal of the Proxmox host. For this guide, however, we use the web interface because it is easier to follow.

Create the VM for Hermes

Once the Debian 13 ISO is on our Proxmox server, we can create the virtual machine for Hermes.

In the Proxmox web interface, we click "Create VM" at the top right.

As the name we use: hermes
Under OS we select the previously downloaded Debian 13 ISO.
For the virtual hard disk we use 32 GB on local-lvm.
For the CPU we configure 1 socket with 4 cores.
We give the VM 4 GB of RAM.
For the network we use the existing bridge vmbr0. The IP address can initially be obtained automatically via DHCP.

Then we create the VM.
The virtual machine is now prepared. In the next step we install Debian 13.

Install Debian 13

Once the virtual machine has been created, we first start the VM. To do this, we select our VM hermes on the left in the Proxmox web interface and click "Start".

Then we open the console via ">_ Console". A new browser window opens with the Debian installer.

Debian installation

In the installation menu we first select:

Graphical install

Then the basic settings are requested:

• Language: English (or your preferred language)
• Location: your country
• Keyboard: your keyboard layout

Next we set the hostname:

lon3s-bot

We leave the Domain name field empty.

Then we set the root password and enter it again to confirm.

In the next step the user is created:

• Full name: Senol
• Username: hermes
• Password: your choice

The user account password can, for example, be the same as the root password. For security reasons you can of course also use a separate password.

Partition the disk

For partitioning we select:

Guided – use entire disk

As the disk we select:

SCSI1 (0,0,0) (sda) – 32.0 GB QEMU HARDDISK

Then we select:

All files in one partition

After that we select "Finish partitioning and write changes to disk".

When asked "Write the changes to disks?", we select:

Yes

Now the actual installation of Debian 13 begins. This process can take a few minutes.

Configure the package manager

After the basic system components have been installed, we are asked whether another installation medium should be scanned.

Here we select:

No

Then the package manager is set up.

When choosing the Debian mirror, we use a server close to us, for example:

debian.netcologne.de

If you are asked for an HTTP proxy, leave the field empty and confirm.

Install the desktop and SSH

Now we get to the software selection.

Here we choose XFCE as the desktop environment. XFCE is comparatively lightweight and needs considerably fewer resources than more extensive desktop environments.

In addition we select:

• XFCE
• SSH server
• Standard system utilities

Then we start installing the selected software.

Depending on the speed of your internet connection and system, this step can take a while.

Install the GRUB boot loader

After the software has been installed, the installer asks whether the GRUB boot loader should be installed.

Here we select:

Yes

As the target we select:

/dev/sda

Then we confirm with Continue.

Finish the installation

When the installation has completed, a corresponding message appears.

We click Continue and the virtual machine restarts.

After the restart, the Debian login appears.

We log in with the previously created user hermes and the corresponding password.

Update the system

After the first login we open a terminal.

First we switch to the root user with su:

su -

Then we enter the root password we set earlier.

After that we update the system:

apt update && apt upgrade -y

And install the required programs:

apt install curl git -y

Once the update and the installation of the required programs have finished, we leave the root shell again:

exit

Debian 13 is now installed, XFCE is set up, the SSH server is available and the system is up to date.
In the next step we install Hermes Agent.

Install Hermes Agent

Once Debian 13 has been set up and updated, we can install Hermes Agent.

For the installation we use the official installer from Nous Research:

curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash

The installer downloads the required components and sets up Hermes on our system.

During the installation, Hermes may offer to sign in via the Nous Portal. For our setup, however, we don't need this account, because we will later connect Hermes to our own local LLM API.

If Hermes waits for a sign-in at this point, we can cancel the process with Ctrl+C.

After this installation dialog has finished or been cancelled, we open a new terminal and start the Hermes configuration:

hermes setup

Select the terminal backend

First we select the terminal backend.

Here we use:

Local – run directly on this machine

This way Hermes runs directly on our Debian 13 VM.

Set up the LLM provider

Next, Hermes asks:

Set up a provider now?

Here we select:

Yes

This lets us configure our local LLM server right away.

So that your console knows the newly installed paths and commands, reload your Bash configuration.

source ~/.bashrc

Connection to the local LLM

As the API Base URL we enter the address of our llama-server:

http://192.168.178.136:8080/v1

Our llama-server runs in a separate Proxmox LXC container with Bonsai 2.

We leave the API Key field empty for now and confirm with Enter.

Since our local llama-server currently doesn't require authentication, we don't need an API key for the connection for now. If we want to secure access later, we can add this accordingly.

Select the API compatibility

For Select API compatibility mode we choose:

Auto detect – best for standard OpenAI-compatible endpoints

Hermes then automatically detects the appropriate API compatibility of our endpoint.

Model and context size

We leave the Model Name field empty.

We also leave Context Length empty so that Hermes can detect these values automatically.

As the Display Name we use:

local-llm

Reasoning effort

For Reasoning Effort we keep the default setting:

Medium

and confirm with Enter.

Hermes is now installed and at the same time prepared for the connection to our local LLM.

Our current configuration now looks like this:

Terminal Backend: Local – run directly on this machine
Provider: Yes
API Base URL: http://192.168.178.136:8080/v1
API Key: empty
API Compatibility: Auto detect
Model Name: empty
Context Length: empty
Display Name: local-llm
Reasoning Effort: Medium

In the next step we test whether Hermes can actually communicate with our local LLM over this connection and carry out tasks as an agent.

First tasks

Now that Hermes is installed and connected to our local LLM, we can try out the agent for the first time.

We test three typical abilities:

• Tool calling
• Working with files
• Running terminal commands

The difference from a normal chat with an LLM quickly becomes apparent. Hermes can not only generate an answer, but, depending on the task, also use tools and then process the results further.

Tool calling

First we give Hermes a simple task in which a tool should be used.

For example:

Which files are in the current directory?

Hermes can call a suitable tool for this, evaluate the output and then return the result as an answer.

This shows us the basic principle of an agent for the first time:

Task
   ↓
Hermes Agent
   ↓
LLM
   ↓
Choose tool
   ↓
Run tool
   ↓
Evaluate result
   ↓
Answer

Working with files

Next we test file access.

For example, we can ask Hermes to create a file:

Create a file named test.txt in the current directory
and write into it: Hermes works.

Then we can check whether the file was actually created.

This means Hermes can not only talk about a file, but perform a concrete action on the system.

Using the terminal

Finally we test terminal access.

A simple task is, for example:

Show me the current system time and the available disk space.

Hermes can run the corresponding terminal commands for this and then summarize the results.

With that we've tested the most important basics of our agent. Through the LLM, Hermes can not only generate text but, depending on the task, also use tools, edit files and run commands on the system.

In the next step we set up Hermes so that the agent can run permanently as a service in the background.

Set up Telegram and use Hermes from your smartphone

Now that Hermes works locally, we can also make the agent reachable via Telegram. This lets us use Hermes conveniently from a smartphone later on.

Create a Telegram bot

First we create a new bot in Telegram.

To do this, we search Telegram for:

@BotFather

There we create a new bot using the corresponding command.

Telegram then generates a bot token. We need it for the connection with Hermes.

Connect Telegram to Hermes

Now we switch back to our Hermes installation and ask Hermes to set up Telegram.

For example:

Set up Telegram for me.

Hermes then carries out the necessary configuration steps by itself.

We only need to provide the information of the Telegram bot we created earlier.

Hermes then takes care of the setup and automatically restarts the required Hermes service.

Test the connection

Once Hermes has completed the setup and restarted the service, we can open our Telegram bot on the smartphone.

We send the bot a simple message, for example:

Hi Hermes, can you hear me?

If the setup was successful, Hermes replies via Telegram.

This means we can now use our local AI agent not only directly on the Debian VM, but also reach it from a smartphone.

Hermes is therefore no longer just a local chat, but can receive tasks via Telegram as a permanently running agent and work on them with our local LLM.

Conclusion & outlook

With Hermes Agent, our local AI series has reached a point where a simple language model has turned into a real AI agent.

We started with the installation of Proxmox VE, then set up the NVIDIA GPU for an LXC container, used llama.cpp and GGUF models, set up Bonsai 2 with the PrismML fork and finally built an additional agent layer with Hermes Agent.

Step by step, we turned a normal PC into a local AI server that today can not only generate text, but also use tools, edit files, run terminal commands and be reachable from a smartphone via Telegram.

Not everything works on the first try

With a project like this, you should be prepared for the fact that not every step works on the first attempt.

For me, that is just as much a part of a homelab project as the installation itself. Drivers, dependencies, networking, containers, APIs or individual software versions can cause problems. That's exactly why I also document the errors and solutions that come up during the build.

This guide is therefore not meant to give the impression that you only have to copy a few commands and everything will then work automatically. It's precisely when building it yourself that you learn how the individual components work together.

What can the hardware do?

With our Bonsai 2 setup too, you shouldn't lose sight of the hardware being used.

Here we use an RTX 3060 with 12 GB of VRAM and run a comparatively small model on it with a context size that is acceptable for this hardware.

An RTX 3060 can be had used, or depending on the offer, for around €300. For getting started with local AI, that's an interesting option without having to invest several thousand euros in an AI server right away.

Of course, this hardware has its limits. If the performance or the available VRAM is no longer enough, you can also build a system with several GPUs. Depending on the software and model, two, three or four RTX 3060s can be used, for example. This provides considerably more GPU resources, and larger or less heavily quantized models become interesting too.

Technically, you can of course push this even further. A system with several RTX 3090s would also be possible and, with 24 GB of VRAM each, offers much more memory.

But at some point the question arises how much hardware actually makes sense for your own use case.

Four RTX 3090s offer enormous GPU resources, but at the same time come with high purchase costs, considerably higher power consumption, more waste heat and higher demands on the power supply and case.

In the end it's therefore not just a technical question, but also a question of budget and personal benefit.

For me, learning comes first

This is exactly where I see the real point of this project.

You don't have to build a high-end AI server right away. With comparatively inexpensive hardware you can already try out and understand a lot:

• How does a local LLM server work?
• What are GGUF models and quantization?
• How is a GPU made available to a container under Proxmox?
• How do different services communicate with each other via APIs?
• What distinguishes an LLM from an agent?
• What is tool calling?
• How can an agent use files and the terminal?
• How can a local AI system become reachable via Telegram?

Once you've understood these basics, you can still decide later whether more performance is needed.

Outlook

This concludes the series for now.

Step by step, a computer running Proxmox VE has become a complete local AI platform – from virtualization through the GPU and the LLM infrastructure to the AI agent.

But that doesn't mean the project is over. On the contrary: this is where the interesting part actually begins.

Many more projects can be built on top of this infrastructure. Possible examples are Immich, Jellyfin, other local AI applications, ComfyUI, additional agents or connecting different services with each other.

The existing infrastructure thus provides a good foundation for trying out further applications and finding out step by step what is possible with local AI on your own hardware.

And that's exactly what this series is about: trying things yourself, making mistakes, finding solutions and understanding along the way how the individual components work together.