Deploying GLM-5.2 Locally or via Z.ai Cloud API & Connecting it to Hermes-Agent

//Why GLM-5.2 is an Agentic Powerhouse

GLM-5.2 is highly optimized for multi-turn tool calling and reasoning, but its raw size (800GB for full FP8) is a major local deployment bottleneck.

To run GLM-5.2 as your model backbone for Hermes-Agent, you have two radically different paths: Local Quantization (using GGUF via llama.cpp or Ollama) or Cloud API via Z.ai (formerly Zhipu AI - Z.ai platform).

//Path A: Local Deployment (No-Secrets, 100% Offline)

If you have the hardware (unified-memory Macs or multi-GPU RTX 4090/A100 clusters), running it locally gives you total privacy and zero API bills.

1. Spin Up the Local GGUF Endpoint

Hermes-Agent requires a context window of at least 64,000 tokens to prevent tool-loop memory failures. Ensure you pass the context size limit when booting your local server via llama.cpp:

./llama-cli -m glm-5.2-q4_k_m.gguf -c 65536 --port 8080 --host 0.0.0.0

2. Configure Hermes-Agent for Local

Open ~/.hermes/.env (secrets store) and add:

export CUSTOM_API_BASE="http://localhost:8080/v1"
export CUSTOM_API_KEY="ignored"
export CUSTOM_MODEL_NAME="glm-5.2"
export CUSTOM_MODEL_CONTEXT_LENGTH=65536

//Path B: Cloud Deployment via Z.ai API (Fast & Cheap)

If you don't own a DGX array or massive unified-memory hardware, the professional way to run GLM-5.2 is through the Z.ai Cloud Platform (OpenAI-compatible API).

1. Grab your Z.ai API Key

Create an account on the Z.ai Developer Platform (formerly Zhipu AI) or the global portal at z.ai to get your API credentials.

2. Configure Hermes-Agent for Z.ai Cloud

Add the credentials directly to your local ~/.hermes/.env file:

export CUSTOM_API_BASE="https://api.z.ai/api/paas/v4/"
export CUSTOM_API_KEY="your_z_ai_api_key_here"
export CUSTOM_MODEL_NAME="glm-5.2"
export CUSTOM_MODEL_CONTEXT_LENGTH=128000

Note for coding workflows: Z.ai offers a dedicated, subscription-based Coding Plan optimized for code generation. To use it, simply change the API base to:

https://api.z.ai/api/coding/paas/v4/

//Step 3: Wire the Default Provider in config.yaml

Now, tell Hermes-Agent to use your newly configured custom provider by editing your ~/.hermes/config.yaml file:

model:
  default: custom/glm-5.2
  provider: custom

//Step 4: Launch and Verify

Boot the Hermes terminal interface to verify the tool-calling loop compiles correctly and executes local commands using your GLM-5.2 backbone:

hermes --tui