Global Configuration Guide¶

Share provider credentials, model defaults, and execution settings across all your agents so you don't repeat them in every agent.yaml.

Quick start¶

Put shared settings in ~/.holodeck/config.yaml (user-level) or a config.yaml next to your agents (project-level), and keep secrets in a .env referenced as ${VAR}.

Precedence (highest wins): shell env > project .env > ~/.holodeck/.env; and for config values: agent.yaml > project config.yaml > ~/.holodeck/config.yaml.

# ~/.holodeck/config.yaml — shared defaults for every agent
providers:
  azure_openai:
    provider: azure_openai
    name: gpt-4o
    endpoint: ${AZURE_OPENAI_ENDPOINT}
    api_key: ${AZURE_OPENAI_API_KEY}

# ~/.holodeck/.env — secrets, never committed
AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com/
AZURE_OPENAI_API_KEY=${AZURE_OPENAI_API_KEY}

holodeck config init -g   # scaffolds ~/.holodeck/config.yaml
holodeck test agent.yaml  # agent picks up the shared providers automatically

How it works¶

A global config.yaml provides defaults that apply to all agents — provider credentials, model settings, vectorstore connections, and execution timeouts. An agent.yaml only needs to name a provider; the matching block from global config fills in the rest. Anything an agent sets explicitly overrides the global default. Provider routing is by model.provider: openai/azure_openai use the OpenAI Backend, anthropic/ollama use the Claude Backend.

Configuration Precedence¶

When multiple configuration sources define the same setting, HoloDeck applies them in priority order:

1. agent.yaml (Highest Priority)
   ├─ Explicit values in agent configuration
   │
2. Environment Variables (High Priority)
   ├─ ${VAR_NAME} patterns in agent.yaml or global config
   │
3. Project-level config.yaml (Medium Priority)
   ├─ Same directory as agent.yaml
   │
4. ~/.holodeck/config.yaml (Lowest Priority)
   └─ User home directory global defaults

Precedence Diagram¶

┌──────────────────────────────┐
│   agent.yaml explicit        │  Takes precedence
├──────────────────────────────┤
│   Environment variables      │  Used if agent.yaml absent
├──────────────────────────────┤
│  Project-level config.yaml   │  Used if env var absent
├──────────────────────────────┤
│ ~/.holodeck/config.yaml      │  Fallback default
└──────────────────────────────┘

Example¶

Global config sets defaults; agent.yaml overrides what it names explicitly:

# global config
providers:
  openai:
    name: gpt-4o-mini
    temperature: 0.7
    api_key: ${OPENAI_API_KEY}

# agent.yaml
model:
  provider: openai
  name: gpt-4o      # overrides the global name
  temperature: 0.5  # overrides the global temperature

Result: the agent uses gpt-4o at temperature 0.5, with the API key resolved from ${OPENAI_API_KEY}. Any field the agent omits (here, none) would fall back to the global default.

Inheritance Rules¶

Not all agent settings are inherited from global config. Here's what you can and cannot configure globally:

Settings That CAN Be Inherited¶

LLM provider credentials (API keys, endpoints)
Default model names and settings (temperature, max_tokens)
Vectorstore configurations
Deployment settings

Settings That CANNOT Be Inherited (Agent-Level Only)¶

response_format: Define structured output schema in each agent.yaml, not in global config
tools: Tools must be defined per agent based on agent capabilities
instructions: System prompt must be specific to each agent
evaluations: Evaluation metrics are typically agent-specific
test_cases: Test cases validate individual agent behavior

# .holodeck/config.yaml - Only shared settings here
providers:
  openai:
    api_key: ${OPENAI_API_KEY}
    organization: my-org
    temperature: 0.7

deployment:
  default_port: 8000

# agent.yaml - Agent-specific settings here
name: my-agent

model:
  provider: openai
  # Inherits temperature: 0.7 from global, can override

response_format: # Cannot be inherited, must define here
  type: object
  properties:
    answer:
      type: string

instructions: # Must be defined here
  inline: "You are a helpful assistant"

Providers Section¶

Defines LLM provider configurations with credentials and defaults.

providers:
  azure_openai:
    provider: azure_openai # Required: provider type
    name: gpt-4o # Required: model name
    temperature: 0.3 # Optional: temperature (0.0-2.0)
    max_tokens: 2048 # Optional: max tokens
    endpoint: ${AZURE_OPENAI_ENDPOINT} # Required: Azure endpoint
    api_key: ${AZURE_OPENAI_API_KEY} # Required: API key

  openai:
    provider: openai
    name: gpt-4o-mini
    temperature: 0.7
    api_key: ${OPENAI_API_KEY}
    organization: my-org # Optional

  anthropic:
    provider: anthropic
    name: claude-3-sonnet
    temperature: 0.5
    api_key: ${ANTHROPIC_API_KEY}

Provider Configuration Fields¶

Each provider must have:

provider (Required): Provider type - openai, azure_openai, or anthropic
name (Required): Model identifier (e.g., gpt-4o, claude-3-sonnet)
api_key (Required): API authentication key (use ${ENV_VAR} for environment variables)

Optional fields:

temperature (Optional): Float 0.0-2.0, defaults to provider's default
max_tokens (Optional): Maximum response length
endpoint (Required for Azure): Azure OpenAI endpoint URL
organization (Optional for OpenAI): Organization ID

Execution Section¶

Configures execution settings for agent test runs and file processing.

execution:
  file_timeout: 30 # Timeout for file processing (seconds)
  llm_timeout: 60 # Timeout for LLM API calls (seconds)
  download_timeout: 30 # Timeout for downloading files (seconds)
  cache_enabled: true # Enable caching of file downloads
  cache_dir: .holodeck_cache # Directory for cache storage
  verbose: false # Enable verbose logging
  quiet: false # Enable quiet mode

Execution Fields¶

file_timeout (Optional): Seconds to wait for file operations (default: 30)
llm_timeout (Optional): Seconds to wait for LLM API calls (default: 60)
download_timeout (Optional): Seconds to wait for file downloads (default: 30)
cache_enabled (Optional): Enable caching of downloaded files (default: true)
cache_dir (Optional): Directory for storing cached files (default: .holodeck_cache)
verbose (Optional): Enable verbose logging output (default: false)
quiet (Optional): Enable quiet mode, suppressing non-critical output (default: false)

Environment Variables¶

Replace sensitive values with environment variables using ${VAR_NAME} syntax:

providers:
  openai:
    api_key: ${OPENAI_API_KEY} # Reads from environment
    organization: my-org # Literal value

Setting Environment Variables¶

On Linux/macOS:

export OPENAI_API_KEY="sk-..."
export ANTHROPIC_API_KEY="sk-ant-..."

On Windows:

set OPENAI_API_KEY=sk-...
set ANTHROPIC_API_KEY=sk-ant-...

In .env file (automatic loading):

Create .env in project directory:

OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-ant-...

HoloDeck automatically loads .env file if present.

Variable Precedence¶

For ${VARIABLE_NAME}:

Check environment variable
Check .env file
Return empty string if not found (error at agent runtime)

File Locations¶

Default Location¶

~/.holodeck/config.yaml

On different operating systems:

Linux: /home/username/.holodeck/config.yaml
macOS: /Users/username/.holodeck/config.yaml
Windows: C:\Users\username\.holodeck\config.yaml

Custom Location (Future)¶

holodeck --config /path/to/custom.yaml ...

Complete Example¶

# config.yaml (project root or ~/.holodeck/config.yaml)

# LLM Provider Configurations
providers:
  azure_openai:
    provider: azure_openai
    name: gpt-4o
    temperature: 0.3
    max_tokens: 2048
    endpoint: ${AZURE_OPENAI_ENDPOINT}
    api_key: ${AZURE_OPENAI_API_KEY}

  openai:
    provider: openai
    name: gpt-4o-mini
    temperature: 0.7
    max_tokens: 1024
    api_key: ${OPENAI_API_KEY}
    organization: acme-corp

  anthropic:
    provider: anthropic
    name: claude-3-sonnet
    temperature: 0.5
    api_key: ${ANTHROPIC_API_KEY}

# Execution Configuration
execution:
  file_timeout: 30
  llm_timeout: 60
  download_timeout: 30
  cache_enabled: true
  cache_dir: .holodeck_cache
  verbose: false

Creating Configuration¶

You can create configuration files using the holodeck config init command or manually.

Using the CLI (Recommended)¶

The CLI provides a convenient way to initialize configuration files with default settings.

Initialize Global Configuration:

holodeck config init -g
# Creates ~/.holodeck/config.yaml

Initialize Project Configuration:

holodeck config init -p
# Creates config.yaml in the current directory

Manual Creation¶

Global config can be created at two locations with different precedence:

Project-level: config.yaml in same directory as agent.yaml (higher priority)
User-level: ~/.holodeck/config.yaml in home directory (lower priority)

Project-Level Config (Recommended for Teams)¶

Create config.yaml alongside your agents:

my-project/
├── config.yaml          # Project-specific configuration
├── agent1/
│   └── agent.yaml
└── agent2/
    └── agent.yaml

Content of config.yaml:

providers:
  azure_openai:
    provider: azure_openai
    name: gpt-4o
    api_key: ${AZURE_OPENAI_API_KEY}
    endpoint: ${AZURE_OPENAI_ENDPOINT}

execution:
  llm_timeout: 60

User-Level Config (Global Defaults)¶

Create ~/.holodeck/config.yaml in your home directory:

mkdir -p ~/.holodeck

cat > ~/.holodeck/config.yaml << 'EOF'
providers:
  azure_openai:
    provider: azure_openai
    name: gpt-4o
    api_key: ${AZURE_OPENAI_API_KEY}
    endpoint: ${AZURE_OPENAI_ENDPOINT}
EOF

Setting Environment Variables¶

export AZURE_OPENAI_API_KEY="..."
export AZURE_OPENAI_ENDPOINT="https://..."

Or in .env file at project root:

AZURE_OPENAI_API_KEY=...
AZURE_OPENAI_ENDPOINT=...

Running an Agent¶

# Uses agent.yaml in current directory by default
holodeck test

# Or specify explicit path
holodeck test agent.yaml

The agent will automatically load config from project root or ~/.holodeck/.

Troubleshooting¶

Error: "api_key not found"¶

Check global config exists at ~/.holodeck/config.yaml
Verify environment variable is set: echo $OPENAI_API_KEY
Check variable name matches in config

Error: "invalid provider"¶

Check spelling of provider in agent.yaml
Valid providers: openai, azure_openai, anthropic

Agent ignoring global config¶

Verify global config file exists
Check file permissions: ls -la ~/.holodeck/
Verify YAML syntax: cat ~/.holodeck/config.yaml

Environment variable not expanding¶

Check syntax: ${VAR_NAME} (with braces)
Verify variable exists: env | grep VAR_NAME
Note: $VAR_NAME (without braces) is not expanded

Best Practices¶

Keep Secrets Secure: Never commit API keys to version control
Use Environment Variables: Store keys in env, not YAML
Global Defaults: Use global config for shared organization settings
Per-Agent Overrides: Use agent.yaml for agent-specific settings
Don't Over-Configure: Keep global config minimal and focused
Document Settings: Add comments to explain why settings exist
Version Control: Commit config.yaml.example with placeholders, not real keys

Example: Secure Setup¶

# 1. Create project-level config
cat > config.yaml << 'EOF'
providers:
  azure_openai:
    provider: azure_openai
    name: gpt-4o
    api_key: ${AZURE_OPENAI_API_KEY}
    endpoint: ${AZURE_OPENAI_ENDPOINT}

execution:
  llm_timeout: 60
  file_timeout: 30
EOF

# 2. Create agent config
cat > agent.yaml << 'EOF'
name: my-agent

model:
  provider: azure_openai

instructions:
  inline: "You are a helpful assistant."

test_cases:
  - input: "Hello!"
    ground_truth: "Hi there! How can I help?"
    evaluations:
      - f1_score

evaluations:
  model:
    provider: azure_openai
  metrics:
    - metric: f1_score
      threshold: 0.7
EOF

# 3. Create .env file with secrets (DO NOT commit)
cat > .env << 'EOF'
AZURE_OPENAI_API_KEY=your-key-here
AZURE_OPENAI_ENDPOINT=https://your-endpoint.openai.azure.com/
EOF

# 4. Run agent (config and env automatically loaded)
holodeck test

Next Steps¶

See Agent Configuration Guide for agent-specific settings
See File References Guide for path resolution