# Advanced Configuration

Master sophisticated Gonzo configurations for production deployments, high-performance scenarios, and complex multi-environment setups. This guide covers performance tuning, optimization strategies, and advanced configuration patterns.

{% hint style="info" %}
**Prerequisites:** Familiarity with Configuration Files and CLI Reference recommended before diving into advanced topics.
{% endhint %}

### Performance Optimization

#### High-Volume Log Processing

**Scenario:** Handling thousands of log entries per second

```yaml
# high-volume.yml - Optimized for throughput
files:
  - "/var/log/high-traffic/*.log"
follow: true

# Large buffers for throughput
log-buffer: 50000              # Store more entries
memory-size: 200000            # Track more patterns

# Slower updates to reduce overhead
update-interval: 30s           # Update every 30 seconds

# Batch processing optimization
batch-size: 1000               # Process in large batches

# Disable expensive features if not needed
# AI analysis can be enabled on-demand with 'm' key
```

**Command-line equivalent:**

```bash
gonzo -f /var/log/high-traffic/*.log --follow \
      --log-buffer=50000 \
      --memory-size=200000 \
      --update-interval=30s
```

**Monitoring Impact:**

```bash
# Monitor Gonzo's resource usage
watch -n 1 'ps aux | grep gonzo | grep -v grep'

# Check memory consumption
top -p $(pgrep gonzo)

# Profile CPU usage
perf record -p $(pgrep gonzo) sleep 30
perf report
```

#### Resource-Constrained Systems

**Scenario:** Running on systems with limited RAM/CPU

```yaml
# low-resource.yml - Minimized footprint
files:
  - "/var/log/app.log"
follow: true

# Minimal buffers
log-buffer: 500                # Small buffer
memory-size: 2000              # Minimal pattern tracking

# Slower updates
update-interval: 10s           # Reduce CPU usage

# Conservative processing
batch-size: 50                 # Smaller batches

# Disable features that aren't critical
# Enable AI only when needed
```

**Pre-filtering for Performance:**

```bash
# Filter before Gonzo to reduce load
tail -f /var/log/app.log | grep -E "(ERROR|WARN)" | gonzo \
      --log-buffer=500 \
      --memory-size=2000 \
      --update-interval=10s
```

#### Optimizing for Different Log Volumes

{% tabs %}
{% tab title="Low Volume (<10/sec)" %}

```yaml
# Optimized for responsiveness
log-buffer: 1000               # Standard buffer
memory-size: 10000             # Standard tracking
update-interval: 1s            # Fast updates
batch-size: 50                 # Small batches for low latency
```

{% endtab %}

{% tab title="Medium Volume (10-100/sec)" %}

```yaml
# Balanced performance
log-buffer: 5000               # Larger buffer
memory-size: 25000             # More pattern capacity
update-interval: 2s            # Moderate updates
batch-size: 200                # Balanced batch size
```

{% endtab %}

{% tab title="High Volume (100-1000/sec)" %}

```yaml
# Throughput optimized
log-buffer: 20000              # Large buffer
memory-size: 100000            # Extensive tracking
update-interval: 10s           # Infrequent updates
batch-size: 500                # Large batches
```

{% endtab %}

{% tab title="Extreme Volume (>1000/sec)" %}

```yaml
# Maximum throughput
log-buffer: 50000              # Very large buffer
memory-size: 200000            # Maximum tracking
update-interval: 30s           # Minimal UI updates
batch-size: 1000               # Maximum batch size

# Consider pre-filtering or sampling
# Process only ERROR/WARN levels
# Or sample 10% of logs
```

{% endtab %}
{% endtabs %}

### Multi-Environment Configurations

#### Environment-Specific Optimization

**Development Environment:**

```yaml
# ~/.config/gonzo/dev.yml
# Optimized for fast iteration and debugging

files:
  - "logs/*.log"
  - "debug/*.log"
follow: true

# Fast, responsive
update-interval: 500ms         # Very fast updates
log-buffer: 2000              # Moderate history
memory-size: 10000            # Standard tracking

# Cost-effective AI
ai-model: "gpt-3.5-turbo"
ai-context-size: 2000

# Full visibility for debugging
show-host: true
show-service: true
verbose: false
```

**Staging Environment:**

```yaml
# ~/.config/gonzo/staging.yml
# Balanced between dev speed and prod reliability

files:
  - "/var/log/staging/app/*.log"
  - "/var/log/staging/nginx/*.log"
follow: true

# Moderate performance
update-interval: 2s
log-buffer: 5000
memory-size: 25000

# Balanced AI (test prod models)
ai-model: "gpt-4"
ai-context-size: 4000

# Production-like settings
show-host: true
show-service: true
```

**Production Environment:**

```yaml
# ~/.config/gonzo/prod.yml
# Optimized for reliability and comprehensive analysis

files:
  - "/var/log/production/app/*.log"
  - "/var/log/production/nginx/*.log"
  - "/var/log/production/db/*.log"
follow: true

# Reliable, comprehensive
update-interval: 5s            # Conservative updates
log-buffer: 20000             # Large history
memory-size: 100000           # Extensive tracking

# Best quality AI
ai-model: "gpt-4"
ai-context-size: 8000
ai-response-timeout: 90s

# Full context for incidents
show-host: true
show-service: true

# Production monitoring
# Use separate terminal or tmux for continuous monitoring
```

#### Configuration Switching

```bash
# Shell aliases for quick switching
alias gonzo-dev='gonzo --config ~/.config/gonzo/dev.yml'
alias gonzo-staging='gonzo --config ~/.config/gonzo/staging.yml'
alias gonzo-prod='gonzo --config ~/.config/gonzo/prod.yml'

# Environment-based automatic selection
GONZO_CONFIG="${ENVIRONMENT:-development}.yml"
alias gonzo='command gonzo --config ~/.config/gonzo/${GONZO_CONFIG}'

# Usage
export ENVIRONMENT=production
gonzo  # Automatically uses prod.yml
```

### Advanced Integration Patterns

#### CI/CD Pipeline Integration

**Build Analysis Configuration:**

```yaml
# ci-build-analysis.yml
# Automated analysis of build logs

files:
  - "build-logs/*.log"
  - "test-results/*.log"
follow: false                  # Batch processing

# Moderate performance for CI
update-interval: 2s
log-buffer: 5000

# Fast, cost-effective AI
ai-model: "gpt-3.5-turbo"
ai-context-size: 2000
ai-response-timeout: 30s

# CI-friendly output
test-mode: true
quiet: false
json-output: false
```

**Integration Script:**

```bash
#!/bin/bash
# ci-analyze-logs.sh
# Automated log analysis in CI/CD

set -e

LOG_DIR="build-logs"
GONZO_CONFIG="ci-build-analysis.yml"

# Run build and capture logs
build-command 2>&1 | tee "${LOG_DIR}/build.log"
BUILD_EXIT=$?

# Analyze logs with Gonzo
echo "Analyzing build logs..."
gonzo --config "${GONZO_CONFIG}" 2>&1 | tee "${LOG_DIR}/analysis.txt"

# Check for critical patterns
if grep -qi "error\|critical\|fatal" "${LOG_DIR}/build.log"; then
    echo "Critical issues found in build logs"
    exit 1
fi

exit $BUILD_EXIT
```

#### Monitoring System Integration

**Prometheus-Style Metrics:**

```yaml
# monitoring.yml
# Configuration for metrics collection

files:
  - "/var/log/app/*.log"
follow: true

# Optimized for metric collection
update-interval: 60s           # 1-minute metrics
log-buffer: 10000
memory-size: 50000

# Export metrics format
# Implement custom metrics exporter
# Count errors, warnings, patterns over time
```

**Custom Metrics Exporter:**

```bash
#!/bin/bash
# gonzo-metrics-exporter.sh
# Export Gonzo analysis as metrics

while true; do
    # Analyze logs
    gonzo --config monitoring.yml --test-mode > /tmp/gonzo-analysis.txt
    
    # Extract metrics
    ERROR_COUNT=$(grep -c "ERROR" /tmp/gonzo-analysis.txt || echo 0)
    WARN_COUNT=$(grep -c "WARN" /tmp/gonzo-analysis.txt || echo 0)
    
    # Export to metrics file
    cat > /var/lib/node_exporter/gonzo.prom << EOF
# HELP gonzo_errors_total Total error count
# TYPE gonzo_errors_total counter
gonzo_errors_total ${ERROR_COUNT}

# HELP gonzo_warnings_total Total warning count
# TYPE gonzo_warnings_total counter
gonzo_warnings_total ${WARN_COUNT}
EOF
    
    sleep 60
done
```

#### Alerting Integration

**Alert-Triggered Analysis:**

```yaml
# alert-analysis.yml
# Deep analysis when alerts trigger

files:
  - "/var/log/app/*.log"
follow: false                  # Analyze recent history

# Quick analysis
update-interval: 1s
log-buffer: 5000

# Best AI for alert context
ai-model: "gpt-4"
ai-context-size: 8000

# Generate alert context
# Use AI to explain what caused the alert
```

**Alert Handler Script:**

```bash
#!/bin/bash
# alert-handler.sh
# Triggered by monitoring alerts

ALERT_NAME="$1"
ALERT_TIME="$2"
LOG_FILE="$3"

echo "Alert: ${ALERT_NAME} at ${ALERT_TIME}"

# Analyze logs around alert time
# Extract ±15 minutes around alert
START_TIME=$(date -d "${ALERT_TIME} - 15 minutes" +%Y-%m-%dT%H:%M)
END_TIME=$(date -d "${ALERT_TIME} + 15 minutes" +%Y-%m-%dT%H:%M)

# Filter logs to timeframe
grep -E "${START_TIME}|${END_TIME}" "${LOG_FILE}" > /tmp/alert-logs.txt

# Analyze with Gonzo + AI
gonzo -f /tmp/alert-logs.txt \
      --config alert-analysis.yml \
      --ai-model="gpt-4" > /tmp/alert-analysis.txt

# Send analysis to incident channel
cat /tmp/alert-analysis.txt | send-to-slack "#incidents"
```

### Team Configuration Management

#### Shared Configuration Standards

**Base Team Configuration:**

```yaml
# team-base.yml
# Shared standards across entire team
# Version controlled in team repository

# Standard performance settings
update-interval: 2s
log-buffer: 5000
memory-size: 15000

# Team AI standards
ai-model: "gpt-3.5-turbo"      # Default cost-effective model
ai-auto-select: true           # Allow auto-upgrade
ai-context-size: 4000

# Display standards
show-service: true             # Always show service context

# Documentation
# Last updated: 2024-01-15
# Owner: DevOps Team
# Changes require PR review
```

**Personal Overrides:**

```yaml
# personal-overrides.yml
# Personal preferences layered on team standards

# Import team base (if supported)
# extends: team-base.yml

# Personal files (additional to team standards)
files:
  - "~/logs/*.log"              # Personal log directory

# Personal AI preferences
ai-model: "gpt-4"              # Personal API key, better model

# Personal display preferences
verbose: true                  # More output for learning
```

**Usage Pattern:**

```bash
# Team members create combined configs
cat team-base.yml personal-overrides.yml > ~/.config/gonzo/config.yml

# Or use layering with explicit configs
gonzo --config team-base.yml --log-buffer=10000  # CLI override
```

#### Configuration Repository Structure

```bash
# Team configuration repository
gonzo-configs/
├── README.md                   # Documentation
├── base/
│   ├── team-standard.yml      # Base team config
│   └── security-policy.yml    # Security requirements
├── environments/
│   ├── development.yml        # Dev environment
│   ├── staging.yml            # Staging environment
│   └── production.yml         # Production environment
├── profiles/
│   ├── debugging.yml          # Debugging profile
│   ├── security.yml           # Security monitoring
│   └── performance.yml        # Performance analysis
├── examples/
│   ├── kubernetes.yml         # K8s integration
│   ├── cloudwatch.yml         # AWS CloudWatch
│   └── docker.yml             # Docker containers
└── scripts/
    ├── install-configs.sh     # Install team configs
    └── validate-configs.sh    # Validate before commit
```

### Performance Monitoring

#### Benchmarking Configuration Impact

**Performance Testing Script:**

```bash
#!/bin/bash
# benchmark-config.sh
# Test configuration performance

CONFIG_FILE="$1"
TEST_LOG="test-logs/high-volume.log"
ITERATIONS=5

echo "Benchmarking configuration: ${CONFIG_FILE}"

for i in $(seq 1 $ITERATIONS); do
    echo "Run $i/${ITERATIONS}..."
    
    # Measure time and resources
    /usr/bin/time -v gonzo \
        --config "${CONFIG_FILE}" \
        -f "${TEST_LOG}" \
        --test-mode 2>&1 | tee "benchmark-run-${i}.txt"
    
    # Extract metrics
    MAX_RSS=$(grep "Maximum resident" "benchmark-run-${i}.txt" | awk '{print $6}')
    CPU_TIME=$(grep "User time" "benchmark-run-${i}.txt" | awk '{print $4}')
    
    echo "  Memory: ${MAX_RSS} KB"
    echo "  CPU Time: ${CPU_TIME}s"
done

# Calculate averages
echo "Calculating averages..."
```

#### Resource Monitoring

**Continuous Monitoring:**

```bash
#!/bin/bash
# monitor-gonzo-resources.sh
# Monitor Gonzo resource usage

LOG_FILE="gonzo-resources-$(date +%Y%m%d-%H%M%S).log"

echo "Monitoring Gonzo resources to ${LOG_FILE}"
echo "Timestamp,PID,CPU%,MEM%,VSZ,RSS" > "${LOG_FILE}"

while true; do
    GONZO_PID=$(pgrep gonzo)
    
    if [ -n "$GONZO_PID" ]; then
        ps -p $GONZO_PID -o pid,pcpu,pmem,vsz,rss --no-headers | \
        awk -v ts="$(date +%Y-%m-%d\ %H:%M:%S)" '{print ts","$1","$2","$3","$4","$5}' >> "${LOG_FILE}"
    fi
    
    sleep 1
done
```

### Advanced Troubleshooting

#### Configuration Debugging

**Verbose Configuration Analysis:**

```bash
# Show what configuration is actually being used
gonzo --config myconfig.yml --show-config

# Detailed verbose output
gonzo --config myconfig.yml --verbose --dry-run

# Test mode with logging
gonzo --config myconfig.yml --test-mode 2>&1 | tee config-test.log

# Validate configuration syntax
python3 -c "import yaml; yaml.safe_load(open('myconfig.yml'))"
```

**Common Configuration Issues:**

{% tabs %}
{% tab title="Performance Problems" %}
**Symptoms:**

* High CPU usage
* High memory consumption
* Slow UI updates

**Diagnosis:**

```bash
# Check current settings
gonzo --show-config | grep -E "(buffer|memory|interval)"

# Monitor resource usage
top -p $(pgrep gonzo)

# Profile performance
perf record -p $(pgrep gonzo) sleep 10
```

**Solutions:**

```yaml
# Reduce resource usage
log-buffer: 2000              # Smaller buffer
memory-size: 5000             # Less memory
update-interval: 10s          # Slower updates
batch-size: 100               # Smaller batches
```

{% endtab %}

{% tab title="AI Not Working" %}
**Symptoms:**

* AI analysis not available
* Model selection fails
* Timeout errors

**Diagnosis:**

```bash
# Check AI configuration
echo $OPENAI_API_KEY
echo $OPENAI_API_BASE

# Test AI connectivity
curl -H "Authorization: Bearer $OPENAI_API_KEY" \
     https://api.openai.com/v1/models

# Verbose AI debugging
gonzo --ai-model="gpt-4" --verbose
```

**Solutions:**

```yaml
# Increase timeouts
ai-response-timeout: 90s      # Longer timeout
ai-max-retries: 5             # More retries

# Use more reliable model
ai-model: "gpt-3.5-turbo"     # Faster, more reliable

# Check API key environment variable
# OPENAI_API_KEY should be set
```

{% endtab %}

{% tab title="File Access Issues" %}
**Symptoms:**

* Files not found
* Permission denied
* Glob patterns not working

**Diagnosis:**

```bash
# Check file exists
ls -la /path/to/logfile.log

# Check permissions
ls -l /path/to/logfile.log

# Test glob expansion
ls -la /var/log/*.log

# Run with verbose mode
gonzo -f /path/to/log --verbose
```

**Solutions:**

```yaml
# Use absolute paths
files:
  - "/absolute/path/to/logs/*.log"

# Check file permissions
# Run with appropriate user permissions
# sudo gonzo -f /var/log/secure.log

# Verify glob patterns work
# Test: ls -la "your-glob-pattern"
```

{% endtab %}
{% endtabs %}

### Security Hardening

#### Secure Configuration Practices

**Sensitive Data Management:**

```yaml
# secure-config.yml
# Production configuration with security focus

# File paths (no sensitive data)
files:
  - "/var/log/app/*.log"
follow: true

# Performance settings (no secrets)
update-interval: 5s
log-buffer: 10000

# AI configuration (NO API KEYS IN CONFIG FILE)
ai-model: "gpt-4"
# API key set via environment variable: OPENAI_API_KEY

# Notes:
# - NEVER commit API keys to version control
# - Use environment variables for secrets
# - Rotate API keys regularly
# - Use different keys for dev/staging/prod
```

**Environment Variable Management:**

```bash
# .env.production (NOT version controlled)
OPENAI_API_KEY=sk-prod-key-here
OPENAI_API_BASE=https://api.openai.com/v1

# Load in production
set -a
source .env.production
set +a

gonzo --config prod.yml
```

#### Audit and Compliance

**Audit Logging Configuration:**

```yaml
# audit-config.yml
# Configuration with audit requirements

# Log all analyzed files
files:
  - "/var/log/app/*.log"
  - "/var/log/audit/*.log"

# Comprehensive tracking
log-buffer: 20000             # Large history for audit
memory-size: 100000           # Extensive pattern tracking

# Detailed logging
verbose: true                 # Full audit trail
json-output: false            # Keep structured logs

# AI usage tracking
# Track all AI queries for compliance
# Implement custom AI usage logger
```

### What's Next?

You've mastered advanced Gonzo configuration! Continue exploring:

* **Integration Examples** - Apply advanced configs to real scenarios
* **API & Architecture** - Understand how Gonzo works
* **Troubleshooting** - Advanced troubleshooting techniques

Or start implementing advanced configurations:

```bash
# Create your advanced configuration
mkdir -p ~/.config/gonzo/{profiles,environments}

# Set up environment-specific configs
cp examples/prod.yml ~/.config/gonzo/environments/
cp examples/dev.yml ~/.config/gonzo/environments/

# Create performance profiles
cp examples/high-volume.yml ~/.config/gonzo/profiles/
cp examples/low-resource.yml ~/.config/gonzo/profiles/

# Use them
gonzo --config ~/.config/gonzo/environments/prod.yml
gonzo --config ~/.config/gonzo/profiles/high-volume.yml
```

***

**Master advanced configuration for production-ready Gonzo deployments!** 🚀 From performance optimization to multi-environment management, sophisticated configuration strategies ensure Gonzo scales with your needs.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.controltheory.com/backup/advanced-configuration.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.