Advanced Configuration

Master sophisticated Gonzo configurations for production deployments, high-performance scenarios, and complex multi-environment setups. This guide covers performance tuning, optimization strategies, and advanced configuration patterns.

Prerequisites: Familiarity with Configuration Files and CLI Reference recommended before diving into advanced topics.

Performance Optimization

High-Volume Log Processing

Scenario: Handling thousands of log entries per second

# high-volume.yml - Optimized for throughput
files:
  - "/var/log/high-traffic/*.log"
follow: true

# Large buffers for throughput
log-buffer: 50000              # Store more entries
memory-size: 200000            # Track more patterns

# Slower updates to reduce overhead
update-interval: 30s           # Update every 30 seconds

# Batch processing optimization
batch-size: 1000               # Process in large batches

# Disable expensive features if not needed
# AI analysis can be enabled on-demand with 'm' key

Command-line equivalent:

gonzo -f /var/log/high-traffic/*.log --follow \
      --log-buffer=50000 \
      --memory-size=200000 \
      --update-interval=30s

Monitoring Impact:

# Monitor Gonzo's resource usage
watch -n 1 'ps aux | grep gonzo | grep -v grep'

# Check memory consumption
top -p $(pgrep gonzo)

# Profile CPU usage
perf record -p $(pgrep gonzo) sleep 30
perf report

Resource-Constrained Systems

Scenario: Running on systems with limited RAM/CPU

# low-resource.yml - Minimized footprint
files:
  - "/var/log/app.log"
follow: true

# Minimal buffers
log-buffer: 500                # Small buffer
memory-size: 2000              # Minimal pattern tracking

# Slower updates
update-interval: 10s           # Reduce CPU usage

# Conservative processing
batch-size: 50                 # Smaller batches

# Disable features that aren't critical
# Enable AI only when needed

Pre-filtering for Performance:

# Filter before Gonzo to reduce load
tail -f /var/log/app.log | grep -E "(ERROR|WARN)" | gonzo \
      --log-buffer=500 \
      --memory-size=2000 \
      --update-interval=10s

Optimizing for Different Log Volumes

# Optimized for responsiveness
log-buffer: 1000               # Standard buffer
memory-size: 10000             # Standard tracking
update-interval: 1s            # Fast updates
batch-size: 50                 # Small batches for low latency

# Balanced performance
log-buffer: 5000               # Larger buffer
memory-size: 25000             # More pattern capacity
update-interval: 2s            # Moderate updates
batch-size: 200                # Balanced batch size

# Throughput optimized
log-buffer: 20000              # Large buffer
memory-size: 100000            # Extensive tracking
update-interval: 10s           # Infrequent updates
batch-size: 500                # Large batches

# Maximum throughput
log-buffer: 50000              # Very large buffer
memory-size: 200000            # Maximum tracking
update-interval: 30s           # Minimal UI updates
batch-size: 1000               # Maximum batch size

# Consider pre-filtering or sampling
# Process only ERROR/WARN levels
# Or sample 10% of logs

Multi-Environment Configurations

Environment-Specific Optimization

Development Environment:

# ~/.config/gonzo/dev.yml
# Optimized for fast iteration and debugging

files:
  - "logs/*.log"
  - "debug/*.log"
follow: true

# Fast, responsive
update-interval: 500ms         # Very fast updates
log-buffer: 2000              # Moderate history
memory-size: 10000            # Standard tracking

# Cost-effective AI
ai-model: "gpt-3.5-turbo"
ai-context-size: 2000

# Full visibility for debugging
show-host: true
show-service: true
verbose: false

Staging Environment:

# ~/.config/gonzo/staging.yml
# Balanced between dev speed and prod reliability

files:
  - "/var/log/staging/app/*.log"
  - "/var/log/staging/nginx/*.log"
follow: true

# Moderate performance
update-interval: 2s
log-buffer: 5000
memory-size: 25000

# Balanced AI (test prod models)
ai-model: "gpt-4"
ai-context-size: 4000

# Production-like settings
show-host: true
show-service: true

Production Environment:

# ~/.config/gonzo/prod.yml
# Optimized for reliability and comprehensive analysis

files:
  - "/var/log/production/app/*.log"
  - "/var/log/production/nginx/*.log"
  - "/var/log/production/db/*.log"
follow: true

# Reliable, comprehensive
update-interval: 5s            # Conservative updates
log-buffer: 20000             # Large history
memory-size: 100000           # Extensive tracking

# Best quality AI
ai-model: "gpt-4"
ai-context-size: 8000
ai-response-timeout: 90s

# Full context for incidents
show-host: true
show-service: true

# Production monitoring
# Use separate terminal or tmux for continuous monitoring

Configuration Switching

# Shell aliases for quick switching
alias gonzo-dev='gonzo --config ~/.config/gonzo/dev.yml'
alias gonzo-staging='gonzo --config ~/.config/gonzo/staging.yml'
alias gonzo-prod='gonzo --config ~/.config/gonzo/prod.yml'

# Environment-based automatic selection
GONZO_CONFIG="${ENVIRONMENT:-development}.yml"
alias gonzo='command gonzo --config ~/.config/gonzo/${GONZO_CONFIG}'

# Usage
export ENVIRONMENT=production
gonzo  # Automatically uses prod.yml

Advanced Integration Patterns

CI/CD Pipeline Integration

Build Analysis Configuration:

# ci-build-analysis.yml
# Automated analysis of build logs

files:
  - "build-logs/*.log"
  - "test-results/*.log"
follow: false                  # Batch processing

# Moderate performance for CI
update-interval: 2s
log-buffer: 5000

# Fast, cost-effective AI
ai-model: "gpt-3.5-turbo"
ai-context-size: 2000
ai-response-timeout: 30s

# CI-friendly output
test-mode: true
quiet: false
json-output: false

Integration Script:

#!/bin/bash
# ci-analyze-logs.sh
# Automated log analysis in CI/CD

set -e

LOG_DIR="build-logs"
GONZO_CONFIG="ci-build-analysis.yml"

# Run build and capture logs
build-command 2>&1 | tee "${LOG_DIR}/build.log"
BUILD_EXIT=$?

# Analyze logs with Gonzo
echo "Analyzing build logs..."
gonzo --config "${GONZO_CONFIG}" 2>&1 | tee "${LOG_DIR}/analysis.txt"

# Check for critical patterns
if grep -qi "error\|critical\|fatal" "${LOG_DIR}/build.log"; then
    echo "Critical issues found in build logs"
    exit 1
fi

exit $BUILD_EXIT

Monitoring System Integration

Prometheus-Style Metrics:

# monitoring.yml
# Configuration for metrics collection

files:
  - "/var/log/app/*.log"
follow: true

# Optimized for metric collection
update-interval: 60s           # 1-minute metrics
log-buffer: 10000
memory-size: 50000

# Export metrics format
# Implement custom metrics exporter
# Count errors, warnings, patterns over time

Custom Metrics Exporter:

#!/bin/bash
# gonzo-metrics-exporter.sh
# Export Gonzo analysis as metrics

while true; do
    # Analyze logs
    gonzo --config monitoring.yml --test-mode > /tmp/gonzo-analysis.txt
    
    # Extract metrics
    ERROR_COUNT=$(grep -c "ERROR" /tmp/gonzo-analysis.txt || echo 0)
    WARN_COUNT=$(grep -c "WARN" /tmp/gonzo-analysis.txt || echo 0)
    
    # Export to metrics file
    cat > /var/lib/node_exporter/gonzo.prom << EOF
# HELP gonzo_errors_total Total error count
# TYPE gonzo_errors_total counter
gonzo_errors_total ${ERROR_COUNT}

# HELP gonzo_warnings_total Total warning count
# TYPE gonzo_warnings_total counter
gonzo_warnings_total ${WARN_COUNT}
EOF
    
    sleep 60
done

Alerting Integration

Alert-Triggered Analysis:

# alert-analysis.yml
# Deep analysis when alerts trigger

files:
  - "/var/log/app/*.log"
follow: false                  # Analyze recent history

# Quick analysis
update-interval: 1s
log-buffer: 5000

# Best AI for alert context
ai-model: "gpt-4"
ai-context-size: 8000

# Generate alert context
# Use AI to explain what caused the alert

Alert Handler Script:

#!/bin/bash
# alert-handler.sh
# Triggered by monitoring alerts

ALERT_NAME="$1"
ALERT_TIME="$2"
LOG_FILE="$3"

echo "Alert: ${ALERT_NAME} at ${ALERT_TIME}"

# Analyze logs around alert time
# Extract ±15 minutes around alert
START_TIME=$(date -d "${ALERT_TIME} - 15 minutes" +%Y-%m-%dT%H:%M)
END_TIME=$(date -d "${ALERT_TIME} + 15 minutes" +%Y-%m-%dT%H:%M)

# Filter logs to timeframe
grep -E "${START_TIME}|${END_TIME}" "${LOG_FILE}" > /tmp/alert-logs.txt

# Analyze with Gonzo + AI
gonzo -f /tmp/alert-logs.txt \
      --config alert-analysis.yml \
      --ai-model="gpt-4" > /tmp/alert-analysis.txt

# Send analysis to incident channel
cat /tmp/alert-analysis.txt | send-to-slack "#incidents"

Team Configuration Management

Shared Configuration Standards

Base Team Configuration:

# team-base.yml
# Shared standards across entire team
# Version controlled in team repository

# Standard performance settings
update-interval: 2s
log-buffer: 5000
memory-size: 15000

# Team AI standards
ai-model: "gpt-3.5-turbo"      # Default cost-effective model
ai-auto-select: true           # Allow auto-upgrade
ai-context-size: 4000

# Display standards
show-service: true             # Always show service context

# Documentation
# Last updated: 2024-01-15
# Owner: DevOps Team
# Changes require PR review

Personal Overrides:

# personal-overrides.yml
# Personal preferences layered on team standards

# Import team base (if supported)
# extends: team-base.yml

# Personal files (additional to team standards)
files:
  - "~/logs/*.log"              # Personal log directory

# Personal AI preferences
ai-model: "gpt-4"              # Personal API key, better model

# Personal display preferences
verbose: true                  # More output for learning

Usage Pattern:

# Team members create combined configs
cat team-base.yml personal-overrides.yml > ~/.config/gonzo/config.yml

# Or use layering with explicit configs
gonzo --config team-base.yml --log-buffer=10000  # CLI override

Configuration Repository Structure

# Team configuration repository
gonzo-configs/
├── README.md                   # Documentation
├── base/
│   ├── team-standard.yml      # Base team config
│   └── security-policy.yml    # Security requirements
├── environments/
│   ├── development.yml        # Dev environment
│   ├── staging.yml            # Staging environment
│   └── production.yml         # Production environment
├── profiles/
│   ├── debugging.yml          # Debugging profile
│   ├── security.yml           # Security monitoring
│   └── performance.yml        # Performance analysis
├── examples/
│   ├── kubernetes.yml         # K8s integration
│   ├── cloudwatch.yml         # AWS CloudWatch
│   └── docker.yml             # Docker containers
└── scripts/
    ├── install-configs.sh     # Install team configs
    └── validate-configs.sh    # Validate before commit

Performance Monitoring

Benchmarking Configuration Impact

Performance Testing Script:

#!/bin/bash
# benchmark-config.sh
# Test configuration performance

CONFIG_FILE="$1"
TEST_LOG="test-logs/high-volume.log"
ITERATIONS=5

echo "Benchmarking configuration: ${CONFIG_FILE}"

for i in $(seq 1 $ITERATIONS); do
    echo "Run $i/${ITERATIONS}..."
    
    # Measure time and resources
    /usr/bin/time -v gonzo \
        --config "${CONFIG_FILE}" \
        -f "${TEST_LOG}" \
        --test-mode 2>&1 | tee "benchmark-run-${i}.txt"
    
    # Extract metrics
    MAX_RSS=$(grep "Maximum resident" "benchmark-run-${i}.txt" | awk '{print $6}')
    CPU_TIME=$(grep "User time" "benchmark-run-${i}.txt" | awk '{print $4}')
    
    echo "  Memory: ${MAX_RSS} KB"
    echo "  CPU Time: ${CPU_TIME}s"
done

# Calculate averages
echo "Calculating averages..."

Resource Monitoring

Continuous Monitoring:

#!/bin/bash
# monitor-gonzo-resources.sh
# Monitor Gonzo resource usage

LOG_FILE="gonzo-resources-$(date +%Y%m%d-%H%M%S).log"

echo "Monitoring Gonzo resources to ${LOG_FILE}"
echo "Timestamp,PID,CPU%,MEM%,VSZ,RSS" > "${LOG_FILE}"

while true; do
    GONZO_PID=$(pgrep gonzo)
    
    if [ -n "$GONZO_PID" ]; then
        ps -p $GONZO_PID -o pid,pcpu,pmem,vsz,rss --no-headers | \
        awk -v ts="$(date +%Y-%m-%d\ %H:%M:%S)" '{print ts","$1","$2","$3","$4","$5}' >> "${LOG_FILE}"
    fi
    
    sleep 1
done

Advanced Troubleshooting

Configuration Debugging

Verbose Configuration Analysis:

# Show what configuration is actually being used
gonzo --config myconfig.yml --show-config

# Detailed verbose output
gonzo --config myconfig.yml --verbose --dry-run

# Test mode with logging
gonzo --config myconfig.yml --test-mode 2>&1 | tee config-test.log

# Validate configuration syntax
python3 -c "import yaml; yaml.safe_load(open('myconfig.yml'))"

Common Configuration Issues:

Symptoms:

High CPU usage
High memory consumption
Slow UI updates

Diagnosis:

# Check current settings
gonzo --show-config | grep -E "(buffer|memory|interval)"

# Monitor resource usage
top -p $(pgrep gonzo)

# Profile performance
perf record -p $(pgrep gonzo) sleep 10

Solutions:

# Reduce resource usage
log-buffer: 2000              # Smaller buffer
memory-size: 5000             # Less memory
update-interval: 10s          # Slower updates
batch-size: 100               # Smaller batches

Symptoms:

AI analysis not available
Model selection fails
Timeout errors

Diagnosis:

# Check AI configuration
echo $OPENAI_API_KEY
echo $OPENAI_API_BASE

# Test AI connectivity
curl -H "Authorization: Bearer $OPENAI_API_KEY" \
     https://api.openai.com/v1/models

# Verbose AI debugging
gonzo --ai-model="gpt-4" --verbose

Solutions:

# Increase timeouts
ai-response-timeout: 90s      # Longer timeout
ai-max-retries: 5             # More retries

# Use more reliable model
ai-model: "gpt-3.5-turbo"     # Faster, more reliable

# Check API key environment variable
# OPENAI_API_KEY should be set

Symptoms:

Files not found
Permission denied
Glob patterns not working

Diagnosis:

# Check file exists
ls -la /path/to/logfile.log

# Check permissions
ls -l /path/to/logfile.log

# Test glob expansion
ls -la /var/log/*.log

# Run with verbose mode
gonzo -f /path/to/log --verbose

Solutions:

# Use absolute paths
files:
  - "/absolute/path/to/logs/*.log"

# Check file permissions
# Run with appropriate user permissions
# sudo gonzo -f /var/log/secure.log

# Verify glob patterns work
# Test: ls -la "your-glob-pattern"

Security Hardening

Secure Configuration Practices

Sensitive Data Management:

# secure-config.yml
# Production configuration with security focus

# File paths (no sensitive data)
files:
  - "/var/log/app/*.log"
follow: true

# Performance settings (no secrets)
update-interval: 5s
log-buffer: 10000

# AI configuration (NO API KEYS IN CONFIG FILE)
ai-model: "gpt-4"
# API key set via environment variable: OPENAI_API_KEY

# Notes:
# - NEVER commit API keys to version control
# - Use environment variables for secrets
# - Rotate API keys regularly
# - Use different keys for dev/staging/prod

Environment Variable Management:

# .env.production (NOT version controlled)
OPENAI_API_KEY=sk-prod-key-here
OPENAI_API_BASE=https://api.openai.com/v1

# Load in production
set -a
source .env.production
set +a

gonzo --config prod.yml

Audit and Compliance

Audit Logging Configuration:

# audit-config.yml
# Configuration with audit requirements

# Log all analyzed files
files:
  - "/var/log/app/*.log"
  - "/var/log/audit/*.log"

# Comprehensive tracking
log-buffer: 20000             # Large history for audit
memory-size: 100000           # Extensive pattern tracking

# Detailed logging
verbose: true                 # Full audit trail
json-output: false            # Keep structured logs

# AI usage tracking
# Track all AI queries for compliance
# Implement custom AI usage logger

What's Next?

You've mastered advanced Gonzo configuration! Continue exploring:

Integration Examples - Apply advanced configs to real scenarios
API & Architecture - Understand how Gonzo works
Troubleshooting - Advanced troubleshooting techniques

Or start implementing advanced configurations:

# Create your advanced configuration
mkdir -p ~/.config/gonzo/{profiles,environments}

# Set up environment-specific configs
cp examples/prod.yml ~/.config/gonzo/environments/
cp examples/dev.yml ~/.config/gonzo/environments/

# Create performance profiles
cp examples/high-volume.yml ~/.config/gonzo/profiles/
cp examples/low-resource.yml ~/.config/gonzo/profiles/

# Use them
gonzo --config ~/.config/gonzo/environments/prod.yml
gonzo --config ~/.config/gonzo/profiles/high-volume.yml

Master advanced configuration for production-ready Gonzo deployments! 🚀 From performance optimization to multi-environment management, sophisticated configuration strategies ensure Gonzo scales with your needs.

Last updated 2 months ago