AWS CloudWatch ☁️

AWS CloudWatch Integration

Analyze AWS CloudWatch Logs with Gonzo for powerful, interactive log analysis right from your terminal. Stream logs in real-time or analyze historical data with AI-powered insights.

Prerequisites: AWS CLI installed and configured with appropriate credentials. Use aws configure to set up your AWS access.

Why Use Gonzo with CloudWatch?

AWS CloudWatch Logs is a powerful log aggregation service, but analyzing logs in the console can be limiting. Gonzo brings:

🎨 Visual Analysis - Beautiful terminal interface vs web console
🤖 AI Insights - Intelligent pattern recognition and root cause analysis
⚡ Real-Time Streaming - Live log analysis as events occur
🔍 Advanced Filtering - Powerful regex and search capabilities
📊 Pattern Detection - Automatic clustering of similar log entries
💰 Cost Efficiency - Analyze logs locally without CloudWatch Insights costs

Quick Start

Getting Your Log Groups

First, discover what log groups are available in your AWS region:

# List all log groups with names and ARNs
aws logs describe-log-groups \
  --query 'logGroups[*].{Name:logGroupName,ARN:arn}' \
  --output json

# List log groups in table format
aws logs describe-log-groups \
  --query 'logGroups[*].{Name:logGroupName,ARN:arn}' \
  --output table

# Filter by prefix
aws logs describe-log-groups \
  --log-group-name-prefix /aws/lambda/ \
  --query 'logGroups[*].logGroupName' \
  --output text

Basic Log Tailing with Gonzo

The simplest way to use Gonzo with CloudWatch Logs is to pipe aws logs tail output:

# Basic tail (default: last 10 minutes)
aws logs tail "/aws/eks/stage/cluster" --follow | gonzo

# Tail with JSON output for better attribute extraction
aws logs tail "/aws/eks/stage/cluster" --follow --format json | gonzo

# Tail with AI analysis
aws logs tail "/aws/eks/stage/cluster" --follow --format json | \
gonzo --ai-model="gpt-4"

Output Format: The default "detailed" format outputs one log per line (non-JSON). Use --format json to gain attribute visibility in Gonzo, though each JSON log message spans two lines.

Multiple Log Groups

Monitor multiple log groups simultaneously:

# Method 1: Using process substitution
gonzo - < <(
  aws logs tail "/aws/eks/stage/cluster" --follow --format json &
  aws logs tail "RDS" --follow --format json &
  wait
)

# Method 2: Named pipes (alternative)
mkfifo log_stream
(aws logs tail "/aws/eks/stage/cluster" --follow --format json &
 aws logs tail "RDS" --follow --format json) > log_stream &
gonzo - < log_stream

# Method 3: File-based aggregation
aws logs tail "/aws/eks/stage/cluster" --follow --format json > logs1.txt &
aws logs tail "RDS" --follow --format json > logs2.txt &
gonzo -f logs1.txt -f logs2.txt --follow

How This Works:

Starts multiple aws logs tail processes in the background (&)
Combines their output streams
Pipes the combined output to Gonzo
Uses wait to ensure all background processes complete properly

AWS CLI Live Tail Integration

Live Tail provides near real-time streaming of log events as they are ingested into CloudWatch.

Basic Live Tail Usage

# Single log group with ARN
aws logs start-live-tail --log-group-identifiers \
  "arn:aws:logs:us-east-1:767397775588:log-group:/aws/eks/stage/cluster" | gonzo

# Multiple log groups
aws logs start-live-tail --log-group-identifiers \
  "arn:aws:logs:us-east-1:767397775588:log-group:/aws/eks/stage/cluster" \
  "arn:aws:logs:us-east-1:767397775588:log-group:RDS" | gonzo

# With AI analysis
aws logs start-live-tail --log-group-identifiers \
  "arn:aws:logs:us-east-1:767397775588:log-group:/aws/eks/stage/cluster" | \
gonzo --ai-model="gpt-4"

Live Tail Benefits:

✅ Near real-time streaming (lower latency than tail)
✅ Native JSON output (better attribute extraction)
✅ Supports multiple log groups natively
✅ Automatic handling of high-throughput log groups

Live Tail vs Regular Tail

Feature

aws logs tail

aws logs start-live-tail

Latency

~1-2 seconds

Near real-time (<1 second)

Multiple Log Groups

Requires workarounds

Native support

Output Format

Detailed (default) or JSON

JSON (default)

Historical Logs

Yes (--since flag)

No (only new logs)

Best For

Historical analysis

Real-time monitoring

Choose Live Tail When:

Monitoring production incidents in real-time
Need lowest possible latency
Monitoring multiple log groups simultaneously
JSON output is preferred

Choose Regular Tail When:

Need historical log analysis
Want to start from specific time (--since)
Need simple single log group monitoring

AWS CLI Integration

Installing and Configuring AWS CLI

# Install AWS CLI (if not already installed)
# macOS
brew install awscli

# Linux
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install

# Windows
# Download from: https://awscli.amazonaws.com/AWSCLIV2.msi

# Configure AWS CLI
aws configure
# Enter:
#   AWS Access Key ID
#   AWS Secret Access Key
#   Default region (e.g., us-east-1)
#   Default output format (json recommended)

# Verify configuration
aws sts get-caller-identity

Essential CloudWatch Logs Commands

# List all log groups
aws logs describe-log-groups

# List with specific prefix
aws logs describe-log-groups --log-group-name-prefix /aws/lambda/

# Get specific log group details
aws logs describe-log-groups --log-group-name-prefix /aws/lambda/my-function

# Count log groups
aws logs describe-log-groups --query 'length(logGroups)'

# Basic tail (last 10 minutes)
aws logs tail /aws/lambda/my-function

# Follow in real-time
aws logs tail /aws/lambda/my-function --follow

# Since specific time
aws logs tail /aws/lambda/my-function --since 1h
aws logs tail /aws/lambda/my-function --since 30m
aws logs tail /aws/lambda/my-function --since 2024-01-15T10:00:00

# Filter pattern
aws logs tail /aws/lambda/my-function --filter-pattern "ERROR"

# Format options
aws logs tail /aws/lambda/my-function --format short
aws logs tail /aws/lambda/my-function --format detailed

# Filter by pattern
aws logs tail /aws/lambda/my-function --filter-pattern "ERROR"

# Multiple terms (AND logic)
aws logs tail /aws/lambda/my-function --filter-pattern "ERROR timeout"

# JSON field filtering
aws logs tail /aws/lambda/my-function --filter-pattern '{ $.level = "error" }'

# Numeric filtering
aws logs tail /aws/lambda/my-function --filter-pattern '{ $.duration > 1000 }'

# Combined with Gonzo
aws logs tail /aws/lambda/my-function --filter-pattern "ERROR" --follow | gonzo

Real-World Integration Patterns

Lambda Function Monitoring

# Monitor specific Lambda function
aws logs tail /aws/lambda/my-function --follow | gonzo

# Monitor with AI for error analysis
aws logs tail /aws/lambda/my-function --follow | \
gonzo --ai-model="gpt-4"

# Monitor multiple Lambda functions
(aws logs tail /aws/lambda/function1 --follow & \
 aws logs tail /aws/lambda/function2 --follow) | gonzo

# Focus on errors only
aws logs tail /aws/lambda/my-function --follow | \
grep -i error | \
gonzo --log-buffer=5000

ECS/Fargate Container Logs

# ECS task logs
aws logs tail /ecs/my-service --follow | gonzo

# Filter by container
aws logs tail /ecs/my-service --follow | \
grep "container-name" | \
gonzo

# Multiple ECS services
(aws logs tail /ecs/api --follow & \
 aws logs tail /ecs/worker --follow) | \
gonzo --log-buffer=10000

API Gateway Logs

# API Gateway execution logs
aws logs tail /aws/apigateway/my-api --follow | gonzo

# Filter by HTTP status
aws logs tail /aws/apigateway/my-api --follow | \
grep -E "(4[0-9]{2}|5[0-9]{2})" | \
gonzo --ai-model="gpt-4"

# Access logs analysis
aws logs tail /aws/apigateway/my-api/access --since 1h | \
gonzo --log-buffer=20000

RDS Database Logs

# RDS error logs
aws logs tail /aws/rds/instance/my-db/error --follow | gonzo

# Slow query logs
aws logs tail /aws/rds/instance/my-db/slowquery --follow | \
gonzo --ai-model="gpt-4"

# General logs
aws logs tail /aws/rds/instance/my-db/general --since 1h | gonzo

CloudTrail Logs

# Monitor CloudTrail events
aws logs tail aws-cloudtrail-logs --follow | gonzo

# Security-focused monitoring
aws logs tail aws-cloudtrail-logs --follow | \
grep -E "(unauthorized|denied|failed)" | \
gonzo --ai-model="gpt-4"

# Specific service events
aws logs tail aws-cloudtrail-logs --follow | \
grep "ec2.amazonaws.com" | \
gonzo

Advanced Usage Patterns

Multi-Account Monitoring

# Switch AWS profiles
export AWS_PROFILE=production
aws logs tail /aws/lambda/function --follow | gonzo

# Monitor multiple accounts simultaneously
AWS_PROFILE=prod aws logs tail /aws/lambda/api --follow > prod-logs.txt &
AWS_PROFILE=staging aws logs tail /aws/lambda/api --follow > staging-logs.txt &
gonzo -f prod-logs.txt -f staging-logs.txt --follow

# Account-specific aliases
alias gonzo-prod='AWS_PROFILE=production aws logs tail'
alias gonzo-staging='AWS_PROFILE=staging aws logs tail'

# Usage
gonzo-prod /aws/lambda/function --follow | gonzo

Cost-Optimized Analysis

# Export logs once, analyze multiple times
aws logs tail /aws/lambda/function --since 24h > logs-$(date +%Y%m%d).txt

# Analyze without streaming (no ongoing CloudWatch costs)
gonzo -f logs-20240115.txt --ai-model="gpt-4"

# Batch processing for cost savings
for log_group in $(aws logs describe-log-groups --query 'logGroups[*].logGroupName' --output text); do
    echo "Analyzing $log_group..."
    aws logs tail "$log_group" --since 1h > "logs/${log_group//\//_}.txt"
done

# Analyze all exported logs
gonzo -f logs/*.txt

Automated Monitoring Scripts

#!/bin/bash
# monitor-cloudwatch.sh
# Continuous monitoring with alerting

LOG_GROUP="$1"
ALERT_PATTERN="${2:-ERROR}"
ALERT_WEBHOOK="${3}"

echo "Monitoring $LOG_GROUP for pattern: $ALERT_PATTERN"

aws logs tail "$LOG_GROUP" --follow | \
while IFS= read -r line; do
    echo "$line"
    
    # Check for alert pattern
    if echo "$line" | grep -qi "$ALERT_PATTERN"; then
        # Send alert
        if [ -n "$ALERT_WEBHOOK" ]; then
            curl -X POST "$ALERT_WEBHOOK" \
                -H "Content-Type: application/json" \
                -d "{\"text\": \"Alert: $line\"}"
        fi
        
        # Analyze with Gonzo + AI
        echo "$line" | gonzo --ai-model="gpt-4"
    fi
done

Performance Analysis

# Analyze Lambda cold starts
aws logs tail /aws/lambda/function --since 1h | \
grep "Init Duration" | \
gonzo --ai-model="gpt-4"

# Memory usage analysis
aws logs tail /aws/lambda/function --since 1h | \
grep "Max Memory Used" | \
gonzo

# Duration analysis
aws logs tail /aws/lambda/function --since 1h | \
grep "Duration" | \
gonzo --log-buffer=20000

CloudWatch Logs Insights Integration

Export Insights Query Results

# Run CloudWatch Logs Insights query
query_id=$(aws logs start-query \
    --log-group-name /aws/lambda/my-function \
    --start-time $(date -u -d '1 hour ago' +%s) \
    --end-time $(date -u +%s) \
    --query-string 'fields @timestamp, @message | filter @message like /ERROR/ | sort @timestamp desc | limit 1000' \
    --query 'queryId' \
    --output text)

# Wait for query to complete
aws logs wait query-complete --query-id "$query_id"

# Get results and analyze with Gonzo
aws logs get-query-results --query-id "$query_id" | \
jq -r '.results[] | .[1].value' | \
gonzo --ai-model="gpt-4"

Configuration for CloudWatch

Optimized Configuration File

# ~/.config/gonzo/cloudwatch.yml
# Optimized for CloudWatch Logs

# CloudWatch logs are streamed via stdin
# No file configuration needed

# Performance for CloudWatch streaming
update-interval: 2s              # Responsive updates
log-buffer: 10000               # Large buffer for CloudWatch volume
memory-size: 50000              # Extensive pattern tracking

# AI configuration for AWS analysis
ai-model: "gpt-4"               # Best for complex AWS issues
ai-context-size: 8000           # Large context for AWS logs

# Display
show-service: true              # Show AWS service context

Usage:

aws logs tail /aws/lambda/function --follow | \
gonzo --config ~/.config/gonzo/cloudwatch.yml

Environment-Specific Aliases

# Add to ~/.bashrc or ~/.zshrc

# Production monitoring
alias gonzo-prod='AWS_PROFILE=production aws logs tail --follow'

# Development monitoring
alias gonzo-dev='AWS_PROFILE=development aws logs tail --follow'

# Lambda-specific
alias gonzo-lambda='aws logs tail --follow'

# With AI analysis
alias gonzo-lambda-ai='aws logs tail --follow | gonzo --ai-model="gpt-4"'

# Usage
gonzo-lambda /aws/lambda/my-function
gonzo-lambda-ai /aws/lambda/critical-function

Complete Guide

For step-by-step CloudWatch integration: 📖 Gonzo: A Better Way to Tail AWS CloudWatch Logs

Best Practices

🎯 Efficient Log Retrieval

# Use time ranges to limit data
aws logs tail /aws/lambda/function --since 1h    # Last hour only

# Use filter patterns for pre-filtering
aws logs tail /aws/lambda/function --filter-pattern "ERROR" --follow

# Export for repeated analysis
aws logs tail /aws/lambda/function --since 24h > daily-export.txt
gonzo -f daily-export.txt  # No repeated CloudWatch API calls

Cost Considerations

Important: AWS costs may be incurred for using the AWS CLI tail or start-live-tail commands. Please consult the AWS CloudWatch Logs pricing documentation for details on potential charges related to log data retrieval.

Understanding CloudWatch Logs Pricing

Data Retrieval Costs:

GetLogEvents API: Used by aws logs tail - included with storage
StartLiveTail API: Used by aws logs start-live-tail - additional charges may apply
Data Transfer: Egress charges may apply for large volumes

Cost Optimization Strategies

# 1. Use time-based filtering to reduce data volume
aws logs tail /aws/lambda/function --since 1h --follow | gonzo  # Last hour only

# 2. Use filter patterns at the source
aws logs tail /aws/lambda/function --filter-pattern "ERROR" --follow | gonzo

# 3. Export once, analyze multiple times (no repeated API calls)
aws logs tail /aws/lambda/function --since 24h > daily-export.txt
gonzo -f daily-export.txt  # Analyze locally
gonzo -f daily-export.txt --ai-model="gpt-4"  # Re-analyze with AI

# 4. Use regular tail instead of live tail when real-time isn't critical
aws logs tail /aws/lambda/function --follow | gonzo  # Lower cost

# 5. Monitor specific log groups instead of all logs
aws logs tail "/aws/lambda/critical-function" --follow | gonzo  # Focused monitoring

Cost Comparison Example:

# Higher cost: Live tail multiple groups for 24 hours
aws logs start-live-tail --log-group-identifiers arn1 arn2 arn3 | gonzo

# Lower cost: Regular tail with filtering
aws logs tail "/aws/lambda/function" --filter-pattern "ERROR" --follow | gonzo

# Lowest cost: Export and analyze locally
aws logs tail "/aws/lambda/function" --since 1h > export.txt
gonzo -f export.txt  # No ongoing API costs

🔒 Security Best Practices

# Use IAM roles instead of access keys
aws configure set role_arn arn:aws:iam::123456789012:role/LogAnalyst
aws configure set source_profile default

# Least privilege IAM policy for Gonzo usage:
# {
#   "Version": "2012-10-17",
#   "Statement": [{
#     "Effect": "Allow",
#     "Action": [
#       "logs:DescribeLogGroups",
#       "logs:DescribeLogStreams",
#       "logs:GetLogEvents",
#       "logs:FilterLogEvents"
#     ],
#     "Resource": "arn:aws:logs:*:*:log-group:/aws/lambda/*"
#   }]
# }

# Use MFA for production access
aws configure set mfa_serial arn:aws:iam::123456789012:mfa/user

# Audit log access
aws cloudtrail lookup-events --lookup-attributes AttributeKey=ResourceName,AttributeValue=/aws/lambda/function

⚡ Performance Optimization

# Pre-filter for performance
aws logs tail /aws/lambda/function --filter-pattern "ERROR" --follow | gonzo

# Adjust Gonzo buffers for volume
aws logs tail /aws/lambda/high-traffic --follow | \
gonzo --log-buffer=20000 --update-interval=5s

# Use local AI for sensitive logs
export OPENAI_API_BASE="http://localhost:11434"
aws logs tail /aws/lambda/sensitive --follow | \
gonzo --ai-model="llama3"

Troubleshooting

Common Issues

Authentication Errors:

# Verify AWS credentials
aws sts get-caller-identity

# Check IAM permissions
aws iam get-user

# Use specific profile
AWS_PROFILE=your-profile aws logs tail /aws/lambda/function

Log Group Not Found:

# List available log groups
aws logs describe-log-groups | grep logGroupName

# Search for specific pattern
aws logs describe-log-groups --log-group-name-prefix /aws/lambda/

# Verify log group name (case-sensitive)
aws logs describe-log-groups --log-group-name /aws/lambda/my-function

No Logs Appearing:

# Check if logs exist
aws logs tail /aws/lambda/function --since 24h

# Verify time range
aws logs tail /aws/lambda/function --since 5m  # Last 5 minutes

# Check filter pattern
aws logs tail /aws/lambda/function --filter-pattern ""  # All logs

Performance Issues:

# Reduce log volume
aws logs tail /aws/lambda/function --filter-pattern "ERROR"

# Increase Gonzo buffers
aws logs tail /aws/lambda/function --follow | \
gonzo --log-buffer=20000 --memory-size=50000

# Use longer update intervals
aws logs tail /aws/lambda/function --follow | \
gonzo --update-interval=10s

What's Next?

Now that you've mastered CloudWatch integration, explore related topics:

Kubernetes Integration - Analyze EKS cluster logs
Configuration - Optimize for CloudWatch workflows
AI Integration - Enhanced AWS log analysis
Advanced Features - Pattern detection in AWS logs

Or try these advanced CloudWatch patterns:

# Multi-region monitoring
for region in us-east-1 us-west-2 eu-west-1; do
    AWS_REGION=$region aws logs tail /aws/lambda/function --follow > logs-${region}.txt &
done
gonzo -f logs-*.txt --follow

# Cross-account analysis
AWS_PROFILE=account1 aws logs tail /aws/lambda/function --since 1h > account1.txt
AWS_PROFILE=account2 aws logs tail /aws/lambda/function --since 1h > account2.txt
gonzo -f account1.txt -f account2.txt --ai-model="gpt-4"

Unlock the full potential of AWS CloudWatch Logs with Gonzo! ☁️ From real-time Lambda monitoring to comprehensive AWS infrastructure analysis, Gonzo brings powerful log analysis to your CloudWatch workflows.

PreviousDocker 🐳NextGrafana Loki Integration 🔥

Last updated 5 months ago

hashtagAWS CloudWatch Integration

hashtagWhy Use Gonzo with CloudWatch?

hashtagQuick Start

hashtagGetting Your Log Groups

hashtagBasic Log Tailing with Gonzo

hashtagMultiple Log Groups

hashtagAWS CLI Live Tail Integration

hashtagBasic Live Tail Usage

hashtagLive Tail vs Regular Tail

hashtagAWS CLI Integration

hashtagInstalling and Configuring AWS CLI

hashtagEssential CloudWatch Logs Commands

hashtagReal-World Integration Patterns

hashtagLambda Function Monitoring

hashtagECS/Fargate Container Logs

hashtagAPI Gateway Logs

hashtagRDS Database Logs

hashtagCloudTrail Logs

hashtagAdvanced Usage Patterns

hashtagMulti-Account Monitoring

hashtagCost-Optimized Analysis

hashtagAutomated Monitoring Scripts

hashtagPerformance Analysis

hashtagCloudWatch Logs Insights Integration

hashtagExport Insights Query Results

hashtagConfiguration for CloudWatch

hashtagOptimized Configuration File

hashtagEnvironment-Specific Aliases

hashtagComplete Guide

hashtagBest Practices

hashtag🎯 Efficient Log Retrieval

hashtagCost Considerations

hashtagUnderstanding CloudWatch Logs Pricing

hashtagCost Optimization Strategies

hashtag🔒 Security Best Practices

hashtag⚡ Performance Optimization

hashtagTroubleshooting

hashtagCommon Issues

hashtagWhat's Next?

AWS CloudWatch Integration

Why Use Gonzo with CloudWatch?

Quick Start

Getting Your Log Groups

Basic Log Tailing with Gonzo

Multiple Log Groups

AWS CLI Live Tail Integration

Basic Live Tail Usage

Live Tail vs Regular Tail

AWS CLI Integration

Installing and Configuring AWS CLI

Essential CloudWatch Logs Commands

Real-World Integration Patterns

Lambda Function Monitoring

ECS/Fargate Container Logs

API Gateway Logs

RDS Database Logs

CloudTrail Logs

Advanced Usage Patterns

Multi-Account Monitoring

Cost-Optimized Analysis

Automated Monitoring Scripts

Performance Analysis

CloudWatch Logs Insights Integration

Export Insights Query Results

Configuration for CloudWatch

Optimized Configuration File

Environment-Specific Aliases

Complete Guide

Best Practices

🎯 Efficient Log Retrieval

Cost Considerations

Understanding CloudWatch Logs Pricing

Cost Optimization Strategies

🔒 Security Best Practices

⚡ Performance Optimization

Troubleshooting

Common Issues

What's Next?