Input Methods Overflow

Gonzo offers multiple powerful ways to feed log data into its analysis engine. From simple file analysis to real-time OpenTelemetry streams, this guide covers all input methods and their optimal use cases.

Input Method Overview

Gonzo supports four primary input methods, each optimized for different scenarios:

Method
Best For
Command Example
Real-time?

📁 File Input

Static log analysis, debugging

gonzo -f app.log

No

📈 File Following

Real-time monitoring

gonzo -f app.log --follow

Yes

🔄 Stdin Piping

Integration with tools

kubectl logs -f pod | gonzo

Yes

🌐 OTLP Receiver

OpenTelemetry integration

gonzo --otlp-enabled

Yes

File Input Methods

Single File Analysis

Perfect for debugging specific issues or analyzing archived logs:

# Basic file analysis
gonzo -f application.log

# Absolute paths
gonzo -f /var/log/nginx/access.log

# Relative paths  
gonzo -f logs/app.log

# Compressed files (via piping)
zcat app.log.gz | gonzo
gunzip -c error.log.gz | gonzo

Best Practices:

  • Start with single files to understand the data structure

  • Use absolute paths to avoid "file not found" errors

  • Check file permissions if Gonzo can't access the file

Multiple File Analysis

Combine logs from related services for comprehensive analysis:

Glob Pattern Input

Automatically include multiple files using pattern matching:

Glob Pattern Examples:

Pattern
Matches
Use Case

"*.log"

All .log files in current dir

Development debugging

"/var/log/*.log"

All .log files in /var/log

System administration

"app-*.log"

app-2024.log, app-error.log

Application-specific logs

"/logs/**/*.log"

All .log files recursively

Complex log hierarchies

"*-{error,warn}.log"

Files ending in -error.log or -warn.log

Specific severity levels

circle-exclamation

Advanced Glob Techniques:

Real-Time File Following

Monitor logs as they grow, similar to tail -f:

Basic Following

How Following Works:

  • Gonzo monitors files for changes using inotify (Linux) or similar mechanisms

  • New log entries appear automatically in the interface

  • Handles log rotation gracefully

  • Efficiently processes only new content

Advanced Following Scenarios

Development Environment:

Production Monitoring:

Following Best Practices

Performance Optimization:

Monitoring Multiple Sources:

Stdin Processing (Piping)

Integrate Gonzo with existing tools and command pipelines:

Basic Piping

Real-Time Streaming

Traditional Unix Tools:

System Monitoring:

Container Integration

Single Container:

Docker Compose:

Advanced Piping Scenarios

Network Sources:

Complex Processing:

Input Method Selection Guide

Choose the right input method based on your scenario:

Development & Debugging

Production Monitoring

System Administration

Integration Scenarios

Performance Considerations

Buffer Sizing

Optimize for your log volume and available memory:

Update Intervals

Balance responsiveness with system resources:

Input-Specific Optimizations

Input Method
Optimization
Configuration

File Following

Use larger buffers

--log-buffer=5000

Stdin Piping

Ensure smooth pipeline

`command

OTLP

Tune batch sizes

OpenTelemetry collector config

Multiple Files

Balance file count vs performance

Combine related files

Troubleshooting Input Issues

File Access Problems

Following Issues

Piping Problems

Advanced Input Patterns

Hybrid Approaches

Combine multiple input methods for comprehensive monitoring:

Custom Input Scripts

Create wrapper scripts for complex scenarios:

Last updated