# Input Methods Overflow
Gonzo offers multiple powerful ways to feed log data into its analysis engine. From simple file analysis to real-time OpenTelemetry streams, this guide covers all input methods and their optimal use cases.
### Input Method Overview
Gonzo supports four primary input methods, each optimized for different scenarios:
| Method | Best For | Command Example | Real-time? |
| --------------------- | ------------------------------ | ------------------------------ | ---------- |
| **📁 File Input** | Static log analysis, debugging | `gonzo -f app.log` | No |
| **📈 File Following** | Real-time monitoring | `gonzo -f app.log --follow` | Yes |
| **🔄 Stdin Piping** | Integration with tools | `kubectl logs -f pod \| gonzo` | Yes |
| **🌐 OTLP Receiver** | OpenTelemetry integration | `gonzo --otlp-enabled` | Yes |
### File Input Methods
#### Single File Analysis
Perfect for debugging specific issues or analyzing archived logs:
```bash
# Basic file analysis
gonzo -f application.log
# Absolute paths
gonzo -f /var/log/nginx/access.log
# Relative paths
gonzo -f logs/app.log
# Compressed files (via piping)
zcat app.log.gz | gonzo
gunzip -c error.log.gz | gonzo
```
**Best Practices:**
* Start with single files to understand the data structure
* Use absolute paths to avoid "file not found" errors
* Check file permissions if Gonzo can't access the file
#### Multiple File Analysis
Combine logs from related services for comprehensive analysis:
```bash
# Multiple specific files
gonzo -f api.log -f database.log -f cache.log
# Mix different log types
gonzo -f application.log -f /var/log/syslog -f error.log
# Different directories
gonzo -f /var/log/app.log -f /home/user/debug.log
```
{% tabs %}
{% tab title="Related Services" %}
**Microservices Architecture:**
```bash
# Analyze all service logs together
gonzo -f api-gateway.log -f user-service.log -f payment-service.log
# Include infrastructure logs
gonzo -f app.log -f nginx.log -f postgres.log
```
**Benefits:**
* Correlate events across services
* See timing relationships
* Identify cascade failures
{% endtab %}
{% tab title="Different Severities" %}
**Separate Log Levels:**
```bash
# Combine different severity files
gonzo -f application.log -f error.log -f debug.log
# Include system and application logs
gonzo -f app.log -f /var/log/syslog
```
**Benefits:**
* Complete picture of system state
* Context around error events
* Debug information correlation
{% endtab %}
{% endtabs %}
#### Glob Pattern Input
Automatically include multiple files using pattern matching:
```bash
# All .log files in current directory
gonzo -f "*.log"
# All log files in a directory
gonzo -f "/var/log/*.log"
# Specific patterns
gonzo -f "/var/log/app-*.log"
gonzo -f "/logs/**/*.log"
# Multiple patterns
gonzo -f "/var/log/app*.log" -f "/var/log/nginx*.log"
```
**Glob Pattern Examples:**
| Pattern | Matches | Use Case |
| ---------------------- | --------------------------------------- | ------------------------- |
| `"*.log"` | All .log files in current dir | Development debugging |
| `"/var/log/*.log"` | All .log files in /var/log | System administration |
| `"app-*.log"` | app-2024.log, app-error.log | Application-specific logs |
| `"/logs/**/*.log"` | All .log files recursively | Complex log hierarchies |
| `"*-{error,warn}.log"` | Files ending in -error.log or -warn.log | Specific severity levels |
{% hint style="warning" %}
**Important:** Always quote glob patterns to prevent shell expansion. Use `"*.log"` not `*.log`.
{% endhint %}
**Advanced Glob Techniques:**
```bash
# Rotated log files
gonzo -f "/var/log/app.log*" # app.log, app.log.1, app.log.2, etc.
# Date-based logs
gonzo -f "/logs/app-2024-*.log" # app-2024-01-15.log, etc.
# Multiple services with patterns
gonzo -f "/var/log/{nginx,apache}/*.log"
# Exclude certain files (use find + piping)
find /var/log -name "*.log" ! -name "*debug*" -exec gonzo -f {} +
```
### Real-Time File Following
Monitor logs as they grow, similar to `tail -f`:
#### Basic Following
```bash
# Follow a single file
gonzo -f /var/log/app.log --follow
# Follow multiple files
gonzo -f app.log -f error.log --follow
# Follow with glob patterns
gonzo -f "/var/log/*.log" --follow
```
**How Following Works:**
* Gonzo monitors files for changes using inotify (Linux) or similar mechanisms
* New log entries appear automatically in the interface
* Handles log rotation gracefully
* Efficiently processes only new content
#### Advanced Following Scenarios
{% tabs %}
{% tab title="Application Monitoring" %}
**Development Environment:**
```bash
# Follow application logs during development
gonzo -f logs/app.log --follow
# Include error logs
gonzo -f logs/app.log -f logs/error.log --follow
# With configuration
gonzo -f logs/*.log --follow --log-buffer=5000
```
**Production Monitoring:**
```bash
# Monitor production logs
gonzo -f /var/log/myapp/*.log --follow
# Include system logs
gonzo -f /var/log/myapp.log -f /var/log/syslog --follow
```
{% endtab %}
{% tab title="System Administration" %}
**Security Monitoring:**
```bash
# Monitor security logs
sudo gonzo -f /var/log/auth.log --follow
# Multiple security sources
sudo gonzo -f "/var/log/{auth,secure,audit}*.log" --follow
```
**Performance Monitoring:**
```bash
# Monitor performance-related logs
gonzo -f /var/log/nginx/access.log -f /var/log/app/perf.log --follow
# System resource logs
gonzo -f /var/log/syslog -f /var/log/kern.log --follow
```
{% endtab %}
{% endtabs %}
#### Following Best Practices
**Performance Optimization:**
```bash
# Large files - adjust buffer size
gonzo -f huge.log --follow --log-buffer=10000
# High-frequency logs - slower updates
gonzo -f busy.log --follow --update-interval=5s
# Memory-constrained systems
gonzo -f app.log --follow --memory-size=5000
```
**Monitoring Multiple Sources:**
```bash
# Start with most important logs
gonzo -f app.log --follow
# Add more sources gradually
gonzo -f app.log -f error.log --follow
# Full monitoring setup
gonzo -f "/var/log/app/*.log" --follow --log-buffer=5000 --update-interval=2s
```
### Stdin Processing (Piping)
Integrate Gonzo with existing tools and command pipelines:
#### Basic Piping
```bash
# Basic piping from other commands
cat application.log | gonzo
# Process compressed logs
zcat app.log.gz | gonzo
gunzip -c logs.gz | gonzo
# Stream from remote systems
ssh server "cat /var/log/app.log" | gonzo
```
#### Real-Time Streaming
**Traditional Unix Tools:**
```bash
# Stream with tail
tail -f /var/log/app.log | gonzo
# Multiple files with tail
tail -f /var/log/*.log | gonzo
# Advanced tailing with multitail
multitail -f /var/log/app.log /var/log/error.log | gonzo
```
**System Monitoring:**
```bash
# System logs
sudo tail -f /var/log/syslog | gonzo
# Journal logs (systemd)
journalctl -f | gonzo
# Specific service logs
journalctl -f -u nginx | gonzo
```
#### Container Integration
{% tabs %}
{% tab title="Docker" %}
**Single Container:**
```bash
# Current container logs
docker logs my-container | gonzo
# Follow container logs
docker logs -f my-container 2>&1 | gonzo
# Multiple containers
docker logs -f container1 container2 2>&1 | gonzo
```
**Docker Compose:**
```bash
# All services
docker-compose logs -f | gonzo
# Specific services
docker-compose logs -f web api database | gonzo
# With timestamps
docker-compose logs -f -t | gonzo
```
{% endtab %}
{% tab title="Kubernetes" %}
**Pod Logs:**
```bash
# Single pod
kubectl logs -f pod/my-app | gonzo
# Deployment logs
kubectl logs -f deployment/my-app | gonzo
# Multiple pods
kubectl logs -f -l app=backend | gonzo
```
**Advanced Kubernetes:**
```bash
# Previous container instance
kubectl logs -f pod/my-app --previous | gonzo
# Specific container in pod
kubectl logs -f pod/my-app -c sidecar | gonzo
# All containers in pod
kubectl logs -f pod/my-app --all-containers | gonzo
```
**With Stern (if installed):**
```bash
# All pods with label
stern backend | gonzo
# Pods in specific namespace
stern . -n production | gonzo
# With context and coloring
stern api --context=prod --color=always | gonzo
```
{% endtab %}
{% endtabs %}
#### Advanced Piping Scenarios
**Network Sources:**
```bash
# Remote syslog
nc -l 514 | gonzo
# SSH tunneling
ssh -L 8514:logserver:514 server "nc localhost 8514" | gonzo
# HTTP log streaming
curl -s http://logserver/stream | gonzo
```
**Complex Processing:**
# Filter before Gonzo
tail -f /var/log/app.log | grep -v DEBUG | gonzo
# Transform data
tail -f /var/log/app.log | sed 's/old/new/g' | gonzo
# Multiple sources combined
( tail -f app.log & tail -f error.log ) | gonzo
### Input Method Selection Guide
Choose the right input method based on your scenario:
#### Development & Debugging
```bash
# Local development - single application
gonzo -f logs/app.log --follow
# Debugging specific issues
gonzo -f error.log
# Multiple local services
gonzo -f "logs/*.log" --follow
```
#### Production Monitoring
```bash
# Container environments
kubectl logs -f deployment/app | gonzo
# Traditional servers
gonzo -f "/var/log/app/*.log" --follow
# OpenTelemetry-enabled applications
gonzo --otlp-enabled
```
#### System Administration
```bash
# System health monitoring
sudo gonzo -f "/var/log/{syslog,auth.log,kern.log}" --follow
# Security analysis
sudo tail -f /var/log/auth.log | gonzo
# Performance investigation
gonzo -f "/var/log/nginx/*.log"
```
#### Integration Scenarios
```bash
# CI/CD pipelines
build-logs | gonzo
# Monitoring alerts
alert-stream | gonzo
# Log aggregation
rsyslog-stream | gonzo
```
### Performance Considerations
#### Buffer Sizing
Optimize for your log volume and available memory:
```bash
# High-volume logs
gonzo -f busy.log --log-buffer=10000 --memory-size=20000
# Memory-constrained systems
gonzo -f app.log --log-buffer=1000 --memory-size=5000
# Balanced configuration
gonzo -f app.log --log-buffer=2000 --memory-size=15000
```
#### Update Intervals
Balance responsiveness with system resources:
```bash
# Real-time responsiveness (default)
gonzo -f app.log --follow --update-interval=1s
# Balanced performance
gonzo -f app.log --follow --update-interval=2s
# Resource conservation
gonzo -f app.log --follow --update-interval=5s
```
#### Input-Specific Optimizations
| Input Method | Optimization | Configuration |
| ------------------ | --------------------------------- | ------------------------------ |
| **File Following** | Use larger buffers | `--log-buffer=5000` |
| **Stdin Piping** | Ensure smooth pipeline | \`command |
| **OTLP** | Tune batch sizes | OpenTelemetry collector config |
| **Multiple Files** | Balance file count vs performance | Combine related files |
### Troubleshooting Input Issues
#### File Access Problems
```bash
# Permission denied
sudo gonzo -f /var/log/secure.log
# File not found with globs
gonzo -f "/path/to/*.log" # Use quotes!
# Check file permissions
ls -la logfile.log
```
#### Following Issues
```bash
# File rotation problems
# Gonzo handles this automatically, but check:
ls -la /var/log/app.log*
# High CPU usage
# Reduce update frequency:
gonzo -f app.log --follow --update-interval=5s
```
#### Piping Problems
```bash
# Broken pipe errors
# Ensure Gonzo can handle the data rate:
slow-producer | gonzo --log-buffer=10000
# Buffer full issues
# Increase buffer size or filter at source:
noisy-source | grep -v DEBUG | gonzo
```
### Advanced Input Patterns
#### Hybrid Approaches
Combine multiple input methods for comprehensive monitoring:
```bash
# OTLP + file backup
gonzo --otlp-enabled -f backup.log --follow
# Real-time + historical analysis
# Terminal 1: gonzo -f app.log --follow
# Terminal 2: gonzo -f historical-logs.log
# Primary + secondary sources
gonzo -f primary.log --follow & kubectl logs -f secondary | gonzo
```
#### Custom Input Scripts
Create wrapper scripts for complex scenarios:
```bash
#!/bin/bash
# monitor-all.sh
exec gonzo \
-f "/var/log/app/*.log" \
-f "/var/log/nginx/*.log" \
--follow \
--log-buffer=5000 \
--update-interval=2s
```
---
# Agent Instructions: Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.
Perform an HTTP GET request on the current page URL with the `ask` query parameter:
```
GET https://docs.controltheory.com/backup/input-methods-overflow.md?ask=
```
The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.