Log Format Issues

Troubleshooting guide for log parsing and format detection problems in Gonzo.

Format Detection Issues

Logs Not Being Parsed

Symptom: Structured logs appear as plain text instead of being parsed.

Diagnosis:

Gonzo automatically detects formats per line based on these rules:

• JSON: Lines starting with {

• Logfmt: Lines containing key=value patterns

• Plain text: Everything else

Solutions:

1. Verify JSON is valid

   # Test each line individually
   head -1 logs.json | jq .
   
   # If error, JSON is malformed

2. Check line starts with {

   # JSON must start with opening brace
   head logs.json
   
   # ✅ Good: {"level":"info"...
   # ❌ Bad: 2024-01-15 {"level":"info"...

3. Inspect logfmt format

   # Should have key=value pairs
   head logs.txt
   
   # ✅ Good: level=info service=api msg="started"
   # ❌ Bad: [INFO] service api started

4. Use custom format for non-standard logs

   gonzo --format=my-custom-format -f logs.txt

Mixed Format Logs

Symptom: Some lines parse correctly, others don't.

Explanation: Gonzo detects format per line, so mixed formats will have inconsistent parsing.

Example:

{"level":"info","msg":"API started"}     ← Parsed as JSON
level=info msg="processing request"     ← Parsed as logfmt  
[ERROR] Connection timeout              ← Plain text

Solutions:

1. Accept mixed display

   • This is expected behavior

   • Each format renders appropriately

2. Pre-filter to single format

   # Extract only JSON lines
   grep '^{' mixed.log | gonzo
   
   # Or only logfmt
   grep '=' mixed.log | gonzo

3. Convert to uniform format

   # Convert all to JSON with jq
   cat mixed.log | jq -R -c '. | fromjson? // {"message": .}' | gonzo

Attributes Not Extracted

Symptom: Logs parse but attributes panel is empty.

Causes & Solutions:

1. Plain text logs have no structured attributes

   • Plain text can't be parsed into fields

   • Only JSON and logfmt have attributes

2. Check JSON structure

   # Verify JSON has expected fields
   head -1 logs.json | jq .
   
   # Should show key-value pairs

3. Nested JSON

   {
     "log": {
       "level": "info",
       "service": "api"
     }
   }

   • Gonzo extracts nested attributes

   • May need to check nested paths

4. Missing common attribute names

   • Gonzo looks for common fields: level, service, host, etc.

   • Custom fields may not be highlighted

JSON Issues

Malformed JSON

Symptom: JSON logs show as plain text or cause errors.

Common JSON Issues:

1. Trailing commas

   {"level":"info","msg":"test",}  ❌ Invalid
   {"level":"info","msg":"test"}   ✅ Valid

2. Single quotes instead of double

   {'level':'info'}     ❌ Invalid
   {"level":"info"}     ✅ Valid

3. Unescaped quotes in strings

   {"msg":"He said "hi""}           ❌ Invalid
   {"msg":"He said \"hi\""}         ✅ Valid
   {"msg":"He said 'hi'"}           ✅ Valid (single quotes in string)

4. Missing quotes on keys

   {level:"info"}       ❌ Invalid
   {"level":"info"}     ✅ Valid

Validate & Fix:

# Validate JSON
cat logs.json | jq . > /dev/null
# If errors, shows line number

# Pretty-print to find issues
jq . logs.json

# Fix and re-format
jq -c . logs.json > fixed.json
gonzo -f fixed.json

Multi-line JSON

Symptom: JSON objects span multiple lines, not parsed correctly.

Example:

{
  "level": "info",
  "message": "test"
}

Solution:

Gonzo expects one JSON object per line (JSONL/NDJSON format).

# Compact multi-line JSON to single lines
jq -c . pretty.json > compact.json
gonzo -f compact.json

# Or pipe directly
jq -c . pretty.json | gonzo

JSON with Metadata Prefix

Symptom: Lines have timestamp or metadata before JSON.

Example:

2024-01-15 10:30:05 {"level":"info","msg":"test"}

Solution:

Remove prefix before piping to Gonzo:

# Remove timestamp prefix
sed 's/^[0-9-]* [0-9:]* //' logs.txt | gonzo

# Or use awk
awk '{$1=$2=""; print}' logs.txt | gonzo

# Extract just JSON part
grep -o '{.*}' logs.txt | gonzo

Escaped JSON in Strings

Symptom: JSON contains escaped JSON strings.

Example:

{"log":"{\"level\":\"info\",\"msg\":\"test\"}"}

Solution:

# Unescape inner JSON
jq -r '.log | fromjson' logs.json | gonzo

# Or handle both levels
jq -c '.log | fromjson? // .' logs.json | gonzo

Logfmt Issues

Logfmt Not Detected

Symptom: Key=value logs appear as plain text.

Requirements for logfmt detection:

• Must have key=value patterns

• Multiple pairs per line

• Values can be quoted: key="value with spaces"

Example:

level=info service=api user=123 msg="request completed"  ✅ Detected
INFO service api user 123 request completed              ❌ Not logfmt

Solutions:

1. Verify format

   # Check for key=value pattern
   grep -E '\w+=\w+' logs.txt | head

2. Add more key=value pairs

   • Single pair may not trigger detection

   • Multiple pairs more reliably detected

3. Use custom format

   gonzo --format=my-logfmt -f logs.txt

Spaces in Logfmt Values

Symptom: Values with spaces not parsed correctly.

Examples:

msg=hello world         ❌ Breaks: "world" seen as separate key
msg="hello world"       ✅ Correct: quotes preserve spaces
msg=hello\ world        ✅ Correct: escape preserves spaces

Solution:

Ensure spaces in values are properly quoted or escaped:

# Fix unquoted spaces (requires log generation fix)
# Or accept partial parsing of problematic lines

Logfmt with Nested Structures

Symptom: Nested objects in logfmt don't parse well.

Example:

user.id=123 user.name=john

Explanation: Logfmt is flat by design. Nested structures need JSON.

Solution:

1. Accept flat representation

   • Gonzo extracts user.id and user.name as separate attributes

2. Convert to JSON if needed

   # If you control log format, use JSON for nested data

Plain Text Issues

No Structure Extracted from Text Logs

Symptom: Plain text logs show no attributes.

Explanation: Plain text logs can't be parsed into structured fields automatically.

Examples:

[2024-01-15 10:30:05] ERROR: Connection failed
INFO - api-service - User login successful

Solutions:

1. Accept plain text display

   • Logs still searchable and analyzable

   • Just no structured attributes

2. Create custom format parser

   # Define regex-based parser
   # See Custom Formats Guide
   gonzo --format=my-text-format -f logs.txt

3. Convert logs to structured format

   • Modify application to output JSON/logfmt

   • Use log shipper to add structure (Fluent Bit, Logstash)

Severity Not Detected in Text Logs

Symptom: Plain text logs don't show color-coded severity.

Explanation: Gonzo looks for common severity keywords in text logs:

• ERROR, FATAL, CRITICAL → Red

• WARN, WARNING → Yellow

• INFO → Green

• DEBUG → Blue

• TRACE → White

Solutions:

1. Include severity keywords

   [ERROR] Connection failed       ✅ Detected
   Connection failed              ❌ Not detected
   
   WARN: Low disk space           ✅ Detected  
   Disk space is low              ❌ Not detected

2. Use consistent format

   • Put severity at start of line

   • Use standard keywords (ERROR, WARN, INFO, DEBUG)

3. Create custom format

   • Define severity extraction pattern

   • Map custom levels to standard severities

OTLP Format Issues

OTLP Logs Not Appearing

Symptom: OTLP receiver running but no logs in Gonzo.

Diagnosis:

1. Verify receiver is enabled

   # Check Gonzo started with OTLP
   gonzo --otlp-enabled
   
   # Should show listening on ports 4317 and 4318

2. Check sender configuration

   # Verify endpoint in sender
   endpoint: localhost:4317      # gRPC
   endpoint: http://localhost:4318/v1/logs  # HTTP

3. Test with curl (HTTP)

   curl -X POST http://localhost:4318/v1/logs \
     -H "Content-Type: application/json" \
     -d '{"resourceLogs":[]}'
   
   # Should return 200 OK

4. Check for port conflicts

   lsof -i :4317
   lsof -i :4318

Solutions:

See Common Issues - OTLP Receiver for detailed fixes.

OTLP Attributes Missing

Symptom: OTLP logs appear but without expected attributes.

Causes:

1. Attributes in resource vs log record

   • Resource attributes: service.name, host, etc.

   • Log record attributes: user_id, request_id, etc.

   • Both should be extracted

2. Verify sender includes attributes

   # Ensure attributes are set
   logger_provider.add_log_record_processor(processor)
   # Check resource attributes and log attributes

3. Check attribute names

   • Gonzo shows all attributes

   • May just be named differently than expected

Custom Format Issues

Custom Format Not Working

Symptom: --format=my-format shows error or doesn't parse.

Diagnosis:

1. Verify format file exists

   ls ~/.config/gonzo/formats/my-format.yaml

2. Check YAML syntax

   cat ~/.config/gonzo/formats/my-format.yaml
   
   # Validate if you have yamllint
   yamllint ~/.config/gonzo/formats/my-format.yaml

3. Test with built-in format first

   # Verify custom formats work at all
   gonzo --format=loki-stream -f test.json

Solutions:

See Custom Formats Guide for:

• Format file syntax

• Regex patterns

• Field mapping

• Testing formats

Regex Not Matching

Symptom: Custom format regex doesn't extract fields.

Solutions:

1. Test regex separately

   # Test your regex pattern
   echo "sample log line" | grep -E "your-regex-pattern"

2. Use online regex tester

   • Test at regex101.com

   • Use example log lines

   • Verify capture groups

3. Check for special characters

   # Escape special characters
   pattern: '\[(\d+)\]'  # Brackets escaped

4. Start simple, iterate

   # Begin with basic pattern
   pattern: '(\w+)'
   
   # Then add complexity
   pattern: '(\w+)=(\w+)'
   
   # Finally, full pattern
   pattern: '(\w+)="([^"]*)"'

Encoding Issues

Special Characters Garbled

Symptom: Non-ASCII characters display incorrectly.

Solutions:

1. Ensure UTF-8 encoding

   export LANG=en_US.UTF-8
   export LC_ALL=en_US.UTF-8

2. Check file encoding

   file logs.txt
   # Should show: UTF-8 Unicode text
   
   # Convert if needed
   iconv -f ISO-8859-1 -t UTF-8 logs.txt > logs_utf8.txt

3. Terminal font support

   • Use font with good Unicode support

   • JetBrains Mono, Fira Code, Cascadia Code

Binary or Non-Text Data

Symptom: Binary data causes display issues.

Solution:

# Filter out binary data
strings logs.bin | gonzo

# Or ensure only text logs are processed
file logs.txt  # Should be "text" not "data"

Performance with Complex Formats

Slow Parsing with Complex Regex

Symptom: Custom format with complex regex causes slowdowns.

Solutions:

1. Simplify regex patterns

   # Instead of: .*?(\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\.\d+Z).*
   # Use: (\d{4}-\d{2}-\d{2}T[\d:.]+Z)

2. Reduce backtracking

   • Avoid nested quantifiers: (.*)*

   • Use possessive quantifiers when possible

   • Anchor patterns: ^ and `# Log Format Issues

Troubleshooting guide for log parsing and format detection problems in Gonzo.

Format Detection Issues

Logs Not Being Parsed

Symptom: Structured logs appear as plain text instead of being parsed.

Diagnosis:

Gonzo automatically detects formats per line based on these rules:

• JSON: Lines starting with {

• Logfmt: Lines containing key=value patterns

• Plain text: Everything else

Solutions:

1. Verify JSON is valid

   # Test each line individually
   head -1 logs.json | jq .
   
   # If error, JSON is malformed

2. Check line starts with {

   # JSON must start with opening brace
   head logs.json
   
   # ✅ Good: {"level":"info"...
   # ❌ Bad: 2024-01-15 {"level":"info"...

3. Inspect logfmt format

   # Should have key=value pairs
   head logs.txt
   
   # ✅ Good: level=info service=api msg="started"
   # ❌ Bad: [INFO] service api started

4. Use custom format for non-standard logs

   gonzo --format=my-custom-format -f logs.txt

Mixed Format Logs

Symptom: Some lines parse correctly, others don't.

Explanation: Gonzo detects format per line, so mixed formats will have inconsistent parsing.

Example:

{"level":"info","msg":"API started"}     ← Parsed as JSON
level=info msg="processing request"     ← Parsed as logfmt  
[ERROR] Connection timeout              ← Plain text

Solutions:

1. Accept mixed display

   • This is expected behavior

   • Each format renders appropriately

2. Pre-filter to single format

   # Extract only JSON lines
   grep '^{' mixed.log | gonzo
   
   # Or only logfmt
   grep '=' mixed.log | gonzo

3. Convert to uniform format

   # Convert all to JSON with jq
   cat mixed.log | jq -R -c '. | fromjson? // {"message": .}' | gonzo

Attributes Not Extracted

Symptom: Logs parse but attributes panel is empty.

Causes & Solutions:

1. Plain text logs have no structured attributes

   • Plain text can't be parsed into fields

   • Only JSON and logfmt have attributes

2. Check JSON structure

   # Verify JSON has expected fields
   head -1 logs.json | jq .
   
   # Should show key-value pairs

3. Nested JSON

   {
     "log": {
       "level": "info",
       "service": "api"
     }
   }

   • Gonzo extracts nested attributes

   • May need to check nested paths

4. Missing common attribute names

   • Gonzo looks for common fields: level, service, host, etc.

   • Custom fields may not be highlighted

JSON Issues

Malformed JSON

Symptom: JSON logs show as plain text or cause errors.

Common JSON Issues:

1. Trailing commas

   {"level":"info","msg":"test",}  ❌ Invalid
   {"level":"info","msg":"test"}   ✅ Valid

2. Single quotes instead of double

   {'level':'info'}     ❌ Invalid
   {"level":"info"}     ✅ Valid

3. Unescaped quotes in strings

   {"msg":"He said "hi""}           ❌ Invalid
   {"msg":"He said \"hi\""}         ✅ Valid
   {"msg":"He said 'hi'"}           ✅ Valid (single quotes in string)

4. Missing quotes on keys

   {level:"info"}       ❌ Invalid
   {"level":"info"}     ✅ Valid

Validate & Fix:

# Validate JSON
cat logs.json | jq . > /dev/null
# If errors, shows line number

# Pretty-print to find issues
jq . logs.json

# Fix and re-format
jq -c . logs.json > fixed.json
gonzo -f fixed.json

Multi-line JSON

Symptom: JSON objects span multiple lines, not parsed correctly.

Example:

{
  "level": "info",
  "message": "test"
}

Solution:

Gonzo expects one JSON object per line (JSONL/NDJSON format).

# Compact multi-line JSON to single lines
jq -c . pretty.json > compact.json
gonzo -f compact.json

# Or pipe directly
jq -c . pretty.json | gonzo

JSON with Metadata Prefix

Symptom: Lines have timestamp or metadata before JSON.

Example:

2024-01-15 10:30:05 {"level":"info","msg":"test"}

Solution:

Remove prefix before piping to Gonzo:

# Remove timestamp prefix
sed 's/^[0-9-]* [0-9:]* //' logs.txt | gonzo

# Or use awk
awk '{$1=$2=""; print}' logs.txt | gonzo

# Extract just JSON part
grep -o '{.*}' logs.txt | gonzo

Escaped JSON in Strings

Symptom: JSON contains escaped JSON strings.

Example:

{"log":"{\"level\":\"info\",\"msg\":\"test\"}"}

Solution:

# Unescape inner JSON
jq -r '.log | fromjson' logs.json | gonzo

# Or handle both levels
jq -c '.log | fromjson? // .' logs.json | gonzo

Logfmt Issues

Logfmt Not Detected

Symptom: Key=value logs appear as plain text.

Requirements for logfmt detection:

• Must have key=value patterns

• Multiple pairs per line

• Values can be quoted: key="value with spaces"

Example:

level=info service=api user=123 msg="request completed"  ✅ Detected
INFO service api user 123 request completed              ❌ Not logfmt

Solutions:

1. Verify format

   # Check for key=value pattern
   grep -E '\w+=\w+' logs.txt | head

2. Add more key=value pairs

   • Single pair may not trigger detection

   • Multiple pairs more reliably detected

3. Use custom format

   gonzo --format=my-logfmt -f logs.txt

Spaces in Logfmt Values

Symptom: Values with spaces not parsed correctly.

Examples:

msg=hello world         ❌ Breaks: "world" seen as separate key
msg="hello world"       ✅ Correct: quotes preserve spaces
msg=hello\ world        ✅ Correct: escape preserves spaces

Solution:

Ensure spaces in values are properly quoted or escaped:

# Fix unquoted spaces (requires log generation fix)
# Or accept partial parsing of problematic lines

Logfmt with Nested Structures

Symptom: Nested objects in logfmt don't parse well.

Example:

user.id=123 user.name=john

Explanation: Logfmt is flat by design. Nested structures need JSON.

Solution:

1. Accept flat representation

   • Gonzo extracts user.id and user.name as separate attributes

2. Convert to JSON if needed

   # If you control log format, use JSON for nested data

Plain Text Issues

No Structure Extracted from Text Logs

Symptom: Plain text logs show no attributes.

Explanation: Plain text logs can't be parsed into structured fields automatically.

Examples:

[2024-01-15 10:30:05] ERROR: Connection failed
INFO - api-service - User login successful

Solutions:

1. Accept plain text display

   • Logs still searchable and analyzable

   • Just no structured attributes

2. Create custom format parser

   # Define regex-based parser
   # See Custom Formats Guide
   gonzo --format=my-text-format -f logs.txt

3. Convert logs to structured format

   • Modify application to output JSON/logfmt

   • Use log shipper to add structure (Fluent Bit, Logstash)

Severity Not Detected in Text Logs

Symptom: Plain text logs don't show color-coded severity.

Explanation: Gonzo looks for common severity keywords in text logs:

• ERROR, FATAL, CRITICAL → Red

• WARN, WARNING → Yellow

• INFO → Green

• DEBUG → Blue

• TRACE → White

Solutions:

1. Include severity keywords

   [ERROR] Connection failed       ✅ Detected
   Connection failed              ❌ Not detected
   
   WARN: Low disk space           ✅ Detected  
   Disk space is low              ❌ Not detected

2. Use consistent format

   • Put severity at start of line

   • Use standard keywords (ERROR, WARN, INFO, DEBUG)

3. Create custom format

   • Define severity extraction pattern

   • Map custom levels to standard severities

OTLP Format Issues

OTLP Logs Not Appearing

Symptom: OTLP receiver running but no logs in Gonzo.

Diagnosis:

1. Verify receiver is enabled

   # Check Gonzo started with OTLP
   gonzo --otlp-enabled
   
   # Should show listening on ports 4317 and 4318

2. Check sender configuration

   # Verify endpoint in sender
   endpoint: localhost:4317      # gRPC
   endpoint: http://localhost:4318/v1/logs  # HTTP

3. Test with curl (HTTP)

   curl -X POST http://localhost:4318/v1/logs \
     -H "Content-Type: application/json" \
     -d '{"resourceLogs":[]}'
   
   # Should return 200 OK

4. Check for port conflicts

   lsof -i :4317
   lsof -i :4318

Solutions:

See Common Issues - OTLP Receiver for detailed fixes.

OTLP Attributes Missing

Symptom: OTLP logs appear but without expected attributes.

Causes:

1. Attributes in resource vs log record

   • Resource attributes: service.name, host, etc.

   • Log record attributes: user_id, request_id, etc.

   • Both should be extracted

2. Verify sender includes attributes

   # Ensure attributes are set
   logger_provider.add_log_record_processor(processor)
   # Check resource attributes and log attributes

3. Check attribute names

   • Gonzo shows all attributes

   • May just be named differently than expected

Custom Format Issues

Custom Format Not Working

Symptom: --format=my-format shows error or doesn't parse.

Diagnosis:

1. Verify format file exists

   ls ~/.config/gonzo/formats/my-format.yaml

2. Check YAML syntax

   cat ~/.config/gonzo/formats/my-format.yaml
   
   # Validate if you have yamllint
   yamllint ~/.config/gonzo/formats/my-format.yaml

3. Test with built-in format first

   # Verify custom formats work at all
   gonzo --format=loki-stream -f test.json

Solutions:

See Custom Formats Guide for:

• Format file syntax

• Regex patterns

• Field mapping

• Testing formats

Regex Not Matching

Symptom: Custom format regex doesn't extract fields.

Solutions:

1. Test regex separately

   # Test your regex pattern
   echo "sample log line" | grep -E "your-regex-pattern"

2. Use online regex tester

   • Test at regex101.com

   • Use example log lines

   • Verify capture groups

3. Check for special characters

   # Escape special characters
   pattern: '\[(\d+)\]'  # Brackets escaped

4. Start simple, iterate

   # Begin with basic pattern
   pattern: '(\w+)'
   
   # Then add complexity
   pattern: '(\w+)=(\w+)'
   
   # Finally, full pattern
   pattern: '(\w+)="([^"]*)"'

Encoding Issues

Special Characters Garbled

Symptom: Non-ASCII characters display incorrectly.

Solutions:

1. Ensure UTF-8 encoding

   export LANG=en_US.UTF-8
   export LC_ALL=en_US.UTF-8

2. Check file encoding

   file logs.txt
   # Should show: UTF-8 Unicode text
   
   # Convert if needed
   iconv -f ISO-8859-1 -t UTF-8 logs.txt > logs_utf8.txt

3. Pre-filter logs

   # Filter before complex parsing
   grep "pattern" logs.txt | gonzo --format=complex

4. Use built-in formats when possible

   • JSON and logfmt parsing is optimized

   • Custom regex is slower

Timestamp Issues

Timestamps Not Recognized

Symptom: Logs appear in wrong order or timestamp not extracted.

Common timestamp formats Gonzo recognizes:

2024-01-15T10:30:05Z                    # ISO 8601
2024-01-15T10:30:05.123456Z             # ISO 8601 with microseconds
2024-01-15 10:30:05                      # Common format
Jan 15 10:30:05                          # Syslog format
1705315805                               # Unix timestamp

Solutions:

1. Use ISO 8601 format (recommended)

   {"timestamp":"2024-01-15T10:30:05Z","msg":"test"}

2. Ensure timestamp field name

   • Common names: timestamp, time, @timestamp, ts

   • Gonzo checks these automatically

3. Custom format for unusual timestamps

   • Define timestamp extraction in format file

   • Specify timestamp format

Timezone Issues

Symptom: Timestamps appear in wrong timezone.

Solutions:

1. Use UTC in logs (recommended)

   {"timestamp":"2024-01-15T10:30:05Z"}  # Z indicates UTC

2. Include timezone offset

   {"timestamp":"2024-01-15T10:30:05-05:00"}  # EST

3. Gonzo displays timestamps as received

   • No automatic conversion

   • Format logs consistently at source

Large Log Line Issues

Very Long Lines Truncated

Symptom: Extremely long log lines appear cut off.

Solutions:

1. Use horizontal scrolling

   ← → or h l  # Scroll horizontally

2. View in detail modal

   Enter  # Opens full log entry

3. Split long lines at source

   • Configure application to use reasonable line length

   • Use structured logging to avoid massive single-line logs

Lines Exceed Buffer

Symptom: Some log lines cause errors or don't appear.

Solution:

Gonzo handles lines up to typical buffer limits. For extremely large lines:

# Pre-process to truncate lines
cut -c 1-10000 massive.log | gonzo

# Or filter out problematic lines
awk 'length($0) < 10000' massive.log | gonzo

Debugging Format Issues

Test Format Detection

# Test with minimal sample
echo '{"level":"info","msg":"test"}' | gonzo
# Should parse as JSON

echo 'level=info msg=test' | gonzo
# Should parse as logfmt

echo 'INFO test message' | gonzo
# Should show as plain text

Examine Raw Logs

# Check first few lines
head -5 logs.txt

# Check for hidden characters
cat -A logs.txt | head

# Validate JSON structure
head -1 logs.json | jq .

# Check line endings
file logs.txt  # Shows CRLF vs LF

Compare with Known-Good Format

# Test with working format first
echo '{"level":"info","message":"test"}' > good.json
gonzo -f good.json

# Then compare with problem logs
diff <(head -1 good.json) <(head -1 problem.json)

Common Format Patterns

Application Logs

Go/Logrus:

{"level":"info","msg":"started","time":"2024-01-15T10:30:05Z"}

✅ Parses as JSON automatically

Python/Logging:

2024-01-15 10:30:05,123 INFO module: message

⚠️ Plain text - create custom format for structure

Node.js/Winston:

{"level":"info","message":"started","timestamp":"2024-01-15T10:30:05Z"}

✅ Parses as JSON automatically

System Logs

Syslog:

Jan 15 10:30:05 hostname service[123]: message

⚠️ Plain text - consider custom format

Systemd Journal:

MESSAGE=Test log
PRIORITY=6
_HOSTNAME=server

⚠️ Key=value but special format - needs custom parser

Container Logs

Docker JSON:

{"log":"application log message\n","stream":"stdout","time":"2024-01-15T10:30:05Z"}

✅ Parses as JSON, extracts nested log

Kubernetes:

{"level":"info","msg":"test","pod":"app-123"}

✅ Parses as JSON with K8s attributes

Format Best Practices

When Choosing Log Format

1. Prefer structured formats

   • JSON or logfmt over plain text

   • Easier to parse and analyze

   • Better attribute extraction

2. Use consistent format

   • Same format across all services

   • Easier to aggregate and search

3. Include standard fields

   • level or severity: ERROR, WARN, INFO, DEBUG

   • timestamp: ISO 8601 format

   • message or msg: Human-readable message

   • service or service.name: Service identifier

4. Example good JSON log:

   {
     "timestamp": "2024-01-15T10:30:05Z",
     "level": "error",
     "service": "api",
     "message": "Database connection failed",
     "error": "connection timeout",
     "host": "prod-server-01"
   }

When You Can't Change Format

1. Use custom format definition

   • Create regex-based parser

   • Map fields to standard attributes

2. Pre-process logs

   • Use awk/sed to restructure

   • Convert to JSON/logfmt before Gonzo

3. Use log shipping layer

   • Fluent Bit, Logstash, Vector

   • Transform logs to standard format

Getting Help

Provide This Info for Format Issues

# Sample log lines (3-5 lines)
head -5 logs.txt

# File encoding
file logs.txt

# Attempted command
echo "gonzo -f logs.txt [--format=...]"

# Expected vs actual behavior
echo "Expected: Parse as JSON"
echo "Actual: Shows as plain text"

# Gonzo version
gonzo --version

Resources

• Common Issues - General troubleshooting

• Custom Formats Guide - Creating format parsers

• GitHub Issues - Report format bugs

• Examples Directory - Sample format files

Testing formats? Start with small sample files before processing large logs. This makes debugging much faster.