# Log Format Issues

Troubleshooting guide for log parsing and format detection problems in Gonzo.

### Format Detection Issues

#### Logs Not Being Parsed

**Symptom**: Structured logs appear as plain text instead of being parsed.

**Diagnosis**:

Gonzo automatically detects formats per line based on these rules:

* **JSON**: Lines starting with `{`
* **Logfmt**: Lines containing `key=value` patterns
* **Plain text**: Everything else

**Solutions**:

1. **Verify JSON is valid**

   ```bash
   # Test each line individually
   head -1 logs.json | jq .

   # If error, JSON is malformed
   ```
2. **Check line starts with `{`**

   ```bash
   # JSON must start with opening brace
   head logs.json

   # ✅ Good: {"level":"info"...
   # ❌ Bad: 2024-01-15 {"level":"info"...
   ```
3. **Inspect logfmt format**

   ```bash
   # Should have key=value pairs
   head logs.txt

   # ✅ Good: level=info service=api msg="started"
   # ❌ Bad: [INFO] service api started
   ```
4. **Use custom format for non-standard logs**

   ```bash
   gonzo --format=my-custom-format -f logs.txt
   ```

#### Mixed Format Logs

**Symptom**: Some lines parse correctly, others don't.

**Explanation**: Gonzo detects format per line, so mixed formats will have inconsistent parsing.

**Example**:

```
{"level":"info","msg":"API started"}     ← Parsed as JSON
level=info msg="processing request"     ← Parsed as logfmt  
[ERROR] Connection timeout              ← Plain text
```

**Solutions**:

1. **Accept mixed display**
   * This is expected behavior
   * Each format renders appropriately
2. **Pre-filter to single format**

   ```bash
   # Extract only JSON lines
   grep '^{' mixed.log | gonzo

   # Or only logfmt
   grep '=' mixed.log | gonzo
   ```
3. **Convert to uniform format**

   ```bash
   # Convert all to JSON with jq
   cat mixed.log | jq -R -c '. | fromjson? // {"message": .}' | gonzo
   ```

#### Attributes Not Extracted

**Symptom**: Logs parse but attributes panel is empty.

**Causes & Solutions**:

1. **Plain text logs have no structured attributes**
   * Plain text can't be parsed into fields
   * Only JSON and logfmt have attributes
2. **Check JSON structure**

   ```bash
   # Verify JSON has expected fields
   head -1 logs.json | jq .

   # Should show key-value pairs
   ```
3. **Nested JSON**

   ```json
   {
     "log": {
       "level": "info",
       "service": "api"
     }
   }
   ```

   * Gonzo extracts nested attributes
   * May need to check nested paths
4. **Missing common attribute names**
   * Gonzo looks for common fields: `level`, `service`, `host`, etc.
   * Custom fields may not be highlighted

### JSON Issues

#### Malformed JSON

**Symptom**: JSON logs show as plain text or cause errors.

**Common JSON Issues**:

1. **Trailing commas**

   ```json
   {"level":"info","msg":"test",}  ❌ Invalid
   {"level":"info","msg":"test"}   ✅ Valid
   ```
2. **Single quotes instead of double**

   ```json
   {'level':'info'}     ❌ Invalid
   {"level":"info"}     ✅ Valid
   ```
3. **Unescaped quotes in strings**

   ```json
   {"msg":"He said "hi""}           ❌ Invalid
   {"msg":"He said \"hi\""}         ✅ Valid
   {"msg":"He said 'hi'"}           ✅ Valid (single quotes in string)
   ```
4. **Missing quotes on keys**

   ```json
   {level:"info"}       ❌ Invalid
   {"level":"info"}     ✅ Valid
   ```

**Validate & Fix**:

```bash
# Validate JSON
cat logs.json | jq . > /dev/null
# If errors, shows line number

# Pretty-print to find issues
jq . logs.json

# Fix and re-format
jq -c . logs.json > fixed.json
gonzo -f fixed.json
```

#### Multi-line JSON

**Symptom**: JSON objects span multiple lines, not parsed correctly.

**Example**:

```json
{
  "level": "info",
  "message": "test"
}
```

**Solution**:

Gonzo expects one JSON object per line (JSONL/NDJSON format).

```bash
# Compact multi-line JSON to single lines
jq -c . pretty.json > compact.json
gonzo -f compact.json

# Or pipe directly
jq -c . pretty.json | gonzo
```

#### JSON with Metadata Prefix

**Symptom**: Lines have timestamp or metadata before JSON.

**Example**:

```
2024-01-15 10:30:05 {"level":"info","msg":"test"}
```

**Solution**:

Remove prefix before piping to Gonzo:

```bash
# Remove timestamp prefix
sed 's/^[0-9-]* [0-9:]* //' logs.txt | gonzo

# Or use awk
awk '{$1=$2=""; print}' logs.txt | gonzo

# Extract just JSON part
grep -o '{.*}' logs.txt | gonzo
```

#### Escaped JSON in Strings

**Symptom**: JSON contains escaped JSON strings.

**Example**:

```json
{"log":"{\"level\":\"info\",\"msg\":\"test\"}"}
```

**Solution**:

```bash
# Unescape inner JSON
jq -r '.log | fromjson' logs.json | gonzo

# Or handle both levels
jq -c '.log | fromjson? // .' logs.json | gonzo
```

### Logfmt Issues

#### Logfmt Not Detected

**Symptom**: Key=value logs appear as plain text.

**Requirements for logfmt detection**:

* Must have `key=value` patterns
* Multiple pairs per line
* Values can be quoted: `key="value with spaces"`

**Example**:

```
level=info service=api user=123 msg="request completed"  ✅ Detected
INFO service api user 123 request completed              ❌ Not logfmt
```

**Solutions**:

1. **Verify format**

   ```bash
   # Check for key=value pattern
   grep -E '\w+=\w+' logs.txt | head
   ```
2. **Add more key=value pairs**
   * Single pair may not trigger detection
   * Multiple pairs more reliably detected
3. **Use custom format**

   ```bash
   gonzo --format=my-logfmt -f logs.txt
   ```

#### Spaces in Logfmt Values

**Symptom**: Values with spaces not parsed correctly.

**Examples**:

```
msg=hello world         ❌ Breaks: "world" seen as separate key
msg="hello world"       ✅ Correct: quotes preserve spaces
msg=hello\ world        ✅ Correct: escape preserves spaces
```

**Solution**:

Ensure spaces in values are properly quoted or escaped:

```bash
# Fix unquoted spaces (requires log generation fix)
# Or accept partial parsing of problematic lines
```

#### Logfmt with Nested Structures

**Symptom**: Nested objects in logfmt don't parse well.

**Example**:

```
user.id=123 user.name=john
```

**Explanation**: Logfmt is flat by design. Nested structures need JSON.

**Solution**:

1. **Accept flat representation**
   * Gonzo extracts `user.id` and `user.name` as separate attributes
2. **Convert to JSON if needed**

   ```bash
   # If you control log format, use JSON for nested data
   ```

### Plain Text Issues

#### No Structure Extracted from Text Logs

**Symptom**: Plain text logs show no attributes.

**Explanation**: Plain text logs can't be parsed into structured fields automatically.

**Examples**:

```
[2024-01-15 10:30:05] ERROR: Connection failed
INFO - api-service - User login successful
```

**Solutions**:

1. **Accept plain text display**
   * Logs still searchable and analyzable
   * Just no structured attributes
2. **Create custom format parser**

   ```bash
   # Define regex-based parser
   # See Custom Formats Guide
   gonzo --format=my-text-format -f logs.txt
   ```
3. **Convert logs to structured format**
   * Modify application to output JSON/logfmt
   * Use log shipper to add structure (Fluent Bit, Logstash)

#### Severity Not Detected in Text Logs

**Symptom**: Plain text logs don't show color-coded severity.

**Explanation**: Gonzo looks for common severity keywords in text logs:

* ERROR, FATAL, CRITICAL → Red
* WARN, WARNING → Yellow
* INFO → Green
* DEBUG → Blue
* TRACE → White

**Solutions**:

1. **Include severity keywords**

   ```
   [ERROR] Connection failed       ✅ Detected
   Connection failed              ❌ Not detected

   WARN: Low disk space           ✅ Detected  
   Disk space is low              ❌ Not detected
   ```
2. **Use consistent format**
   * Put severity at start of line
   * Use standard keywords (ERROR, WARN, INFO, DEBUG)
3. **Create custom format**
   * Define severity extraction pattern
   * Map custom levels to standard severities

### OTLP Format Issues

#### OTLP Logs Not Appearing

**Symptom**: OTLP receiver running but no logs in Gonzo.

**Diagnosis**:

1. **Verify receiver is enabled**

   ```bash
   # Check Gonzo started with OTLP
   gonzo --otlp-enabled

   # Should show listening on ports 4317 and 4318
   ```
2. **Check sender configuration**

   ```yaml
   # Verify endpoint in sender
   endpoint: localhost:4317      # gRPC
   endpoint: http://localhost:4318/v1/logs  # HTTP
   ```
3. **Test with curl (HTTP)**

   ```bash
   curl -X POST http://localhost:4318/v1/logs \
     -H "Content-Type: application/json" \
     -d '{"resourceLogs":[]}'

   # Should return 200 OK
   ```
4. **Check for port conflicts**

   ```bash
   lsof -i :4317
   lsof -i :4318
   ```

**Solutions**:

See Common Issues - OTLP Receiver for detailed fixes.

#### OTLP Attributes Missing

**Symptom**: OTLP logs appear but without expected attributes.

**Causes**:

1. **Attributes in resource vs log record**
   * Resource attributes: service.name, host, etc.
   * Log record attributes: user\_id, request\_id, etc.
   * Both should be extracted
2. **Verify sender includes attributes**

   ```python
   # Ensure attributes are set
   logger_provider.add_log_record_processor(processor)
   # Check resource attributes and log attributes
   ```
3. **Check attribute names**
   * Gonzo shows all attributes
   * May just be named differently than expected

### Custom Format Issues

#### Custom Format Not Working

**Symptom**: `--format=my-format` shows error or doesn't parse.

**Diagnosis**:

1. **Verify format file exists**

   ```bash
   ls ~/.config/gonzo/formats/my-format.yaml
   ```
2. **Check YAML syntax**

   ```bash
   cat ~/.config/gonzo/formats/my-format.yaml

   # Validate if you have yamllint
   yamllint ~/.config/gonzo/formats/my-format.yaml
   ```
3. **Test with built-in format first**

   ```bash
   # Verify custom formats work at all
   gonzo --format=loki-stream -f test.json
   ```

**Solutions**:

See Custom Formats Guide for:

* Format file syntax
* Regex patterns
* Field mapping
* Testing formats

#### Regex Not Matching

**Symptom**: Custom format regex doesn't extract fields.

**Solutions**:

1. **Test regex separately**

   ```bash
   # Test your regex pattern
   echo "sample log line" | grep -E "your-regex-pattern"
   ```
2. **Use online regex tester**
   * Test at regex101.com
   * Use example log lines
   * Verify capture groups
3. **Check for special characters**

   ```yaml
   # Escape special characters
   pattern: '\[(\d+)\]'  # Brackets escaped
   ```
4. **Start simple, iterate**

   ```yaml
   # Begin with basic pattern
   pattern: '(\w+)'

   # Then add complexity
   pattern: '(\w+)=(\w+)'

   # Finally, full pattern
   pattern: '(\w+)="([^"]*)"'
   ```

### Encoding Issues

#### Special Characters Garbled

**Symptom**: Non-ASCII characters display incorrectly.

**Solutions**:

1. **Ensure UTF-8 encoding**

   ```bash
   export LANG=en_US.UTF-8
   export LC_ALL=en_US.UTF-8
   ```
2. **Check file encoding**

   ```bash
   file logs.txt
   # Should show: UTF-8 Unicode text

   # Convert if needed
   iconv -f ISO-8859-1 -t UTF-8 logs.txt > logs_utf8.txt
   ```
3. **Terminal font support**
   * Use font with good Unicode support
   * JetBrains Mono, Fira Code, Cascadia Code

#### Binary or Non-Text Data

**Symptom**: Binary data causes display issues.

**Solution**:

```bash
# Filter out binary data
strings logs.bin | gonzo

# Or ensure only text logs are processed
file logs.txt  # Should be "text" not "data"
```

### Performance with Complex Formats

#### Slow Parsing with Complex Regex

**Symptom**: Custom format with complex regex causes slowdowns.

**Solutions**:

1. **Simplify regex patterns**

   ```yaml
   # Instead of: .*?(\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\.\d+Z).*
   # Use: (\d{4}-\d{2}-\d{2}T[\d:.]+Z)
   ```
2. **Reduce backtracking**
   * Avoid nested quantifiers: `(.*)*`
   * Use possessive quantifiers when possible
   * Anchor patterns: `^` and \`# Log Format Issues

Troubleshooting guide for log parsing and format detection problems in Gonzo.

### Format Detection Issues

#### Logs Not Being Parsed

**Symptom**: Structured logs appear as plain text instead of being parsed.

**Diagnosis**:

Gonzo automatically detects formats per line based on these rules:

* **JSON**: Lines starting with `{`
* **Logfmt**: Lines containing `key=value` patterns
* **Plain text**: Everything else

**Solutions**:

1. **Verify JSON is valid**

   ```bash
   # Test each line individually
   head -1 logs.json | jq .

   # If error, JSON is malformed
   ```
2. **Check line starts with `{`**

   ```bash
   # JSON must start with opening brace
   head logs.json

   # ✅ Good: {"level":"info"...
   # ❌ Bad: 2024-01-15 {"level":"info"...
   ```
3. **Inspect logfmt format**

   ```bash
   # Should have key=value pairs
   head logs.txt

   # ✅ Good: level=info service=api msg="started"
   # ❌ Bad: [INFO] service api started
   ```
4. **Use custom format for non-standard logs**

   ```bash
   gonzo --format=my-custom-format -f logs.txt
   ```

#### Mixed Format Logs

**Symptom**: Some lines parse correctly, others don't.

**Explanation**: Gonzo detects format per line, so mixed formats will have inconsistent parsing.

**Example**:

```
{"level":"info","msg":"API started"}     ← Parsed as JSON
level=info msg="processing request"     ← Parsed as logfmt  
[ERROR] Connection timeout              ← Plain text
```

**Solutions**:

1. **Accept mixed display**
   * This is expected behavior
   * Each format renders appropriately
2. **Pre-filter to single format**

   ```bash
   # Extract only JSON lines
   grep '^{' mixed.log | gonzo

   # Or only logfmt
   grep '=' mixed.log | gonzo
   ```
3. **Convert to uniform format**

   ```bash
   # Convert all to JSON with jq
   cat mixed.log | jq -R -c '. | fromjson? // {"message": .}' | gonzo
   ```

#### Attributes Not Extracted

**Symptom**: Logs parse but attributes panel is empty.

**Causes & Solutions**:

1. **Plain text logs have no structured attributes**
   * Plain text can't be parsed into fields
   * Only JSON and logfmt have attributes
2. **Check JSON structure**

   ```bash
   # Verify JSON has expected fields
   head -1 logs.json | jq .

   # Should show key-value pairs
   ```
3. **Nested JSON**

   ```json
   {
     "log": {
       "level": "info",
       "service": "api"
     }
   }
   ```

   * Gonzo extracts nested attributes
   * May need to check nested paths
4. **Missing common attribute names**
   * Gonzo looks for common fields: `level`, `service`, `host`, etc.
   * Custom fields may not be highlighted

### JSON Issues

#### Malformed JSON

**Symptom**: JSON logs show as plain text or cause errors.

**Common JSON Issues**:

1. **Trailing commas**

   ```json
   {"level":"info","msg":"test",}  ❌ Invalid
   {"level":"info","msg":"test"}   ✅ Valid
   ```
2. **Single quotes instead of double**

   ```json
   {'level':'info'}     ❌ Invalid
   {"level":"info"}     ✅ Valid
   ```
3. **Unescaped quotes in strings**

   ```json
   {"msg":"He said "hi""}           ❌ Invalid
   {"msg":"He said \"hi\""}         ✅ Valid
   {"msg":"He said 'hi'"}           ✅ Valid (single quotes in string)
   ```
4. **Missing quotes on keys**

   ```json
   {level:"info"}       ❌ Invalid
   {"level":"info"}     ✅ Valid
   ```

**Validate & Fix**:

```bash
# Validate JSON
cat logs.json | jq . > /dev/null
# If errors, shows line number

# Pretty-print to find issues
jq . logs.json

# Fix and re-format
jq -c . logs.json > fixed.json
gonzo -f fixed.json
```

#### Multi-line JSON

**Symptom**: JSON objects span multiple lines, not parsed correctly.

**Example**:

```json
{
  "level": "info",
  "message": "test"
}
```

**Solution**:

Gonzo expects one JSON object per line (JSONL/NDJSON format).

```bash
# Compact multi-line JSON to single lines
jq -c . pretty.json > compact.json
gonzo -f compact.json

# Or pipe directly
jq -c . pretty.json | gonzo
```

#### JSON with Metadata Prefix

**Symptom**: Lines have timestamp or metadata before JSON.

**Example**:

```
2024-01-15 10:30:05 {"level":"info","msg":"test"}
```

**Solution**:

Remove prefix before piping to Gonzo:

```bash
# Remove timestamp prefix
sed 's/^[0-9-]* [0-9:]* //' logs.txt | gonzo

# Or use awk
awk '{$1=$2=""; print}' logs.txt | gonzo

# Extract just JSON part
grep -o '{.*}' logs.txt | gonzo
```

#### Escaped JSON in Strings

**Symptom**: JSON contains escaped JSON strings.

**Example**:

```json
{"log":"{\"level\":\"info\",\"msg\":\"test\"}"}
```

**Solution**:

```bash
# Unescape inner JSON
jq -r '.log | fromjson' logs.json | gonzo

# Or handle both levels
jq -c '.log | fromjson? // .' logs.json | gonzo
```

### Logfmt Issues

#### Logfmt Not Detected

**Symptom**: Key=value logs appear as plain text.

**Requirements for logfmt detection**:

* Must have `key=value` patterns
* Multiple pairs per line
* Values can be quoted: `key="value with spaces"`

**Example**:

```
level=info service=api user=123 msg="request completed"  ✅ Detected
INFO service api user 123 request completed              ❌ Not logfmt
```

**Solutions**:

1. **Verify format**

   ```bash
   # Check for key=value pattern
   grep -E '\w+=\w+' logs.txt | head
   ```
2. **Add more key=value pairs**
   * Single pair may not trigger detection
   * Multiple pairs more reliably detected
3. **Use custom format**

   ```bash
   gonzo --format=my-logfmt -f logs.txt
   ```

#### Spaces in Logfmt Values

**Symptom**: Values with spaces not parsed correctly.

**Examples**:

```
msg=hello world         ❌ Breaks: "world" seen as separate key
msg="hello world"       ✅ Correct: quotes preserve spaces
msg=hello\ world        ✅ Correct: escape preserves spaces
```

**Solution**:

Ensure spaces in values are properly quoted or escaped:

```bash
# Fix unquoted spaces (requires log generation fix)
# Or accept partial parsing of problematic lines
```

#### Logfmt with Nested Structures

**Symptom**: Nested objects in logfmt don't parse well.

**Example**:

```
user.id=123 user.name=john
```

**Explanation**: Logfmt is flat by design. Nested structures need JSON.

**Solution**:

1. **Accept flat representation**
   * Gonzo extracts `user.id` and `user.name` as separate attributes
2. **Convert to JSON if needed**

   ```bash
   # If you control log format, use JSON for nested data
   ```

### Plain Text Issues

#### No Structure Extracted from Text Logs

**Symptom**: Plain text logs show no attributes.

**Explanation**: Plain text logs can't be parsed into structured fields automatically.

**Examples**:

```
[2024-01-15 10:30:05] ERROR: Connection failed
INFO - api-service - User login successful
```

**Solutions**:

1. **Accept plain text display**
   * Logs still searchable and analyzable
   * Just no structured attributes
2. **Create custom format parser**

   ```bash
   # Define regex-based parser
   # See Custom Formats Guide
   gonzo --format=my-text-format -f logs.txt
   ```
3. **Convert logs to structured format**
   * Modify application to output JSON/logfmt
   * Use log shipper to add structure (Fluent Bit, Logstash)

#### Severity Not Detected in Text Logs

**Symptom**: Plain text logs don't show color-coded severity.

**Explanation**: Gonzo looks for common severity keywords in text logs:

* ERROR, FATAL, CRITICAL → Red
* WARN, WARNING → Yellow
* INFO → Green
* DEBUG → Blue
* TRACE → White

**Solutions**:

1. **Include severity keywords**

   ```
   [ERROR] Connection failed       ✅ Detected
   Connection failed              ❌ Not detected

   WARN: Low disk space           ✅ Detected  
   Disk space is low              ❌ Not detected
   ```
2. **Use consistent format**
   * Put severity at start of line
   * Use standard keywords (ERROR, WARN, INFO, DEBUG)
3. **Create custom format**
   * Define severity extraction pattern
   * Map custom levels to standard severities

### OTLP Format Issues

#### OTLP Logs Not Appearing

**Symptom**: OTLP receiver running but no logs in Gonzo.

**Diagnosis**:

1. **Verify receiver is enabled**

   ```bash
   # Check Gonzo started with OTLP
   gonzo --otlp-enabled

   # Should show listening on ports 4317 and 4318
   ```
2. **Check sender configuration**

   ```yaml
   # Verify endpoint in sender
   endpoint: localhost:4317      # gRPC
   endpoint: http://localhost:4318/v1/logs  # HTTP
   ```
3. **Test with curl (HTTP)**

   ```bash
   curl -X POST http://localhost:4318/v1/logs \
     -H "Content-Type: application/json" \
     -d '{"resourceLogs":[]}'

   # Should return 200 OK
   ```
4. **Check for port conflicts**

   ```bash
   lsof -i :4317
   lsof -i :4318
   ```

**Solutions**:

See Common Issues - OTLP Receiver for detailed fixes.

#### OTLP Attributes Missing

**Symptom**: OTLP logs appear but without expected attributes.

**Causes**:

1. **Attributes in resource vs log record**
   * Resource attributes: service.name, host, etc.
   * Log record attributes: user\_id, request\_id, etc.
   * Both should be extracted
2. **Verify sender includes attributes**

   ```python
   # Ensure attributes are set
   logger_provider.add_log_record_processor(processor)
   # Check resource attributes and log attributes
   ```
3. **Check attribute names**
   * Gonzo shows all attributes
   * May just be named differently than expected

### Custom Format Issues

#### Custom Format Not Working

**Symptom**: `--format=my-format` shows error or doesn't parse.

**Diagnosis**:

1. **Verify format file exists**

   ```bash
   ls ~/.config/gonzo/formats/my-format.yaml
   ```
2. **Check YAML syntax**

   ```bash
   cat ~/.config/gonzo/formats/my-format.yaml

   # Validate if you have yamllint
   yamllint ~/.config/gonzo/formats/my-format.yaml
   ```
3. **Test with built-in format first**

   ```bash
   # Verify custom formats work at all
   gonzo --format=loki-stream -f test.json
   ```

**Solutions**:

See Custom Formats Guide for:

* Format file syntax
* Regex patterns
* Field mapping
* Testing formats

#### Regex Not Matching

**Symptom**: Custom format regex doesn't extract fields.

**Solutions**:

1. **Test regex separately**

   ```bash
   # Test your regex pattern
   echo "sample log line" | grep -E "your-regex-pattern"
   ```
2. **Use online regex tester**
   * Test at regex101.com
   * Use example log lines
   * Verify capture groups
3. **Check for special characters**

   ```yaml
   # Escape special characters
   pattern: '\[(\d+)\]'  # Brackets escaped
   ```
4. **Start simple, iterate**

   ```yaml
   # Begin with basic pattern
   pattern: '(\w+)'

   # Then add complexity
   pattern: '(\w+)=(\w+)'

   # Finally, full pattern
   pattern: '(\w+)="([^"]*)"'
   ```

### Encoding Issues

#### Special Characters Garbled

**Symptom**: Non-ASCII characters display incorrectly.

**Solutions**:

1. **Ensure UTF-8 encoding**

   ```bash
   export LANG=en_US.UTF-8
   export LC_ALL=en_US.UTF-8
   ```
2. **Check file encoding**

   ```bash
   file logs.txt
   # Should show: UTF-8 Unicode text

   # Convert if needed
   iconv -f ISO-8859-1 -t UTF-8 logs.txt > logs_utf8.txt
   ```
3. **Pre-filter logs**

   ```bash
   # Filter before complex parsing
   grep "pattern" logs.txt | gonzo --format=complex
   ```
4. **Use built-in formats when possible**
   * JSON and logfmt parsing is optimized
   * Custom regex is slower

### Timestamp Issues

#### Timestamps Not Recognized

**Symptom**: Logs appear in wrong order or timestamp not extracted.

**Common timestamp formats Gonzo recognizes**:

```
2024-01-15T10:30:05Z                    # ISO 8601
2024-01-15T10:30:05.123456Z             # ISO 8601 with microseconds
2024-01-15 10:30:05                      # Common format
Jan 15 10:30:05                          # Syslog format
1705315805                               # Unix timestamp
```

**Solutions**:

1. **Use ISO 8601 format (recommended)**

   ```json
   {"timestamp":"2024-01-15T10:30:05Z","msg":"test"}
   ```
2. **Ensure timestamp field name**
   * Common names: `timestamp`, `time`, `@timestamp`, `ts`
   * Gonzo checks these automatically
3. **Custom format for unusual timestamps**
   * Define timestamp extraction in format file
   * Specify timestamp format

#### Timezone Issues

**Symptom**: Timestamps appear in wrong timezone.

**Solutions**:

1. **Use UTC in logs (recommended)**

   ```json
   {"timestamp":"2024-01-15T10:30:05Z"}  # Z indicates UTC
   ```
2. **Include timezone offset**

   ```json
   {"timestamp":"2024-01-15T10:30:05-05:00"}  # EST
   ```
3. **Gonzo displays timestamps as received**
   * No automatic conversion
   * Format logs consistently at source

### Large Log Line Issues

#### Very Long Lines Truncated

**Symptom**: Extremely long log lines appear cut off.

**Solutions**:

1. **Use horizontal scrolling**

   ```
   ← → or h l  # Scroll horizontally
   ```
2. **View in detail modal**

   ```
   Enter  # Opens full log entry
   ```
3. **Split long lines at source**
   * Configure application to use reasonable line length
   * Use structured logging to avoid massive single-line logs

#### Lines Exceed Buffer

**Symptom**: Some log lines cause errors or don't appear.

**Solution**:

Gonzo handles lines up to typical buffer limits. For extremely large lines:

```bash
# Pre-process to truncate lines
cut -c 1-10000 massive.log | gonzo

# Or filter out problematic lines
awk 'length($0) < 10000' massive.log | gonzo
```

### Debugging Format Issues

#### Test Format Detection

```bash
# Test with minimal sample
echo '{"level":"info","msg":"test"}' | gonzo
# Should parse as JSON

echo 'level=info msg=test' | gonzo
# Should parse as logfmt

echo 'INFO test message' | gonzo
# Should show as plain text
```

#### Examine Raw Logs

```bash
# Check first few lines
head -5 logs.txt

# Check for hidden characters
cat -A logs.txt | head

# Validate JSON structure
head -1 logs.json | jq .

# Check line endings
file logs.txt  # Shows CRLF vs LF
```

#### Compare with Known-Good Format

```bash
# Test with working format first
echo '{"level":"info","message":"test"}' > good.json
gonzo -f good.json

# Then compare with problem logs
diff <(head -1 good.json) <(head -1 problem.json)
```

### Common Format Patterns

#### Application Logs

**Go/Logrus**:

```json
{"level":"info","msg":"started","time":"2024-01-15T10:30:05Z"}
```

✅ Parses as JSON automatically

**Python/Logging**:

```
2024-01-15 10:30:05,123 INFO module: message
```

⚠️ Plain text - create custom format for structure

**Node.js/Winston**:

```json
{"level":"info","message":"started","timestamp":"2024-01-15T10:30:05Z"}
```

✅ Parses as JSON automatically

#### System Logs

**Syslog**:

```
Jan 15 10:30:05 hostname service[123]: message
```

⚠️ Plain text - consider custom format

**Systemd Journal**:

```
MESSAGE=Test log
PRIORITY=6
_HOSTNAME=server
```

⚠️ Key=value but special format - needs custom parser

#### Container Logs

**Docker JSON**:

```json
{"log":"application log message\n","stream":"stdout","time":"2024-01-15T10:30:05Z"}
```

✅ Parses as JSON, extracts nested log

**Kubernetes**:

```json
{"level":"info","msg":"test","pod":"app-123"}
```

✅ Parses as JSON with K8s attributes

### Format Best Practices

#### When Choosing Log Format

1. **Prefer structured formats**
   * JSON or logfmt over plain text
   * Easier to parse and analyze
   * Better attribute extraction
2. **Use consistent format**
   * Same format across all services
   * Easier to aggregate and search
3. **Include standard fields**
   * `level` or `severity`: ERROR, WARN, INFO, DEBUG
   * `timestamp`: ISO 8601 format
   * `message` or `msg`: Human-readable message
   * `service` or `service.name`: Service identifier
4. **Example good JSON log**:

   ```json
   {
     "timestamp": "2024-01-15T10:30:05Z",
     "level": "error",
     "service": "api",
     "message": "Database connection failed",
     "error": "connection timeout",
     "host": "prod-server-01"
   }
   ```

#### When You Can't Change Format

1. **Use custom format definition**
   * Create regex-based parser
   * Map fields to standard attributes
2. **Pre-process logs**
   * Use awk/sed to restructure
   * Convert to JSON/logfmt before Gonzo
3. **Use log shipping layer**
   * Fluent Bit, Logstash, Vector
   * Transform logs to standard format

### Getting Help

#### Provide This Info for Format Issues

```bash
# Sample log lines (3-5 lines)
head -5 logs.txt

# File encoding
file logs.txt

# Attempted command
echo "gonzo -f logs.txt [--format=...]"

# Expected vs actual behavior
echo "Expected: Parse as JSON"
echo "Actual: Shows as plain text"

# Gonzo version
gonzo --version
```

#### Resources

* **Common Issues** - General troubleshooting
* **Custom Formats Guide** - Creating format parsers
* [**GitHub Issues**](https://github.com/control-theory/gonzo/issues) - Report format bugs
* [**Examples Directory**](https://github.com/control-theory/gonzo/tree/main/examples) - Sample format files

{% hint style="info" %}
**Testing formats?** Start with small sample files before processing large logs. This makes debugging much faster.
{% endhint %}