> For the complete documentation index, see [llms.txt](https://docs.controltheory.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.controltheory.com/controltheory-documentation/gonzo-docs/integration-examples/victorialogs.md).

# VictoriaLogs 🪵

Stream and analyze logs directly from VictoriaLogs with Gonzo's native integration. Leverage VictoriaLogs' high-performance log storage with Gonzo's powerful analysis capabilities, including AI insights and pattern detection.

{% hint style="info" %}
**Native Integration:** Gonzo has built-in VictoriaLogs support with custom field parsing that properly maps VictoriaLogs format to Gonzo's internal log representation.
{% endhint %}

### Why Use Gonzo with VictoriaLogs?

VictoriaLogs is a high-performance, cost-effective log database. Combining it with Gonzo provides:

* **🚀 High-Performance Streaming** - Direct streaming from VictoriaLogs without intermediaries
* **🤖 AI-Powered Analysis** - Intelligent insights on your stored logs
* **📊 Pattern Detection** - Automatic clustering of similar log entries
* **🔍 LogsQL Queries** - Powerful query language for precise log filtering
* **🏷️ Smart Field Mapping** - Automatic mapping of VictoriaLogs fields to Gonzo attributes
* **⚡ Real-Time Analysis** - Live streaming and analysis of logs as they arrive

### Features

#### Custom Field Parsing

Gonzo automatically maps VictoriaLogs fields to provide optimal display and analysis:

| VictoriaLogs Field   | Gonzo Mapping  | Description                              |
| -------------------- | -------------- | ---------------------------------------- |
| `_msg`               | **Log Body**   | Displayed as main message (not raw JSON) |
| `_time`              | **Timestamp**  | Parsed as log timestamp                  |
| `_stream`            | **Attribute**  | Stream identifier                        |
| `_stream_id`         | **Attribute**  | Stream ID                                |
| **All other fields** | **Attributes** | Available in Attributes panel            |

#### Special Field Mappings

**Host Detection** - Checked in priority order:

1. `k8s.node.name` (Kubernetes node)
2. `kubernetes.pod_node_name` (Alternative K8s format)
3. `kubernetes_pod_node_name` (Underscore format)

**Severity Detection** - Checked in priority order:

* `level`
* `severity`
* `log.level`
* `log_level`
* `loglevel`
* `levelname`

#### Format Transformation Example

**VictoriaLogs JSON Input:**

```json
{
  "_msg": "Database connection established",
  "_stream": "app-logs",
  "_stream_id": "stream-001",
  "_time": "2024-01-15T10:30:46.456Z",
  "level": "INFO",
  "k8s.node.name": "node-02",
  "service": "database",
  "pool_size": 10
}
```

**Gonzo Display:**

* **Message:** "Database connection established" ✅ (clean message, not raw JSON)
* **Severity:** INFO (with appropriate color coding)
* **Timestamp:** 2024-01-15T10:30:46.456Z
* **Host:** node-02 (mapped from `k8s.node.name`)
* **Attributes Panel:**
  * `service` = "database"
  * `pool_size` = 10
  * `_stream` = "app-logs"
  * `_stream_id` = "stream-001"

### Configuration

#### Command Line Options

```bash
# Required
--vmlogs-url        # Victoria Logs URL endpoint

# Optional
--vmlogs-user       # Basic auth username
--vmlogs-password   # Basic auth password
--vmlogs-query      # LogsQL query (default: "*")
```

#### Environment Variables

```bash
# Set VictoriaLogs connection parameters
export GONZO_VMLOGS_URL="http://localhost:9428"
export GONZO_VMLOGS_USER="myuser"
export GONZO_VMLOGS_PASSWORD="mypass"
export GONZO_VMLOGS_QUERY="service:'my-app'"

# Then run Gonzo
gonzo
```

#### Configuration File

```yaml
# ~/.config/gonzo/victorialogs.yml
vmlogs-url: "http://localhost:9428"
vmlogs-user: "myuser"
vmlogs-password: "mypass"
vmlogs-query: "*"

# Performance settings for VictoriaLogs
update-interval: 2s
log-buffer: 20000
memory-size: 100000

# AI configuration
ai-model: "gpt-4"
```

**Usage:**

```bash
gonzo --config ~/.config/gonzo/victorialogs.yml
```

### Usage Examples

#### Basic Streaming

```bash
# Stream all logs from VictoriaLogs
gonzo --vmlogs-url="http://localhost:9428" --vmlogs-query="*"

# Stream with AI analysis
gonzo --vmlogs-url="http://localhost:9428" \
      --vmlogs-query="*" \
      --ai-model="gpt-4"

# Stream with custom configuration
gonzo --vmlogs-url="http://localhost:9428" \
      --vmlogs-query="*" \
      --log-buffer=20000 \
      --update-interval=2s
```

#### Authenticated Access

```bash
# Basic authentication
gonzo --vmlogs-url="https://vmlogs.example.com" \
      --vmlogs-user="myuser" \
      --vmlogs-password="mypass" \
      --vmlogs-query="*"

# With specific query
gonzo --vmlogs-url="https://vmlogs.example.com" \
      --vmlogs-user="myuser" \
      --vmlogs-password="mypass" \
      --vmlogs-query='level:error'

# Using environment variables for credentials
export GONZO_VMLOGS_USER="myuser"
export GONZO_VMLOGS_PASSWORD="mypass"
gonzo --vmlogs-url="https://vmlogs.example.com" \
      --vmlogs-query='level:error'
```

#### LogsQL Query Examples

{% tabs %}
{% tab title="Service Filtering" %}

```bash
# Specific service
gonzo --vmlogs-url="http://localhost:9428" \
      --vmlogs-query='service:"payment-processor"'

# Service with severity
gonzo --vmlogs-url="http://localhost:9428" \
      --vmlogs-query='service:"payment-processor" AND level:error'

# Multiple services
gonzo --vmlogs-url="http://localhost:9428" \
      --vmlogs-query='service:("api" OR "worker")'
```

{% endtab %}

{% tab title="Severity Filtering" %}

```bash
# Error logs only
gonzo --vmlogs-url="http://localhost:9428" \
      --vmlogs-query='level:error'

# Errors and warnings
gonzo --vmlogs-url="http://localhost:9428" \
      --vmlogs-query='level:(error OR warning)'

# Exclude debug logs
gonzo --vmlogs-url="http://localhost:9428" \
      --vmlogs-query='NOT level:debug'
```

{% endtab %}

{% tab title="Field-Based Queries" %}

```bash
# Kubernetes node filtering
gonzo --vmlogs-url="http://localhost:9428" \
      --vmlogs-query='k8s.node.name:"node-01"'

# Container name
gonzo --vmlogs-url="http://localhost:9428" \
      --vmlogs-query='kubernetes.container_name:"nginx"'

# Namespace filtering
gonzo --vmlogs-url="http://localhost:9428" \
      --vmlogs-query='kubernetes.namespace:"production"'

# Combined filters
gonzo --vmlogs-url="http://localhost:9428" \
      --vmlogs-query='kubernetes.namespace:"production" AND level:error'
```

{% endtab %}

{% tab title="Text Search" %}

```bash
# Message contains text
gonzo --vmlogs-url="http://localhost:9428" \
      --vmlogs-query='_msg:"database connection"'

# Pattern matching
gonzo --vmlogs-url="http://localhost:9428" \
      --vmlogs-query='_msg:*timeout*'

# Complex text search
gonzo --vmlogs-url="http://localhost:9428" \
      --vmlogs-query='_msg:("error" OR "failed") AND service:"api"'
```

{% endtab %}
{% endtabs %}

### Real-World Integration Patterns

#### Production Monitoring

```bash
# Monitor production errors with AI
gonzo --vmlogs-url="http://vmlogs.prod.company.com" \
      --vmlogs-user="$PROD_USER" \
      --vmlogs-password="$PROD_PASS" \
      --vmlogs-query='kubernetes.namespace:"production" AND level:error' \
      --ai-model="gpt-4"

# Multi-service monitoring
gonzo --vmlogs-url="http://vmlogs.prod.company.com" \
      --vmlogs-query='service:("api" OR "worker" OR "scheduler") AND level:(error OR warning)' \
      --log-buffer=20000
```

#### Kubernetes Log Analysis

```bash
# Specific namespace
gonzo --vmlogs-url="http://localhost:9428" \
      --vmlogs-query='kubernetes.namespace:"production"'

# Pod pattern matching
gonzo --vmlogs-url="http://localhost:9428" \
      --vmlogs-query='kubernetes.pod_name:*backend*'

# Node-specific logs
gonzo --vmlogs-url="http://localhost:9428" \
      --vmlogs-query='k8s.node.name:"node-01" AND level:error'

# Container-specific analysis
gonzo --vmlogs-url="http://localhost:9428" \
      --vmlogs-query='kubernetes.container_name:"nginx" AND level:warning'
```

#### Time-Based Analysis

```bash
# Recent logs (VictoriaLogs handles time filtering)
# Use VictoriaLogs Web UI or API to export specific time ranges
# Then analyze with Gonzo

# Stream recent logs
gonzo --vmlogs-url="http://localhost:9428" \
      --vmlogs-query='_time:>now-1h'

# Specific time range (if supported by your VictoriaLogs version)
gonzo --vmlogs-url="http://localhost:9428" \
      --vmlogs-query='_time:>2024-01-15T10:00:00Z AND _time:<2024-01-15T11:00:00Z'
```

#### Performance Troubleshooting

```bash
# High-duration operations
gonzo --vmlogs-url="http://localhost:9428" \
      --vmlogs-query='duration:>1000'  # Duration > 1000ms

# Slow queries
gonzo --vmlogs-url="http://localhost:9428" \
      --vmlogs-query='_msg:*slow* AND service:"database"'

# Memory warnings
gonzo --vmlogs-url="http://localhost:9428" \
      --vmlogs-query='_msg:(*memory* OR *oom*) AND level:warning'
```

### Implementation Details

#### VictoriaLogs Client Architecture

Gonzo's VictoriaLogs integration includes:

1. **Streaming Client** (`internal/vmlogs/client.go`)
   * Connects to VictoriaLogs `/select/logsql/tail` endpoint
   * Maintains persistent streaming connection
   * Handles reconnection on failures
2. **Format Converter**
   * Transforms VictoriaLogs JSON to OTLP format
   * Preserves all field information
   * Optimizes for Gonzo display
3. **Field Mapper**
   * Handles special fields (`_msg`, `_time`, etc.)
   * Maps host fields in priority order
   * Extracts all attributes automatically
4. **Severity Detector**
   * Identifies log levels from various field names
   * Provides consistent severity detection
   * Enables proper color coding

#### Streaming Endpoint

Gonzo uses the VictoriaLogs streaming tail endpoint:

```
http://victorialogs:9428/select/logsql/tail?query=<logsql_query>
```

This provides:

* Real-time log streaming
* Efficient resource usage
* Native LogsQL query support
* Automatic format conversion

### Configuration Optimization

#### High-Performance Setup

```yaml
# ~/.config/gonzo/vmlogs-production.yml
vmlogs-url: "http://vmlogs.company.com:9428"
vmlogs-user: "production-reader"
vmlogs-password: "${VMLOGS_PASSWORD}"  # From environment
vmlogs-query: "kubernetes.namespace:production AND level:(error OR warning)"

# Optimized for high volume
update-interval: 5s
log-buffer: 50000
memory-size: 200000

# Best AI for production
ai-model: "gpt-4"
ai-context-size: 8000
```

#### Development Setup

```yaml
# ~/.config/gonzo/vmlogs-dev.yml
vmlogs-url: "http://localhost:9428"
vmlogs-query: "*"  # All logs in development

# Fast updates for development
update-interval: 1s
log-buffer: 5000
memory-size: 20000

# Cost-effective AI
ai-model: "gpt-3.5-turbo"
```

### Best Practices

#### 🎯 **Efficient Querying**

```bash
# ✅ Good: Specific queries
gonzo --vmlogs-url="http://localhost:9428" \
      --vmlogs-query='service:"api" AND level:error'

# ❌ Avoid: Overly broad queries
gonzo --vmlogs-url="http://localhost:9428" \
      --vmlogs-query='*'  # All logs (high volume)

# ✅ Good: Use field filters
gonzo --vmlogs-url="http://localhost:9428" \
      --vmlogs-query='kubernetes.namespace:"production"'

# ✅ Good: Combine multiple filters
gonzo --vmlogs-url="http://localhost:9428" \
      --vmlogs-query='service:"api" AND level:error AND kubernetes.namespace:"production"'
```

#### 🔒 **Security Best Practices**

```bash
# Use environment variables for credentials
export GONZO_VMLOGS_USER="readonly-user"
export GONZO_VMLOGS_PASSWORD="secure-password"

# Don't hardcode passwords in commands
gonzo --vmlogs-url="https://vmlogs.company.com"  # Credentials from env

# Use HTTPS for remote connections
gonzo --vmlogs-url="https://vmlogs.company.com" \
      --vmlogs-query='level:error'

# Least privilege: Use read-only credentials
# Create VictoriaLogs user with SELECT-only permissions
```

#### ⚡ **Performance Optimization**

```bash
# For high-volume logs, adjust buffers
gonzo --vmlogs-url="http://localhost:9428" \
      --vmlogs-query='kubernetes.namespace:"production"' \
      --log-buffer=50000 \
      --update-interval=10s

# Use specific queries to reduce volume
gonzo --vmlogs-url="http://localhost:9428" \
      --vmlogs-query='level:(error OR warning)'  # Less volume

# Filter at source when possible
gonzo --vmlogs-url="http://localhost:9428" \
      --vmlogs-query='service:"critical-api" AND level:error'
```

### Troubleshooting

#### Connection Issues

```bash
# Test VictoriaLogs connectivity
curl "http://localhost:9428/select/logsql/query?query=*&limit=10"

# Verify authentication
curl -u myuser:mypass "http://localhost:9428/select/logsql/query?query=*&limit=10"

# Check VictoriaLogs is running
curl "http://localhost:9428/health"

# Test with verbose output
gonzo --vmlogs-url="http://localhost:9428" \
      --vmlogs-query="*" \
      --verbose
```

#### Query Issues

```bash
# Test LogsQL query directly
curl "http://localhost:9428/select/logsql/query?query=level:error&limit=10"

# Verify field names
curl "http://localhost:9428/select/logsql/query?query=*&limit=1" | jq .

# Check query syntax in VictoriaLogs docs
# https://docs.victoriametrics.com/VictoriaLogs/LogsQL.html

# Start with simple query
gonzo --vmlogs-url="http://localhost:9428" --vmlogs-query="*"

# Then add complexity
gonzo --vmlogs-url="http://localhost:9428" --vmlogs-query="level:error"
```

#### Performance Issues

```bash
# Reduce query scope
gonzo --vmlogs-url="http://localhost:9428" \
      --vmlogs-query='service:"specific-service"'

# Increase update interval
gonzo --vmlogs-url="http://localhost:9428" \
      --vmlogs-query="*" \
      --update-interval=10s

# Increase buffer sizes
gonzo --vmlogs-url="http://localhost:9428" \
      --vmlogs-query="*" \
      --log-buffer=50000 \
      --memory-size=100000
```

#### Field Mapping Issues

```bash
# Check raw VictoriaLogs output
curl "http://localhost:9428/select/logsql/query?query=*&limit=5" | jq .

# Verify field names in Gonzo
# Press Tab to navigate to Attributes panel
# Check if expected fields are present

# Ensure using JSON output (default for VictoriaLogs)
# Gonzo automatically handles VictoriaLogs JSON format
```

### What's Next?

Now that you've mastered VictoriaLogs integration, explore related topics:

* **Configuration** - Optimize for VictoriaLogs workflows
* **AI Integration** - Enhanced log analysis with AI
* **Advanced Features** - Pattern detection in VictoriaLogs data
* **Kubernetes Integration** - Combine with K8s log analysis

Or try these advanced VictoriaLogs patterns:

```bash
# Multi-environment monitoring
gonzo --vmlogs-url="http://vmlogs.company.com" \
      --vmlogs-query='kubernetes.namespace:("production" OR "staging") AND level:error' \
      --ai-model="gpt-4"

# Service mesh analysis
gonzo --vmlogs-url="http://localhost:9428" \
      --vmlogs-query='service:"istio-proxy" AND level:warning'

# Custom field analysis
gonzo --vmlogs-url="http://localhost:9428" \
      --vmlogs-query='response_time:>1000 AND service:"api"'
```

***

**Unlock powerful log analysis with VictoriaLogs and Gonzo!** 📊 Native integration, intelligent field mapping, and AI-powered insights make VictoriaLogs logs easy to analyze and understand.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.controltheory.com/controltheory-documentation/gonzo-docs/integration-examples/victorialogs.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.