# Filtering & Search Master Gonzo's powerful filtering and search capabilities to quickly find the logs that matter. From simple text matching to advanced regex patterns, this guide covers all techniques for efficient log analysis. {% hint style="info" %} **Quick Start:** Press `/` to filter logs with regex patterns, or `s` to highlight text without filtering. Most users start with simple text filters and progress to regex mastery. {% endhint %} ### Filtering vs Search Overview Gonzo provides two complementary approaches to finding relevant logs: | Feature | Key | Behavior | Best For | | ----------------------- | ---------------- | -------------------------------------- | --------------------------------------- | | **🔍 Filtering** | `/` | Shows only matching logs, hides others | Reducing data volume, focusing analysis | | **📍 Search** | `s` | Highlights matches, shows all logs | Finding text while preserving context | | **🎯 Attribute Filter** | Click attributes | Filters by structured data values | JSON/logfmt field-based analysis | | **📊 Word Filter** | Click words | Filters by frequency panel selections | Quick filtering from common terms | ### Basic Filtering Techniques #### Text-Based Filtering The simplest way to filter logs is by searching for specific text: ```bash # Enter filter mode with '/' # Then type any of these patterns: error # Show logs containing "error" database # Show logs containing "database" timeout # Show logs containing "timeout" user_id # Show logs containing "user_id" ``` **Workflow:** 1. Press `/` from any panel 2. Type your search term 3. Press `Enter` to apply filter 4. Navigate filtered results with normal keys 5. Press `/` again and clear to remove filter #### Case Sensitivity ```bash # Case-insensitive by default error # Matches: error, Error, ERROR, eRrOr # Force case-sensitive with regex (?-i)Error # Matches only: Error (not error or ERROR) # Mixed case patterns [Ee]rror # Matches: Error or error (but not ERROR) ``` #### Multiple Term Filtering ```bash # AND logic - both terms must appear error.*database # Logs containing both "error" AND "database" user.*login.*fail # Logs with "user" AND "login" AND "fail" # OR logic - either term can appear (error|warning) # Logs containing "error" OR "warning" (timeout|slow|lag) # Performance-related terms ``` ### Regular Expression Filtering Gonzo supports full regex patterns for powerful filtering: #### Basic Regex Patterns ```bash # Exact word boundaries \berror\b # "error" as complete word (not "errors") \btimeout\b # "timeout" but not "timeouts" # Start/end of line ^ERROR # Lines starting with "ERROR" failed$ # Lines ending with "failed" # Any character wildcards user.id # "user_id", "user-id", "userid", etc. data.*base # "database", "data_base", "data-base", etc. ``` #### Advanced Regex Techniques {% tabs %} {% tab title="Log Level Patterns" %} ```bash # Specific log levels level.*error # JSON: "level":"error" level.*warn # JSON: "level":"warn" \[(ERROR|WARN)\] # Plain text: [ERROR] or [WARN] # Multiple severity levels level.*(error|warn|fatal) # Multiple JSON levels \[(ERROR|WARN|FATAL)\] # Multiple plain text levels # Exclude debug logs ^(?!.*DEBUG).* # Everything except DEBUG logs level.*(?!debug) # JSON levels except debug ``` {% endtab %} {% tab title="Service & Component" %} ```bash # Specific services service.*api # JSON: "service":"api" service.*(web|api) # Multiple services # Component patterns component.*auth # Authentication component module.*payment # Payment module # Microservice patterns (user-service|payment-service|auth-service) ``` {% endtab %} {% tab title="Time & Date Patterns" %} ```bash # Specific dates 2024-01-15 # January 15, 2024 2024-01-1[5-9] # January 15-19, 2024 # Time ranges (10|11|12):[0-5][0-9] # 10 AM to 12:59 PM 1[0-2]:[0-5][0-9] # 10 AM to 12:59 PM # Recent timestamps (last hour) $(date +%Y-%m-%d\ %H) # Current hour (requires shell) ``` {% endtab %} {% tab title="Error & Exception Patterns" %} ```bash # Exception types (Exception|Error|Fault)$ # Lines ending with exception types NullPointer.*Exception # Specific exception types timeout.*exception # Timeout-related exceptions # HTTP status codes status.*[45][0-9]{2} # 4xx and 5xx HTTP status codes "status":\s*[45][0-9]{2} # JSON HTTP error statuses # Database errors (connection.*refused|deadlock|constraint) ``` {% endtab %} {% endtabs %} #### Performance-Oriented Regex ```bash # Efficient patterns (fast) ^ERROR # Anchored to start \berror\b # Word boundaries (error|warn) # Simple alternation # Avoid these patterns (slow on large datasets) .*error.* # Unnecessary .* at start (.*error.*|.*warn.*) # Complex nested patterns error.*database.*timeout # Too many .* wildcards ``` ### Structured Data Filtering For JSON and logfmt logs, leverage structured field filtering: #### JSON Log Filtering ```bash # Field-specific filtering "level":"error" # Exact JSON field match "user_id":"12345" # Specific user "status":50[0-9] # 500-509 status codes "duration":[1-9][0-9]+ # Duration > 10 # Nested field filtering "request".*"method":"POST" # Nested object fields "metadata".*"trace_id" # Nested trace information # Array and complex structures "tags".*"production" # Array containing "production" "errors".*\[.*\] # Non-empty error arrays ``` #### Logfmt Filtering ```bash # Key-value pattern filtering level=error # Exact logfmt match user_id=12345 # Specific user ID method=(POST|PUT|DELETE) # HTTP methods duration=[1-9][0-9]+ # Numeric values # Combined logfmt patterns level=error.*user_id= # Error logs with user context method=POST.*status=4[0-9]{2} # POST requests with 4xx errors ``` #### Attribute-Based Filtering Use the Attributes panel for point-and-click filtering: 1. **Navigate to Attributes panel** (bottom-left) 2. **Click any attribute name or value** 3. **Gonzo automatically creates filter** for that attribute 4. **Refine with additional patterns** if needed **Example Workflow:** ```bash # 1. See "trace_id": "abc123" in Attributes panel # 2. Click "abc123" # 3. Filter automatically becomes: trace_id.*abc123 # 4. All logs with that trace ID are shown ``` ### Search and Highlighting Use search mode to find text while preserving full context: #### Basic Search Operations ```bash # Enter search mode s # Press 's' to start search # Type search term error # Highlights all instances of "error" # Navigate matches n # Jump to next match N # Jump to previous match # Clear search ESC # Remove highlighting and exit search ``` #### Advanced Search Patterns ```bash # Search supports regex too timeout.*second # Highlight timeout patterns user.*[0-9]+ # Highlight user IDs \b[A-Z]{3,}\b # Highlight all-caps words (like HTTP, API, SQL) ``` #### Search vs Filter Decision Guide | Use Search When | Use Filter When | | ---------------------------------- | ----------------------------------- | | Want to see context around matches | Want to focus only on relevant logs | | Investigating pattern frequency | Reducing data volume | | Understanding log flow | Debugging specific issues | | Learning about log structure | Performance analysis | ### Interactive Filtering Workflows #### Word Frequency Panel Filtering Leverage the Word Frequency panel for quick filtering: 1. **Focus Word Frequency panel** (Tab to top-right) 2. **Navigate to interesting word** (↑/↓ arrows) 3. **Press Enter** to filter by that word 4. **Examine filtered results** in Log Viewer 5. **Clear filter** (press `/` and clear) to see all logs again **Power User Tip:** ```bash # Quick workflow for problem investigation: 1. Tab to Word Frequency 2. Look for "error", "fail", "timeout" near top 3. Enter on highest frequency problem word 4. Tab to Log Viewer to see filtered results 5. Enter on specific log for details ``` #### Multi-Step Filtering Build complex filters progressively: ```bash # Step 1: Start broad level.*error # Step 2: Add service context level.*error.*service.*api # Step 3: Add time context level.*error.*service.*api.*2024-01-15 # Step 4: Add specific component level.*error.*service.*api.*2024-01-15.*auth ``` #### Filter Refinement Techniques ```bash # Start with common terms, then refine database # Too broad - 1000 results database.*error # Better - 50 results database.*connection.*error # Specific - 5 results # Use exclusion to remove noise error.*(?!debug) # Errors but not debug errors timeout.*(?!test) # Timeouts but not test timeouts ``` ### Severity-Based Filtering Quickly focus on specific log levels using both regex patterns and the interactive severity filter modal: #### Severity Filter Modal (Ctrl+F) The fastest way to filter by log level is using the severity filter modal: **Quick Access:** Press `Ctrl+F` from anywhere in Gonzo **Features:** * Visual selection of severity levels * Real-time count of logs at each level * Color-coded severity display * Quick select all/none options * Combines with other active filters **Workflow:** ```bash # 1. Open severity filter Press Ctrl+F # 2. Quick clear all (optional) Navigate to "Select None" → Press Enter (applies and closes instantly) # 3. Select desired levels Press Ctrl+F to reopen Navigate to "ERROR" → Press Space (toggles checkbox) Navigate to "FATAL" → Press Space (toggles checkbox) # 4. Apply filter Press Enter # Result: Only ERROR and FATAL logs are displayed ``` **Interactive Features:** | Action | Result | | --------------------------------- | ---------------------------------------------- | | Navigate to "Select All" + Enter | Show all severity levels (applies immediately) | | Navigate to "Select None" + Enter | Hide all levels (applies immediately) | | Space on individual level | Toggle that specific level | | Enter (after changes) | Apply selections and close | | ESC | Cancel changes and close | **Combining with Other Filters:** ```bash # Show only database-related errors: 1. Press / → Type "database" → Enter (regex filter) 2. Press Ctrl+F → Select only "ERROR" → Enter (severity filter) # Both filters are active simultaneously ``` #### Standard Severity Patterns ```bash # JSON log levels level.*error # Error level only level.*(error|warn) # Error and warning levels level.*info # Info level only # Plain text log levels \[ERROR\] # [ERROR] markers \[(ERROR|WARN)\] # Multiple levels with brackets (ERROR|WARN|FATAL): # Colon-separated levels ``` #### Custom Severity Systems ```bash # Priority-based systems priority.*high # High priority logs priority.*(high|critical) # High and critical # Numeric severity severity.*[89] # Severity 8 or 9 severity.*1[0-9] # Severity 10+ # Custom level names (CRITICAL|ALERT|EMERGENCY) # Custom high-severity terms ``` ### Performance Filtering Find performance-related issues efficiently: #### Response Time Analysis ```bash # Slow requests duration.*[5-9][0-9]{3} # Duration > 5000ms response_time.*[1-9][0-9]+s # Response time > 10s elapsed.*[1-9][0-9]+ # Elapsed time patterns # Timeout patterns timeout # Any timeout mentions .*timeout.*[0-9]+ # Timeouts with specific durations connection.*timeout # Connection timeouts specifically ``` #### Resource Usage Filtering ```bash # Memory issues (memory|mem).*high # Memory alerts out.*of.*memory # OOM errors heap.*size.*exceeded # Heap exhaustion # CPU issues cpu.*high # CPU alerts load.*average.*[5-9] # High load averages processing.*slow # Slow processing alerts ``` ### Security-Focused Filtering Identify security-related events: #### Authentication Filtering ```bash # Login failures (login|auth).*fail # Authentication failures invalid.*credential # Credential issues access.*denied # Access control issues # Suspicious activity brute.*force # Brute force attempts rate.*limit.*exceeded # Rate limiting triggers suspicious.*activity # General suspicious events ``` #### Network Security Filtering ```bash # Network issues (firewall|iptables).*block # Firewall blocks connection.*refused # Connection rejections unauthorized.*access # Unauthorized access attempts # Attack patterns (sql.*injection|xss|csrf) # Common attack types malicious.*request # Malicious activity ``` ### Filter Management #### Clearing and Modifying Filters ```bash # Clear current filter / # Enter filter mode # Delete all text Enter # Apply empty filter (shows all logs) # Modify existing filter / # Enter filter mode # Edit existing pattern Enter # Apply modified filter # Quick clear Ctrl+C # Clear filter immediately (context-dependent) ``` #### Filter History and Patterns **Common Filter Patterns to Remember:** ```bash # Error investigation starter pack level.*error (error|fail|exception) timeout.*[0-9]+ status.*[45][0-9]{2} # Performance investigation (slow|timeout|high|exceed) duration.*[5-9][0-9]{3} response.*time.*[1-9][0-9]+ # Security investigation (auth|login|access).*fail (block|deny|refuse) rate.*limit ``` ### Advanced Filter Techniques #### Lookahead and Lookbehind ```bash # Positive lookahead error(?=.*database) # "error" followed by "database" user(?=.*login) # "user" in login context # Negative lookahead error(?!.*test) # "error" not followed by "test" timeout(?!.*retry) # "timeout" without retry # Positive lookbehind (?<=POST\s).*error # "error" preceded by "POST" (?<=user\s).*fail # "fail" after "user" ``` #### Context-Aware Filtering ```bash # Same-line requirements user.*login.*success # All three on same line error.*code.*[0-9]+ # Error with numeric code # Multi-line context (limited support) error.*\n.*stacktrace # Error followed by stacktrace exception.*\n.*at.*line # Exception with line reference ``` ### Filter Performance Optimization #### Efficient Filter Design ```bash # Fast filters (use these patterns) ^ERROR # Anchored to start \berror\b # Word boundaries level.*error # Structured field matching # Slow filters (avoid when possible) .*error.* # Unanchored wildcards (.*user.*|.*admin.*) # Complex alternation error.*user.*login.*fail # Too many wildcards ``` #### Large Dataset Strategies ```bash # Pre-filter at source when possible tail -f /var/log/app.log | grep ERROR | gonzo # Use progressive filtering # Start: error # Refine: error.*database # Specific: error.*database.*connection # Combine with buffer adjustments gonzo -f large.log --log-buffer=10000 # More history gonzo -f busy.log --log-buffer=1000 # Less memory ``` ### Troubleshooting Filters #### Common Filter Problems **Filter not working:** ```bash # Check regex syntax # Test patterns: https://regex101.com # Use simple text first: error # Then add complexity: level.*error ``` **Too many/few results:** ```bash # Too many results - add specificity error # Too broad error.*database # Better error.*database.*timeout # Specific # Too few results - remove constraints level.*error.*database.*connection.*timeout # Too specific level.*error.*database # Broader level.*error # Even broader ``` **Performance issues:** ```bash # Simplify complex patterns (.*error.*|.*warn.*|.*fail.*) # Complex (error|warn|fail) # Simple # Avoid nested wildcards error.*database.*timeout.*connection # Slow error.*connection # Faster ``` #### Filter Testing Strategies ```bash # Test filters progressively 1. Start with simple text: "error" 2. Add structure: "level.*error" 3. Add context: "level.*error.*database" 4. Refine specificity: "level.*error.*database.*timeout" # Validate with known logs # Find a specific log entry you can see # Create filter that should match it # Verify it appears in results ``` ### Filter Best Practices #### 🎯 **Effective Filter Strategies** 1. **Start broad, narrow down** - Begin with simple terms, add specificity 2. **Use structured fields** - Leverage JSON/logfmt field names 3. **Combine with search** - Filter to reduce, search to highlight 4. **Save common patterns** - Remember frequently used regex patterns #### 🔍 **Investigation Workflows** 1. **Problem identification** - Use Word Frequency panel to spot issues 2. **Initial filtering** - Filter by problem type (error, timeout, etc.) 3. **Context gathering** - Search for related terms while preserving context 4. **Deep dive** - Progressive filtering to isolate specific issues #### ⚡ **Performance Tips** 1. **Anchor patterns** - Use ^ and $ when possible 2. **Word boundaries** - Use \b for exact word matches 3. **Avoid excessive wildcards** - Minimize .\* usage 4. **Pre-filter when possible** - Filter at source before Gonzo ### What's Next? Now that you've mastered filtering and search, explore these advanced topics: * **Detailed Usage Guide** - Real-world patterns and power-user workflows * **AI Integration** - Let AI help identify patterns * **Log Analysis** - Advanced pattern detection techniques * **Configuration** - Save filter patterns and preferences *** **You now have complete mastery over finding relevant information in your logs!** 🚀 From simple text filtering to complex regex patterns, you can quickly isolate the logs that matter for any investigation. --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.controltheory.com/backup/user-guide/filtering-and-search.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.