Structured Logging & Performance

🚀 Major Features & Enhancements
Complete Structured Logging System 📊

Structured Logging architecture with component-based organization
JSON & Pretty-print formats - Choose structured JSON logs or human-readable pretty logs
Configurable log management with automatic rotation, trimming, and size limits
Component-specific logging with fine-grained control over log levels
Suppression controls - Silence INFO/DEBUG logs from specific components
Centralized logger initialization for consistent behavior across the entire codebase

Advanced Pubkey Caching System 💾

Intelligent caching of whitelist and blacklist pubkeys for dramatic performance improvements
Background refresh with configurable intervals for whitelist and blacklist data
Domain pubkey caching - Maintains cached domain pubkeys even when domains are temporarily offline
Separate validation logic - Cache always available for sync/purge operations, respects enabled state for validation
Memory-efficient bulk operations using cached data

Enhanced Connection Management 🔗

Smart memory monitoring with automatic connection dropping when memory thresholds exceeded
Improved WebSocket handling with proper timeout configuration and graceful closure
Connection statistics and monitoring for better observability
Per-connection timeouts for read, write, and idle operations
Enhanced error handling with connection state tracking

Query Performance Optimization ⚡

Unified cross-collection queries using MongoDB $unionWith for improved performance
Intelligent collection selection based on event kinds
Configurable implicit REQ limits to prevent resource exhaustion
Optimized sort ordering with chronological guarantees
Reduced query complexity for no-kind filters


🛠️ Technical Improvements
Event Processing Enhancements

Improved signature verification with NIP-01 compliant JSON normalization
Enhanced event validation with consolidated validation package
Better timestamp validation with dynamic time constraint calculation
Optimized event storage with improved MongoDB operations

Configuration System

Centralized configuration management with better type organization
Enhanced config watching with debounced auto-restart functionality
Improved example configurations with better documentation

User Sync Improvements

Cached whitelist filtering for sync operations
Better relay discovery and outbox event processing
Improved error handling and logging throughout sync process


🐛 Bug Fixes
Critical Fixes

Fixed signature validation that was incorrectly invalidating certain events
Resolved query issues in REQ handling that caused dropped messages
Fixed client connection dropping with proper message buffer allocation
Corrected event purging to properly consider domain-whitelisted pubkeys

Stability Improvements

Enhanced WebSocket timeout handling preventing connection hangs
Better error recovery in database operations
Improved memory management preventing resource leaks
Fixed race conditions in concurrent operations


🔧 Configuration Changes
New Configuration Options
```yaml
config.yml
logging:
  level: "info" # Log levels: "debug", "info", "warn", "error"
  file: "debug" # Log file name
  max_log_size_mb: 10 # Maximum log file size in MB before trimming
  structure: false # true = structured JSON logs, false = pretty logs
  check_interval_min: 10 # Check every 10 minutes
  backup_count: 2 # Keep 2 backup files (.bak1, .bak2)
  suppress_components: # Components to suppress INFO/DEBUG logs from (WARN/ERROR still shown)
    - "util" # Utility functions (file ops, IP detection, metadata loading)
    - "conn-manager" # Connection management (memory stats, connection counts)
    - "client" # Client connection details (connects/disconnects, timeouts)
    - "mongo-query" # Database query operations (can be very verbose)
    - "event-store" # Event storage operations (insert/update confirmations)
    - "close-handler" # Subscription close operations (routine cleanup)
Available components for suppression:
- "main"             # Main application lifecycle (startup, shutdown, restarts)
- "mongo"            # MongoDB connection and database operations
- "mongo-store"      # High-level event storage coordination
- "mongo-purge"      # Event purging and cleanup operations
- "event-handler"    # Event processing and validation coordination
- "req-handler"      # Subscription request handling
- "auth-handler"     # Authentication processing (NIP-42)
- "config"           # Configuration loading and caching
- "event-validation" # Event signature and content validation
- "user-sync"        # User synchronization operations
- "log"              # Logging system internal operations
Note: Suppression only affects INFO and DEBUG levels.
WARN and ERROR messages are always shown regardless of suppression.
server:
  implicit_req_limit: 1000     # Default limit for REQ without explicit limit
event_purge:
  disable_at_startup: false    # Disable purge at server startup
user_sync:
  disable_at_startup: false    # Disable sync at server startup
blacklist.yml
mutelist_cache_refresh_minutes: 30 # Refresh mutelist cache every 30 minutes
whitelist.yml
pubkey_whitelist:
  cache_refresh_minutes: 60 # Refresh cache every hour
domain_whitelist:
  cache_refresh_minutes: 120 # Refresh domain cache every 2 hours
```

📦 Installation & Upgrade Instructions
Fresh Installation

Download the appropriate binary for your platform
Extract the www folder to the same directory as the executable
Run the program - it will create default configuration files

Upgrading from v0.3.1

Backup your current configuration files
Replace the old executable with the new one
Replace the app folder with the new www folder
Review the new example configurations - several new options have been added
Update your configs with new options as needed
Restart the relay

Important Upgrade Notes

The app directory has been renamed to www
New logging configuration options available
Enhanced caching system improves performance significantly
Review new configuration options for optimal performance


📈 Performance Improvements

Dramatically faster whitelist/blacklist operations through intelligent caching
Reduced database queries with optimized MongoDB operations
Improved memory usage with better connection management
Faster event validation with consolidated validation logic
Enhanced query performance with unified collection operations


💡 What's Next for v0.5.0?

Enhanced Client Package offering a more functional front end
Advanced relay statistics and monitoring dashboard

Enhanced NIP support with additional protocol implementations


Extended API endpoints for relay management



🎯 Highlights of This Release
This is the largest release since the project began, featuring a complete overhaul of the logging system, major performance improvements through intelligent caching, and numerous stability enhancements. The new structured logging system provides unprecedented visibility into relay operations, while the pubkey caching system dramatically improves performance for large-scale deployments.

Compare with v0.3.1

Released: June 1, 2025

Commit: f9b98c154966b017296cdce7fd750808154ffeee
Full Changelog: https://github.com/0ceanSlim/grain/compare/v0.3.1...v0.4.0

Thanks for your patience during the extensive development cycle! This release represents months of refinement and optimization. As always, feedback and testing from the community are invaluable. 🚀