Skip to main content

Logging Configuration

Table of Contents​

  1. Introduction
  2. Project Structure
  3. Core Components
  4. Architecture Overview
  5. Detailed Component Analysis
  6. Dependency Analysis
  7. Performance Considerations
  8. Troubleshooting Guide
  9. Conclusion
  10. Appendix

Introduction​

This document systematically explains the logging configuration system of this project, covering log level settings, output format selection, file rotation configuration and other core capabilities; explains the differences between development mode and production mode (including log format, output target, performance considerations); provides logging configuration best practices (log aggregation, monitoring integration, performance optimization) and methods for extending custom log handlers and formatters. Readers can understand and correctly use the logging system without deep diving into source code.

Project Structure​

Logging related code is mainly distributed in the following modules:

  • Configuration Layer: Defines logging configuration structure and default values
  • Logging Implementation Layer: Encapsulates zap, provides development/production two logger factories and unified Logger wrapper
  • Application Startup Layer: Loads configuration and creates global logger instance during application initialization phase
  • Usage Example Layer: Various business modules output logs through global Logger

Core Components​

  • Logging Configuration Structure: Contains key fields such as mode, level, format, output, filename, etc.
  • Logger Wrapper: Unified external API, provides Info/Error/Fatal/Panic/Debug/Warn and their formatted versions
  • Development Logger: Console output, colored levels, short caller path
  • Production Logger: File rotation (lumberjack), JSON format, multi-target output (file + console)

Architecture Overview​

The logging system adopts "configuration driven + factory pattern" design:

  • Configuration layer responsible for reading and validating logging configuration
  • Factory chooses different logger implementations according to mode
  • Global Logger injected during application startup
  • Business modules output logs through global Logger

Detailed Component Analysis​

Logging Configuration Structure and Default Values​

Structure Fields​

  • Mode: Running mode, determines logger type (dev or prod/production)
  • Level: Log level, supports debug/info/warn/error/fatal
  • Format: Output format (current implementation uses console for development mode, JSON for production mode)
  • Output: Output target (current implementation outputs to console for development mode, outputs to file + console for production mode)
  • Filename: Log file path (file rotation target in production mode)

Default Values​

  • Default level: info
  • Default format: json
  • Default output: stdout
  • Default filename: ./logs/info.log
  • Default mode: dev

Logger Factory​

NewLogger decides which logger to create based on Mode:

  • dev: Creates development logger (console output, colored levels)
  • prod/production: Creates production logger (file rotation, JSON format)

In production mode, log level is mapped from Level field to zap level

Development Logger Implementation​

Encoder Configuration​

  • Time field name, level field name, caller field name, message field name, stacktrace field name
  • Level encoding: With color (convenient for terminal reading)
  • Time encoding: ISO8601
  • Caller encoding: Short path (filename:line number)

Output Target​

  • Console (standard output)

Log Level​

  • Mapped according to Level (debug/info/warn/error/fatal)

Features​

  • Enables caller information and error stacktrace (error level and above)

Production Logger Implementation​

File Rotation (lumberjack)​

  • File size threshold: 100MB
  • Backup count: Maximum 10
  • Maximum retention days: 7 days
  • Compression: Enabled
  • Local time: Enabled

Encoder Configuration (JSON)​

  • Time field name, level field name, caller field name, message field name, stacktrace field name
  • Level encoding: Uppercase
  • Time encoding: ISO8601
  • Caller encoding: Full path (convenient for locating)
  • Duration encoding: Milliseconds

Output Target​

  • Multi-target (file + console)

Log Level​

  • Mapped from Level to zap level

Features​

  • Enables caller information and error stacktrace (error level and above)

Logger Wrapper​

  • Provides unified method family externally: Info/Error/Fatal/Panic/Debug/Warn and their formatted versions
  • Internally uses zap.SugaredLogger's Infow/Errorw/Fatalw/Panicw/Debugw/Warnw
  • Supports field key-value pairs (fields) and formatted strings

Application Startup Integration​

  • Loads configuration (including logging configuration) during application startup
  • Creates global Logger and injects into App instance
  • Business modules use logging through App.Logger

Usage Examples​

NATS Event Bus​

  • Publish, subscribe, error, close and other scenarios all use Logger output

RabbitMQ Event Bus​

  • Publish, subscribe, error, message processing and other scenarios all use Logger output

Dependency Analysis​

Configuration Layer​

  • LogConfig defines logging configuration structure
  • SetDefaults sets default values for logging configuration
  • Load responsible for reading configuration file, environment variables and validation

Logging Implementation Layer​

  • NewLogger chooses logger according to Mode
  • NewDebugLogger builds development logger
  • NewRotatingLogger builds production logger
  • Logger wrapper provides unified API

Application Startup Layer​

  • App creates and injects global Logger during startup

Usage Example Layer​

  • Various business modules output logs through global Logger

Performance Considerations​

  • Production mode enables file rotation and compression by default, helps control disk occupation and log lifecycle management
  • Production mode uses JSON encoding, convenient for log collection and structured analysis
  • Development mode uses console output, convenient for quick debugging, but not suitable for high-throughput production environment
  • Recommendations in production environment:
    • Fix log level (such as info or higher), avoid debug excessive output
    • Reasonably set rotation parameters (size, backup count, retention days), balance disk and retrieval requirements
    • When using multi-target output, pay attention to additional overhead brought by console output
    • In high concurrency scenarios, avoid expensive field computation in hot paths (lazy construct fields)

Troubleshooting Guide​

Configuration Not Taking Effect​

  • Check configuration file path and format, ensure configuration loads successfully
  • Confirm environment variable override rules and key name replacement

Log Level Incorrect​

  • Confirm Level field value and mapping relationship (debug/info/warn/error/fatal)
  • Confirm Mode field value (dev/prod/production)

File Rotation Abnormal​

  • Check file path permissions and if directory exists
  • Check disk space and file handle limits

Output Target Abnormal​

  • Development mode should output to console
  • Production mode should output to file and console (multi-target)

Log Format Does Not Meet Expectations​

  • Development mode uses console encoder
  • Production mode uses JSON encoder

Conclusion​

The logging configuration system of this project is centered on configuration driven, seamlessly switches between development and production modes through factory pattern. Development mode emphasizes readability and debugging efficiency, production mode emphasizes stability and operability. Combined with reasonable rotation strategy and structured output, can meet logging requirements of most application scenarios. Recommend further refining rotation parameters and output targets according to business scale and compliance requirements in actual deployment, and pay attention to logging performance impact in high concurrency scenarios.

Appendix​

Development Mode vs Production Mode Comparison​

Log Level​

  • Development Mode: Mapped according to Level (debug/info/warn/error/fatal)
  • Production Mode: Mapped according to Level (debug/info/warn/error/fatal)

Output Format​

  • Development Mode: Console encoder (colored levels, short caller path)
  • Production Mode: JSON encoder (uppercase levels, full path caller)

Output Target​

  • Development Mode: Console (standard output)
  • Production Mode: File + Console (multi-target)

Performance and Features​

  • Development Mode: Convenient for debugging, suitable for low-throughput scenarios
  • Production Mode: File rotation and compression, suitable for high-throughput and long-term retention

Extension Guide​

Custom Encoder​

  • Can refer to existing JSON encoder configuration, adjust field names, time encoding, caller encoding, etc.
  • Suitable for specific log platforms or audit requirements

Custom Output Target​

  • Can add new WriteSyncer in multi-target scenarios (such as remote log service)
  • Pay attention to synchronization and error handling

Custom Factory​

  • Can extend more mode branches in NewLogger
  • Can introduce external log middleware (such as OpenTelemetry exporter)

Best Practices​

Log Aggregation and Monitoring Integration​

  • Production mode uses JSON output, convenient for structured collection and analysis
  • Output logs to centralized log system (such as ELK, Loki, Cloud Logging)
  • Combine with alerting rules for real-time alerting on error/fatal levels

Performance Optimization​

  • Fix log level, avoid debug excessive output
  • Reasonably set rotation parameters, balance disk and retrieval requirements
  • In high concurrency scenarios, avoid expensive field computation in hot paths

Observability​

  • Add structured fields for key paths (such as request ID, user ID, event type)
  • Retain sufficient context information (caller, stacktrace), but avoid redundancy