Skip to main content

Configuration Management System

Table of Contents​

  1. Introduction
  2. Project Structure
  3. Core Components
  4. Architecture Overview
  5. Detailed Component Analysis
  6. Dependency Analysis
  7. Performance Considerations
  8. Troubleshooting Guide
  9. Conclusion
  10. Appendix

Introduction​

This document systematically explains the overall design and implementation of the configuration management system, covering configuration file loading mechanism, configuration item hierarchy structure, default value setting rules, various configuration types (application, HTTP server, logging, database, message queue, etc.), configuration file format and environment variable override mechanism, configuration validation rules, methods for extending new configuration items, and best practices and security recommendations. The goal is to help developers quickly understand and correctly use the configuration system, while providing clear reference for future extensions.

Project Structure​

The configuration system is located in the pkg/config directory, adopting a "layered aggregation" organization:

  • Root configuration structure centrally defines each sub-configuration domain
  • Each sub-domain has independent file defining its structure and related behaviors
  • Loader is responsible for reading configuration files, setting default values, environment variable override and validation
  • Bootstrap module loads configuration during application startup and injects into global application object

Core Components​

  • Root Configuration Structure: Aggregates all sub-configuration domains, exposes uniformly externally
  • Sub-configuration Structures: Divided by functional domains, such as application, HTTP server, logging, database, cache, event bus, message queue, permissions, etc.
  • Loader: Responsible for configuration file discovery and reading, default value setting, environment variable override, deserialization and validation
  • Bootstrap Module: Loads configuration during application startup phase and injects into global application object

Architecture Overview​

The configuration system follows the principle of "convention first, environment variable override, strict validation", implements multi-format configuration file reading and environment variable mapping through Viper, combined with mapstructure structure tags to complete type-safe deserialization.

Detailed Component Analysis​

Configuration File Loading Mechanism​

Configuration File Name and Path​

  • File name determined by application name passed by caller; if empty, falls back to default name, maintains backward compatibility
  • Search paths include current directory, configs directory, and when application name is non-empty, automatically appends system-level and user-level configuration paths

Multi-format Support​

  • Supports yaml/yml, json, toml, properties/props/prop formats, tries reading in order, returns on non-"file not found" error

Default Value Setting​

  • Sets reasonable default values for key fields through SetDefaults before loading, avoids runtime missing

Environment Variable Override​

  • Enables automatic environment variable injection through AutomaticEnv()
  • Replaces dots in configuration keys with underscores through SetEnvKeyReplacer, makes environment variable naming consistent with hierarchy (e.g., server.host corresponds to environment variable SERVER_HOST)
  • .env file can be explicitly loaded, convenient for development environment variable override

Deserialization and Validation​

  • Uses mapstructure to deserialize key-value pairs in Viper into strongly-typed structures
  • Executes validateConfig after deserialization, ensures key constraints are met (such as application name non-empty, port range legal)

Configuration Item Hierarchy and Default Values​

Hierarchy Structure​

  • Root structure Config aggregates each sub-configuration domain, each sub-domain corresponds to an independent structure
  • Sub-domain internal fields correspond one-to-one with configuration keys through mapstructure tags

Default Value Setting​

  • Sets default values for key fields in each sub-domain through SetDefaults, coverage includes application, HTTP server, CORS, logging, NATS, Redis, Badger, Casbin, SQL, RabbitMQ, etc.
  • Default value override strategy: first set default values, then read configuration file and environment variables, finally override with environment variables

Configuration Type Details​

Application Configuration (AppConfig)​

  • Fields: Name, version, secret key, permission file path
  • Purpose: Identifies application identity, version information and permission file location
  • Default Values: Name, version, secret key, permission directory all have default values

HTTP Server and CORS Configuration​

HTTP Server

  • Fields: Host address, port
  • Default Values: Host binds to all interfaces, port default value reasonable

CORS

  • Fields: Allowed origins, allowed methods, allowed headers, credentials, maximum cache duration (hours)
  • Special Tags: []string type parses comma-separated string into slice through "csv" tag
  • Default Values: Wildcard origins, common methods and headers, allow credentials, default cache duration

Logging Configuration​

  • Fields: Mode, level, format, output, filename
  • Default Values: Development mode, info level, JSON format, standard output, log file path

Database Configuration​

SQL Configuration

  • Fields: Driver, host, port, username, password, database name, event store database name
  • Behavior: Provides convenient methods for DSN and event store DSN, convenient for directly concatenating connection strings
  • Default Values: Postgres driver, local host, common default credentials and database name

Redis Configuration

  • Fields: Host, port, password, database number, event store database number
  • Default Values: Local host, common port, default database and event store database

Badger Configuration

  • Fields: View data directory, event store directory, threshold, compactor count
  • Default Values: Reasonable local directory and threshold settings

Event Bus and Message Queue Configuration​

NATS Configuration

  • Fields: Connection URL, event stream name, event stream maximum retention time (days)
  • Default Values: Local default URL, default stream name, permanent retention

RabbitMQ Configuration

  • Fields: Host, port, username, password, virtual host, exchange
  • Default Values: Local default credentials and exchange

Kafka Configuration

  • Fields: Broker list, consumer group ID, topics
  • Default Values: Empty list and empty strings, requires explicit configuration

Permission Configuration​

  • Fields: Model file path, policy file path
  • Default Values: Default model and policy file paths

Retry Configuration​

  • Fields: Maximum retry count, initial backoff time, maximum backoff time, backoff multiplier, enabled
  • Default Values: Reasonable default values, convenient for reuse in business

Configuration Validation​

  • Required Field Validation: Application name must not be empty
  • Numeric Range Validation: Server port must be between 1 and 65535
  • Extension Suggestions: Can add more constraints in validateConfig, such as database connection parameter validity, event bus URL format, etc.

Configuration File Format and Environment Variable Override​

  • Configuration File Format: Supports yaml/yml, json, toml, properties/props/prop, etc.
  • Environment Variable Override: Through AutomaticEnv() and SetEnvKeyReplacer(".") set to "_" mapping rule, makes configuration key and environment variable naming consistent
  • .env File: Can achieve development environment variable override through explicitly loading .env file

Extending New Configuration Items​

Add New Structure​

  • Add new file under pkg/config, define new configuration structure, and add mapstructure tags for fields
  • Add this field in root Config structure, makes it part of aggregation

Set Default Values​

  • Set default values for new fields in SetDefaults, ensures normal operation when not configured

Loading and Validation​

  • If involving key constraints, can add corresponding validation in validateConfig

Usage Example​

  • Access new configuration through app.Config in bootstrap module, or inject this configuration structure in specific module

Dependency Analysis​

Configuration System and Bootstrap Module Coupling​

Configuration system has low coupling with bootstrap module, decoupled through interfaces and structures, convenient for extension and testing.

Performance Considerations​

Configuration Loading Timing​

  • Complete configuration loading and validation early in application startup, avoids runtime repeated parsing

Environment Variable Override​

  • Uses AutomaticEnv and SetEnvKeyReplacer to improve environment variable mapping efficiency

Deserialization​

  • mapstructure tag and structure field type matching helps reduce conversion overhead

Validation Cost​

  • validateConfig should be as lightweight as possible, avoid complex IO operations

Troubleshooting Guide​

Configuration File Not Found​

  • Confirm configuration file name and path; check if exists in current directory, configs directory or system/user-level paths

Environment Variables Not Taking Effect​

  • Check if key name follows "dot replaced with underscore" rule; confirm .env file is loaded

Deserialization Failure​

  • Check if configuration key and structure field type match; confirm mapstructure tags are correct

Validation Failure​

  • Pay attention to validateConfig error messages, correct required fields and numeric ranges item by item

Conclusion​

The configuration management system is based on clear hierarchy structure, complete default values and strict validation mechanism, combined with Viper's multi-format support and environment variable override capability, provides flexible and reliable configuration management solution. Through the guidance of this document, developers can quickly get started, correctly extend and safely use this system in production environment.

Appendix​

Configuration Key and Structure Mapping Relationship​

Best Practices​

Naming Conventions​

  • Configuration keys use dot to separate hierarchy, environment variables use underscore, maintain consistency

Default Values and Sensitive Information​

  • Provide reasonable default values for non-sensitive fields; sensitive information (such as secret keys, passwords) override through environment variables

Validation Front-loading​

  • Perform configuration validation as early as possible in application startup phase, avoids runtime exceptions

Environment-specific Management​

  • Use different configuration files or environment variables to distinguish development, testing, production environments

Documentation and Comments​

  • Add brief description for each configuration item, convenient for maintenance and collaboration

Security Recommendations​

Secret Keys and Passwords​

  • Use environment variables to inject secret keys and database/middleware passwords, avoid hard-coding

Ports and Network​

  • Server ports should be in safe range; production environment recommends listening only on internal network or controlled network

Logging and Output​

  • Production environment recommends using JSON format logging and controlling output location, avoids sensitive information leakage

Event Bus and Message Queue​

  • Configure independent credentials and minimum privilege access policies for NATS/RabbitMQ/Kafka, etc.

Production Environment Configuration Example​

The following are recommended configuration points for production environment (please adjust according to actual environment):

Application Configuration​

  • Name and version clear; secret key properly kept and injected through environment variables

HTTP Server​

  • Bind to controlled IP; port in 8000-65535 range; CORS only allows necessary origins

Logging​

  • Mode: production; Level: info or higher; Output to file or standard output; Rotate as needed

Database​

  • Use dedicated credentials; Enable SSL/TLS; Set appropriate connection pool parameters

Cache and Local Storage​

  • Redis and Badger directories have sufficient disk space and permissions

Event Bus and Message Queue​

  • NATS/RabbitMQ/Kafka use independent credentials and minimum privileges; Monitor connection status and latency