Skip to main content

Tech Stack

Table of Contents​

  1. Introduction
  2. Project Structure
  3. Core Components
  4. Architecture Overview
  5. Detailed Component Analysis
  6. Dependency Analysis
  7. Performance Considerations
  8. Troubleshooting Guide
  9. Conclusion
  10. Appendix

Introduction​

This document systematically outlines the Go ecosystem technology stack adopted by the Sparrow microservice framework, covering key modules including web framework, messaging middleware, permission control, configuration management, logging, caching, databases and document storage, key-value storage, etc. It also explains the reasons for technology selection, responsibility positioning in the framework, dependency relationships, and integration methods based on code implementation. Additionally, it provides practical suggestions for Go 1.25 compatibility, dependency management strategies, version control mechanisms, upgrade paths, and alternative solutions, along with learning resources and best practice guidelines.

Project Structure​

Sparrow adopts an organization divided by functional domains and layers:

  • Configuration Layer: Concentrated in pkg/config, unified loading and default value setting
  • Bootstrap Layer: pkg/bootstrap responsible for application initialization, container assembly, service startup, and graceful shutdown
  • Adapter Layer: pkg/adapter provides HTTP routing, middleware, and handlers
  • Use Case and Domain: pkg/usecase, pkg/entity, pkg/errs
  • Infrastructure: pkg/eventbus, pkg/messaging, pkg/persistence, pkg/logger, pkg/auth
  • Task Scheduling: pkg/tasks
  • WebSocket: pkg/ws
  • Web Framework: Gin, responsible for HTTP routing and middleware, initialized during application startup and carries business handlers
  • Message Bus: NATS, provides event publish/subscribe capabilities, supports JetStream (code also provides non-JetStream implementation)
  • Permission Control: Casbin, implements RBAC/ABAC and other access controls through model and policy files
  • Configuration Management: Viper, centrally reads environment variables, files, and default values, maps to structured configuration
  • Logging: Zap, provides high-performance structured logging, supports production mode rotation and development mode console output
  • Caching: Redis, provides key-value caching and TTL management, supports batch operations and pipelining
  • Database: PostgreSQL, ORM uses gorm/sqlx, provides strong consistency transactions and complex queries
  • Document Storage: MongoDB (driver imported), used for unstructured data scenarios
  • Key-Value Storage: BadgerDB, embedded KV, suitable for event views or lightweight persistence
  • Task Scheduling: Framework built-in task scheduler, supports concurrent/hybrid/sequential scheduling strategies
  • Middleware: JWT, RSA permission verification, CORS, etc.

Sparrow's runtime architecture centers around the "bootstrap layer": during application startup, it loads configuration, initializes logging, establishes database/cache connections, assembles event bus and message Hub, then starts the HTTP server and enters graceful shutdown process. Event bus and repository implementations together form the data and event infrastructure layer.

Web Layer (Gin)​

  • Responsibilities: Routing, middleware (JWT/CORS/RSA permissions), controllers
  • Integration Points: Create Engine during application startup and mount middleware; HTTP server carries routing
  • Key Behaviors: Set timeout parameters during startup; wait for request completion during graceful shutdown
  • Responsibilities: Publish/subscribe domain events; supports both JetStream and standard NATS implementations
  • Integration Points: Application resolves NATS connection through container; event publisher/subscriber assembled in Hub
  • Key Behaviors: Exponential backoff retry for subprocesses; event serialization/deserialization; subscribe/unsubscribe

  • Responsibilities: Access control based on model and policy files

  • Integration Points: Configuration provides model and policy paths; middleware calls Casbin for authorization decisions

  • Key Behaviors: Model and policy file paths configurable; policy changes require reloading

  • Responsibilities: Unified configuration reading, default value setting, mapping to structured objects

  • Integration Points: Load configuration during application startup; logging, database, cache, NATS, Casbin, etc. all depend on configuration

  • Key Behaviors: Default value override, environment variable priority, configuration hot update strategy

  • Responsibilities: High-performance structured logging; production mode log rotation; development mode console output

  • Integration Points: Create Logger during application initialization; all modules use the wrapped Sugar interface uniformly

  • Key Behaviors: Level switching, file rotation, field attachment

  • Responsibilities: Key-value caching, TTL management, batch operations, pipeline optimization

  • Integration Points: Repository implementation based on Redis client; supports prefix isolation and entity type distinction

  • Key Behaviors: Pipeline batch processing; full table scan queries (no index); TTL setting and querying

  • Responsibilities: Relational data persistence; transactions, batch operations, complex queries

  • Integration Points: Repository implementation based on sqlx; provides generic entity support; soft delete and pagination

  • Key Behaviors: Batch save transactions; dynamic column building; string array scanning; conditional queries

  • Responsibilities: Embedded KV storage; event views or lightweight persistence

  • Integration Points: Repository implementation based on Badger; supports prefix iteration and random reads

  • Key Behaviors: Transactional read/write; full table scan queries; key prefix management

  • Responsibilities: Unstructured data storage; decoupled from other framework components

  • Integration Points: Driver imported, no specific repository implementation seen; can be extended as needed

  • Responsibilities: Concurrent/hybrid/sequential scheduling strategies; integrated with application lifecycle

  • Integration Points: Register subprocesses during application startup; exponential backoff retry on failure

  • Responsibilities: Token structure and authentication flow; JWT middleware and RSA permission verification

  • Integration Points: Middleware chain; token struct definition

  • Version and Compatibility: go.mod specifies go 1.25; most dependencies are stable versions, recommend following semantic versioning

  • Indirect Dependencies: Large number of indirect dependencies declared through go.mod, note version conflicts and upgrade impacts

  • Component Coupling: Event bus and repository implementations are relatively independent; Web layer coupled with permission modules through middleware; configuration and logging run throughout globally

  • Logging: Zap structured logging, production mode rotation; avoid constructing large objects in high-frequency paths
  • Caching: Redis Pipeline batch processing; reasonable TTL settings; avoid full table scans
  • Database: Postgres batch transactions; parameterized queries; index and pagination strategies
  • Event Bus: NATS JetStream (provided in code) has persistence and reliable delivery; standard NATS is more lightweight
  • Memory and GC: Avoid frequent allocation in hot paths; reuse buffers; monitor Badger/Redis memory usage

Troubleshooting Guide​

  • Configuration Loading Failure: Check viper default values and environment variable overrides; confirm configuration file path and permissions
  • NATS Connection Anomalies: Verify URL, reconnection strategy; check event bus closed state
  • Redis/Postgres/Badger Connection Failure: Verify host, port, password, and database name; check network and firewall
  • Logging Output Anomalies: Confirm log level and output destination; production mode rotation strategy
  • Task Retry: Observe exponential backoff and maximum retry count; adjust retry strategy if necessary

Sparrow's technology stack centers on Gin, NATS, Casbin, Viper, Zap, Redis, PostgreSQL, and BadgerDB, forming a high-cohesion, low-coupling microservice infrastructure. Through configuration center, event bus, and repository abstraction, the framework balances ease of use with performance and scalability. It is recommended to enable JetStream in production environments, improve monitoring and alerting, and continuously monitor dependency version evolution and security patches.

Appendix​

Go 1.25 Compatibility and Dependency Management​

  • Compatibility: go.mod explicitly specifies go 1.25; recommend fixing version in CI and enabling go mod tidy

  • Dependency Management: Use go.mod/go.sum; recommend regularly executing go upgrade and go mod tidy

  • Version Control: Follow semantic versioning; set upper version limits for critical dependencies to avoid breaking updates

  • Gin: Keep major version stable; if upgrading to new major version, focus on middleware and routing compatibility

  • NATS: Migrate from standard NATS to JetStream (if persistence and reliable delivery are needed)

  • Casbin: Have rollback plan when migrating model/policy files

  • Viper: Provide default values and migration scripts when adding new configuration items

  • Zap: Production mode rotation strategy and metrics collection

  • Redis: When upgrading from go-redis/v8 to v9, focus on API changes and connection pool configuration

  • PostgreSQL: gorm/sqlx dual-track parallel, gradually migrate complex queries

  • BadgerDB: Focus on compaction and memory threshold configuration

Learning Resources and Best Practices​

  • Gin official documentation and middleware ecosystem
  • NATS JetStream and event-driven architecture
  • Casbin model design and policy maintenance
  • Viper configuration best practices (environment variables, files, default values)
  • Zap performance optimization and production deployment
  • Redis high availability and persistence strategies
  • PostgreSQL transactions and index optimization
  • BadgerDB embedded storage and operations