Service Lifecycle Management
Table of Contents
- Introduction
- Project Structure
- Core Components
- Architecture Overview
- Detailed Component Analysis
- Dependency Analysis
- Performance Considerations
- Troubleshooting Guide
- Conclusion
- Appendix
Introduction
This document systematically explains service lifecycle management, unfolding around the three phases of "startup—runtime—shutdown", focusing on the following:
- Implementation and usage of GracefulClose interface
- Startup process of Startable interface
- Service registration and cleanup process
- Subprocess management and retry strategy (exponential backoff, maximum retry count)
- Service interdependencies, startup sequence control and fault isolation
- Best practices: resource cleanup, error handling, monitoring and alerting
Project Structure
This project adopts layered and modular organization, with key modules as follows:
- bootstrap: Application entry point, container, authentication, database connection and other infrastructure
- usecase: Business use case interfaces (Startable, GracefulClose)
- tasks: Task schedulers (concurrent/sequential/hybrid), supports retry and exponential backoff
- messaging: Message bus and subscribers, supports graceful shutdown and startup
- config: Configuration loading and retry configuration
Core Components
- App: Application controller, responsible for starting HTTP server, starting subprocesses, handling signals, graceful shutdown and resource cleanup
- Container: IoC container, supports anonymous/named registration and dependency injection
- Startable/GracefulClose: Service lifecycle interfaces, unify startup and graceful shutdown behavior
- TaskScheduler: Task scheduler interface, supports concurrent/sequential/hybrid modes
- StreamHub/EventSubscriber: Message bus and subscribers, supports Start/Close and event distribution
- Config Loader/RetryConfig: Configuration loading and retry strategy configuration
Architecture Overview
When application starts, App sequentially completes:
- Load configuration and initialize logging
- Build Gin engine and container
- Register subprocesses (implementing Startable/GracefulClose)
- Start HTTP server
- Start subprocesses (with exponential backoff retry)
- Listen for system signals, trigger graceful shutdown
- Shutdown HTTP server and cleanup subprocesses
Detailed Component Analysis
App Startup and Shutdown Process
Startup Phase
- Initialize configuration, logging, Gin engine and container
- Start HTTP server (ListenAndServe)
- Iterate SubProcesses, identify subprocesses implementing Startable and call Start
- For subprocesses that fail to start, put into retry channel, background goroutine implements exponential backoff retry (maximum 10 times, maximum 1 minute)
Shutdown Phase
- After receiving signal, first Shutdown HTTP server (with timeout)
- Call CleanUp: cancel retry goroutine, call subprocess Close(ctx) one by one, timeout 5 seconds
Startable and GracefulClose Interfaces
- GracefulClose: Defines Close(ctx context.Context) error, used for graceful shutdown and resource cleanup
- Startable: Defines Start(ctx context.Context) error, used for starting services
- App identifies subprocesses implementing Startable through assertion and starts them; performs cleanup through unified GracefulClose interface
Retry Strategy and Exponential Backoff
- Exponential Backoff: backoff = 2^attempt seconds, upper limit 1 minute
- Maximum Retry Count: 10 times
- Retry Channel: After failure, send retryItem to channel, background goroutine consumes and retries
- App.CleanUp cancels retry goroutine to avoid leak
Task Scheduler Lifecycle
- Interface TaskScheduler: Provides Schedule/Cancel/Start/Stop/Close capabilities
- Concurrent Scheduler concurrentScheduler
- Concurrent execution, limited by maximum concurrency
- On task failure, retries with linear increasing delay (upper limit 500ms), exceeds maximum retry enters dead letter
- Sequential Scheduler sequentialScheduler
- Serial execution, supports exponential backoff retry (upper limit 1 minute)
- Supports periodic tasks and cancellation
Message Bus and Subscriber Lifecycle
- StreamHub: Implements Startable/GracefulClose, encapsulates underlying Subscribers
- EventSubscriber: Responsible for event subscription, topic encoding, handler registration and deserialization
- Both support Start/Close, convenient for unified inclusion in App's subprocess management
Configuration and Retry Configuration
- Configuration Loading: Supports multiple formats and environment variable override, validates required fields
- Retry Configuration: Provides maximum retry count, initial/maximum backoff time, backoff multiplier and switch
Container and Dependency Injection
- Container supports anonymous/named registration and dependency resolution, automatically caches instances
- App obtains service instances through Container.ResolveInstance/ResolveByName
- Test cases validate container behavior (singleton, named/anonymous, complex dependency chains)
Dependency Analysis
- App depends on Container to provide service instance resolution
- App depends on Startable/GracefulClose to unify subprocess lifecycle
- Task schedulers and message bus both implement Startable/GracefulClose, can be uniformly registered to App
- Configuration module provides configuration loading and validation for App
Performance Considerations
- Exponential Backoff and Maximum Retry Count Limit: Prevents cascading failures, reduces system pressure
- Task Scheduler Concurrency Control: Balances throughput and resource occupation through maximum concurrency and condition variable/polling mechanism
- HTTP Server Timeout: Shutdown(ctx) 30-second timeout, avoids long-time blocking
- Subscriber Shutdown Timeout: Close(ctx) 5-second timeout, ensures quick resource reclamation
Troubleshooting Guide
Startup Failure
- Check configuration loading and validation (port, application name, etc.)
- View App startup logs and retry channel output
- Confirm subprocess implements Startable/GracefulClose and is correctly registered
Shutdown Exception
- Confirm Close(ctx) timeout setting is reasonable (HTTP 30s, subprocess 5s)
- Check if there are blocking subscribers or tasks not completed
Task Retry
- Concurrent Scheduler: Linear increasing delay (upper limit 500ms), exceeds maximum retry enters dead letter
- Sequential Scheduler: Exponential backoff (upper limit 1 minute), supports periodic tasks and cancellation
Conclusion
This project achieves through App unified management of service lifecycle, combined with Startable/GracefulClose interfaces and IoC container:
- Clear startup—runtime—shutdown process
- Robust retry strategy with exponential backoff and maximum retry count
- Concurrent/sequential execution and retry control of task schedulers
- Graceful startup and shutdown of message bus
- Configuration loading and validation, ensures runtime stability
Appendix
Best Practices Checklist
- Resource Cleanup: All subprocesses implement GracefulClose, uniformly closed in App.CleanUp
- Error Handling: Start/Close should both return error, avoid silent failure
- Monitoring and Alerting: Record key metrics and errors during App startup/retry/shutdown phases
- Dependency Management: Register and resolve through Container, avoid hard-coded coupling
- Configuration Security: Strictly validate configuration, prioritize environment variable override