Skip to main content

Service Lifecycle Management

Table of Contents

  1. Introduction
  2. Project Structure
  3. Core Components
  4. Architecture Overview
  5. Detailed Component Analysis
  6. Dependency Analysis
  7. Performance Considerations
  8. Troubleshooting Guide
  9. Conclusion
  10. Appendix

Introduction

This document systematically explains service lifecycle management, unfolding around the three phases of "startup—runtime—shutdown", focusing on the following:

  • Implementation and usage of GracefulClose interface
  • Startup process of Startable interface
  • Service registration and cleanup process
  • Subprocess management and retry strategy (exponential backoff, maximum retry count)
  • Service interdependencies, startup sequence control and fault isolation
  • Best practices: resource cleanup, error handling, monitoring and alerting

Project Structure

This project adopts layered and modular organization, with key modules as follows:

  • bootstrap: Application entry point, container, authentication, database connection and other infrastructure
  • usecase: Business use case interfaces (Startable, GracefulClose)
  • tasks: Task schedulers (concurrent/sequential/hybrid), supports retry and exponential backoff
  • messaging: Message bus and subscribers, supports graceful shutdown and startup
  • config: Configuration loading and retry configuration

Core Components

  • App: Application controller, responsible for starting HTTP server, starting subprocesses, handling signals, graceful shutdown and resource cleanup
  • Container: IoC container, supports anonymous/named registration and dependency injection
  • Startable/GracefulClose: Service lifecycle interfaces, unify startup and graceful shutdown behavior
  • TaskScheduler: Task scheduler interface, supports concurrent/sequential/hybrid modes
  • StreamHub/EventSubscriber: Message bus and subscribers, supports Start/Close and event distribution
  • Config Loader/RetryConfig: Configuration loading and retry strategy configuration

Architecture Overview

When application starts, App sequentially completes:

  • Load configuration and initialize logging
  • Build Gin engine and container
  • Register subprocesses (implementing Startable/GracefulClose)
  • Start HTTP server
  • Start subprocesses (with exponential backoff retry)
  • Listen for system signals, trigger graceful shutdown
  • Shutdown HTTP server and cleanup subprocesses

Detailed Component Analysis

App Startup and Shutdown Process

Startup Phase

  • Initialize configuration, logging, Gin engine and container
  • Start HTTP server (ListenAndServe)
  • Iterate SubProcesses, identify subprocesses implementing Startable and call Start
  • For subprocesses that fail to start, put into retry channel, background goroutine implements exponential backoff retry (maximum 10 times, maximum 1 minute)

Shutdown Phase

  • After receiving signal, first Shutdown HTTP server (with timeout)
  • Call CleanUp: cancel retry goroutine, call subprocess Close(ctx) one by one, timeout 5 seconds

Startable and GracefulClose Interfaces

  • GracefulClose: Defines Close(ctx context.Context) error, used for graceful shutdown and resource cleanup
  • Startable: Defines Start(ctx context.Context) error, used for starting services
  • App identifies subprocesses implementing Startable through assertion and starts them; performs cleanup through unified GracefulClose interface

Retry Strategy and Exponential Backoff

  • Exponential Backoff: backoff = 2^attempt seconds, upper limit 1 minute
  • Maximum Retry Count: 10 times
  • Retry Channel: After failure, send retryItem to channel, background goroutine consumes and retries
  • App.CleanUp cancels retry goroutine to avoid leak

Task Scheduler Lifecycle

  • Interface TaskScheduler: Provides Schedule/Cancel/Start/Stop/Close capabilities
  • Concurrent Scheduler concurrentScheduler
    • Concurrent execution, limited by maximum concurrency
    • On task failure, retries with linear increasing delay (upper limit 500ms), exceeds maximum retry enters dead letter
  • Sequential Scheduler sequentialScheduler
    • Serial execution, supports exponential backoff retry (upper limit 1 minute)
    • Supports periodic tasks and cancellation

Message Bus and Subscriber Lifecycle

  • StreamHub: Implements Startable/GracefulClose, encapsulates underlying Subscribers
  • EventSubscriber: Responsible for event subscription, topic encoding, handler registration and deserialization
  • Both support Start/Close, convenient for unified inclusion in App's subprocess management

Configuration and Retry Configuration

  • Configuration Loading: Supports multiple formats and environment variable override, validates required fields
  • Retry Configuration: Provides maximum retry count, initial/maximum backoff time, backoff multiplier and switch

Container and Dependency Injection

  • Container supports anonymous/named registration and dependency resolution, automatically caches instances
  • App obtains service instances through Container.ResolveInstance/ResolveByName
  • Test cases validate container behavior (singleton, named/anonymous, complex dependency chains)

Dependency Analysis

  • App depends on Container to provide service instance resolution
  • App depends on Startable/GracefulClose to unify subprocess lifecycle
  • Task schedulers and message bus both implement Startable/GracefulClose, can be uniformly registered to App
  • Configuration module provides configuration loading and validation for App

Performance Considerations

  • Exponential Backoff and Maximum Retry Count Limit: Prevents cascading failures, reduces system pressure
  • Task Scheduler Concurrency Control: Balances throughput and resource occupation through maximum concurrency and condition variable/polling mechanism
  • HTTP Server Timeout: Shutdown(ctx) 30-second timeout, avoids long-time blocking
  • Subscriber Shutdown Timeout: Close(ctx) 5-second timeout, ensures quick resource reclamation

Troubleshooting Guide

Startup Failure

  • Check configuration loading and validation (port, application name, etc.)
  • View App startup logs and retry channel output
  • Confirm subprocess implements Startable/GracefulClose and is correctly registered

Shutdown Exception

  • Confirm Close(ctx) timeout setting is reasonable (HTTP 30s, subprocess 5s)
  • Check if there are blocking subscribers or tasks not completed

Task Retry

  • Concurrent Scheduler: Linear increasing delay (upper limit 500ms), exceeds maximum retry enters dead letter
  • Sequential Scheduler: Exponential backoff (upper limit 1 minute), supports periodic tasks and cancellation

Conclusion

This project achieves through App unified management of service lifecycle, combined with Startable/GracefulClose interfaces and IoC container:

  • Clear startup—runtime—shutdown process
  • Robust retry strategy with exponential backoff and maximum retry count
  • Concurrent/sequential execution and retry control of task schedulers
  • Graceful startup and shutdown of message bus
  • Configuration loading and validation, ensures runtime stability

Appendix

Best Practices Checklist

  • Resource Cleanup: All subprocesses implement GracefulClose, uniformly closed in App.CleanUp
  • Error Handling: Start/Close should both return error, avoid silent failure
  • Monitoring and Alerting: Record key metrics and errors during App startup/retry/shutdown phases
  • Dependency Management: Register and resolve through Container, avoid hard-coded coupling
  • Configuration Security: Strictly validate configuration, prioritize environment variable override