Task Scheduling System

Table of Contents

• Task Scheduling System
  • Table of Contents
  • Introduction
  • Project Structure
    • Concurrent Scheduler (concurrentScheduler)
  • Appendix
    • Task Lifecycle and State Transitions

Introduction

This document provides detailed technical documentation for the Sparrow task scheduling system, covering the design principles and implementation details of the concurrent scheduler, sequential scheduler, and hybrid scheduler. It explains task lifecycle management, state tracking, priority handling, and retry mechanisms. It provides usage methods for the task builder and scheduler wrapper. It includes monitoring and statistics, best practices, performance optimization, and fault recovery strategies, and demonstrates how to extend the scheduler to support custom scheduling algorithms and task types. This document is intended for task scheduling developers, providing complete guidance from concept to implementation.

Project Structure

The task scheduling system is located in the pkg/tasks directory and adopts a layered design:

• Interface and Models: Defines task scheduler interfaces, task models, states, and scheduling types
• Scheduler Implementations: Concurrent, Sequential, and Hybrid schedulers
• Builder and Wrapper: Simplifies task building and provides a unified scheduling entry point
• Tests: Covers concurrent, sequential, hybrid, retry, timeout, and memory management scenarios

• Task Scheduler Interface: Unified scheduling, cancellation, status query, concurrency control, and lifecycle management capabilities
• Task Model: Task information, state, scheduling type, execution mode, and retry configuration
• Task Builder: Chain configuration of task ID, type, handler, callbacks, scheduling time, timeout, retry, and priority
• Scheduler Wrapper: Unified entry point, responsible for task submission, mode selection, concurrency control, and default retry
• Three Scheduler Implementations: Concurrent, Sequential, and Hybrid, meeting different execution needs and strategies

The scheduling system adopts an "interface abstraction + multiple implementations + wrapper entry" architecture:

• Interface layer defines unified capabilities
• Implementation layer provides three strategies: concurrent, sequential, and hybrid
• Wrapper layer provides easy-to-use API and default behaviors (such as default retry, concurrency setting, task type mode mapping)

Concurrent Scheduler (concurrentScheduler)

The concurrent scheduler allows multiple tasks to execute in parallel, supporting maximum concurrency limits, task TTL, memory cleanup, and retry mechanisms. Its key features include:

• Concurrency slot control: Achieves concurrency throttling through active task counting and condition variables/polling
• Task monitoring: Periodically scans waiting and retrying tasks to trigger execution
• Retry strategy: Linearly increasing delay (capped at 500ms), entering dead letter after reaching maximum retry count
• Memory management: Removes oldest completed/cancelled/dead letter tasks when maximum task count is reached; cleans up after TTL expiration
• Task cancellation: Calls cancel callback and marks status as cancelled

The sequential scheduler ensures only one task executes at a time, suitable for timed and periodic tasks. Its key features include:

• Ordered execution: Maintains pending execution queue, takes out and executes in order
• Task queue and notification: Uses buffered queue and completion notification channel to coordinate execution
• Monitor: Periodically checks if timed tasks have reached execution time and adds to queue
• Retry strategy: Exponential backoff (capped at 1 minute), marks as failed after reaching maximum retry count
• Memory cleanup: Removes oldest completed/cancelled/failed tasks by update time

The hybrid scheduler selects execution mode based on task type and scheduling time:

• Task type mapping: Can bind specific task types to sequential or concurrent mode
• Scheduling decision: If task type is sequential or is timed/periodic, uses sequential scheduler, otherwise uses concurrent scheduler
• Unified interface: Encapsulates concurrent and sequential schedulers, providing unified scheduling, cancellation, status query, and listing

The task builder provides chain configuration, supporting:

• ID, type, handler, completion/cancellation callbacks
• Scheduling time (immediate, timed, periodic), timeout, retry count, priority
• After building, generates built-in task object, implementing Task and TaskInfoProvider interfaces

The scheduler wrapper provides a unified entry point and default behaviors:

• Task submission: Supports immediate, timed, periodic, delayed execution, and typed tasks
• Mode mapping: Can configure mapping of task types to execution modes
• Concurrency control: Supports setting maximum concurrency (effective for concurrent/hybrid schedulers)
• Default retry: Injects default retry count through options
• Lifecycle: Unified Start/Stop/Close

• Interface coupling: All schedulers implement TaskScheduler interface, facilitating replacement and combination
• Composition relationship: Hybrid scheduler composes concurrent and sequential schedulers; wrapper holds scheduler instance
• Task cohesion: builtTask embeds TaskInfo, implements Task and TaskInfoProvider, reducing cross-layer coupling
• External dependencies: Uses testify assertions and atomic counters in tests, no external runtime dependencies

• Concurrency control

  • Concurrent scheduler uses active task counting and condition variables/polling to control concurrency slots, avoiding excessive contention
  • Hybrid scheduler can set concurrent mode for specific task types, balancing throughput and consistency

• Task TTL and memory management

  • Both concurrent and sequential schedulers provide TTL and maximum task count limits to prevent memory bloat
  • Periodically cleans up expired tasks, reducing invalid data occupation

• Retry strategy

  • Concurrent scheduler uses linearly increasing delay (capped at 500ms), sequential scheduler uses exponential backoff (capped at 1 minute)
  • Enters dead letter/failed state after reaching maximum retry count, avoiding infinite retries

• Timeout control

  • Task execution supports timeout context, marks as cancelled rather than failed when timeout or cancellation occurs

• Cost of sequential execution

  • Sequential scheduler serializes execution, suitable for timing-sensitive tasks, but has lower throughput

• Task state anomalies

  • Both concurrent and sequential schedulers provide GetTaskStatus and ListTasks for diagnosis
  • If tasks remain in retrying state for a long time, check retry delay and maximum retry count configuration

• Concurrency blocking

  • Concurrent scheduler controls concurrency through active task counting and waiting mechanism; if blocking occurs, check maximum concurrency setting and task execution time

• Task loss

  • Check task TTL and maximum task count limits, confirm if cleaned up

• Cancellation invalid

  • Confirm task ID is correct and task is still in waiting state; sequential scheduler will remove task from pending execution queue

Sparrow task scheduling system provides flexible concurrent, sequential, and hybrid execution modes through clear interface abstraction and multiple implementation strategies. With the task builder and scheduler wrapper, developers can quickly build, configure, and manage tasks. The system has comprehensive retry, timeout, memory management, and state tracking capabilities, suitable for high-concurrency and timing-sensitive scenarios. Through extended interfaces and mode mapping, it can further support custom scheduling algorithms and task types.

Appendix

Task Lifecycle and State Transitions

• Concurrent scheduler state transition: Waiting -> Running -> Completed/Retrying/Dead Letter

• Sequential scheduler state transition: Waiting -> Running -> Completed/Retrying/Failed

• Difference between dead letter and failure: Dead letter usually indicates maximum retry count reached or unrecoverable error

• Task priority: Builder supports priority setting (1-10, 1 is highest), can be configured during build phase

• Scheduling types: Immediate, Timed, Periodic, corresponding to different execution strategies

• State query: Get task status and list through wrapper or scheduler's GetTaskStatus/ListTasks

• Memory statistics: Evaluate memory usage through ListTasks length and TTL cleanup frequency

• Performance metrics: Can estimate throughput through concurrent scheduler's maximum concurrency and active task count

• New scheduler: Implement TaskScheduler interface to integrate with wrapper or hybrid scheduler

• Custom mode mapping: Bind task types to new scheduler through wrapper's mode mapping options

• Custom retry strategy: Configure retry count and timeout during task build phase, or implement custom retry logic in task handler

• Task type classification: Classify timing-sensitive tasks (such as payment, order) as sequential execution

• Concurrency setting: Set maximum concurrency based on hardware resources and task characteristics, avoiding resource contention

• Timeout and retry: Set reasonable timeout and retry strategies for long-running tasks, avoiding blocking

• Memory management: Reasonably set TTL and maximum task count, regularly clean up expired tasks

• Monitoring and alerting: Establish task execution monitoring through state query and logging

• Initialize scheduler wrapper at application startup, inject default concurrency and task type mapping

• Submit tasks uniformly through wrapper, supporting timed, periodic, and delayed execution

• Graceful shutdown: Call wrapper's Close method in application shutdown flow