Sequential Scheduler

Table of Contents

1. Introduction
2. Project Structure
3. Core Components
4. Architecture Overview
5. Detailed Component Analysis
6. Dependency Analysis
7. Performance Considerations
8. Troubleshooting Guide
9. Conclusion
10. Appendix

Introduction

This document systematically describes the design and implementation of the sequential scheduler, focusing on the following aspects:

• Task queue management and execution order assurance
• Serialization processing and concurrency comparison
• Task queuing mechanism, execution order control, and task dependency handling
• Differences from concurrent scheduler and applicable scenarios
• Configuration methods, performance considerations, and error handling strategies
• Practical usage scenarios and code example paths

The sequential scheduler is suitable for task scenarios requiring strict serial execution, such as timed tasks, periodic tasks, batch processing tasks requiring avoidance of resource competition or state conflicts, etc.

Project Structure

The sequential scheduler is located in the task scheduling module, together with the concurrent scheduler and hybrid scheduler, forming a unified task execution infrastructure. Its core files are as follows:

• Sequential scheduler implementation: pkg/tasks/sequential_scheduler.go
• Concurrent scheduler implementation: pkg/tasks/concurrent_scheduler.go
• Scheduler interface and data models: pkg/tasks/core.go
• Task builder: pkg/tasks/builder.go
• Scheduler wrapper (unified entry): pkg/tasks/wrapper.go
• Hybrid scheduler (selects execution mode by task type): pkg/tasks/hybrid_scheduler.go
• Unit tests and application layer tests: pkg/tasks/tasks_test.go, pkg/bootstrap/app_scheduler_test.go

• Sequential Scheduler (sequentialScheduler): Single-threaded serial execution, ensuring strict order between tasks; built-in pending execution queue, task completion notification channel, task TTL and maximum task count limits.
• Concurrent Scheduler (concurrentScheduler): Multi-task concurrent execution, limited by maximum concurrency; supports timed/retry task polling processing.
• Hybrid Scheduler (HybridScheduler): Automatically selects sequential or concurrent execution based on task type or scheduling time; supports configuration of task type to execution mode mapping.
• Scheduler Wrapper (SchedulerWrapper): Unified external entry point, responsible for task submission, status query, cancellation, shutdown, etc.; supports default retry count, concurrency configuration, task type mode setting.
• Task Builder (TaskBuilder/builtTask): Encapsulates task ID, type, scheduling time, handler function, callbacks, timeout, retry, priority, and other attributes to generate schedulable tasks.

The sequential scheduler adopts a dual-goroutine architecture of "Executor + Monitor":

• Executor (runTaskExecutor): Traverses the pending execution queue, takes out tasks in order and executes them; marks isRunning during execution to prevent concurrent execution.
• Monitor (runTaskMonitor): Scans timed tasks and expired tasks every second, adds expired tasks to the pending execution queue; regularly cleans up expired completed tasks.

Sequential Scheduler (sequentialScheduler)

• Data Structures and Key Fields
  • Task mapping and mutex: tasks, taskMutex
  • Pending execution queue and mutex: pendingTasks, pendingMutex
  • Execution control: isRunning, taskDoneChan, taskQueue, taskQueueClosed
  • Lifecycle: startOnce, stopChan, wg
  • Task lifecycle: taskTTL, maxTaskCount
• Core Processes
  • Schedule: Validate task, store task, enqueue to pending execution queue, notify executor, attempt to join task queue
  • Cancel: Call task cancellation callback, update status, remove from pending execution queue
  • runTaskExecutor: Take tasks in order, execute, update status, handle retry/periodic
  • runTaskMonitor: Periodic scanning, enqueue expired tasks, cleanup expired tasks
  • executeTask: Timeout control, error handling, retry exponential backoff, periodic task rescheduling
  • Cleanup strategies: cleanupExpiredTasks (by TTL), cleanupOldTasks (by oldest completed)

• Design Points: Maximum concurrency control, active task counting and condition variables, timed/retry task polling processing, expired task cleanup.

• Key Processes: Schedule (immediate tasks execute directly), Cancel (update status and trigger callback), executeTask (concurrency slot waiting and release).

• Design Points: Select sequential/concurrent based on task type or scheduling time; merge state and list of two schedulers; only open maximum concurrency setting for concurrent scheduler.

• Key Processes: Schedule (type mapping/timed/periodic default sequential), Cancel (attempt on both schedulers), GetTaskStatus (concurrent priority), ListTasks (merge lists).

• Design Points: Unified external entry; supports default retry count, maximum concurrency, task type execution mode mapping; provides convenient timed/periodic task submission methods.

• Key Processes: RunTask series (immediate/timed/periodic), RunTypedTask (submit by type), SetTaskTypeMode (dynamic mode switching), SetMaxConcurrentTasks (effective on concurrent scheduler).

• Design Points: Chain configuration of task ID, type, handler function, callbacks, scheduling time, timeout, retry, priority; generates built-in task implementation.

• Key Processes: Build (validate parameters/calculate execution time), builtTask (implements Task and TaskInfoProvider interfaces).

• Interface Contracts

  • TaskScheduler: Unified scheduler interface, sequential/concurrent/hybrid schedulers all implement this interface.
  • Task/TaskInfoProvider: Task and task information interfaces, builtTask implements both.

• Component Coupling

  • Sequential scheduler internally depends on task information interface to read/update status; depends on task timeout and retry configuration.
  • Wrapper depends on scheduler interface, provides unified API upward; delegates to specific scheduler implementations downward.
  • Hybrid scheduler composes sequential/concurrent schedulers, distributes tasks by rules.

• Sequential Execution Throughput: Single-threaded serialization, throughput limited by single task processing speed; suitable for scenarios sensitive to order or resource competition.

• Memory and Cleanup: Maintains task mapping and pending execution queue, supports TTL and maximum task count limits; expired tasks and oldest completed tasks cleanup reduce memory pressure.

• Channels and Mutexes: Uses buffered task queue and notification channel, cooperates with mutex to protect shared state, avoiding race conditions.

• Retry and Timeout: Sequential scheduler supports exponential backoff retry and task timeout control, avoiding blocking other tasks.

• Comparison with Concurrent Scheduler: Concurrent scheduler controls parallelism through maximum concurrency and condition variables, suitable for high throughput, stateless or parallelizable tasks.

• Task Not Executing

  • Check if task status is waiting and expired; confirm if runTaskMonitor scanned timed tasks and added them to pending execution queue.
  • Confirm if executor is blocked due to isRunning being true; check if taskDoneChan is correctly sent/received.

• Task Stuck or Timeout

  • Check if task Handler is blocked; sequential scheduler creates context with timeout for tasks.
  • Observe retry count and NextRetryAt; confirm if exponential backoff causes delay.

• Memory Growth

  • Check if cleanupExpiredTasks and cleanupOldTasks cleanup as expected; adjust taskTTL and maxTaskCount.

• Concurrent/Sequential Confusion

  • Hybrid scheduler selects execution mode based on task type or scheduling time; confirm type mapping and scheduling time configuration.

The sequential scheduler provides reliable assurance for tasks requiring order consistency and resource security through strict serialization execution and comprehensive task lifecycle management. Combined with hybrid scheduler and scheduler wrapper, it can flexibly select optimal execution modes for different task types within the same system and provide unified configuration and usage experience.

Appendix

Differences and Applicable Scenarios Between Sequential and Concurrent Schedulers

• Sequential Scheduler

  • Advantages: Strict order, resource safety, easy debugging and auditing
  • Disadvantages: Limited throughput, higher latency
  • Applicable: Timed tasks, periodic tasks, state machine driven, external system interaction

• Concurrent Scheduler

  • Advantages: High throughput, low latency
  • Disadvantages: Need to pay attention to race conditions and resource conflicts
  • Applicable: Stateless or parallelizable tasks, batch data processing

• Create Sequential Scheduler Using Scheduler Wrapper

  • Example path: wrapper.go

• Set Default Retry Count

  • Example path: wrapper.go

• Set Maximum Concurrency (effective on concurrent/hybrid schedulers)

  • Example path: wrapper.go

• Specify Task Type Execution Mode (hybrid scheduler)

  • Example path: wrapper.go

• Directly Use Sequential Scheduler

  • Example path: sequential_scheduler.go

Performance and Error Handling Strategies

• Performance

  • Reasonably set task TTL and maximum task count to avoid memory bloat
  • Sequential execution suitable for small batch, high-value tasks; large batch tasks suggest concurrent or hybrid mode

• Error Handling

  • Sequential scheduler supports exponential backoff retry and task timeout; can enter retry or failed state after failure
  • Monitor regularly cleans up expired tasks to avoid zombie task accumulation

• Sequential Execution Verification (Unit Test)

  • Scenario: Submit multiple tasks, verify execution order
  • Example path: tasks_test.go

• Hybrid Scheduler Concurrent/Sequential Comparison (Application Layer Test)

  • Scenario: Same batch of tasks go concurrent/sequential execution by type respectively
  • Example path: app_scheduler_test.go

• Task Build and Submission

  • Scenario: Immediate execution, timed execution, periodic execution
  • Example path: wrapper.go

• Task Cancellation and Status Query

  • Example path: wrapper.go