module JoobQ

Overview

Module JoobQ

The main JoobQ module, which acts as the central orchestrator for a job queueing system. This module integrates various components like job queueing, scheduling, statistics tracking, and logging. Here's a detailed documentation of the JoobQ module:

Overview

The JoobQ module is the core module of a job queue system designed in Crystal. It sets up and manages the entire job processing environment, including configuration, queue management, scheduling, and statistics.

Constants

• REDIS: Initialized with the Redis client instance from Configure.instance, used for all Redis operations within the module.

Initialization and Configuration

• Log.for("JoobQ"): Initializes logging for the JoobQ system.
• Log.setup_from_env: Sets up logging configuration from environment variables.

Methods

• .configure: Provides a way to configure the JoobQ system. Yields to a block with Configure.instance for setting up configurations.
• .config: Returns the configuration instance (Configure.instance).
• .queues: Returns the hash of queues set up in the configuration.
• .statistics: Returns an instance of the Statistics for tracking and managing statistical data.
• push(job): Adds a job to its respective queue in Redis and logs the action. Returns the job's unique identifier (jid).
• .scheduler: Returns an instance of the Scheduler for managing job scheduling.
• .[](name : String): A shorthand method to access a specific queue by name from the configured queues.
• .reset: Clears the Redis database and re-creates the statistical series.
• .forge: The main method to boot the JoobQ system. It initializes the statistics, starts the scheduler, starts all queues, and logs the initialization process.

Usage

• To initialize and start the JoobQ system, call JoobQ.forge.
• Use JoobQ.configure to set up system configurations.
• Jobs can be pushed to the queue using JoobQ.add(job).
• Access specific queues or the scheduler as needed.

Notes

• The JoobQ module brings together different components like queues, scheduler, and statistics into a cohesive system.
• The use of a centralized Redis client ensures consistent database interactions.
• Logging and statistics creation are integral parts of the module, facilitating monitoring and debugging.
• The module's design allows for flexible configuration and easy management of job queues.

Defined in:

joobq.cr
joobq/api_server.cr
joobq/configure.cr
joobq/dead_letter.cr
joobq/fail_handler.cr
joobq/global_stats.cr
joobq/job.cr
joobq/job_schema_registry.cr
joobq/metrics.cr
joobq/queue.cr
joobq/queue_metrics.cr
joobq/redis_store.cr
joobq/scheduler.cr
joobq/store.cr
joobq/throttler.cr
joobq/timeout.cr
joobq/worker.cr
joobq/worker_manager.cr

Constant Summary

CONFIG = Configure.new

Class Method Summary

• .[](name : String)
• .add(job)
• .config
• .configure(&)
• .forge
• .queues
• .reset
• .scheduler
• .statistics
• .store

Class Method Detail