A generic background task runner for crystal applications supporting periodic (CRON) and manually queued jobs

CircleCI Crystal Version GitHub

Mosquito is a generic background job runner written specifically for Crystal. Significant inspiration from my experience with the successes and failings of the Ruby gem Sidekiq.

Mosquito currently provides these features:

  • Delayed execution
  • Scheduled / Periodic execution
  • Job Storage in Redis
  • Crystal hash style ? methods for parameter getters which return nil instead of raise
  • Automatic rescheduling of failed jobs
  • Progressively increasing delay of failed jobs
  • Dead letter queue of jobs which have failed too many times
  • Rate limited jobs

Current Limitations:

  • Job failure delay, maximum retry count, and several other variables cannot be easily configured.
  • Visibility into the job queue is difficult and must be done through redis manually.

Project State

Updated 2019-04-26

Sufficient working beta.

Use in a production environment at your own risk, and please open issues and feature requests.


Update your shard.yml to include mosquito:

+  mosquito:
+    github: robacarp/mosquito

Further installation instructions are available for use with Amber as well as a vanilla crystal application:


Step 1: Define a queued job

class PutsJob < Mosquito::QueuedJob
  params message : String

  def perform
    puts message

Step 2: Trigger that job

PutsJob.new(message: "ohai background job").enqueue

Step 3: Run your worker to process the job

crystal run bin/worker.cr


> crystal run src/worker.cr
2017-11-06 17:07:29 - Mosquito is buzzing...
2017-11-06 17:07:51 - Running task puts_job<...> from puts_job
2017-11-06 17:07:51 - [PutsJob] ohai background job
2017-11-06 17:07:51 - task puts_job<...> succeeded, took 0.0 seconds

More information about queued jobs in the wiki.

Periodic Jobs

Periodic jobs run according to a predefined period.

This periodic job:

class PeriodicallyPutsJob < Mosquito::PeriodicJob
  run_every 1.minute

  def perform
    emotions = %w{happy sad angry optimistic political skeptical epuhoric}
    puts "The time is now #{Time.local} and the wizard is feeling #{emotions.sample}"

Would produce this output:

2017-11-06 17:20:13 - Mosquito is buzzing...
2017-11-06 17:20:13 - Queues: periodically_puts_job
2017-11-06 17:20:13 - Running task periodically_puts_job<...> from periodically_puts_job
2017-11-06 17:20:13 - [PeriodicallyPutsJob] The time is now 2017-11-06 17:20:13 and the wizard is feeling skeptical
2017-11-06 17:20:13 - task periodically_puts_job<...> succeeded, took 0.0 seconds
2017-11-06 17:21:14 - Queues: periodically_puts_job
2017-11-06 17:21:14 - Running task periodically_puts_job<...> from periodically_puts_job
2017-11-06 17:21:14 - [PeriodicallyPutsJob] The time is now 2017-11-06 17:21:14 and the wizard is feeling optimistic
2017-11-06 17:21:14 - task periodically_puts_job<...> succeeded, took 0.0 seconds
2017-11-06 17:22:15 - Queues: periodically_puts_job
2017-11-06 17:22:15 - Running task periodically_puts_job<...> from periodically_puts_job
2017-11-06 17:22:15 - [PeriodicallyPutsJob] The time is now 2017-11-06 17:22:15 and the wizard is feeling political
2017-11-06 17:22:15 - task periodically_puts_job<...> succeeded, took 0.0 seconds

More information: periodic jobs on the wiki

Throttling Jobs

Jobs can be throttled to limit the number of messages that get executed within a given period of time. For example, if 10 messages were enqueued for ThrottledJob at one time; 5 would be executed immediately, then pause for a minute, then execute the last 5.

class ThrottledJob < Mosquito::QueuedJob
  params message : String
  throttle limit: 5, period: 60

  def perform
    puts message

Connecting to Redis

Mosquito currently reads directly from the REDIS_URL environment variable to connect to redis. If no variable is set, it uses redis connection defaults to connect to redis on localhost.


Contributions are welcome. Please fork the repository, commit changes on a branch, and then open a pull request.


This repository uses minitest for testing. As a result, crystal spec doesn't do anything helpful. Do this instead:

make test

In lieu of crystal spec bells and whistles, Minitest provides a nice alternative to running one test at a time instead of the whole suite.


  • robacarp robacarp - creator, maintainer




  • Update to crystal 0.27, thanks @blacksmoke16



  • Logo contributed by @psikoz
  • Add several more automated tests
  • Add configuration for CI : make test demo will run all acceptance criteria
  • Add demo section
  • Release version 0.2.1


  • Update to specify crystal 0.26
  • Add several tests
  • Add makefile



  • Update to specify crystal-redis 2.0 and crystal 0.25
  • Release version 0.2.0



  • Breaking: Update Mosquito::Model type alias to match updates to Granite
  • Misc typo fixes and flexibility upgrades
  • Update Crystal specification 0.23.1 -> .24.2
  • Correctly specify and sync version numbers from shard.yml / version.cr / git tag
  • Release version 0.1.1


  • Use configurable Logger instead of writing directly to stdout
  • Job classes can now disable rescheduling on failure
  • Log output is now colorized and formatted to be read by human eyes
  • BUG: task id was mutating on each save, causing weird logging when tasks reschedule.


  • PERFORMANCE: adding IDLE_WAIT to prevent slamming redis when the queues are empty. Smarter querying of the queues for work.
Github statistic:
  • 90
  • 5
  • 7
  • 7
  • 10
  • 7 days ago


MIT License