Introduction

CeleryManagement is an open source Management Library designed to monitor and manage Celery installations. It has two primary functions:

• Visualize task and worker performance and statistics

• Policy driven monitoring of celery infrastructure to achieve

  • task latency
  • task throughput
  • worker utilization

Celery can be a critical component in server and web service architectures. Having knowledge about the performance and reliability of those celery functions is valuable. Through monitoring CeleryManagement can help identify problems before they happen. If problems have already happened, CeleryManagement can assist in visualizing any strange occurances.

CeleryManagement is also designed to be fully interoperable with celery add on packages django-celery, celery-pylons, and Flask-Celery add-on packages. This ensures that CeleryManagement can provide value to the Django, Pylons and Flask communities.

Overview

This is a high level overview of the architecure.

Event data is enabled on all celery workers by using the -E option on celeryd. This event data is sent through your existing AMQP message bus and received by CeleryManagement which delivers this information into a database. The celery data is examined and displayed such that.

Example

@TODO link to the examples page with pretty graphs

Features

Visualization	Graphically renders all worker and task data visually. This helps give you a quick look at what your celery infrastructure is organized.
Analytics	Drill down into your task and worker performance data visually.
Auto-scaling	Brings online and offline worker nodes to dynamically grow and shrink the number of worker nodes.
Cloud Enabled	Full cloud management and provisiong capabilities through `libcloud`_.
Monitoring	Celery event monitoring with email notifications capability. Monitoring can happen continuously on definable schedules.
Scheduling	Supports recurring tasks like cron, or specifying an exact date or countdown for when after the task should be executed.
Policy Driven	Control all monitoring of your system through a simple policy language. This allows you to fully customize the policies that govern your infrastructure.
Routing	Using AMQP’s flexible routing model you can route tasks to different workers, or select different message topologies, by configuration or even at runtime.
Managed Settings	Remote contorl functions have been added to manage and modify worker settings at runtime

Documentation

The latest documentation with user guides and tutorials is hosted at Github.

Installation

You can install CeleryManagement either via the Python Package Index (PyPI) or from source.

To install using pip,:

$ pip install CeleryManagement

To install using easy_install,:

$ easy_install CeleryManagement

Downloading and installing from source

Download the latest version of CeleryManagement from http://pypi.python.org/pypi/celerymanagement/

You can install it by doing the following,:

$ tar xvfz celery-management-0.0.0.tar.gz
$ cd celery-management-0.0.0
$ python setup.py build
# python setup.py install # as root

Using the development version

You can clone the repository by doing the following:

$ git clone git://github.com/bmbouter/CeleryManagement.git