Skip to content

Latest commit

 

History

History
207 lines (135 loc) · 4.99 KB

README.rst

File metadata and controls

207 lines (135 loc) · 4.99 KB
Raw

Caravan

Light python framework for AWS SWF

About this Project

Caravan is being used at Ludia for projects like marketing campaign system and on-demand distributed processing systems (thanks to hundreds of Lambda functions).

Feedbacks, ideas and contributions are highly welcomed. (Just open a Github issue).

Focus of this project

Similar projects exists (like Simpleflow which seems mature). Here is where Caravan differs from existing projects:

  • Support of AWS Lambda tasks
  • Boto3
  • KISS
  • Bring your own workflow framework (standard implementations as contribs)
  • No coupling between Decider code and Activity code
  • Paster compatible config file

Features

  • Decider worker
  • Activity task worker TODO
  • Commands to start/signal/terminate an arbitrary workflow execution
  • Command to list open workflow execution
  • Command to register a domain / list domains

Configuration

Caravan uses Boto3 to connect to AWS. See Boto 3 configuration guide for the complete documentation.

Warning

On AWS EC2, the metadata provider only provides the credentials, the must be provided by configuration.

Environment Variables

AWS_ACCESS_KEY_ID
The access key for your AWS account.
AWS_SECRET_ACCESS_KEY
The secret key for your AWS account.
AWS_DEFAULT_REGION
The default region to use, e.g. us-east-1.
AWS_PROFILE
The default credential and configuration profile to use, if any.

Configuration Files

The credentials file is located at ~/.aws/credentials:

[default]
# The access key for your AWS account
aws_access_key_id=<YOUR ACCESS KEY ID>

# The secret key for your AWS account
aws_secret_access_key=<YOUR SECRET KEY>

The settings file is located at ~/.aws/config:

[default]
# The default region when making requests
region=<REGION NAME>

It also supports profiles:

[profile dev-profile]
# The default region when using the dev-profile account
region=<REGION NAME>

Demo

Setup a SWF domain to run this example:

$ caravan-domain-register -n CaravanDemo --retention-days 1

Write a workflow type (see full demo)

from caravan import Workflow


class Demo(Workflow):

    """Noop workflow using the bare caravan API."""

    name = 'Demo'
    version = '0.1'
    default_execution_start_to_close_timeout = '600'
    default_task_start_to_close_timeout = '10'

    def run(self):
        self.task.print_events()
        self.task.add_decision('CompleteWorkflowExecution')

Run the decider with the Demo workflow:

$ caravan-decider -d CaravanDemo -m caravan.examples.demo -t default --verbose

Start an execution of the Demo workflow:

$ caravan-start -d CaravanDemo -n Demo -v 0.1 -i 1

(The Demo workflow will wait for 5 minutes)

List the executions:

$ caravan-list -d CaravanDemo
$ caravan-list -d CaravanDemo --oldest 2015-01-01

Send a signal to an execution:

$ caravan-signal -d CaravanDemo -i 1 -s PRINT --input 'Hello World!'
$ caravan-signal -d CaravanDemo -i 1 -s PRINT --input 'Lorem ipsum'
$ caravan-signal -d CaravanDemo -i 1 -s STOP

Terminate an execution:

$ caravan-terminate -d CaravanDemo -i 1

Similar projects

Python:

Ruby:

References

Development

Possibly use virtualenvwrapper to manage your virtualenvs.

$ mkvirtualenv caravan

or

$ virtualenv caravan

Clone and install development dependencies:

(caravan)$ git clone [email protected]:pior/caravan.git
(caravan)$ cd caravan
(caravan)$ pip install -e .[dev]

Run tests:

(caravan)$ nosetests

Release

The release process use zest.releaser:

$ fullrelease

License

MIT licensed. See the bundled LICENSE file for more details