Building an Event-Driven Application With RabbitMQ using CloudAMQP and Acorn Services – Part 1

Acorn Services allow developers to easily provision databases, caches, storage buckets, and other as-a-service offerings from cloud providers and Kubernetes operators within the Acorn context.

In this tutorial, we will build an Acorn Service for provisioning a RabbitMQ instance within CloudAMQP, a hosted message queuing service in the cloud.

This guide has two parts: 1) Packaging RabbitMQ as an Acorn Service, and 2) Consuming the Acorn Service from other Acorns. You can read the second part here.

By the end of this tutorial, we will have a running Acorn Service that exposes RabbitMQ within the Acorn environment.

Step 1 – Signing up with CloudAMQP

Visit CloudAMQP.com and complete the signup process. CloudAMQP has a free tier that can provision RabbitMQ instances without needing a credit card.

Once logged in, visit the API Access section and create an API key. This is useful for programmatically managing the service.

Save the API key before proceeding with the next steps. Initializing an environment variable with the API key is a good idea.

Step 2 – Create an Acorn Secret based on the CloudAMQP API Key

Let’s create an Acorn Secret for the API Key. This will be used by the Service to manage the instance’s lifecycle.

acorn secret create cloudamqp-account --data api_key=$API_KEY

Step 3 – Declare the Acorn Service

It’s time to define the Acorn Service in an Acornfile.

args: {
    instanceName: "rabbitmq-1"
    plan: "lemur"
    region: "us-east-1"
}

services: "rabbitmq-cloudamqp-server": {
    generated: job: "create-rabbitmq-cloudamqp-server"
}

jobs: "create-rabbitmq-cloudamqp-server": {
    build: context: "."
    env: {
        CLOUDAMQP_API_KEY: "secret://cloudamqp-account/api_key"
        CLOUDAMQP_INSTANCE_NAME: "\(args.instanceName)"
        CLOUDAMQP_REGION: "\(args.region)"
        CLOUDAMQP_PLAN: "\(args.plan)"
    }
    events: ["create","delete"]
}

secrets: {
    "cloudamqp-account": {
        external: "cloudamqp-account"
        type: "opaque"
        data: {
            api_key: ""
        }
    }

    "rabbitmq-instance": {
        type: "generated"
        params:{
            job: "create-rabbitmq-cloudamqp-server"
        }
    }

}

Let’s take a look at each section of the Acornfile.

The args section initializes a few defaults that can be changed during the runtime. The name of the service, the plans supported by CloudAMQP, and the AWS region parameters are defined here.

The next section, services, defines the actual service – rabbitmq-cloudamqp-server – by pointing it to the job that does the initialization and lifecycle management of the RabbitMQ instance.

The jobs section is a classic Acorn Job that builds an artifact from the Dockerfile and passes the environment variables gathered from the command line arguments and the external secret created in the previous step.

The next section is critical. It refers to the external secret storing the CloudAMQP API key and the secrets generated by the service at runtime. The second secret becomes available at runtime to other Acorns that consume the service.

For more information on Acorn Services, refer to the documentation.

Step 4 – Write the Script to Manage the Lifecycle of Acorn Service

With the Acornfile in place, it’s time to create the actual job responsible for the lifecycle of the RabbitMQ instance.

Create a file named render.sh in a directory called scripts.

#!/bin/sh

set -eo pipefail
set -x

# Set your CloudAMQP API KEY and parameters
CLOUDAMQP_API_KEY="${CLOUDAMQP_API_KEY}"
CLOUDAMQP_INSTANCE_NAME="${CLOUDAMQP_INSTANCE_NAME}"
CLOUDAMQP_PLAN="${CLOUDAMQP_PLAN}"
CLOUDAMQP_REGION="${CLOUDAMQP_REGION}"

# Function to create a RabbitMQ Instance
create_rabbitmq() {
    echo "Creating RabbitMQ instance..."
    output=$(curl -s -u :"${CLOUDAMQP_API_KEY}" -d "name=${CLOUDAMQP_INSTANCE_NAME}&plan=${CLOUDAMQP_PLAN}&region=amazon-web-services::${CLOUDAMQP_REGION}&tags=${CLOUDAMQP_INSTANCE_NAME}" https://customer.cloudamqp.com/api/instances | jq)    
    echo "RabbitMQ instance created successfully."
    CLOUDAMQP_INSTANCE_ID=`echo $output | jq -r .id`
    url=`echo $output | jq -r .url`

}

create_rabbitmq_acorn_service() {
    cat > /run/secrets/output<<EOF
    services: "rabbitmq-cloudamqp-server": {
        data: {
            "url": "${url}"
        }
    }
EOF
}

# Function to delete a RabbitMQ Instance
delete_rabbitmq() {
    echo "Deleting RabbitMQ instance..."
    output=$(curl -s -u :"${CLOUDAMQP_API_KEY}" https://customer.cloudamqp.com/api/instances | jq) 
    ID=`echo $output | jq "first(.[] | select(.name == \"$CLOUDAMQP_INSTANCE_NAME\") | .id)"`
    curl -XDELETE -u :"${CLOUDAMQP_API_KEY}" "https://customer.cloudamqp.com/api/instances/${ID}"
    echo "RabbitMQ instance deleted successfully."
}

# Main script
if [ "${ACORN_EVENT}" = "create" ]; then
    create_rabbitmq
    create_rabbitmq_acorn_service
elif [ "${ACORN_EVENT}" = "delete" ]; then
    delete_rabbitmq
else
    echo "Invalid argument. Please use 'create' or 'delete'."
fi

The script receives the parameters as environment variables during the runtime and executes the curl commands to provision and terminate the RabbitMQ instance. The events create and delete call the appropriate sections of the script to manage the lifecycle. For example, when acorn rm is called, it invokes the delete_rabbitmq function to terminate the instance.

In the parent folder, where the Acornfile is located, create a Dockerfile with the below content:

FROM python:alpine3.9 RUN apk add --no-cache curl jq ADD . /acorn ENTRYPOINT [ "/acorn/scripts/render.sh" ]

This Dockerfile becomes a part of the Acorn Job, which is a part of the Acorn Service.

Step 5 – Build and Run the Acorn Service

Let’s build, push, and run the Acorn Service. If everything goes well, running the service should create a RabbitMQ instance at CloudAMQP.

acorn build -t index.docker.io/janakiramm/rabbitmq-service:latest .

After logging into Docker Hub, push the image.

acorn push index.docker.io/janakiramm/rabbitmq-service:latest

Finally, let’s run the Acorn Service with the defaults and check if the RabbitMQ instance is created.

acorn run -n rabbitmq-01 index.docker.io/janakiramm/rabbitmq-service:latest

We can see the instance running within CloudAMQP. In the next part of this tutorial, we will build Acorns that act as a publisher and subscriber of the RabbitMQ instance running as an Acorn Service.

Conclusion

In this tutorial we’ve used the Acorn Services capabilities to create a service that provisions a RabbitMQ instance using CloudAMQP. In the second part of this post, we demonstrate how to leverage this service in a sample application. Acorn Services can be used to connect to any cloud service and incorporate it into your Acorn ecosystem. You can explore tutorials for other services, such as NATS and MongoDB Atlas.