Building a Service Acorn for NATS using NGS by Synadia

When Acorn 0.7 released earlier this month, it introduced a new feature called Service Acorns, which allows developers to easily provision instances of cloud components (database, message queues, …) so applications can consume them. If you’re not familiar with Service Acorns, Bill Maxwell wrote a good overview of how the services feature works, and how to nest Service Acorns into your application descriptions.

In my last tutorial, I explained how to create a Service Acorn which creates a MongoDB Atlas cluster and showed how a sample Webhooks app can use this cloud managed database to persist data.

In this article I want to focus on building a Service Acorn for NATS which automates the creation of a NGS user, NGS being a global distributed NATS system developed by Synadia. Once we’ve created the Service Acorn for NGS, I’ll show how the Webhooks app can use this cloud managed NATS to exchange messages via its Pub/Sub mechanism.

Prerequisites

First we need a Kubernetes cluster with Acorn 0.7+ installed on it as the Service functionality was released in Acorn 0.7.0.

Next we need a free NGS account: it can be created from https://ngs.global

We just need to go into the Sign in/Sign up menu and provide an email address

Next we select a free account (no credit card needed)

Then we get the private key which allows us to connect to this newly created account (the private key in the screenshot below is not a valid one).

For this demo we store the private key in the NGS_KEY environment variable.

Definition of the Service Acorn for NATS

To keep things clean and simple we will consider the three following files (we used the same approach in the article defining the MongoDB Atlas Service):

• Acornfile
• Dockerfile
• render.sh

The Acornfile contains the definition of the Service

This Acornfile contains 4 top level keys:

• args
• services
• jobs
• secrets

The args key defines the argument that can be passed to the Service, here it specifies the NATS subject to which the NGS user should be allowed to publish and subscribe.

The ngs Service is generated by the create-ngs-service Job. This Job contains the logic to create a NGS user, it uses the ngs cli binary present in the container image lucj/ngs-cli:v0.1.0.

The user-creds Secret is also generated by the create-ngs-service Job; it contains the credentials of the NGS user.

The ngs-creds Secret defines an external Secret containing the credentials to connect to the NGS account. This Secret must exist in the Acorn project before the Service can be used.

The Job runs within a container built with the following Dockerfile:

It basically contains the nsc cli and executes a ** render.sh** script to perform all the NGS related actions.

The render.sh file contains the logic to create / delete a user attached to the current NGS account, it gives the user the rights to :

• publish and subscribe to the subject provided as a string in the pubsub argument
• publish to the ngs.echo subject and to subscribe to the _INBOX.> subject for test purposes

This script generates the file /run/secrets/output which is an Acornfile containing the details of the Service:

• the address of the NGS distributed system
• the credentials file of the NGS user

Running the Service

Before running the Service we need to create the ngs-creds Secret containing the NGS account’s private keys.

Note: the following example uses the NGS_KEY environment variable already defined in the current shell

Next we run the Acorn:

In a few tens of seconds a new user will be created in the current NGS account, its credentials file is stored in a new Secret.

The following command shows all the existing Acorn resources:

We can get the content of the credentials file contained in the ngs.user-creds Secret:

We will verify that using this credentials file we can publish and subscribe to the NATS data. subject. This can be easily tested locally with the nats client as follows:

First we save the content of the credentials file in test.creds.

Next we run a local subscriber on the data.* subject, this one using the credentials file to authenticate:

Next we publish a payload on data.demo (a sub-subject of data.*) still using the authentication embedded in the credentials file

Then we should see the payload has been received by the subscriber

This shows the newly created user (which belongs to the current NGS account) is authorized to publish and subscribe to the data.* NATS subject

This user is not authorized to publish or subscribe to other subject as illustrated below:

We have defined the Service and verified it creates a user with the right to publish to data.*.

We can now delete the application, this will delete the associated user:

Publishing the Service

First we need to build an image of the Service:

Next we log to the DockerHub (the registry we will push the image to)

Next we push the image:

Once the image is in the registry it can be used by other applications.

Using the Service with a simple container

The following Acornfile defines an application which:

• uses a Service based on the image we built above
• runs a container relying on this Service: this container regularly sends a request to the ngs echo server waiting for a reply

This sample application can be run with the following command:

After a few tens of seconds we can see the container is running and manages to send a request (and get a response) from an NGS echo servier:

We can now delete the Acorn app, this will remove the NGS user at the same time:

Using the NGS Service with a microservice application

The Webhooks application (live version available on https://webhooks.app/) is a microservice application whose purpose is to provide a webhook (HTTP POST endpoint) on the fly. It is mainly used for tests and demos. Under the hood this application embeds a NATS server, it uses NATS for its Pub/Sub capabilities to exchange messages between several microservices.

Note: if you want to know more about this sample application, feel free to check the source code in https://gitlab.com/web-hook.

Recently the application’s Acornfile has been modified to allow the usage of the ngs Service if the –ngs flag is provided at launch time. Using the ngs Service allows the application to use NGS instead of the default embedded NATS server.

Let’s run the Webhooks app using this flag:

After a few tens of seconds a HTTP endpoint is returned (in the current example the endpoint is http://default-webhook-a1a950fb.awawu9.oss-acorn.io) to access the application.

Going to the dashboard we can see it is currently empty, which is normal as we haven’t sent any payload yet.

We will now make sure the application uses NGS under the hood.

First we get the content of the credentials file of the NGS user which was automatically created by the Service:

Next we save this content locally in ngs.creds.

Next we use ngs.creds to subscribe to the data.* subject on the connect.ngs.global server:

Next we send a test payload from the application’s dashboard (clicking on the Test Payload button):

We can see the application is working correctly as the payload was received by the frontend (under the hood the payload is sent to the API, which publishes it to NATS before it is received by a web socket server responsible for updating the frontend). As the same time we can also see the payload was received by our local subscriber (this one listening on the connect.ngs.global endpoint)

This shows that running the webhooks application with the –ngs flag ensures it uses the globally distributed NGS system instead of a local NATS server.

We can then remove the application and the associated components:

Key takeaways

In this article we explained how the NGS / NATS Service has been created and published. First we illustrated how this Service can be used with a single container, then we showed how the webhooks application can use this Service thus relying on the NGS connectivity to exchange payloads.

The code used in this article is available in https://github.com/lucj/acorn-services/tree/main/ngs.

Learn more on Acorn Services with the Acorn documentation https://docs.acorn.io/reference/services

Luc Juggery is a software engineer with 18+ years of experience and co-founder of 2 startups located in Sophia-Antipolis, southern France. You can chat with him on Twitter, read more of his work on Medium, find his tutorials on YouTube, or take one of his Docker or Kubernetes training courses on Udemy.

Header Photo by Raspopova Marina on Unsplash