Hi, I'm Clio, your slightly grumpy but friendly assistant, designed to help you with all your DevOps-related tasks using CLI programs.
I’ve been using GPTScript extensively to build agents to perform specific tasks and improve my productivity for a few months. My previous post discussed AI-powered DevOps and how I built an agent to work with GCP and Kubernetes.
With microservices and distributed applications, testing has become more complex than ever. From creating and scaling tests across distributed systems to keeping up with the pace of deployments, managing tests efficiently is difficult.
In this post, I focus on testing and see how we use GPTScript to work with Testkube, a Kubernetes-based testing framework. I’ll show how to build an agent using GPTScript to modernize testing in Kubernetes, making it faster, easier, and more adaptive to the constantly changing environment.
This blog post is built up to my talk at DevOpsDays Kerala, where I’ll speak on “Is Your Testing Stuck In the Past? Modernizing Kubernetes Testing With GPTScript and Testkube”
Testkube is a Kubernetes native testing framework designed to simplify testing. It Allows you to orchestrate and automate complex testing workflows using different testing tools, all from a single intuitive UI.
Unlike other testing frameworks and tools, Testkube executes your tests within the Kubernetes clusters, ensuring a secure and production-like environment and thus improving the reliability of your test outcomes.
Testkube integrates with all popular testing tools, including k6, Cypress, and Postman, to name a few. Testkube allows you to write Test Workflows that are single yaml file configurations to define your entire test requirement. Further, Testkube allows you to combine tests for any of these tools in any way required to perform your system tests.
With such versatility, Testkube makes testing in Kubernetes a breeze. By integrating it with GPTScript, the process becomes much easier and smarter.
GTPScript is a scripting language that automates your interactions with an LLM using natural language. It uses natural language syntax, which is easy to understand and implement. One of the highlights of GPTScript is its ability to mix these natural language prompts with traditional scripts, making it extremely flexible and versatile.
To understand the versatility of GPTScript to build agents and use it to build one for Testkube, here’s what we’ll do. On a Kubernetes cluster, we’ll deploy a simple Nginx application. The application simply prints “Hello Kerala”. We’ll build the Testkube Agent to generate a Postman Test Workflow to test the application's response. We will then update the application to print “Hello World” - upon which the Postman test will fail. We’ll ask the Testkube agent to figure out why it failed and update the Test Workflow to fix the failing test.
Note: Ensure you’re using the latest version of GPTScript.
Follow the steps mentioned in the Testkube documentation to install and configure Testkube on your Kubernetes cluster. Once configured, you can access the Testkube dashboard at testkube.io to connect with your cluster and create test workflows.
Install and configure the Testkube CLI and log in to your Testkube account. To do that, execute the “testkube login” command to initiate the login process. You can choose “GitHub” to log in to the account.
After that, select the cluster your default organization followed by the environment where you’ve configured your cluster; in this case, I have a “local-kind” cluster configured.
Once successfully logged in, you can use the Testkube CLI to perform various operations. You can learn more about the Testkube CLI by running the “testkube –help” command.
The first step is to create the GPTScript for the Testkube assistant. Since GPTScript is written primarily in natural language, writing the script to perform code review is easy. Below is the testkube.gpt file.
Name: Testkube Description: A tool to help you perform testing of your application on your Kubernetes clusters using Testkube. Context: learn-testkube, learn-kubectl Tools: sys.exec, sys.http.html2text?, sys.find, sys.read, sys.write, github.com/gptscript-ai/browse-web-page chat:true You are an assistant for Testkube and help the user create, manage and execute test workflows. You can also perform kubernetes related tasks. Rules 1. Access the testkube workflow docs at https://docs.testkube.io/articles/test-workflows and remember the latest specification to create testworkflows. 2. Use testkube CLI to interact with Testkube. 3. Use kubectl CLI to interact with the Kubernetes cluster. 4. Based on the user's request, perform actions on the Kubernetes cluster and create, manage, delete test workflows. --- Name: learn-testkube Description: A tool to help you learn testkube cli #!/bin/bash testkube --help testkube create --help testkube create testworkflow --help testkube run --help --- Name: learn-kubectl Description: A tool to help you learn k8s and related commands #!/bin/bash CMDS="kubectl helm" echo 'The additional CLI commands are available locally, use the `exec` tool to invoke them:' for i in $CMDS; do if [ -e "$(command -v $i)" ]; then echo ' ' $i fi done
You can find this GPTScript in our examples folder.
To execute this script, you must first configure the “OPENAI_API_KEY” environment variable.
Execute the script using the following command:
gptscript testkube.gpt
This will initialize the assistant and start in chat mode.
Note: Since this is an AI assistant, the responses you receive will differ from what is shown here, even if you provide the same prompts. Hence, you might have to take some extra steps/prompts to get this working.
Since the agent has access to kubectl, let's first check the pods running in the default namespace.
We then check for the associated services.
With the pod and service running, let’s ask it to make a Curl request to the hello-world-deployment pod to verify the response it gets.
The pod gives the response, “Hello Kerala”. With this, we have verified that the pods and services are running fine, and we are getting the expected response. Let’s now go ahead and create a test.
Testkube allows you to write single YAML file specifications to define your test workflow. Let us ask our Testkube AI Agent to write a Postman Test Workflow to test the response.
Below is the test-workflow yaml file it generates:
apiVersion: testworkflows.testkube.io/v1 kind: TestWorkflow metadata: name: postman-hello-world-validation spec: content: files: - path: hello-world.postman_collection.json content: | { "info": { "name": "Hello World Validation", "_postman_id": "12345-67890-abcdef", "description": "Collection to validate the response of the hello-world service", "schema": "https://schema.getpostman.com/json/collection/v2.1.0/collection.json" }, "item": [ { "name": "Validate Hello World Response", "request": { "method": "GET", "header": [], "url": { "raw": "http://hello-world-service.default.svc.cluster.local", "protocol": "http", "host": [ "hello-world-service", "default", "svc", "cluster", "local" ] } }, "response": [] } ], "event": [ { "listen": "test", "script": { "exec": [ "pm.test(\"Status code is 200\", function () {", " pm.response.to.have.status(200);", "});", "pm.test(\"Response is 'Hello, World!'\", function () {", " pm.response.to.have.body(\"Hello, World!\");", "});" ], "type": "text/javascript" } } ] } container: resources: requests: cpu: 256m memory: 128Mi workingDir: /data/repo steps: - name: Run Postman Test run: image: postman/newman:5-alpine args: - run - hello-world.postman_collection.json
We can see that it has understood how Postman tests are to be written, so it created a collection along with a test to validate the response from the hello-world-service to be “Hello Kerala”. It followed the latest guidelines on Testkube Test Workflow documentation and created a specification.
We ask it to save it in a yaml file and use that file to create the Test Workflow.
Once the Test Workflow is created, we can also validate that in our Testkube dashboard.
We’ll now ask it to execute the test workflow, and we’ll validate it on the Testkube dashboard as well.
The test has successfully passed. The response is “Hello Kerala”, which was validated by our Postman test.
Now, let us upgrade the image of our Nginx application. This upgraded image will give a response of “Hello World” instead of “Hello Kerala”. Since our agent has the knowledge and access to kubectl, we’ll ask it to update the image too.
We execute the Postman Test Workflow again to the response.
As expected, the test failed because the response is “Hello World” and our Postman test is looking for “Hello Kerala”.
The Testkube AI agent is smart enough to figure out why the test failed and provide a resolution, let’s see how that works.
We can see that it very accurately figured out why the test failed. It suggests a few next steps, which include:
We ask it to fix the Postman test and update the Test Workflow.
It updates the Test Workflow and executes it, and this time, it should pass.
As expected, the updated Postman Test Workflow passed this time as it was now looking for a “Hello World” response, and it got “Hello World”.
And that’s how we can “talk” to an AI agent to smartly and effortlessly manage our Testkube Test workflows. It was similar to talking to an intern and asking them to create and manage tests. The difference is that I interacted with an LLM, and the process was faster and smarter.
However, creating a GPTScript in the first place requires planning and attention to detail. What assistant do you want to build? What are its capabilities? What are the tools it should support? These are some questions that you need to ask yourself before writing one. And for basic tasks, this can be too much of an effort.
That’s where Clio comes in.
Clio is an AI-powered copilot designed to assist with DevOps tasks using command-line interface (CLI) agents. It harnesses OpenAI's capabilities to deliver intelligent, context-aware support directly from your terminal, simplifying the management of complex workflows and multi-cloud environments.
Installing Clio is as simple as running
brew install gptscript-ai/tap/clio
and you can use your GitHub account and get access to OpenAI - without an OpenAI account - or bring your own OpenAI API key.
You can easily configure Clio to use the Testkube agent we built. Simply follow the add
context: github.com/gptscript-ai/clio/context
~/config/clio/agents
You can see that we now have our Testkube agent as part of the clio. Do read my review of Clio. Read it to understand the nuances.
Overall, GPTScript is a wonderful and easy-to-use tool for building AI Agents. We saw that in this blog post as we created a Testkube AI agent to streamline and automate our testing.
Clio is a good sidekick if you’re into DevOps and perform many tasks using various cloud providers, Kubernetes and GitHub. While I’ve shared my opinions here, go ahead and give Clio a try and see how it fares.
For the latest updates on Clio, follow @Clio_ai on X.