What should I cook today? Using GPTScript to build an AI recipe generator

November 30, 2022, marked the start of the GenAI revolution with the birth of ChatGPT. I remember being in awe of it and bombarding it with queries of all types. I also saw my team utilize ChatGPT for technical tasks, from coding to unit testing.

Personally, I felt that ChatGPT was replacing Google for me. If you look at my Google search history, you'll mostly find "What do I eat?" queries. So, I started asking ChatGPT to help me. And I was surprised that some combinations it suggested turned out to be good! But it was a tedious process to type in the ingredients all the time and wait for a response.

To make the process more intuitive, I decided to cook up an app that analyzes uploaded photos to suggest recipes based on available ingredients.

I knew I had to leverage OpenAI's vision API, but how could I do that without going through the learning curve? That's when I came across this tweet about GPTScript that simplifies interaction with OpenAI APIs.

In this blog post, I'll tell you how I built RecipAI using GPTScript to analyze a photograph for ingredients and, based on the ingredients, suggest a recipe.

But First - What Is GPTScript?

Most of us have interacted with ChatGPT at least once and found it fascinating. While the interactive UI is good for generating text, getting coding assistance, and performing similar tasks, leveraging the full potential effectively requires understanding the nuances of crafting prompts and interacting with APIs, which means that it’s a steep learning curve.

Enter GPTScript, a scripting language that is designed to bridge this gap. It is tailored to automate your interactions with an LLM like OpenAI. What makes GPTScript so interesting is that it uses natural language as the programming language for building apps powered by LLMs.

How does GPTScript work?

At the core of GPTScript lies tools. Tools are like functions that perform a particular set of actions. And just like function calls, tools can call other tools as well. These tools are also written using natural language prompts. The best part is using custom commands to invoke a program instead of natural language.

All these tools are written in a ".gpt" file, and each file can have multiple tools. The decision of whether a tool will be executed or not is determined by the AI model. Read more about how GPTScript works.

With GPTScript, users can build powerful tools using natural language or seamlessly mix natural language prompts with conventional scripting elements to interact with LLMs. This makes it a good option for anyone who wants to harness the power of generative AI without the traditional barriers. You can read more on the project’s GitHub page about how GPTScript works.

Building RecipAI - a smart recipe generator

The idea of building the recipe generator stemmed from my constant battle with the question, “What to cook today?”. While I had used ChatGPT in the past to get some ideas, the biggest pain was telling it all the ingredients I had in my kitchen or had bought at the store. When I first saw GPTScript, I thought I might be able to whip up an app that would make the process much easier by uploading a photo of the ingredients and letting AI tell me what to do; I had to short-circuit the process.

How RecipAI works

I built the application in python, using GPTScript under the hood, and powered by OpenAI APIs. It uses GPTScript vision for analyzing an image and GPTScript to generate a recipe based on the ingredients.

Below is a high-level overview of how it works:

1. A user takes/uploads a photo of groceries and uploads it to the web app.
2. Using GPTScript Vision, it analyzes the photo and identifies the ingredients.
3. These ingredients are taken as inputs by GPTScript to generate a recipe.

I’ve made the application available in the examples section of the GPTScript examples repo, but I’ll also walk through the process of how I built it so you can build your own or something similar.

Prerequisites

Before you can start, make sure you have all of the following:

• An OpenAI API Key
• Python 3.8 or later
• Node.js and npm
• Flask

Writing the GPTScript

Creating the GPTScript is the first step in building the Recipe Generator. Since GPTScript is written primarily in natural language, it’s very easy to write the script for generating recipes. Below is the

script

tools: sys.find, sys.read, sys.write, recipegenerator, tool.gpt

Perform the following steps in order:
1. Give me a list of 5 to 10 ingredients from the picture grocery.png located in the current directory and put those extracted ingredients into a list.
2. Based on these ingredients list, suggest one recipe that is quick to cook and create a new recipe.md file with the recipe

---

name: recipegenerator
description: Generate a recipe from the list of ingredients
args: ingredients: a list of available ingredients.
tools: sys.read
Generate 1 new recipe based on the ingredients list

The script calls

, part of GPTScript vision, the default tool for Vision CLI. So we are essentially using two tools here, the default and our custom tool named which calls tool.gpt to analyze the image. Apart from that, the script is self-explanatory.

We’re asking the model to:

1. Analyze the image and identify 5-10 ingredients from it.
2. Analyze these ingredients and generate a recipe.

Executing the GPTScript

To execute this script, you must first configure the

environment variable. Further, you need to install the npm modules required by GPTScript vision CLI. You can simply run in the directory, and it will install all the dependencies.

With this, our recipegenerator.gpt is ready to be executed.

The script requires an image named

in the current directory. So you can click a picture or use any image and save it with the name in the directory. We’ll use the following image as an example:

Execute the script using the following command:

• The script uses
  tool.gpt
  to interact withl OpenAI LLM and starts by analyzing the image.
• Once the ingredients are identified, the recipe generator takes those ingredients, generates a recipe, and saves it into a
  recipe.md
  file.

The whole process takes a few seconds to complete, and you can see the output in the terminal or the generated files.

Python App

With the scripts running as expected, let us add a UI. We’ll be using Flask to build one, but you can use any other framework.

The web app has a simple UI that allows the user to upload a file to the directory and then execute the

script using subprocess.

Backend Logic

Below are the steps that were involved in building this:

• After the environment setup, the first step was to integrate
  recipegenerator.gpt
  in Python. I created an
  app.py
  file and used Python’s subprocess capability to invoke the GPTScript. I defined the path to the GPTScript and used popen to invoke the script using
  gptscript
  prompt

# Execute the script to generate the recipe
   subprocess.Popen(f"gptscript {SCRIPT_PATH}", shell=True, stdout=subprocess.PIPE, cwd=base_dir).stdout.read()

• Once I got the
  recipegenerator.gpt
  to work from my Python app, I added Flask to the project and made necessary changes with respect to routes. Added a
  /
  route that would execute when the app is launched, and another
  /upload
  route that will execute when the file is uploaded.
• Finally, I added the logic to handle file uploads. This function takes the file uploaded by the user and saves it as
  grocery.png,
  which will be used by the recipegenerator.

Frontend UI

• After the backend was ready, it was time to build the front end. I used simple HTML, CSS, JS, and Bootstrap to build the front end.
• Since we are using Flask, all the UI-related files go to
  templates
  directory. So created an index. html file with a few elements, including a loader that shows up when the script is still executing, form elements to allow the user to upload a file
• The page also has client-side logic written in JS to take the form input, send it to the backend, get the response, and display it on the screen.

Pro Tip: You can even use ChatGPT to build the UI for this.

Once both these parts are ready, execute the application. If you’re using an IDE like VSCode, simply hit F5, or you can execute

or

The application will be available at http://127.0.0.1:5000/. Go ahead and upload a photo with your grocery/ingredients. The GPTScript runs in the background, analyzes the photo for ingredients, and shows the recipe.

Watch the video

I recorded a quick video to show you how this all works.

Closing Thoughts

GPTScript makes it super easy to use natural language to interact with OpenAI. This recipe generator is just one of the many things you can do with AI and GPTScript. There are many more examples that you can try or even contribute your existing use cases and show the world what you’re up to.

If you’re struggling to interact with LLMs, give GPTScript a try. If you’re new here or need guidance, join GPTScript discord server and get all your queries answered.