Hi, I'm Clio, your slightly grumpy but friendly assistant, designed to help you with all your DevOps-related tasks using CLI programs.
Making Smart Homes even Smarter with AI: Enhancing Home Assistant Usage with GPTScript
Jul 5, 2024 by Thorsten Klein
Have you ever considered turning your home into a smart hub but felt overwhelmed by the complexity and jargon? You're not alone. The journey into home automation can seem daunting, with a steep learning curve and an array of devices that need to be perfectly synchronized to work seamlessly. However, the rewards of a smart home — energy efficiency, increased security, and convenience — are too compelling to ignore. This article delves into the world of Home Assistant, a powerful tool for home automation, combined with an AI Copilot powered by GPTScript, making it as easy as ever to transform your living space into a smart (or even smarter) home.
Home Automation with Home Assistant
Home Assistant is an open-source platform for home automation with a thriving community. It’s easy to install (e.g. on a Raspberry Pi) and, thanks to the extensive documentation and forum, fairly simple to setup. There is a huge set of integrations and AddOns to integrate virtually any smart (and some not-so-smart) devices into one cohesive platform.
Let the position of the sun control your blinds, configure a morning routine to have your coffee ready once you get up or check if you closed the backdoor even after you left - there is a lot you can do with this free piece of software.
Sometimes it can feel like it’s too much that you can do with it and that there’s just so much to configure and you’re lacking the time to browse the forum to find help. Or you’re just searching for that one entity that you can’t remember the name of…
GPTScript’s AI Copilot is here to help you out!
Building a Copilot for Home Assistant Using GPTScript
A Copilot for Home Assistant is your day-to-day companion for controlling and configuring your smart home with just natural language. It knows your devices, your existing automations and scripts and can help you control them without any “Clicking around in the UI” and without any extra configuration.
Let’s get started with it.
Pre-Requisites
What you’ll need to follow along:
- Home Assistant installed somewhere and, to have the most fun with it from the start, some devices integrated already
- The Home Assistant CLI (hass-cli) installed on your current device
- An API Key for your Home Assistant installation (make sure that the API is reachable from your current device)
- GPTScript installed on your current device
Note: You may as well run this directly on your HAOS (Home Assistant OS) device which has the hass-cli pre-installed
The Copilot in a few lines of text
The Copilot, let’s call it Harry, consists of a single GPTScript file:
name: Harry the Home Assistant context: github.com/gptscript-ai/context/workspace context: github.com/gptscript-ai/context/cli tools: github.com/gptscript-ai/answers-from-the-internet chat: true You are a friendly assistant for the open source software Home Assistant. Ask the user if they like some help with Home Assistant You have the hass-cli CLI available to you. Use that to perform tasks or answer questions about the environment. For connection and authentication, use the HASS_SERVER and HASS_TOKEN environment variables, if available.
Let’s break this down:
name: Harry the Home Assistantcontext: [github.com/gptscript-ai/context/workspace](http://github.com/gptscript-ai/context/workspace)context: [github.com/gptscript-ai/context/cli](http://github.com/gptscript-ai/context/cli)tools: [github.com/gptscript-ai/answers-from-the-internet](http://github.com/gptscript-ai/answers-from-the-internet)chat: true< the rest >Bringing it to life
All you need to be able to chat with Harry (and with it effectively to your Home Assistant installation) is a single command. This assumes that you have the aforementioned script stored in the current working directory as
ha.gptHASS_SERVER=<your-home-assistant-server-url> HASS_TOKEN=<your-token> gptscript ./ha.gpt
That’s really all it takes - have a look below at what it can do in very simple terms (I manually truncated some output to make it more readable):
$ gptscript ha.gpt Hi there! Do you need any help with Home Assistant? Whether it's setting up new devices, automating tasks, or troubleshooting issues, I'm here to assist you. Let me know how I can help! > What devices do I have available? ┌────────────────────────────────────────┐ │ Call Arguments: │ │ │ │ exec {"command":"hass-cli state list"} │ └────────────────────────────────────────┘ Running hass-cli state list ┌──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── │ ENTITY DESCRIPTION STATE │ sensor.stromverbrauch_summe Stromverbrauch Summe 408.0 │ sun.sun Sun above_horizon │ sensor.sun_next_dawn Sun Next dawn 2024-07-04T02:38:14+00:00 │ sensor.sun_next_dusk Sun Next dusk 2024-07-03T20:36:45+00:00 │ sensor.sun_next_midnight Sun Next midnight 2024-07-03T23:37:30+00:00 │ sensor.sun_next_noon Sun Next noon 2024-07-03T11:37:13+00:00 │ sensor.sun_next_rising Sun Next rising 2024-07-04T03:25:03+00:00 │ sensor.sun_next_setting Sun Next setting 2024-07-03T19:49:56+00:00 │ sensor.nachste_abholung Müll: Nächste Abholung Gelb, Grau in 5 Tagen | ... truncated ... └──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── You have a wide variety of devices integrated into your Home Assistant setup. Here are some of the categories and examples of devices you have: ### Persons ... truncated ... ### Scenes ... truncated ... This is just a snapshot of the many devices and entities you have. If you need more detailed information or help with a specific device, feel free to ask! > Where's the sun at? ┌───────────────────────────────────────────────┐ │ Call Arguments: │ │ │ │ exec {"command":"hass-cli state get sun.sun"} │ └───────────────────────────────────────────────┘ Running hass-cli state get sun.sun ┌──────────────────────────────────────────────────────────────────────────┐ │ ENTITY DESCRIPTION STATE CHANGED │ │ sun.sun Sun above_horizon 2024-07-03T03:24:16.453015+00:00 │ └──────────────────────────────────────────────────────────────────────────┘ The sun is currently above the horizon. If you need more specific details like the next sunrise or sunset times, let me know! > When do I have to take out the trash? ┌───────────────────────────────────────────────────────────────┐ │ Call Arguments: │ │ │ │ exec {"command":"hass-cli state get sensor.nachste_abholung"} │ └───────────────────────────────────────────────────────────────┘ Running hass-cli state get sensor.nachste_abholung ┌──────────────────────────────────────────────────────────────────────────────────────────────────────────┐ │ ENTITY DESCRIPTION STATE CHANGED │ │ sensor.nachste_abholung Müll: Nächste Abholung Gelb, Grau in 5 Tagen 2024-07-02T22:00:00.469792+00:00 │ └──────────────────────────────────────────────────────────────────────────────────────────────────────────┘ You need to take out the trash in 5 days. The next collection will be for the Gelb (Yellow) and Grau (Gray) bins. > It's a little too bright in my office - please help. ┌───────────────────────────────────────────────────────────────────────────────────────────────────────────┐ │ Call Arguments: │ │ │ │ exec {"command":"hass-cli service call light.turn_off --entity_id light.shelly_shellyplus1_7c87ce726fc4"} │ └───────────────────────────────────────────────────────────────────────────────────────────────────────────┘ Running hass-cli service call light.turn_off --entity_id light.shelly_shellyplus1_7c87ce726fc4 ┌────────────────────────────────────────────────┐ │ ERROR: exit status 2 │ │ OUTPUT: │ │ Usage: hass-cli service call [OPTIONS] SERVICE │ │ Try 'hass-cli service call --help' for help. │ │ │ │ Error: No such option: --entity_id │ └────────────────────────────────────────────────┘ ┌───────────────────────────────────────────────────────────────────────────────────────────────┐ │ Call Arguments: │ │ │ │ exec {"command":"hass-cli service call light.turn_off light.shelly_shellyplus1_7c87ce726fc4"} │ └───────────────────────────────────────────────────────────────────────────────────────────────┘ Running hass-cli service call light.turn_off light.shelly_shellyplus1_7c87ce726fc4 ┌──────────────────────────────────────────────────────────────────────────────┐ │ ERROR: exit status 2 │ │ OUTPUT: │ │ Usage: hass-cli service call [OPTIONS] SERVICE │ │ Try 'hass-cli service call --help' for help. │ │ │ │ Error: Got unexpected extra argument (light.shelly_shellyplus1_7c87ce726fc4) │ └──────────────────────────────────────────────────────────────────────────────┘ ┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ │ Call Arguments: │ │ │ │ exec {"command":"hass-cli service call light.turn_off --arguments entity_id=light.shelly_shellyplus1_7c87ce726fc4"} │ └─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ Running hass-cli service call light.turn_off --arguments entity_id=light.shelly_shellyplus1_7c87ce726fc4 ┌────┐ │ [] │ └────┘ The light in your office has been turned off. It should be less bright now. If you need any further adjustments, just let me know! > Lights are out, but there's too much sunlight still. ┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ │ Call Arguments: │ │ │ │ exec {"command": "hass-cli service call cover.close_cover --arguments entity_id=cover.shelly_shsw_25_3494546bbb23"} │ │ exec {"command": "hass-cli service call cover.close_cover --arguments entity_id=cover.shelly_shsw_25_4c7525344903"} │ └─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ Running hass-cli service call cover.close_cover --arguments entity_id=cover.shelly_shsw_25_4c7525344903 ┌────┐ │ [] │ └────┘ Running hass-cli service call cover.close_cover --arguments entity_id=cover.shelly_shsw_25_3494546bbb23 ┌────┐ │ [] │ └────┘ The blinds in your office have been closed to reduce the sunlight. It should be more comfortable now. If you need any further adjustments, feel free to ask! > What's the temperature in my son's room? ┌────────────────────────────────────────────────────────────────────────────────────────┐ │ Call Arguments: │ │ │ │ exec {"command":"hass-cli state get sensor.temperatursensor_kinderzimmer_temperature"} │ └────────────────────────────────────────────────────────────────────────────────────────┘ Running hass-cli state get sensor.temperatursensor_kinderzimmer_temperature ┌────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ │ ENTITY DESCRIPTION STATE CHANGED │ │ sensor.temperatursensor_kinderzimmer_temperature Temperatursensor Kinderzimmer Temperature 21.4 2024-07-03T09:27:43.378435+00:00 │ └────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ The temperature in your son's room is currently 21.4°C. If you need any further information or adjustments, feel free to ask! > Is there someone in my driveway? ┌───────────────────────────────────────────────────────────────────────────────┐ │ Call Arguments: │ │ │ │ exec {"command":"hass-cli state get binary_sensor.einfahrt_person_occupancy"} │ └───────────────────────────────────────────────────────────────────────────────┘ Running hass-cli state get binary_sensor.einfahrt_person_occupancy ┌───────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ │ ENTITY DESCRIPTION STATE CHANGED │ │ binary_sensor.einfahrt_person_occupancy Einfahrt person occupancy off 2024-07-03T09:41:27.846592+00:00 │ └───────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ There is currently no one in your driveway. If you have any other questions or need further assistance, just let me know! > When was the last person seen on the terrace? ┌───────────────────────────────────────────────────────────────────┐ │ Call Arguments: │ │ │ │ exec {"command":"hass-cli state get sensor.terrasse_person_name"} │ └───────────────────────────────────────────────────────────────────┘ Running hass-cli state get sensor.terrasse_person_name ┌──────────────────────────────────────────────────────────────────────────────────────────────┐ │ ENTITY DESCRIPTION STATE CHANGED │ │ sensor.terrasse_person_name Terrasse Person Name 2024-07-03T07:22:19.884563+00:00 │ └──────────────────────────────────────────────────────────────────────────────────────────────┘ The last person was seen on the terrace on July 3rd, 2024 at 07:22:19. If you need any more information or assistance, just let me know!
Findings
What’s interesting about those super simple use cases is:
- From my natural language instructions/questions, the copilot finds the correct devices/sensors/triggers/etc. to read/control to achieve my imposed goal
- Even though some of my Home Assistant’s entities are named in German and have German descriptions, the copilot finds them and interprets them correctly due to the power of the Large Language Model
- Have another look at the actions taken for my prompt “It's a little too bright in my office - please help.” - The copilot tries a command, then sees that it fails and tries to correct itself until it succeeds
- No matter how the data is returned from Home Assistant, the copilot pimps it up and returns it to us in a clear and easily understandable way.
Prompt Ideas / Outlook
There are so many possibilities, that they can’t be shown in the example above, but here are some ideas what to do with this agent:
- “Generate an automation for my morning routine: I want to get up with the sunlight, have the radio in the bathroom turned on and receive a notification with the daily news.”
- With no more typing involved, your copilot could generate the whole Home Assistant automation config (YAML) for you and even test it against the Home Assistant API to validate that it’s working and won’t break your setup
- “I’m leaving home, make sure everything’s secure until I come back”
- Security system gets armed, door contacts turned on and motion detection on the cameras enabled - another way to achieve this is creating a scene or an automation for it or to go to each entity manually to toggle it… but it’s way cooler to just ask your copilot, isn’t it?
- “When do I have to take out the paper bin?”
- Just as in the example, this will tell you when the paper trash will be collected according to the waste collection schedule integration.
- “Am I currently producing or consuming energy?”
- This will tell you, whether, according to your smart power meter, you’re currently consuming energy from the grid or feeding back some self-made energy from your solar panels
Conclusion
Home Assistant is incredibly powerful on its own, letting you integrate all your devices and automate just about everything in your smart home. But in combination with the GPTScript-powered Copilot, we can open up this mighty platform to even more people, without them having to watch lots of videos and read up on documentation upfront. That’s for Home Assistant - but with GPTScript, you can easily build an assistant or copilot for any other system yourself. Visit our GitHub to find out more and explore some tools we already built for you - Get started today.
