ChatGPT Plugins: What is the future of the interface? And is it AI?

The internet is alive with ChatGPT and generative AI discussions. Some say “it’s all hype!”, like the recent crypto trend, whilst others believe this is the biggest single legitimate step change in technology we’ve seen in decades. Regardless, it’s interesting to keep up on and see the opportunities as they emerge. Let’s talk about the recently announced ChatGPT plugin system. This won’t be an in depth explanation of AI, LLM’s or ChatGPT, but rather my initial response looking at the social media reactions to the ChatGPT plugin system. I’m curious about the response, where people think it’s going, where I think it’s going, the limitations, etc.

Here’s the TL;DR: of how it works…

In a nutshell, ChatGPT announced the ability to make plugins, this essentially allows ChatGPT to interface with APIs and data that it does not yet have access to. Opening up a wealth of potential opportunities. Mitchells tweet caught my eye in particular though, notably: “absolutely no glue code”. Anyone who has built, or interacts with APIs knows that on the surface APIs appear simple, but under the hood APIs get fiddly: versioning, API chaining, rate limiting, authentication, backwards compatibility. Can ChatGPT drastically simplify how we consume and work with API’s?

The answer: Actually… kinda… yeah.

Here are a couple quotes that I plucked from the release announcement that I thought particularly useful or interesting…

That’s the current known limitation of LLM… that it’s mostly based on text, and doesn’t have a great understanding of the world, like physics, gravity, mechanics, that type of thing. So, are ChatGPT plugins going to help AI better understand the real world context needed to reach another level of intelligence beyond a very, very advanced game of playing “guess the next word in this string of words”? That’s the question.

So, there is actually an interface between ChatGPT and your API’s. It’s not just magically understanding the API’s and their context, sadly. You need to provide an OpenAPI specification, so that the data, behaviours and parameters can be understood, and a manifest file, which is (mostly static metadata), but does have a few interesting instructional properties, such as the descriptions you provide the model to instruct about how to work with the API set.

So, really there’s not that much to it. You feed in a manifest file with some metadata, you tell ChatGPT which endpoints to use, it roughly figures out how to call GET API’s and feed in parameters, etc. This is super cool, but it’s mostly fun to think about: what next? What does this enable? What are some opportunities? Flaws? Where could we see this going?

Since this technology is super new, we’ll simply have to wait and see what comes of it. So, here instead of answers, I wanted to pose a series of questions that came to me as I was reading through the announcements and the reaction…

OpenAPI have mentioned that they are looking to standardise the interface (this is really like an API for APIs that… read APIs?) which would mean some interoperability between AI providers, so it might be less of the “app store” moment than it first appears. This should go some way to calming those who say it’s too early to jump on this trend.

And it does seem that OpenAPI also intends to expose this behaviour beyond embedding directly in ChatGPT itself and through their own plugin ecosystem. Again, less of the “app store” moment, seems that they’re just starting with plugins inside ChatGPT, but eventually they can be used elsewhere. Unlike Google, I don’t see the ChatGPT interface being the core way we interact with this technology, so I fully expect plugins to move outside of the ChatGPT interface.

An immediate thought was the engineer in me wondering about about how effective and efficient these clients will be, won’t they flood API requests? What about optimisation? These OpenAPI plugins are almost like “GraphQL”, but for humans. An abstraction on top of APIs, but you pay for the flexibility in API request cost because the queries are not optimised for minimising data transfer and number of requests, it’s all about getting the “right” or most complete answer. It’s like a much more flexible query language, but the underlying API is potentially going to get flooded with requests.

Another thought coming to me is where this leaves front-end development. If ChatGPT plugins wrap API’s, isn’t that the job of a front-end developer? To take a set of APIs and make them coherent to a human? There are still many use cases for regular UI’s, especially when the interface is particularly relevant to the query, but for things like: booking a restaurant, getting directions, etc. I don’t see any of these use cases being served better than through natural language anyway, there is rarely a visual component. Certainly natural language and augmented, or mixed reality for the visual component.

One other wacky thought I have is… can ChatGPT generate API clients? ChatGPT itself is essentially an API client that reads OpenAPI specs, so why can’t we use that for regular programming clients? Let’s say we are on version 1.0 of an API, and the API is updated to 2.0. If all clients are using AI interfaces that could immediately adapt to the new API contract, could that eliminate the need for backwards compatibility?

So, that’s it for now. Not too in depth, just some initial thoughts and ideas that I wanted to share as I wrapped my own head around this super interesting announcement. If you’re keen to learn more, the announcement post and the documentation goes a long way to understanding how the plugin system works, these two links are mostly all that you need. If you have thoughts about where things are headed, reach out, I’d love to hear the ideas.