As it turns out, GPT-4 was not the only ace up OpenAI’s sleeve, as the platform has just released a massive announcement: ChatGPT is getting Plugin support. And not just any other plugins either, as ChatGPT is going to get support for real-time web browsing for sourcing data, as well as numerous integrations with platforms such as Zapier, Wolfram Alpha, FiscalNote, and many others.
This announcement will send shivers down the spine of giants like Google, who have just released Bard to US and UK users. In fact, I don’t think anyone expected OpenAI to do this so soon, which might also indicate that they had been working on adding Plugins support for quite some time already.
Let’s break the announcement down.
An overview
OpenAI has taken a significant step forward in enhancing the capabilities of ChatGPT by introducing initial support for plugins. These tools are designed with safety as a core principle, enabling ChatGPT to access up-to-date information, perform computations, and utilize third-party services.
Implementing plugins aligns with OpenAI’s iterative deployment philosophy, allowing the organization to study real-world usage, impact, and safety challenges.
OpenAI is starting with a small user base and plans to gradually roll out larger-scale access, ultimately allowing API users to integrate plugins into their products after an alpha period.
A select group of developers from the waitlist will be invited to use OpenAI’s documentation to create plugins for ChatGPT. The first plugins have been created by notable companies, including Expedia, FiscalNote, Instacart, KAYAK, Klarna, Milo, OpenTable, Shopify, Slack, Speak, Wolfram, and Zapier.
OpenAI is also hosting two plugins – a web browser and code interpreter – and has open-sourced the code for a knowledgebase retrieval plugin for developers to self-host.
Plugin alpha access will be extended to users and developers from the waitlist, with initial prioritization given to a limited number of developers and ChatGPT Plus users.
OpenAI plans to roll out larger-scale access over time.
Expanding ChatGPT with a Browsing Plugin
The latest addition to ChatGPT’s capabilities is the browsing plugin, an experimental model inspired by previous work like WebGPT, GopherCite, BlenderBot2, and LaMDA2. By allowing language models to access information from the internet, the browsing plugin expands the scope of content ChatGPT can discuss beyond its training corpus, bringing fresh and up-to-date information to users.
The browsing plugin enables ChatGPT to perform thorough and interpretable research, enhancing user experience by providing answers to questions that previously would have been limited by the model’s training data.
For example, ChatGPT can now retrieve recent information about the latest events and even compose poetry based on the retrieved data, demonstrating the additive nature of browsing in enhancing user interactions.
The browsing plugin leverages the Bing search API to retrieve content, benefiting from Microsoft’s work on source reliability, truthfulness of information, and “safe-mode” for preventing the retrieval of problematic content. By operating within an isolated service, ChatGPT’s browsing activities remain separate from the rest of the infrastructure.
To respect content creators and adhere to web norms, the browser plugin’s user-agent token is ChatGPT-User, configured to honor websites’ robots.txt files.
OpenAI has also published its IP egress ranges and implemented rate-limiting measures to prevent excessive traffic to websites.
Introducing the Code Interpreter Plugin for ChatGPT
OpenAI has developed an experimental code interpreter plugin for ChatGPT that allows the model to use Python, handle file uploads and downloads, and run code in a sandboxed and firewalled execution environment.
This gives users a powerful tool to solve mathematical problems, perform data analysis and visualization, convert files between formats, and more, all within a chat conversation.
The interpreter plugin evaluates code in a persistent session that is active for the duration of the chat conversation (with an upper-bound timeout). It supports uploading files to the current conversation workspace and downloading the results.
The plugin enables completely new workflows, making tasks more effortless and efficient and opening up the benefits of programming to new audiences.
In the example provided, ChatGPT uses the code interpreter to solve a system of equations to determine the weight of a brick given information about the weight of bricks and GPUs in relation to each other and the total weight of a set of bricks and GPUs.
Are we witnessing a superpower?
We’re living in an unprecedented timeline, one that is going to affect a large number of industries by rendering certain jobs and tasks “worthless”.
In fact, the societal implications of plugins are far-reaching. A recent working paper by OpenAI suggests that language models with access to tools will have a much greater economic impact than those without.
In line with other researchers’ findings, OpenAI expects the current wave of AI technologies to affect job transformation, displacement, and creation significantly. OpenAI is eager to collaborate with external researchers and customers to study these impacts and navigate the future of human-AI interaction with a focus on safety and societal benefits.
