4 AI predictions—and what they mean for product managers
With the rapid advances in AI technology, what’s coming next for us as PMs?
Glenn Jaume
Product Manager at Coda
4 AI predictions—and what they mean for product managers
By Glenn Jaume
Share this article
As a kid, one of my hobbies was “going on the internet”—a novelty at the time. Fast forward 25+ years and we are, of course, all “on” the internet all the time—it’s now an integral part of our lives. I believe we’re in the midst of something equally transformational with the rapid advances in artificial intelligence (AI). Like many, I found the release of AI models like ChatGPT and DALL·E 2 awe-inspiring. The rapid advancement and expanding capabilities of AI over the last two years—from media generation to code creation—have continued to inspire. Every time I think we’ve hit a peak, a new update comes out that’s ten times better than what came before it. These advancements bring a mix of both excitement and caution. But this isn’t a post about the dangers of AI or how it’s coming for your job (there are plenty of those already). Instead, this post is about how we, as product managers (PMs), can use AI to help us in our roles and how it might impact the products we build. Let’s first take a look at the advancements we’ve seen so far, and then I’ll dive into four predictions I have for the next year and what they’ll mean for us.
A quick look back at how far we’ve come.
Given the recent explosion of AI products, it’s easy to forget that just a couple of years ago, the idea that your non-techy friends and family members would be regularly talking about—and potentially using—AI wasn’t even on the radar. As different AI models and versions have been released, I’ve been collecting examples that show just how far we’ve come already. You can see the comparisons I’ve collected across all different media types—including video, voice, and images—here, but here are three of my favorites:1. Writing an Oscars speech.
Back in 2022, I asked Open AI’s GPT-3.5 to “write a Jimmy Kimmel opening monologue about the Oscars.” In return, it gave me a stilted and generic speech with lazy and obvious references (Kimmel would be ashamed!). Fast forward to now, and Google’s Gemini 1.0 Ultra generates a speech that reads much more like Kimmel, with topical references and jokes that genuinely made me laugh. You can read both versions in full here.
2. Designing a movie poster.
In 2022, a Reddit user asked recently-released DALL·E 2—Open AI’s image generator—to create a very meta “DALLE-2: Revenge of the AI, Pixar Studios movie poster.” While at first glance the poster looks passable, when you start to look closer you’ll notice the odd text, the many artifacts (i.e., ”noise” or irregularities in the image),and that it doesn’t really look like Pixar’s style. I tried out the same prompt with DALL·E 3 (released in August 2023) and the result is much more accurate: believable imagery in the right style and a tagline that actually makes sense.
3. Creating new music.
In May 2023, Google released the first version of MusicFX, a music generator powered by their MusicLM. I asked it to make me “a catchy pop song about Coda AI.” What it returned was indeed catchy but also a fairly generic song without any lyrics. Compare that to the result from another, newer model, Suno: a Europop banger I can imagine dancing to in a club at 2am (before promptly realizing it’s about software). You can listen to both versions here.
Generated by Suno (Mar 2024)
AI advancements extend to taking action, too.
In addition to rapidly improving content generation, many models have gone “multimodal”—meaning they can take inputs like images and output text or code—and are much more capable of taking actions, too. I believe this is where we’ll see the real value of AI. There are many examples of script-based tasks that AI models are able to perform without dedicated UI or requiring you to learn new software. For example, I’ve used ChatGPT to find the volume of an MP3 sound effect I’m using in my tropical home bar, and then adjust other MP3s to the same volume—all without leaving the chat interface (full chat here). I’ve also used it for tasks like modifying 3D printer files and cropping a video to then convert it into a gif.
MP3 volume adjustments generated by ChatGPT (Feb 2024)
My 4 AI predictions—and what they mean for product managers.
Clearly, AI is developing rapidly and we’ll see many more advancements in the coming years. But what does this mean for us as PMs and the products we’ve built? Here are my four AI predictions for the next year and what opportunities they present for us.Prediction 1: Significantly cheaper models will enable new use cases.
In November 2022 GPT-3(the pre-cursor to ChatGPT, aka GPT-3.5)cost $0.02 per 1,000 tokens. You can think of a token as roughly equivalent to the number of words in both the input and output; the longer the question and response, the more tokens it uses. Today, GPT-3.5 costs $0.0005 per 1,000 tokens—a 97.5% price cut in just over a year. And GPT-4 Turbo—considered the world’s most advanced AI model—is only $0.01 per 1,000 tokens. In other words, the models are getting more capable and cheaper at the same time. This has mostly been driven by OpenAI’s aggressive pricing strategy and pressure from open-source models that can be self-hosted and, therefore, cheaper for some use cases. This, plus the inevitable advances in chip technology to run these models, leads me to confidently predict that by the end of the year, we’ll have models comparable to GPT-4 at significantly lower prices and models at the quality of GPT-3.5 that are virtually free to use.What this means for PMs:
We’ve already seen how quickly companies jumped to building new AI products and adding AI capabilities to existing ones. With this change—plus the likelihood of models getting faster, too—development of AI features will only accelerate and more compelling use cases will emerge. Without cost being an issue, it’ll be possible to use AI in many more features and even have it running persistently in our products. For example, for us at Coda, it could mean having Coda’s AI features continuously reading your doc and suggesting next sentences while you write or using AI to categorize much bigger data sets with thousands of rows. Think about what opportunities would exist in your product if you had access to cheaper AI models and start planning for them, because it’s coming!Prediction 2: “RAG” will become the fastest-growing product category.
RAG (retrieval augmented generation) is a technique that directs the AI to retrieve specific sets of data first—internal company docs, for example—and then ask it to look at those sources for the answer to the user’s question. This gives the AI more context and means it can use your specific data, which leads to more accurate and relevant responses. Coda’s AI features already uses RAG to deliver answers about the content in your Coda docs, so you can ask it questions like, “what’s our expenses policy?” and get an accurate answer. I predict that we’ll soon see an explosion in AI features using RAG to give more useful answers using your data, not just data found on the general internet.
What this means for PMs:
Expect to see many new AI tools and features that can understand your context—i.e., your product specs, your team’s progress, company knowledge, and so on—to give you more accurate and relevant answers. This will make them much more useful for you and your team in your day-to-day work. Try out Coda AI for an example. If you’re building AI into your own products, RAG will allow you to create much more personalized AI features without having to train your own models. Instead, you’ll simply be able to direct your AI where to look for the information and give your customers a smarter, more contextually aware AI experience.Prediction 3: Context windows will stop mattering (when time and money are no object).
Currently, most LLMs have token limits that constrain the length of the question or prompt you give to an AI model. But Google’s latest model, Gemini 1.5, supports up to a million tokens, meaning the input can be far longer than ever before. For context, that’s enough tokens to stuff every Harry Potter book into a single prompt or to analyze every frame of a 44-minute video. More importantly, Google promises incredible recall accuracy even with a prompt this long, as shown here. Most models find it harder to find an accurate answer the larger the data volume and the further it is from the start. For example, it’s easier to retrieve a fact from page one of a book than it is from page 500. Google’s testing showed Gemini 1.5 only missed three facts across 1M tokens. Compare this to GPT-4, which missed many more facts across only 125k tokens.
Gemini’s recall accuracy within different context windows
What this means for PMs:
Similarly to the advancements in RAG, these improvements will broaden the viable use cases for AI and provide much more accurate answers—whether in our own work or the products we build. For example, you could build an AI feature that lets customers find answers within a how-to video or webinar recording, without having to watch the whole thing—something that would be much too slow and inaccurate today. Larger context windows also mean we can ask the AI model to retrieve more documents or context for a question, which increases the chance that it can find the right answer and massively improves the quality of the results. Imagine how much time you’d be able to save if you were able to ask an AI to find insights from a huge dataset of customer feedback or survey results. With larger context windows, this will be possible. In the near term, the main shortcomings will be speed (long context windows mean slower answers) and cost—although for high-value use cases, this likely won’t be a blocker.Prediction 4: Action will surpass content generation.
In mid-2023, OpenAI added “function calling” to their models, which gives developers more control over the output the model produces, so that it can be used to generate code or connect to external tools and APIs. For example, an AI response in the right JSON format can be used to display something in a widget. There are many rumors that OpenAI’s next model, GPT-5, will be heavily action-orientated and we already know that OpenAI is working on agents that can take action across apps. It seems likely that GPT-5 will be a huge leap forward in all areas but mostly in its ability to take sophisticated actions with complex reasoning. We’re also seeing companies develop models and devices that are trained to take actions, such as rabbit r1. These are the first steps in having AI models go beyond finding information and generating media, to actually performing tasks for us automatically. This opens up possibilities like AI being able to code applications to a user’s needs without them having to actually understand the code. Or AI models performing actions by using and combining APIs without needing to build an integration—like a chat-based Zapier but with more flexibility.What this means for PMs:
I predict the existing tech is close, and soon, AI “agents” will reach a quality level that takes them from gimmicks to actually being useful for complex work tasks. Companies like Retool and Equals are already doing interesting work here, and I expect many more will follow soon.For PMs, this shift means two things:- There will be huge opportunities to use AI in our products to reduce repetitive actions for our customers and make certain use cases much easier. For example, AI could run a relevant action in an integration without a user needing to go and set it up, or it could modify content or create formulas via a chat interface. This has great benefits in making products more accessible, delivering a better user experience, and generating greater adoption and retention.
- These opportunities to automate tasks will extend to our own work as PMs, too. It’s easy to imagine that we’ll soon have access to products that can take some of the “busywork” off our plates, like turning meeting notes into Jira tasks for your team or converting feature ideas into a visual roadmap.
AI will make us, and our products, work smarter.
There are already many opportunities to use AI in our day-to-day work and within the products we build for our customers. I’m excited to see what creative new products and capabilities emerge as the AI models further advance. We’re certainly excited to continue with our work on Coda AI, and welcome your ideas and feedback. You can read more about our approach to AI here, or check out our Ultimate Handbook for Product Teams for more about how PMs are using Coda for a more efficient, effective workday.Related posts
Explore more resources for product teams.