For decades, the way we find information on the internet changed only in small ways. Doing a traditional Google search today doesn’t feel all that different from when, in the 1990s, you would Ask Jeeves. Sure, a lot has changed under the hood, the results are likely far more relevant and the interface has some new features, but you’re still typing in keywords and getting a list of websites that might hold the answer.
That way of searching, it seems, is starting to go the way of AltaVista, may it rest in peace.
In May, Google announced the rollout of its new AI Mode for search, which uses a generative AI model (based on the company’s Gemini large language model) to give you conversational answers that feel a lot more like having a chat and less like combing through a set of links. Other companies, like Perplexity and OpenAI, have also deployed search tools based on gen AI. These tools, which merge the functionality of a chatbot and a traditional search engine, are quickly gaining steam.
You can’t even escape AI by doing just a regular Google search: AI Overviews have been popping up atop those results pages since last year, and about one in five searches are now showing this kind of summary, according to a Pew Research Center report. I’m surprised it’s not even more than that.
These newfangled search tools feel a lot like your typical chatbot, like ChatGPT, but they do things a little differently. Those differences share a lot of DNA with their search engine ancestors. Here’s a look under the hood at how these new tools work, and how you can use them effectively.
Search engines vs. AI search: What’s the difference?
The underlying technology of a search engine is kinda like an old library card catalog. The engine uses bots to crawl the vast expanses of the internet to find, analyze and index the endless number of web pages. Then, when you do a search to ask who played Dr. Angela Hicks on ER, because you’re trying to remember what else you’ve seen her in, it will return pages for things like the cast of ER or the biography of the actor, CCH Pounder. From there, you can click through those pages, whether they’re on Wikipedia or IMDB or somewhere else, and learn that you know CCH Pounder from her Emmy-winning guest appearance on an episode of The X-Files.
“When customers have a certain question, they can type that question into Google and then Google runs their ranking algorithms to find what content is the best for a particular query,” Eugene Levin, president of the marketing and SEO tool company Semrush, told me.
Generally, with a traditional search, you have to click through to other websites to get the answer you’re looking for. When I was trying to figure out where I recognized CCH Pounder from, I clicked on at least half a dozen different sites to track it down. That included using Google’s video search — which combs an index of videos across different hosting platforms — to find clips of her appearance on The X-Files.
Google announced AI Mode at its I/O developer conference in May.
These multiple searches don’t necessarily have to happen. If I just want to know the cast of ER, I can type in “cast of ER” and click on the Wikipedia page at the top.
You’ll usually find Wikipedia or another relevant, trustworthy site at or near the top of a search result page. That’s because a main way today’s search algorithms work is by tracking which sites and pages get most links from elsewhere on the web. That model, which “changed the game for search” when Google launched it in the 1990s, was more reliable than indexing systems that relied on things like just how many times a keyword appeared on a page, said Sauvik Das, associate professor at Carnegie Mellon University’s Human-Computer Interaction Institute.
“There’s lots of cookie recipes on the web, but how do you know which ones to show first?” Das said. “Well, if a bunch of other websites are linking to this website for the keywords of ‘cookie recipe,’ that’s pretty difficult to game.”
AI-powered search engines work a little differently, but operate on the same basic infrastructure. In my quest to see where I recognized CCH Pounder from, I asked Google’s AI Mode, literally, “Where do I recognize the actress who plays Dr. Angie Hicks on ER from?” In a conversation that felt far more like chatting with a bot than doing searches, I narrowed it down. The first result gave me a list of shows and movies I hadn’t seen, so I asked for a broader list, which featured her guest appearances on other shows. Then I could ask for more details about her X-Files appearance, and that narrowed it down.
While the way I interacted with Google was different, the search mechanisms were basically the same. AI Mode just used its Gemini model to develop and process dozens of different web searches to gather the information needed, Robby Stein, vice president of product for Google Search, told me. “A user could’ve just queried each of those queries themselves.”
Basically, AI Mode did the same thing I did, just a lot faster.
So many searches, so little time
The approach here is called “query fan-out.” The AI model takes your request and breaks it down into a series of questions, then conducts searches to answer those components of the request. It then takes the information it gathers from all those searches and websites and puts it together in an answer for you. In a heartbeat.
Those searches are using the same index that a traditional search would. “They work on the same foundation,” Levin said. “What changes is how they pull information from this foundation.”
This fan-out process allows the AI search to pull in relevant information from sites that might not have appeared on the first page of traditional search results, or to pull a paragraph of good information from a page that has a lot more irrelevant information. Instead of you going down a rabbit hole to find one tiny piece of the answer you want, the AI goes down a wide range of rabbit holes in a few seconds.
“They will anticipate, if you’re looking for this, what is the next thing you might be interested in?” Levin said.
Read more: AI Essentials: 29 Ways You Can Make Gen AI Work for You, According to Our Experts
The number of searches the AI model will do depends on the tool you’re using and on how complicated your question is. AI Mode that uses Google’s Deep Search will spend more time and conduct more searches, Stein said. “Increasingly, if you ask a really hard question, it will use our most powerful models to reply,” Stein said.
The large language models that power these search engines also have their existing training data to pull from or use to guide their searches. While a lot of the information is coming from the up-to-date content it finds by searching the web, some may come from that training data, which could include reams of information ranging from websites like this one to whole libraries of books. That training data is so extensive that lawsuits over whether AI companies actually had the right to use that information are quickly multiplying. (Disclosure: Ziff Davis, CNET’s parent company, in April filed a lawsuit against OpenAI, alleging it infringed Ziff Davis copyrights in training and operating its AI systems.)
AI search isn’t just a chatbot
Not relying on training data is one thing that sets an AI-powered search engine apart from a traditional chatbot, even though the underlying language model might be largely the same. While ChatGPT Search will scour the internet for relevant sites and answers, regular ChatGPT might rely on its own training data to answer your question.
“The right answer might be in there,” Das said. “It might also hallucinate a likely answer that isn’t anywhere in the pre-training data.”
The AI search uses a concept called retrieval-augmented generation to incorporate what it finds on the internet into its answer. It collects information from a source you point it to (in this case, the search engine index) and tells it to look there instead of making something up if it can’t find it in its training data. “You’re telling the AI the answer is here, I just want you to find where,” Das said. “You get the top 10 Google results, and you’re telling the AI the answer is probably in here.”
Perplexity offers AI-powered search through its app and through a newly announced browser.
Can you really trust AI search results?
These AI-powered search tools might be more reliable than just using a chatbot itself, because they’re pulling from current, relevant information and giving you links, but you still have to think critically about it. Here are some tips from the experts:
Bring your human skepticism
Consider how bad people are at telling when you’re sarcastic on the internet. Then think about how bad a large language model might be at it. That’s how Google’s AI Overviews came up with the idea to put glue on pizza — by pulling information from a humorous Reddit post and repeating it as if it were real culinary advice. “The AI doesn’t know what is authentic and what is humorous,” Das said. “It’s going to treat all that information the same.”
Remember to use your own judgement and look for the sources of the information. They might not be as accurate as the LLM thinks, and you don’t want to make important life decisions based on somebody’s joke on an internet forum that a robot thought was real.
AI can still make stuff up
Even though they’re supposed to be pulling from search results, these tools can still make things up in the absence of good information. That’s how AI Overviews started creating fake definitions for nonsensical sayings.
The retrieval-augmented generation might reduce the risk of outright hallucinations but doesn’t eliminate it, according to Das. Remember that an LLM doesn’t have a sense of what the right answer to a question is. “It’s just predicting what is the next English word that would come after this previous stream of other English words or other language words,” Das said. “It doesn’t really have a concept of truthiness in that sense.”
Check your sources
Traditional search engines are very hands-off. They will give you a list of websites that appear relevant to your search and let you decide whether you want to trust them. Because an AI search is consolidating and rewriting that information itself, it may not be obvious when it’s using an untrustworthy source.
“Those systems are not going to be entirely error-free, but I think the challenge is that over time you will lose an ability to catch them,” Levin said. “They will be very convincing and you will not know how to really go and verify, or you will think you don’t need to go and verify.”
But you can check every source. But that’s exactly the kind of work you were probably hoping to avoid using this new system that’s designed to save you time and effort.
“The problem is if you’re going to do this analysis for every query you perform in ChatGPT, what is the purpose of ChatGPT?” Levin said.