in this issue
|
CASE STUDY
Making Answers Easier to Find for Millions of Users |
Cisco tackled one of its biggest customer pain points by using AI to improve findability, showing that the best AI initiatives start with customer problems, not technology.
by Jyotsna Khandekar, Director of Product Management at Cisco
Every AI initiative promises transformation. The ones that succeed usually begin somewhere much less glamorous: a persistent customer frustration.
For Cisco, that frustration was findability. Millions of users relied on Cisco Community to troubleshoot issues, but long discussion threads and inconsistent post titles made it difficult to find answers quickly.
This case study explores how Cisco resisted the urge to implement AI for AI’s sake. Instead, the team started with a friction point users had been raising for quite some time, then designed, validated, and deployed an AI solution that prioritized trust, accuracy, and measurable customer value from day one.
Cisco Community is one of Cisco’s most heavily used destinations, visited by millions of users every year. But for a long time, the biggest pain point our customers reported was findability — simply getting to a relevant answer quickly.
The problem was structural. When customers post questions and answers, titles are inconsistent and often uninformative, and popular threads can run to 10 or more replies. Someone in the middle of troubleshooting a live network issue doesn’t have time to read an entire thread just to work out whether it’s even relevant to them. They need a fast, relevant answer.
This wasn’t a single complaint or a spike in one quarter. It came through clearly and consistently in the ways we listen to customers: our surveys, a feedback form on Cisco Community itself, and one-on-one customer interviews at events like Cisco Live, alongside the overall customer satisfaction signal on the site. Findability was, time and again, the friction point customers cited. It was a widespread, well-understood problem across a user base in the millions.
Importantly, this became an AI project because of the problem, not the technology. We didn’t set out to “add AI” to the Community. We already knew findability was one of our biggest pain points, and we recognized that AI could finally solve it quickly and at scale. Problem first, technology second.
| “ | This became an AI project because of the problem, not the technology. We didn’t set out to ‘add AI’ to the Community. We already knew findability was one of our biggest pain points. | ” |
Rather than dropping a generic summary paragraph on top of every thread, we designed a structured summary built for fast scanning. Each summary is organized into clear sections: what the issue is (including the product and any error message), whether there’s an accepted answer, any recommendation from experts or VIP community members, and a final section for other important information.
The intent is that, within a few seconds, a customer can understand the issue, see whether there’s a solution, decide whether the thread is relevant, and scroll down to read the full discussion only if they want more. We also added a deliberately minimal thumbs-up/thumbs-down control directly in the interface to gauge whether the summary was helping.
Trust shaped the design from the start. Because our team sits in operations and manages Community as a platform, we are not the in-depth experts on Cisco’s products, their error messages, or their solutions, so we couldn’t fully judge on our own whether an AI-generated summary was accurate. That drove a deliberate decision: bring in the people who could. We were especially careful because a customer arriving at Community is already dealing with a problem; an inaccurate or misleading summary would only add to their frustration, so guarding against hallucination and inaccuracy was central.
We also built in economic discipline. We also built in economic discipline. For threads that are already trivially simple — for example, a straightforward question with a single, obvious accepted answer — we deliberately don’t generate a summary, because it adds no value and isn’t worth the infrastructure cost. Our original thinking had been to summarize every post; ruling that out for low-value threads was one of the clearest tradeoffs we made. Our original thinking had been to summarize every post; ruling that out for low-value threads was one of the clearest tradeoffs we made.
From “let’s try this” to the first board going live took a little over a quarter, deliberately paced, because we validated before we scaled.
| “ | Autonomy without boundaries isn’t innovation. It’s risk. | ” |
We started by identifying the boards for the pilot phase, then reached out to product leaders, who nominated specific testers from their domains. We ran a kickoff with those testers, and each product leader reviewed roughly 20–30 posts in their area, comparing the output and flagging any discrepancies in the summaries. To keep the evaluation rigorous, we tested two different implementations of the summary output — using different techniques and prompts — and asked the testers which produced better results.
That expert review directly improved the product. Two clear examples: when an accepted answer linked to an official Cisco document, our early summaries didn’t surface that link, and testers told us that having the official documentation accessible right from the summary really matters to customers. Similarly, the summaries were sometimes omitting the specific software version name, which is often critical context for a networking issue. In both cases, we tweaked the feature to capture that detail. Testers also flagged instances of inaccurate information, which we addressed before expanding.
| “ | The goal isn’t to remove people as fast as possible. It’s to know exactly where human judgment matters most, and design for it.” |
A practical lesson came from the process itself. Our validators were senior people doing this on top of their day jobs, so we were careful not to waste their time. We hit some friction early. Product leaders don’t normally have access to the testing environment, which is owned by IT operations, and we had to work through access and logins before testing could really begin. Rather than bombard testers with emails, we ran one-on-one working sessions to show them how to test quickly. That kept them engaged and made the whole process faster for everyone.
Underneath all of it was a simple principle: the goal isn’t to remove people as fast as possible; it’s to know exactly where human judgment matters most, and design for it. On this build, that meant putting product experts precisely where their expertise was irreplaceable.
It’s deliberately early to make big claims. We won’t say overall CSAT has improved yet — that needs at least a quarter of data before we’d responsibly state it, and we’re not going to over-claim ahead of the evidence.
What we can point to is the early signal, and it’s encouraging: on the boards where the summaries are live, we haven’t seen a single negative comment, thumbs-down, or negative sentiment so far. The feedback has been consistently positive — spontaneous reactions like “nice summary” and “good summary,” along with thumbs-up on the interface. These are unprompted, in-the-moment responses, which are exactly the kind of signal we watch for before expanding.
| “ | Build trust in from the start. Don’t bolt it on at the end. | ” |
The thesis behind the feature is straightforward: act on what customers have told us and remove the friction they’ve described in our surveys and interviews — reducing time-to-answer and directly addressing the findability pain point. That’s what we’re building toward, and it’s how we’ll judge success. As the rollout matures, we’re targeting measurable outcomes like CSAT and time-to-answer.
Availability is expanding to more discussion boards, guided by the same two things that have guided us throughout: a validation-first approach and the feedback signal. We expand as we confirm accuracy and see positive sentiment, while gathering feedback and testing the remaining boards in parallel with broader availability planned in the coming months.
From here, the direction is toward more relevant, helpful experiences that directly address customer pain points. Now that we have AI, the goal is to solve more and more real customer problems with it, but the discipline stays the same. We’re not embedding AI for its own sake; we look for where it’s truly needed and genuinely improves the experience, rather than adding it because we can.
| “ | Customer first. Technology second. Innovation follows. | ” |
Community is a good example of where that balance matters. Even as we add AI, we’re intentional about preserving what makes Community valuable in the first place: the human-to-human interaction. Customers come to connect with peers who’ve solved the same problem, and they look forward to that. Our aim is to use AI to reduce friction and help people find answers faster, while protecting that sense of community and human connection rather than replacing it.
Two lessons stand out. First, lead with the customer problem, not the technology. Start with customer value, a real pain point you understand deeply, then ask how AI can solve it, and solve it faster. Customer first, technology second, and innovation follows. On this project, we didn’t set out to “use AI”; we started with a friction point we’d heard about for quite some time and recognized AI could finally solve it.
Second, build trust from the start — don’t bolt it on at the end. How much rigor you need depends on what you’re building, who it’s for, and at what scale. When you’re launching something to millions of users who are already dealing with a problem, you cannot afford to lose their trust in the process. That’s why we validated accuracy with the right experts, guarded against hallucination, and rolled out gradually rather than all at once. Autonomy without boundaries isn’t innovation; it’s risk. Explainability, validation, and human oversight aren’t overhead; they’re what let you move fast without breaking the very trust you’re trying to earn.
|
About the Author
Jyotsna Khandekar is a Director of Product Management at Cisco, where she leads customer-facing digital platforms and AI initiatives that serve millions of users at a global enterprise scale. Her portfolio spans Cisco Community — where customers turn to peers to solve problems — and the digital infrastructure that delivers the right insight to the right customer at the right moment, driving adoption and helping customers realize the full value of their investment. With 17 years in technology — roughly a decade in product management — Jyotsna brings an unusually horizontal view of how a business actually works. She has built products for internal sellers in the go-to-market domain, for partners in the commerce domain, and now in customer experience — giving her a rare end-to-end perspective on the full customer and revenue lifecycle. Her foundation was built at Tata Consultancy Services, where large-scale delivery sharpened her instincts for building systems that hold up in the real world. Beyond her product portfolio, Jyotsna leads her organization’s transformation to AI-native product management — moving an entire practice from manual to AI-assisted to AI-native ways of working. She was also a judge for the 2026 Globee Awards for Technology. She holds a Bachelor of Engineering from VJTI, Mumbai University, and completed product management studies at Berkeley. At the heart of it all is a conviction she has carried since writing her first lines of code: stay curious, keep learning, and keep the customer at the center of every decision. Interested in taking part in a future Case Study feature and sharing your story? Visit our Get Involved page. |
