Zoom AI Studio - Ai Agents
Zoom’s Virtual Agent offering was an effective chatbot, but lacked advanced modern AI capabilities and was difficult to self service and maintain for customers. Also, there was no voice capabilities for customers wanting to implement AI voice support.
Overview
Zoom currently offers chatbots to customers who want to add a self servicing tool for customer support, sales, HR needs, and more - which is called Zoom Virtual Agent. There is a growing need to build more types of agents to be used in multiple modalities. As the industry has adopted technologies that utilize large language models (LLM), there is an increasing need to adapt from supervised model-powered bots to a more modern solution.
The Problem
The existing chatbot offered by Zoom was an effective solution, but it was time consuming to set up and not as 'intelligent’ as more modern LLM-powered chat solutions that exist today. There was growing interest from customers that they wanted a bot that was more intelligent and could be more conversational than the existing bot - which was powered by supervised AI models.
There also was no support for customers wanting an intelligent voice AI agent to help their customers with various tasks when they call in. This was a rapidly requested functionality which left customers and potential customers wanting more.
In order to gain a deeper understanding, I needed to learn more about the opportunity space.
Discovery Research
During the discovery period, I primarily focused on learning from our customers about some of the thoughts they have about our product. In addition, while we were talking to customers, I started looking at the competitive landscape to see what others in the industry are doing to alleviate these issues. Initially, I met with our Competitive Team to understand their expertise about who the best are in the field in this area, and then expanded into my own research to identify the optimal experiences available.
Discovery Findings
Findings from the discovery research yielded some common themes.
Existing customers as well as potential customers were unhappy with the amount of effort it took to initially set up, and then maintain bots.
Once successfully implemented, they were not always happy with how the bot responded. As the industry has gotten comfortable with using LLM-based products such as ChatGPT, Microsoft Copilot, etc. , they expected a more conversational experience, rather than a traditional intent based chatbot - which lacks nuances of the human conversational experience.
Customers were more and more requesting a voice bot to help with when customers would call in for support or other reasons. We determined that our existing bot model using supervised models would not be an ideal experience for voice - as speech conversations have a lot more nuance, which requires a more powerful AI model.
After aligning with the triad on some of these opportunity spaces, we decided it was time for us to upgrade our bots to AI Agents - which utilize both Large Language Models, as well as Small Language Models to assist our customers - which would alleviate the above issues. Updgrading to AI Agents allows the following benefits:
Enhanced conversational experience - the ability to jump seamlessly between topics without getting lost.
Ease of set up and maintenance - using a low/no code approach, our customers could explain how the agent should function using plain speech.
Easily scalable to other channels - building this for chat, makes it simple to apply to voice and other modalities in the future.
Design Solution
After presenting our findings to the executive team, as well as the rest of the company we were able to start thinking about how this could be done. We needed to:
Allow customers the ability to build this new agent quickly and intuitively.
Existing customers needed a seamless experience to try this new bot type out while still remaining all existing functionality.
To help understand this new way of setting up agents, we wanted to create a guided setup experience.
Ability to set up a variety of agents - ranging from customer service chat and voice to agents used in other interfaces, as well as within Zoom.
Design Explorations
Design Validation
Once we built a couple low fidelity options which solved the above issues, we worked with our Professional Services team, as well as a few existing customers which requested this functionality to make sure we were heading the right direction. I created a few click-through prototypes to do a walkthrough so we can make sure we are building a cohesive product that won’t leave more questions. Overall there was a strong signal in one direction - which most of the team was aligned on.
Success Metrics
How did I know I was successful or not once the project was released?
Our initial goal was that we wanted customers to be able to set up and deploy a chat or voice agent in 30 minutes or less.
We also tracked how many customers were switching from the traditional bot to the new AI agents in the first 30 days - wanting 50% adoption in the first month.
In addition to this, we wanted to conduct some qualitative interviews after release to hear how they were experiencing the new product, as well as feedback for future iterations.
Results
We were able to build the new agents and the movement was met with a lot of enthusiasm. In addition, building an agent took a fraction of the time it took to build the traditional bot, which aligned with our success metrics.
It is worth noting that this is a huge change for customers, and there was a lot of feedback for the next release to update to add additional functionality, as well as a fair amount of tweaks needed on the AI models to get the agent responding in the expected manor.
Challenges
This endeavor had a lot of challenges which we had to overcome. We initially had a lot of issues with hallucination - which we were able to mostly fix by enhancing AI guardrails, to make sure the AI won’t make up content. Also, we found that users’ learning curve to create skills was very high, so we are now working on ways to simplify skill creation, and educate on best prompting practices.
