Overview Panel

Transcript:

All right. Good morning, everybody. Thanks for it's already ten o'clock. so, we're going to make up some time. But this panel is important because it's kind of frame up conversation for the rest of the day. so, don't let everyone introduce themselves. But we have a really good representation of ecosystem and a lot of folks on the analytics side that are have a God's eye view of what Bots are doing and what's happening in the ecosystem. so, just apologize start with you. Alright low, my name is Joseph Holguin. I'm the co-founder of smart Loop and Loop is a I chopped up a chatbot platform for marketing and customer service. Hi, my name is Ashe, I lead the chatbot practice for Ernst & Young. It's a big for consulting firm Professional Services. We help businesses Implement chatbots on there and using the products that are available in the market. Hi everyone. I'm Milker. I'm the CEO of pause analytics is the conversational analytics platform for conversational interfaces and physically like that. Hi, my name is Jesse Hall. I'm CTO of - bah, we do analytics for conversational interfaces as well. Our view is that building these interfaces is really hard. so, you need really great tools to help you continuously optimize and improve these interfaces. Well, thanks gentlemen to this is question. Everybody gives us perspective on the ecosystem. Is a growing and al so, let's look at some use cases for chatbots an AI that are winning today. Yeah. so, question is that ecosystem growing. I think it is growing when we first kind got into here into this industry. Probably about three years ago. There was a lot of hype around Bots can replace everything like I was going to get in an Uber and was going to just use a bot. We're really not seeing that but the good thing is that now we're singing like a very good use cases emerging and businesses seen really good results around. From my perspective customer service marketing and sales. so, I think that's where we're seeing the industry kind of solidify around those use cases, which is good because businesses are seeing positive Returns on those. Absolutely, the I think I don't see anybody saying the ecosystem is not growing. I think the in last couple of years. There are three ways that we have seen how ecosystem is going first is like people were doing Q's fa through the chat Bots. Now people are moving onto towards doing more transactions and smarter transactions. so, that's one area. The second area is largely the chat Bots were always focused towards the Stammers red end-users providing themselves service capability now people are thinking about how can I increase my efficiency accuracy consistency of my employees like contact center agents, back office operations, their job is lot tougher. So, that is the area where it is improving and the third piece is the business drivers, you know, like initially it was all about customer experience, now people are thinking about okay customer experience is one of the areas, how can I grow my revenue or reduce my risk towards my business. Like there are the drivers are also, expanding. so, absolutely the ecosystem is expanding. No doubt. Yeah, iding is growing. Well, if you look at the last three years, I think for the first two years companies failed a lot on the Enterprise level because they couldn't have put the borders of the project in a very well. The scope is really important while building a virtual agent. Specific cases and they failed a lot but I think for the last couple years is going great. They failed a lot and tools are like analytics tools design tools building virtual agent tools are emerging and there. Locating the people in the company's in all levels and then they have started to put our great scope for the projects and then it's getting good results. I have seen a lot of great results on the banking side. Yeah. Yeah, I mean I think customer care is still King, you know, there are large Enterprises investing significant resources, and they're really starting to see Returns on those Investments. so, customer satisfactions going up user retention goes up. So, you know, these interfaces are working very well in the customer care case. There's lots of other cases that are really appearing sales generation lead generation is a really growing business on The Voice side. Gaming continues to really gain speed and moment. There's been a lot of growth there. They're even starting to see some reasonable revenues on The Voice side, although still quite far behind mobile, but you know, we had a spot of processed over 75 billion messages. so, clearly this space is hot and growing interest interesting to see kind of gaming into entering the ecosystem. It's very much like the app economy that we saw growth over the last 20 years. so, looks on the Enterprise side, you know, let's see how our business is using chat today and then maybe pick one use case that you've identified that hey, this is actually this is really awesome on the Enterprise side for a use case. So, on the Enterprise side again from our company, we focus a lot on Marketing sales and customer service, but from the customer service side, we're seeing a lot of Enterprises being able to fully identify a large percentage of their customer inquiries and then build Bots around those. so, they're able to reduce the amount of conversations that a live agent has to do, but they don't remove the live agent aspect. We have to remember that Bots really can't solve all problems right now. They've been able to identify the top inquiries that their customer service reps are already facing buildbot surround that to automate that and then if the bot fails or they need to speak with an agent, then they're able to escalate to an agent. Yeah. so, the other talk about a use case where people are using chatbots internally for employees. I've better solution, which actually kick started my practice we call it as ey procedure board. But it is a use case for the back-office operations people like who process insurance claims as an example, their job is extremely difficult. It's very procedural. so, we thought we can use the analogy of how Google Maps change the way we get directions. We changed how people use desktop procedures today to do the job to kind of chat bot telling them what to do and al so, automate certain transactions as they are doing it and that use case is saving millions of dollars for don't chill services company and I think that is something that we are really proud of. I can give an example on the banking side. I'm using it my daily routine for that virtual agent companies called a spank on the banking side and their virtual agent is called Maxi they have put the virtual agent on their website and mobile phone and you can order why a voice or on the website. You can order a lot of things and they have built this agent with many third parties like new alskling Seewald and I'm using in my daily routine and their main purpose was to reduce the customer support on their rent and they reduce 5% and all over the world. so, I think this is good. I use it a lot and many of my friends use it a lot and it's just starting point lighting and next couple years. I think we will. See the numbers like 20% reduced to all support loads. Yeah, I mean supporting our one of our one of our big customers into it, you know, as has invested heavily in these conversational interfaces, you know, I can't go into detail numbers, but they have a team of you know, over 20 developers working on the conversational interface and they've seen significant Returns on that investment. so, you know, they have this large team working on it and one of the things you really learn building these conversational interfaces is a they're very hard to build but be customers give you. Get this constant stream of really great data you get this constant stream of feedback to your interface working because customers tell you right customers. They type it out in the message know your boat's not understanding me. You're not listening to me. Right? And so, if you have the right tools to extract these things and really figure out okay, where is my conversational interface not working? And where is it working? And if you have tools that can do that at scale your goal should be when building a conversational interface launch some minimal viable product and then constantly iterate it constantly make it understand more things make it handle more use cases and by constantly iterating your chap on using great tools to find out where your body is not working can make these things much better and Intuit seeing that happening. It's got verticals and in Enterprise ecosystem is it you know, what's being used the most is entertainment Hospitality travel what verticals you think that are leveraging or is it Banking and leveraging Bots today? So, on our platform, I think we're seeing a lot of just engine. I guess we can't really generalize the customer service because it's anybody that business our user is coming to your business. They're asking questions. so, customer service is just probably across the board when it comes to marketing and lead generation. We're seeing a lot of Professional Services where you have to were maybe you can target somebody specifically on Facebook you engage with them educate them and then you capture their lead. so, that's a lot of Professional Services for the marketing / Lee generation site. Yeah, think you're thinking correctly said it like it's more about functional areas within a company than a vertical across sectors. so, I cover all the sectors but predominantly I'm seeing more traction in the power and utility energy sector Banking and insurance obviously and then the other part is Healthcare. Those are the three sectors where we go to the clients and talk about chatbots. They are getting more excited and they are wanting to know how they and increase efficiency as well as considering the cyber security portion of it all so, intact. so, I would say rather the sectors are those three but verticals within a company they are more customer service and HR is where we are seeing more traction. Yeah, I think the insurance trouble and banks are growing well whilst giving their solutions and on the other hand. There's messenger side lot of companies, agencies doing messenger marketing there are two verticals in up there. So, most of the banks are focusing on the fa Q's in the customer support side, but on the messenger side, they're all doing messenger marketing and releasing their new products to their phones and the followers and their iterating their products with this base and on the other hand on the Travel side Insurance side. They just solve specific problems such as customer support. Terrific use. Yeah, I mean those are in agreement with what we've seen as well. so, Finance is a huge area. A lot of people are making large Investments there, also, Big investments in health care and you know, I think a sign that this industry is really maturing and that enterprises are really starting to take it seriously is you know, kind of our first two or three years in existence, you know, we took security seriously, but now as we get into this really big Enterprise deals, you know, they really need us to be compliant to significant standards and are really demanding the top level of security from companies like ours. I think that's a sign that they're really starting to take this space seriously. I want to toss in a question about voice. So, it is voice emerging. It's and definitely on the analytic side. Are you seeing more customers for growth? Obviously, it's an emerging and maybe quite not so, mainstream, but I just want to get your guy your opinions on in addition just like traditional chatbots house voice impacting. Yeah. I mean as I mentioned we started Voice is still a space that's growing pretty significantly I think gaming is really the place that voice is it we're really seeing significant gains there. There are some games with very high retention that are actually making real money, you know Amazon has in store or in speaker purchasing now and yeah, there are some games that are seeing quite good success. You know, I think one of my predictions are kind of over the next 12 to 18 months is we're really going to see a lot more bot Commerce shopping happening over these voice interfaces or V Commerce. I think that's really going to start to take off as people get more familiar with, you know adding items to their list and okay just go ahead and buy that and send it to me. I think that's coming over the next sort of 12 to 18 months. Yeah. We have seen all those cases on Alex aside on the Google side very much and we see they solve their problems. They see the bottlenecks points in there. Whirlpool of conversation flow and then iterate their conversation flows and we look at each of our virtual agent on The Voice side their life and up to six months. They are live update month the alive in the beginning of the like let's look at the three years ago two years ago. The voice apps are dying like after one month a tied after one-month data, but right now we see more live cases and this is good and more than 88 8K boss voice walls in the Amazon. Alex is right now growing well and we see more like is I think this is much important than like in other cases it if it's live then it's good. I think the one area that it is al so, increasing is the demand is in the real-time voice analytics during the customer call. Like when you are talking to the customer the customer can get mad but he can be very subtle, right? You know, it doesn't. To be always The Voice modulation doesn't need to actually Spike up. The choice of words can be different. It differs by every individual how they express that they are not satisfied with the service lot of voice to text or analytics that happens is in the batch. There are lot of vendors who are doing real-time voice analytics, but that we think is going to mature a lot. Like I was in Customer Care conference in Las Vegas this June every vendor has a voice real-time voice Analytics. Offering but when we have demos with them, we see that okay, it needs lot of maturity before we start implementing in the in the live environment. so, that's one area that I am really excited about very cool and let's a look at when your customers if you can talk to you have to name the customer but let's talk about what the ROI impact you've helped make with these customers and maybe it's a Cinderella story that hey we know that you need integration and wow, we were hey you know cut prices in half on Etc. But let's see if you have tell us about you know, customer Cinderella story. So, I really can't talk about specific numbers, but I'll just tell you about use case with it's a company where they help they help people get subsidized smartphones. so, previously there was this long educational process about okay. If you meet the certain government requirements, you can get a phone you have to fill out this application. You have to send in this proof of benefits you have to do all this stuff and the client was seeing just a lot of drop off and they're using Facebook ads to go to a landing page and it was very hard. so, they created a chatbot that was an issue where they initially were just sending Facebook ads into messenger and they had live agents. I think in like Latin America trying to go through this whole process of educating them and getting the correct information and then getting them to sign an application and all this so, they created a bot which actually automate sets 75% of that whole process. so, they can go through they can learn about the program. They can put in their zip code to find out if they actually even can get that in their state. They send in some photos. We push that to their CRM an agent can go back and verify. Okay. Is this actually a legitimate government docent? If so, they don't have to actually chat with the per son they just flip a switch in the CRM and initiates the next step and they can automate pretty much the whole process. If at one point they need to really step in Then they can do a live chat and they can kind of wrap up the conversation but I really can't talk in numbers, but they're really seen a lot higher conversion rate and especially because Facebook Messenger allows them to really reach back to these people and touch them versus email or some other channel like that.

Great. Yeah, So, I can talk an example about the evil eye or unsteady on procedure bought the solution that I paired at a particular Bank. The problem statement was around 15% of their calls require the agent to put the customer on hold and go into the knowledge repository find the information and come back and all that added around 90 seconds into the call. The company is huge. The call vole is immense that 80 90 seconds translated into 35 40-million-dollar business problem and because of re by procedure bought we kind of built it like a like a chatbot that kind of fetches the information for you proactively knowing what the call is we were able to save 60% of that cost. so, that is something that I think a real example of Roi in the customer service space, but I want to take another thing is like our Roi is interesting because there is I believe there is an imbalance between the advancement into the products that are happening and the Readiness of the companies to kind of implement them. We still have lot of companies with decade-long antique systems non digitized processes. The data is not available in the system's forget structured unstructured and all of that and specifically talking about like Financial Services insurance companies. so, that is a problem to actually get the full benefit of chat Bots and get the true Ry, So, it's there is lot of potential but it's capped by the ability of the organization itself. Yeah, I think at first place let's look at how the virtual agent processes going on at did at first place Company open the project and then the first phase is designing the conversation Falls and second phase is building to Virtual agent. Aunt and I'll text a space in our side and comes in end. so, what we see in the most of the Enterprise side, they scaled theater virtual agent then comes to the analytics part on. This part all takes is waiting. And in that part what we see most of the Enterprise's Telco’s insurance, companies’ banks are want to store their data on their end. so, as at Baltics we give our solutions is a software as service and on private cloud or on-premise adopts. so, the main thing what we see in last hiding one year after we set up the on premise and the private class setups to there. Cite, the things are getting much easier because it it took to process in a long way in the south part because they don't want to store it and after they started it they find the bottlenecks point the iterator flows in every week every two weeks and we see a lot of shifts in a good way and the bank sites in that process, but we need to look at the picture in it white. Yeah, I mean, I think the flow you described about how people build these conversational interface is pretty typical, but I actually think pretty flawed the you know, we had a we had a pretty big customer, you know, invest heavily in conversational interfaces designed, you know this whole body to help with customer care integrate our analytics launch didn't realize it was not working at all. so, they took a big step back. They built a much simpler conversational interface integrated analytics launched it and then started to iterate from there and over the past six to eight months. They've really, they've gotten back to that same feature place. They were when they first tried to launch but now it works beautifully and I think that's really the les son here about how you have to build these conversational interfaces you get tons of feedback from users, but you have to listen to that feedback and use that feedback. so, start small and then slowly improve the interface. An overtime another one of our customers that benefited greatly Vale one of the top gaming voice gaming companies. They basically doubled their attention using our tools so, they went in and looked and see. Okay, where we losing users where exactly are flows not working and by fixing those, they were able to essentially double their attention. Awesome examples guys. so, what's next in the body ecosystem? Like what's on what's on the horizon if developer out here or an Enterprise says like hey, we this is that we need to start investing in these areas. What do you guys what's your what's your crystal ball? Say over the next five years where we should be investing? That's a hard one from us. I think again like we just right now we focus a lot on Marketing sales customer service I think for us it's really developing these like natural language processing systems or I guess goal systems where if I'm trying to generate a lead, I'm trying to get this per son through a funnel, but they go off on a tangent. They start talking about maybe their favorite pizza place or something like that. Like, how do we have a conversation with them like a chitchat? But then how do we bring them back in a way that we don't lose the lead or yeah, they fall out of the funnel. so, I think when we think about stuff like that for us again what we focus on I think being able to have these side conversations where user falls out but then organically bring them back in where it's not like a forced or broken situation. I think the three are a couple of pieces that we think that we are going to get into a problem sooner than later in the come. When a large company there are, they operate very siloed structure and there are a lot of chatbot Frameworks out there in the market and the it's cooler. Continue to grow there are a lot of products in general like analytics you talk about it you talk about the prototyping these things are increasing in numbers with need to think about as an organization. so, I will talk more about from the organization standpoint as I work with more businesses than with the product companies. We need to have a thought about how do you kind of select few enterprise-level chat bot framework source software’s that you can work with because otherwise it's going to create a lot of mess in your internal systems on how to make all of these function and drive consistency be compliant with your regulations and all of that the second piece I think we need to think about as a company is to digitize our processes. I think that is something that is really important for any product to work like chatbot the fuel for At bot if their data is not data available, like even if your NLP engine is through like amazing like human grade, but there is no data for an organization to implement in the back end. It's not worth it. so, it's striking that balance of the technology advancement and your ability to implement it. I think that needs to improve significantly. So, the adoption at the c-suite level which we interact with on a daily basis, they need to understand that what we have at least at all. Not that so, the technology can grow and get you more business out more Roi out of it. I think we'll see a lot of cases. Just look at again three years ago. There were no AI leads VP of AI those kind of roles in the companies, but right now we see a lot and we have been working with the company for two and a half years in the first year. There were just two Bots virtual agents in different fields of the company right now. There's it's more than 10 and I think it will more than 20 in the next couple years. so, there will be a lot of AI lead roles we play rolls on the AI side and the thing is if you have a tool if you have a project or if you have some offering while going to those people you just Define the simple thing, you just put the Border in and really narrow way and find a specific problem. And then you will see those Ai and MLS will be more in those fields of the company with your solution and we will see tons of cases hiding in it two to three years. Yeah, I mean, I think people like to think that you know Improvement happens in these big steps, but I think this is one case where really, it's just kind of incremental Improvement on these interfaces. So, you know, your customer support interface is this bot is handling 30 percent of your traffic and then 40 and then 50 and then 60. so, I think over the you know, next five years know that will really start to increase till you're handling almost all that traffic via your conversational interface. I mean the thing that really got me excited about this is that you know three years ago, these are the natural ways in which we as hans communicate, right? We want to use language. We that's what we use to communicate with each other and that's what we want to use to communicate with computers. We don't want to have to type and press buttons and fill out forms and do all that bullshit. We want to use language but computers need to kind of step up their game a little bit so, that they can understand us better and just for something like really far out. I feel like VR is still very new industry, but relational interfaces and language interfaces to VR is Supernatural having to like type when you're in a VR world is super awkward, but just being able to speak and talk to the environment around you is very natural. So, I think that's going to be really interesting and fun space when that starts to take off the ottoman awesome insights to kick off the day to day. I want to thank everyone in the panel. Thanks for being out here and guess you guys will be Are we going to do questions or we're going to do is like yeah, I guess we did couple questions raise your hand. Hey Jesse, one of the things you mentioned earlier is that creating this conversational like platform is especially hard. What would you say would be the one thing that makes it especially difficult from what you've seen? I think people just underestimate how rich and complex languages. You know, there’s so, many different ways. You can say things so, many different ways that you can express an idea and when you're kind of designing your Bot, you're like, okay, you know, there's like they're going say this and then they might say that and then they might say this but really there's like a thousand ways you didn't think of they're going to say that first thing and then half the time they're going then go on to something you didn't think of so, and honestly, you don't need to know that You just need to start small. so, you get this feedback. That's the amazing things about this unlike designing a website. You actually users tell you all the things that they want to say. And do you just have to collect that data and act on it. so, that would be the biggest thing people just understand misunderstand the sophistication of language.

Hi, this is for anyone in the panel. so, how do you measure the success? Especially, how do you convince the customer on the way to measure the success of a chatbot? Is there any structure procedure or is it varying from use case to use case? How what would be your experience with that? But yeah, I mean it really think it really depends a lot on the industry. So, you know if you're looking and it's something you wanted to find upfront with the customer, so, You're looking at you're looking at customer support you're really trying to reduce escalation to an agent. You're really trying to make sure that customer satisfaction is up. so, those are the two main things we're looking at like how often to does it escalated to an agent and then how satisfied was the user with conversation and usually you do that by just asking them and then measuring Basically, it depends on the specific use case and the team and if the product team just focus on the payments thing and then if you if they want to end the converse all the conversations with a payment they can set up segments and funnels and they can see how many of the people reach out the payments tab. so, this might be a kpi for them and but it all depends on the case and the product owner team so, they can set up lot of kpis in about their Well, then they can track it. That one tangible. so, when we go and Pitch our chatbot services to contact centers or operations or even HR and help desk shared services. Typically, they are looking for efficiency like shown earlier showed. The average call cost is 4 and a half dollars in some institutions. It's up to eight or ten dollars as well. so, reducing that call vole, right? That is the one of the kpi that can immediately. Directly below the customer away through the self-service the other pieces like even if you have increasing the efficiency of the contact Centre agent or back office operations or whoever who is picking up your call the HD. The average handle time reduction of that is another tangible metric that people get excited about. so, if you talk about those two things and if the use case supports these two things that's a very great way to start the conversation and address the problem on his head One more, what would be some of the best metrics to keep track of two to see how well the interface is doing? And then from there how do you grow that to where you have a lot of exchanges coming from people where one or two people are just not able to keep up with looking through a conversation, sir? I will be anyone your thoughts on that. Yeah, I mean the first thing you need to look at is how many of your messages are just going to the not handled intent? Like how often is your Bot just have no idea what the per son is talking about and then really digging into those messages. Okay. Can our tool allow you to Cluster those messages see what are the most common things people are asking that you're not even trying to answer? So, you know find those messages put those in NLP fix that problem and then and that’s something you're just always going to be doing always going to be looking at. You're not handled intent. When am I breaking down and then you know, but once you get that under control, it's really starting to look at? Okay, what are the bigger use cases that people really want to do with this pot? And how can I kind of expand this functionality and they'll tell you and then and this is really focused on kind of the customer care case, you know where they satisfied with conversation and you can look at things like sentiment and other things but we really discovered that the best way to do that is just to ask them and track that over time. Yeah, basically you need to dig into the details in a very well way in our so, you can create a specific audience. For example, you can say that people having an urgent intent and having located in SF having an English language and average conversation staff is more than 10 you can dolls those kind filters and you can list all those audience and you can see all the details of their conversation details, and you can export those and maybe you Put them in a Google retargeting or custom Facebook custom audience to retarget them. This is the main thing to dig in more details. I think that's it. Yeah one more question. I think you push. My name is Russian. I work with Walmart. I think what my question was very much aligned to the last question. I think you partially answered my question so, a bit more can bit more clarity on that. so, we're talking about explicit way of feedback and implicit way of feedback and when we look into the data itself you are looking to implicit be implicit way. Is there any way we have we have implemented shared board conversation with chat pod itself is explicitly asking for Back and you're collecting that conversational data from user or you just looking into the conversational back, back, back in data and getting feedback? Have you tried conversational way of getting feedback from the users before? Yeah, I mean absolutely and I think looking implicitly is fine, but actually asking is actually really good thing you and I encourage adding that to the end, you know, if it's there's a natural place in the conversation adding that and then tracking that something a lot of our customers do and yeah, it's really important thing to explicitly ask the customers how the experience was for them. Okay. Hi, what are some of the best practices you would you would recommend for bought training once it's once it goes live for when the training of the board once it goes live?

What some of the best practices you would recommend two organizations? Yeah, I mean, it's sort of what I said is you want to look at what exactly users are saying? You want to find the most common things people are saying that you're not handling so, taking those and then put Back into your NLP engine you want to spend time, you know sort of thoughts go through this process, right? You initially launched if things are relatively low vole you spend a lot of time kind of looking through transcripts trying to see where users go wrong. But then as you start to get scale that doesn't really work anymore. You need to you need better tools that tell you okay, here's a percentage of users got lost off of this path in your conversation flow. Here are the most common things users are saying that you're not understanding put that back into your Tianjin retrain your engine make sure that the engine then still works after you did that retraining and sort of constantly iterating that process. Yeah, the basic thing you just you just focus on the iteration conversation all these tools provide tons of features.

And the main goal is finding the bottleneck point and iterate your conversation love this thing. so, there are tons of features, but you need to find bottleneck point and iterator conversation that the follow. Who think right guys? No, thank you. Thank Thanks panel around with flyers in if I can get everyone to just wave at the camera. They leave everybody wave and take a picture is 1 3 say yay. Okay. We need more caffeine in these people this morning. Okay. Thank you.