Virtual Assistant Ability Model

Transcript:

Sorry, there is no coffee break. but thank you for patient. So, I’m the co-founder of rule. I and I’m also a professor in university of California. Santa Cruz is my honor to give the talk here. just want to briefly introduce rule. I we are the conversation computing platform and a solution provider. and so, we're honored this year. we are selected by Forbes as the most promising ai company in America and of the only conversation platform on the list while also one of the top seven conversation platforms selected by for forester last year and Goodwin and gather clove endure. and what we're focusing on is provide empower enterprise digital transformation and innovations what we human-centered automation with the advance of a I would be we combine chatbots and the robot process automation to build a virtual assistant that can help enterprise to remove a lot of frictions. for example, in customer service. you can improve resolution rate better customer satisfaction. You can help a customer to do organ other management check order status and also help support customer to customer. our onboarding and if customer have troubles you can help them and you can also use that virtual assistant to manage no loyalty programs and so on. and then you can also use virtual assistant to do sales and lead generation. the previous talk is in some example, you can actually use the virtual assistants to the whole sales process from customer acquisition to improve conversion rate and to lead qualification asking customer questions catch nurture them or finally you can even provide personalized promotion person personalized recommendations. We have actually research publications. those topics in the way also help build virtual assistant for enterprise internal user’s usages. For example, in the it helps desk. you can help and employers to solve troubles issues and also for her the virtual assistant can help to do recruiting employee onboarding help employee feel all kinds of forms, and it can also help employee to better access their negative systems that it hasn't been changed for like decays very on difficult to use digital systems through the chat bots and we have done all those for different customers. So, actually the promise of virtual assistant is very is there and forester has this report and we have seen various numbers in this conference. that's I believe that's why you are here. you are spending your valuable time in this conference. however, although promise is there but the reality is very bad. This is this year for easter report more than half of the customer are very frustrated with dumb bots. So, there is a big gap between the reality and the promise and some people who do it right get a good thing but a lot of companies get bad results and we look at his say why that happens actually if you look at a chat board market, the marketer is a total mass there are more than a than the companies in the space. and then if you look at ai there are many ai companies don't promise a deliver their promise and there's a financial times reports like for European companies more than 40% of the company who claimed that ai company has no ai at all and I think it seemed for the chat board space and that's where the problems are. as a researcher, I’m a researching this field for more than 20 years. I started natural language processing in 1996. We build virtual assistant in Carnegie Mellon university in 1998. and we worked on this for more than 20 years and we know it's a very difficult problem because the virtual assistant need to understand what you are talking about and it need to provide coherent meaningful consistent responses, which requires a virtual assistant know if this his company's virtual assistant need to know the knowledge about the company. It also needs to have common sense knowledge. it also needs to have discourse understanding to understand the whole conversation. not just a sentence and the virtual system need to do take business process. we are doing human center automation to automate business process through virtual system. So, it needs to take actions and decide one to take action how to take an action and so on besides it's need to be interactive. put those together building virtual state and really need deep ai so the problem is in the market. most people don't understand what ai means in this virtual system field and how to evaluate his ai ability of virtual assistant and motivated by the field of autonomous driving car. they have very clear definition of what AI ability in autonomous driving column level 1 to level 5. We introduced the virtual assistant ability model. it's a model it's a standard for evaluate virtual assistant. So, and even the name if you look at the names of different levels is very similar to automatic driving car because autonomous driving car is a special kind of virtual assistant. it also gives us some guidance on how to improve the virtual system in the future. if you are on level why you may want to work towards a level 2 and level 3 and so on so let me briefly talk about the different levels. the first level one is human agent.

Human agent assists where the virtual assistant doesn’t talk with customer here. we have the customer, right? we have human agent the virtual system actually provides suggestions recommendations to the agent. this is very basic but it tends out to be very useful and then you can go to level 2 level 2 is the virtual system will sometimes talk directly with the with a customer for visual of what asks if the tasks happens as designed or four questions that the virtual assistant knows the answer that's level 2 but to the virtual system if the cask conversation happens unexpectedly the virtual assistant need to transfer it to human then for level 3 the for design virtual designer tasks and then questions virtual assistant pretty much can handle all of those no matter how the conversation flows so the virtual assistant need to transfer human for answering problems and then for level 4, it's called a full automation in designer domain. That means the virtual assistant can take over all the conversation human any need to train the virtual assistant or recruited virtual assistant and then humankind just go to sleep have a cup of coffee and virtual assistant does all the work in that domain and level 5 is called human-like automation level 4 level 5 is what? people call general artificial intelligence and then it will actually go beyond a particular domains generally open domain human-like automation. that's a long-term future now I’m going to talk about the different levels. and as you go from one level to the next level and your expectation about the performance of the virtual system will be different and there from one level to the next. level you actually need technical breakthroughs to go to the next level. that's why they have those five levels. So, for the level one that virtual assistant help agents, you only need basic technique like classification search engine as a retrieval techniques single run of retrieve results that based on the current conversation show it to the agents if it's run that's okay. it's useful and those are the techniques that I teach classes in university my students. sold build those techniques for level 2.

You need the virtual system now support martirano conversation and we have discussed before for the same. meaning there are always different ways to say about him for example, users will say please book a flight for me to San Francisco tomorrow or say, I want to go to San Francisco. can you find the ticket for me? those are different meanings of virtual assistant need to have the ability to handle the variation of natural? to tell the intense of the user that's booked a flight then to fill the slots. the slots are destination in San Francisco date is tomorrow. So, it needs to do intend prediction and slot feeling and that's what you so when we come to level three where we are looking at the scenario. actually, it's more like human conversation where when for the same task of us has different tasks the conversation can happen in all kinds of different ways. for example, the conversation control can switch between two speakers, if user can lead to the conversation later the boss can either conversation that's called mixing initiative. and sometimes you may need to have multiple tasks mixed together in the whole conversation to get the users problem solved and sometimes user when you are asked talking with user about one things user. My toe goes suddenly switch topic to talking with something else and you also need to support context switching and sometimes user will to say multiple problems and things in was when all and the same time then you need to handle them strategically. And so, verse in this those are the things that happens when we have real human conversation because the conversation flow could be very dynamic for level three virtual system is capable of handling variations of conversation flows. that's the next thing. That's actually the state of art the state of what part of virtual assistant is level 3 so the level 4 of fully automation in a special domain is where I have seen research and development in the lab that has virtual system that's capable of handling announcing problems. they have never seen before how they do that. they need to have the ability to gather information to do mathematical and logical reasoning than fighting the answer and they need to have ability to go already at if you tell virtual assistant or goal or problem it need to figure out those planning to figure out how to solve the problem even without being told how to solve it and it also need to learn about the particular domain interactively learning from human from other places and so simple languages in that domain. So, those are various ability needed to support solving announcing problems in a domain once you can achieve that you, I’m more likely to get fully automation in a design domain next and level 5 is handling announcing problems in open domain for open domain. you need to the virtual system now need to plan to solve problems. they have never seen before and complicated that human. do we serve a lot of new problems and the virtual assistant need to answer basic how questions it's not a spider search engine to find the answer? it actually needs to work out the answer itself. and it also need to process comprehend abstract information and need to modify its own knowledge behavior skills and preferences and from instructions using more open language. So, there is a research field called a general artificial intelligence and there is annual conference on that topic. they are focused on level 5 as I mentioned that the state of art is levels, it's conditional automation where the virtual assistant is capable of handling the variations of natural language and variations of conversation flow when you can handle both then for a task is a virtual assistant has already been taught to do it can solve them. no matter how users say about it. no matter how the conversation flow is. So, it's we can achieve condition automation. So, now I’m going to eat show two demos to illustrate the difference between gene level two and level three because level two is where the most of the virtual assistant and a child boss level 1 level 2 are where most of the chat bots are in this market and level three is a state of art in this demo first I will show you level 2. it's a virtual assistant insurance domain. it should have the ability to handle natural language understanding slot feeling and so on but it cannot handle diverse conversation flow flexibilities there. So, the virtual system is capable of intent protection as lot willing to fill this form are not capable of flexible conversation. So, that's the level tune. now, let's see the level 3 virtual assistant in the same domain. Now we're doing context switching to a different task. now user is telling too intense as boss is capable of handling both then the boss also has memory automatic. go back to the original problem getting insurance quota. and also capable of transferring to human as an idiot. okay, as you can see for level 3 you the virtual assistant not any has the ability to handle natural language variations. it also has the ability to dynamically generate a conversation flows based on the context. and those flow is never designed there. it's automatically doing the flow itself and planning itself. and as I mentioned there are also level 4 and number 5 people are doing research on it. I expect in the future how you train virtual system will be how you're treating human you no longer need to take virtual assistant a workshop or do some programming or all you don't need to do drag and drop design council. you can actually just talked of as a virtual assistant like you talk with your new employee and thousand them knowledge the virtual system will learn by listening observing and trying and working them from penalty and rewards then eventually achieve level 4 and level 5 virtual assistant level for could be short middle term and a level five. will be a much longer term but people are working on that and a little bit about what we do is rule I is a platform is conversation platform and solution provider will provide us a platform for level 3 virtual assistant where you can do. and design without coding drag-and-drop always limited coding where when you do integration and this empower us to build smart virtual systems are capable of handling variation of natural language and variation of conversation flow. So, rich condition automation, and then the platform can be managed by central team and the design can be deployed to be different business units of the company. So, and then it's generous scalable approach for people to do like a company to build a like different virtual system for different markets for different business unit. So, we are use case of gnostic platform that empowering people to do that and besides we also realized even with the right technology, although we have the level 3 virtual assistant technologies, but ai is not perfect. So, people actually need to have the right design methodology and development methodology to build virtual assistant. So, we actually have a ruler instead. jeweled which is a research and teaching education part of our company and the team members has published more than 400 research paper related in ai, and we also offer courses. So, here are some examples of the instructors in this in the institute. we have of course, I’m teaching there and we also have professor jimmy cannon from Carnegie Mellon university. he's actually organizing the virtual assistant evaluation conducted by national institute of standards. Deej this year and we also have here as an ai professors. we have some are professors. we also have user experience professors from like a university of Washington university of California and also industry practitioners teaching the course and we generate it's more like an all you need to know about building a virtual assistant to start building a virtual assistant. you will learn about the design and analysis how to build a how to evaluate how to deploy how to iteratively improve and the some basic as I needed. that and the in this just one or two day training you actually build a virtual assistant. our homework is like a building a coffee ordering home virtual assistant besides we talk cover very practical topics of how you build a meta boards to handle many boards in the company and so on. and so that's all for my talk. thank you. if so if you are interested in the virtual assistant ability model here is a link where you can download. our whitepaper is like 12 pages more details in it. thank you. thank you. appreciate it. right above you.

We're going to do a quick qa5 questions first hands up. I got one of your one two, three, four five. that's it. and I feel a little bit generously. So, you're going to get twice the amount of tickets. So, we'll start here. hi you thank you. what are we missing right now that we don't have to get to a level for getting level 4? yes. what are we missing right now that we don't have planning is very difficult? right? and the planning is something you actually don't have their actual long list of things. we I’m using to handle our sin problem its importance. So, we currently actually can solve some housing problem already like in our company, but not to the level that like 90% or 100% the way are like 50% question answering and then and for program planning we can do planning for certain type of question tasks. but if you think about all kinds of tasks and planning will be challenging and planning reasoning inferencing all those are things where when you look at state of water natural language processing. and they don't have much research on that. Yes. hi. my name is Nikhil. I’m for I work in an automotive oem my question is in the map that you had all the levels of automation. I don't see it translating directly to a to a bot that is voice paste. it seems like it works for the framework works for a text a spot but not necessarily void space a specific example. I could give is like a didn't see agency as an aspect that any of the bought any of the levels had in your framework voice space. Yeah, voice-based. So, I want to mention that this five-level. is focus out a I focus on AI and there are some other reports and levels the talk about that focus on product features product features for product features. then you will come like a voice multimeter interaction and so on and then those features are your company build it then you have those feature there. but here when I talked about the ai is not whether you build it or not. you actually need technical breakthrough and when you come to a higher level, we don't even have it in this world, right? so it's a different perspective. So, for example, when you cannot say text retrieval question-answering is you easier or harder that speech recognition, so there is not a level in terms of difficulty or technical challenge and I deaths hi, how do you resolve the problem with the training data in the different domain and the different languages?

I’m sorry. how did you resolve the problem with the training data in the different domains and the different languages? okay. I didn't talk about it. it's some other thing we do right if you look as advance of natural language processing and the major advances in the last few years are deep learning based on language models and so on. and we actually have publication backing lighting mm about deep learning for language models. by the way at that time people don't believe us now. it's very hot. if you look has a deep learning models and which can train from like the internet data. we have a lot of internet data and then you look as a for most people it's black box, but for us, it's not you look at it. You will find it. it's already known freaks. and then you look at the weights inside the model link the waist. then you see something more like dependency parsing tree inside the words and then you look at the model structure further detail you find the model is actually capable of learning composition ability of human language more like some people causing talking passing and so the pre-training mode already have quite some ability there. then when you come to a domain with those abilities, they are what's really seeing our domain spectacles things. So, because those pre-training model you can you don't need as much big data as you probably thought you need to that's advancing in here besides the company already have domain like an unstructured data structure database or knowledge graph like your emails are knowledge graph and those information can start with help boost start to the boss with information when you combine all those things together. we can better solve the limited training data problem. thank you. there is 2 more where you guys at, we are giving away dots echo dots. So, two more questions. if not, we'll move on. all right, let's keep it moving. round of applause thank you very much.