all right I think we have enough to get started and if folks are just continuing to come in welcome to the session today so uh today we're going to talk about analysis for itm right we're really going to drill into the details and we're going to talk about the capabilities why we've built the product and we're also going to show you a sneak peek at what this capability can do um and then throughout this session we're going to ask questions and feedback because at the end of the day we are here to build product that works for you and this is our forum and opportunity to really get feedback on the things that we are working on so please please please add some feedback and let us know what you think about the session all right so before we get started I want to start off with some housekeeping we've saved some time at the end for Q&A so please use that button and ask us questions we will get back to you as soon as we can uh and if we can't we'll try and follow up with you offline this session will also be recorded on our community so uh if you have colleagues that you want to share it with it will be available there and at the end of this event we are going to ask you for some feedback via a short survey so please we take this seriously in fact we're going to talk about some of the feedback we've heard from the last session and how we're planning to address it in this session so what you guys tell us really does make a difference in how we present and the content that we share with you so we're really just trying to make sure that we make these sessions as valuable as POS possibly can for you all right so for today's agenda like I mentioned earlier we're going to talk about some of the feedback we heard in the last session and then we're going to go into an overview of analysis for itsm then we'll jump into a demo and we'll get we'll conclude the session with some Q&A all right so here's what we've heard right again we value feedback what we've heard was we want more technical information and you know we really need to walk that line so we're going to see a few slides that talk about the capabilities in Greater detail um but we want to tiptoe into this right and if you want more after this let us know in the feedback and we'll add more in the next session and then secondly I would like to be more demo focused right less focus on an overview and more focus on the demo we have that in store for you so you're going to see us jump right into the content into the demo um and at that point hopefully we have ADD that feedback a lot of people asked for a road map session so again we're going to ask you if you want to be attend a private roadmap session so when we ask that question in our poll please do respond uh and you know what air on the side of optimism right if you don't have the time you can always change your mind later but we need to know if you're interested now so that we can make sure that we accommodate you when the time comes and then finally knowledge or sorry knowledge article creation this is going to be something that we can't share right now it's planned for a future release but in our road map session we'll make sure to include that demo there and then the last was is going to be PDF handouts and we have done a lot of of community article creation and in fact I'm going to actually talk about that right now so I'm not going to go through all the this the the resources that we've created for you in anticipation of this session instead I'm just going to link them here and you can click on them or explore them and read them in your leisure um but we have written about six articles so far and you know what this is our start we're going to make these better we're going to make them um as engaging as possible accompany them with videos and so forth so that really you get the information you need and again when you're there add comments because we read those comments and we want to make sure that the content is there and it meets your needs so please uh take the time when you have it to read through our content and uh you know let us know what you think all right so Safe Harbor will be sharing some content here that might be forward looking um but for the most part honestly everything that we're going to share with you is GA today so you can um you know take this uh at face value all right so let's start off and talk about why we're doing this right we've heard from you in the past we've asked you questions on our live on service now webinar and what we've heard is look it's not just efficiency and user experience it's actually efficiency and productivity these are things that we are trying to help our customer customers achieve while maintaining that exceptional user experience this is Mission critical right and it makes sense when you look at the problems that we're all facing and that is the self-service that we have is not always there right there are gaps in our knowledge base we don't always have the best catalog items or maybe we don't have all of them in place right and you know what if we get connected with the knowledge article or catalog item sometimes it's just not engaging enough it doesn't quite meet the bar for our most fickle of end users right if I see a KB article I can read it but honestly if you give me an answer from that KB article wow that meets my needs so we have limited self-service and it's not that engaging that reduces its Effectiveness next there's collating information right we know as an agent the information is going to be in different places it's going to be in our work notes it's going to be in a transcript it's going to be in the description the short description that makes it very difficult to understand what is happening with this incident right and sometimes we just skip steps we say hey you know what let me just reach out to this requestor or let me let me in an attempt to save time I'm not going to read as much and that actually makes triaging more difficult because we're more prone to making mistakes or asking repeat questions so again this is a challenge then we have undocumented steps and solution Solutions right when it comes to those handoffs sometimes we do just enough to hand it off to the next person but we're not really telling them what we've done we've just told them that we can't solve it and when we solve these incidents we don't always write down the solution that really resolved the incident sometimes we just write done or resolved and you know what that makes it difficult for other people to benefit from the great work that we've done so these are real challenges that we've heard and when we look at it through a process flow it becomes very apparent starting with that requestor the self-service content doesn't always exist I might start my journey off with the in the portal or the virtual agent and if the content doesn't exist it doesn't exist right and at that point I need to be handed off to an L1 agent and when I do you know what that L1 agent is not always going to be looking at uh the trend transcript to get up to speed right they might ask repeat questions because reading the transcript is hard right you got to scroll up and especially if it's a long running conversation between an employee and a virtual agent or another agent I don't have have that presence of mind to read through the transcript in detail and you know what if the solution or the content didn't exist for the employee chances are it doesn't exist for that L1 agent so they might be using agent assist or searching the knowledge based for agent facing knowledge but they may not find it there and when they don't they usually have to transfer it off to the L2 agent this is where things get expensive right and when they transfer it they're not always documenting what they've attempted so far so that L2 agent doesn't know what the employees attempted it doesn't know what the L1 agent has attempted instead they're forced to think about this aresh right and they're busy and they're trying to get on with their day so so when they resolve the incident you know documenting the solution is not always top of mind so they might leave a note like resolved or done not going to be helpful for everybody especially not making the incident smarter and that's why we've introduced nysis for itm with nysis for itsm we increase self-service right we make that content more engaging we reduce mttr we make it easier for agents to get up to speed and resolve incidents faster the quality of service has gone up tremendously as well because we're not asking repeat questions to that end user and we're getting up to speed and we're ensuring that the collaboration across agents is as good as possible and the experience for everyone just got so much better the content is more engaging right I'm generating answers or making catalog items more conversational and when I'm interacting with an incident or I'm getting that live engine handoff it's so pleasant to get a summary of the transcript or the incident and when it comes to generating resolution notes oh my God that's so much better than having to write them from scratch and finally if you're a platform owner we really lower the total cost of ownership we can get up and running with all of this capability in about 15 minutes most of the time is going to be spent waiting for the system to load the plugins and install the configurations none of it is really on the on the effort of um finding documentation and so forth we really thought about how to make this easy to turn on so you can get value out of it immediately so let's talk about how this system got Smarter with analysis for itsm so again starting with that requester again if on day one we can't solve all the problems with our content and our employees aren't going to be able to find those knowledge articles or those catalog items if they don't exist right so it it can't happen right but in those inst instances where that requestor needs to get connected with an L1 agent we make that handoff efficient and effective the L1 agent doesn't have to scroll through the transcript instead they get to see a summary of everything that has been attempted so far by the virtual agent or another agent which is great and when they do the handoff we're helping Ensure that they are documenting the actions that they have taken so far so we're mining through the work notes and the transcripts to ensure that those handoffs become as efficient and effective as possible and when the solution is there when they resolve the incident we make sure that it is documented and better than done or resolved making ensuring the system is smarter what this does is it makes it much more easier for the content to be provided back to that L1 agent that is we can shift left so if you're an L1 agent now that article or those helpful work notes or resolution notes become available to you in the system right so if you're using agent assist you can find an article that was generated or you can look at an incident with with its resolution notes but what's better than that if you're a requester you get access to that article right away and when you're engaging with now assist you get generated anwers so you don't have to read through the entire article you just get the portion of information that is helpful to you so to tell you a little bit more about this right analysis for itm is really for the business right starting with those requesters they like self-service they want to self- serve and when the content is increases in quantity uh I'm just reading through the um the notes the qu the the chat here it says Stan you said that the presenter has gone quiet um Can can everybody else hear me can I get some Jeff is everything okay I'm hearing you just the same actually M okay appreciate that thank you so to to resume and and hopefully um hopefully Dan you can watch the uh recording uh and if you hear this you know we stop for you um analysis for itm is for requesters right starting with the requesters we want to make sure that more content is available in the system and when it's there we want to make it more engaging for agents we want to help them process information faster and document that pertinent information so those that content and the system gets smarter as a result and for platform owners we want to make sure that this is not something that they really have to think too hard about turn it on get it into production immediately get an Roi right help getting some uh system information out of the system and and make it smarter so that your agents and requesters can get value out of this and you don't have to spend a lot of time maintaining this solution so a lower total cost of ownership and a faster time to Value equals a higher Roi all right so let's talk about the capabilities and this is what I said when we're addressing that feedback of trying to get more technical so analysis capabilities Ed its core do three things one is it helps you understand the context and the intent this is the prompt that we talk about right so when we ask questions like how can I ask get access to a VPN we can understand that as a request and the specific entity here is going to be the VPN right this really helps frame the model so that it understands what exactly is the problem and who it's for and why it matters then we synthesize information so this is ensuring that we search across multiple sources and synthesize a summary right so if you have a long running information or long running incident we can take all that information and really condense it into the things that matter most that means going past all of the the um the noise as we call it making sure that only the important information is coming through to the agent when we do a summary and when we talk about generating content we are looking through all of the information that is out there and helping generate a meaningful response in return right this ensures that the virtual agent can respond to answers uh questions quickly so taking information from the knowledge article and generating an answer from it this is generating an answer from the content that exists right the the benefit here is that we reduce our um the likelihood of a hallucination because we are grounding the model and our solution in your knowledge base and we want to talk a little bit about our our generative AI approach right so we have general purpose models like open AI Microsoft and you know Google cloud and things that are coming out in the future this is not going to stop right and we have asked you in the past are you using any of these models and it's great to hear that you are right we understand that we all need to be up toate on this technology now what's great about those models is that they're general purpose you can use them and they're a great starting point for understanding the technology and building your own use cases for your business we on service now are taking an addition to that approach we're adding domain specific models right and the benefit for that is that we have such a passion and interest in ensuring the quality of the solution and the predictions that are out for our solution that we are developing our own models we have the expertise to do this because in the past we have been developing Solutions like nlu predictive intelligence task intelligence to us this is just another evolution in the step of artificial intelligence so we're building our large language models so that we can be more in in control of the quality of this solution and that is really top of mind for us right we want to make sure that you get the best experience out of the system and that the system is responding in the domain that it we understand best we are the leaders in itsm so when we combine that with techn our technical prowess in the space we can ensure that you get high quality and really awesome experiences from the system all right so let's talk about our vision for all of this and then with this I think I'm going to just jump into the demo because I want to address that feedback of getting more demo less content our vision is not to stop with the solutions that I'm about to demo you uh demo to you we have a vision of ensuring that every single application and workflow will be touched by this technology now assist for itsm is going to be something that becomes our um vision for all of our Solutions everything needs to get better because of this technology and it will because this technology is really awesome and we are ensuring that it is woven SE seamlessly into every single one of these workflows and applications right that means that it's just going to become easier to work with and the experiences are fundamentally going to shift they're going to be more llm powered and and and easier to interact with and more powerful and better to use right and that's going to yield more resolutions for you as well as a lower mttr all right enough talk let's talk let's get into the demo and I'll show you how this capability works okay so the first thing I'm going to do is just to get us all started on this is I want to show you how our retrieval augmented generation capability works that stands that is short uh the acronym there is going to be Rag and you're going to hear this from other vendors and and in the industry at large and what I'm going to do is just to show you how this work uh how this capability works is I'm going to start with a simple use case called what is Spam now the idea here is that in the past would have looked at this I would have searched for this query and I probably would have clicked on this article here and I would have read it and eventually I would have gotten my answer right but you know what's better than that is if I search for that same query and the system leverages that knowledge article and generates an answer from it so in short what it's doing is it's retrieving all the articles that are similar in meaning to my search query and when it finds an article that is very similar in meaning to my search query it is going to search that article for pertinent information and it's going to generate an answer for us okay that's retrieval augmented and generation right rag now that's the architecture that we are using for our analysis for itsm specifically our search capabilities here and the benefit is very apparent right I don't have to search I don't have to read through an article to look for the information that may answer my question instead what it's doing is it's telling me the answer to my question right very directly now this is a pretty simple use case but it gets better when we look for things like uh how to connect a VPN on my iPhone okay so let me show you what that looks like here when I search for this search query it is going to find an article how to configure VPN for Apple devices so it retriev articles that were similar and then what it's doing is it's generating an answer from that article to help me self- serve faster okay again the article title is how to configure VPN for Apple devices however the answer is how to connect to VPN on iPhone and here are the steps so it's really saving me that step of clicking and finding the relevant passage and it's just giving me the answer so that I can selfs serve immediately Jeff um let me pause here any questions so far hey a yeah there was one that came into the chat um says this is great but is generalized AI Fair applied to snow serviced data domains whose underlying inference technology are you using uh so we'll likely want to do that like when I when we uh when we launch the poll for a road map followup um please please say yes to that and we'll get into the details there um we can talk about that all day and I don't want to sell that topic short with a quick answer on this webinar at this point but thank you for that for that question and we will be sure to follow up with that again the feedback always does get addressed um if we don't get in this this session we'll get in the next okay one other question who owns uh and where is the synthesized data stored the synthesized information uh it is generated on the fly so we're not necessarily storing this information anywhere what we're doing is we're taking your information that came from the knowledge article and generating an answer on the Fly for the requester so this will change if you change your article this will change as well um and that's the beauty of this system it doesn't have to retrain the model at all it is simply using our retrieval augment and generation technology was really powerful but it is elegant in the sense that you don't have to retrain or do anything on your side just turn it on and it continues to work very good thank you um all right right that's it for now okay great all right so the next thing I want to do is show you how our virtual agent takes advantage of this new capability so I'm going to do a sidebyside comparison and what I want to show you is how we can show how catalog items become conversational in the virtual agent so in the virtual agent I'm going to ask it a question I'm going to say I'm in the product Division and I need access to Dynamics and the reason why I'm doing that is because I want to show you how I took this catalog item and we made it more engaging for the requester so again I get connected with that same catalog item you know the search query is different over from from over here uh because it's more conversational and when I click get started the first thing I want to draw your eyes to is the fact that it doesn't ask me for what division I'm a part of and that's because I've already made that information available to the virtual agent I've told it I'm in the product division so it skipped right past that question next it's going to say what region do I belong to again there is nothing here on this catalog item that says what region do you belong to instead it's just a region parameter okay and what I can do is I can type in na for North America and for the sake of this demo it's going to ask me the purpose of this and I'm going to say for a webinar and we're off the the races here right I just conversationally engaged with a catalog item I can submit it and I can say no to adding attachments and I'm done now I'm going to pause here and just state that I did not create a topic here this is not a virtual agent topic instead what it is it's a catalog item that was made conversational for the virtual agent this is an asset that is available to the virtual agent just like it is available to our real life agents and what it can do is it can reference that catalog item and talk to the requestor like they're taking an order from them right so that's the beauty of the virtual agent with now powered by now assist is that you can get up and running with this capability in 15 minutes because it can consume your catalog items and make them conversational for you so real powerful Stu and I just want to make that clear because I know in the past we've had virtual agent powered by nlu and we might have thought that that needed to be created with a topic but with now assist you don't have to do that instead you can turn it on with just a click of the button and it's going to um make your catalog items conversational uh Jeff I'm going to pause here because I want to make that clear for the audience are there any questions about this topic about making catalog items and KB articles conversational see here I'm looking in the Q&A uh one question around catalog items is we often found limitations to submitting catalog items via virtual agent in a conversational manner because we use many clients scripts and UI policies that contain scripting is that still a limitation yeah so that is a very good question I'm glad you uh whoever asked that asked it because yeah I also want to be transparent not everything is going to be conversational on day one there are simply way too many uh scripts that go on with some of these catalog items that make it very difficult for them to render as conversational so the fallback experience is that they become a popup experience meaning that instead of conversationally interacting with them I get connected and I pop into this catalog item on the portal page so that's something that is just going to happen now I will say that our leadership has made this a mandate for our product team are the various product teams that all working on this together and what we they have told us is we need to make sure that that does not happen as frequently as it should right if there are patterns that we detect in the system we are addressing that actively in our road map and again when we when we give you that option to sign up for a road map please please hit yes uh because we will like to share our plans there and then also you know get a sense for your catalog items right if you want to help us understand how to make this more conversational for you that is something that we are really actively looking into okay all right uh anything else Jeff or can I move on um around catalog conversations I'm not seeing anything else at the moment okay and if there are any good questions or or you know any questions that you think would be relevant or well yeah please please feel free to to pause me and we can address them okay all right so the next thing I want to do is talk about how we show um chat summarization okay now to do that I need to make sure that my agent is available to pick up a live agent chat request and what I want to do is I want to start off by seeding the conversation with some content that can be summarized so I'm GNA ask the virtual agent for help on how to install the data analysis pack in Excel and what it's going to do is it's going to try and connect me with some KB articles again these aren't generated answers because as you can tell these aren't strong enough matches in meaning to my search query if it's close Big Data analysis or Excel functionality it's very close but it's not exactly the answer so it's not going to generate an answer for me so it's very good at understanding what questions it should not try to answer directly so what I'm going to do is I'm going to say that wasn't what I was looking for and can you please connect me with a live agent and when I do that it is going to prompt me as David for that ability to summarize and get get involved here okay it's not happening for David okay maybe it's happening for Abel here we go all right so Abel is going to see this and he's going to accept this chat request I'm gonna rearrange my my browser so it's easier to work and what I can see is a quick summary of what Eliza has attempted to do with the virtual agent okay we see that they're looking for information about the data analysis pack in Excel we provide in information but that wasn't what they were looking for and now it's time for Abel to get involved okay so what Abel can do at this point is he can reply back with some information like what have you tried so far right and when he does that um Eliza can reply back in kind and say things like you know what I've I'm having issues installing the data analysis pack in Excel u i really have this important meeting that to prep for and I need to perform some statistical analysis okay so enough information for for able to understand that they are that this is important okay now AEL is going to look for things like hey what's my uh do I have any information here right so he might look for things like data analysis pack and Excel and uh you know nothing shows up right and and that's okay right we get the same articles that she had seen and that's great however it doesn't help AEL in this situation so what he can do is he can create an incident at this point and now assist for itsm is going to help him generate a summary a short description a description so that when he hands off to the second the pure two agent that um all that information is documented already he doesn't have to say anything or it makes his life easier and it also makes that other agent's life easier as well so at this point what he's going to do is he is going to hand it off and assign this incident to David so let me go ahead and do that and upon that transfer he's going to say can you help Eliza with this incident it's an urgent request okay so at this point what Abel has done as he has saved a lot of time not asking repeat questions like hey have you tried this uh Excel functionality or Big Data analysis catalog and what he's done is he's understand that he can't help in this situation because he's not trained on this matter and the content available to him is not sufficient enough for him to answer the question so he has deferred to an expert in L2 in this case and what Abel can do now is he can find the incident that is assigned to him and he can pick up exactly where Abel left off all right so to do that what he's going to do is he is going to to take a look at the the short description the description everything helps him understand that um he needs to get involved in a different way and that means contact contacting Eliza directly so he's going to start a sidebar discussion with Eliza all right and now what Eliza can do is see that response see just a second seems like there's an issue with my teams integration at this point I think my developer was telling me earlier this morning that that was a problem so what I'm going to do is I'm going to take a different approach to this and I am going to add that as a comment in the work note and what Eliza can do at this point is she can look at the incident that was just created on her behalf and she can reply back at this point okay so he's gonna she's going to say that the instructions worked and what Davey could do now is he can move on to the next step of this incident which is to resolve it and to generate resolution notes so let's do that so when I click on the resolve button what it's going to do is it's generating resolution notes based off of the work notes that are in the system so it says that they were provided with instructions to update 365 run as a local administrator navigate to Excel and reinstall the data analysis pack which were the instructions that we see right here if you can see behind this right and that's awesome because what David status quo here is likely as I mentioned earlier to write things like done or result because he doesn't know that this is really going to be beneficial for folks in the end right so when he hits resolve what he has done effectively is made the system that much smarter let me pause here for a second does does anybody have any questions Jeff any questions that came in ASF there is a question on the accuracy of the automated statements if could talk a little bit about that the accuracy of automated statements um and can we get some clarification on on what the automated statements were there were a few things that I showed here there's the generation of the resolution notes then there is the summary the chat summary that occurred over here and then there were some of the generated answers in um our AI search experience uh the chat summary specifically chat summarization okay yeah yeah so this is something that uh I mean it's pretty accurate if you were to ask us how are we quantifying this um we would need to have a follow-up conversation on that one and we more than happy to share how we're approaching this so please let us know that that's something that you're interested in and we can we can make sure that we set up those calls um but in terms of the accuracy it is we know if we just look at this example and I know it's not the best I'm just going to be transparent like it's it is enough for us to demo on a session like this with everybody on a call but what we really need to do is if you're curious let's get Hands On Together get a long running chat summary transcript and it'll show you how accurate this thing can be right um this is great for a demo because we only have so much time together um so we know that like look two two bullet points data analysis pack in Excel um that's what they were asking for yep that's correct we we also know that they were provided with information yep that's correct we know that that was not helpful we know that that's correct so in terms of accuracy here I think it checks the boxes I'm not going to say that that's the the the ex the the north star or the golden standard for how we judge accuracy because as a team we look at long running in like transcripts really complex ones where it's going to take you you might not even consider looking at the transcript that just went so long right that's not where the value is created with this capability the value is created when the transcript is so long that you you your instinct as a live agent is to actually ask repeat questions because it's just way too complex and too much time is going to go on for you to actually solve this uh question right again we have to remember that the trade-off here is when an when your requestor has to wait for a respon resp because you are taking the time to do your due diligence and redo the transcript that's the tradeoff right so the amount of time even if you were to spend an enormous amount of time doing that uh you're not going to get a lot of value U because your requestor might just leave at that point and that's the worst case scenario so hopefully that answers your question um Brian any other questions sorry OIP there I was looking at a beach ball for a minute sorry on my system there's a ton of questions coming through let me see what we have here that's currently open technical question I'm just picking one have we seen any performance issues specifically with time to load in larger instances once now assist is enabled no no performance issues once this is turned on this thing just works right we we're very conscious about the speed and its ability to help especially live agents you can't be waiting around for like a beach ball like you said uh for this thing to generate a summary we need this thing to be instant and and readily available in fact we're working with a large company right now that works with has a lot of live agent transfers and when that happens they need to be able to get up to speed fast and respond fast so this thing needs to work uh as soon as it possibly can okay great um thanks another question I mean oh don't want to hold up the demo here there's other questions we're not to the Q&A portion just yet right so uh I think we um I I think so what I can do is I will jump straight to our you the one thing that I want to I think we can actually um but before we do that maybe what I'll do is just make that invitation uh available right now right so so again I think we have had this in the past and it was really successful and and was really beneficial for our customers than for us we got a lot of insight and feedback with um on our road map and we got feedback on product that we're actively developing so we wanted to make sure that this was something that was available again because we think that this is really helpful for us and when it's helpful for us we're really it's helping us ensure that we're building product that you guys want to use at the end of the day so if we can launch that poll Jeff that would be great and we can get some feedback on uh whether you'd be interested in in a follow-up conversation all right the poll's going in folks if you just respond yes we will capture your response and have your email address so we'll be able to follow up with you thank you so much and thanks thanks Brian and and Jeff I know we just jumped right into it but thank you guys both um this is a great team effort yep yep cool all right than have a great day everyone thank you bye