all right let's go ahead and get started again thank you all for joining the week bi-weekly conversational interfaces Academy as always uh feel free to visit our virtual agent and nlu Community for content and experts uh you know get to see there for the latest uh content and guides we have go there to ask questions or answer questions all of you at the Academy I'm sure you're all experts by now and you know as we have new users coming in we they could always use your advice use your inputs we also have an agent chat AWA and sidebar Forum I'd like to thank Sam Utah for uh presenting the last couple weeks for the academy uh and then of course as always this video is recorded yes the academy is recorded So if you or your colleague missed out or have to drop early it is all available on YouTube just search conversational interfaces academy uh the playlist on the servicenow channel or just searching in the search bar I'm sure it'll come up and I assure you everything is there if you like what you're seeing by the way go ahead and you're on watching on YouTube go ahead and click that like or subscribe button that way you'll be notified every time this new video uh comes up such as the one we're having right now great so today's topic is on automated testing and group choice for the virtual agents so these are two new features on the Utah release uh I shouldn't say new because Utah's been out for uh several months now so I hope you guys taking a look at this but maybe you haven't or maybe we're just wondering what all that's all this is about so we're going to go into all of this today and and show you how cool and useful these two features are before that an announcement get ready our Vancouver releases coming soon so uh the Vancouver General availability date is September 20th the Vancouver release is available at developer.servicenow.com uh in as a PDI I just stood mine up yesterday so that's available today so you can check out the new Vancouver release and you can be sure that uh you know in the coming weeks and months we will have uh academies on the Vancouver features for the virtual agent and for sidebar Etc um I believe in early October look for that date uh we'll be doing a what's new for conversational interfaces for Vancouver as we as we have done for all past uh releases all right so let's first talk about the automatic testing framework for virtual agent now this is a new feature in the Utah release and it is by far or if not one of the highest upvoted idea portal idea for the virtual agent folks have been asking for ATF for virtual agent since as long as I can remember and we now have the foundation foundational support for ATF for the virtual agent okay so um it doesn't require a store app called automatic testing for virtual agent you can see it in your plugins page you're probably in the servicenow store if you haven't already and it allows you to create test cases of virtual agent topics and we've created or we've provided a virtual agent topics test suite for to distort all of those you can create more than one test case uh per topic and of course with ATF you can run more than one test at a time so instead of testing each topic one at a time you can just run them all at once and it'll say success or failure and and where it fails Etc so this is built in the virtual language designer the it's also included in in the ATF test Suite uh one thing to note you do need and the ATF Runner enabled role to run test cases this is just um in the same role that you use for for any other ATF test ATF testing if you're an admin you're probably you know inheriting this or also just been look out if you don't see ATF reference for whatever reason just make sure you have that role enabled make sure you got the store app installed and you'll be good to go okay so with that said by the way if you have any questions about what you're seeing in the academy or about almost anything for virtual agent feel free to pop it into the Q a channel not chat q a um just so that I can read and and do some effective uh question and answering there um so now let's jump into a demo for a TF for virtual agent so I'm going to switch out of my presentation mode and switch to an instance so this is on Utah Utah patch 5 I believe is the latest patch as always I recommend uh upgrading to the latest patch whenever available so let's go to the designer and I'm going to go to a new button up top here called manage test cases and this isn't this is allows me to manage all my uh test cases for ATF you can see I've run some already but I can create new test cases here okay this tab here and these are essentially all the topics that are published okay that are active that do not have test cases so I can choose amongst here which one topic I want to create a test case score right so let's say I want to create a test case for a topic email setup and it's going to open up the test case right this is a topic now when I when I click test it's going to open up my tab as always right and we have a new UI here called create test case so let me start that and what that's going to do that's going to start testing recording the topic for the test case now I can now uh you know now uh you know run a flow or record a flow that I want a recorded for ATF right so I'll just pick I want to you know test uh the iOS flow and it's going to go do it staying these steps work for you yes and there we go and now I could uh stop the capture I can restart if I want but I'll just stop here and I can give this test case a name email setup uh the original title's recommendation but I'm gonna change this to iOS right got me a time stamp which is great click save I can also trash if I want but I'll click save here and it's going to save the capture of that test case so now uh this blue bar comes up I'll click manage here and it will take me back to here you now see I have this email nope not this one here we go email setup not a huge setup uh test case here which is ready it has not run before uh so I can run it so I can check this box and click run like what I could also do I could also check multiple topics and run them both at the same time so let's let's do that I'll click two I'll click run and it'll run my test cases and I'll just do that spinny thing uh ready uh it failed I don't know why but we could take a look at why and this was in progress okay so it all failed I'd wonder I wonder why let's take a look see I can now click into this failed uh result link here and it'll take me to the ATF test Suite I believe there we go uh failed to create conversation interesting start topic receive everything else skipped uh uh test log test transaction so I'm not going to troubleshoot why this failed here uh now but you can see that like all the logs here as to what happened and in one order is captured here uh the titles here Etc so I can do this for email setup and I can do this for a crate problem which is also here this has more steps so for this one several things uh was successful uh per my last recording then something happened here I don't know what exactly and uh the rest of the things skipped here okay so yeah I could basically create test cases for all my topics any way I want I could create if I want to another test case if I go back to uh email setup for a moment click test again create a new test case you know maybe you want I want to test the Apple flow did you steps work yes great stop Apple save manage and my Apple one's right there so yeah I can get like I mentioned I can create multiple test cases for one topic and I can run you know these two test cases at the same time again if I wanted to okay one thing one other thing I wanted to show you related to ATF for virtual agent if I were to Let's close some of these windows here I can easily just go back to all I can go to uh you know automated testing framework maybe you're more um familiar with um this window here and you'll see here all the test Suites and I could look for virtual agent or actually or let's see this is test Suites yeah here's virtual agent tests retreation topics looking here and you can see here all the here all the test Suites we've run I don't believe any child ones and then I can click into any one of them I think this was the one I ran last time you can schedule them probably and uh yeah there we go so yeah so now virtual agent and ATF are are now uh foreign parts of each other and it'll be a great way for you to test your test Suites and automate this of course you don't have to uh you know test every variation of every topic and uh you know do all that manually so that's ATF for virtual agent in a nutshell so going back to the slides here let me go back full screen here and let me see if there's any questions uh but I do see some chats should we keep client test Runner open while running the test I'm not sure I'm not an ATF expert by the way so I'm not sure client test Runner is uh but I didn't need to use it so you know maybe I'm not sure and then Diana asked oh yeah good point is it because it's not published probably not or probably that is the case because I didn't um because I I threw it back into test or I may I made some changes uh it and activated it for me even though it was published I don't know back in June so yeah maybe if I published it and ran the test case it would work so yeah that's a good that's a good call I didn't notice that inactive flag um turned on as well and I kind of the you know flew past that so maybe that was a good a good reason why it wasn't uh starting it or wasn't successfully doing the test in the first place all right so let's talk about the next feature here which is called group Choice input and this was a um an ask that came to the virtual agent developers um actually via our customer service organization that said hey you know we wanted this kind of UI that you see here and this is very familiar to you if you've ever you know booked the uh you know book to flight booked a hotel or order something online on doordash you'd have this group Choice like hey I have some couple sets of questions that I want to answer ask you all contained in one node how do I do that well this is how you do it and we'll show I'll show you how to do it it's a brand new input node called group Choice it supports both text and images as you can see here you get both single select and multi-select uh and the the last thing to note is that it is supported on a web only however uh another channel is supported on is Apple business chat so that is a a relatively new channel we're supporting it is only for customer service management customers right because we understand some of you like to use apple business chat for your virtual agent conversations helping customers right so that is a channel that's available if you want more information talk to your account exec uh for for CSM for customer service and they could um you know connect you to the right uh customer service outbound product manager for that so that's those are the two channels that this group choice is supported for web and apple business chat okay so let's go ahead and build one so I'm going to exit out the uh slides here go back to here go this is test Suites let's go back to designer all right so let's I'm just going to create a brand new group Choice from scratch just to show you how one's built and then we'll test it and see what it looks like so I'll just call this a group choice test you can see I've built a group Choice topic before let's create here and here is the new grouped Choice node I'm going to drag that in give it a name it doesn't matter I'm going to cost sandwiches I'm going to build a topic that or yeah a topic that asks what sandwich you want to order right so I'll just ask in the prompt here this is going to be my main question right and I'm going to call it you know order your sandwich and um you know when it first pops up the node it'll have a title and it'll have an image which if I want I don't have to this is optional right so I can update the title I can just you know call this sandwich ordering page something like that and I can give it an image from your server or from your desktop I'm going to get it from the desktop so I'll click on this upload image here and uh let's do I have a sandwiches folder and Let's do let's add a bologna sandwich to the um uh card all right so let's see what happened or I need a I can't test it now I need to add a group so let's add the first group and you can see it's all kind of um you know low code don't need to do any jsonian or scripting here um it does tell you hey where does where where goes what exactly so uh group name uh let's call it sandwich order and of course if things don't make sense we could always um change it up a bit I think this is just the name here the display as is sandwich order and we'll say you know we'll call this a grilled cheese and um I don't know veggie sandwich I could add another one uh let's do I have images for all of these I think I had an image for sub okay now I can add details for all of these uh grilled cheese is cheesy right this is like the little subtitle header that you saw earlier I can upload an image here uh I got a grilled cheese image here now you're gonna have to be careful with the images they're going to be shrunken so you're gonna have to do some testing here and just make sure uh you you got that taken care of um I don't know why my description disappeared I'll just add it back veggie sandwich I got an image for that oops not scripting upload uh I think I did oh I don't have that image okay I'll leave that one blank I guess I'm gonna copied from uh or save the image from Google Images didn't let me do it but uh I'll skip that for now but I'll call it um you know vegetarian and then for sub pretty sure I got a so much for that Italian sub description Italian sub now uh I can select here from multi-select I'm not going to do that here because you know we're going to let our users select just one sandwich okay so I'll click save here so that's my first group of choices and I'm going to add my next group of choices okay so this one's going to be called um I don't know condiments uh you know what's what what did I say um select condiments and I'll just say I won't add any uh descriptions here lettuce tomatoes you know Mayo pickles and this one I will make multi-select so yours was like more than one click yes click save okay so I've now created a group choice and this group choice is going to ask me two questions right it's gonna in one single node or one single you know input is going to ask me what sandwich do I want and I have three options and what condiments am I going to want it's got four options um multi-select okay so now let's give this a test so I'm going to click test here now let's see what it looks like might take a lot of load because it's all the images and stuff so let's see what let's let's see what we get okay got my order your sandwich prompt you can see I got my sandwich ordering page here title along with an image note again it's pretty small so you know don't don't relo don't uh create images that the users rely on to read anything so if I click into this I now have my ordering page now apologies for the the bad branding UI I I think um uh you know that my branding isn't set up or it's set up but like I'm not not for group Choice here but you can see here's my sandwich order choose one I can choose grilled cheese veggie sandwich or sub I'll choose sub I like subs and then uh here again apologies for the brand in here select condiments choose Moto because multi select uh here yeah cheesy yeah we got the subtitles which is good and then I'll select you know lettuce and tomatoes and I click submit I'll read it back to me which is great and that's the node all right um the next thing I want to show you is how do I you know yes someone selects a sub sandwich order in The Economist that's all well and good but you know what if I want to use these inputs somewhere right uh so uh let's close this for now let's put in let's grab in a text I'm going to grab in a script for a little bit later uh you know call this grab inputs and let's say Hey you ordered a blank right so you notice because this group group Choice it still acts like every other input it's you can grab it from input variables and it's Globe icon or I think it's a globe globe icon input variables you can see here condiments or users uh the condiments because it was multi-select it's going to be an array the order was a choice so if I select order here I can do that and then scripting here let's say uh you know just I'll call it or grab from script I could uh let's see here it's going to act the same as every other input node here so uh let's see here um or I can say return VA inputs Dot condiments order right I'll say condiments Dot and I can just say get value you know I didn't have to type this at all auto Dr you know that drop down was automatic click save or I can say um with so I don't know this I don't know if this uh if the syntax was right but we'll find out soon enough or maybe we should put a space here yeah so now if I test it again I'm gonna head order my sandwich and order a grilled cheese with lettuce and tomatoes submit you ordered a grilled cheese and my syntax was wrong on the script I'm sure there's a concatenate function actually um I'll it's just for Simplicity I'm just going to do this people in Q a or Charter probably already telling me what I did wrong foreign yeah lettuce tomatoes and they can parse it if you want uh you know uh with zeros with a zero or one if you just wanted one of the condiments Etc but there you have it there's how you use group Choice build your inputs put some snazzy UI on it and then you can of course uh not only will I read it back to you by default but also uh out you know how to grab the outputs as well so all this is available in Utah definitely check it out especially this group Choice especially ATF of course you know in the past couple of months I know we've showed a lot of the Utah features this is sort of the uh the last of the bunch we've been able to show in um uh you know in our academies uh I'll now pause you know that's the that's the end of my prepared remarks uh I'll now go back to answer any questions that you may have so I'm going to open up the Q a panel here uh let's see here Katie asked hold on let me uh there we go kitty asked can the test case check case a check if links added and topic are working for example uh I don't know because if a link goes if um you'd have to you definitely have to try that out I don't think it would work if links go out to like the outside world but if links are supposed to open something up in uh within the virtual agent um I've never tried it I'd be curious to see if that works and do we have any options to handle multi-roll row variable set and virtual agent I don't know what that is I've heard it before I understand it's some platform capability but I don't know so sorry I can't help you out there uh a general question about virtual agent uh where can I learn more about having a user completed HR record producer I want to have to use complete the question and then you come and run around them to the portal submit the case um for record we so I I don't know Cheryl off top of my head it's more of an HR question and HR might have we so HR does have a an out of box virtual agent topic for you to use I believe it's called create case so it may be possible that rather you having to create a your own topic about you know having to create an HR case we already have one for you and that probably uses some form of record producer I know itsm you know by a by analogy they have um you know they they do um service catalog uh topics uh so that people don't need to create record producer for that so um yeah I would definitely take a look at if you go to our documentation just type in you know or search for hrsd for HR Service delivery hrsd virtual agent topics we do have out-of-box topics I believe they may use some form of record producer when you want to say create a case or update an HR case rather you have to build one yourself foreign Russ asks is it possible to map the choices to a reference list oh no it is not uh possible unfortunately Russ uh this is all um it has to be manually manually created but it's a good suggestion though I can see how that could be helpful so no it can't be linked to say a table or anything like that um it's again meant for manual creation I believe in the past you know customers complained oh in order to have a group Choice UI we'd have to script all this and and um you know pull in the group reference that way well we've sort of indexed on let's create a low code experience but I don't think unfortunately we were able to um support reference choice at the time but it's a good good request can nlu be configured with the group choice um uh hmm I don't or let me see I I don't believe so but if I were to go back real quick I'll just see if it's possible to skip yeah I don't see a skip entity uh node option here so my guess is unfortunately not um of course the nlu will work to get you into the sandwich topic or this group choice test topic but then I don't think it's going to be able to input anything here uh to skip entities unfortunately not Dustin uh Russ I think I just answered your question hopefully hopefully that answered that and then uh by web I assume you mean the portal yes and group choice so group choice is not supported in teams although you know the moment I share I share this to customers they asked you know Will Group Choice be in teams or or slack for those slack users uh unfortunately not but it's a good request I think uh you know RPMs heard that pretty loud and clear when they built this uh but yeah for now it's it's web only or the aforementioned Apple business chat Channel will gen air replace V8 topics [Music] um not entirely we you know it'll replace some topics I know that some customers like creating topics that just answer one question right like you how to set up my email earlier right topics like that will probably go away on the flip side we do have advanced topics such as like you know ordering a sandwich right or booking a conference room or resetting your password and these action-oriented uh topics that we have uh they will not be as easily replaced by gen AI right so definitely the the knowledge oriented topics they are already being replaced by AI search and I definitely certainly don't recommend people creating topics just for the sake of displaying a knowledge article right so those types of quest topics probably will be more and more replaced by generative Ai and we're gonna you know talk more about generative AI in the coming months leading up to our Vancouver release for virtual agent uh but then topics like these uh topics like again all the ones I talked about with that are action oriented that are workflow oriented probably not as much great uh so with that let me go to the chat here um yeah I mean um hear you loud and clear on the team support for group Choice uh and and and for slack although I know teams I from personal experience I usually hear more about teams but every so often uh you know you also get this 100 pound gorillas like oh we're a slack shop so yeah I mean input her loud and clear hopefully in the coming future releases we'll take a look at support group choice for uh for teams and from all believing great well I don't see any other questions uh but thank you for the questions and thank you for your participation thank you for joining our Academy today uh We've we're at the top of our we're going to wrap things up but uh as always if you like what you saw we're going to be on YouTube and then uh stay tuned for our next uh conversation interfaces Academy thank you everyone