ATF on Custom Filters - July 1, 2020 - Performance Analytics Academy

performance analytics and reporting office hours we are very glad that you're able to join us for this session we do this session every two weeks and for those of you who are new welcome and for those of you who have joined us before I'll come back my name is David Van Huston I work for the now intelligence team now intelligence consists of several products including performance analytics predictive intelligence and reporting for example we will be expanding in the future with additional products but today's session is mainly staged still around performance analytics and reporting so but before we get started I'm going to run you through a few logistical items for today's session remember this session is for you this is your opportunity to get some fresh ideas gain a better understanding of performance analytics or reporting or dashboard aim or the interactive filters and get some practical advice from some of the world's experts in the field there may be cases where we can't solve a technical issue or go or get into details about your instance so we may ask you to open up a case on high or we may need to follow it up off off line and that way we can cover as many questions as possible during this session please feel free to ask questions by the zoom Q&A panel like you see somewhere in your zoom interface so that you can open up those questions and that we can answer either by typing them or we can answer them live and that way we basically retain a repository of questions being asked and answers being given we also then can ask follow-up questions to clarify so we can ensure we're answering appropriately we do have quite a few people on the line so please keep yourself on mute unless you're asking a question but although we prefer you just typing it through the Q&A panel using chat is not advice because we sometimes miss those chats so Q&A is the best way to ask us questions please note that this session is being recorded if you or your organization are not comfortable with that please disconnect the sessions are also posted to YouTube and a link to the playlist is on the page where you registered for this session I will also revisit it and you can just go to YouTube ServiceNow community and then we're one of the playlist is performance analytics office hours today we will have Adam stout giving us some in-depth information on how to make testing perfect so essentially he will guide you through sort of mini getting started with the automated testing framework and then in the concept of responsive dashboards and following his basically presentation and demonstration we will open up the floor for questions Adam over to you Thanks stop it and just to confirm you're seeing the presentation now I am seeing the presentation all right so today we are going to talk about something new for for a lot of us and how do we test how do we test new things we're developing specifically we're gonna talk about custom development for a responsive dashboard so if I'm creating a custom interactive filter that's the use case we're gonna talk about today where I really want to automate my testing we're gonna talk about leveraging the the automated test framework built into ServiceNow so you don't need anything extra so we want to expose you to what you can do and walk you through a little bit how you do it I will say this is not completely in-depth because the automated testing framework is rather extensive and we'll talk about how you can get some more information but our goal today is to get you some exposure and for me success is if you then go look at some more take some more training let's talk about what what we're able to do so we're talking about what is ATF I don't know how many of you have have used it or heard of it before but we're gonna talk about what it does and doesn't do how do I get started and then much as I always think this is a bad idea I'm gonna do a live demo we're gonna build an ATF test on the fly in this call in a matter of minutes okay so let's talk about what ATF with ATF is so ATF allows us to verify that things are good and here's an example of the test that we're actually we're gonna build that allows me to click a button and I can actually schedule it as well to run this it is going in and viewing a dashboard and once that dashboard loads it's going to look at what's on the dashboard it's gonna put text in the dashboard it's gonna verify that things happen that I expect to happen and then really importantly it's gonna it's going to rollback anything I had to do for that test the vast majority of things that I did for that test that allows me to be confident that what I have works and that that is one of the biggest values that we get from ATF I don't have to worry about doesn't work when I upgrade doesn't work if I make this change I press a button and I have a result saying yes this works it's also important when things don't go right when they don't work and then we'll see something like this where I'm gonna run my test and then very quickly it's going to show me a bunch of red saying it just didn't work and now I know something is wrong and I need to fix it by clicking a button I know doesn't work or does it not work and that is that's a big value for ATF so again what what is it so it's a framework ATF is a framework that lets you run create and run the automated tests a lot of applications come with QuickStart tests so if you're if you're using performance analytics using ServiceNow you're using something else as well but a lot of our applications and and all the new ones come with QuickStart tests for ATF so not just responsive dashboard so we're talking about today but whether it's CSM or ITSM or HR there are other tests that you can that you can build on as well for all your other customizations there's different types of tests there's both UI tests and server-side tests both are useful we need a mix of both of those and it also allows you to create reusable test steps so there's several out-of-the-box test steps and you can also create your own test steps for your custom apps or the things you've extended where you just want to say do this you can define with a script what this is and then from there you can easily reuse it so by working with your developers you you can help that they can help you create test steps but then you can have non-technical users create those tests and execute those tests so why do I want to use ATF why does it matter write something else I have to do write you know I had to create a custom filter and now I have to test it myself but ATF gives me a lot of peace of mind it ensures I use it for unit testing so that when I am working on a filter I am confident that that filter works it also allows me to have confidence that when other people are working on things they're not breaking my stuff and if I have a custom filter on the change table well any time we're making a change I should have picked a different table any time we're making a change to change I can run my tests automate it I don't have to spend hours doing it I can just press a button run a test suite and make sure yes when this week's changes go through my code still all works so again that peace of mind whether I'm working on the custom interactive filter or what it's being filtered I can make that I can make sure that that everything is still running smoothly and then the last one is I can detect upgrade issues earlier so for custom interactive filters there was a change there is a change in relay that affects older custom interactive filters that didn't follow best practices New York and below it you can do something kind of sloppily but in Orlando you have to do it the way that it's described in the documentation if I when I have ETF in place I can detect that before I upgrade I can detect that the first day I upgrade my sub prod I run my ATF tests and I know I have an issue there and then I can find and address that that is much more preferable than if we upgrade this didn't get tested and now my customers something's broken for one of my users right or many of my users so it lets me know that my stuff works and lets me know that other people's stuff didn't break mine and then lets me have the peace of mind and service now as we go through upgrades all of my custom code still continues to work and when do I run ATF right so along with what it does I'm gonna run it all my unit I'm gonna use it my unit tests I actually write it before as I'm going along or before I try to write my tests so I know what is right as I've been using this I have yet to use ATF where it did not find a problem with my code I think my stuff is working great but I write out my tests and I go yeah it doesn't support this or what is the right which should the right answer be what should it do in this case using it along with your development is really valuable and then again the rest of regression tests what other code changes which may or which you may not know affects you or not this allows me to make sure that those work as well instant configurations so I'm going to change a property I'm gonna activate a new plug-in I'm gonna run those there now those are generally low risk but particularly activating a plug-in if it's interacting with something it's already there I prefer the peace of mind that I ran my regression tests in ATF and all my code still works the way I think it should and then of course with ServiceNow upgrades you're gonna apply a patch you're gonna upgrade to a family release run your ATF tests we can test our code but we can't test your code this allows you to test your code and make sure that everything is compatible and it's not necessarily that something's gonna break but that there's some incompatibility something doesn't work the way you think it would you were using an undocumented I something you don't know this helps you find identify those very earlier in your upgrade process rather than later on so I'm running it all the time and for my app development and if there's any normal changes I'm running it regularly and then I'm really going through everything with the fine fine-tooth comb when I'm gonna do it upgrade for ServiceNow and the testing paradigm so there are different test tools there's different things you can do the way that ATF works the way that we approach it a key piece of information is that you do have to know what you're looking for we don't do a compare pre-post change so you can't we don't run everything save the results do your upgrade and then process it you actually have to say when I do this this is what I expect so that's just that's how it works and you do need to think about it I find that really useful it does require you to think because it's not just simple tell me if anything's different you need to be more prescriptive about what is right the next key thing is that we create new records whenever possible so I'm gonna test an interactive filter we call this go through this and the demo when I test my new interact custom interactive filter the custom interactive filter is what I'm testing so I'm definitely going to use that exact same thing but I'm gonna create a new user I'm gonna create a new dashboard I'm gonna put the widget the existing widget on the new dashboard in a new tab so that way I'm not relying on an existing dashboard to be there that could change form underneath me I'm not running as a specific user that might get deactivated so we're gonna build new things all the time and then the ATF framework automatically rolls them back it captures all the things that you create and it undoes them all really really powerful built into the platform we also want to make sure we're testing both positive and negative assertions so not just does it work but does it work and does the things that's not supposed to do not work so if I'm if I'm testing something I can this dashboard be shared to this user yes I want to make sure that it works but I also test a user it should not work for and test that as well so your tests can get quite extensive because how many ways can it break you actually want to test them to make sure that something doesn't start working that you weren't expecting to work later on and a key piece of value for you using the ServiceNow 8ef framework which is included in the platform is that it knows the objects and service now and we're expecting to use that so we will see when though we look at the test steps it knows what things are what a UI action is so you're not using some third-party tool that is just looking at the interface and doesn't know what's clicking at but we say run this UI action it knows what a UI action is so regardless of the the browser of the platform of the instance version that will always work so if this saves you a lot of time and headache because you don't have to worry about what is this Dom element called in a kind of reverse engineering service now your allow you are able to leverage ServiceNow understanding what ServiceNow is and there's no other tool that's gonna do that and a couple key things I do want to point out here there's a bunch of parameters for for the ATF so one of the one of the interesting things with ATF is that my tests are actually gonna go to production but I do not want to ever run my tests in production I don't want to run my tests in production we're creating a bunch of new things so there are properties that happen that exist on all your instances to say to configure there's a lot of configuration for for the different test properties on what you want to do and a bunch of options that just kind of fit what you're looking for but the two key ones that highlighted here is do I want to run these tests by default tests cannot run on an instance this property has to be checked to yes to enable it to run so it's possible when I am I in production it's gonna be off it's definitely gonna be off in production you do not want to run a TF in production but as iclone down I might have my clone this might be persistent or I might turn this on when I clone to allow certain instances to allow my tests to run but you have that control this is not something you have to keep offline in something else it goes in your normal update sets it goes normal in your normal applications it moves to production those tests are in production you just have we have a stopper so that we can outrun them in in production to cause issues so that's what ATF is and I'll take a quick breather if anybody has any questions we can you can throw them in the QA and if not we'll move on to how do I get started with with ATF it's great you all want to use it and you're gonna call me up all hours of the night to go how do I get use it how do I use it let's let's talk about that so the first thing is takes their training again we're just scratching the surface there is some very in-depth training in now learning there's lots of stuff available and pretty much everything you want to know about ServiceNow ATF has its own class the ATF fundamentals class it says it takes 8 hours there's a lot here and again we're doing this in 20 minutes but there is a lot of training you can go through to understand what's going on it is self-paced it is free I highly encourage it when you want to make the investment into ATF which I hope you're gonna want to after after seeing this go take the training you're you're you won't regret spending a day understanding what you can do and what your options are and how you can really integrate this into your system into your process there is also a micro certification so once you take the class you can then test your your your understanding with the micro certification I think it's somewhere between 20 to 25 questions open book and you have an hour to do it it shouldn't take you an hour but you have an hour to do it as you're looking to integrate ATF in your process I recommend whoever is running that getting started take this training and then ask to see their micro certification right make sure they actually read it the next part is we're releasing a new project and share hopefully later today that gives you a few additional test steps so we're gonna use these test steps in in our ATF test they are things that you can do without this share project but I'll tell you it makes a lot of much easier so some of the things that this that we're gonna this share project lets you do is create a responsive dashboard and again we want to create new things I don't want to use an existing one because it could change from what I'm expecting so I'm gonna create a new new dashboard I'm going to share a dashboard so I can test security and then I want to add things to the dashboard I want to add a report a PA widget a Content block where we're gonna be testing interactive filters so content block is what we use the debug widget is important for validating that things work and then if I have a custom widget that's not a Content block it's not a report on PA I have an option just add that as well with a couple of properties um you can do all these things you can script them but your life will be much easier if you install this this helper pack application that's in share will send out this link but you'll see the link here as well it's not available right now but again later this afternoon we expect to publish all right let's get started let's do it so we're gonna do a demo here's what we're gonna do in the demo we're gonna create a data we're gonna create a test and then in that test we're gonna create a dashboard add the filter so I've already created a filter on my instance we're gonna have the debug filter again those first three things are all very easy to do because I used because I mean adding in the store project which I've already loaded in that update set if you don't do that then if you just go to how to box you won't see that the filters were those steps we're using so add that share project and you'll see them and then we're gonna open the dashboard form view the dashboard add some text to the add some text to it submit the filter and then validate that the filters applied this is my one unit test to make sure it does my custom interactive filter work okay so now I'm going to stop sharing and move over to my browser so as I'm bringing this up a question came up about what role is needed for this feature there are there are three ATF roles that go through in the training but an admin a runner and a designer I believe are the three so you can grant those as needed Dava Thomas are you saying my browser yeah thank you very much okay so this is what we're gonna test so I'm just going to show you that it actually works will validate that it works so I have a custom interactive filter that lets me do a search incidents on short description and I have a report that shows me the number of active incidents so I'm going to come in and search how many of my active incidents have the word email in it off when I click off of it doesn't do anything I have to click search so I click search I have 19 incidents that are that are open that have email in it it's it's working let's validate this continues to work okay so I'm gonna go to automated test so when I go to the automated test framework I'm an admin right now so I'm seeing everything but I believe test admin would see these same things I have lots and lots of options there's lots of things you can do and templates again that there's very in-depth training that goes through all of these things I'm not gonna go through all of them right now we're just gonna get this test to work hopefully so I'm gonna go to tests again this is out of the box so right now I'm showing 230 tests that come out of the box testing many different things that you can just use make sure they work for you we're gonna create a new one because we're creating something very new so I'm gonna go to new my new test we're going to do this filter custom filter test good and one thing to know I'm actually putting this I have a custom application a custom scope that has my interactive filter I'm putting my tests in that scope these these don't have to go in global they can go in in your applications just like everything else can go into applications and we'll save this so I have a test now underneath the test I need to add the test steps I am going to test that my filter works but now we need to do something so I'm going to add a test step and the first thing we're gonna do is create a dashboard so I am gonna go in again I have loaded in the project from share so under responsive dashboards out of the box we have responsive dashboard sharing responsive dashboard visibility the project from share added in these other options you can also create your own custom custom ones as well for other things you're doing these ones are just going to be really reusable for all of us on this call so I'm going to create a responsive dashboard and again I don't have to be I don't have to know a lot to get this to work what its gonna ask me are the key things to create my dashboard so this is the first thing it's going to do I have my order and I can change the order later on I'm gonna give it a name so my name for my dashboard is my test dashboard I'm very creative in the morning the owner I do need an owner I'm not gonna set it I'm just gonna let it be sysadmin I don't care that's not actually part of my test so it's just admins fine and you can specify here do you want to share this dashboard with certain specific users with specific groups with specific roles so a role I might not create a new one because I'm actually testing there's a role matter but a group or user I can create a new group or user those are out-of-the-box tests so create a user you want to impersonate the user oh that's already supported that is not needed for our test or the tests we are doing today it doesn't have to do with security doesn't have to do with users we are just testing does this filter work so I'm not gonna set anything else this is all afternoon so I'm gonna I'm gonna submit it and now I have my do just my new test step I generally put in notes to the describe it because these create responses - Borg is not necessarily the most descriptive of what I'm trying to test or what I'm trying to see so I'll normally want to put notes in but will will go so let's get that for right now so we're gonna create a responsive dashboard and then we need to add the filter to it the filter I'm going to add is my content block the filter the the interact custom interactive filter that I built is a is a dynamic content lock so I'm going to add a Content block after step one create the dashboard which dashboard do I want to add this to well I actually don't know what the dashboard is this is a reference and we're gonna create it before so in a TF one of the things I'm able to do is instead of so I could I can select an existing dashboard but again that's not what I want to do so I'm going to click on this icon over here which allows me to get parameters so from step one creating a dashboard returns a reference to that dashboard so I'll go ahead and click on that and what it is saying is I'm going to create a gap I'm going to add this widget to this dashboard which is a variable from my previous one now the content block is not a variable we actually have a very specific content block we want to test and that is incidents short description right we saw before I created this before I want to add this very specific content block that I'm trying to test in short description to that dashboard good next and now I have that on the dashboard I want to have the debug widget because that's actually I'm gonna validate did my interactive filter show it because interactive filters I'll be able to see what's going on so we're gonna go ahead and click Next and again what dashboard I want to add it to the same dashboard that we created in step 1 and now we are going to open the dashboard form so how am I going to do that I'm going to add a test step and I am going to go to form open any existing record so again I don't have to know where I'm clicking what's going on I just know I need to open an existing record I'm gonna do that after step 3 what record do I want to open checking all the tables right now which is taking a little bit of time we and I want to point out here so we have workspace and standard UI so again you don't have to worry about what some fully qualified Dom address is of anything I just want it in the standard UI I want to open this record and again I could choose I want to open workspace or the standard UI the record I'm opening up is in PA dashboards I want to open a dashboard if there is a warning that if I change this I actually might have a problem if I'm running in parallel in this case it's okay but the design considerations are important things to read about what record again dynamic record so I'm going to open the dashboard that I created here a view doesn't matter gonna go ahead and submit that and now I want to view it there is not an out-of-the-box way right now to just say open this dashboard so this is how I open a dashboard I open open the record in the form and then I click on a UI action so I want to click a UI action after I've opened it so on the dashboard form there is a UI action which you I actually do I want in its view dashboard so we've opened up the form and now I want to click on view dashboard and we're testing actually everything is we're going is it is a test and we're testing doesn't work does it not work does something reload in this case clicking on it is all we need to do or make sure that it does submit so I have now I'm now viewing the dashboard and now I want to add text this is where it gets pretty exciting to do a live demo especially so I'm going to come in and go to custom UI and I want to set the component values on a custom UI I have my custom UI is my interactive filter oh so I have not I have not enabled running tests on here so I'm gonna go do that and we'll come back this is a global setting so I'm gonna go into global this is not a production instance so I'm gonna go ahead and say yes we're gonna run tests and lots of different options on how you see the data what you what you want to do by default everything you should work but you do have to turn on testing okay then we're gonna go back to my test going back to my test scope and now I'm gonna go back to my test so again we were now on the dashboard but I want to add text to it so we are going to go to the custom UI we're going to set component values what this needs to do now is read well what are the values that are that are optional so I'm going to retrieve the components so as I'm building this it's going to open up a new window a test runner so the test Runner is actually using my browser and it is going to go run through the test steps that I've told it to do so it is already gone and created my new dashboard it's added the the two widgets to the dashboard it's now on the form and it is clicking on the view dashboard related link the V dashboard UI action under related links and now what it's doing is it pulls up that dashboard it's going to examine well what's available so we have exactly we wanted to see we have the short description and we have the the debug widget and what it just did is revert and undid all those things it removed the tabs it deleted the dashboard and went back to normal so it's now retrieved everything and what it lets me do is go to next and now it's giving me a list of all the places I can put text and there's a lot of them on here does the dashboard has a lot of things going on so we are going to look for we're looking for the name of where I put in the search text which is custom filter short description is my name if I had looked at it before I'd see it and what do I want to put in there and what I'm gonna put in there something really unique in my test I don't want this to appear by accident so I'm gonna call this one two three four five six test test test six five four three two one I am I am hoping that this does not occur naturally somewhere else in my dashboard and we're gonna go and click Submit so we've added the search text now I need to filter so I'm gonna go ahead and add another test step we've set text now what we want to do is click a component I have to click the search button this is a behavior of my filter because in my filter itself I don't do it on change or on on focus loss I have to click a button to submit the filter so now this is going through executing the server side steps it's done that it's created the dashboard it's added that you created the tab out of the widgets the tab it's now going to run the UI set a set of this really needs to see what renders and it's testing your job your client-side JavaScript all at the same time and if you can see quickly it's gone now but it has it put that text in to where we want to go to and so now I've retrieved all the things I have rolled back I click on next and now what we're looking for is I'm actually going to look at what I called it's a search I should be looking for something called search I'm just gonna check the text of what is actually called ah so it's actually called it's a button and the button is called add custom filter so I just forgot what that is make sure you have your code ready so I'm going to add custom filter that's clicking the button and then the last thing that we need to do is we need to add another test step which is to validate that the debug filter that it actually shows up in the debug filter this is the really the final validation so we are going to assert text on a page and the text that we're going to assert I'm actually going to do it on here just I'm going to copy and paste exactly what I want to see so that there is no accident in this I am going to manually go through and add the debug widget here okay well it's not letting me do that so I'm gonna assert the text that I put in which was one two three four five six test test test six five four three two one now there is a trick here because this we are looking for text on a page it's actually in this case right now it's actually gonna show up because we typed this text in what I really want to put in here and it's just not letting me do right now and try one more thing I actually want to make sure that the the debug filter text is exactly correct so I'm gonna put in the text when I click search this is what I want to see exactly this so I'm I'm actually gonna search for just this string because this tells me if I see short description like that things are working I could look for this whole string but that this is enough because I'm only typing in that one two three four five six if I see a short description like that that text it means it's working so we're to make sure that that's there I'm going to submit so we're done we built our test now I want to run it so I'm going to run the test it's gonna ask me which which browser to pick so I in this session have created a test runner we launched a client test runner in Chrome but I could run this in Firefox in edge and Safari whatever whatever you have on your instance I'm gonna run the test and we're seeing it's executing on the server so it's creating the dashboard creating the tabs adding the widgets all those steps that we put in then it's going to go to the UI side and on the UI side it's brought up the form this will go quick it is putting in the text it's clicking search it's now looking for the text and debug filter and once it's found that it's gonna come back and revert just say over here till it's finished okay it's done it's now reverting oh it didn't like something we're not gonna completely debug it but let's take a look at what's going on I'm gonna go to the result so when I look at this result so it didn't find this text so that's it should have so I might have something quite a little bit wrong with the test or something's going on but what I get here is still really valuable for me so we've seen that the this took 30 seconds when I ran it that it failed I have all the things that did work and then when did it fail so if I didn't find my text because I changed the name of my button or something else you'd see where it failed and as everything that it did in the background to actually create my records and close them and do everything it needs to do a pretty detailed long about where it's going and what's going on the instance if you need to if you do need to debug where we tested it why why it failed and then screenshots all along the way one of the options is should I take screenshots when it works or can I take should I take screenshots only when it fails so it's all the time or just on failure one of the things that I like to do is to personalize this form in in Mike and global and actually add these as a list so I can view them all really easily but we'll go ahead and view what failed so it tells me this is what failed I can click on that and I don't know if you're gonna see what's shared here but it has a screenshot of what its of what it saw which actually looks like it should be there but I have again pictures it's taking pictures all along the way to see what did you see and what didn't you see you can roll these tests these tests will normally go into test Suites so when I'm when I have a sweet I can then build up all of my tests inside of it so I can have my text search test but I can roll that into a custom filters test which I sweet which I can roll into my responsive dashboard sweet which I can roll into my full regression sweet so again with one button I can run the test suite and it just goes through everything as fast as it can and get those answers and let me know where am I at right I said I want to see a lot of green and if not just point me exactly where it's failing and I don't have to recreate it I can go straight in look at that test look at those screenshots maybe run the test again and make sure it is it succeeded just executing now or it's failing when I make the fix that I need to fix I can then run that test again make sure it's running and then run the full test week a couple buttons and you have peace of mind okay so let's go back all right we seeing PowerPoint again I think that is yes yeah great okay so we went back in in the demo as we went through we went through these test steps to create our single test that was testing doesn't work which actually failed in this case but it allow us to go back and see what's going on I have to build something to test right we can't underestimate that I I am testing custom development here so I had a dynamic content block which was just some JavaScript and some jelly to give me what we were looking at then I went through created my test where I went through all the different test steps again to get these to make this really simple I wanted to use the project from share and then all I had to do is fill out the form you saw through all this testing we actually didn't write any code all we did is that I want a dashboard that does this I want to share it with this group and I can create a user add them to the to a certain group and then validate can that users see this I could remove them from the group and then validate can they not see it now - really good tests and then we're passing that information on in variables so it knows that this test step returns a dashboard when I create a user I get this this idea of a dashboard okay this and get to your questions we will post these links in the wrap-up with this but there's a lot of a lot of reference for you to see here so one you'll get the link to the dashboard test steps to load those in and as a note there are two update sets that are part of this project one is global to interact with the dashboards we actually have to install a global update set a global script include and then all of the test steps are in a scope tap there's no new table so there's no licensing implications with this this should just help you out training there's that fundamentals that ATF fundamentals that has a micro cert that goes along with it and now learning that's available to you couple labs a great one on ATF from K 20 that's only about an hour long or a hour and a half long so you can do that as well do both do one the the training and now learning is considerably longer than just the knowledge lab but you get a lot more out of it there is a ton of depth in ATF and then we talked about custom interactive filters there's a lab on that from K 19 that's available to you as well and then dasu documentation so ATF has a lot of documentation interactive filters custom interactive filters the debug filter and then just general script includes a couple of key pieces of Doc's that would help you walk through this okay so let's get to your questions so a question came in about used ATF for the health dashboards so the health dashboard actually can't predate ATF so we test those with a different framework that doesn't ship with it in our testing we test a lot of the Java a lot of the Java side too so we have a little bit more in-depth framework we use this to some extent but that requires a lot of infrastructure and a lot of a lot of a lot it's pretty in-depth with our QA Department ATF is a little bit lighter weight but allows you and your business users to do your the testing you need to have you absolutely could creates the CMDB the health dashboards you could test those in here and if you are customizing them that you do want to do that you absolutely want to do that to make sure your customizations don't break again really important if you're using undocumented api's that you found somewhere or something else right anything you're writing behaviors change we upgrade versions of you know other libraries so whether it was high charts or it's some JavaScript library that is upgraded over time if you are writing custom code you want to write at ATF tests to go with it if you're not using some other some other testing framework however I don't know of a better testing framework for custom ServiceNow code and we did talk about we focused in on the responsive dashboards the client-side but if I'm writing a script include I'm doing the same thing if I'm writing a dynamic filter option we talked about that a few weeks ago you want to write an ATF test that's just simply testing does my script include return what I think it should return right it should I want one that returns just managers so if I want to have that then I just on the server side you can write scripts that just call your script includes and make sure you know I inserted this user I you get that user back I deleted that user I don't get any users back all that stuff can be built in into ATF as well and again this isn't a good thing about my my coding practices but ATF always finds issues in my code no matter how good I think it is when I write the test and go through all of the edge cases I always find something that I'm missing so it's pretty essential to to having me have quality code can ATF test widgets and indicators and breakdowns how do I use this with PA so we did focus this on responsive dashboards and custom interactive filters you can do this with with everything else with PA we're looking into actually write writing some more that in two things things like run a job and make sure the job runs most of piays functionality we're testing so you don't have to worry about it if you're just configuring something you normally don't have to test it right and it's and we can test it we should test it we're responsible for fixing it we're ATF really comes into play is when you're customizing and customizing meaning you're writing a new script include script includes you're writing new business rules you're modifying the form when you're doing all those things cut dynamic content blocks client script you wrote it you need to test it because we just simply don't have access to it but does the job run we test those things does the breakdown matrix actually exclude things we test those since collecting records collect records we test that that's not to say you can't but you do need to you know I don't think you need to in most cases unless you're doing something pretty pretty special one area that I would look that we're looking at how do we make it easier to test would be PA scripts right pa scripts you're writing that logic so how how are you how are you able to test that you can do it today ATF allows you to read server scripts you can write anything but we are looking at how do we make it easier for you to do that because I do know the answer is if it's not easy to do you don't do it I didn't ask how many people on this call are running ATF today because I'm pretty sure I know the answer but we want to make it easy we want to empower the business users to do it and when I empower you to do it and I want to give you that peace of mind because I'll tell you I have it when I when I run these tests I know my code works I'm not worried about getting a call you know after we did an upgrade at 2:00 in the morning on a Saturday because something is not working right and it needs to be working by Monday morning security does work so yeah you can test anything right anything that you can click on or anything that you can do on the server side you you can test most of ATF you can you can click to impersonate a user so I can create a user and impersonate them so if I wanted to test elements security dashboard security that the responsive visibility is actually that does come out of the box so you certainly create a user put them in a group and validate can they view this dashboard again I've never seen that break because normally it's some nesting group or the person didn't a person didn't get the role that they were supposed to get right or the role wasn't it wasn't assigned to the group correctly that comes though from test using ATF to test your workflows to test your employee onboarding to test your Service Catalog items if you're testing everything that you're gonna have a lot of list of those problems the problems I see with security are typically a process breakdown somewhere along the way so yes you can test it but I'm not sure you're gonna find something but if you'd like to do it especially if you wanna get your hear your hands dirty and try something you know go at it go look at what's there and see what will help you sleep better at night okay well if there aren't any other questions about ATF we can open it up to any other questions you have about any other topic and if something comes to mind about ATF go ahead and throw in the Q&A Adam do you think it's the ATF is it comes more relevant as you mature in the use of the ServiceNow platform or is this for any user any any maturity level it's a very interesting question I I think it's it's probably more important the more immature you are but I've never it's always important any time you're writing custom stuff and again this is not I'm not creating a new indicator I don't need to test that but I'm writing a I'm running a scripted indicator or a scripted mapping I'm writing a client script or a custom interactive filter that's where this really becomes essential because most of the customers I talk to just they're manually testing it best and I know if you're just manually testing it means you're not testing right you test it when you wrote it but nobody has the time to go through a thousand dashboards and make sure there are if and look are they using anything custom and is it working and is it working for you this allows me to make sure everything works and and my kind of my philosophy is I'm not as upset if it breaks once I'm upset if it breaks twice in the same way right so if I do find an issue with it then I'm going to go through my tests and make sure that they're right but simply going through the thought exercise of writing your tests for your custom code makes your code better it just it just makes it better okay some more questions in every Dario first on the demo in the instructions for the test that you also added the step to add the report to the dashboard where the filter was going to search good questions so I actually did not add the report because you don't need to we're not validating the report a wonderful question so when I when I did the demo at the beginning I threw a report on there it had 460 incidents I applied my filter and I got 19 in theory I could look and go is it 19 right I could I could instead of adding a debug filter I could add that report and verify are there 19 the problem is is that that 19 is actually kind of hard to figure out as a human you can read it when I'm clicking through those Brout the UI things it's hard to tell where that number is and I even with that I'm scared of that because that 19 could change for many many reasons on my sub broad right there actually are there's now 20 there's a new incidence come in and my clone data so there's a lot of variables in there that could make it look like it broke even though it didn't break I could get around that if I created a bunch of incidents if I created an incident that have the word this is a test this will never exist in reality I created an incident that has that as the short description and then my filter searches for that text in the short description then I'd see one or not one or get one or zero I could do that to fill the debug filter I find it to be the easiest way because you actually see what the filter is you can see do our my mappings working correctly so the debug filter is what gives me insight into a working that didn't work in my demo for some reason but that's in real life when I write this I use the debug filter that and that tells me it's being applied again I'm not going to test was the report listening do reports handle that but because I trust the service now test that if I have a filter does it apply correctly service now test that the custom part of this code the smallest part that I wrote was do I get this in the dashboard message handling all right for the custom interactive filter so I didn't it was easier for me to see it but in the in the demo but in actually in my test I'm Eileen very heavily on the debug filter okay a couple more questions came in about integrations sure you can use an integrations a ATF works extremely well when you're testing there's some rest things that come in but testing server-side code is very clean very easy to get it to do exactly what you want if you notice the longest part when we were building this test I mean I think we built it in under 10 minutes the longest part of the test was when I had to run through the tests to go well what's on the screen what things do I want to see that does take some time because you have to run through everything you have to build it we have to parse it all but on the server side you don't have to do that the server side runs I think a whole server side a part of this which was actually more of our test took less than a second to run and then the other part took 29 seconds so anything you can do on the server side you can test yeah it works exceptionally well and will there be pre more pre-built 8ef like the content packs probably so we released the share for responsive dashboard it's a very much an ongoing discussion for us as we're giving you more abilities to build things we want to give you the ability to test things and just very simply it is in service now as best interests as well as yours but it's an our best interest to make sure your code doesn't break right so we want to give you all the tools and guidance so that you can make sure that your code works so you don't call us because we have a very vested interest in your success and if your custom code doesn't work you're probably going to call support we're gonna have a hard time debugging it's just a bad situation so the more tools the more things we can give you the test the better off we are we want you to use it you want we want you to want to use it and and it's good right porpora said today was just to get you guys thinking about it how can you use it okay and the share projects I appreciate it we are looking to release some of the stuff and share when it's that that can happen so you'll see a lot of stuff in store coming out across Holub service now and share is share is there as well if you don't look at share there's a lot of really interesting things in there a lot of snippets you know maybe a single script include to help you out so as you look at store store goes through more testing or some more scrutiny it's more supported in share it's actually not supported by us and you'll see the disclaimer about that but there's some really interesting things in share and if you're looking at which you'll find on you get a share that's service now calm your and share a lot of great stuff in there and if you're building something you're thinking about building something some utility take a look and share because it may not be there but maybe 90% of what you want is there so lots of good stuff and differences in Orlando in New York I did do this in Orlando I'm not aware of any changes between Orlando and New York or Paris for that matter I believe everything we did once you add in the share project should work back through probably even Madrid maybe before that ATF's been around for a while the biggest changes you see with with with ATF is that they go along with the with the version so if there's a change to how we insert records or something that would ATF is version with the families and then the test steps are the biggest things and those test steps everything I wrote should work fine with Orlando I'm sorry I tested it in Orlando it should work fine in New York you know make sure it sits in share we didn't go through full regression testing so if it works then go ahead and go ahead and apply it in New York it should it should work and if not ping me in the community ping put a comment and share and and I'll try to get this to work in New York I don't know why it would not work also with share make sure even if you don't install it now maybe go look at share subscribe to it share is a mechanism where if we add new test steps if we fix an issue so this is you know compatible in London and it wasn't about of on London or we need to fix it in Paris again I only did this in Orlando share is version so if as we put the new versions out there if you're subscribed you'll get an update saying hey there is a new version to make this work in Paris so make sure that if something is interesting you and share that you subscribe to it you leave your comments in there leave your ratings you know it's kind of a win-win for all of us but again be it's not officially supported it's not like it's coming straight from the store so we're here to help but you know we all have limited support as it goes forward okay we have just a couple minutes if there's anything else we can we can answer it if not w of the closing part of the deck I do and I'll try to hit the spacebar this time okay so show you some slides and I'll hit it back over to you dolly yeah if there are no more questions this week and we'll close it off nicely in time so let me just share my screen and start clicking away all right all right thanks again for joining us and we look forward to seeing you again in two weeks that would be July 15 if I'm not mistaken what I see a hundred you see it yeah all right good there was an empty slide that was my starting slide so I'm as you may know or may not be aware but all knowledge Tony labs or almost all are available in now learning including those for the now intelligence products so everything around reporting performance analytics dashboards even predictive intelligence are available through now learning and the biggest benefit is that you get to go and watch the presentation on it on each lab and you can actually do the lab you will be assigned an instance the instance will be made available for you for the time that you get a lot of it I think it's three days and then you can do all the exercises and all the exercises are also explained through a series of videos so don't forget basically brush up your skills and expand your knowledge with this free content upcoming sessions and the list in the article on the now community and as you know all previous recordings are also posted to YouTube and PA office hours and we still need your help right we still want to know what topics you would like to have discussed during office hours so don't be shy there's an article on the community that really deserves some extra comment comments so thanks for your help all right actually we just did this so this is a for this week's placeholder next up is actually Adam do we already know what the next topic I think we were still deciding now so well maybe a lace isolate metrics that would be a nice one huh I love sls s alia metrics yes who doesn't love them so thanks again everyone I hope you have a pleasant rest of your day and see you next time