e hi everyone um we'll be starting very soon we'll wait for a couple more guests to join before we um continue e welcome welcome welcome everyone if you can um put the the location you're coming you're joining the call from it to be great to get some um know some of you guys e e so we starting anywh from now I'm waiting for it to be exactly 5 minutes past um the top of the clock awesome thank you for that I think I saw that okay um yeah so we're going to get TI right now um we're going to have a couple of um one other colleague join me later on but right now we're going to talk about um automated test framework for beginners and before I get started I just want to know um the chat is disabled like I can like I can see right now that the chat is disabled well I would like to know um how many of you guys here I've experienced using ATF just so I know who my audience is right now this is supposed to be ATF for for beginners but I just want to know if some of you guys right now understand or have any idea what um ATF is all about you can just type it in the q& a oh awesome um Ian bti so you just end your micro um certificate that's that's I'm happy about that um so no experience that's good so you're the kind of audience we're looking for also um who else want to share their experience with ATF I'd like to make this as um interactive as possible so if you can just um drop your experience with ATF in the chat yeah very little experience with implementing I've bely experience okay awesome another interesting Insight oh um R PR you can just type it in the chat type it in the Q&A that's that works for everyone so we've got we've got three more okay all right um all right oh chat is disabled just just type just type in the Q and A um let's make this as interactive as possible you can type your your your questions or your comments in Q and A this way I can monitor everything and and um type the answers or answer them on or well talking oh someone says they can't hear right now um can everyone hear me I want to believe rest of this F can hear me yes that's correct so ATF makes upgrade testing easier with no much resources exactly you're right about that and ATF goes beyond just upgrade testing well we're trying to position ATF also as a development um testing tool so ATF think of ATF as that testing tool that helps you with upgrades and also helps you with development so the cicd all all those thank you vti um you can hear me oh you can also hear me fine all right good so we've gotten a couple of um um comment from comments and answers from you guys I'll just go right to start right now um please someone says please explain about test compared to test yeah I'll get to that I'm going to get to that um all your questions will be answered today so you guys are okay about the session so I'm going to go right onto this um save AB also um we'll be talking about some some some some new features um for coming up for ATF I just want you guys to know that this is a for looking state statement and um I wouldn't advise you guys to make any purchase decision based on this that's all safe no is for then I'll go I'll go into the agenda for today the the today's goal is to understand um service now ATF the goal is to understand okay how are we positioned the market um how are we trying to help you achieve your goal or achieve your testing goal then the the um basic the agenda today will be we're going to start with basic testing concept which I believe most of you all know um then head on right to getting started with ATF we're also going to do our hands on demo where I'm going to show you how to create your first test case in this case we're talking about a test case that consists of several test steps so we're also going to talk about um working with t suit I'm going to also describe and show you how that works I'll also share a couple of best practices and then we're going to move to Q and A so I'll try and make this as interactive as possible um and um I hope you guys can uh help me make this experience as as wonderful as possible so now um I've got um my first slide here but prior to this first slide I want to ask a question which I want you guys to help type in the answer this will help me understand um uh my LG basically so um the first one being have you used any testing to before so if you have used any testing to before just type yes or no I would like to know this have you used any testing to before so you can you can you can type in your answers if you've used any testing to before okay I can see no no no no no good keep it coming in keep it coming in guys's let's try and make this as as interactive as possible I already have four answers and I would like to get answers from the rest of you so if you've used um a testing tool before can you indicate yes or no okay I guess I guess some of you guys prefer the poll so uh I'm going to make this very easy for you all right seems we have um most answers in um most answers are saying no so that's good so um the the basic idea of testing is is that you want to test you the goal of software application testing is to identify the effect ensure and ensure quality and validate functionality that's basically what you want to do with testing and um historically people are actually tested using um or just opening the application and running through the steps or the flows are involved but um what what we try to make um we we try responsible to make testing as automated which is the whole point of automated test framework which is ATF so what are the pros and cons or what what what are the pros and cons of using all this for example for manual testing it's basically conducted by aans people like you and I are the ones that on manual testing um and it's basically stable for exploratory and usability and adog testing so what that means is that oh you just want to explore something it's not something you want to repeat again so that's when you do man testing but for automated testing we're talking about about a two based testing software there are various testing software out there if you want use uh so there are various testing software out there and um this helps you achieve your goal of of application testing and to identify the effect so um another another Pro of automated testing is it's you can it's repetitive and and you can actually use it for regression testing what that means is that you can schedule your test to run as as as time fit you can also use it to for regression testing so what does regression testing means it means once you develop a software once you customize your application you want to know if when you're moving on to the next family release do you still have the same functionality it is still working as as the way should be you understand that that's the whole point of regression testing and another benefit of automated testing is that enhances speed you can just by the click of a button you can run your test and have your test result it also helps with efficiency and cover so those are the the the um things you should know about now I'm going to ask another I want to ask another question um you can either type your answer in the Q a box or just respond to the poll which I'm going to share right now so um the question is what type of um testing are you familiar with so what type of are you so basically manual testing automated testing or you can tell me both so what type of testing are you familiar with if you can just also type that in the in the chat that would be good sorry in the Q&A box oh we got one answer from Manuel anybody else don't be shy keep be coming guys keep be coming another manual that's good that's good we like to see that also another manual so looks like everybody that's responded so far at least they're doing testing that's good manual testing is also good but we're trying to move you from the manual testing all the way to to automated testing so um if some of you guys can so the the question is which type of testing are you familiar with and the options are manual testing or automated testing all right so I I can see most of the folks here are all oh I can see developer here yes 200 plus tests wow interesting so and if you can share your insights like um how do you do you use Exel sheets to to manage your test or how do you ensure all this test are organized you can just you can just type it in in the Q&A also that that works oh I see Exel good good good so one of the things we're looking forward to to in ATF is a way for manual testers to convert your test automated test so you just fed in your Exel sheet and what you get is um a bunch of automated test based off of your manual test so those are the things we're going to talk about oh awesome I see everyone uses Exel good good good so I'm I'm glad to hear about that so we're going to move to the next slide right now and like I already mentioned earlier one of the benefits of automated test is that you get to be efficient when you when you're executing your test you are also going to more accurate because you you're going to reduce human error as a result of manual testing and another benefit is usability you can always use this test in multiple family releases so that mean the test you created in in your Tokyo release can be used for for use in your Vancouver release and so on and so forth the another benefit also is coverage your um when we talk about coverage we're saying that your test can actually cover different use basically you can test forms you can actually test um work space or as the case may be so those are the benefits of automated testing it helps you with coverage and it helps you to achieve more with less time so now let's talk about ATF now this is what we all year for and before we go into um ATF I have a question I also want to ask again so I see CH here oh nice good so um the question the next question I have here is which feature of a testing tool do you find most important the first one is it EAS of use the second one is it reporting or the third one is it integration or the fourth one is it automation capabilities I'm read the P are not working so I'll just go right on to um what is ATF so ATF ATF is a service Now application that's purposely built for automated functional testing and the whole point of this is we're trying to help you achieve more by doing regression testing and one of the benefits of automated test is that you don't have to be a developer to do testing so we run no code testing and another benefit also is the fact that you get to put all your test cases inside the test suit so what that means is that if you have all test that has to do with itm you can all you can Bunch them all on T suit and them at the same time um bre do you want to introduce Lisa oh sure um hi everybody uh Lisa is one of our um tech leads in the community so she's here to help us with any technical issues that we have and um yeah all right thank you yeah welcome Lisa so um we continue with one of the some some of the benefit of ATF and the next one is the fact that you can actually roll back test data so what that means is that if you have a bunch of data that was created from the test your hand you don't have to worry about the data affecting your Pro Data because in fact what we advise customers is that um you should always run your testing in s and not prod environment so testing should be done in s prod and once you're done doing testing s you can actually move to um production for for um for for your live instances the next one is the fact that we have over 700 plus quick start test so what does Quick Start test means we know people find it difficult using ATF sorry using testing tool so what we what we try to do is for any product you have on service St we ship things are called quick start test so quick start test are the first kind of test that will help you understand how test cases are built in ATF so what that means is that you can see some of these test cases and modify them as you see it or edit them the way you want so that's the benefit of Quick Start test and we have over 700 plus quick start test in ATF the the other one is the fact that you can reduce overhead of manual testing like you you don't need more and directly R your test with man testing you need so many um um manow on man test the the next one also is the fact that you get your results very fast like running your test ensures that you can get your result fast and you can find the place where where you can monitor all the the the results you got or if the if the test feel or they successful so why should you use ATF number one decrease the risk during implementation and upgrade like you want to ensure that when you're moving to the next release all your applications all your customizations are intact you don't want a sitation where you done your upgrade and you have issues with your customizations you want want to avoid that the the next one is you want to R the need for third party testing software so you you want to ensure that you are not leaving the service star environment and you're doing all your testing with service now because you know you're going to get the best and you know you going see the best experience the third one being you want decrease the cost of cost to change it's very expensive when you're when you're changing or when you're when you're trying to um run a testing tool so you want to ensure that you have you using the same testing tool so you can keep building on your test cases the the next one also is you want to save time during testing like I said man testing takes a low of sign when you start running them on Exel sheet so it's um it's it's strous and it's it takes a lot of effort but with automated test you are well assured that you are saving a lot of time now how do you use how to how do you use ATF this chart basically just shows you um how you move from um changing your Dev instance all the way to prod so from the left you're going to see oh new implementations for bug fixes and the likes and when you detect um issues or detect bugs in them you try and fix them before you upgrade or before you move them over to prod so you see you move all the way to build HF test in Dev instance then you execute the test in Dev instance then you m migrate the ATF test to your test instance then you also execute the ATF test in test instance and migrate ATF test to Pro so this is the the cycle of how to use ATF which I'm going to share um some pages in this deck so you can have access to it um on your own time so now today we're going to start with the anzone demo um I think I I saw some people talk about um test cases and likes but before we get to the Anon demo um does anyone have any question that they would like to us too and folks feel free to come off mute and ask questions um oena is the the expert here on ATF this is your opportunity to interact with him so don't be shy so if you can raise your hand then um we'll um allow you to talk so you can contribute to this is everybody just drinking from the fire hose here I know this is a lot of great information he's sharing S I think be nice also maybe Lisa can actually share some of our experience with testing and what she has seen that works good for for her testing team oh rir yeah um did you want to share any of your um testing experience it sounds like quite a bit uh of course thanks for um yeah giving me the chance so I would love to share my experience so I have created around more than 200 plus test scripts so for um for various clients earlier I was working for a service now partner so um so we have created a lot of uh test scripts for um incident problem change the uh itm uh module on top of CSM modules on top of the customer custom applications so especially I would love to share my custom application test script which having around 125 test steps the overall application is huge we were around um more than 12 months we worked on the development um so first we have set up the application uh so what we were thinking is the sequence or the overall functionality should not break on adding the modules which in the custom application we did in the ADF um sorry in um CSM um it's a application so um how the credit card um initiated applying everything we have personated with the agent customer again agent so all the four phases we have covered in the ATF test script so every releases we when we add on the um the functionality we run the ATF and ensure the overall architecture is not breaking that was the um I would say it's a huge um um benefit or W uh the ATF which we were um working on uh on top of that um I have worked on itm all um um incident problem change every um the overall functionality which should not break the business should not break uh in the prod what we are thinking so whenever the customer comes it will create one incident and then it will go to the fulfiller so it will go for approval all the process we captur in the ADF so whenever it is going for a upgrade it will run automatically and ensure it is not breaking or if it is breaking we will go and find the test result and we will figure out why it got test we'll go investigate what got changed what should we need attention on somewhere so that's how we have used I'm so happy to share and I can uh elaborate more more how we created um so I can do more deeper because I have a lot of experience on that we have created lot of Uh custom steps also um there is a mean every release uh we I'm working on it past three years I I can uh see the advancement and the new features adding every time it is amazing that's that's that's great for he like your experience is um very very interesting and considering the fact that you have over 120 test steps in the in the test case that's that's a lot that's massive does anyone else want to share their experience with um testing or mal testing or the case may be I guess we had a shy group here today oinka I can I can see that so uh the next I'm going to do right now I'm going to share my screen so I can run us through um creating um test cases in the at okay I'm [Music] switching all right so good I I've got my instance running so right now the first thing we're going to do is sping screen all right good so the first thing we're going to do is to I think I I saw earlier someone asked about um um products having test that that shipped to them so yes so I'm going to show you where to get them from so um as usual if you know how to navigate like I'm sure you know how to navigate through U through this um I'll type it here just to find this bring this out and this is everything you have under ATF you the only thing you have to do is to activate the plugin you don't have to do more than that so for for the quick start test I was referring to what you do is you find them in quick start search here and once this loads up you can see the test I I'll let me bring in a new column that as the package the package thing so on this side forward so beside the name you can see the different packages and the respective um quick start test or quick start start suit so if I decide to group by packages you would see all the packages I have on this instance and all the number of test or number of Quick Start suits that are within them for predictive intelligence I have for um service level management I've got one so these are the standard quick start suit that you would find in any ATF enabled on your instance now let's go to this particular one now let's try and just understand how the the test cases are built now for this particular test SE we open these are the tests inside so we've got three main test in this in this suit like I said test suit is a compilation or a combination of several test cases now the first test case in this test series is this one and what you would see is that this particular test case adds 16 test steps for respect each of this column so each of these row are what we call test steps all these are test steps and what that means is that inside a test case you would find test St inside the test suit you would find test case so that's how ATF is built and some of you mentioned that you typically use Excel sheet for for for managing your manual test so imagine a situation where the first the first true is that oh this par test or this manual test is service level management test and these are the steps you're going to take or these are the steps the person has to take to ensure that this this test is running perfectly and well so these are all the things you can do in your quick start shoot now I'm going to go to since we've seen all that I now want to take us through setting up your own test case or using an already created test to modify your own test case so all you have to do is navigate to test then you would find all the test that are shied with your application now what you'll notice that some of this have true and some some of this have false so the false means that um these are quick start quick start test which you can copy because they inactive so for example if I want to run this I can't run this test right now there's no run around there and because this test is not active I can't use it so all you have to do is to copy this test once you have the test copied it will also take the demo data with it oh I didn't I didn't change the scope so let me find the one that has the global um scope then I can I can do the same I just said um through matching I guess they all have um a different scope uh all right so um let me let me do let me do that for this particular one that is in the global scope so we know um what what to typically do so once you find um a test you're interested in all you have to do is to copy the test once you copy the test you can save it with whatever name you want or we use copy in this case then you update it once you have it updated you have the new test here then you can all go back into the test and whatever um additional test step you want to add you can add it to this now let me let me walk you through what what I mean by test steps remember I said in each test case you're going to have test steps so um to add more test steps this is what you do uh um you click on the the ADD test button and if you're interested in whatever however your flow is you can get it in this so like I said I want to make this as interactive as possible so if someone can that uses manual test can come off mute and tell me maybe a simple test case there on and we can quickly build um a test case here just right here in front is anyone ready to do that just come off mute um let's build a simple test case using some of the manual test you might have running and I'll show you how to do do that in short s St there an opportunity if you're trying to build some test to walk through it here so again don't be shy is there anyone willing to take my offer to come off mute so we can be maybe um we can try opening a new incident form and try to fill in all mandatory fields and then submit if it's accessible or not okay all right so let's do that then so um the first thing you do is go to the right the top right corner you can be your test anyway but you can go to the top right corner so um we talking so I can see your name and that's Ramia okay so I'm I'm going to make the the test in your name okay all right so that's the first one um you save once you have it saved so what what the first thing you want to do we can open one incident form and fill all mandatory details all right good so we want to open an incident form and F mandatory form all right so the first thing you do is we need a user for that case right we need we need a user so um what you do you can either imp impersonate a user or create a user but we tyly advise customers to always use the create a user um test steps so we use this then um we want to put in the name of the user uh I'm going to use user name also okay um you don't need to fill all this unless you have rules particular rules you want want to use so you don't to fill all this um going to submit you can either leave the Imp personate on or leave it off but right now we leave it leave it off we don't need it so the next thing is we want to open an incident form so you type in the form so um open the new form that's what we want to do so should we use the standard UI or should we use the of or I guess everyone uses um the standard UI so we're going to stick with that the table she mentioned is incident table so let's pick one of the incident tables there submit that's step two the next step is so once you click on for forms you can see every single thing that we forms but right now what we want to do is to is to add those [Music] um those required um values so I think what we're going to do right now is to validate so we going do validate um this we don't want to do record validation we just want to validate that you can insert all this fi value Val validation that's what we're going to do so I'll click on this so um what what are the required one we should do um let's try and do you want to tell me some some M pre you want to F I'm here any mandatory like maybe category um we can do assign to and sign to remember we created a user in the first place now what this button does for you is that it helps you it's a Glide variable the the the the variable you created earlier you can always locate it now if you look at the remember our our first test step we did create a user so this is what you're going to use for this you click on this and you automatically taking that variable we took it we added in the first place now you can use this but button also to add more more fields to it um let's say we want to also um fill color we can also use the same thing you say oh that's the same we we use that as color depending on how your case is set that's what you can do we can also decide to say um let me see what's the next business business impact um starts with um PR call so I guess you can do all this and submit and when you have it submitted you see the description of every single thing you've done here you can see every every single thing you've done here and once you're done with that this is a simple test case that we just created using three steps if you if you notice what what you have here you have the the execution order so if you decide to P this two it automatically changes to two and you can change this other one to three to change the to change how you want it to to look or Let me refresh so this gets changed and you can see the open new form becomes the third one but that's not it should be we want to open the form first before um Feud validation so I'll do this then do refresh and we get our previous order so right now I can now run this test if I run this test you're going to see um either if you if you should run it in test Journal you run it in Cloud Journal I'm going to talk about Cloud now which is uh new feure or more like something we came up with we released back in Tokyo that has been very influential instrumental in helping customers achieve more with ATF test I'm going on this test right now and you'll see um your typical windows or your typical forms in ATF you can you can see it's running the test is running you can see the screenshot the the image of the forms and how we filling them and now our test is done running so once your test F you can always go back to see okay what's the issue what did I miss so that's one of the is the the the good things about ATF you can always go back and see what did you miss what did you did you open the ranking or light so you go to test results this is live debug debug you life and see what the issue is so now it's telling us there's an this and we can open this to see what the issue is so go to test log so it's saying the form did not match the condition assigned to this color this and so I I guess while we were putting some of the required field it didn't work so um we can go back to the test steps and modify since we now have more information about what we try to do so let me go back to the test then we can edit this so let me take this out I think what I should add um I should add short description like I said remember I said that we're supposed to this is field field values validation so what we're supposed to do we're supposed to set the field values and not validate the field values you only validate when you're trying to confirm the the the content of the text and not the content of the form and not validate so what we're supposed to do here is to change this this um test step into set C values you understand so my mistake over there and I'm going to take I'm going I'm going to delete this I'll delete this test steps and write and add the the correct one so um let me add it so what we're supposed to do is us just set field values to insert the test we want so there are different test like I said you can use one to validate you can use one to to set the test now um like suggested earlier um what are the mandatory form should we should we Fu them now this is when we can get to F all this the M mandatory um fields in the form then we can use this Glide record to take the user and if you want to add any other field any other you can use this but these are the two um fields we interested in we'll submit them and let's R our test again and yet we have our test with to run uh so what that's running oena um we're getting short on time I don't know if folks want to stay on a little or if um you know you want to move on yeah so I'm going so since we we've seen how you can convert your manual test into automated test we can I'm going to stop um the anone right now and move on to the next part of our uh our presentation for today um let me share my screen so we can go back to all right so um what are the leading practices within ATF the first one is always validate existing test before you upgrade server side execution is faster screenshots on test step failure so you can generate screenshots as much as possible if you want them to be generated on test failure you can also use consistent use consistent test users so you don't want situation where you're changing always changing the test users or changing the the users for the test so you're using the same test users that has um consistent rules and and groups um um I don't think you're sharing the right screen oh I sorry uh sorry about that no worries I know um this is our first go around with the ask the expert reboot so apologize for some of the hinessmithsonian yes please okay my screen is starting to active all right hold on just a second here um many questions here from Zoom there we go okay all right thank you very much sure um the leading practices you want to validate existing test before you upgrade um you should know that server side execution is always very faster um screenshots on test test results you can generate screenshots as you want I know some of you want to generate screenshots for for your report or Li so you can use ATF test to generate um screenshots um you can also you should test instance functionality and not data although you can do parameterized testing which is um using to test um test mission critical functionality test best sorry yeah it's fine I guess I was hovering Too Close over my mouth it's right so um I'm going to share a link also so you can see additional best practices for ATF and um um help that can actually help you in your test cases so um what should the next thing we'll talk about which I'm going to run through is what we call the test generator and Cloud Journal so that what testator and cl Cloud Journal does for you is that it goes through your instance behavior and generate series of ATF test for you so these are more like disposable test test you generate by asking by by selecting particular tables you're interested in or selecting the entire table um you can activate this or install this by going to the store it's straightforward just by the click of a button you get test generator and Cloud R and um one thing you should know also is that um it's very easy and simple to do once you have test generator and Cloud you get two lanes what that means is that you can run your test as twice as fast but when you when you start queing up test it takes time right but um test generator helps you ensure that your test are running faster than your typical um client test R the next slide so the cloud R is is infrastructure be um yeah Cloud infrastructure where you get to run your test so basically what we're saying is that instead of you running your test on your on your on your browser or your device you can run your test on a dedicated Cloud infrastructure just for your instance and this ensures that you run your test faster and and very effective on the four you get two lanes but if you want to run your test faster you can get additional Lanes it comes at the course you might have to talk to your account executive for that um I think we can we can jump all this and just go to go straight questions so let have some questions let's have some live questions so we can get them answered you also have a community expert here Lisa so also help with our Insight uh I'm gonna butcher this name aogen said can you create multiple users and run the same tus for all of them for example what the results would be like for people with different roles locations Etc yeah you can do that in fact that's that's one of the things we call parameterized test so you you can do that um I'm going to share we have a couple of videos on that would help you easily we call them work through series that will help you easily get all this done so I'm going to share this but just know you can do that yes all right anybody else with any questions here and it doesn't have to be to what was uh displayed um you know the purpose of this session is for you to really interact with our experts and also um you know if it's a question that he can't answer here then that's something you know you can always come back with we'll post this recording and some of the slides from the deck on the community and um we'll continue to interact with this topic this is a very quiet group I thought there'd be a lot of questions today so we we got five minutes left here if there's no questions OA do you want to move into what the Washington DC release um new components or changed changes are yes please yeah let's do that so um we we're run through the new features we have in Washington DC and the first one being configurable workspace I'm sure you all know about configurable workspace prior to Washington ATF didn't support um testing in this but right now you can actually test form related um um test on on ATF right now and the these are the following test step are supported you can open a new form you can open an existing record you can set field values you can validate field values and all the likes this is available on the when you upgrade your instance to Washington DC um the next one so we also have performance testing and performance testing is very important in the sense that you there are times when you make changes to your customizations you want to know what's making your test run longer so with performance testing this gives you like a Pinpoint location of what are the things that that are affecting your test and why are they taking longer to run so that's the whole point of apart testing this also comes on Thea on on Washington DC and you need to have enabled or install Cloud J and test generator for you to have your performance testing um we don't have enough time I would have shown you a couple of demos on how to how these two new features are working but I think that's all for Washington DC in um zanadu and other future releases we are trying to ensure that customers like you can convert manual test to automated test we know you have some of your test um hosted on on Exel sheet but we want to want you to be able to easily convert this test to automated test as easy as possible yeah um really I think I'm done with that right yeah yeah we're not going to have time for this um whoops um let me keep moving here um and then here's some additional resources I know OA posted a lot of these in the chat already but again we'll be sharing this with you after the the recording it becomes available and really appreciate all of you coming on to our first ask the experts sorry we got off to a bit of a shaky start here with the technology and everything and hope you'll bear with us and come back to another one in the future uh we will be looking to run an ATF for more advanced users later on so if um this was too re remedial for you we'll have another session and then also of course look for additional ask the expert sessions um that will be coming out soon um so if there's uh one more chance we got two more minutes here but there's any last minute questions anybody wants to ask um please feel free otherwise we will call it a wrap for today all right well thank thank you everyone for coming and hope to see you at another ask the experts real soon thank you everyone bye