Process Mining Academy - Linking ServiceNow and external process data to get end-to-end visibility
well welcome everyone to today's process mining Academy session today we'll be focusing on how to get endtoend visibility into processes that contain both service now process data and process data that's generated by applications outside of service now so the ability to combine service now and external process data and a multi-dimensional map is something that we added in the Washington release so today in the session we'll go from importing that external data to creating the combined visual a process map but of of course first we need to get through the logistics uh of the session so the purpose of these sessions are to help us all get better educated on service Nows in platform process mining capabilities I will provide howto like today's session some practical guidance and use cases product updates and just as a group exchange ideas and collaborate so we all continue to get better how the sessions worked well we have a schedule topic that we cover usually for the first 30 minutes I think today might be a little bit shorter than that then we'll open up for questions on the chosen topic for the session then we'll open up for questions above and beyond the topic it's sort of like an office hours and I mention office hours just so you know we're we're starting up an official office hours I believe the first one is scheduled for next week uh that is an event that's on the community site as well so if you haven't registered for that go check that out we'll be hosting these office hours monthly uh the sessions get recorded and eventually posted on the community uh and finally feel free to post your questions in the Q&A session as I go through the cont content I do my best to answer them here today but if I can't I usually follow up and post the answers on the process mining Community or in some cases we actually use them as a a future topic for one of these sessions um so I think most of you have that are here today have actually realized that the registration is moved uh onto a live on now registration but if you happen to be watching the recording and you want to register for future events just go out to the community events page and register for the for the new invite uh this is the slide that says hey anything I say and do here today can't be held against me in the court of law it's also the slide that says it's part of these conversations if I happen to make any forward-looking statements about things that might be coming in future releases you should take them as forward-looking statements make no purchasing decisions on purchasing decisions based on them whatsoever because things are always subject to change and I'm assuming you've read the fine print uh for those that have not met before my name is Dan Grady I'm part of the uh process mining product team here at service now what we'll be doing today is we'll do that quick little process mining 101 for the newcomers then we'll dive into the content I got a couple of slides up front just to kind of highlight uh what's different and things you need to be aware of if you want to go through and Link service now data and external process data into multi-dimensional map and then we'll jump right into the demonstration and like I said we'll walk end end uh through the process with a very small little data set and a simple small application that I created on the platform just just for this purpose we'll get your questions answered and and then we'll just wrap up with some additional resources and we'll Al also touch on um all of the knowledge sessions that are coming up in a couple of weeks for those that are attending that are focused on process mining as always we'll start with the quick process mining 101 for our newcomers so with everything we do there's a designed and desired path for in our minds for how it should work and how things should play out whether it be planning an event like these Academy sessions or a workflow and when you design things you design for efficiency as well as completeness to provide the best experience possible for as many people as possible now unfortunately what we design isn't always what ends up happening in reality the reality is not all the work is going to be flowing through the the optimal path and that's going to have a negative impact on both the experience people requesting Services have are having as well as the people that are trying to deliver that service the challenge is identifying what is actually happening within our process CES in our workflows and then improving them isn't always easy what process mining allows us to do is use the audit log data that's generated as records move through a given process to create a visual representation of what is actually happening within that process and then with this new level of visibility this is going to help us accelerate our ability to identify inefficiencies non-conformant process activities and Improvement opportunities essentially process mining gives us the ability to answer process questions that have historic ially been pretty challenging for most of us to answer so where traditional analytics will allow us to answer a lot of the what questions about our processes process mining helps us answer the why so things like where is the process getting stuck where is there any sort of unnecessary rework happening where are incidents cases anything really ping ponging between different groups or teams and where aren't we conforming to what we've actually designed these are all things that have historically been difficult to answer and act upon and now they're available to us in service now with just a couple of clicks we like to say process money is going to get you to the why behind all of your kpis and when we do this right it empowers everyone involved in the process to make more data driven decisions about which improvements to make and which ones maybe we put on the back burner right there's always no shortage of opportunities to improve you need data to help you prioritize which ones to to tackle first or which opportunities to tackle first all right today's topic so um if you've joined us for other sessions you know one of our service now process mining goals is to provide endtoend visibility into all your critical business processes sometimes those processes live in service now sometimes they don't sometimes they start in service now and have some steps in another application and then come back to service now well as of the Washington release uh we have coverage for all three of these scenarios and it's it's been a progression for us to get to this point so in the Tokyo release we made the leap from single a single table process view to multi-process view by giving you the ability to link parent and child tables or reference tables when creating a process model and and that allowed us to get end to-end visibility into processes that were solely contained and ran only inside of service now so like really it's a really powerful capability for workflows like uh requests in the service now platform and if you think about where the bottlenecks are in your typical request process uh you have the approval step which usually lives at the requested level and then you have um any sort of delays that live maybe in the Fulfillment task themselves right and those those live in or or they're two different tables on the service now platform right requested item and SE tasks so multi-dimensional mining or Maps give us the ability to visualize these things together to see exactly where in the request process are we leaking time or losing time losing productivity another Prime opportunity for these multi-dimensional Maps would be an HR onboarding process where we have a apparent onboarding life cycle event and then a whole bunch of activity sets with subtests and requests that that kind of Farm off of them or or Branch off of them that's probably not Farm branch is the right word um and very often an onboarding process is going to have steps that involve other applications in addition to service now so we needed to continue to enhance the solution and start to incorporate more processed data Beyond service now and that's what we we did in the Vancouver release in Vancouver we introduced the ability for customers to import external process data through an automation engine and integration Hub and then apply our in platform process mining Solution on top of that external data so one example of that it would be a hiring process this happens to be data that we can pull from let's say a smart recruiter uh systems we pull the audit log data from Smart recruiters pump it into service now and apply process mining on top of that or maybe you've got a find process that's living inside of sap again we can pull the data from sap pump it into service now and apply our process Mone to that now well this is a super interesting use case if you've not already seen it our now on now team here at service now recently posted a recording of how they used Erp canvas flow designer and process mining together to get visibility into internal fi into an internal Finance process and you if you haven't checked out this recording yet it it's it's a good like hole is the whole is greater than the sum of its parts plat service now platform story I highly encourage you to check it out it's available on the community site just search out that service now process mining for sap and uh you'll find it and then now in the Washington DC release we bring the two capabilities together the ability to combine service now and external process data in a single multi-dimensional process map so we can get that endtoend visibility into these these hybrid processes that cross systems all right so I'm just going to run through a few more slides to highlight the important changes or differences from what you may have been doing from an external data and multi-dimensional map perspective individually in Prior releases um and give you what you'll need to know if you want to create that M map that involves both service now and external process data and then we'll get into the instance and run through the process start to finish with a small sample data set that I created let's just start with the external data so in order to mine external process data we need to acquire that data from that external application very often that the data is in a form of a data extract a CSV or an Excel file but if you have some uh integration Hub skills you could go direct to that data source like that sap example I was mentioning a little bit earlier uh we then have a step-by-step guide guided experience that puts the audit log data in the correct place on the service now platform and makes it available for process mining then once it's there you just create your process mining project and in minutes you have visibility into your process and like I said we're going to do that here today the key piece of this this process is that audit log data um and there are three core pieces of information necessary for process mining uh you need some a case ID some or some sort of unique identifier for the record could be a claim number a patient ID an employee ID or a case number of some sort then you need an activity essentially what the activity is is the thing that is changing over time that we want to track usually this is some sort of status or state value uh but as we often do for Native service now workflows perhaps you want to look at how things are moving between teams or groups that would be your activity or could be an activity for you and then the last core value that we need is a time stamp for when that activity value has changed if you can produce those three pieces of information from an external system we can visualize your process and then highlight bottlenecks and inefficiencies with process mining now to link external process data to internal process data we need some sort of reference value to the record that we want to connect to in service now an incident number a case number request number maybe uh this is the main difference between what you've been doing and what you need to do if you want to link internal and external process data now in addition to those core data points you might want to augment the data set with some contextual information that be will be able to use as part of your analysis things like categories regions types vendors Etc uh for those familiar with our process mining solution these will be values that we'll use as breakdowns in the workspace once we mine the data so once you have a file that looks like this the rest of the process is pretty much a piece of cake um are there other ways to format the data to include multiple activity values yeah but that's like a a topic for a whole another session but this is the key piece if you can produce a file or a data set that looks like this we're we're 80% of the way to where we need to be now with your process event data in in hand you or somebody with the process mining Academy role will go through the process mining workspace to step through the process of loading up that data you'll give you external data set a name you'll then be given instructions on which Fields we're creating for you by default and why essentially what we just walked through in that uh last slide then you'll have the option to add those additional Fields whereas before we were primarily entering string Fields now you have to make sure that you configure that reference field linkage um so uh to the internal table once you've configured the import table you'll be directed to integration Hub to handle the mapping and importing of your data into that import table that was just magically created for you and once the data is loaded into the import table the guided setup completes and the process um there's a process that formats the data on the platform so you can mine it like any other process that is native to service now now this is the the the key step in that process that uh what it will look like essentially when you're adding your additional Fields above and beyond the case the activity and the timestamp that we create for you to your import set table you need to set up this reference link so it's obviously handy to know which table and field you want to use as your reference before you start start this process once the data is loaded up and we're going to create our project um and we'll do this live but I wanted to highlight just this one area here uh before we get into the live demonstration more of as a reference point right um you're likely going to be using your your service now table as your parent uh then you'll choose to configure your child table the first thing and sometimes I forget to make sure that I do this myself is you need to change the source of the data from table to external data right there's a specific flag so when you're choosing which data you want to mine you're going to have things like table archive data report source and then one of those options external data make sure you choose this external data option I've driven myself crazy in the past looking for the the table that I know I just loaded up and it's not there and it's because I haven't chosen that that Source table um and so when you switch that you get access to the right list of tables um then you need to choose kind of whether it's going to be a what relationship you want to use and this in 90% or 99.9% of the cases for external data is going to be a child relationship that you're going to choose here and then you need to choose the source or reference field from your parent table that you you want to link to um and this is most likely going to be the unique identifier or the number ID for that field in the table um and then the one is basically the one that has the value that matches the reference column in the external data set that you loaded up that's it that's that's the magic all right uh once you have that configured and you add your activities and breakdowns just like you do for anything else you mine your data and you'll have your visualized process that contains both service now and external data so let's uh jump on into the instance and do it together live uh just so you can see real quick here's my little data set that we're going to use from an external data perspective we have our unique identifier field we have our activities and the times that those activities are changing we have our related tasks so this is going to be my field that I'm going to reference to and then I have my just additional categorical field of vendor so that's that's the data we're going to load up I kept it small so the load time would be super fast um when we go through the process we can get it all done today so step one let's go to our external data integration we're in the process mining workspace if you've not done this before you come here you go to process mining workspace if you have enabled the external data plugin you'll have this option to choose external data sets and we're going to create an external data set and we'll call this uh Academy external data set and we'll say create data set it just tells me that when we did that it created a an import set table for me that has the case ID our event values and our time stamps and then next we have to choose the additional fields that we add have to add well one will be that vendor field and then we need to add that reference field right that that one that had the reference to the numbers in the application on the platform I'll choose reference here and I we'll call this reference to to Academy ask we'll select our reference table it's going to be this little Academy app that I built for purposes of this exercise and the reference field in there we're going to use is the number field that's going to match what I have in my external data set great now we'll create that import table all right we give you kind of here's the import table it's always good to copy this value because this is what we're going to use when we go over here to integration Hub so we're going to use integration Hub to populate this import set table so it'll proceed to integration hub for me we'll create a new integration this is going to be called Academy external data that load Academy and we'll hit save and continue you select your Source type as I mentioned earlier most of the time you're probably going to be loading up a CSV or an Excel file but you do have these other connections Avail other spokes available to you so if you got some skills and you you want to get a little bit fancier you can totally do it that way as well I've got my little external audit data file it interrogates that it finds the those five fields that were inside of there for me great and then we map it to our Target so we'll come here we'll say add a table this is where we'll paste in that little table that we just copied from the last step we'll hit save and then we have to map our Fields so we can use this autom map button to start the process for us it mapped vendor for me but then I'm going to take my let's say unique identifier field and map that to the case ID I'm going to take my activity value and map it to the event value I'm going to take that time stamp and map it to the time stamp field and then here's my related task and I'm going to map that to the reference field here and we'll hit save so we've got our mapping and then we just need to run that import so we'll come here to scheduled Imports we'll say run import only five or I think 25 records so this should be completed and process pretty quickly for us awesome our external data is now loaded up into our import set if we come back over here it just says let me know when your import is complete in this case it is we'll say next great we could edit the data set check it out not going going to do that here today we'll hit confirm and then it needs to create the case record so what we loaded up is we loaded up the audit logs right the individual changes on these records but we need like a a main primary case table that maps to those audit logs so it's going to take these 25 AIT log records look at those unique identifiers and ultimately in our case here it's going to create a table with five records in it right and that's this one here our five records now right from here if we were just doing um external like a process map on an external data set we could click this create project button but in our case we're going to be now linking this external external data set to an internal data set and the internal data set is going to be our our our parent project so we're just going to come back here and we're going to create a new project from the start uh for those who have not seen the Washington uh release before one of the enhancements that we made in Washington is we added a guided setup experience for project creation so for those of you that have not seen this yet you're probably expected when I hit create project us to go to the traditional list and form view uh We've augmented that now with this new guided setup so hope you enjoy this new guided setup as I walk you through many of you through it for the first time so what we'll do is we'll call this Academy external data set project don't need to give it a short description we'll choose our table and it's going to be our little academy process money Academy app that I've got it's only like 15 records in it we say create project and this will walk us through the process of next what we have to do is set up our activities definitions and our breakdown so here on the left hand side I don't really need any filter conditions for this I only got 15 records in this uh Academy application I'm going to set my activity for it we'll come here we'll say new it's going to be state great and I have to hit save down here then we'll add breakdowns I'm only going to add one breakdown for this parent table here and it'll be category great I'll hit save and now what we'll do is I'll add our child table right we'll link that external data set to this internal data set so this is the piece that I usually forget but here's where you choose your source so table report source is if you have saved a report that's got lots of filter conditions external data is what we're going to do here and in Washington we also added the ability to mine archive data if you're an organization that archives your data so we're going to choose external data I need to find my external data set here's my Academy external data set that we just created through that process so we'll pick that the parent table is going to be my process mining Academy app the relationship is going to be these records are going to be children of of this process mining Academy app and what's the source field well that's that reference to the academy task that I want to link together and we'll hit save now I usually forget to do this but I'm not I'm going to come to my external data set and I'm going to come here and I want to make sure that I have activities set for this external data set as well so I'll come here we'll find our Fields here's those fields that were in that external data set my activity and my vendor and the reference to the academy task I'm going to choose Activity one I'll make that a state value and hit save we'll choose a breakdown and we'll make it that vendor field that was in that external data set and we'll hit save and now we're pretty much done we'll move on to this uh Improvement opportunities we don't have any uh finding definitions configured for these external data set so we're just going to Breeze through this process and at this point we're ready to mine so it'll give us this ability to review everything that we set up here's our parent our child table great and we'll hit mine project we can choose sample mine there's a small amount of data in here it's way less than that 3600 record limit of the the sample mine what's going to happen now it's going to go out harvest the relevant audit log data both from the internal and the external take that data pass it off to the machine learning infrastructure to do all the heavy do the number crunching spit that back in and we'll go view it in the workspace and now what we'll see I'll just move this to the left and to the right here and if I expand this out to be a a little bit larger 100% maybe or maybe let's just make it 75 I'll expand this and what we'll see is now we've got that linkage between our internal app my little process mining Academy app which is native to the service now platform and then this external data set that we loaded up from that Excel file and we've got a linkage between the two and we can start to see when things go to the external processing step there's the linkage the five records that's when it happened we can actually then see all the bottlenecks in this process and then when the stuff came back to the parent table that still lives in service now that's it that's how it works that's how you kind of walk through the process of linking together internal service now process data external service now process data put them into a single combined multi-dimensional map and we did all of that in in maybe 10 or 15 minutes here so pretty powerful capability um like I mentioned earlier there's many use cases for this the one that we've SE has most often for is from an HR onboarding perspective where you've got that higher level onboarding case and then a whole bunch of potential uh subtasks that may involve external systems um you want to start linking those together into a single map let's come back to the slides and we'll just pause I'll open it up for questions if you have questions throw them into the Q&A um portion of this if anything or the chat I'm going to be monitoring both if you if you have any questions now is the time to get them into that Q&A I don't see anything there right now so I'll come back and check again in a minute or two just some other logistical items so we just walked through kind of the the steps necessary to link external data internal data in a multi-dimensional process map there are other Academy sessions out there one that specifically focuses on process mining for external data so if you want a little bit learn a little bit more about process mining for external data go check out that process mining Academy session if you want to learn a little bit more about multi-dimensional process maps and how to do different types of analysis with those there's a whole session focused on just multi-dimensional process Maps um so check out those two sessions if you want to dive deeper into any one of these two areas I highly recommend both of them if you're going to be doing something that involves linking both internal and external data um above and beyond this there are other opportunities to learn more about service Nows process mining solution we do have a now learning course out there on the site uh it's called process mining Essentials a little bit dated at this point we're in the process of updating it but it is a good foundational course for you to take on the now learning site um there is a great now and now white paper that walks through how we've structured our process mining program here at service now and five of the initial use cases and uh processes that we attacked with process Mining and some of our initial findings most of you probably know because you found this session that there is a community form just focused on process mining or product Hub so lots of useful information out there lots of content there one of them is these process mining Academy sessions this happens to be our 25th process mining Academy session all the other prior 24 or recorded and posted uh to this process mining Academy site so there's deep dives into different areas like those two we just focused on there's even a deep dive into things like the histogram if you wanted to learn more about the histogram or root cause analysis or there's a process mining for HR session out there we also have something we call the use case series uh the use case series is a a collection of short little five minute videos that walk you through common use cases like um how to use process mining to do SLA breach analysis is how to use process mining to do multihop analysis how to use process mining to compare uh before and after a process change to see if there's any adverse effects so there's about 10 of those little recordings out there just to help you kind of get inspired about what use cases you might be want to use process mining for and and actually walks you through how to configure the projects and do the analysis itself um and then lastly there's a a live on now session that's focused on getting started with in platform process mining walk show through all the steps of uh turning on the plugins and configuring your first project or two uh the the academy sessions out there also Academy session number one and Academy session number two also have videos that are very good foundational videos to walk you through those types of initial project creation steps I'll go back and I'll check the Q&A to see if there's any questions that have come in so somebody's asking like how many customers have log files or vent files per process available besides that sap example um it I've seen lots of different examples I I've gotten uh data files from uh Salesforce Salesforce doesn't have their own native process mining Solutions so people are looking to do type analysis they'll extract those those types of files from Salesforce the smart recruiters one is that uh we used um uh internally here is one that we used so I've seen I think Jiro was another one in which somebody pulled some data out of jir and sent it on over was using that for analysis so I've seen a number of different examples sometimes it's a custom application I actually had one customer that had a homegrown underwriting system that they produced an extract for us to mine and visualize so lots of different examples I'm always interested to learn more so for those of you that uh are watching this recording or stumble upon it if you've got examples of external data that you're interested in mining please post to the community we're always looking to learn uh interesting use cases where we could apply this solution uh the question about which course do I recommend for linking to internal service now tables uh for that one I I recommend that you you use this one here this process money Academy number 21 uh multi-dimensional process Maps uh That'll Walk Through the creation of a multi-dimensional process map of two uh in service now tables internally that's the best one for that um and then also you could look at there the use case series for um analyzing approval bottlenecks that also walks you through creating a multi-dimensional process map for both the requested item and the SE task table so those are two um two good options to check out right anyone else have any uh additional questions that they want to uh get in before we we wrap up uh as you thinking of other questions that you may want to ask I'll just Rec remind you that there is no uh process mining Academy in May we're taking a little break for for knowledge but there is a ton of process mining sessions that are being run at knowledge itself so um there's uh one that we're running it's basically an introduction to process mining called enable fast datadriven improvements within platform process mining that's going to be myself with my colleague and partner yon vanel we'll do kind of a little intro we'll also cover some roadmap stuff in that session on Wednesday um we have a session that called better together so we'll walk through kind of the combination of performance analytics or plat platform analytics and process mining in fact uh that session we we had one scheduled for Tuesday it's sold out already and they added an additional session so that that's a topic that interests you highly recommend you get yourself in your heading to knowledge sign up for that one because it is filling up fast we have another session that we'll be running through just some of the common use cases some of the things that are in that use case series there um and then there's there's couple of sessions focused specifically on process mining for customer service management and another couple of sessions focused on process mining specifically for hrsd and then our now on now team is also running a session on process mining and how to get started with it so no shortage of opportunities to learn more about process mining at at knowledge if you happen to be attending um and if you are attending I would love it if you came to one of these sessions but I would also love it if you stopped by our our pod or Booth whatever you call it that's going to be in the platform Pavilion area I mean I love would love to meet some of you in person that have been joining on a regular basis always good to put a face with a name so please uh seek us out if you happen to be there sign up for the sessions or recommend the sessions to a friend all right last chance for romance anybody else have any additional questions that you want to get in before we turn into a pumpkin here today if not just want to thank you all for your time and uh Happy mining everyone e
https://www.youtube.com/watch?v=KVh_mhXYjF0