and we're live so welcome to phil goes deep and this is the grc irm full flow series just realized haven't got my headphones in so hold on a second hopefully the audios hopefully the audio is okay yes excellent so uh welcome to phil goes deep this is grc irm full flow where we are converting the grc application in servicenow to flow designer we've got 80 minutes right now it's part 7 and we're going to be focusing on the grc utils api so far we've managed to convert indicators issues and now we're going to move into a bit more of the core api where we see a lot of asynchronous activity things like generation of entities generation of items uh we are starting with a blank sheet of paper today i am going to switch to my screen oh i've got pink hair uh it's breast cancer awareness month october so check the link in the description if you want to find out more information i'm trying to get to 100 supporters on that so thanks to everyone who has sponsored so far i've also got a link in the description for the servicenow community uh post that you can join in the conversation but please feel free to get involved on the chat as well and we're gonna get into the show so what i started with was an approach in here to say we're going to identify the api calls and we're going to convert those methods into actions and then create our flows and sub flows and then replace those calls so grc utils is a bit of a beast let's take a look so we've got a tab over here i know one thing that sticks out is something called generate link so i think that's where we're going to see a lot of a lot of attention what i'm going to do is go into code search and just filter on grc utils i'm going to open up flow designer please let me know as well if the audio levels and everything are okay um the show last week i was a bit concerned but everything turned out okay seems like my instance is running a bit slow also uh yeah if you check out the servicenow community you'll see that i've made it to the number one spot so let's see how long we can we can last there but shout out to everyone on the leaderboards uh out to jan very very helpful posts so we've got the irm full flow article that's been pinned i hope my flow design is up and running and grc utils has got things like dictionary overrides reference qualifiers get on related records okay it's the thing about utils what else are we doing enrichment query that's the entity type filter table is processing okay so these types of calls aren't going to go into flow designer start workflow i wonder whether we should we also need to watch scope okay i'm staying within the grc profile scope for now but what i'm going to be looking for is if i've not got this wrong a lot of stuff i'm hopefully i haven't named the api wrong i'm pretty sure i want to generate links business rule so things like prevent generation i assume let's just take a quick look at these these are going to be something like a before business rule and that's in policy and compliance anyway yeah before so we're not going to even look at before business rules i'm looking for relate profile types that's going to be an interesting one anything that's asynchronous is definitely going to be in scope here remove entity relation on delete i don't know what this is at the moment i'm just trying to find a good starting point uh set discrete fields from applies to this is nice this is on profile but i think that's a before business rule set name yeah this you can see there's no update in there pop-up scratch pads is display business rules start control test see how big here we go update item generation records see how many methods we've got in grc utils validate parent not what i was looking for let's have a look at ui actions add so let's see where i'm thinking about if i go to entity types the m2m tables within entity types when we attach an entity type even the creation of new entities in here if we look at the business rules generate links asynchronous have i called the wrong api for grc yeah that's grc item generator engine okay so i might have to raise a change order and not focus on grc utils right now um grc item generation generator engine let's stick this in our in our document in here so we've got item generator engine um when we are on a an entity type and we change the filter if we look in here there's a ui action update entities from filters and this calls an event queue generating profiles which we call a script action generating profiles and that's glc utils let's generate okay so i was calling it generate links so grc utils is generate profiles equally um so i'm gonna open up glc utils you'll see what i've been doing here messing around with version 11 i'm on paris the script the the schedule the scheduled job that maintains our entities it's called profile generation that never got updated with the nomenclature changes you see this is calling import profiles and import profiles is read only even though it says to override base class functions so that's getting looked out i know i've raised this with the product team but the base is protected okay so this really brought me some alarm bells however what i was able to do was look at the grc utils and what was previously in version 10 getting called directly from within the script within the scheduled job i've actually managed to to intercept these okay so when it calls import profiles this is actually getting logged out so import profiles api actually makes a call back to our classic grcu tool so it's generate profiles that i want to focus on not generate links okay generate links is within um item generator so that's that's something for next time maybe generate links but there's loads of stuff where you've got m2m relationships kicking off uh asynchronous and you can't always keep track of you know what's happening and when so i'm really hopeful that putting this stuff into flow design is really gonna start to see the uh the benefit so generate profiles is getting called let's go back to our code search i know i've been a little bit all over the place right now generate profiles and under grc utils base so i think i've got my script sync set up correctly in here i'm just going to copy this out into vs code it's just huge this file it's great though look at all these apis at the top uh so we'll do methods at the top so lots of checks getting done here checking is check check generate profiles from type i think this is going to be very relevant um get indicator templates from content get item based on table i don't really want to get relate profiles this is definitely of interest related profile types also set discrete fields that's a before that's called from before business rule so there's no update in there so we can't call that from uh we can't call that from a flow designer action set discrete field start workflow cancel workflow i'm curious whether these ones you know i think there's out of the box um actions core servicenow call actions for for workflow so i'll be interested to see about those show add button get default templates some of this stuff's coming from before or sorry from display is check is you know i keep talking to people about variable naming conventions and function naming conventions it really does pay off if i look at this is i should be able to know that that is going to return a boolean value and check isn't check it's going to return true or false right user has role i'm not sure why that needs to be within grc utils to be honest remove users without role get downstream profile class list this stuff's getting used for uh glc workbench get content records so we'll see where some of this is called from um get profiles from profile class generate profiles so this is this one here so generate profiles has got a lot of inputs in there i would have loved if this was private and generate profiles of sitting at the top as a public method all that all that good work and and then the main you know the main style of show doesn't get the benefit so let's tidy this up i think these are the main ones not to say it's completely limited to these so generate profiles and we did a code search gets called from a script action it gets called from a scheduled job okay that comes hmm would we be able to replace the scheduled job if it's going through import profiles i think we can take the old legacy code and assume that within a scheduled job so that's going to be interesting let's put that as a there's a question um ui action generate profiles that doesn't get cooled anymore that ui action yeah it's not active so import profiles it's probably our main one let's take a look at generate profiles from type okay so generate profiles from type probably comes from what we're saying here 49 and 360. 49. go to definition no 3 50 3 60 wasn't it yeah so from within generate profiles equals generate profiles from type so actually it's not public method that gets called from in here relate profiles relate profiles so what's called relay profile then this relate profiles 364. 364 is coming from generate profile so this is the thing when i was looking at some methods in in a previous episode i thought if it's a public method then it's probably getting called from you know an end point like a business rule or a script action and actually it might have been getting called from a different script include a different class so just because it's public you know denoted as public without the uh without the underscore doesn't necessarily mean it's getting called from a relevant end point so that was a massive lesson learned for me when trying to convert the entire application here we go we might relate profile types coming from a business rule and that's on entity when to run after applies to record changes interesting relate profile type so that's a business rule it's called in that and where else scripts include okay start workflow i'm not too bothered about these ones so let's do the generate profiles and then we need to think about what we'll do for the new import profiles api and maybe we'll move over to grc item generator that's i got a little bit muddled up when i was planning this one so generate profiles if we see their generate profiles so generate profiles type query no not this one you know generate profiles oh so it takes in here all these bit only passes them if it goes from cis i don't think this gets called very often if table name and selected cis and user field sorry selected ids i think that might be for a legacy legacy method this is the two main ones profile type entity um need update flag however it's like for like so we're going to convert all of these with values and we'll just pass them as strings so we go into flow designer and create a new action create a new action generate profiles and we need a new i need a new method in here grc irma flow grc utils okay inputs we've got here where can i put these as visible there profile type need update flag oh it's also hacktoberfest as well so shout out to anyone that has been getting involved in hacktoberfest shout out to the rangoo team for friday's efforts out to raph uh sabine out to ali andrei i feel like i'm missing someone um i'll tell you what now components is a real toughy um something that we really excited about and just seemed to be quite difficult to to get some traction on so if you're learning something new you want to learn something new check out now components get involved in in the hackathon still a few days you need to make four pull requests check out the developer site on servicenow and for me the main learning there was was about the use of github not something that we you know i've always backed up my custom apps scoped apps to source control but i think the now component stuff is is going a different way and you you're kind of doing local development pushing that up to get and then once it's compiled you're pushing it into the instance and you get the compiled version so that was probably one of the things that really slowed us down do we get any output from here do we return yes we're returning okay so if i go to that definition well it returns but there's no return in here okay and generate profiles where you've gone generate profiles from cis oh it's a big e there's no returns in here so actually that's a false return i'm not going to bother with an output then we're going to do a script step and this is going to be called generate profiles variables are all going to come in here i'll grab them from this top bit no this top over here i love it when this can just get auto populated somehow but or at least i'd like to understand why it doesn't i guess because you can bring actions you can bring inputs into your top action and your script steps might not actually need to consume all of them so maybe that makes sense i had this before as well where it goes to undefined if you drag over it it's dropping off so that profile type is fine if you come in from the side if i drag over the top of it it goes undefined a bit of a bug there table name so coming from the side use user field content type content type content id excellent then we just do a new grc utils dot generate profiles you can push the profile type inputs dot need let's write this bar profile type equals input stock profile type if anyone's wondering what profile is and you're just getting into grc it used to be called profile okay and it was changed recently which makes a lot of sense but you can't change all the methods and you can't change uh the tables so this stuff's always going to be hanging around i'm just going to copy just do copy and paste user field content type and content id but in fact i can then just copy that back out boom so that's going to take care of whether or not we want to go through the generate from cis or generate the normal one and just use these two we've got the option for these top two i think in terms of making something mandatory profile type definitely has to be mandatory i think need update flag is optional it's not denoted in here what you might expect to see is something like this which is always nice but it's not there right now there's no code documentation on the on that method we've got generate profiles let's save that i'm going to give that a nice name when i call it generate entities right just because it's modern it's up to date and when you see that from the flow designer kind of front end that will make more sense to people i thought what happened here going the wrong way around don't think that matters profile type they seem to have switched around let me just publish that again i don't think it matters because you know it's all going to come in mapped so it's not like a function where the order is is very critical i've got my action i need a subflow okay my subflow is going to be called generate entities grc utils run as the user that initiates the session that's what happens inputs and outputs so let's think about this when we when we saw generate profiles when it's getting called i don't recall seeing anything coming in with the extra extractions here so that scripts generate profiles okay actually this script action has got the options okay just i've overlooked that because it's never happened so we do need to have these things specified so flow designer data table name selected ids okay so that's coming into event palm one so wrapping the gap always nice uh we need to create all these inputs i'm just wondering now when i've done this i'll give the names if you notice the names so the label is that name is that and it doesn't respect camelcase here i think i've done something wrong in my action so i'm just going to go back to that a second feel like i've done something wrong because i went through didn't i and changed all of these to follow this format which is the format that we see in flow design i forget the name of it i can't remember if it's table underscore name ah user field so these are the names when i was calling them from within my script action they weren't called that and actually this should be called like entity type need update and that's true or false right it's a boolean true false table names a string should be called table name selected ids handling data types correctly from within flow designer is also a bit of a new a new one for me because trying to handle why has that done that with a six um yeah handling things like arrays and objects and passing them through just doesn't seem to to like it as much let me go back to here if you notice this is my name and oh sorry sorry that um entity type need update and that should be true or false table name i think i'm calling them wrong in the script step so i'm just giving these a nice friendly name and then in here look i've called this name oh it doesn't matter that's fine that is fine because they're mapped to it so i don't know why why they need that name then if it's kind of invisible are we doing on time right 30 minutes in which means we've got 50 minutes remaining we haven't tested anything yet we don't need an output so we're done with inputs actions guess what we're going to go to profiles utils generate and entity type you can see there's mandatory when entity type comes in let's look at that code when we were looking at our script action entity type comes in so profile type comes in it's actually coming in as a glide record so my data types are wrong here this input is not a string it's a reference to entity type publish that and then in here let me save it hopefully that refreshes no okay i didn't need to refresh this input is a reference to entity type excellent so that's done in here i'm going to pass in entity type need update is a true or false and then we can just map these dragging them across whatever's coming in just gets passed straight through but i think for objects things like um glide records another benefit of flow designer is it's going to make sure that it's the table that it should be especially when you're setting up your flows so that's as simple as it needs to be for a subflow if we just test this i just want to make sure it runs correctly it doesn't matter which one it is if i just say run test let's just make sure it completes successfully obviously haven't passed anything in other than the entity type that's been executed but you can see it did take a while it's completed successfully there's no output yeah we don't know what it did there's no outputs just hope that works so where's generate profiles getting called from within this script action so from our subflow that's an action we want the subflow generate entities just publish this i'm going to do a snippet and this is going to run asynchronously i only want to try actually i don't need the anonymous function in there um if i go to my script action okay so we've got the if statement to deal with let's create a function down here called function call subflow why don't you like that unexpected token okay of course we've got a map in our stuff here so we're going to map in this cool that that won't be true is it always true yeah so we'll just call that true directly in here profile type table name not sure what the order changed selected [Music] ids i'm interested where this other one can actually get called from because it's not something that's really that obvious it did slip my mind okay so we call subflow now we do want to call it asynchronously so we want to take this one out get runner generate entities why don't you like it i'll mix spaces synchronously we don't want any outputs so that's going to go this way call subflow and that's just going to go to end to profile type okay so nothing's going direct to the api it's all going through the subflow here so i'll save that that's called generating profiles script action generate profile so that's going through a subflow a script action here what's calling that script action if we go to our entity type and for example business services uh subflow's called asynchronously if i change this filter and just say i don't know approval goo group is anything when i update that entity filter i now get this ui action when i click this it throws the event which calls the script which calls the script action it should call that you see that's queued up now complete and that ran okay so we've got the ui action forgot what it's called update entities from filter ui action after institution filter profile generation job still going via import profile generate profiles takes care of these two so these ones are done this one's done which means i don't really need to do these but the market was done and then we've got a business rule just let me sort that text out because i can't read text it color there we go um relate profile type so we need to find that method relate profile types wow does it return anything returns results what is results an object where's results defined yes up here okay so this might be interesting this is what i said about the data types so let's start with our action and create a new action and in fact i'm not going to do that i'm going to copy an action copy action so it's called relate profile times yeah and obviously i'm going to rename this relate entity types now my inputs are very limited here we've got profile and profile dot cis id so profile must be glide record so i can get rid of most of these all but one of these inputs and it's going to be called entity oh yeah once you've created it you can't change it so if you copy an actions that are handling the same inputs then that makes a lot of sense if i'd copied it though you'd expect it to not be locked in uh it's smgrc profile mandatory in here you can get rid of these entity i'm going to call this gi entity because it is a glide record coming in and it's called relate profile types and we're going to say vo outputs dot results equals and this is results and this is an object now this is a predefined structural object which is should be fine right i should be able to do this and it won't be a problem because i can look at that and say added existing removed i'm not sure how we map the values yeah when of when i'm passing objects in and out that are a bit more flexible in code that's fine in here it's non-impossible from what i can tell these are integers so this is this is very easy but why or how do i map them i save that and then create an output here which is going to be called results results and this is going to be an object i'll do the same again so if i do this boom why is that expandable i did i did existing it's called existing removed edit exit mode can i drag that there does that mean that's popular in itself hopefully because they match then that matches so relate profile types passes the results back da da input entity save publish um so relate entity types school relate profile types come from a business rule called relate profile types and when this runs and applies to record changes so we need a subflow new subflow late entity times category grc utils input is going to be entity and that's a reference to sngrc profile mandatory outputs is results which is called results and as an object i wonder if i need to specify this here and say that's done then my action is under grc utils relate entity types passing entity and then flow logic assign subflow output hold on a minute here results equals results okay you can just pass out an object depends how you want to access the elements of that object save that and let's test it let's just grab any entity i don't know if it really matters let's run that test hopefully we can see an object with zero zero zero in the results results is an object i don't see anything in here okay let's look at that logic flows we need this now the flow is going to replace the business rule so this is relate entity types we're doing on time 53 minutes half an hour so our trigger is going to be is it insert an update no just update okay updated table sngrc profile table which is our entity table filter applies to changes oh this one always looks a bit slow applies to changes run trigger once we'll just say for every update running the background should be fun i mean it's it's an after business rule so it should actually run in the foreground if we try and match things up we can always optimize them later let's go and get our subflow here subflow generate no it's i've not published it okay that's published delete this at the subflow glc profiles relate entity types passing the entity record i might get an action out of it i'm going to activate that and i'm going to deactivate a business rule i'm just going to remind myself what actually happens here so it goes and matches yeah make sure it belongs okay so as an example take the entity type business services and take one of these entities if i go and point this now so it's called sap financial accounting and you can see that this belongs to the business services entity type but imagine that i changed yes if i said you're no longer that and you're now a user called phil swann imagine i'm getting a buffering message i've got a stream poor status i hope it's not a problem for anyone watching this live or on repeat on demand i'll just slow down a little bit now when i change this applies to what should happen is it will be removed from this business services entity type because it didn't belong there anymore it won't meet the conditions of that entity type if i save that and then go and watch in flow designer for executions just now okay let's reload this entity updated yes subflow called okay well let's go to the subflow and look at these executions complete that came in no this is not the right one how come subflow isn't showing here that went into a subflow so my subflow should have an execution against it shouldn't it not the one from six minutes ago the one from just now hmm i would have thought that i could see the execution of the subflow let's have a look at my action relate entity types executions doesn't look it's got that far does it definitely didn't work because i've now got this entity where the applies to changed and it still got me in the business services so let me go to grc utils and we're talking about relate profile types that's what it's calling isn't it in our flow designer action boom types it's being called running in the foreground so let's fire up script debugger i'm going to close down some extra windows i've got open i made that inactive that's good so that's a flow properties that's just descriptions will the debugger kick in let's change the applies to record and find out i'm gonna send it to a kib yeah shout out to a kid when i save this i'm hoping script debugger kicks in but it doesn't so my flow designer i'll go to executions run just now erid cool subflow failed i wonder if that's because of my let's take this off and change this back to me oh i hate when you drag in a modal window you should be able to drag in a modal window save that and flow designer tidy some of this stuff up look at my executions complete okay so putting a script debugger in debuggers up the uh the execution in flow designer even though it's running in the foreground i would have liked to be able to still step through my code if possible so that execution ran okay if i go to my subflow why does the subflow not show my executions 11 minutes ago what should it be doing then let's turn the let's deactivate that for a moment turn my business rule back on and put the debugger on there let's just see what's happening with the code i'm a bit curious there i thought that the executions would be visible at all levels at the flow subflow and action level it appears not to be the case so i'm just switching between users at the moment so make sure debugger is running yes when i save this should get caught here we go that's what i wanted from flow designer i don't know why that wouldn't be possible if it's running in the foreground so we're going off to enrichment query again all active enrichment queries okay we're going to go through each enrichment query and add the query the profile applies to which is my user id my user sys id well that's interesting i thought it might use the glidefilter.check record but actually it combines the existing condition and the condition on sys id equals and if it's next that's quite clever so it matches if it matches both and it's not going to match any i don't think i haven't got any entity types pointing at the user table so we shouldn't see any so when you did change the applies to on a record it's going to say do i meet the conditions of any entity type filters and if i do then add me to them okay that's obviously quite a lot of filters let's just put a break there and play that so i expect string ids will be pretty pretty empty yeah still undefined ids was an object with nothing in it so gr unwanted is any m2m records that no longer apply so we're looking at the profile sys id which should be good created one off false which means they're automatically created so if you manually mapped it into an entity type it wouldn't matter we didn't have any that can't be right this one there surely that should have come up for i in ids there's nothing in create a one-off equals false let's open this m2m record create one off equals false so this record should have been deleted this should have been a gr unwanted if i take this out switch it back to me it doesn't really matter who it is there's no blood model it doesn't really matter who it is it does not meet the condition script debuggers on save i play that because it's going to catch it after here string ids empty so why would this table if we look at this record open a new record that's the table grcm2m profile type grc m2m profile profile type profile.sys id here so let's do this filter and say yeah entity let's show related records entity sys id run that yes and created manually just created one off is false run that oh create money hold on then equals false but if we say in here created manually is false nice bizarre i am baffled i'm absolutely baffled right now i don't even know what what's going on uh right click show matching it's gone i'm so confused created manually true or false um right okay so it's not going to work this is why it's not going to work because that query is not working why would that query not work is it because it's demo data could it be that simple that this stuff's come through and for some reason if i open this record in explore create one off equals false find element boolean let's say add query sys id and then add another query created one-off folks gr.query and then g output next geo no so just hear me then if i do this get and then gr dot set value gl dot update and by the way never use gr uh for some unknown reason it can break stuff uh something to do with the java level runtime not something i really understand but it's not recommended best practice in fact it's anti-practice and it will show up in your sprint scans and health checks and stuff so you don't really want that it would be nice if we could use it safely but unfortunately we can't so don't do it kids just say no um add query query let's just see gr don't get rocount still zero what the i've no idea what's going on absolutely no idea what if we wrap it nope okay let's get rid of that sis id right so okay it doesn't make any sense but so i've got one there i'll maybe just do my data if i just put it down to that and choose a different record it'll probably work then query next gr why is that shouldn't be difficult query gr.next yeah just took a while so that works so what happens then if i pick a different entity type just conscious of time got kind of three or four minutes remaining i do want to stick to these kind of um 80 minute sessions because it works quite nicely if we do that let me point this one um that's this is this user and you're gonna be able able owns himself script debugger yep save that start debugging i'll play that one out yep nothing in here because it doesn't belong unwanted there we go right so for some reason it's a demo data issue nothing of any interest but let's watch what happens this is going to get removed from that entity type okay let me just play that now and therefore entity's been removed from the entity type if i then turn my business rule off and switch it back to my flow and activate i've got one minute let's just see if i can point this at maintenance task this one which is where it belongs and save that hopefully my script debugger didn't break i know i had that on yeah meanwhile did uh okay let me go to a different existing record do the same thing pick on this one and turn this to a user you know if you there's a good kind of tip here then if you're getting stuck on a certain record and that one record is behaving very strangely go and get a different record that's similar create a new record take it through the process from scratch start with a really vanilla record um and you know then test it there because sometimes there can be something in the data that's just going to screw things up let's just save that access to set abortion oh right okay so set a board action because it already it already exists got a unique key violation i can't drag on a glide model um okay it's coming your way it keeps in the demo data i mean he's my new go to now look at that it removed itself if i go and look at that flow and re-run it look complete i'd really like it if my object had my object had those numbers coming out of it um let's see just the flow that's the flow i want the subflow executions why does it not not show my executions there i'll go to the action it must be only when you get called directly i thought the action should have been in there oh well uh lots lots to take away from this uh not hugely hugely successful but we have done a few bits um and we keep going so thanks everyone for watching let me switch this back um yep thanks everyone for watching have a uh happy wednesday and i'm kind of trying to do it if it's not wednesday then it might be thursday i don't want to do it on a friday because of live coding happy hour so shout out to all those guys that are running that um andrew barnes brad tilton and all the guests chuck et cetera shout out upside down andrew thanks for the help with the hackathon last week uh thanks to everyone that's been tuning in liking the posts getting involved on the community uh share link subscribe etc see you next time thanks everybody um it's been a pleasure and i will uh see you soon you