hello everyone welcome back to our digital product release uh launch and learn Series today we're going to be covering uh automating release and we have a Col and back as as our speaker again let's jump in so before we go further uh we have our Safe Harbor which you've all seen now many times um but there may be some forward-looking statements in this presentation uh they're based on our best understanding at this point uh this point in time so just don't take anything I said Stone as it could change okay so uh hello everybody uh this is uh Colin for those of you that this is the first time I'm a part of the inbound uh Team so I work with our development team on building out these capabilities and also as a refresher for digital product release we are currently focused on being more of the the process orchestrator around governance uh making sure that all of the activities that need to occur um have occurred they've taken Place uh in order to validate that we are ready to release to ready to deploy um and we're going to be focusing on automating that validation uh today so that's what we're going to be be leaning into so if we think about the maturity curve that we've uh reviewed several times this is basically this third leg expanding Integrations and policy uh automation this is by connecting to our uh cicd Pipelines to capture additional information that typically occurs within those pipelines for more validation um a question that I sometimes get when we're talking about these this kind of data that we're trying to now collect is uh where should it really res uh reside so should it be in your your cicd pipeline should it be in the release side um and the answer is effectively both your your cicd pipelines uh if you're leveraging them should should do testing they should do security scans uh what release is really doing is taking that data so that we can validate it that we can authorize it so that teams don't have to go through additional work in order to kind of collect the evidence to say yes we did the activities that are necessary so as we're going through these policies today and talking through these policies and how they work and how uh things get wired up so that we can collect this information uh I am not saying that you should you know not do a check for security scans within your your pipelines you certainly should uh we are now an additional layer that collects it seamlessly behind the scenes team shouldn't have to do additional work in order to help automate that release Readiness validation uh so that's what we're really going to be focusing on uh for uh for today so let's first talk talk about our content pack so with DPR again since we are based on policies and policies are what are used to move from one stage of your uh release to the next one phase to the next uh we shipped with with policies uh we actually have two other policies that that they're not in the content pack they just they come with DPR they're um uh they're they're they're not a separate inst which is validating tasks basically just validating that release tasks are done release approvals are done but when it comes to data outside of service now that's where the content pack comes into play and so we have things like all of our stories have code commits tied to them that was actually an early feedback from a devops uh devops change customer actually a while ago that they just want to make sure that every every story that gets worked on has Associated code commits they need it for audit um or things like all of our plan stories are actually completed they've been approved by the product owner so there's nothing that's still in progress uh which you know might mean that it's not time for us to release because not all of our work is done um code coverage scanning comes up quite a bit if there's certain threshold you want to meet then we pull in data from like star cube to to get that uh and then different kinds of test cases so we have a generic way of collecting tests and categorizing the automated tests and I do want to to keep for right now it's more of automated tests we can definitely get two manual tests but there's um there aren't out-of-box Integrations for it that would be something that we could still do to get information uh but when we classify those tests there's usually different thresholds for different kinds of testing like unit testing might be at like 100% but maybe functional testing is at you know 80% or 90% something like that uh so again there's lots of data that we are using and all of this is pulling off of the devops data model so with devops change velocity um there is behind the scenes a data model with Integrations we are hiding this data model for all intents and purposes we show you the relevant Parts within our dashboards and within different screens of DPR uh but behind the scenes there's a complex data model that Associates things like stories which could be in jira with code commits that could be in GitHub maybe you're using Jenkins for your cicd pipeline and it's running some unit testing tool or some functional testing tool and then that's producing different artifacts that are may be stored in artifactory you know within the world of devops and really just developer Tools in general there are lots of choices and they're not always aware of each other so the point of the devops data model is to connect to different tools in the developer World pull that data into service now and normalize it so and only pull in relevant information so for instance if you have custom fields in jira that have no impact on your release process then we don't need that data um we'd only need what is necessary for the release process so definitely we don't advocate for bringing in everything and also all of this data is in readon form so if we do pull in data from jir you can't edit it in service now because it's meant to reflect what's going on in jira not as a way to then update it in service now and push it back into jira there are definitely ways of doing that in service now but that's not the point for release we want it to reflect whatever is going on in those external tools so we connect to those tools and then we can use the data originally with change we use this data for change authorization and we're going to get to that in today's uh in today's uh session but release can also use this information to make sure that there are necessary approvals that are in place with necessary data that we can trace to and prove if needed for audit so as we go through and run these policies all of the data is stored in service now in readon form uh and especially if you're using our integrated risk management um you can even use like continuous compliance in order to validate uh that data to help you know with your your your audit process and then this just comes with the uh the slide that I pulled over all this information while being used for release purposes uh we also have plenty of metrics that pull from within it so if you are interested in things like the door metrics or the flow metrics um we do include that Within release management itself we also show uh quality related information so that you have access to that data um for you so because we are using the devops change velocity we can use it not only to collect data but as we get into automating more and more of the release process the next step after validating that we are release ready is by helping to automate the deployment process so if you do have a pipeline a cicd pipeline and when it runs you want to have a change automatically created we went through this uh in a previous session but I'm I'm going to go over it again uh in this session since we're covering automation um then you would be able to you know orchestrate your your change creation automate it and have it also look at your release to make sure that everything is good to again streamline your uh your deployment process as well so release release Readiness we validating also deployment authorization uh we can automate with the same data so you don't have to do it once for devops and once for uh release or like once for Change and once for release we're sharing everything so you're able to just use this data in multiple ways to help streamline your um your change and release process um actually I will probably come back to this slide let me let me get I'm going to switch over into my demo so I'm going to I'm going to skip back to it where we can start with reviewing the policies and how we capture build data and then I'll come back to these slides to talk more about the change uh change Automation and change authorization uh before I do that Greg are there any uh chats or Q&A that I should be aware of uh before I switch over to my instance nope nothing so far okay and let's see if I've been logged out quickly change screens and see if it complains okay I'm still in okay so you should be seeing uh my screen now um I'm inside of one of our instances and I am logged in is aen who is a product uh product owner um that is managing some releases so first of all when we are leveraging the data and I'm going to focus first on how we leverage it and then I'll show a little bit about policies the key thing to note is that we do have to know about your Integrations so if I go into products and we're going to use the fmr loan origination product there is a uh screen over here that says external tools external tools are where we tell this product about the other systems that we're using so we have going along the top our plans so plans right now we have two plans connected to fmr loan we have a jira project of fmr demo and we have a GitHub a GitHub issues um that is connected to Joe's fmr um repo so in this case I have multiple plans uh we have one source code repo that we're using and that's the the GitHub fmr uh repo we have two pipelines that we are leveraging these are both using GitHub actions um again you could have different different pipelines using different tools and that's fine and connecting them has two parts to it the part that the product team needs to be mainly aware of is being able to associate and say these are the pipelines or the repositories that we use uh this is going to show across all of them you'll notice we we pull a lot from um uh from our demo data uh but this will be the list of all pipelines that we are aware of these are already set up so the tool to connect to GitHub is already done if there's questions and we have time at the end I can certainly go into how these connections are originally set up but the teams don't set up these connections the administrators set these up before the teams really get in and start using um start using DPR so they're able to to connect whatever their their pipelines are and then they and um once these connections are set up we'll start pulling in data but these are the areas that we look at right now we look at plans repositories pipelines test results come from pipelines so we don't have an additional setup in order to to connect your um your tests uh and then we also have artifacts that we connect to as well which we'll we'll dive into in a moment but this makes DPR and really service now and in general aware of the plans repos pipelines that your teams use and teams are able to set up this Association so this doesn't have to be done by the Admin before onboarding a team once the product is in place and the team has access to it they can manage what their plans are they should know about their Juro project uh they should know about um uh they should know about any repos that they use any uh pipelines they use so we Federated that out to where they can set it up so once these are set up we can start using them within our releases so I'm going to go into our releases and get to my actually on my home screen I've got a closer link to the April fmr early release so we've been using this uh demo release for a while so it is very overdue it was uh it was supposed to be done back in uh in June but for demonstration purposes uh it has a good amount of demo data it just is telling me hey our risk is very high because we are um uh we have no time remaining we're overdue and we have policies failing we have tasks overdue there's no way it's getting on time um so there will be a lot of red but we can still go through the data that we collect so this particular release has one AR artifact that it's looking at so the release artifact is saying that we are tracking this fmr artifact so if a release is more of the product the high level of What's um what we're going to be releasing the artifact is a more detailed like a jar file or a um a dll in Windows or an executable or whatever you know whatever language you're using whatever technology you're using the actual binary itself that's the artifact uh so we track it we pull it in from the associations that were made earlier it allows us to track all of the changes that are going through this artifact over time so this particular artifact has lots of builds that have occurred um you'll see earlier today when testing it out um we ran a a build that produced a new uh new version and so we're tracking that and we use it in order to start collecting our quality information so we can see a uh build from our uh our um yeah our build uh build Pipeline and we can get to some of the test results we didn't classify any functional or performance but we're capturing information that we can see within this quality dashboard but what we're really going to be focusing is through release execution where we have policies so this particular release when we were setting up its template has multiple policies that we are checking before we can exit the development phase these policies some are compliant some are non-compliant um like all of our stories being completed that's not the case we have stories Associated to this release that are not completed and therefore we are not compliant which also means we can't move forward with this um uh with this uh f Days of our release we have several that are non-compliant we have you know our stories are not compliant there are critical vulnerabilities so that one's not compliant and then finally we have this software quality compliance that's non that is also non-compliant this is the one we're going to use today and that's because we've actually uh put into our demo we've kind of forced the fact that this particular uh policy will fail un very specific conditions that we can control so right now it is non-compliant so if we were wanting to validate this manually we'd probably have a task that we had given to somebody to say tell me that it's compliant now all of our tasks are are completed but we would have assigned a task that is the way most organizations start we assign a task to a person they need to approve the fact that the security scans are within our acceptable range but wouldn't it be better if we just used data from within the pipeline in order to figure this out so right now this is non-compliant I am going to you know Joe I probably just should have had you since it takes a couple minutes um run this in the background but I'm going to run uh this build so we have a a build that's been kicked off it takes a moment so while it runs I'm going to discuss a couple of things but this build will now run and when it's finished uh the software quality compliance will switch from being non-compliant to compliant that's the uh the punchline that you'll get to see in just a moment um but these policies these are what are used in order to determine can I move from one phase of this release to the next and we have built in to DPR a couple of rules in place on the movement from one phase to the next and it does vary based on are we using a timeline oriented uh release process or a stage oriented uh release process so in the case of timeline oriented until the last day while we're still going to show the status of our policies we're going to tell you if they're compliant or non-compliant until we reach the last day we're not going to use the status of those to determine whether or not to move to the next phase so until this one was supposed to be done back in in uh March so until March 27th which was the planned in date for the development uh uh phase um everything could be compliant on the 26th but on the 2 7th we will use the status of these policies to determine whether or not we can close the deployment phase and move on to the next phase which was I think packaging yeah package um which is a very short a very short phase so you could be compliant all the way up until the last day but what matters is what's the results on the last day and if you're compliant then we automatically move you uh to uh packaging on the 28th so we would have automatically move this and that's because in a timeline oriented process um you're trying to stick to a timeline so that's why we wait until the last day however in stage oriented stage oriented doesn't have the notion of time once all of your policies are completed in a stage oriented then the stage will move to the next one there could be multiple movements in a single day all stage oriented cares about is that all of your policies are compliant and if they are then you can move from one stage to the next there is no time uh requirement to move through these um through these uh policies and that's really because organizations working different ways for instance within service now uh we all March to the same drum we have our August release coming up and I can't be done early and I can have all of my things done early but it's going to evaluate based on the um based on our timelines so we don't move forward even if uh everything looks good because a new build might make it not good anymore um so that's how we work but I have several other custom I work with where they all good that one's done run those policies uh where as soon as a team is ready they can move forward so we we support both methods of working in order to move from one phase to the next leveraging uh these these policies uh so let's see if uh and we now have the the magic has happened the software quality compliance is now compliant because when I ran that last build um it was going to to trigger and just show and basically pass this particular compliance now there are others that aren't well I never finished any stories so this will still be a hold up I also didn't fix any critical vulnerabilities so this would still be a hold up but this last one here is doing it automated and that again our our policies run at the very least nightly you can configure how often they run in the background or you can trigger a manual check for policies um and now we're doing that without a person having to go find the information compile it and attest to the fact that they um uh that they uh you know whatever requirements they have have been met so I think I just saw a Q&A pop up uh is there a question Greg that I should answer um it looks like it might have been a typo okay will they hit enter too soon no worries so if there are any questions because um going through we're going to go into the anatomy of policies uh in the next session the next thing I was going to go over was now we have you know this part automated so our our Readiness checks are automated um next next I was going to go into the devops change piece again to automate the deployment uh side of release so if there are any questions people want to put in chat now is a a good time otherwise I will go into devops change okay so let me switch back to that PowerPoint and go back okay so devops change uh which again uh devops change velocity and digital product release do not require each other they can be implemented independently they can be implemented together obviously for the full um uh the full kind of automation we recommend using them together uh but which order you implement totally up to you uh and again they don't rely on each other um but when we get into saying we have a release that is ready and we want to automate the authorization of the change based on this data devops change leverages something released oh when was it maybe back in Orlando could have been Madrid a while ago I worked on it but now it's been I think five years so little hazy on time but we introduced change approval policies to help automate the process of saying do we actually need a person involved in saying the change should move forward and devops data is one of the largest uses of uh of sources of information in order to make that determination so a a approval policy basically says based on some criteria do we approve this change do we reject this change do we send it to a person do we send it to a team those are all options available and any data within service now can be used in order to make that determination uh devops change ships with some examples of what to use change management ships with some examples of just using change in order to um in order to make that determination but it allows you to use any data in service now to decide whether or not to Auto approve Auto reject or send it to a person and what devops then adds is the ability to have a dedicated model dedicated Change Model that has the specific criteria needed as well as has Integrations into your cicd tools to actually create the change requ itself now since I've already run one pipeline I'm going to go into GitHub and show a little bit about what that looks like it is more on the technical side so this is something that typically uh development teams who are used to dealing with yaml files or are used to dealing with Jenkins pipe line files uh this is something they typically set up we're going to be going through it because I'm also going to highlight how you would connect especially if you're already using devops change how you would connect this with release because we already have a connection between uh both uh change and release for purposes of automating your change process uh so they would need to at least make an update to be aware of your um uh your release itself uh so Greg I now I got another popup that said there was a Q&A uh is this something I should answer now or answer uh later um I think we can do it later so I I'll just read out um it's it's about feature toggling within DPR so maybe we'll send save it to the end okay yeah that would probably be good to the end yeah okay and let's now switch back and talk about uh how these work so first of all um I'm going to just so that I I have this to point at um multiple change requests so there is a dedicated area within DPR that we've gone over before on all of the change requests Associated to this release uh but when I say It's associated to this release it's actually a bit of a misnomer because the relationship between change and release is actually through the version record so this pre-release 0.9 what we are doing here is saying show me all of the change requests that are referencing this pre-release 0.9 version of fmr loan so by doing that it allows us to tell the change what about the CI is changing instead of saying what release it is so we're we're actually associating a ver to the change and since the release knows what version that it is you trying to validate uh we can pull all of those change records in here the reason that's important is because that is how when we automate change we like that's actually what we need to to connect uh to the change record in order to pull it in here so you'll see we have several automated deployments haveo my very um clever naming system uh when I open manual changes uh so instead of doing the new button to create a change because you could have many change requests for this release maybe you have change requests for pre-production that you want to uh that you want to leverage um actually there's one customer I was working with that one of their policies is that there must be Chang re quests that are closed complete before they can do a deploy to prod so that's that's not an outof box policy right now because I've only had one customer um you know give that as an example but that's that's an example of leveraging changes that are still a part of the release but are for earlier stages of your development in order to say whether or not we can authorize a change to prod they want to make sure that it it's successfully deployed and pre-prod first or you might have different locations that you deploy to if you have um in the devops world you might hear the term blue green where you will have two different uh kind of data centers and you'll deploy to one of them and update and then you kind of fail everybody over uh to the the previous one while an upgrade is going on and then once the upgrade is completed you fail everybody back over and then you might deploy that version to the other data center so in that case we're deploying to two different data centers we should probably have two different change requests to manage it so there are lots of reasons why you would have multiple change requests for release and you always have access to this new button to create it but wouldn't you prefer for the change request to just kind of get created for you uh without having to you know uh manually collect all of the data that is needed for that particular change so that's where devops change comes in if I change over real quickly uh you'll see I have my build Pipeline and then I have a deploy pipeline uh the deploy pipeline um fails a lot because we're not trying to do an actual deployment succeeds a couple of times uh and that's because within the deployment pipeline itself if the change gets rejected then the pipeline will fail um which could stop a an actual deployment going on so if you raise the change wait for its authorization once it's authorized then it deploys that's a great check to have in your deployment pipeline so that you know you make sure that they're authorized uh and it's built right into the deploy pipeline so deployment can't happen if the change is not authorized and the way that happens again now we're going to go into a yo file um is by adding some pieces into um uh into whatever build system you're using like this particular one is using a service now registering a package so it's it's using this particular register package um actually action I think is what GitHub actions calls it so this particular action um and there are some tokens and there's some data that has to be passed in to say what artifact facts are part of this package and then later with the change itself we're saying okay now run a devops change we're going to use the devops change action and we are going to specify what um close this so I get a little bit more screen real estate we're going to specify things like what is the short description like what needs to go into the change request itself um let me scroll over uh this can also use you know dynamic information from your pipeline so this particular one you'll you'll see back here we have a different build number so 0.961 the reason why it changes automatically is because it's specifying here that we're going to use the GitHub version uh so the build number as a part of the uh the short description for the change you can do the same thing with description itself you can do things like give your implementation plan but the part if I scroll over really far this part right here at the end the software model let me see if I can make that bigger so that everybody can see it that's better so the last thing that we just have to make sure to include as a part of all the other things you want filled out is what software model are we using so that's the version of our of our product and the one we want to use is the summer the fmr summer patch that's the 0.9 um that that will get kicked off uh actually this might have been updated so it's not uh if I ran this this would actually do a different version uh but when you're specifying which version you want to run this right here is basically saying what we're associating to the release so if I were to hit play and it were to create the change request what happens back in service now is that a change is raised all of the fields that you'll see on this change request were specified by that pipeline so we'll see that the categor set the software model this was back before we changed it um was the 0.9 we have the configuration item that we're working with uh we have the short description all of the information is coming from uh from the the pipeline although you can do further uh you can do further iterations on this data once it gets to service now and everything gets filled out it's filled out the same way every time there are several times that I've worked with organizations where I've asked them what is like a good change look like what does a good change look like um can you show me an example and then they pull up their change requests and people have just I me they filled in all the required Fields but it's not necessarily good or useful data well by putting it in to your pipeline it's repeatable it's high quality data it's using data in context to fill out the dynamic portions and a person didn't have to spend a lot of time doing it it also will start connecting other information so the change team will also have access this one only has some code commits so I could see what code commit was involved if there were test results that we were capturing when this one ran I just pulled a random one so of course it doesn't have uh a lot of data um but if it had additional like test summary information or quality uh summaries all that gets added to the change record so not only is are we automating the creation we're also automating the authorization and we're giving visibility automatically aut Al to the CH to whoever looks at this change request to get to the details about why this change was authorized hey I actually just kicked off that deployment for the new uh build you created 2.2 okay 182 and and that does indeed show up under the um the summer patch release okay which that was uh summer yeah which one was it Joe oh it's um h222 4 so uh clear out the yeah I'm sorry it was um H2 this one yeah that one yeah yep all right so let going to the change requests here so there's the build yep for 182 and let's see if it's got some yeah this one has some data so this one has here's the test summaries uh that occurred so we can see 100% passing there were 12 tests uh we can see the quality scan that uh that uh occurred we can drill into more details about this scan uh we also have access to the one story that was worked on so all of this information was captured automatically and presented to whomever is needing access to it whether it's for further review or for audit we create this traceability for you to help automate this change uh and release uh process so in uh in the Q&A itself if there are additional questions um because this is what I want I always like to make sure that I've got time to answer any questions that come up during uh remember for the next well next week's session is a uh an office hour so we'll be on to answer questions if you have questions about implementation or you're trying things out or you're having an issue that's the point of next week's but the one after we're going to go into the details and it'll be a little more on the technical side because we're going to go into the details and anatomy of the policies so that if you needed to make your own policies you would know how to do it um but any any question s today go ahead and start loading them into Q&A um and I'll uh I'll start answering them at this point folks can just raise their hand if they have questions and we can uh have them come off mute as well yeah so since since we're there there was one that we had about feature flags that we were um uh that we were uh wanting to discuss so um depending on uh when we talk about feature Flags uh feature Flags actually have a tie in uh conceptually to our product features so right now the majority of customers that I work with when they're trying to validate you know Readiness it's Readiness about a particular build artifact so like I just had build one 82 we were validating build 182 that was what we were validating but with organizations that are doing continuous deployments and actually the last company I worked for they were a continuous deployment shop so we deployed multiple times a day uh however those deployments were behind feature flags and therefore um you know customers couldn't access them we actually released quarterly even though we deployed multip times a day so this is this is sometimes a misconception about continuous delivery versus release um you can still release on whatever Cadence you want even if you're deploying multiple times and what's important is that these features which would be tied to feature flags are what you are validating so in that case I don't have any policies right now and I'm I'm really interested if this is something that you're doing because I um I want to learn more about customers that are using feature Flags because we do have Integrations with uh split.io to capture feature flag information possibly others but I remember split.io because we can use that to instead of validating that a that a uh binary is good we can actually use the same concept like getting feature flagging Telemetry things like that to say are these features that we are planning to release ready so it's less about is the binary ready and more about are the features ready leveraging the same Concepts we went over today leveraging poetry from those feature Flags leveraging tests from those feature Flags leveraging um you know any any other information that you're capturing so we just be looking at instead of validating an artifact we'd be validating these so it's it's definitely possible so on the feature flagging if if there is uh if the person has more information uh we and and we have enough time um does that first of all does that help on on feature flagging and how we look at it and how release supports it even if you're deploying multiple times a day we it's an interesting topic we can on one weave have a hand up so John do you wanna come off mute sure um yeah I was just we were you were discussing back this is from a little bit further back um do you this tool I'm looking at from a testing perspective is Q test is that something that's already integrated into or we can make the connections or is that it's it is a tric centus tool and I did see that you are supporting tric Senus but I was just wondering if you have that they they built that integration uh so it's not owned by service now but they do have that integration on their end for bringing yeah that's kind of what I figured I just didn't know if you had that information here um but okay thanks yep but but it works the same way so right yeah and any any test results that report back to a pipeline tool so like Jenkins or Azure devops or GitHub actions we pick up those results from the pipeline so if if you have a a plugin that reports the results to the pipeline we don't have to directly talk to the testing tool to get the results we'll get it from the pipeline okay yeah and you kind of answered the question that was really what the question was do they do you know if tricentis has built that functionality and it sounds like yeah they have so okay y Yeah we actually demonstrated that not at the last knowledge uh they had a session I think uh in knowledge of 23 where they demonstrated that integration with devops change velocity yes okay cool just to reiterate any integration with devops change velocity is as far as test results or security scans or quality scans that would be uh available for policies within digital product release it's the same underlying data model that we're referencing with those queries yeah and then also if there's any integration that you have into a different area of service now but that you need to use it is a part of release then our policy engine can definitely access it um we'd have to figure out how we discover which records to look at but so long as there is a consistent way of finding information for that particular release then our policies can access it there are even ways of accessing data that doesn't live in service now it's more advanced um right but yeah yeah I figure that was I assumed that but yeah if we had custom tables as well we would have to look into that as well exactly just yeah all right looks like we have a question from Pamela you can come off mute and okay can you hear me sorry so you just mentioned what I wanted to ask and I did put something in the chat related to it for us we're using we're trying to leverage service now right from the product management as well as test management and it sounds like you're evaluating test management how much integration you're going to have so and we do use devops right but my question is like you said you can tap in we can get to any of that data but how easy is it going to be for us to do to do that if it's not out of the box right if I want to get the stories from product management and um you know using agile to .0 and also test management 1.0 how easy is it going to be for us to get this data to then be able to leverage DPR yeah so agile 2.0 um and actually by extension into test management 2.0 uh we already have Integrations uh set up and we integrate with agile 2.0 we treat it like an integration because we wanted to have kind of a clean way where we interact with every tool um the integration is a little stronger with agile 2.o um but you can set that up so when I'm when I'm let me change to being let me change real quick when we are setting this uh setting DPR up we have tools that we set up and when you connect a tool one of the planning tools is agile 2.0 so once 2.0 is is set up then we can pull in stories from aile 2.0 um the same way we do with every with everything else all the okay and and agile 2.0 has references to test management 2.0 not one. two I know um I will go ahead and apologize um I I am the reason for test management too I then changed roles to start work on devops and nobody picked it up from me so I am somewhat to blame for that transition um that that being said we are kind of investigating this is not near- term on my road map but the SPM team really wants me to take test management because we get more requests for test Management in supportive release than we do test management generally so it is definitely an area I'm looking at to bring you know to potentially bring again I've got a I have so many things on my backlog we're balancing out what order we do stuff but I'm getting this request more often where we need testing built in to DPR um so I wouldn't be surprised if that that gets prioritized in and is that do you use it mostly for release testing or testing more broadly yeah we use uh well we use test management 1.0 because it works and we haven't had a need to to move to two .0 uh unless the only way we're going to be able to get to this is through agile 20 and test manager 20 that way right um but it the process we have in place works and um we have automated test results that come into service now from other tools as well and um like certify we'll pull in um and and use the test plans and the test case instances and and automatically execute those and then we have a bunch of manual tests that happen with we have monthly releases and then major releases and then that is a key requirement you have to have a test plan that's sign off in order for you to be able to go to production right so that's one of our key requirements and we pull everything together on what we call a release package which uses the RM release table so you know it's it's very similar to where this is going and we do want to move to more out of the box because when we implemented it um 10 years ago you didn't have some of the things that we already had but you guys are you know we want to get there but we it also has to be a step approach right so first we have to get the common service data model is where are but we're looking at add the box change change models DPR um you know and other things so it's just a lot to get there yeah and test management 1.0 works for us so there's like zero reason for us to move to 2.0 at this point except the fact that all your new stuff doesn't automatically integrate so that's my struggle is how do I get to DPR not change everything at once because you know that's not practical to be able to do that so if I wanted to if I want test results from test manage 1.0 I would it it would have to be able to create a policy to do that right so and my recommendation for that the simplest way is to on your test plan add a reference to the software model table because if you the easiest way to create a policy would be to say um what is what are the test plans that look at this version so if we were you know looking at um if we were looking at that this release was for you know pre-release Z 0.9 if your test plan said this is a test plan for pre-release 0.9 we'd be able to very easily get to that data so okay that's that's probably the easiest way uh and maybe I'll even use that as an example in our next uh session on on Advanced automation When You're Building policies um because that's probably going to be the easiest way to uh to do that and and if we can do that that's great because that at the end of the day we have test stuff going on in devops too um but at the same time there's still you know a lot of manual testing going on and we need to get to it and it's within service now so I should be able to get to it right it's great we can get to all these other tools and that's where the industry is but you know I want to get to what what I have in service now too so that' be that's really helpful so I jotted that down thank you very much no worries and You' given me a good uh good inspiration for a policy to make next time great I'll be there I know that we're at time um I can stay a little bit longer if there are any last minute questions that we want to get to otherwise again next week's session is just open Office hours so you can come ask any questions there as well um uh but if there's something that we want to get to at the moment uh I can stay for a couple more minutes if needed but if not then I hope everyone has a good uh rest of their day and I'll see you at the next session thanks everyone I do have one other thing um sure so you we actually implemented it's been a few years using the service Now API to go to create changes automatically from azzure devops and to close them just like you said so with the yo pipelines so is it recommended that we use this method that this new method that you were showing today it's new to me right but um yeah to switch over to using this vers because the API just creates the change again if the if the if it's pipeline successful it'll close it if it's you know it puts the if it fails it'll put you know why it failed and you know we try and create that same data like you said and so for nonpr we just Auto approve them um but for PR we we didn't we didn't put the Automation in for prod right we you know I like where everything's going with these policies we haven't done that yet but um we Auto approving autoc close uh changes you know from those pipelines yeah so ours you can think of as we support it so you don't have to deal with that aspect of it okay ours has two portions to it uh one is that it creates the change it can also hold the yaml pipeline uh until such time as it is actually authorized so if you're if it's not Auto authorized and it does need to hold for a person um I recommend you have slas that say someone's got to get to it rather quickly to say yes or no but uh at that point it'll hold until the change is authorized and then sends it you know continues the pipeline so it can be used to not only create the change but to wait pending approval uh for that change before it deploys uh so we support both both methods there and again we maintain it so you don't have to if your system is working I'm not saying you have to move right right but again you don't have to worry about the maintenance right yeah and and we so we won't the pipeline can't run if there's not an authorized change right so when you're autoc creating the change at the time you run the pipeline it's it's going to be all fine but we didn't automate prod in the same way because you're you can't run the you're not running the pipe you're you're getting that authorized prior to running the pipeline so if we release on a Saturday we're approving those changes like Thursday and Friday like the final approvals because we we haven't gotten to the full auto approving of the pride changes yet so we can't leverage that automation from prod because people need to approve it right right so if you want to release at 8m on Saturday you can't autocreate that prod one so that's kind of what we've done but we will the pipeline can't go if you don't have an authorized change you know in cases where we even if we don't create the change we have like a check that goes into service now and looks for the product or the CI and says do I have an authorized change in the this current time within that window right we so we do have all that validation in place because even before service now we did stuff like that so it can't go if the change is improved and if you you know missed your window oh well you got to get another change authorized so we do have that automation built in but I'm going to tell our uh Cloud enablement team who supports the whole gate that we created for the service now piece here with changes about this um so that's good stuff so I assume there's information out this in your product documentation too yes so it's a part of devops change velocity and it includes and we support Azure devops I did GitHub actions today but we we do support uh Azure devops and many others okay yeah so they work with our service now developers to integrate and do all that stuff so I'll let them know about it because maybe it's something that we can leverage then okay that's all I wanted to ask thank you for your time I appreciate it thank you