hey everybody good morning and good afternoon thanks for joining us today we definitely tell you to have you on board for another one of our virtual user groups before we actually get started here I just want to do a couple technical checks just to make sure everybody can see my screen and hear me so if you could raise your hand just as an indicator that there are no issues so yeah we got a couple hands up good stuff all right cool so yeah we're gonna get get going here again we're we're definitely excited to have all you with us today we're also excited to have millennium pharmaceuticals with us to share their cool story of how they implemented both compliant and and specialized ServiceNow platform under FDA regulations so definitely a unique challenge to say the least so a lot of you have attended one of our bugs in the past and you probably know a little bit about us already but before jumping in just want to give you a reminder of you know who we are just a little background so up on screen you'll just see a couple numbers that we take great pride in 200 plus cloud experts 10 plus years of ITIL and service management experience and then actually over 325 implementations so I actually say that last that last number for last just because that is the most implementations done by any ServiceNow partner so again that's definitely set we're proud of but it's worth mentioning that we're more than just a implementation partner we offer a full range of products services and technologies to help you during any stage of the service management lifecycle and all these business lines that you see up on the screen are focused exclusively on the ServiceNow platform and that's actually garnered us the exclusive status of being a preferred services partner of ServiceNow so a little introduction to our speakers today again very happy to have millennium with us being represented by John Chun who is their director of IT operations John is actually going to be driving a majority of the presentation today but we also have Dan and Chris from fruition who worked very closely with the we're all in the Millennium project so we have them on hand just to give you a little more technical insight into the project so last last screen here before we get going again this is a user group we emphasize that so please be active engage we don't save questions and answers for the end so anytime you have a question go ahead and enter it in that question box so I'll be keeping a close eye on that so as soon as we can we'll we'll get to that question and answer it the other piece we had and play here are the polls I think we have about three of those for today so be on the lookout for that little icon in the bottom right corner that'll just be an indicator that that specific page or slide will be releasing or opening up a poll for you to participate in so without any further delay I'm going to go ahead and pass the presentation on to John so one second here John let me know if your said John just pass the presentation off to you okay I just picked that up and then I'm sharing my PowerPoint can everyone see my slide here awesome yeah I can see it John good all right hello everyone my name is John Sean I'm director of IT operations and millennium pharmaceuticals in Cambridge Massachusetts I also have a share of the day here with me today she's our manager of IT operations and she was the pm4 project manager for ServiceNow deployment we transition from remedy 7.5 and when live with the service now on April 1st and we've been very happy with it so far and I want to introduce a little bit about millennium pharmaceuticals Malayan millennium Pharmaceuticals is a on ecology focus the pharma company here in Cambridge Massachusetts and we are actually a wholly owned subsidiary of a number one Japanese pharma company called the Takeda I'm not sure if many of you are familiar with the Takeda but they're not really a household brand in the US but they actually had just expanded to European market as well as the rest of global market by acquiring another European company called and I committed about two years ago and and that millennium itself is just a focused on developing and marketing cancer drugs and our flagship product is called a Velcade and it's for multiple myeloma and we currently have actually a quite a few drugs candidates in the pipeline so today's agenda is going to cover what is validation and what happens during validation and what problem are we trying to solve here and then can we reduce the documentation time by say about a factor of 1,000 and that I think is a pretty ambitious goal but that's something that we accomplished here so I'm going to walk you through those steps of how we actually got to that and and then we'll be giving out some technical tidbits so you can actually do some of these on your own possibly with the help from fruition and then I'll give you a top three takeaways from today's presentation and then again as Mike said that we're going to have a questions coming in throughout the whole presentation so please do feel free to ask questions at any time so at this point I like to conduct a poll of Mike do you have that question ready I do thanks John or send that up so yeah we we're gonna start off with a kind of quick basic poll here and just ask and I'm gonna launch right now so if you can't see it I'll read it out loud so the first question or poll question is do you work in an FDA regulated company so the options are yes no I don't know so hopefully nobody votes I don't know but we'll see so we'll give everybody probably about twenty seconds to go ahead and vote and then once the votes stop stop coming in we'll close it up and share the results with everybody so we'll probably give it about ten more seconds here all right so we're gonna go ahead and close it it looks like we had 80% of you vote so thank you for participating so here are the results which actually is pretty surprising I'm surprised there's that many yeses so again the poll question is do you work in an FDA regulated company forty percent said yes and sixty percent said no so I'm gonna go ahead and go ahead and close it and pass it back to you John but at the end of the of the webinar I can actually go ahead and a follow up and share these results with everybody as well just so you have an and so hide the results in John your backup okay great okay that's that's good to know thank you Mike and the reason why I asked that question was that at Millennium Pharmaceuticals being an FDA regulated company here we had to follow through some very strict protocol and what's called a validation process and I'm going to walk you through some of the things that we have done here but understanding that who works in that FDA regulated companies is going to be helpful for me too just to make make sure that you all understand what we're talking about and one more thing is that what I'm presenting today is applicable to anyone who is outside of FDA regulated companies as well so you are going to have takeaways at the end of the presentation that will help you to make your documentation process better and possibly your the deployment process better as well so please bear with me as I go through here and as I come across any FDA specific lingo I'm going to explain to you what that is and one thing that I also want to tell you at this point is when I talk about QA QA stands for quality assurance and in many of the industries QA means software testing within the FDA space QA means ensuring that our processes do comply with the FDA regulations so when I say QA that means there's some specific department within my company where they oversee our processes and documentation just to make sure that they are conforming to FDA regulations and they do not necessarily our software testing departments so having said that I just want to give you a definition of what is a validation so within the FDA space we do define a validation as this here and the key phrase here is documented evidence so establishing documented evidence which provides a high degree of assurance that a specific computer system will consistently produce a result meeting at the predetermined specifications and quality attributes this was originally written or written around drug manufacturing but I kind of rephrase this a little bit to be applicable to in our case where we're more concerned with the computer systems than manufacturing machines so I rephrase that a little bit but the key phrase here again is we have to produce documented evidence that everything that we are producing that when we deploy ServiceNow is going to meet that specific requirements that we set forth in the beginning so moving through that what happens during validation we do a lot of documentation and then more documentation and more documentation and then we also have to go through lots of revisions and more revisions and this is a very labor-intensive and very time-consuming error-prone as you can imagine and it's very hard to track changes with accuracy and I'm sorry as you can imagine the we as we go through the deployment process we these the requirements that we gather in the beginning are not static meaning that they're going to change over time and with this kind of very rigid documentation process it's very hard to track those changes as things become very dynamic and in the next slide here I just want to show you graphically how the whole process of flows GAMP GMP is an industry organization and they promote it see that that's actually GMP actually stands for good automated manufacturing practice it's a practice that's being promoted by an industry organization called is PE and their primary interest is in drug manufacturing but again they have extended over the years they are a good practice into computer systems as well so this diagram is showing you the overall process of going from user requirements which starts at the top left and then going down into functional requirements and then turning into detail design and then you configure a system such as ServiceNow and then you conduct a unit testing functional testing eventually leading up to user acceptance testing before you release the product to the end-users so over on the left side as you come down you go through the specification process and then on the right side as you go back up you go through the revocation process so this is this diagram is called the V diagram for validation now when you look at this the process here it's not really specific to pharma companies it's not really specific to FDA regulated industries rather I'm sure all of you can relate to having having to go through these steps in and in your everyday life as you're deploying any system including ServiceNow so here what problem are we trying to solve what were some of the pain points that we were trying to resolve here and before I go through the steps again I like to turn the microphone over to Mike for him to conduct another poll here please thanks John so yeah our next poll question the question is does your organization keep test cases and test scripts for all your live processes so available options are yes no I don't know so again we'll give everybody a few seconds to go ahead and vote and then we'll share the results so we still have some it's coming in thanks everybody for participating I'm gonna go ahead and close it up and we'll share the results here so again the the poll question was does your organization keep test cases and test scripts for all your life processes so 67% said yes 27% said no and then 7% said I don't know so get go ahead and hide those and pass it back to you John all right thank you very much again and I'm just back to here okay okay everybody can see my slide here okay I'll take that as a yes okay and so some of the challenges that we ran into was that we were collecting requirements in Excel and Word and some of you may also be doing that but I felt that that was not a efficient way of a collecting requirements and then what we also had to do was we had to then transcribe those requirements into word templates again when it comes to producing these documents we have to follow very specific guidelines and requirements are dictated by what the FDA is expecting from us and our QA Department again the department that dictates that oversees our processes they have produced a certain set of document templates that we all follow and then we have to maintain traceability across all these here and I'll show you as we go through those the process diagram that I just showed you we now have to maintain what's called a traceability meaning that which user requirement is tied to which functional requirement and then which functional carmen is tidy which detailed design specification and in all the test cases that all the test scripts so if you have a test script that failed you must be able to trace that back all the way to the user requirement and and and be able to tell say to an FDA inspector that hey yes so we had this test script of failing on us and then that actually tied all the way back to this particular very specific user requirement so all that relationship is documented in what's called a traceability matrix document and then whenever we add or remove any of the requirements or test scripts and whatnot we have to revise the traceability matrix document in addition to any of these specific documents we're producing as part of our project and when we asked our contractors we had actually had technical writers as part of the project and we when we asked actually two vendors about the effort that takes to produce such traceability metrics they came back to us both actually the same numbers 40 hours of work just to create traceability matrix so on the next page I want to show you a sample traceability matrix and this is kind of a document that you have to update maintain and then again takes about 40 hours effort for producing this and and and and make sure that there are no errors and as you can imagine here you know missing a dot there is going to alter the meaning of that particular section of the document so you can imagine the type of scrutiny that you have to go through to make sure that this kind of a document is is correct and throughout the entire cycle of the project deployment so at this point I want to just give you just quick background information on how why I came up with this idea of using additional tools to maintain our requirements and then eventually be able to generate this kind of a traceability matrix automatically about ten years ago I used to work at a financial company in New York and at that time I transitioned from being a project manager for website development to IT operations and my first task was to revamp change management and at that time we were using old remedy system I mean that was a new remedy at the time and the remedy web client was broken so the engineers could not submit to change tickets through remedy so what they were doing was they were just sending email in free text about upcoming changes and I said well that's not gonna work so I try to fix that remedy at the time but my manager to decline to provide any money for hiring contractors so having had some web development experience myself I started actually putting together some web-based tools just for me to keep up with all the changes eventually that tool became very popular we decided to use that company-wide and that became our requirement tracker Incident tracker change management tool as well as a release management tool and we also tracked the the testing software testing records in there as well and by that time I was doing change management release management and software testing so that kind of web tool gave us a very good way of tracking everything making sure everything tied to one another and be able to produce the reports and whatnot almost instantaneously so when I came to this point of millennium pharmaceuticals I did ask others hey what kind of other tools are you using and when I realized that they were not using any tools I said oh wait a minute you know we cannot go in like this and that's just going to be a recipe for failure for the project so so that actually was a bit of a background knowledge that I had that prompted me to go and explore better ways of producing this type of traceability matrix and the overall project documentation so moving on to the next slide here I'm just gonna walk you through how we solve the problem again I did leverage some of the background knowledge experience that I had from my back in the old days of a financial company but as soon as I saw fruition requirements enter this actually came with our instance of service now when we engaged a fruition one thing that they also introduced besides their professional services was that they installed the fruition requirements center in our instance of service now that only came with well with what we would consider functional requirements and about tracking modules so what we did was we added additional modules for user requirement specifications detailed design specifications test cases test script and test runs and then we linked all these using what's called the reference field and related list in service now what that means is that traceability across all those different records are maintained automatically and again based on some of the background that I had I was able to just put this together in about one hour of time with in-service now based on what fruition had already provided plus the additional modules and then just relating all those together so that took only about an hour and then we were able to generate traceability metrics in about two minutes of times so that's 40 hours of work again that's that was an estimation from our the two vendors got reduced down to two minutes now that's more than a factor of 1,000 so that's the number that I actually gave out in the beginning but that's we were actually be able to see that in our experience and at this point I like to turn microphone over to Chris Damon who was our Engagement Manager and then and then he can explain to you about a little bit more about fruition requirements Center Chris hello well hopefully hopefully he's there John okay I'll just be able to do it if not we can move forward or maybe have Dan jump in and explain a little bit the requirements Center fees available either you there Chris or Dan yeah can you hear me this is Dan yep can you ready Dan thanks okay cool yeah and Chris if you if you're muted you can jump on any time too so yes I'm here okay thanks dr. overview sure no problem sure so one of the things that you know myself and a lot of us apparitions have experienced what we've experienced over the years with all the implementations we've done is that the quality of requirements that you get can range anywhere from someone took a picture of a whiteboard with their big vision to tons of requirements that their pickup hole multiple three-ring binders and the team of bas parse through it all to figure out what it is how we really want to design this system so knowing that there's great variances across what how we get requirements from from our folks the clients are engaging with we develop this first requirement center on ServiceNow and what it allows us to do is have a you know we still may have lots of requirements we still have may still have very few requirements but what if those allow us to do is allows us to put those requirements into the ServiceNow platform so a couple of quick benefits on that one it's collaborative so what we like to do with requirement center is open it up for the team members participating key stakeholders those that are participating in the project to submit requirements into requirement center so the collaborative nature is great I mean how many people on this call have been on a project where there's some Excel spreadsheet that just keeps bouncing from inbox the inbox to inbox or it's on a drop chere and a SharePoint site and you know on someone's desktop and you never know which version is the you know the authoritative source of those requirement so we get away from that requirement center we also have added more flow to it so we have states where it goes through a status of new when somebody submitted it it's just an idea or it's the shell of a requirement or something we want to consider to take on in the engagement we will then review it as the fruition team to look at see how complex this or what's the level of effort to define it to make these requirements realize on the platform we will then discuss it with the client in this case you know with John and his team at Millennium and then they would approve it so once it's approved that goes into ready for development State and that's where our team or we're partnering up with let's folks on the client side we will develop the work it will go through a series of unit tests it will then go into formal QA and hopefully the the expected result is that through formal QA it gets proves and it gets this flag for ready for deployment ready for rollout and from a traceability perspective we now have who entered it who is interacting with the requirement you know adding additional information when it was approved - approved it who did the development work and actually we capture update sets - so if you're familiar with updates ed serves now what it allows us to do is say this requirement was built in was in this particular up they said that we can then promote the other instances so that's one half of the requirement center this is the entering of requirements the other half of it is and again in the spirit of traceability we also have the defect side of the requirements in our application and the defects we can say against this requirement I'm opening this defect it failed here was the the expected result here was the actual result we can assign those defects the folks doing the development work and the configuration work they can then resolve it of course send it back through to a and hopefully it gets resolved and then at that point we not only have when it was submitted when it was approved to built it what updates that it was applied we also now know all the defects that were logged against that requirement and then when that defect was introduced into which update set so again traceability to where this is stored and how it was promoted across the instances so that allows us to have full traceability from entering through promotion and it's been it's had very very positive results it's taken us away from that the old days of like I said pouncing around you know Excel files through inboxes and it's been you know it's been extremely helpful a lot of our engagements and engagements and it certainly was helpful at Millennium so all right thank you much questions about that okay back to good job thank you anyone have any questions to Chris I'll show you a couple more slides here and show you how we actually worked with the pollution requirement Center plus some of the extended features that we implement it and then again if you have any questions please do feel free to ask at any time so here is a screenshot of what it looks like within service now again if you look at on the left side here the fruition requirement Center came installed with our instance and then there are several modules there and the second module functional requirement is what came with the fruition recording Center and at the bottom the defect that also came with the fruition requirement Center and and then we added those other modules in there as we actually needed them to complete our release cycle and documentation and if you look on this here the application drop-down this is was actually where we were keeping track of the different applications we were configuring with in service now so we deployed the incident problem change and config so we had those four applications listed in the draw down plus we had some like a general or system-wide requirements as well as like a FDA specific requirement so we were keeping track of all those by using that that drop down there and then as Chris as I mentioned here we were just working through the workflows through the state drop down value over there so in this particular case it's showing death pending meaning that whoever is assigned to here now is working on configuring that particular requirement in the system and that that's showing that assign the assignee I'm sorry and then so this is a screenshot of the requirements the user requirements and then on the next page I'm going to show you how the reference fields were tied to the related lists so at the top of the slide here I'm showing you the user requirement on this screen that you just saw on the previous slide and then at the bottom of the of the user requirement form are the list of functional requirements that are related to that particular user requirement so in this case there was one user requirement that ended up with the four functional requirements and at the bottom I'm showing you a a section of a functional requirement the top section of the functional requirement form where it is showing the reference field so that that red box over there that I'm showing you is actually how the this particular functional requirement is tied to the URS the user requirement specification so in this case that urs number is specified there that means this particular functional requirement is related to that particular user requirement and then using the related list in the urs now you are able to tie the two together so this is how we establish the relationships and the traceability across user requirement and functional requirement and we repeated the same thing across all the different modules that we were using and moving on to just showing you all the modules that we configured here so at the top left we are showing user requirements which had one-to-many relationship with the functional requirements and then functional requirements also had one-to-many relationship with the test cases to the right and then each test case may also spin off one or more test scripts so there was one to many relationship and then each test script may had one or more test runs so again that turned into one-to-many relationships each test run may create a zero or more defects so there you can again see either zero too many relationship and again those defects will eventually have traceability tying it all the way back to those functional requirements and then user requirements and as for detailed design there at the bottom I'm gonna get to a little more detail around that in a couple more slides but we decided to go with many to one relationship for detail design and I'll actually show you why we chose to go that route in terms of the coloring here the green boxes were the modules provided my fruition and the purple boxes were the ones the wet that that we custom-designed on our own and at this point Mike there's another poll question please yes there is and this is actually our last poll question for the day so actually this poll question might tie back to an earlier slide a little better but it's still relevant to the overall discussion so I'm gonna go ahead and launch it and read it out loud for everybody so the question is does your organization currently capture communicate and report on project requirements and the options are yes no I don't know so again we'll give everybody about 15 seconds to vote and and I'll go ahead and share the results afterwards and again as John mentioned earlier you know don't hesitate to ask any questions at all there's no such thing as a bad question so we're available I guess all this means John you're doing a heck of a job so far good I'm glad everyone it's clear to everyone so we are going to close the poll and share the results so again the question was does your organization currently capture communicate and report on project requirements seventy-five percent said yes which is actually very good eight percent said no and then seventeen percent said I don't know so we'll go ahead and hide those and John you're back on great that's great and at this point I also like to turn the microphone over to Dan Dube who actually worked with us on creating test cases and test scripts and also executed test runs and encaptured any failures in defect tickets so that we could come around and fix those as necessary and I just want to clarify here that as part of the FDA validation process we actually did our test runs in two segments one was that sort of a dry run and that's what we did with with the fruition working with the Dan debate when it comes to official testing conforming to the FDA regulations we actually conducted those in-house separately but we we were able to leverage what Dan had done before so those are two actually two separate steps of testing in our case but when it comes to the first a dry run a dance team conducted all that so I like to just ask a Dan to just give us how this worked out for you yeah yeah you pretty much took all the words right on my mouth but yeah definitely as far as this is a good slide to have up here is basically like as John was saying millennium came to us and wanting us to do the first dry run of their of testing their functional requirements before they actually got into the kind of meat and potatoes of the FDA testing so yes so working with John we you know he built the shaded areas and purple which allowed us to take the requirements you know they were already in fruition requirements center and then write all the test cases in subsequent test scripts to those cases and then once those were approved by millennium our testing Services team went off in and executed all those all those test scripts subsequently logging defects into the into the green shaded area which which in essence turned around and were assigned to the fruition and millennium technical consultants that were on the implementation project who could you know quickly turn around those defects and allow us to you know go in and and run through again and execute them again to make sure that they were fixed before millennium moved on to the next phase and in their in their in their validation process Thank You Diane thanks John and and would this kind of work a flow in the process we defined between millennium and fruition we were able to very rapidly execute these cycles so there really was a very little downtime for us in terms of communication or misunderstanding or anything falling through the cracks and on our side cheryl day was the project manager and part of her daily routine was making sure that everything was being captured and processed and being updated and worked on so at this point i'd like to just turn over the microphone over to Cheryl just to hear from her side of experience and how this entire process and the tools helped with her project management module affarin thank you John having those modules there and the information clearly available to me as the project manager was extremely helpful we had several engineers working to configure the various requirements it was very easy to see who is assigned what tasks and what condition it was in so every day I could go in and see what was going on and who had it when it came to the testing it was very very helpful to having fruition help with us they were able to draft the test cases and the test scripts and do the initial test run when those test runs turn into defects we were able to process those quite often it turned out that the functional requirement that we had written ourselves was unclear to be able to update that and get that updated all the way through to new test scripts so by the time we got to our internal official test run we had very few variances and they saved us a lot of time Thank You Cheryl and when sure is talking about variances again the FDA regulation dictates that we fully document any test of failures and it actually takes a lot more documentation effort trying to document those failures then in my opinion fixing the defect sometimes so again not having had too many variances whether with the results with the help from fruition the initial dry runs or dry run testing it was a great help for us and that eventually saved us a lot of time as well so I'm just gonna move on and then explain to you why we chose to do in certain ways for the detailed design specifications hey John so when it comes to detailed designs but yes hey John sorry to interrupt we actually have a question for the audience and it oh yes it's back to that last slide we were on yeah so feel free to answer Dan jump in as well cuz this might be more up your alley but the question is how do you differentiate between cases test scripts and test runs okay I think I'll explain that to you because these were actually something again we were subject to the way that our QA you know the the team that dictates how we define our processes in documentation so this the those distinctions came from them so let me just give you a quick explanation on the differences so whenever you have a functional requirements for example when you click on that button it has to do certain activities around the screen those are those come down as what we call test cases so a test case would be would be representing for example if you have to approve a change then test case would be around approving changes and then from that particular test case you would actually spin off multiple test scripts meaning that if the test was a for example a major change then then you have the particular steps you click Li click here and there a pic this value and then submit the change for approval and this is what's expected results of that particular step or a set of steps for major changes but if you have a minor change for example then you may also have a little bit different steps of processing the ticket and getting approval as well so that may be test script two number two or you could have a multiple test scripts based on different combinations of those change attributes so and all those different test scripts test scripts are just set of steps that you go through those may all fall under a particular test case again called change management approval process so that's how they are differentiated so when you have a test script written that means you have a steps you have documented and a test run would be someone following those test steps and executing those test steps and then documenting the results of the of the of the test run so that would be the box the module represented by test runs is that is it clear do you have anything to add I just another a good example of test cases and their corresponding scripts would be like on a you know when you're making sure fields are mandatory on an incident form or something like that where you'd have a positive and a negative test so the test overall test case would be the you know having these fields as mandatory and then the scripts would you know one script would be one child script under that test case would be around you know filling all of filling out all the mandatory fields and hitting submit and it working and then like a negative test would be you know around leaving that one of those each one of those fields blank and hitting submit and getting an error message kind of thing be a good another good example of a test case an end script yes a very good example there then you know given your experiences were working with the multiple clients in various industries we just say that like having test cases test scripts and test runs so would they be more of a typical of just about any industry or is that was it something specific to our industry no it's definitely it's definitely not just with with with FDA and millennium it's everywhere every every testing project that fruition takes on goes through the same these same steps yeah okay that's good to know and as I've said in the beginning I do think that all the millennium in our particular case we did have to follow through more of FDA regulated practices I think that overall I mean this is a good practice that anybody any industry would be following so I think there's a lot that we can share together even if you do not work in the FDA regulated companies so move on to the next slide here and again this I'm trying to explain to you why we went with the way that we capture detail design and the way that we define detail design was there hey if we had some kind of a functional requirement detailed design is how that was implemented in ServiceNow meaning that was how it was configured in ServiceNow so plan a was that as Chris was explaining all of the configuration that you do in ServiceNow are captured in the update sets so we said okay we'll just use the update sets and and update sets I don't know if everyone is familiar with the update sets and ServiceNow but it basically captures like the configuration that you're making so it captures like the values and what not and which table or tables that they are stored in so it does capture everything that you do and in fact you can save that update set and then apply that to another instance of your service now and then that updates that will be applied to that other instance it's just like the way that you had to manually configure in the original instance so we said okay we're going to use that and it's a very exact XML document very technically technically in in nature but and has a lot of details but we'll just use that and then we'll just and then update set contains multiple what's called the customer updates and the customer update is the individual update being made as you're going through the configuration steps and we say we're going to access a link that customer update to each functional requirement specification but but the update sets are not really for human consumption and I don't know how many of you actually bother to open up some of those update sets and customer updates and look inside and and it's a it's a just a little mind-boggling and then also they're very very many many customer updates are being produced as you were configuring your service now so at the end we said wow there's just too many and then it's really hard to track them manually and then try to tie them into the functional requirement specifications so what we did was we moved on to plan plan me where at the end of all the configurations so we're not going to bother trying to pick out the update sets as we are configuring a service now so what we said was all right let's just wait until we're done with all the configurations and bug fixes and everything that that we're trying to accomplish here and then what we do is before we release that we just go against the system's again I'm not sure if how many of you are familiar with the systems and service now but this tables in ServiceNow capture all your configurations so basically what's being captured through update sets they are in fact being written to sis tables in in ServiceNow so we went against the assist tables and then we just read all the configurations that we that matter to us meaning that if it's an instant form what are the fields that we're exposing there and how do they tie in to the underlying database table and the columns so we wrote pretty extensive set of a sequel statements reading off those cyst tables and then we actually converted them into word excuse me some more human-friendly format so that you can actually output the cysts able contents into Word document or in our case the templates that we had to use as part of our the QA process so this final document would just capture a snapshot of at any given time of all the configurations that we had done and then at the end we were able to auto-generate about a 1,000 page document so so this is a just pictorial diagram of how we went about Auto documenting some things that we did so once again we captured all our requirements and all these subsequent steps in service now using Friesians requirements center as well as some additional modules that we built and then we use the ODBC connection into service now to read off everything that we did in those ticketing ticket records as well as in the system and then we used Microsoft Office automation to to extract the data from service now through ODBC and then format them in the way that we want it and then produce Word documents and Excel documents and when I say script here in office automation it's mostly around like VBA Visual Basic for applications for Word and Excel so I'm sure many of you have like run macros in Word or Excel and then that's essentially what helped us produce those documents so top three takeaways so what we decided to do was we wanted to use ServiceNow for what it does best and these are the things that ServiceNow does best in our opinion and that is like tracking tickets rep like the requirements and collaboration as Dan and Chris said anything that we capture in there are is visible to everyone so across fruition engineers and any other members of fruition anyone at Millennium we could see everything that everyone else was doing and everything was up to date so you don't have any Excel spreadsheet that's sitting there being stale and also relating tickets so again we use the related list feature within ServiceNow to tie all the tickets all the requirements and specifications so we were able to have the traceability built in automatically by design and then we integrated with the microsoft office automation particular using the ODBC n scripting and then we in the end that we had of increased efficiency in accuracy and transparency so that is the end of my presentation here and I'll be very happy to take any questions yeah thanks Joe actually lucky you we do have a couple from the audience and the first one is so did millennium follow a waterfall or agile or iterative type of development methodology so overall our our sdlc process is waterfall but we had like miniaturized iterative or I should say miniaturized the waterfall built into the in the trends in in the you know in the overall waterfall model meaning that so what we did was we did not just had a requirements fixed in the beginning and then just to worked off that what we found out along the way it was that the original requirements were not quite the way we wanted once we implemented it or once we tested it so we had to go back and then we rise the requirements along the way and then that triggered iteration and that we went through actually about three four iterations before we said okay now this is good enough for release so again the overall it's a waterfall in that in our the QA process each document that we generate must be reviewed by our QA personnel and then they actually have to sign off on those so so that kind of keeps our process rigid in that we do have to produce the user requirements specifications in the beginning we have to get the approval and then move on to the functional requirements etc etc but along the way if we do find something to be revised then we are allowed to go back and then revise those documents we had submitted and got approval for but that that review process will now have to take place again those additional changes that we introduced so that's the process that we followed nice thanks John yeah good question good answer and we actually have one more question here so do you see the potential to reuse the requirements Center for other systems you need to validate that are not ServiceNow yes and the fact that question includes the word validation indicates that it's coming from somebody in the FDA regulated company yes so we did have some discussions with our QA personnel here and we did not have to validate the requirements center that we used based on the fact that we were using the resulting word document as the basis of our GXP process and when I say GXP that's the FDA regulated process gxb stands for like a good practice and actually means that there's like a good clinical practice good laboratory practice etc so overall whenever we have an empty regulated the process we just call that GXP so that was the basis for our not having to validate that particular functionality of service now that we leveraged going forward we like to eventually validate this tool by the way we did not migrate the requirement center in our eventual the the final production release of service now that we deployed we actually stripped all that off because we did not want that to be part of our validated platform but in the future we'd like to actually have a requirement center like tool available to us in production environment that is validated so that we can capture requirements for any other systems that we may be deploying in the future right is that answer your question I think so John you know and I'll keep a keep an eye on the question box here to see if there's any follow-ups but I think that's a great answer so thank you and one last question here and and John you could probably take a step back so I think this is more for Chris and Dan on the fruition side the question is and hopefully Chris and Dan yourself out there is it possible to use the requirements center if fruition did not do our initial implementation yeah I mean right now it's just available to all service are all fruition customers so whether you're in you know managed services virtual admin or doing some other project with us it's it's available and load it up for you I don't think there's a decision has been made yet on making it available for the general ServiceNow customer base I know it we are working on our ServiceNow app store which will allow you to go out very soon and kind of download any of the fruition applications that we've built and release packaged up and released that that kind of tag onto the ServiceNow platform and I'm and the requirement Center will probably be part of that but I don't know if there will be a cost or what what that will entail yet so it's kind of stay tuned for that I think marketing will probably be releasing more information about that as it gets closer to a decision made on what we're gonna do with the requirements tracker for non fruition customers cool thanks Dan yeah it looks like no more questions right now so John if you could just move to the next slide Thanks yeah and if and if you guys think of any questions at any time related to the millennium story it's okay to send us an email afterwards both my email address and John's email address are up here so if you want to again learn more talk more about millenniums FDA nation process if you have any questions about the requirements enter or even the virtual user group send us an email and we'll will definitely get back to you so just a couple closing remarks just want to say thanks to John and Cheryl and millennium for participating today it's a the whole millennium story is just amazing so we're very happy to be in a relationship and partnership with them so thanks to the two of you again and thanks to everybody else who attended today for for spending an hour with us lucky for you actually get thirty minutes back in your day so enjoy it a couple quick notes we're gonna send out a follow-up blast probably I'm thinking early next week Monday or Tuesday before the holiday with a PDF of today's deck along with the recording of the webinar so if you missed anything today or want to revisit some of the material you'll have both those resources available to you and also stay tuned we'll be sending out some more details on our next virtual user group scheduled for August 2013 so we hope to see you all of you there for that and I think that is that is it everybody so again thank you for participating and we look forward to talking to you all soon thanks everybody bye