Quality and UAT Testing for Developers

I have covered the following in my previous articles:

What's left? You as a developer no longer care after you are done Unit Testing right? Wrong!

There are still a couple of steps in the process that need to be covered by the developer. These are JUST as important as any of the previous steps to ensure quality of your changes. Yet another checkpoint to reduce risk of failure.

NOTE: At this point there should be a significantly reduced risk of a design flaw. The QA team will review it, obviously, and UAT might uncover a problem, but frankly if your Business Owner has been involved up to this point then there should be no surprises for anyone. QA and UAT really should be for verifying there are no defects in the execution, and that performance is nominal. Suggested usability "tweaks" are acceptable at this stage IF 1) they do not impact the schedule, 2) they do not introduce complexity that will require going back and writing new functionality (this is known as a CHANGE, and needs to be set aside for a future release - manage the Business Owner's expectations; even though this might not be part of your job! ).

So, what are the responsibilities of the Developer at this stage in the process?

Quality Assurance Testing

1. After the update set has been promoted to QA the developer is responsible for addressing defects with their code, or that their code might create (remember that integration thing?).

2. QA proceeds to test the new features. Here there should be a mechanism for defect tracking. The ServiceNow SDLC plugin has a basic version for defect tracking.

NOTE: If you are going to use this mechanism you should probably enhance it to include a couple of new fields: 1) Test number with a tie-in to the test case to be able to look at the test steps, expected, and actual results. Then on the Test Case application (from PPS application) you might want to add a field to contain requirements number(s). This will aid in traceability.

3. The developer is responsible for resolving each defect. Defects are not to be worked until QA has completed the round of testing.

4. A new Update Set is created. Use a naming convention that will identify it with the current release being tested. I usually label mine "_2", "_3", etc. as it will contain the changes/fixes.

5. This back-and-forth continues until all QA tests have passed.

User Acceptance Testing

1. When QA has passed all tests it is ready for User Acceptance Testing (UAT).

2. The developer is then responsible for merging all updates sets into one, and then re-releasing this to QA for a quick smoke test to verify that everything is present.

a. The ServiceNow Admin who installed all of the update sets during testing will be responsible for backing these out in the reverse order.

3. QA notifies the business owner to begin UAT. The same process for defect discovery and resolution applies. However, the process is a bit more involved.

a. When UAT finds a problem they will record the test used, the steps to recreate, and the expected result.

b. When UAT has completed a round of testing they will communicate the list of testing results (good and bad) to QA for review. If defects were found by UAT these will be retested by QA before being communicated to the developer for resolution. The reason for doing this is that the UAT test failure may be a false-positive, and needs to be reviewed by QA prior to being passed on.

4. When UAT testing has successfully completed, and there were any additional fixes required the developer; these will have to merged into the core update set as well.

NOTE: I would recommend rolling this merge out one last time to QA and have them do a quick (smoke) test of the release.

5. The release is now ready for roll-out to production.

Release Preparation

1. After successful UAT the QA team notifies Project Management that the User Story is ready for release to production.

2. Project Management is responsible for creation of the release plan, the rollback plan, and identification of support personnel (contact information, etc.). The developer is brought in to fill in any blanks.

3. The roll-back plan is a list of ordered steps that will be used to remove the new release from production (should it be incompatible or fail). Don't skip this step!

a. The roll-back plan needs to include Update Set back out order, data removal and replacement, and Fix Scripts that might be used to reverse data changes (need to be taken into consideration as part of the development process - sorry, forgot this in my last article ). Essentially work done to make sure of a clean reversal if possible.

b. NOTE: If a new plug-in is installed; this will not be able to be backed out.

c. NOTE: Data updates are a bit trickier. You will want to back up any tables to be modified locally before applying a data import (update transform or upload).

4. At this point, or earlier, the Project Manager cuts a change request for the entire release and sends it through the approval process. The Change Request is a great engineering step to schedule the change; this is a planned release after all. This should place the modification for roll-out at an off-hour or maintenance window that will not, and should not, impact production users!

Production Release

1. The developer is responsible to assist the ServiceNow Admin if necessary. Unless something goes badly the rollback should not be needed. If problems are encountered and they are configuration issues then they can be dealt with in production. If an actual defect or coding issue is found this needs to be handled via a hot-fix roll-out process (simply a sped-up version of the SDLC).

a. The Admin will pull the merged update set for the release from the DEV environment.

2. Should a hot-fix be required, for a post-production defect, it goes through the same process as a normal release.

3. Should a rollout to production fail it may be necessary to roll it back. This is done through the roll-back process created for the release.

4. Admin smoke-tests the application. This is simply to see if things look ok; a quick-and-dirty look at the modifications to see if they were really applied.

5. If everything looks good then Admin updates the change request.

6. The Project Manager will then announce to concerned parties that the release has been completed successfully with a list of features pushed. Victory has been achieved! Time to party!

Post Release

1. Here the developer is basically done. If problems do occur then defect fixes need to go through the process starting at the beginning and working their way back through the requirements and so on. This will end up being what is called a maintenance release. Again, planned not slammed through.

Conclusion

The entire development process I have described in this set of articles is not meant to be written in stone (although that would be cool). It is meant as best practice advice from someone who has had to do this a lot in development over the last 30+ years, and in ServiceNow over the last 4+ years. It is of course flexible; to meet your company's business needs. I highly recommend adopting some form of development process if you do not already have one. You will see your defect rate drop like a rock, and, of course, your quality and user satisfaction soar!

Steven Bell

If you find this article helps you, don't forget to log in and "like" it!

Also, if you are not already, I would like to encourage you to become a member of our blog!