Build developer friendly APIs with JSON Schema Validation

CCB1129-K22

## Transcript X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:0 [MUSIC PLAYING] Hi. I'm Jarod Mundt, vice president of engineering at Clear Skye. I've been working on the ServiceNow platform since 2014, and I've spent time at customers, implementation partners, and am now working at a product partner where we publish identity security and governance solutions into the ServiceNow store. Today, we'll talk about JSON schema validation and how you can take your APIs to the next level. Now, this is not going to be a deep dive into schema validation and also not going to be a deep dive for importing external JavaScript libraries into your instance, but we will look at the JSON schema validation concepts and why it will be worth the investment to implement this into your instance. We'll also take a look at some scripted REST API examples, but this validation can be used in any other server-side or client-side code in your instance. Both the schema and input object are both written in JSON. JSON stands for JavaScript Object Notation, and it's the data format that powers most of the internet's APIs, including ServiceNow. There are things like curly brackets for objects, square brackets for arrays, and things can be nested inside of other things. Now, the schema on the left we see has a very specific format so that the validator code knows where to find each object. There are many validators available online in a variety of programming languages. For these examples today, I'm using Tiny Validator a.k.a. Tv4 out on npmjs. I'll show a link to where to find this library at the end of the presentation. Now, in our first example, we have a very basic flat object. So hypothetically, in this example, we have a client that is sending the input object into our system, and the result of that will be some expensive database long-running queries and a variety of inserts and updates across multiple tables. So ideally, this will be an all-or-none transaction. We don't want to get half of the transactions completed and then find out that some of the data is missing or in the wrong format. So what we do is we pass both the schema object and the input object into the validator, and then we'll get a third JSON object that we'll call the validation result. Now, in this case, valid is equal to true. Great, we won't have to stop processing or send any errors back to the client. We can begin our processing. But what if our input object-- what if we had an age of negative 21? Technically, according to this schema, that's still an integer, but that may cause errors when we start processing. So in example two, we've added a few more attributes to the schema definition. Now, if we try and process it now, in theory, negative 21 is out of the scope so we'll receive an error. Notice the message and data path attributes of our validation object. We see that it says specifically that the parameter of age is less than a minimum of zero. And we did this with very limited code. This is part of the JSON schema validation standard. Also notice that we're on only example two and both the schema and the validation result are starting to get rather long for a relatively simple example. The definition on the left is 22 lines long and the validation result is 17 lines long already. So pretty quickly, when we start nesting objects and arrays, both our input object is going to get big but also our schema validation or our JSON schema definition also start to get pretty big. So for the next example, I'll only highlight the things that have been added or changed so we won't have to shrink down all of that code to get it to display all on the screen at the same time. So here we've added a few more items, specifically favorite movies. So in the schema, we've added a type of array, and the items inside of that array must be strings. On the input object on the right, we see that we've added some favorite movies, but one of them is in a integer format and not a string. So let's take a look at that. We now get valid equals false. And specifically, back to the message and data path portions, we see invalid type, number-- it wanted a string-- and data path, under favorite movies, position one. So arrays start at zero, so that would be the second item that was submitted is the problem item. Now, there are quite a few other additional JSON schema attributes to add value and to fill out the schema definition. Specifically-- I won't go into a variety of them, but one to look out for is called additional properties. So if we set that to false, any extra items that are submitted in the input object, if they don't have a specific definition, we can error out on those. Now let's move this stuff out of theory, get it into our instance, and build some scripted REST API checks and responses. So prior to this step, we must import a library into our instance. So like any good TV baking show, I'm going to skip over that step of selecting the open-source library and also the import to script include process and I'll go right to the code execution. Now, as a side note, if you don't recognize this interface, it's called the Xplore toolkit and I do highly recommend it for scripting and working with data on the platform. It's available from the ServiceNow Share out on the Developer Portal. Now, once our script is proven to work, we can just start coding away and start creating scripted REST APIs, right? Well, not so fast. I want to have a quick chat about architecture standards first. So when I've demoed this and implemented this in the past, one of the common questions is, why not just use the table API? That way, if somebody wants to integrate with ServiceNow, we just point them at the docs, give them a service account, and then they do all the work. We don't have to do anything as the ServiceNow team. My response to that is, as a whole, what needs to happen is the organization has a requirement, the organization needs to get some work done, and the organization needs to support it. So we get to kind of think about which teams should best pick and choose to support that. So I typically recommend centralizing as much of the development and centralizing as much of the pain as possible, and that allows us to reduce friction. What we don't want to have is making a junior Linux admin on another team that's writing an integration inside of a Python script have to learn the table API, do four lookups to get various sysIDs of locations, users, change request numbers, CMDB CIs, other things like that. We want them to just go right to building their query. What this also does is it increases happiness, can reduce the time to value, and also reduce costs and time on the other teams. Another advantage to building things in scripted REST APIs is that you can conform to the culture of your organization. I've worked at places where they had shorthand codes for locations. Everyone referred to people in every system by the employee ID. And so, by building it into a scripted REST API and not the table API, we get to choose which value we use for each of these things. And third, we get to continue turning ServiceNow into the control tower. We get to put ServiceNow at the center. Now, whether that data lives in ServiceNow or only an up-to-date copy of that data lives there, it allows all the developers across your organization to have access to all that data with a single service account. A fourth thing is also that it makes people love and trust the platform more when it's easier and lower friction to get access to the proper data. So don't underestimate that part either. I've seen, at some organizations, where the data center operations people get tired of the table API and they just want to-- they go the other way with their APIs. They replicate CIs, incidents, and changes out of ServiceNow into their custom database, do all the work in that custom database, and only feed back the very minimum updates and states into ServiceNow. So that way, the ServiceNow record only has the bare minimum of updates and activities. Also technical-- flow triggers are great, but everything's asynchronous and you have very limited control over your inputs and your outputs, especially for a flow trigger that uses an HTTP GET. So here, we get better control, better logging, better error handling. For example, if we wanted to open up an incident or an event or alert when APIs are sending in bad data, in a scripted REST API, we'll have control over that. So overall, before implementing this, I do want to recommend getting buy-in from the top technical people responsible for your instance to make sure that what we're doing here conforms with the roadmap for all internal and external data flows. So anyway, back to the tech demo where we'll be doing some validation directly in a scripted REST API. So here we have a HTTP POST example. So above, what we're doing is just grabbing the body that's sent into the POST body. We're loading up the schema definition that's stored somewhere in the instance-- it could even be directly in the POST record here-- doing the comparison, and then checking to make sure if it's valid. So again, here, we'll take that next step that we didn't do in those earlier examples and build out our error message. So I'll zoom in on that part. And notice, again, we're using the data path and message fields. One of the favorite features that I have inside of a scripted REST API is the ability to control specific error messages. So there's a one-line command that gives you the option to just send a error, but here we see the four-line option to set specific HTTP status message and the details of which is the concatenated string of that error array above. So if there is no error, we can begin processing away. If there is an error, we stop the transaction and reply very specific things back to the developer. So let's take a look and see what the remote developer sees during this error process. So when I'm coding integrations, I always make sure everything looks good using Postman and, like we saw, the Xplore toolkit to make sure all the data and authentication works. Now, in this example, I've set three errors intentionally in what I'm submitting. And what comes back from our scripted REST API is exactly this, the HTTP 400 error. What we see is, under the message, we see that static string that we programmed in. And then, under detail, we see the specifics. Where was the problem? Under first name. What happened? The string is too short. It was only one character long. It needs to be at least two. And then it loops through all the other errors as well. So this was a basic scripted REST API example, but like I started with, this could be used outside of these APIs and could be used anywhere on the platform where you want to validate parameters. For example, critical script include function, we can validate that everything's within range before we do any expensive processing or any record inserting or updating. Thank you for watching. If you want to continue the conversation, please reach out to us out on the ServiceNow Community Forums, out on the SNDevs Slack community at sndevs.com, or find me on Twitter. I'll see you around the community. [MUSIC PLAYING]

How ANY developer can powerfully leverage Script Debugger

CC20-CCB2007

360 Exchange: Securing your instance in the new age of AI

SES2897

Join us for an illuminating session where we’ll explore the intersection of AI and security within the ServiceNow ecosystem. By connecting with ServiceNow experts and your peers, you will gain practical insights to help you fortify your instance against emerging risks. Tailored for practitioners and implementors, 360 Exchange provides a distinctive opportunity to connect directly with ServiceNow experts and peers. Prepare to delve into open dialogues, share invaluable insights, and collectively explore solutions to common challenges.

A practical guide to cloning

CCL1310

Data preservers, exclude tables, clone profiles, dev work oh my! Cloning instances is an important step for effective development and testing. This is an in-depth, hands-on session on how to clone. We will walk through step-by-step cloning tips and tricks, setting up and using Preservers and Excluders, and backing up and restoring in-flight development work. We will explore the new Clone Admin Console and how the newly released features make cloning easier.

ServiceNow Veterans Connected in Tech Meetup

CCA3048

<p>Get ready for a 30-minute power session with our new Veterans' Connected in Tech Group, led by other veterans at ServiceNow. You'll have an opportunity to network, learn, and share your experiences with other veterans in the ServiceNow Community.</p>

KMF: Your new best friend for integration encryption

CCE1453

The Key Management Framework (KMF) is your new best friend when it comes to encrypted data integrating across multiple systems. These moving parts can get complicated quickly, don’t let custom scripting add to the confusion! We’ll review the new APIs, as well as the deprecated ones to stay away from. By doing it the right way the first time, you’ll reduce the chance of an accidental mis-configuration and the result will be a system with increased supportability. Your security team (and your future self) will thank you!

Data Privacy 101

LAB3148

What you don’t know can in fact hurt you! Learn how to protect hidden data from misuse in this hands-on lab. You'll explore how Data Discovery and Data Anonymization leverage AI to classify and secure sensitive information in real-time. Gain practical experience implementing security controls on the Now Platform to ensure data stays protected in lower environments and in AI models. Walk away with the knowledge to safeguard your enterprise while harnessing the power of AI!

Unlocking the secrets: Reverse engineering in ServiceNow

CCL1449

In this lab, we will dive into the art and science of reverse engineering code on the Now Platform. Whether you’re troubleshooting complex scripts, understanding third-party integrations, or enhancing existing functionality, mastering reverse engineering techniques is essential. You’ll learn how to leverage built-in and third-party tools, inspect code, and apply effective debugging. By the end of this session, you'll be equipped with practical skills to reverse engineer code, enhancing your problem-solving capabilities and driving better outcomes in your ServiceNow projects.

Protect your sensitive data with access controls

CCL1447

We’ll implement defense-in-depth to prevent unauthorized access to your sensitive records and fields. You’ll receive practical strategies that you can use immediately, as all the tools in this lab require no additional entitlements or licensing. This session will implement various access control features, Allow and Deny ACLs, Query ACLs, Related Record Access and Security Data Filters for protection, as well as routine review and auditing strategies using Access Analyzer.