- Hello, and welcome to "Three Levels of Advanced Integration Design on Custom Inbound REST." My name is Jake Gillespie. I'm the Manager of Technical Excellence at Enable, a Fujitsu company. I'm a architect by role, certified as Master Architect and Technical Architect. I've been working with ServiceNow for about a bit over a decade, and I'm very passionate about design. And my favorite thing to say is, "You should measure twice and cut once." Sid. - Thanks, Jake. Hi, everyone, I'm Siddharth Tiwari. I'm Practice Director at Fujitsu. I look after telecom and healthcare portfolio. I am also leading the gen AI solution space in Fujitsu as well. I've got 15 years of consulting experience in ServiceNow. I'm also a service and certified Master Architect, and I love to solve complex business problems in my day-to-day life. Thank you. - Thanks, Sid. So, definitely, over the years we've, we've worked on many integration solutions and we thought this would be a really fun topic and a really valuable topic to bring to knowledge this year. So in saying that, let's just take a quick look at our agenda for today, so you can get a better understanding about the lab that we're about to undertake. I'll start us off with a bit of a background as to why we put this lab together and why we chose the topics that we did. Essentially, we're talking about some of the inbound REST API capabilities that ServiceNow has. Then Sid will take over and give us an overview on what those APIs are, how they work, dig in a bit deeper. And we'll sort of give you an intro into each lab exercise, so that you know what we're expecting from that. We'll pause, give you a little bit of time to do that, and then at the end we will summarize what we think are some of the best practices and best usage for these APIs. And that'll wrap us up for today. So let's start with the background. So, as you have probably felt, working with ServiceNow, if you've been doing it for a couple of years, there's a lot of flexibility in the platform, a lot of capability that we can unlock, and none so much with integrations. So I guess the scope of our session today is inbound, and by inbound we're talking about third-party platforms initiating a connection into ServiceNow and leveraging one of the APIs that we'll talk to. So a very popular API is the Table API. Sid will definitely talk about it more in the technical sense, but I guess from my perspective one of the things that comes up is what's it best used for. If you think about integrations, that are inbound, that is, we might be pushing data to ServiceNow, pulling data back, or perhaps doing something a little different, like triggering an event or a workflow, or a flow, or some kind of orchestration. So when you use, say, the Table API to push data to ServiceNow, for example, you're actually interacting directly with the tables. And that means, well, that should raise some questions around the security element, what can this connection do, you know, this service account that's connecting in this platform? Are they gonna create data? Are they gonna update existing records? Should they delete records, or what records can they read? You know, these are all questions that should be asked in those early phases, and when you're certainly assessing whether something like the Table API is the right solution, you know, for your requirement. Another point is that a lot of third party applications, that is, things like spokes, you may have heard of integration spokes on the store, et cetera, these will actually leverage things like the Table API or the Import Set API, and with that, it does raise also that question around governance. You know, what should they be able to do when they leverage those APIs? And then, lastly, the final reason we put this lab together is that concept of build versus buy. If ServiceNow has these APIs already, should you leverage them, should you, you know, go and build that, or is there something that's out there already? To try to narrow the focus a little bit and give us a bit of context for the exercises today, I thought it would be good to bring out a couple of problem statements that lean into these APIs. So the first one is: As a vendor, I need to retrieve the details for incidents that are assigned to my team. Now, this might be a common scenario for some of you, particularly if you've worked with IT service management, but often, an incident can't be resolved without the support of a vendor. And if you don't have an integration in place, something that's commonly called eBonding, then what may need to happen is someone from the service desk will need to log into the vendor's portal and swivel chair. That is, they're gonna copy and paste details in, you know, effectively logging a new incident, or a ticket of some kind with that vendor, and from then on, it's manual updates each way. If we build an integration to bond those two records together, that solves a lot of time. It's a cleaner approach, and, certainly, it scales quite well. Now, if we were to do that, Table API does seem like a great option, but should it be used for writing data or just reading data? So this is where this lab exercise will, I think, give you some value, or bring some value to your decision-making. So for example, it's our view, as architects with a lot of experience, that Table API is most suitable for reading data in ServiceNow. It's less suitable for writing data, or creating data, or even deleting. It's not to say you can't do it. Certainly, you can, and Sid will give you some background as to the security elements of how you might do it, but Table API is most suitable for reading. We have another API that will, we have a lab on called Import Set API, and if you've used import sets and transform apps before, this is really just a front door to that. It's just a way of getting data into that Import Set Table, but it's through a REST API. So that is a much better way of pushing data into ServiceNow because we've got some validations, we've got some mechanisms for controlling that data, et cetera. Our next use case here is: As a third party system, I need to publish events to ServiceNow to trigger automated processes. Now, notice that this one is not specifically talking to any record. We're not creating a record, we're not reading a record, we're just talking about triggering a process. So for something like this, the Table API and the Import Set API probably aren't the best tools to use, because both of those are utilizing the table records. So in this case you might want more of a custom API built, and that is something you could do through a scripted REST API or a Flow Trigger API. And as the name suggests, a Scripted API is gonna run a script, and a Flow API Trigger is gonna run a flow. But either way, you can build your processes out behind that, whatever that may be; you know, firing an event or triggering a new flow to run, and whatever logic you may build in that. Alright, I might pause myself for a little while and hand over to Sid to run us through the lab overview. - Thank you so much, Jake. Hi, everyone. Siddharth here, and welcome to the lab section of today's session. So today we'll see how we can really harness the power of REST API capabilities in ServiceNow and amplify the capabilities in a ServiceNow platform magnificently. Whether you are an end user, whether you are a developer or an administrator in our organization, it is very essential for you to understand the REST API capabilities that ServiceNow has to offer, so that you can really think about how you can automate all the more business processes, you can workflow more use cases, and much more. So what you have on the screen right now is a portfolio for REST interfaces that ServiceNow gives you to be leveraged, first one being Table API. So Table API, I think of Table API as a gateway to your backend production tables directly. You can conduct CRUD operations, you can create read, update, delete data directly onto the backend production tables while using Table API. You can use the POST method available to create records into the backend table. You can use the GET method to retrieve single or batch records from production table. You can use BATCH method to modify existing records. And lastly, you can use DELETE method to delete records in production table. We have touched upon this before as well. Oh, and Jake mentioned that we should not be using Table APIs for the purposes other than GET that much, and we as architects recommend that too. The most common use case you would come across for Table API is where you would have to retrieve records in single or in bulk fashion from the ServiceNow instance. Next, moving on to Import Set API. Import Set API is another specialized REST interface designed by ServiceNow which allows you to ingest data, import it into ServiceNow. It allows you, by doing that, by giving you a staging table. You can stage data into an interim table, which ServiceNow calls as Import Set Table. And in that table, the data which is imported can be reviewed, validated and then be transformed into your target tables. ServiceNow gives you something called as transform map, which is a very strong, very potent utility that we have in this space, using which you can draft out some mapping scripts, some mapping logic, and hence making more conscious decision as in what data should be landing in your target table. You would see all these activities in the labs today, anyways. And lastly, Custom REST API. Custom REST API is the answer to those edge case scenarios, those edge use cases which are not be sufficed by Table API, Import Set API. You can use Scripted API component in your custom REST space and you can draft out Scripted web services, Scripted REST services using in-house ServiceNow REST libraries. As an extension of that is a flow trigger, which is a configurable API which can be used. If an incoming event comes into a ServiceNow platform, you can get that event and you can kickstart, you can invoke any flow in ServiceNow platform to execute an end-to-end business process. Now let's move on and see how that works out in your reality. This is a Scoped Application, right, so whatever customizations we would be doing in today's lab, it's gotta be tracked in something called a Scoped App. Now, what exactly is a Scoped App? Think of Scoped App as a container in your ServiceNow platform. Using Scoped Application, You can encapsulate any aspect of your application. Be it a UI element, be it script, be it any component, you can encapsulate that and control it within a Scoped App. If I were to use a general analogy, then I would say think of a Scoped App in your Java world as your privately managed scope, right? So whatever your variables, whatever components you would define in that scope, it's gonna be controlled by only you. Obviously, it gives you much more capability in security perspective if only you would be able to control what happens with your application. Whatever happens, whatever activities happens outside your application in ServiceNow platform, your application would completely be abstracted from those activities. It's gonna be based on role level privileges. You would be the one who can assign what individuals can have what sort of access in your application. That's one advantage. Second advantage is it gives you versioning control. So your version one might be live. If you launch some more updates, you can control that, hence giving you all the more governance structure and base. And lastly, it can be developed using ServiceNow's very intuitive App Engine Studio. So irrespective of what is your level of experience in ServiceNow development, you can use App Engine Studio by ServiceNow to publish your application and launch incremental updates as the time go. Let's move on to our first exercise, and here's where you'll be creating a scoped app. Please refer to the resources in this section, in the in Resources section, and I will see you in a short while. Alright, so I hope you were able to create the Scoped App in the previous exercise. Welcome to the new exercise. Here's where we would be talking about how to leverage Table API's component of ServiceNow. Now Table APIs, as we talked about before, is your gateway into the backend production tables of ServiceNow, and what we will do today is we will interact with something called as REST API Explorer, which is an intuitive interface given by ServiceNow, using which you can tap onto the Table API REST interface and interact with the backend table. In this lab today, in this exercise today, we will be fetching records from your backend incident table. The way we would do that is by consuming a GET method that will be available on your REST API Explorer. When we would use that? I know that we have created a service account already, so that service account needs to have right privileges on your backend incident table as well. So if your GET option is not really working, that might be a good place to start to check what exactly rules have been assigned to the service account. I would suggest to assign an admin account for now, so that your end-to-end execution works. And lastly, we also have a section which talks about how we can use flow actions to further augment the capability Table API. So in your existing flows, you can put in a component for Table API so that if you want to conduct any CRUD operations in an existing flow, you can execute that by a flow action trigger. So I will give you some time to go out and test out for our second exercise, Alright, I hope you were able to see the power of Table APIs, and welcome to the Import Set API. Now, as we were talking about before, Import Set API is a specially designed interface allowing you to ingest data in a single or batch fashion, and stage that into an interim table on which you can do some data massaging and transform that into your backend tables. So, extend that message, you can use the Import Set APIs for a couple of key use cases. You would ideally use imports at API for scenarios where you are importing large chunks of data. You might be decommissioning an existing application onto ServiceNow, and there's where you might want to free up resources of a ServiceNow platform. And there also, you can use Import Set API's asynchronous functionality, so that the data gets consumed in, gets transformed in without really consuming all those important resources of the ServiceNow platform. The other use case can be where you want to maintain that end-to-end transaction synchronous, and here's where you would want to hold off on that transaction. The use case might be, suppose ServiceNow is a slave system and there's an external system which is the master data, which holds the master data of certain type. And if that system triggers a transaction, then you would want to ensure that the transaction actually got transformed into ServiceNow so that your teams functioning from ServiceNow can actually use that data. So you'd want, there is where your synchronous transaction's gonna really come in the picture. Import Set APIs gives you all the more efficiency, because it gives you the utilities like transform map, using which you can play with the data before it actually gets transformed into the backend tables. And it obviously enhances the ServiceNow security, because you are abstracting the backend tables via a staging table in between. So you're not giving a direct access to the backend table, hence the security gets improved, gets enhanced via Import Set APIs. Now let's test out in the lab today. So moving on to exercise three, I will give you some time to test out that, and I'll see you in a short while. Thank you for taking us through those first couple of labs with Table API and Import Set API. Now we'll do a lab on Custom API. So I guess you might be wondering, "Well, what is custom about it? Aren't we talking about capabilities that already exist in the platform?" And I guess our view on this is that the Table API and the Import Set API, these are APIs that already exist and you simply just need to plug values in to drive them. You know, the table name for example, the Import Set table name, and the data, of course, that you want to exchange. With a Custom API, the API itself doesn't actually exist yet. And by API we're talking about a path, you know, a URL to that resource. So if we were to create a Scripted REST API, for example, we would give it a name, that would form part of the path in the URL to access that. We can also do things like configure the path parameters. These are variables that are essentially in the path as well, or query parameters that sit off to the end of that as well. You may have noticed, in the Resources, we gave you a glossary of terms at the beginning, and we called out some of these things, so if you're a little unfamiliar with it, please take that away and have a good read, and hopefully that's some value you can take from today's session. Now, coming back to the concept of a Custom API, we've grouped these two together for a reason. So that is the scripted REST API and the Flow Trigger API. And the reason for that is that when you're looking at building your own integration solution, we think you should really look at the Table API and the Import Set API first, and if those are not suitable, then consider coming in and actually, you know, using this particular technique. Because there's a lot of components here, it is more work to set it up. If you're doing the scripted one, obviously you need to know something about scripting, which is not always ideal, it's not always desirable, and perhaps you might pivot to the flow trigger. Now, I'll just call out a couple of differences here before we jump into the exercise. So with a Scripted REST API, you actually, it's actually a synchronous transaction. So Sid mentioned before, with Import API you can actually determine which one you want. What we're talking about here when we say synchronous is it means when it triggers the script to run, that script will run all the way through before the response is provided back to the API consumer. Now, you might know from your testing in the previous labs, what we're looking for in that response is a status code. You know, was it successful? Was it a 200 for example, was it a 500, which is a an error of some kind, and was there actually a message in the body? So those things are generated during that script and can be provided back to the consumer. That is something that cannot happen, at least at the current point in time, with Flow API Triggers, which are asynchronous in their nature. So by asynchronous, it's essentially a fire-and-forget type of a pattern. So you consume the API. As long as you've authenticated correctly and the flow starts, you will get a successful response, but the flow itself can't set that status code, it can't build a response and provide that back. So that could be a key differentiator there between the two. Now, if you don't like scripting, like many people, you do want to use flow but you do need to do something in a synchronous fashion, then perhaps you can mix both worlds together. And by that I mean with your Scripted API you could actually start a subflow using the Flow API, which is available on the documentation. Alright, well, with that in mind, I know everyone's keen to get in this lab, so... Now, what we did with this one, we wanted to try to give you the best of both worlds. But we are limited with time today, so exercise four is officially the Scripted REST API. However, there is a challenge exercise following that, which is on the Flow Trigger API. We'll pause now, and you can have some time to do both of them, and we'll come back soon. Okay. Hopefully, you enjoyed the Scripted REST API lab and, hopefully, had enough time to get through that Flow API Trigger. And ideally, hopefully, you saw some of the benefits of each that we mentioned earlier. We're almost at the end of our lab session today, so let's just run through a summary and we'll do a wrap up. Now, Table API, as Sid mentioned earlier, this is a very powerful API. Essentially, it's allowing third parties to access any table in the system. Of course, with, what is it, the saying? "With great power comes great responsibility," so we really need to do our homework there and make sure that the security elements are set up correctly, you know, the right roles have being assigned, et cetera, to the accounts that are using that, because, not unlike an interactive user, they'll have access to those tables based on that. Ideally, best used for reading, although it does have the full CRUD set of operations available. The Import Set API, as Sid mentioned in a lot of detail there, it's really good for ingesting data, perhaps in batch. It can be a single record at a time, or multiple records at a time, multiple rows. It's quite powerful in that sense, and it's leveraging that out-of-the-box capability through Import Set Tables and transform maps. So perhaps if there was an existing import set and transform map that had a different source, you can actually leverage that as well by simply just creating an API integration into that same process. Other than that, obviously, you can raise your own tables and set that up. It's most suitable for creating and updating data. Lastly, we had our Custom REST APIs, split between the Scripted REST API or the Flow Trigger API, each with their own benefits as we went through, but certainly a lot of capability there that you can unlock. And these are, of course, most suitable when Table API and Import Set API are not applicable. You know, there's no table record, for example, or perhaps there is a limitation in the Import Set API. And one quick example here is if you have another platform that is pushing a payload to you, say, a JSON payload, and that JSON can't be structured in a way that it matches the Import Set API or the table. In other words, it's a custom JSON. In that case, you would need something like a Scripted or Flow API to be able to parse that out and work with it. Alright, with that in mind, we might wrap up for today. So thank you very much for joining us, and I hope you found the lab valuable.