Join us for this webinar to find out how fast and easy it is to configure a ServiceNow-to-ServiceNow® data integration with Instance Data Replication (IDR). Every day more customers like you are increasing the number of instances they use. Additionally, they are exchanging data with other ServiceNow customers. Why not supercharge your efficiency while making things easier, too?

Originally aired: December 15, 2020 8:00 AM PT

Featured Experts

Kreg Steppe is a Staff Programmer Lead within ServiceNow Training and Certification developing and supporting cloud training infrastructure. He specializes in developing integration solutions, automating repeatable processes and Cloud Management in ITOM. Kreg's prior experience includes operating his own ISP, developing web applications in PHP, network integration, managing network support, Application Development on cloud based networks, DNS and email server maintenance. He is a Linux enthusiast and enjoys Photography.

Chuck Tomasi is a Sr. Developer Advocate for ServiceNow. He is a computer science major with over 35 years of IT experience. As a former ServiceNow customer, Chuck won the first Innovation of the Year Award at Knowledge 10. Since joining ServiceNow in 2010 as a Technical Consultant, he has done many large scale ITSM implementations and custom applications, acted as an adjunct instructor for Education Services, created and lead the Technical Best Practices program, makes appearances on Live Coding Happy Hour, created dozens of fit for purpose custom applications, and co-hosts the ServiceNow series "TechNow".

Jeremy Duncan is a Platform Architect on the Workflow Design Studio team at ServiceNow. Jeremy is an Enterprise Architect with over 16 years of IT and Service Management experience. His experience and certifications span across retail and manufacturing with focuses on hardware, software, and information security specialties. He has done many implementations in both the Enterprise Federal and Commercial spaces since 2011 and brings a breadth of platform knowledge that aids him in sound architectural decisions. He achieved his B.B.A. at MTSU and majored in Information Systems. He spends his free time fighting crime as a volunteer certified police officer in North Nashville since 2012 and enjoys family time, camping, with his wife and kids!

Questions and Answers

can an MSP use it for its customers?

Yes, but I'd be curious to know the use case. Normally the MSP has the customer data already.

Is this IDR not for an on-premise customer? If so, why?

It's not available for on-premise. I don't have the details, but it has something to do with the Kafka server and key management.

What happens to two task records (for example, incident) with same number in bidirectional sync?

It uses the sys_id as the unique identifier.

Thank you, I can see that, however "number" INC123435 could exist in both instances with different SYS_IDS but also have unique actual incident needs.

That's true. Like any other integration, IDR doesn't free you from the philosophical things you need to consider. :)

What roles do we require for approval and can we gave this authority to anyone

You need either idr_admin or admin roles to administer IDR. Technically, yes, you can assign these roles to anyone. Given that this involves two systems, and potentially massive amounts of data, consider this role (like many others) very carefully.

Can IDR be used to replicate Group, Roles, Users records?

What about PA Dashboards???

I wouldn't as dashboards are part of the sys_metadata construct (captured by update sets, app repo, etc.)

Are you going to show how to make a catalog item?

Sorry, that's beyond the scope of this webinar topic on IDR.

Sorry if I missed this. Can you use IDR between 2 Production instances (separate companies) but own by one.

Yes you can. Any two ServiceNow instances can use IDR.

how does it keep track of order? sys_updated_on?

It does it in the background. It sends the latest modifications last. I don't know the specific details.

Use case for domainsepraion:I have a Legal MSP (domain separated) who uses their central instance to manage incidents for multiple clients in a first line capacity, but these are sync'd to multiple other SN instances for 2nd and 3rd line support within the clients instances.

Yes, it could potentially be used for that Karl.

Is this enabled by a plug-in? The only "replication" I see in Filter Nav is "Replication Properties" and it talks about master/slave for data replication. Same ? Different?

You need to reach out to customer support on this one. Even as 'maint', I couldn't activate it as it requires some set up with keys and the replication server (kind of like a store.)

Are there any limits for the amount of exchanging records as they are in IntegrationHub? Or is this 'inclusive' because it's a paid plugin?

There's a practical limit of 3 million records. Yes, it uses your IH transactions to the best of our knowledge. Please check with your account team with any pricing questions.

Will we be able to know how much time will it take to sent the data. As in prod there we will be dealing with bulk data.

It's 'near real-time', so if you import a lot of data to prod, it will take some time to replicate to the consumer(s). If it's just a record here or there changing from time to time, you won't notice it.

what happens with the associated tasks

They're not brought over unless you add that table to your replication set.

If I'm the producer (Company A) and my customer(Company B) is the consumer. Do both companies need to purchase this product?

I believe so, but please check with your account team.

are there any limitations with the attachments or attachment size?

I haven't heard, but that doesn't mean much. Please reach out to customer support.

I am struggling to understand the use case for IDR if it isn't meant to replace cloning (especially to get configuration/code changes to consumer) and if a more formal e-bonding solution is coming in a future release. Can you expand a bit on the slide #1 speaking to specific use cases IDR was meant to handle?

The primary use case is to share data between ServiceNow instances quickly and easily. eBonding traditionally involves non-SN systems and potentially other data transormations.

Are you going to show how to work on the change management application?

Sorry, that's beyond the scope of this webinar topic on IDR.

Does IDR replace a back-up? What are ultimate benefits of IDR?

No, it does not replace backups. Benefits are listed on that Top Takeaways slide.

I am sorry, can you repeat about on-premise customer for IDR? I missed it.

IDR is currently not supported.

What is Discrete for?

It allows you to create a main producer set with a "sub filter" of sorts. E.g. send incidents, but only where company=Amazon.

Are the incident records on the Consumer view-only, or they can be modified/acted upon from the Consumer instance?

No, they are live on both instances (ACLs dependent of course.) This is not a read-only replica feature.

What if the incident table are configured differently on both instances? Like Producer has 4 incident states and Consumer has 5 incident states?

It will send over the value, but obviously, if the sys_choice entry is not available, it won't be able to display (and show up in the list blue). In the case of a reference field, you'll just get a blank entry (because we don't like to display sys_ids)

can you synchronise fields with different names on producer and the consumer?

Yes. It's in the transform section.

What will happen if the consumer already has existing incident or problem numbers that the producer is sending? (But different sys_ids)

You'll end up with duplicates

Am I right that the vendor is paying for IDR Plugin in a Customer/Vendor model? Like they bought support from the vendor and both companies are using ServiceNow.

I don't recall. Please check any licensing questions with your account team.

What happens to fields like - Opened, Resolved values?. They are timestamps. Sure that they are not getting imported? Or do they?

The values (not display values) of the date and date/time fields comes over and is stored on the consumer. If your instances are in two different time zones, it will be addressed/displayed as expected.

Can you use scripts to transform data?

Is SN considering ever allowing data replication to other systems, say a SQL database?

I haven't heard of such plans. You may want to suggest it to the Idea Portal.

Are there any tables that are restricted from use in the producer?

There are. I don't have that at this time. It's documented in the IDR docs page.

I'd think one would need to setup any related/referenced tables as well, e.g. sys_user table for caller etc

Yes. Related records also need to be replcated.

Does IDR work with replicating to Developer instances?

Yes. I assume you mean sub-prod, and not PDI.

what about business rules and client scripts that may be associated with incidents? We have to replicate those separately?

Please do not put things like BRs in IDR. That would be moving data in the wrong direction. Hope I explained that fairly well about 40min in.

isn't export xml and import is faster for this?

I haven't done a timed trial. Remember, import/export XML is once, this keeps data updated in near-real time.

IDR is not a back-up use. Is it correct? What are the benefits of IDR?

No, it is not a replacement for backups. The benefits of IDR were mentioned several times in the webinar, notably at the beginning and end.

how do you manage duplicate numbers?

You can do that with the adapter field on the numbers.

Is ebonding for TSM Telecommunication Service Management build on top on IDR.

Not that I'm aware. We are planning on a configurable eBonding solution in Q. #safeharbor

Does the replication use standard table APIs or another method?

It uses another method to keep it fast and secure.

Is there a cost associated with IDR?

Yes. This is a paid for product with its own SKU

I would imagine the answer yes... If I have more than 3M rows, I can do this in stages to commit all 4M rows?

If you have that much data, I'd reach out to your account team to talk to the product team if you have requirements that go beyond the limits we suggest.

Would/ could this be an eBonding replacement?

Theoretically yes, but we have an optimal solution coming specifically for configurable eBonding.

It's pretty much like Sys ID is mandatory for import Data.

Paid option or included with the monster subscription charge ?

Separate product.

We saw the Producer request and Consumer approve. That's like a push. Does it work the other way round? for pull as well?

Each consumer needs to make a request to the producer. The push/pull concept doesn't really apply here (at least in my mind.)

is it easy to map fields (eg if systems have different categories)

Yes, You can do that with the Adapter.

Is it possible to configure problem data in producer table to be replicated to problem_task data in consumer table using IDR?

Why is the "Producer URL" not Mandatory ?

Good point. It should be.

How do you know if IDR is enabled in your instance?

You will have an application menu called "Instance Data Replication"

back to business rules, do they get triggered on the consumer instance?

The BRs on the consumer will run as they normally would. The incident, for example, doesn't change behavior when it goes from the producer to the consumer. It's just incident data.

Can we make each instance (producer and consumer) to use their own record numbering to keep the local numbering sequence?

By default, they use their own numbering sequence. Remember, numbering is assigned when the record is created. Since IDR is functionally the same as exporting data from one system to another, you'll get the numbers from the other system. You may want to look at using an adapter to change the value in the number field if you need, say, PRBs in one instance and DEFs in another.

When we enable Transform, does it provide options for Mapping Assist?

Not the same UI we use for import sets, but there is a field matching UI.

Does it work perfectly with On-premise implementations

On prem is currently not supported.

How does this deal with collisions of ID's between the instances?

Sys_ids are the unique identifier.

how the systems are talking without authorization?

I hope we cleared that up in the discussion (around 28 min past)

How do you I enable attachments?

Replicate data in the sys_attachment and sys_attachment_doc table.

In case of bi-directional IDR, does the consumer need to approve as well (as it is also a producer)?

For bi-directional, the consumer is also a producer. Hopefully your question was answered in the demo.

In case of Requested Items if I replicate - "sc_req_item" table will it automatically send the variables as well which are in items ? Or i need to send all the related sc_item_mtom tables as well ?

I suspect they will not come over. I haven't tested this case specifically.

If we just want to once in a while update sub-prod with some tables on a schedule, is this a solution for that?

That sounds like a scheduled export. Take a look at https://docs.servicenow.com/bundle/geneva-servicenow-platform/page/administer/export\_sets/concept/c\_ScheduledExportSets.html

does IDR manage encrypted fields ?

just joined, does IDR also helps in real time syncing of data between instances like similar to ebonding?

Essentially, yes. It's what we call "near real-time"

IDR is not supported for on-premise customers. Why?

It has to do with the Kafka server, key management, and some other things that I'm not totally privvy to. If you like, we can reach out to a product manager and broker an introduction if you would like to understand further.

Is it OOTB that edge encrypted fields will not be allowed to be selected?

How is the referenced field handled? do we need set IDR for each table? is the IDR order matters?

Yes, you need to send any related records that may be involved or they will display as blank entries. Order generally doesn't matter.

How does choices gets managed, if consumer does not has the choices which are send by Producer ?

The values will be transferred in the record, similar to any other values (dates, integers, etc) but will show up as blue in a list to indicate there's no corresponding choice in sys_choice.

Why isn't IDR part of Integration HUB already?

Someone above my pay grade makes decisions like that. :)

How to map choice lists items, such as categories and subcategories in case if they don't match between producer and consumer tables?

You would do that the same way you would without IDR. An update set, an app repo, etc. to move config changes like choice lists between instances.

Is IDR supported for different Release Versions like Customer is still on Orlando, Vendor is already on Paris?

You can replicate between different versions, but there may be some limitations. Talk to customer support for more details.

Does IDR have back-out option?

In UNI directional IDR, if you enable transform at the consumer, is there an option/transform script to add a prefix or suffix in the short description or any field so that we can differentiate the incidents

Yes, you should be able to do this with an adapter

Are sys_ tables eligible with IDR, e.g. sys_auto_script?

Some are, some not. Check the docs for full details.

That pretty much simplified the way SNOW instances will talk in the future.

What if the consumer number has a a record with the same incident number?

You will end up with duplicate numbers.

Is this practical limit of 3mio records/transactions per year or one time? Is it only for 'real' records like Incidents or also for each update on the records? Also, for each connection or for all connections (e.g. the Instance is connected to multiple Instances)?

Three million records is total (for all time.) For the second part, it would be for each connection. If you are considering IDR for millions of records, reach out to customer support.

Is the configuration of IDR being kept in update set? I want to config that on dev first and validate on test before I enable it on production.

It doesn't appear to be and I can understand why given the security considerations behind the scenes. Given that it only takes a few minutes to set up each consumer, I don't consider this a major liability.

can we update replication set to bi-directional ?

IDR implementation is first step for TSM?

I don't understand the question, but since it's not the first step to anything, I'll say No.

Is it possible to test IDR using two temporary PDIs or are there any restrictions?

IDR is not currently available on PDIs. Wish it was. If you feel strongly about this and have a use case, contact the developer program people and ask. Things can change. We may also have instances available for a lab/workshop at Knowledge 21. Stay tuned.