- these work in conjunction with ACLs, however they are executed BEFORE the ACLs

- data filtration is a 'deny' principle whereas ACL a 'grant' principle

- data filtration reduces the need for scripting

- they run AFTER before query business rules

- will still see the 'removed due to security contraints' message unfortunately

- requires security_admin role just like ACLs but there is no admin override feature

- specific to scoped app it's defined in (might not have access to certain global/other app tables if defined in a scoped app)

- key design criteria: machine enforceable and human readable (so improves on ACLs)

- declarative option over scripted option reduces technical debt

- No ACL will grant you access that a data filtration has already taken out

- may need to install the 'data filtration' plugin as not installed by default yet on Tokyo version

- remember to elevate privilege to security admin first

https://www.youtube.com/watch?v=UsjbPMHVs7U

(ServiceNow )

(full video transcript:

good morning good afternoon good evening wherever you are and whenever you are welcome to creator toolbox this is the

show that helps you put tools into your toolbox to be a better service now i'll get this out servicenow

developer or administrator the wonders of live streaming there's no redo

my name is chuck tobasi senior developer advocate at servicenow i'm going to

let's do some intros going that way to earl hi everyone i am earl duque i am a

servicenow developer advocate and i've been with the team for about a year now and before that i was in a lot of higher

ed customers doing development work and i'll kick it off to scott hey everyone my name is scott kaufman

i'm a product manager in platform security so i'm actually the product manager of data filtration

i've been with servicenow for nine years going on nine years and had a customer for eight more years before

that so i'm very familiar awesome and i'm laura mcmahon it's nice

to meet y'all i am the newest member of the developer advocate team and i've worked at service now i just had my uh

five year anniversary so very excited to still be here and very excited to hear about data filtration today

and i didn't do much of an intro for me but uh i've been here since mid-2010 at

servicenow as a customer for a couple of years before that and a long long resume of erroneous information before that

but enough about us thank you for joining we are going to be talking about data

filtration one of the new features in tokyo not just talking about it but demonstrating it as well

i've built out a scenario but i want to i want to frame this up of what data filtration is

it's it's it's a way to provide a declarative way uh to protect high value data

because in many of the high security environments finance

uh federal that kind of thing the auditors don't allow scripting and they they

weren't really living up to their regulatory requirements and they were getting some compliance issues so this is our way to

help address that it it provides not only access to the data like at acl

but it also provides in access to what we call the subject criteria or the

user's environment whether that's the ip address or what groups you belong to

which obviously helps reduce scripting now data filtration complements acls

it runs before the acls so you have to pass a data filtration rule if it's

there before you get to all the other acl checks now whereas acls are an allow operation

and scott's going to correct me if i'm saying anything out of order here this is why he's here to make sure you

know that we're saying what we really heard uh in our appropriate training

acls are a grant base you you allow access to something

data filtration works in reverse it's a deny it's still it still has a condition

part that you need to correct and we'll show this to you in the demo it has to be true to pass and then

you get the filter almost like a funnel if you will

what i miss these are post query operations like acls one of the questions for acls but after still

possible before acls but after

before query business rules yes wow there's a whole lineup of things we got to keep track of now

uh we do have something on the roadmap that we'll share for later of where we're headed

with that uh one of the questions in tech now was does this get rid of that annoying message at the bottom of the

screen that says you know so many records are removed due to security constraints the answer is no we will still see that in the demo that we've

got here let me oop i skipped ahead on my notes

uh quick reminders and i'll get this in the demo as well but it does require the

security admin role just like acls you are configuring security rules and who can

see what so you do need that elevated security there is no admin override we'll also

see that because my chuck tomasi account on this particular pdi

it has admin access but you're going to see these data filtration rules apply to my account because i don't satisfy some

of the conditions that we've set up it's also at the table level or the record level

you can't get granular down to the fields so you're filtering out or

filtering those records before that um

and if you are creating these just like acls it does respect scopes so if you're

creating them put them in a scope and you'll notice on ours we will be doing this in a scoped app the table list is

filtered to just those scoped apps so if i'm configuring a a data filtration rule

for my app my scoped app i don't have access to things like sysuser or sysuser group

or task or cmdbci you won't see those it is specific to my application

any further notes earl uh a clarity a clarification security

admin is required to do create re uh update and deletion of records but with

admin you can still read records you can read correct same same basic outline as acls

turning to mr wizard scott any any words before we jump into our story here yeah

i i think i'll kind of um one of the things to understand is is kind of the order of operations when it comes to

accessing data um i think i'll touch on a few of the the high level points there's a whole lot of

nuances that go and go on in each step but once you get past the authentication point you know the step that we go

through to determine you are who you say you are um then you go and you say i want to see this data let's just say i want to see

all incidents um so there's a there's a series of things that happen there's there's some pre-query activities um what most people

are familiar with is you know before query business rules or query business rules however you want to call them um

and then there's a lot of database stuff happening behind the scenes one of the things and this kind of this kind of

goes into explaining a little bit more about the security constraints error and why we see these things in in security

that's happening after the query um if you show me if you say i want to see all incidents

and there's a million records in that table we fetch all million records

and um because of some efficiency requirements we actually uh we actually

paginate we do uh total record count calculations we do pagination

calculations because those are all very resource intensive activities and uh the database does those really well

if you did it after the database it would be a lot slower and performance would be

not anywhere near where it is now so you imagine we have these million records that we've paginated now we've done some

pagination calculations then we get handed off to other things um like access handlers and whatnot now

we have data filters coming into play so we're evaluating the data we're evaluating who you are and and making a

determination of whether you're allowed to see certain data subset of data or all that data in that million records

and then we hand it off to acls so within that order there's some artifacts

as a result of that some of the benefits is for example you mentioned um data filters runs before an

acl the the value there is um because acls are an allow model it's a bunch of ores

um if you have 10 acls that determine some you know criteria some scope of data access and then 11th one can come

along and give access to more than the previous 10 has already you know gives you access to

it's a it's a it's a downside of something that makes acl so great unfortunately

but it causes some headaches for people that are trying to really secure high value data um so data filters is kind of

designed to target you know high value data and know that there's no acl afterward that's going to come along and

give access to it because what's been handed to acls is that has already been removed so there's nothing to add

anymore um we're trying to prove upon this as well so that that order you know data filters happening after the query but

before acls we actually are gonna our focus is on getting data filters to happen before the query so we've built

um a query augmenter so the notion of taking a query that goes before it goes to the database we actually can

intercept it now and we can decorate it with conditions um and do that very performantly

we are running into some you know challenges that kind of you know slow that process but that's kind of the direction we're headed and ultimately

what happens then is that chunking the the pagination of those million records if data filters

and acl starts removing things after the query that that pagination becomes inaccurate

and that's where you get the security constraints error and the pages unfortunately that have you know numbers

that that vary if there's been records filtered out by moving data filters before the query

um we are now removing data um before that pagination process occurs

so if if you're only supposed to see you know a hundred thousand of the million records there's only going to be a

pagination on the 100 000 records not the million records that eventually goes to acls

so that kind of gives you some insight into kind of the whole you know life cycle of you know accessing data and the order of things and you know what

artifacts you know are a result of that and what we're trying to change do we know when that would be expected

the moving data filtration to before query uh we don't um

we we're we're yeah we're so with data filters we actually are touching on an area that um

we haven't we haven't needed to touch in i don't know how long i mean we're creating a new you know data control

like ecl's existed since the beginning of time and so we're we're uncovering some areas where we can make

improvements on that we didn't need to before um we're discovering some new things about the platform um that we can

change to make improvements everywhere as because we're understanding this these operations that are happening and

where the bottlenecks are um but we're we're looking at probably two releases

um before we can really crack that that problem i know a lot of people will be excited

when that comes out especially i mean you and me writing before query business rules are not fun you know yeah the scripting

gets really complicated and it adversely affects the rest of the platform like right exporting and they're 100 scripted

which is another problem i think the key word for data filtration is that it's declarative and

that's why uh regular regulatory auditors are um accept

declarative statements because it's so much easier to build when it's when it's easy to describe kind of yeah

unexpected and there's no one-offs from the script or something like that that's i'll be excited when

that comes out too yeah go ahead key design criteria was you

know machine enforceable and human readable uh that's just kind of the the basis of

of any sort of security that is is fully auditable and and today with acls that's

not that's not the case yeah and anytime you've got a declarative option versus a scripted

option take the declarative one it uh it really reduces that technical

debt and your future self and your people who come after you will thank you for it i don't know how

many times i've walked into someone else's application or integration or situation and going

this is a lot of script to unwind as opposed to going into some records and saying i can read this condition it's

very easy to figure out i'll point out in some areas when we kind of get into the demo um

you know why uh you know declarative is is great and we can and where we'll

start expanding and where we're focused on expanding to kind of you know accommodate more capabilities yeah but

you'll see why using data filters now is still advantageous because you'll start inheriting some new stuff as we bring it

along well good segue what do you say we do a quick screen share earl i'm going to

bring up my scenario i can't see you guys so you gotta tell

me when it's ready it's ready okay uh we've got three personas that we're gonna be using hubble tudor he's our

regional director i've created an application to control stores

there are 800 and what did we say 31 records in here so we've got some demo data that we can

work with in various regions various countries each has an id we've got some

rules that we want to apply for various sections of these ids like the lowest 1000 from 1000 to 1999 is special

and secure we're going to put some data filtration rules around that uh and notice that abel and dennis don't have

any roles this is a situation you may run into i've got groups but the groups

don't necessarily have a role attached to them don't ask me what that means for licensing i'm not going to even try to

comprehend it let alone answer it today if you want to know more about licensing my standard disclaimer applies go talk

to your account account manager and they can help you now i have no groups but i do have the

admin role kids don't do this at home always assign roles through groups this was only done for demo purposes

so let's take a look at this here's my list of store 813 i transpose two numbers

sorry 813 records some are active some aren't they all have an id they're in

various countries there's even some additional information inside

the record that has you know an ip address i don't know what even that means but and a code

maybe that's some sort of backdoor hacker security code demo data random demo data it's not significant to

anything but this will help us tell our story so in

our our first scenario uh let me just also show you that i have

dennis's record here dennis is logged in he sees the same 813

and abel also has the same 813. we can see that down in the bottom

okay nothing special going on the acls at this point are pretty much wide open it's

you can read if you have public access so that goes back to what scott was saying

that no acl will grant you access that a data filtration rule has already taken

out and we'll see that in a second now the first scenario is is this first

thousand you see they're red they're red they're red and then they stop being red

okay that means something to somebody and according to the requirement i got is

admins should only be able to see those records they're special in some way i don't know what that is but only people

with the admin role should see these and yes you could secure this with an acl

but we're here to show you data filtration rules and then i'll show you some of the things you can do easily

with data filtration that you can't do with an acl as easily or without script

so let's start by going to oh should also mention we did enable a plug-in called data filtration

i thought it would scott correct me if i'm wrong but i thought that was on by default for new instances or was it

upgrades um no it's in in tokyo we made the decision to not make it on by default

okay we wanted people to opt into it because of this this you know it's it's kind of an mvp product at this point oh

sure great way to put it too i love that the um so you do have to turn it on if you

choose this is a great place to do it on your personal developer instance go to developer.servicenow.com

and get yourself a free pdi and then you can use it as a sandbox before you start exploring this in your

subprod and broad instances in your organization so once you have it turned on you will see

the data filtration menu which has data filtration records that's where we're going to be defining our

rules we've also got three more here ipfilter criteria role filter criteria and group

filter criteria we'll explore that in a little bit but these allow you to create

uh more involved and reusable components to the data filtration records is that a

fair assessment scott yeah yeah and then finally you can use a combination of these

as a subject criteria and interestingly enough i discovered that subject criteria uses

decision tables underneath like yeah you go to edit a certain part like the decision builder pops up that was kind

of fun table exclusions is exactly what it sounds like it says i am not going to

let you see or modify or do anything with these tables you can add to these tables if

you like so back to data filtration records let's

create a new one what's the first thing i need to do earl okay remember you must have security

admin so you got to escalate yourself yes and we won't turn this into a drinking game

how many times does chuck forget

there we go now i have a new button if you don't see the new button you don't have security admin that's your clue

so rule number one pick your table and it will happen to everyone it will happen it has happens

everybody anyway for acls notice that i only have one table if you are going where are my other

tables if you say show notice that we have a table choice script that does what it's

supposed to and says this is what i'm going to show you in this table field so i'm going to pick my store table

and it is active just like most records in the system you can still keep it just

make it deactive try not to delete things cascading means if you set one of these

data filtration rules on a parent class such as task this will apply to all the records in

the extended tables incident change problem i would i use that one because it's

easier than going through several cmdb examples let's give it a nice friendly

description special low numbers for admins only

uh scott i also discovered that description is only 40 characters so it's not even a

short description it's a shorter description oh wow okay we might want to look at that

i think that's in a couple of places it might be on the subject criteria as well all right i had a lot of fun going

through everything we discovered lots of stuff this is you know patch zero this is the first

release of the first release of the mvp okay we've got data filter these are the

two important sections here data filter and subject condition

if it were me designing this form i probably would have reversed these because i tend to think of them in the

reverse order like if this then that but

you do what you want to do i think that the idea here was we wanted to

continue some familiarity with acls which is really focused on the data first ah okay

i can but behind that you might see some you might see some design you know user experience changes around this because

we are very concerned about fragmentation of features so there is some consolidation on the

horizon here as well nice so let's do our

condition here we want to only show where the id

is between uh this is something you definitely would have to script for in an acl

1000 and 1999 right easy enough i'm

getting a little concerned so i save off and save early that's what i want to show if you meet

these conditions before you jump to that go back to the data filter absolutely

two things here one is that the going back to the cascading as it relates to the conditions

so if you're on the task table for example and you want to cascade this down to all table extensions it's

important to understand that the the conditions available to you are only on the table that you selected so you're

not going to see incident specific conditions even though you're cascading this in down into the to potentially

into the incident table so the the ids are of the table you selected in the in the actual table

field good point um so if you do need to have an incident specific field as part of your condition you would need to

create a data filter at the incident table level i see it the drop down doesn't have the

extended fields option then i'm assuming right it has it has show related fields

even though oh no i guess it doesn't on this one yeah just on the on a regular on the on different interfaces it's the

show extended fields to be able to do that additionally but in this in data

filtration you would want to build it in a different record yeah so like cascading in order to do that it would have to collect all the

fields across all the tables and then you start causing potential issues where

an incident specific field is not in the task table and so how does it interpret that and enforce it properly

the whole point is trying to be declarative and if you have a billion filters on one data filtration

maybe you need to re-look at how you do data possibly yeah the other thing to point out is similar to acls that we have that preview button

next to data condition um and that's an easy way to see what records match the criteria

and then also um uh it will allow you to click on the

actual record match count and view the preview those records

i was wondering okay condition builder equals v2 attribute is what puts up the nice preview button instead of the

little circley circley thing gotcha yeah all right you you you

definition fields when you use these three [Laughter] thank you so many times we've created

condition fields without readable equals true and i'm so annoyed when i see a code yeah in the list

so now you can actually see the records themselves by clicking on it it's just like acls but that's awesome yeah i mean if we do that you'll see

right there there's our data condition it actually reads instead of you know this encoded query let's go back in

there and set our subject condition so step two is what do i need

in order to see that according to what i said before the user needs to have the admin role

so i use the chooser here again and i've got the ability to look at network criteria

could be something i've built or could be yeah no that has to be something i built in that network criteria list

or group or role or subject criteria subject criteria subject group and

subject role apply to what i am i am the subject in this

so let's make it easy and say subject role is admin

easy enough this is now live

what do you say we go look favorites ungrouped stores i am admin i see

everything below one thousand hooray there's two thousand six at the bottom of the list below two thousand

the follow two thousand thank you in the one time if it starts with a one i win now let's refresh dennis's list

and haha can't see those and he starts counting at 2006

and then from there on it's pretty much what you would expect sounds a lot honestly that's a lot

easier to configure than an acl and abel should be the same way

right minus 50 minus 48 there's the other two

yeah that wasn't too bad pretty straightforward what do you say we do another one

okay i have another requirement uh only show

these ams records like this one if you are in the ams

support group okay again you would have to resort to scripting with an acl for something like

this we're going to do it of course with data filtration so i have a favorite for my rules

let's create a new one again on the store table well dynamic filters would be able to do

it in the condition builder for nacl uh yes you could you're right you're

right so our data filter is going to be

if the support group is

store ams support that's what i'm going to show you and that would give me 19 records or take

away 19 records from my list and the subject condition

is going to be this time subject group

is ams support now i did hear a question on another

video that said can i do show me all of my groups and this is not a

dynamic filter it just has an is operator and this

does have a dynamic filter which you could do here so i'd be saying if i was in the ams

group show me all of the records for all of my groups which might be a little too

broad so you got to think closely about that recognizing that let's go back to my

original one just because it will stay on script this doesn't have any more conditions

for the groups than just ears there isn't is one of or is dynamic

yet should i say scott yeah possibly let me touch on that for sure

um it's something to kind of peel apart a little bit and understand um

so you have to imagine that uh we are building something that

is uh potentially being called in every transaction yeah

as you add more data filters we're talking about high transaction um part of the the system

the when we start adding the the is dynamic right there's it's basically you know mini javascript call

there is an extra operation from the subject side so if you think about it from a query standpoint so our goal is trying to get

this to be done before the query [Music] as far as the dynamic

variables for data that can be that's basically translated into a modified sql

query in order to determine whether or not

you satisfy the criteria you know the subject conditions if we started adding dynamic

capabilities we would actually be running scripts and potentially database calls

before we did our main database calls so we're adding we're adding extra steps in inexpensive steps

so we have to be careful in how we evaluate you you know you as a person to determine your level of access

um if we touch on kind of what the the area that we're going to be going more on and i can i can go into this a little bit

later when it makes more sense but um the the uh subject conditions are going to

grow we we just maintain security role group and ip address and actually if any

of you are familiar with adaptive authentication which is also something fairly new to us we're sharing

the same code so the same rules that you define to determine authentication adaptive

authentication rules we you can carry that over into your data conditions as

well oh nice yeah so we so we we plan on um getting more into the subject and

environment attributes and and allow for kind of customization and configurability of that um and i can

talk about that a little bit in a little bit if if we want to but that's where we want to grow beyond just role group and

network adding network is actually a great thing because that doesn't exist today in the acl so right um that's key

that's that's that's this you know we're sharing code we're eventually going to bring all this all the way into the acl as well so a

lot of the stuff the declarative stuff we're going to be adding to acls um and so you know that that's that's

kind of where we're where we're focused on growing all right and i am am i safe to assume

you can't do you know javascript colon something in here um

well there's a matter of if you can or you should which i'm not sure which is

good valid good answer just because you can do a thing doesn't mean you should do a thing right

i wouldn't be married if that happened okay let's see how it worked so

theoretically chuck is not part of ams support group according to that slide earlier

but if i look at stores ungrouped and

actually you know what i have a shortcut just for that so grouped by support group i don't have

any ams let's see what dennis gets by support group dennis is also not in

he's in the emit mes support group and if we go over and look at

our old friend abel tutor not history hey where's my favorites anyway

group by support group he does have ams because he's in that group so proof that we can easily create a data filtration

record for that specific group i have one more but i want to show you something first

if you're ever questioning why don't i see this you can do

debug all security up our favorite debugger window i'm

going to allow and when i refresh this list

see if i can get that back there we go i can

search on filter right and hit

this and determine exactly which ones applied and which ones did not so data

filtration applied subject criteria not met support group blah blah blah

data filtration not applied subject criteria met the id is between so i can

tell exactly why did this record show up or not is based on these records so use the uh

security debugging to your advantage thank you for making sure that's included with its own messages by the

way you're welcome we wanted to uh yeah do better than

we've done in the past [Laughter]

you're driving a good point which is which is also going to be more meaningful in the future

right so as we you know you see right now that oh there's i'm missing some data because i don't satisfy some

criteria um and so that prompts you to you know go into the debug

in the future when we are able to move this at a quick you know before the query as part of the query um those

those uh those notices will go away and um everyone will be

seeing the data that they presumed are allowed to see so even more so um the debugger is going to come into play as

kind of like your go-to rather than just seeing the error itself or the message itself saying okay what's being hidden

and why and then go to the debugger oh good point

good point okay i'm going to do a different approach to that last example where i

said the ams support group gets to see the ams records we're going to satisfy dennis so you can see

only dennis can see the email support records but i'm going to do it in a slightly different way using something called support criteria

so underneath oh what was i typing yesterday

paf i like i like working on the middle of the word for uniqueness

we're going to create a support criteria that we can then use in our data filtration record this is a small

example of where you can go with this so we are going to create a new

called i like to just call these i'm going to prefix this because ams or emea support is going to come in all over the

place and this will help me tell them apart i mean by no means condoning a best practice here i only got confused

yesterday so maybe this turns into a best practice

description hey this description field is bigger i like this

so save that and down here i've got two more tabs and this is where i may lean on scott a

little bit to help us understand what the criteria inputs are

so um as as you mentioned before this is go this is using uh

decision tables uh and so the the prerequisite for decision tables is to define what inputs you want as part of

that okay so it's essentially just yeah adding what criteria

what of the three that you know we're able to evaluate what you want to add to this um

and it's just kind of a nuance with using decision tables all right and this is going to these

three correspond to these three correct so it's going to be giving me records in these other tables implicitly yeah the

the the other yeah those those other um links in the menu are probably um you

know just extraneous they're just there you probably don't need to interact so much with those but we kind of guide you through the process

oh hardcore people love them true that's probably why they're still there

now i could create any one of those ipfilter criteria to say hey i want this ip range or cider address or

whatever uh you can make these as simple or as complex as you want this is a group

filter criteria for email support again reusable components in

here and i'm going to assign it the group store support

that is that piece of it and notice that it's currently says used

false now this is the cool part these records know where they're referenced so you can't delete the wrong thing first

if i came and tried to delete the support criteria it would go i i'm sorry if you try to create the

group filter criteria it would go hang on this is being used somewhere you don't want to yank

the carpet out from under this or or saw the branch off that you're sitting on use whatever metaphor you like for

disaster happening here the system will actually tell you that you can't delete this record because it's used in a

a subject criteria which i think is really cool though something i did notice is you can actually still

set that field as active or false even though it should be it is automatically determined by the criteria conditions so

maybe in the future we can make that a read-only field oh yeah protect it whoop

yep you're right good call earl mvp product

yeah and we we want to do better in this area um make it a bit more intuitive because

either we we realize it's not but um that's just kind of how it has the components are necessary uh we want to

do better though and scott if i add more criteria inputs let's say i say you've got to be in this group on this ip range

etc you have this role are those ended or ored together uh that that's defined in the conditions

so all you're doing right now in this first tab is just bringing uh listing you're bringing all

the available inputs that you want to evaluate in the condition okay so it's

it's kind of like atf in a way where you're setting things up that you can use later or flow designer yeah so think of you know in our condition builder

there's the left side and then the right side right so you're just basically uh creating the the list of left options

right now let's go do that let's let's show these people how that works so label this is my subject criteria

creation for me support you see why i'm starting to do that

and in here oh look yeah i've got the different filter criteria uh earl and i were wondering

what sis id and keywords are doing in here that's a good question okay

generic to the condition field i don't know we were

conjecturing about those two for a little bit last night no attributes on this one so maybe

maybe if you use v2 they go away or something i don't know so what we're asking is

when this decision table evaluates our decision we want it to come back as true

now you could invert this which says you are not part of that group is that correct yeah

okay easy enough pretty straightforward well

yes uh i love that yes and

i'm i'm doing mental gymnastics right now to to verify that answer okay

we can we can easily flip that and see who can see what records in a minute but if you want to see the decision

table up comes decision builder our old friend thank you julia perlis and her team for

creating this it's not that hard to glance at it and go what's happening here uh if you're part

of this group it's true okay let's go back i didn't make any changes let's go back

again okay show the first tab again first tab is now true it says you have

been used so if i combine three of these and i like that if if you said

i've got a group one and of course you can always personalize the list to say you know

what's the display name what's the filter criteria blah blah blah

to make it a little more readable you go oh my filter criteria for email support is

used but i better go back to the criteria conditions because the one with the role or the one with the ip address wasn't

included in that condition so it gives you a little visual before hopping over here is this right

let me open that up and see if it actually says what i wanted to say

so in the in the midst of all this i think it's important not to lose context of what we're trying to achieve

um and and the value of it so if you think about um you think about

creating um a number of data filters that have some let's just call them

compound conditions so you have to be a member of a couple different groups or be from this role or from the corporate

network um and you want to repeat those that that those compound

conditions and and have consistency across multiple data filters this is

this is how you would go about that um almost almost like in it to building a persona that's using a combination role

group and network right oh my thunder man

actually your thunder have added yeah the reusability and i'll show you

real quickly now that we've got the subject criteria defined that persona that you know scott alluded to

i'm going to go to like the data filtration and create a new filtration record

to enact that or enroll that store let's see uh email for email

or it's like a protest statement or something beta filtration similar to before we're

going to say the support group is store emea that's what you get to see

if you satisfy this condition now this is where subject criteria comes in i'm

going to use that other subject now i have to start typing it so fortunately i started with

an sc for subject criteria and there we go

so in theory if this worked fingers crossed i go to chuck's list chuck is not in

ams or email so that's working dennis darren i don't know i keep calling

dennis because it's darren ennis or enos inus nice

something what's his name darren nice okay

the answer is yes to emea no to ams and if we go over to abel

and refresh the list you know you can refresh the list just by clicking all aha

things have been taken away no no nobody in the first thousand because that rule is applying

and group by support group says

no email should we flip it around just to test your theory

scott okay we're off script just letting everybody know that's like

we have a really he's like we have a really good batting average we don't want to change it up now right this is we're developing into the

criteria and we flip this to false so this would be a way to exclude data

from people not in that group let's try it again favorites grouped by support group

uh chuck doesn't see it interesting

refresh the list i'm not sure what happened to my refresher here

still don't see it for dennis and

still don't see it here now this is where

there's like there's a couple of things we could talk about now chuck based off of what we found last night we're going to just try clearing the

cache let's see what impact that has we could

also try logging out logging in but i'm not ready to go there yet okay cache is cleared chuck refreshes

does chuck see records we have to rebuild that cash which is why this is taking a couple extra seconds

this is when you are glad you have a cash no

i noticed another decoration i don't know if this was san diego or tokyo you see up here where you got these new

little icons here you can see what is grouped by so many times i've turned on grouping and i

don't remember which column was on totally this is nice i'm thank you for whoever

did that i think this is a great time to mention

some of the uh quirks that we ran into um in our

testing check so when we were figuring out uh and setting up this demo

some some things came up where we realized two things one uh impersonation the reason why

chuck is showing them in its own session windows right now instead of impersonating a user to demonstrate the records is

because what we've noticed is that the the essentially the criteria that is hitting

for the data filtration persist to the original person that's logged in even if you impersonate to

somebody else so impersonation is not a great thing to use while trying to test for these

records in data filtration so i see scott writing things down i think

so that's um uh i see him on the side um the other thing is uh right now chuck

is in a scoped application and we saw some irregularities while being

in a scoped application when it came to the subject criteria yeah subject criteria specifically

um so if we're running into interesting things here we don't know necessarily if it's a

scoped issue or if it's a bug for the entire the actual feature itself but um

when it comes to subject criteria we knew that it worked in global but not in a scoped application

yeah for scoping um there is we can definitely do better in

this area um we obviously as you can tell we're we support scopes but then you when you add

like scoped administration or scope security in into the picture it complicates it significantly

um so we've we've identified some areas where we can definitely improve and um

have more expected behavior around that um and also kind of you know do bet be

better than what scopes allow us to do

one more reminder is some of you may know if you use a glide record query you can navigate around acls

and if i'm correct you can you can do that today with glide record

on data filtration but you cannot do it i mean the glide record secure honors the acls and the data filtration rules

but the goal is no more no more working around with glide record they are what they are

especially if they're pre-query you got no choice yeah that's something good to kind of

dive in a little bit on um you know as we as we kind of

thought of data filters you know charter is to kind of protect high value data one of the one of the things that we

identified an area that we identified that we wanted to try and improve upon was

um i feel like a better term kind of centralizing security so taking taking the choice out of the hands of the

developer of whether or not security is enforced or not so we kind of took a position of

let's let's not make data filters bypassable

that led us that led us down a very long rabbit hole that ultimately was a dead end

not because we can't solve the problem but data filters it wasn't data filters

job or responsibility to solve what we ended up finding is a

very high risk of regression so if you think about it you know all of

our what we ship out you know from in our baseline code we have some expectations around what level

of access certain processes have and you can't get around you can't get around the need for a

process to be able to bypass security you know the system itself needs access to data in order to process the data so

you have to have some level of you know bypass ability of that security and today that mechanism is glide record

versus glad record secure and unfortunately both of those apis are publicly available as in script so

you can essentially create scripts that bypass security um this is nothing new um this is just

that's just the way it is so the notion of trying to change the um the point where data

filters was enforced i mean we wanted to go down to the db um the db query layer

um but what we found is that we were um we were creating an environment of a

very high risk of regression so like you should you know there's no data filters shipped out of the box but suddenly you start creating you know

data filters and applications start misbehaving because they weren't tested

um you know with with security in place maybe that that that that application needs acts full access to data but you

were creating a data filter that was removing access and and you'd start seeing anomalies so we we didn't make it made a

determination that um we need to solve that problem more at the plat you know general platform layer rather than

within data filters itself um so we've kind of resolved to re you know reevaluating our

approach to that problem one other quirk that

we came up with is you can see abel here doesn't have access to any of

the 1000 records but if i were to drill into one of those records

and say email the link to abel

away we go there we go had to remind

so that's something to consider as well i don't know if if that's on the road

map as well scott but yeah we actually have a defect uh we're working at hurricane as a defect at the moment

good so if you're watching this in the future and it doesn't work for you you know why

all right that's um you know unless we wanted to go down the global path for a quick demo i think looking at the clock

we better get it back on track but we're aware of these features this is patch zero uh i

got nothing else to demo here earl i think we can probably run through some

of the questions that were asked inside of the chat during the demo and just make sure we covered everything oh wait i do want to show one more thing hold on

okay work filtration uh if you try to delete that group criteria i said it's got a smart delete

and i go i'm going to be sneaky and try and delete it it says no you can't

it's already being used click here to review and it brings you right to

you know where you need to go okay yes let's see some questions little things like that save a lot of headaches thanks for

making sure that's there that was fun

give me a little warning first chuck dates i'm gonna fix that too

a few questions popped them in chat um some things we a lot of these we did answer on air um just by coincidence but

i just want to make sure that we covered everything um stefan had a question about phil data filtration and how

we're expecting admins to be able to uh work on records that are being reported

to us if there's no admin override button and stuff like that and one of the key things is debug security still

happens um so you'll there's key messages in there saying hey data filtration is getting rid of this record

for this person for for you that's why you can't see it and that's why there's a bug um so you need to go look at data

filtration or if it's not that go to acls then um but one of the other things is uh one

thing we did we mentioned uh the url bypass chuck

yes so that when when you went into the specific record right okay

um so there is a stuff that we uh noticed about that that um

uh it's mainly a table-less kind of thing known defect

yeah uh we had a lot of questions for clarity about acls working with data filtration

and just for clarity uh they they work to complement each other so remember they're both post query uh data

filtration works first and then passes those lists of records over to acls to run so they work together and they

always work together you can't have one or the other if you turn on the data filtration uh what else is in there

uh another clarification of about does this get rid of the security prevented showing the rows that's part of post

query so because data filtration is also post query it doesn't get rid of that what you're looking for is a before

query business rule right now but as uh scott has mentioned in the future we

hope to move data filtration or give the option to have that run before query which would solve that problem also

um nitish gave a good summary of what happens and

then i clarified it with uh hey um it's more clear if you say where the query was so that you know what happens before

querying what happens after query uh to be a good picture of the the workflow that's happening on records behind

what else has been asked there was the one about cascading too

yeah so earlier in the episode um we talked about cascading and scott gave great insight on uh

gotchas to make sure to pay attention for about the condition builder for a data filtration record and to make sure

to build the if you have a cascading field that you need to be addressing you need to you

would want to uh because of the declarative way of how this works is to build a new data filtration record on that table

so a lot of cool clarifications a lot of um questions in the chat which is great because we wanna that's how we get all

the information out there and how we get uh all the information to scott too this is great this is why we love having a product manager on our um on our shows

uh but yeah i think that was a lot of questions a lot of the things a lot of the chat was a lot of clarification on things um that we did answering the show

so i won't go over um a lot of these questions again uh but did you notice anything else in

the chat lauren um did you go through all your notes chuck yep

earl point up with your left finger this is part of the tokyo content

you can tell because we have this cool badge we are in the midst of the tokyo season you can find out everything that's out there or going to be out

there on our tokyo calendar i think we may need to update a few links because it's been a couple of

weeks now we've got content rolling out creator toolbox live coding happy hour breakpoint podcasts blog entries lots of

stuff coming in devlink dot sn slash tokyo earl's got the banner right there

[Laughter] we're on cue

well-oiled machine any other announcements earl

we are we have a uh do you want to talk ab challenge at all

build with aes challenge yes lauren heck yeah so i believe it was about

three or two or three weeks ago we kicked off the build with aes challenge uh all the information about the

challenge is listed on our blog and also on the community but essentially if you can write a short blog post or film a

short video on anything that you've built with aes why you liked it what you liked about the experience

we will send you a free shirt it is very similar to this shirt that was oh that went like lickety-split out of stock at

creatorcon so if you missed out on the shirt there uh this is an awesome opportunity to get a very good uh dupe

without the uh little creatorcon specific logo but the whole shirt is the same

i'll be the one to ask what does aes stand for app engine studio thank you so much i

appreciate that [Laughter]

well i know i just want to make sure we say just because um natasha asked about it uh

he's saying that if the if you have trouble with the dev links uh just go to

developer.servicenow.com and check out uh all of our resources um on the blog

and you'll be able to see the tokyo content uh breakpoint content this shows content

our friday live coding happy hour content all that's there on developer.servicenow.com

and then click on blog plus you'll see all all the other resources that you have available to you as a developer so

check it out all right i just tested it we did have a technical issue last week because rit

people changed something on the back end that we had to track down so it was broken for about 12 hours but hopefully

that's all cleared up i just tested devlin slash tokyo is working but you can also find that on the blog as well

thank you earl thank you lauren for joining and definitely thank you scott for the depth of knowledge on this

product absolutely thanks guys definitely another couple degrees higher than where we could have taken it uh had i had a

little more time i would have made nicer slides until next time everybody take care stay

safe and we will see you soon )

Haven't tested these all recently within global/local scopes, so feel free to have a play! option 1 use an encoded query embedded in the GlideRecord , e.g. var grProf = new GlideRecord ( 'x_cls_clear_skye_i_profile' ); grProf . addQuery ( 'status=1^ owner=NULL ' ); grProf . query (); even better use the glideRecord addNotNullQuery or addNullQuery option 2 JSUtil.nil / notNil (this might be the most powerful. See this link ) example: if ( current . operation () == 'insert' && JSUtil . notNil ( current . parent ) && ! current . work_effort . nil ()) option 3 there might be times when you need to get inside the GlideRecord and perform the check there, for example if the code goes down 2 optional routes depending on null / not null can use gs.nil : var grAppr = new GlideRecord ( 'sysapproval_approver' ); var grUser = new GlideRecord ( 'sys_user' ); if ( grUser . get ( 'sys_id' , current . approver )){

Classic UI : var sURL_editparam = gs . action . getGlideURI (). getMap (). get ( ' sysparm_aparameter ' ); if ( sURL_editparam == 'true' ) { gs . addInfoMessage ( 'parameter passed ); } Portal : var sURL_editparam = $sp . getParameter ( " sysparm_aparameter " ); if ( sURL_editparam == 'true' ) { gs . addInfoMessage ( 'parameter passed ); }

Call a script include to apply a reference qualifier on a catalog item variable: - variable reference qualifier dependent on another variable selection, in this case a variable referencing sys_user (requested_for) On the catalog item form. variable name to apply ref qual filter : retail_equipment variable reference qualifier (on cmdb table ): javascript : new refqual_functions (). lostStolen_getAssignedCIs (); client-callable script include ( refqual_functions) function : lostStolen_getAssignedCIs : function (){ //--called from variable set client script, for lost/stolen request (service catalog) gs . log ( current . variables . requested_for , 'retail_lostStolen_getAssignedCIs' ); return ( 'install_statusNOT IN8,7owned_by=' + current . variables . requested_for ); //owned_by=1269b79937f1060041c5616043990e41install_statusNOT IN8,7 },