Measure your Application Inventory with the Indicator and Scoring Framework

If you want to improve something, you likely need to measure it first. When it comes to management, disposition and rationalization of a Business Application inventory, the Indicator and Scoring framework is a powerful and flexible tool.

On June 18, 2024, we have presented a workshop on this topic. Please see the link below to watch the recording.

Indicators and scoring has been part of APM for years now and has grown and evolved with the product and concepts like CSDM. Here, we seek to help you configure and get best value from the framework to help govern your Application AND Capability inventory.

These are configurable business metrics that provide a key basis for evaluating both Applications and Capabilities. Indicators provide a normalized score for whatever condition they are evaluating.

There are five types of indicators:

Assessments – indicator that uses a score derived from a configurable Assessment/Survey. Assessments are configurable in content, target user and evaluation.
Query Condition – a query condition indicator is a configurable query against tables or other data sources such as performance analytics with ability to define differing
Custom Script – utilize a custom script to generate an indicator score
PA Analytics – Utilize Performance Analytics to gather historic data such as incidents, problems change to derive an indicator score.
Indicators – calculated sum or average of a set of child indicators

There are now (as of Washington release) 14 out of the box indicators in the Washington DC release (see details below).

The chart below also show 7 older cost related indicators that used the ServiceNow IT Financial Modeling engine. That functionality is no longer available but we include these for completeness with legacy usage. However, as of Washington release, we have a new Total Cost of Ownership (TCO) indicator.

The system also provides a mechanism, Scoring Profiles, for Business Applications and Capabilities to provide a single score value derived from the weighted individual indicator values.

The following table details the out of the box Business Application indicators in Tokyo and above.

Indicator name	Type	Source	How is it calculated?	Desc ription
Customer satisfaction (CSAT)	Assessments	Assessment Metric Type: Customer SatisfactionAssessment Metric Category: CSAT		Template NPS
Technical risk	Assessments	Assessment Metric Type: Technical RiskAssessment Metric Category: Performance		Technical risk captured through survey for the fiscal period.
Business value	Assessments	Assessment Metric Type: Business ValueAssessment Metric Category: Business Value		Business Value captured through survey for the fiscal period.
Functional fit	Assessments	Assessment Metric Type: Functional FitAssessment Metric Category: Functional Fit		Functional Fit captured through survey for the fiscal period.
Usage	Query Condition	APM product. cmdb_ci_business_app table	Calculated from the Active User Count field	Number of user sessions and users for the application for a given fiscal period.
Number of Incidents via Service	Performance Analytics	Mapped to Performance Analytics > Indicators > Automated Indicators > Number of new incidentsSource = Business Application via Service	Utilizes the Impacted Business Applications related list on Incident	Indicator that gets the count of all incidents associated to the business application tied to the scoring profile of which the indicator is part.
Number of Changes via Service	Performance Analytics	Mapped to Performance Analytics > Indicators > Automated Indicators > Number of new changesSource = Business Application via Service	Utilizes the Impacted Business Applications related list on Change	Indicator that gets the count of all changes associated to the business application tied to the scoring profile of which the indicator is part.
Number of Problems via Service	Performance Analytics	Mapped to Performance Analytics > Indicators > Automated Indicators > Number of new problemsSource = Business Application via Service	Utilizes Business application reference on Problem	Indicator that gets the count of all Problems associated to the business application tied to the scoring profile of which the indicator is part.
Application Operational Metrics	Indicators	Sum of Incidents via Service Problems via Service Changes via Service	Sum of operational values with a registration date (change_request.opened_at) on collection date	Number of incident, problem, and changes opened over the period. Daily and historic data collection.
Total change hours	Performance Analytics	Mapped to Performance Analytics > Indicators > Automated Indicators > Summed duration of closed changesSource = Changes.Closed (Change_Request table)Fields: Opened, ClosedState = Closed, Business Application = any of the APM Business Applications, Closed today	Script: Change.CloseTime.Hours. All Change Requests closed today considered	Summed duration of closed changes for an application for the given fiscal period. Time taken to close the changes in hours. Daily and historical data collection.
Portfolio TCO	Custom Script	Total Cost of Ownership [sn_apm_tco]	Sums the costs within the specified fiscal period across various cost types	Sum of costs per application. the costs are provided by externa; source such as Proven Optics, Nicus, Apptio or other financial modeling systems including simple Excel import.
Technology Lifecycle Risk	Custom Script	Based on Business Application risk as evaluated in the TPM Technology Risk table [sn_apm_tpm_technology_risk]	Risk due to out of date or soon to be out of date technology in TPM are rolled up to the Business Applications	Normalized score of the Technology Portfolio Management (TPM) risk evaluation.
Application's Incident Count*	Custom Script	incident	Data will be available in the incident table only after the business application is associated to the incident. Not often used since Business Application is not OOB exposed.	Indicator that gets the count of all incidents associated to the business application tied to the scoring profile of which the indicator is part.
Application's Instance – Incident Count*	Custom Script	incident	Gets incident count attached to all Application Instances, which are mapped to a business application and rolls it up to application.	Indicator that gets the count of all incidents associated with application instances. The application instances, in turn, are associated to a business application tied to a scoring profile of which the indicator is a part.The incident count is calculated first at the application instance or application service level, and then it is rolled up to the business application level.

* these Indicators are no longer best practice. They allow for only one Business Application and require some form of mechanism such as a business rule to set the Business Application on incident. They shoudl be disabled and removed from an scoring profile unless it is explicietly part of your process and you have made appropriate considerations in the Incident process.

Retired/Deprecated Indicators

These indicators rely upon an older and less mature data model where Business Application has to be directly specified on the Incident/Problem/Change. CSDM and latest maturity woudl advise use of Impacted Service/Business Application lists.

Indicator name	Type	Source	How is it calculated?	Description
Number of incidents*	Performance Analytics	Mapped to Performance Analytics > Indicators > Automated Indicators > Number of new incidentsSource = Incidents.New (Incident table)	Number of incidents opened today	Number of new incidents. Daily and historic data collection
Number of changes*	Performance Analytics	Mapped to Performance Analytics > Indicators > Automated Indicators > Number of new changesSource = Changes.New (change_request table)	Number of changes with a registration date (change_request.opened_at) on collection date	Number of changes opened today. Daily and historic data collection
Number of problems*	Performance Analytics	Mapped to Performance Analytics > Indicators > Automated Indicators > Number of new problemsSource = Problems.New (Problem table)	Problems created today	Number of problems opened today. Daily and historic data collection

Cloud Readiness Indicators (from the Store download)

The APM product team provides a ServiceNow Store plugin designed to aid in analysis of Cloud Readiness. This plugin adds a few new attributes and the following indicators.

Indicator name	Type	Source	How is it calculated?	Description
Strategic Importance	Custom Script	APM product. cmdb_ci_business_app table	Calculated from the Emergency tier (emergency_tier) field of the business application record.	Evaluates the emergency tier of the business application.
Data Classification	Custom Script	APM product. cmdb_ci_business_app table	Calculated from the Data classification (data_classification) field of the business application record.	Evaluates data classification of the business application.
Cloud Version Available	Custom Script	APM product. cmdb_ci_business_app table	Calculated from the Cloud version available (cloud_version_available) field of the business application record.	Evaluates if there is an available version of this application that fits cloud deployment.
Core Architecture	Custom Script	APM product. cmdb_ci_business_app table	Calculated from the Core architecture (core_architecture) field of the business application record.	Evaluates if this business application is part of the core architecture.
Business Criticality	Custom Script	APM product. cmdb_ci_business_app table	Calculated from the Business criticality (business_criticality) field of the business application record.	Evaluates the business criticality of the business application.
Regulatory Implications	Custom Script	APM product. cmdb_ci_business_app table	Calculated from the Regulatory implications (regulatory_implications) field of the business application record.	Evaluates the regulatory implications of the business application.
Business Impact	Indicators	Not applicable	Calculated from the sum of the following indicators: Core architecture (core_architecture) Strategic importance (emergency_tier) Business criticality (business_criticality)	Evaluates the business impact score for the business application, based on other indicators.
Cloud Readiness	Indicators	Not applicable	Calculated from the sum of the following indicators: Data classification (data_classification) Cloud version available (cloud_version_available) Complexity of integration (integration_complexity) Regulatory implications (regulatory_implications)	Evaluates the cloud readiness score for the business application based on other indicators.
Complexity of Integration	Custom Script	APM product. cmdb_ci_business_app table	Calculated from the Integration complexity (integration_complexity) field of the business application record.	Evaluates the complexity of integration to the business application.
User Base	Custom Script	APM product. cmdb_ci_business_app table	Calculated from the User base (user_base) field of the business application record.	Evaluates the user base of the business application.

Old Financial Modeling (ITFM) indicators

These indicators are no longer provided though older APM installation might have them. The ITFM module for SPM (formerly ITBM) is no longer available. Our Partners, Proven Optics, Nicus, and Apptio each have solutions.

Application TCO
Facilities Cost
Hardware Cost
Software Cost
Service Cost
Labor Cost
Other Cost

** note that as of March 2021, the Financial Modeling application and therefore the APM TCO and other cost indicators based upon the ITFM functionality are no longer available.

Operational Indicators

In order to support viable Operational scoring, you must first understand how Incident, Problem and Change are managed at your company. The key is to understand how Application Services are related in the Incident, Problem and Change processes and gather your metrics at the Application Service level then roll these up to the Business Application.

For example, if your company follows CSDM recommended practice, Application Services may be found as the Configuration Item Or, in the Impacted Services related list on Incident. Your configuration of the indicators must follow that data model.

For Incident, you might consider activating Populate the Business Application related list for incidents (com.snc.incident.populate_business_application ) system property and utilizing the related list of Business Applications.

Assessments

Assessments are a platform facility in ServiceNow. In the system, an Assessment is called a Metric Type. Within the assessment, you define Metric Categories and categories contain Metrics (questions). An Assessment, when completed, calculates a score on each contained category based on weighted values of the metric/question results and then derives and overall assessment score based on weighted category values.

Once you have defined the Assessment contents, you identify the SME(s) that will complete the assessment. Assessments can be automated to be sent on a schedule such as quarterly, bi-annual, etc. They may also be launched on demand. The user will see assigned assessments in Self Service > My Assessments & Surveys.

The four Application assessments are Customer Satisfaction, Business Value, Functional Fit and Technical Risk. Typical target users for the assessments are shown below.

Customer Satisfaction – a select sub-set of users for the Business Application
Business Value – Application Business Owner
Functional Fit – IT Owner/Business Owner
Technical Risk – IT Owner

The out of the box assessment questions will need to be defined as they are merely place-holders. Also, it is not required to use all four assessments.

Each Business Application is evaluated on a scale of 1(Low) to 5 (High) for each of the following example criteria.

Customer Satisfaction
- What is the overall customer/consumer satisfaction?
- What is the customer/consumer perceived performance? Does the application perform well during peaks for the business process?
- What is the customer/consumer perceived availability?
- Is the data within the application current and up to date, trustworthy and readily available (per business goals)?
Business Value
- Does the application deliver all of the requirements for the capability it supports?
- Does the business intend to maintain and scale this application for the future or get rid of it?
- How widely is the application used within the assigned user base?
- Is the application able to our changing business needs?
- Are all aspects of the application in use and fit for purpose?
- The application requires little in depth training and user support?
- The application positively impacts our bottom line?
Functional Fit
- Does the application deliver all of the requirements for the capability it supports with none or minimal configuration and no customization?
- Does this application fully automate the processes it supports?
Technical Risk
- Does the application adhere to established architecture (data, integration, infrastructure) guidelines/patterns?
- Is the application used out of the box or customized?
- Does introducing new functionality break existing functionality?
- Is the base technology on a current version that is supported by an internal team or vendor?
- Is the basic technology stack for the application allowed within the enterprise or similar to the greater ecosystem?
- Can the application be easily scaled to add more business functionality, users or integrations to other systems?
- Does the application execute functionality with efficiency (low error rates, average time to execute queries low resource utilization, etc. are all within desired thresholds)?

Business Capabilities and scoring

Business Capabilities are evaluated out of the box via Assessment. the three key, and traditional, evaluations are: People, Process, Technology. See the following examples of questions to configure for these assessments.

People
- Is the Capability supported from a Knowledge / Skills Staffing perspective to deliver on business goals?
- Do people consistently have the same knowledge or awareness of the capability?
Technology
- Rate the relative health of the technologies providing this Capability.
- Are there redundant Business Applications providing this Capability?
- Is the capability supported by the right technology in order to deliver the business goals?
Process
- How mature are the processes underlying this Capability?
- Are these processes evaluated for continual improvement?
- Is the process related to the capability structured, streamlined and automated?

Each dimension is scored using the following scale:

1 – Capability exists and meets future-state business needs

2 – Capability exists and will need minor changes to meet future-state business needs.

3 – Capability exists and need major changes to meet future-state business needs.

4 – Capability does not exist but will be needed for future-state business needs.

5 – Capability exists but requires significant changes to meet current needs and/or future-state business needs.

Indicator Score Calculations

When an indicator is run, it captures a value that is called the Application Weight for each Business Application. Then the system will generate a Normalized Value.

Once the Indicator has processed all application for all date ranges (the system passes in the date ranges for which the indicator will evaluate the business Application that way things such as incidents and Changes don’t keep getting counted if they are outside the date range), it will calculate the Normalized value of the Application indicator.

This normalized value is essentially a “ranking” value from 1-10 of the Application’s value or weight compared to the Minimum and Maximum values found across the set of Applications evaluated. See the normalized calculation below:

Normalized Value = [ (app weight – minWeight)/ (maxWeight – minWeight) ] * 9 + 1

If the Indicator direction is set to Minimize (you want to minimize incidents and the like), the calculation is:

Normalized Value = (10 - Normalized Value(above)) + 1.

For example, if I get a score of 8 for # of Incidents meaning out of all the apps, I am ranking 8 out of 10 in highest number of incidents, I should have a bad Indicator score or at least worse than another app with a 2 or 3.

Indicator Score

After finding the Normalized Value, the system can now generate an Indicator Score which is the weighted value of the Normalized Value within the Scoring Profile for which the indicator score is being generated.

When an Application (or Capability) Scoring Profile is created, the user selects one or more Indicators and assigns the “Weight” it carries in the overall score for the Application.

The Indicator Score is the “weighted” value calculated as:

Indicator Score = Normalized Value * ( Indicator Weight from Profile / Total of all Weights on Profile)

Application Score

Now that we have the Indicator Score, the system can calculate the Application Score for the fiscal period.

Application Score = per Application, Sum of all Indicator Scores for the Application during the fiscal period identified.

Application Profile

As you can tell from the discussion above, it is very important to consider the indicators in use and properly configure your Scoring Profile then appropriately set the correct profile on the Business Applications.