Discovery of Solaris Zones

Solaris can run zones since Solaris 10 (SunOS 5.10). A zone acts as a virtual server.

In a Solaris Server machine running zones, there will always be one global zone and 0 or more child zones (also called just zones).

The global zone is the real, top-level Solaris instance.

The zones are isolated virtual machines that can be run in two modes: fully isolated or sparse-root (some resources as /usr are shared with them by the global zone).

Solaris zones and serial number

In Solaris, the command sneep is used to retrieve the serial number of the machine.

SPARC machines have a EEPROM where a sysadmin can store data, including the serial number of the machine. This serial number can be stored in the EEPROM with the sneep command.

x86/x64 based machines don't have a EEPROM but they can be emulated with the file /boot/solaris/bootenv.rc

It is known of Solaris Zones that unless the serial number has been explicitly set on the EEPROM, the sneep command will return the same serial number as in the global zone. So that all zones will share the same serial number with that of the global zone.

That is a problem for Discovery OOTB Identifiers as we will see next.

Discovery Identification

As part of the Discovery process, when Discovery classifies a discovered device as a Solaris Server it will then try to identify it.

Identify the device means it will try to find out whether there is a record already existing in CMDB (supposedly as a Solaris Server record in the [cmdb_ci_solaris_server] table).

For that, Discovery will try to match comparing part of the information just retrieved from the device (discovered information) against the existing information in CMDB.

Based on the results of matching:

if one and only one record in CMDB is matched, then that is the record that Discovery has to update with the new discovered information from the device.
if no record in CMDB was matched, then a new record has to be created with the discovered information.
in case there are multiple matches, Discovery can not know if the information discovered correspond to one or none of those matches, thus it will just do nothing.

In order to match the discovered information with the existing records in CMDB, Discovery runs Identifiers.

There are different Identifiers shipped OOTB, many of them are disabled. All of them are tried to match in an specific order, the lower the highest priority.

Among the most common Identifiers there are:

Serial Number Table & Class Name
Name & Class Name

Good scenario

Suppose two Solaris Servers that have different serial numbers (XXXXXX and YYYYYY), different names (Solaris1 and Solaris2).

We have a clean empty CMDB and we are going to run some discoveries over these two servers.

1. We discover Solaris1.

As this is the first discovery of this device no Identifier would match so Discovery creates a new CI in CMDB for this record.

2. We discover Solaris1 again.

The Serial Number Table & Class Name identifier would match uniquely the existing CI with serial XXXXXX, so this is the record that Discovery will update with the new information.

3. We discover Solaris2.

As this is the first discovery of this device no Identifier would match so Discovery creates a new CI in CMDB for this record.

4. We discover Solaris2 again.

The Serial Number Table & Class Name identifier would match uniquely the existing CI with serial YYYYYY, so this is the record that Discovery will update with the new information.

Real life scenario

1. We manually add a Oracle Instance [cmdb_ci_db_ora_instance] record with name Solaris1 and serial number XXXXXX.

2. We try to discover the Solaris Server Solaris1.

The "Serial Number Table & Class Name" identifier would still uniquely match the existing Solaris Server by serial number "XXXXXX" and not the Oracle Instance record because the oracle instance record is not of the same class as the Solaris Server (cmdb_ci_db_ora_instance vs cmdb_ci_solaris_server). So Discovery would update the Solaris Server record.

3. We manually add a Solaris Server record with serial number XXXXXX and name Solaris3.

4. We try to discover a new Solaris Server named Solaris3.

Now the Serial Number Table & Class Name identifier will match two existing CIs (Solaris1 and Solaris3). Discovery will run the next identifier trying to find out which one of these two CIs is the one it has to update.

The Name & Class Name identifer will also match the same two CIs.

At this point, Discover can not know which of the two CIs has to update. So it will just do nothing.

Bad scenario

1. We remove the manually created Solaris3 record. Now we just have Solaris1 and Solaris2 in CMDB, with serial numbers XXXXXX and YYYYYY respectively.

2. We go to the Solaris3 server and manipulate the serial number in its EEPROM to be XXXXXX (same as Solaris1).

3. We try to discover the Solaris Server Solaris3.

Now the Serial Number Table & Class Name identifier will match uniquely the existing CI with serial XXXXXX corresponding to Solaris1.

Discovery will update the CI of Solaris1 with the information discovered from Solaris3.

This is similar scenario to discovering Solaris Zones.

The CI Solaris1 with serial number XXXXXX exists in CMDB.

If we try to discover Solaris1Z1 (ZONE1 in Solaris1), Discovery will update the CI record corresponding to the global zone Solaris1.

If we try to discover Solaris1Z2 (ZONE2 in Solaris1) now, Discovery will update the same CI record.

Solutions

There are two solutions to this:

Solution 1 - Append zone to serial number

The probe Solaris - Serial Number runs the command sneep -T on the Solaris Servers (global and child zones) to retrieve the serial number.

This command can return something like the next on global and child zones:

"ChassisSerialNumber" "PX60951106"

We will modify the OOTB Solaris - Serial Number probe so that it runs our own script on the Solaris Servers to retrieve the serial number.

The script will run sneep, but will add the name of the zone to the output of sneep.

On the global zone it will return something like this:

"ChassisSerialNumber" "PX60951106" "global"

On the zones it will return something like this:

"ChassisSerialNumber" "PX60951106" "ZONE1"

There, ZONE1 is the name of the zone retrieved with the command zonename.

Next, we will modify the Multisensor Script that reacts to the Solaris - Serial Number probe so that it parses the zone. For zones that are not the global zone, the script will append the zone to the serial number that will be stored in CMDB.

That way, every Solaris Server record will have its own exclusive serial number and the OOTB Serial Number Table & Class Name identifier should be able now to identify the correct CIs.

This solution is easy to implement but is specific to Solaris (though you could adapt this to others like AIX).

Solution 2 - Custom Identifiers

As discussed above, servers in Solaris Zones can return the same serial number as the parent Solaris Server, or global zone.

We will modify the existing Serial Number Table & Class Name and Serial Number & Class Name identifiers so that they do the next:

check a System Property com.snc.discovery.identify.serial.skip_classes
skip the identifier if one of the classes in that property is cmdb_ci_solaris_server.

Doing that we are letting other identifiers as Name & Class Name match the discovered device to an existing CI in CMDB.

We can add a third custom identifier to match only Solaris on Serial Number & Network if we don't want to relay on Name & Class Name for identification of our CI. This identifier will match an existing CI if the next three conditions are true:

The serial of the device being discovered matches a serial in the table [cmdb_serial_number]
The device discovered has a NIC whose MAC and IP matche those of a NIC in the table [cmdb_ci_network_adapter]
The matched serials and NICs correspond to the same CI

With this solution we can also address the same bad identification for AIX WPARs. We just need to add cmdb_ci_aix_server to our System Property:

com.snc.discovery.identify.serial.skip_classes = cmdb_ci_solaris_server, cmdb_ci_aix_server

Solution 1 - Append zone to serial number

Go to Discovery > Probes.
Search and edit the probe Solaris - Serial Number.
Set the field ECC queue name to sh ${file:get_serials.sh}
Go to the related list Probe parameters of the Solaris - Serial Number probe and add a parameter with Name "get_serials.sh" and Valuecontaining the next script:
#!/bin/sh
FAILED=0
ERR=""
SNEEP="sneep -T"
ZONE=""
get_sneep(){
if [ -x /opt/SUNWsneep/bin/sneep ] ; then
SNEEP="/opt/SUNWsneep/bin/sneep" && return
elif [ -x /usr/sbin/sneep ] ; then
SNEEP="/usr/sbin/sneep" && return
else
SNEEP=which sneep && return
fi
FAILED=1
ERR="Could not find sneep"
}
get_zone(){
ZONE=zonename 2>&1
}
make_serials(){
$SNEEP -T | awk '{ printf("%s \"%s\"\n", $0, z); }' z="$ZONE" && return
if [ $? -eq 0 ] ; then return ; fi
FAILED=1
ERR="Could not parse the output of sneep -T"
}
runit(){
get_sneep
get_zone
make_serials
return
}
runit
[ "$FAILED" != 0 ] && echo "${ERR}"
exit 0
Go to the related list Multisensor Scripts of the _Solaris - Serial Number_probe and edit the existing script. Replace it by the next
function(result, ciData, debug, sensor) {
var output = result.output;
if (output === null || gs.nil(output))
return;
run(result, ciData, debug);
function run(result, ciData, debug) {
var serials = getSerialNumbers(result);
for (var i=0; i<serials.length; i++) {
var serial = serials[i];
var sn = new SncSerialNumber();
gs.print("Is serial valid? " + serial.value + ": " + (sn.isValid(serial['value'])?"true":"false"));
if (sn.isValid(serial['value']))
addToCIData(serial);
}
}
function addToCIData(serial) {
gs.print*("Adding serial " + serial.value);
ciData.getData()['serial_number'] = serial['value'];
var sl = new CIRelatedList('cmdb_serial_number', 'cmdb_ci');
var sr = {};
sr[ 'serial_number_type' ] = serial['type'];
sr[ 'serial_number' ] = serial['value'];
sr[ 'valid' ] = true;
sl.addRec(sr);
ciData.addRelatedList(sl);
}
function getSerialNumbers(result) {
var output = result.output;
var serials = [];
var lines = output.split(/\n/);
for (var i=0; i<lines.length; i++) {
var line = lines[i];
var m = line.match(/"(.+?)"."(.+?)"."(.+?)"$/);
if (!m) {
m = line.match(/"(.+?)".*"(.+?)"$/);
if (!m)
continue;
}
var serial = {};
serial['type'] = m[1];
serial["value"] = makeSerial(m[2], m[3]);
serials.push(serial);
}
return serials;
}
function makeSerial(serial, zone) {
if (gs.nil(zone) || zone == "" || zone == "global")
return serial;
return serial + "-" + zone;
}
}

This script is able to parse output from both the stock sneep -T and our own commands.

Solution 2 - Custom Identifiers

This solution might be more difficult to implement but is not specific to Solaris only.

Go to System Properties > All Properties and add a new property with the next values:
- Name: com.snc.discovery.identify.serial.skip_classes
- Description: Classes of CMDB records for which the "Serial Number Table & Class Name" identifier will be skipped.
- Type: string
- Value: cmdb_ci_solaris_server, cmdb_ci_aix_server
Search and edit the identifier Serial Number Table & Class Name and replace the script by the next:
function(ciData, log, identifier) {
var discoveredSerials = getDiscoveredSerials(ciData);
var matching = matchingCIs(discoveredSerials, ciData);
return new CIIdentifierResult(matching, discoveredSerials.length > 0);
function matchingCIs(discoveredSerials, ciData) {
var matches = [];
// If there's no discovered serial numbers, skip the lookup
if (discoveredSerials.length == 0)
return matches;
// If the device class is one of those that could have duplicated serial numbers, skip the identifier
if (isSkipIdentifier(ciData.getData().sys_class_name))
return matches;
// find matches for all valid serials in the serial number table...
var gr = new GlideAggregate('cmdb_serial_number');
gr.addEncodedQuery(buildEncodedQuery(discoveredSerials));
gr.addAggregate('COUNT', 'cmdb_ci');
gr.addQuery('valid', 'true');
gr.addQuery('absent', 'false');
gr.groupBy('cmdb_ci');
gr.addHaving('COUNT', '=', discoveredSerials.length);
gr.query();
// populate our return value...
while (gr.next())
if (ciData.getData().sys_class_name == gr.cmdb_ci.sys_class_name)
matches.push('' + gr.getValue('cmdb_ci'));
return matches;
}
function isSkipIdentifier(ciClass) {
var classes = getClassesToSkip();
var i;
for (i = 0; i < classes.length; i++) {
if (isParentOf(classes[i], ciClass)) return true;
}
return false;
}
function getClassesToSkip() {
var classes = gs.getProperty('com.snc.discovery.identify.serial.skip_classes', 'cmdb_ci_solaris_server').split(',');
var i;
for (i=0; i<classes.length; i++)
classes[i] = trim(classes[i]);
return classes;
}
function isParentOf(parent, child) {
var tu = new TableUtils(parent);
var extensions = tu.getAllExtensions().toArray();
var i = 0;
for (i = 0; i < extensions.length; i++) {
if (extensions[i] == child) return true;
}
return false;
}
function buildEncodedQuery(discoveredSerials) {
var conds = [];
for (var i = 0; i < discoveredSerials.length; i++)
conds.push('serial_number=' + discoveredSerials[i].serial_number + 'serial_number_type=' + discoveredSerials[i].serial_number_type);
return conds.join('NQ');
}
function getDiscoveredSerials(ciData) {
var srls = ciData.getRelatedList('cmdb_serial_number', 'cmdb_ci');
var result = [];
for (var i = 0; i < srls.length; i++) {
var srl = srls[i];
var sn = new SncSerialNumber();
if (sn.isValid(srl.serial_number))
result.push(srl);
}
return result;
}
}
Search and edit the identifier Serial Number & Class Name and replace the script by the next:
function(ciData, log, identifier) {
var discoveredSerials = getDiscoveredSerials(ciData);
var matching = matchingCIs(discoveredSerials, ciData);
return new CIIdentifierResult(matching, discoveredSerials.length > 0);
function matchingCIs(discoveredSerials, ciData) {
var matches = [];
// If there's no discovered serial numbers, skip the lookup
if (discoveredSerials.length == 0)
return matches;
// If the device class is one of those that could have duplicated serial numbers, skip the identifier
if (isSkipIdentifier(ciData.getData().sys_class_name))
return matches;
// find matches for any discovered serial number...
var gr = new GlideRecord(identifier.applies_to);
gr.addQuery('sys_class_name', ciData.getData().sys_class_name);
gr.addQuery('serial_number', discoveredSerials);
gr.query();
while (gr.next()) {
if (_shouldSkip(discoveredSerials, gr))
continue;
matches.push('' + gr.getUniqueValue());
}
return matches;
}
function isSkipIdentifier(ciClass) {
var classes = getClassesToSkip();
var i;
for (i = 0; i < classes.length; i++) {
if (isParentOf(classes[i], ciClass)) return true;
}
return false;
}
function getClassesToSkip() {
var classes = gs.getProperty('com.snc.discovery.identify.serial.skip_classes', 'cmdb_ci_solaris_server').split(',');
var i;
for (i=0; i<classes.length; i++)
classes[i] = trim(classes[i]);
return classes;
}
function isParentOf(parent, child) {
var tu = new TableUtils(parent);
var extensions = tu.getAllExtensions().toArray();
var i = 0;
for (i = 0; i < extensions.length; i++) {
if (extensions[i] == child) return true;
}
return false;
}
/* Typically if we have serial number table available to match against, we should use it by leveraging
* the "serial number table and class name" identifier. However...
* If we've only discovered one serial number, we should always match with it.
* If the serial number table identifer doesn't exist or it's been deactivated, we don't want to skip.
* If the CI we've just found by looking at the serial number field doesn't have anything in the serial number table, then we shouldn't skip
/
function _shouldSkip(discoveredSerials, gr) {
// We've only got one valid serial to match, don't skip
if (discoveredSerials.length == 1)
return false;
// Get the Serial Number Table & Class Name identifier
var idgr = _getSerialNumberTableIdentifier();
// if we can't find the serial number table identifier, don't skip!
if (!idgr)
return false;
// if it's been deactivated, don't skip!
if (idgr.active == false)
return false;
if (hasSerialNumberTable(gr))
return true;
return false;
}
function _getSerialNumberTableIdentifier() {
// Look up by sys_id or name.
var gr = new GlideRecord("ci_identifier");
if (gr.get("0e9717c40a0a0b4508e68214b8335cbd"))
return gr;
else if (gr.get("name", "Serial Number Table & Class Name"))
return gr;
else
return null;
}
/
* Check for skip here because if a CI already has serial numbers populated in the serial number table, we shouldn't even bother
* with checking the serial number field value anymore.
*/
function hasSerialNumberTable(cigr) {
var sgr = new GlideRecord("cmdb_serial_number");
sgr.addQuery("cmdb_ci", cigr.sys_id);
sgr.addQuery('absent', 'false');
sgr.addQuery('valid', 'true');
sgr.setLimit(1);
sgr.query();
if (sgr.next())
return true;
else
return false;
}
function getDiscoveredSerials(ciData) {
var srls = ciData.getRelatedList('cmdb_serial_number', 'cmdb_ci');
var result = [];
for (var i = 0; i < srls.length; i++) {
var srl = srls[i];
var sn = new SncSerialNumber();
if (sn.isValid(srl.serial_number))
result.push(srl.serial_number);
}
return result;
}
}
Go to Discovery > CI Identification > Identifiers and add a new CI Identifier with the next values:
- Name: Serial Number Table & Network
- Applies to: Solaris Server [cmdb_ci_solaris_server] (or Server [cmdb_ci_server] if you want it to apply to others including AIX)
- Order: 1060 (after the other Serial indentifiers)
- Description: Solaris zones can return the same serial number as the global zone. This Identifier will match on serial and also compare the MAC and IP address to identify the Solaris Server record
- Script:
  function(ciData, log, identifier) {
  var discoveredSerials = discoveredSerials = getDiscoveredSerials(ciData);
  var discoveredNICs = getDiscoveredNICs(ciData);
  var matching = matchingCIs(discoveredSerials, discoveredNICs, ciData);
  return new CIIdentifierResult(matching, discoveredSerials.length > 0 && discoveredNICs.length > 0);
  function matchingCIs(discoveredSerials, discoveredNICs, ciData) {
  var matches = [];
  // If there's no discovered serial numbers, skip the lookup
  if (discoveredSerials.length == 0)
  return matches;
  // If there's no discovered NIC, skip the lookup
  if (discoveredNICs.length == 0)
  return matches;
  // find matches for all valid serials in the serial number table...
  var serialMatch = new GlideAggregate('cmdb_serial_number');
  serialMatch.addEncodedQuery(buildSerialsEncodedQuery(discoveredSerials));
  serialMatch.addAggregate('COUNT', 'cmdb_ci');
  serialMatch.addQuery('valid', 'true');
  serialMatch.addQuery('absent', 'false');
  serialMatch.groupBy('cmdb_ci');
  serialMatch.addHaving('COUNT', '=', discoveredSerials.length);
  serialMatch.query();
  var serialMatches = [];
  while (serialMatch.next()) {
  if (ciData.getData().sys_class_name == serialMatch.cmdb_ci.sys_class_name)
  serialMatches.push('' + serialMatch.getValue('cmdb_ci'));
  }
  // find matches for all NICs in the network adapter table...
  var nicMatch = new GlideAggregate('cmdb_ci_network_adapter');
  nicMatch.addEncodedQuery(buildMacsEncodedQuery(discoveredNICs));
  nicMatch.addAggregate('COUNT', 'cmdb_ci');
  nicMatch.addQuery('valid', 'true');
  nicMatch.addQuery('install_status', '!=', '100');
  nicMatch.groupBy('cmdb_ci');
  nicMatch.addHaving('COUNT', '=', discoveredNICs.length);
  nicMatch.query();
  // get only those NICs whose CI matches the CI of the serials found
  var nic_ci;
  var i = 0;
  while (nicMatch.next()) {
  nic_ci = '' + nicMatch.getValue('cmdb_ci');
  for (i = 0; i < serialMatches.length; i++) {
  if (nic_ci == serialMatches[i])
  matches.push(nic_ci);
  }
  }
  return matches;
  }
  function getDiscoveredSerials(ciData) {
  var srls = ciData.getRelatedList('cmdb_serial_number', 'cmdb_ci');
  var serials = [];
  for (var i = 0; i < srls.length; i++) {
  var srl = srls[i];
  var sn = new SncSerialNumber();
  if (sn.isValid(srl.serial_number)) {
  serials.push(srl);
  }
  }
  return serials;
  }
  function getDiscoveredNICs(ciData) {
  var adapters = ciData.getRelatedList("cmdb_ci_network_adapter", "cmdb_ci");
  var result = [];
  for(var i = 0; i < adapters.length; i++) {
  if (gs.nil(adapters[i].mac_address) || gs.nil(adapters[i].ip_address))
  continue;
  result.push(adapters[i]);
  }
  return result;
  }
  function buildSerialsEncodedQuery(discoveredSerials) {
  var conds = [];
  for (var i = 0; i < discoveredSerials.length; i++)
  conds.push('serial_number=' + discoveredSerials[i].serial_number + 'serial_number_type=' + discoveredSerials[i].serial_number_type);
  return conds.join('NQ');
  }
  function buildMacsEncodedQuery(discoveredNICs) {
  var conds = [];
  for (var i = 0; i < discoveredNICs.length; i++) {
  var ma = SncMACAddress.getMACAddressInstance(discoveredNICs[i].mac_address);
  if (!ma)
  continue;
  conds.push('mac_address=' + ma.getAddressAsString());
  }
  return conds.join('NQ');
  }
  }