Starting/Stopping multiple VMs in AWS

This article deals with how we can implement starting/stopping multiple VMs in AWS. The same technique can be used for other clouds as well.

AWS has an API to start and stop multiple VMs in a single API call. We are going to use those APIs.

https://ec2.amazonaws.com/?Action=StopInstances&InstanceId.1=i-1234567890abcdef0 https://ec2.amazonaws.com/?Action=StartInstances&InstanceId.1=i-1234567890abcdef0

We will be adding a Day-2 operation at the AWS Datacenter level. The flow will be such that the users will select a particular AWS Datacenter first and then choose the operation (start / stop multiple VMs) and then identify those instances that need to be targeted.

---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

==>

---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Now let us look a the steps that are needed. In the end we will provide an update set which will do all the above as well.

Make sure that the AWS Datacenters show up in the End User portal.
Extend the OOB AWS Cloud API MID server scripts to do the multi VM Start and Stop.
Create a new operation in AWS Datacenter for Starting and Stopping multiple VMs.
Create a Catalog Item and use List Collector for choosing multiple VMs

Show AWS Datacenters in the End User Portal

This is captured elaborately in this article https://community.servicenow.com/community?id=community_article&sys_id=b28f7f711b4f7348ada243f6fe4bc.... Please read this. It is about making sure the AWS Datacenter CIs are visible in the portal. The change will be about setting the CAPI Resource type for AWS Datacenter to be visible and have proper category.

You can also import the attached xml into your instance and skip this step.

Extend OOB AWS Scripts

Create MID Scripts

The OOB Cloud API takes care of authentication and many operations. We will extend the OOB classes to do the specific Start/Stop multiple VMs.

We will use the AWSCloudAPIBase class that comes with the product. We will provide the proper service and APIVersion.

var AWSVMOperations = Class.create();
AWSVMOperations.prototype = Object.extendsObject(AWSCloudAPIBase, {
    service: 'ec2',
    apiVersion: '2016-11-15',
    initialize: function(parameters, headers) {
        AWSCloudAPIBase.prototype.initialize.call(this, parameters, headers);
    },
    multiVMDirectOps: function(action, vmInstanceIds) {
        var index = 1;
        var vms = new JSON().decode(vmInstanceIds);
        var actionParameters = {};
        for(var vmindex = 0; vmindex < vms.length; vmindex ++) {
            actionParameters['InstanceId.'+(vmindex+1)] = vms[vmindex];
        }
        return this.executeAction(action, actionParameters);
    },
    addTags: function(tags,vmInstanceIds) {     
        var action = 'CreateTags';
        var customTags = new JSON().decode(tags);
        var index = 1;
        var vms = new JSON().decode(vmInstanceIds);
        var actionParameters = {};
        for(var vmindex = 0; vmindex < vms.length; vmindex ++) {
            actionParameters['ResourceId.'+(vmindex+1)] = vms[vmindex];
            var i = 1;
            for(var key in customTags){
                actionParameters['Tag.'+(i+1)+'.Key'] = key;
                actionParameters['Tag.'+(i+1)+'.Value'] = customTags[key];
                i++;
            }
        }
        return this.executeAction(action, actionParameters);
    },
    type: 'AWSVMOperations'
});

Look for the 'multiVMDirectOps' method. It takes in the action and array of VMs that need to be operated upon. The action could be either 'StartInstances' or 'StopInstances'. The array notation in the parameters are specific to the way AWS does it.

https://ec2.amazonaws.com/?Action=StopInstances&InstanceId.1=i-1234567890abcdef0 https://ec2.amazonaws.com/?Action=StartInstances&InstanceId.1=i-1234567890abcdef0

Create CAPI Artifacts

Create a single AWS Extensions Interface where you can put in all your methods. For starters, added 'StartNodes' and 'StopNodes' as the operation. Both these operation will take in 'Location' and 'NodeIDs' as parameters.

Create one CAPI API impl and call it as 'AWS Extensions API'. This could be your single API for all extensions. As a start it would have just two operations. Have the API Config Overrides setup as these pertain to the passing of the credentials seemlessly.

By default both the Start and Stop Nodes will be tied to a MID script include which does nothing. Get into that script and make the call to the AWSVMOperations class to do the needful.

Click on the record indicator as depicted in the image below:

---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Click on the Request Script record indicator to get to the auto-generated mid script include.

---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Create StopInstances Operation in AWS Datacenter Resource Block

Get to Cloud Admin Portal ==> Design ==> Resource Blocks ==> AWS Datacenter ==> Operation.

Override any operation (for example ConnectAndStopVirtualMachine) in Compute Interface and name it as 'AWS Extensions'. This resource interface will have the custom operations that will be exposed to the end user via a Day-2 catalog item.

Then add an operation called 'StopInstances' in the extension. Call the CAPI method which we did in the first step. The NodeIDs parameter is something which should be exposed to the catalog item. Check the checkbox for 'Create Form Parameter' and 'Mandatory'.

Now let us turn our attention to the catalog item as this is what the end user will be consuming. Click on the 'Generate Catalog' button to create the catalog item for this operation. After creation, the button will change to 'View Catalog'. You can click that to get to the catalog item. It will create a variable for the NodeIDs in addition to the others. The NodeIDs is something which we want to be multi-selectable. The service catalog provides a type called 'List Collector' and we will use this.

Let us make the NodeIDs to be List Collector and be backed by Virtual Machine Instance. Add a proper reference qualifier so that the only those VMs that are pertinent to the chosen datacenter/region will be picked.

The List Collector will always return a CSV of sys IDs. The AWS API though will need the resource IDs (object_id). We will use a script include to convert the CSV of sys_IDs to a list of Object IDs. We will use a method in CMDBResourceUtils to make this happen. The mapping would be '$(Script:sn_cmp.CMDBResourceUtils.convertVMSysIDs_to_ObjectIDs[arg=${parameter.NodeIDs}])' as indicated in the image below.

All the above changes are captured in the update set here : https://developer.servicenow.com/app.do#!/share/contents/4708895_stop_and_start_multiple_vms_in_aws?...