Skip navigation
1 2 3 Previous Next

Server Automation

101 posts
Share:|

This seems to be a fairly common case:  you want to run a job from the blcli and wait for it to finish and then do something with the result.  You can use something like Job.executeJobAndWait but this can be problematic if the job runs for long enough that the blcli to appserver connection will be deemed idle.  The blcli sits there with no output so you aren't getting any indication anything is going on (hence the idle timeout problem), if that blcli to appserver connection is still alive or anything else.  Instead maybe you can start the job and get something back you can use to check the run status and then loop over running that command until you see the job run has finished.

 

The first thing to do is to find the right blcli command to run.  There are a couple that look promising: Job.executeJobAndReturnScheduleID and Job.executeJobAndWaitForRunID.  We need a command that will output something we can use to get the running status and I see a couple commands in the JobRun namespace that look like they will do that: JobRun.getJobRunStatusByScheduleId and JobRun.getJobRunIsRunningByRunKey  (One thing that's a little confusing that well see later is the executeJobAndWaitForRunID actually returns the jobRunKey, not the jobRunId.  And that's ok because we can use that with the getJobRunIsRunningByRunKey command)  There are some other commands that let you 'execute against' a set of targets for that run and they also return the scheduleId or jobRunKey.  As long as the execution command returns the scheduleId or jobRunKey or jobRunId it looks like we will be in good shape.  As you can tell there is no right blcli command.

 

We've found a couple different sets of commands that look like they will accomplish our goal of starting a job run and returning something that we can use to check status with another command.  We can use either command set.  The schedule one looks nice to me because it gives more information about status than running or not.  Let's look at that one first.  We need to get the jobKey, run it and get the scheduleId and then do a loop until the job returns a 'COMPLETE' status.  That's pretty straightforward:

 

blcli_execute NSHScriptJob getDBKeyByGroupAndName "/Workspace" "myJob"
blcli_storeenv JOB_KEY
blcli_execute Job executeJobAndReturnScheduleID ${JOB_KEY}
blcli_storeenv JOB_SCHEDULE_ID
JOB_STATUS="INCOMPLETE"
while [[ -n "${${JOB_STATUS//COMPLETE}//$'\n'}" ]]
     do
     sleep 10
     blcli_execute JobRun getJobRunStatusByScheduleId ${JOB_SCHEDULE_ID}
     blcli_storeenv JOB_STATUS
     echo "Schedule ${JOB_SCHEDULE_ID} status is: ${JOB_STATUS//$'\n'/ }"
done

 

There are some zsh-isms going on there I can explain: ${JOB_STATUS//$'\n'/ } removes the trailing new line from the return of getJobRunStatusByScheduleId.  ${${JOB_STATUS//COMPLETE}//$'\n'} does the same thing and also removes the string COMPLETE from the output and then the while test looks to see if the variable is non-zero in size.  When the job completes then it will show a status of COMPLETE only and by removing that string the variable will be zero and the loop will break.

 

You could put a counter in there as well if you want to prevent it from getting stuck in the loop, so something like:

 

blcli_execute NSHScriptJob getDBKeyByGroupAndName "/Workspace" "myJob"
blcli_storeenv JOB_KEY
blcli_execute Job executeJobAndReturnScheduleID ${JOB_KEY}
blcli_storeenv JOB_SCHEDULE_ID
JOB_STATUS="INCOMPLETE"
COUNT=1
while [[ -n "${${JOB_STATUS//COMPLETE}//$'\n'}" ]] && [[ ${COUNT} -le 100 ]]
     do
     sleep 10
     blcli_execute JobRun getJobRunStatusByScheduleId ${JOB_SCHEDULE_ID}
     blcli_storeenv JOB_STATUS
     echo "Schedule ${JOB_SCHEDULE_ID} status is: ${JOB_STATUS//$'\n'/ }"
     let COUNT+=1
done

Now I might want to check if the job run had errors or dump the job run log items out or use one of the Utility commands to export a job result or log to a file.  With the scheduleId approach I'll need to use a couple unreleased blcli commands to convert the schedule id into something i can use.

 

I can run something like this to convert the scheduled into a jobRunId or jobRunKey:

blcli_execute JobRun findByScheduleId ${JOB_SCHEDULE_ID}
blcli_execute JobRun getJobRunId
blcli_storeenv JOB_RUN_ID

or

blcli_execute JobRun findByScheduleId ${JOB_SCHEDULE_ID}
blcli_execute JobRun getJobRunKey
blcli_storeenv JOB_RUN_KEY

 

Depending on what other commands you want to run, they will take the jobRunId or jobRunKey choose the appropriate sequence above to feed into the commands you want to run.

 

 

Now that I have shown how to poll for status using the scheduleId we will look at how to poll for status using the getJobRunIsRunningByRunKey approach.  It's very similar - we execute the job and get the jobRunKey returned to us and then we do a loop checking to see if the job is running:

blcli_execute Job executeJobAndWaitForRunID ${JOB_KEY}
blcli_storeenv JOB_RUN_KEY
JOB_IS_RUNNING="true"
while [[ "${JOB_IS_RUNNING}" = "true" ]]
        do
        sleep 10
        blcli_execute JobRun getJobRunIsRunningByRunKey ${JOB_RUN_KEY}
        blcli_storeenv JOB_IS_RUNNING
        echo "${JOB_IS_RUNNING}"
done

 

In a normally functioning environment these will more or less work the same.  A problem with the getJobRunIsRunning approach is that if the job doesn't start running, for example you have a job routing rule sending the job to a downed appserver, your appservers are maxed out on WorkItemThreads, or you've reached the MaxJobs threshold, then your executeJobAndWaitForRunID is going to sit waiting for the job to start running and you are back to the possible idle connection timeout happening.

 

There are other commands that look interesting in the JobRun namespace (released and unreleased).  The JobRun.showRunningJobs looks neat - it outputs a nice table - but it only shows the job name which is not enough to know if your job is actually running since you can have duplicate job names in different workspace folders.

 

The above two examples are pretty simple.  They can be extended to manage multiple job runs - for example if you wanted to kick off a number of jobs at once and then wait until they were all complete, check their status, and then move on to some other action.  One approach to that would be some arrays to hold the job information an another loop around what I've done above.  That may be a future post.

Share:|

The Utility.exportPatch* and related commands only pull out the patch analysis results.  That won't help if you want to look at the log messages to troubleshoot an error during the run.  For that you need to get the contents of the job run log.  There's no command to dump that (yet?).  Also it would make sense to only get the log messages for the servers that threw a warning or error, since you probably don't care about the successful ones.

 

The Patching Job is like a Batch Job - it contains the Analysis Job and possibly the Remediation and Download Jobs.  Since the Analysis Job is doing most of the work that's where the information we need will be stored.  So we start with getting the latest run of a Patching Job, getting the associated Analysis Job Run (there should be only one and we exclude the Download and Remediation), then finding the failed targets for that run and pulling out the log messages for each server and write it off to a file.

 

You could also

  • Look at a particular Patching Job run, not the latest, knowing the start date or if you just ran it and had the run key.
  • Email off the file(s), collect them, ship them to some other system
  • get the logs for all the servers in the job
  • get the top-level job run logs for the patching job and analysis job runs with the 'LogItem.getLogItemsByJobRun' or 'JobRun.getLogItemsByJobRunId' (second one calls the first actually)

 

 

 

#!/bin/nsh
blcli_setjvmoption -Dcom.bladelogic.cli.execute.quietmode.enabled=true
blcli_setoption serviceProfileName defaultProfile
blcli_setoption roleName BLAdmins
blcli_connect

patchingJob="/Workspace/Patching Jobs/RedHat Patching Job"
blcli_execute PatchingJob getDBKeyByGroupAndName "${patchingJob%/*}" "${patchingJob##*/}"
blcli_storeenv jobKey
blcli_execute JobRun findLastRunKeyByJobKey ${jobKey}
blcli_storeenv jobRunKey
blcli_execute JobRun jobRunKeyToJobRunId ${jobRunKey}
blcli_storeenv jobRunId
blcli_execute JobRun findPatchingJobChildrenJobsByRunKey ${jobRunId}
blcli_execute JobRun getJobRunId
blcli_execute Utility setTargetObject
blcli_execute Utility listPrint
blcli_storeenv patchChildJobRunIds
patchChildJobRunIds="$(awk 'NF' <<< "${patchChildJobRunIds}")"
while read patchChildJobRunId
        do
        blcli_execute JobRun findById ${patchChildJobRunId}
        blcli_execute JobRun getType
        blcli_storeenv jobRunType
        blcli_execute JobRun getJobKey
        blcli_storeenv analysisKey
        # skip remediation
        if [[ ${jobRunType} -ne 7033 ]] && [[ ${jobRuntype} -ne 7031 ]]
         then
           blcli_execute JobRun getServersStatusByJobRun ${patchChildJobRunId}
           blcli_execute Utility mapPrint
           blcli_storeenv serverStatusMap
           serverStatusMap="$(awk 'NF' <<< "${serverStatusMap}")"
           while read serverStatus
                do
                     serverKey="$(awk '{print $1}' <<< "${serverStatus}")"
                     echo "ServerKey: ${serverKey}"
                     serverResult="$(awk '{print $3}' <<< "${serverStatus}")"
                     # if it is an error get the job run log
                     if [[ ${serverResult} -eq 1 ]] || [[ ${serverResult} -eq 2 ]]
                          then
                          blcli_execute Server findByDBKey ${serverKey}
                          blcli_execute Server getName
                          blcli_storeenv serverName
                          blcli_execute Server getServerId
                          blcli_storeenv serverId
                          blcli_execute LogItem getLogItemsByDevice ${analysisKey} ${patchChildJobRunId} ${serverId}
                          blcli_execute Utility listPrint
                          blcli_storeenv jobRunLog
                          echo "${jobRunLog}" > "/tmp/${patchChildJobRunId}-${serverName}"
                fi
           done <<< "${serverStatusMap}"
    fi
done <<< "${patchChildJobRunIds}"
Share:|

This has come up at a couple customers recently so I figured I'd post something about it.

 

Problem: You have a number of approved exceptions that should be set on a components associated w/ a component template.  As new servers are added and components created there is no mechanism to have exceptions applied to the new components so they are in place before the next compliance run.

 

Solution: While there is no built-in mechanism to do this in BSA currently this can be accomplished with some blcli and nsh scripting.

 

There's a couple parts to this - first is where to keep the exceptions.  A couple obvious places are the Property Dictionary or a text file.  So of course I'm going to use the Property Dictionary because BSA! but the text file is fine also.

 

So I have a custom property class called ApproveExceptions that contains properties that define the exception (exceptionName, exceptionDescription, etc).  I ended up writing a script to create this and populate it, more on that later. So in this class I created sub-classes for each Component Template that I need to set exceptions on.  Under each sub-class I create the instances for each exception I need to set.  That's a pretty simple setup since I'm only working w/ the out-of-the-box component templates.  You might need to make something with more depth, or add some properties to better identify the template.  The point here was to make this as automated and self-resolving as possible.  I also define the Templates workspace folder my templates are in - I'm assuming I'll loop through them all, then loop through the sub-class that list what templates should get some exceptions and match them up.  Once I know what template I'm looking at I grab all the exceptions I need to set.  Then I grab all the components for the template and loop through them.  I see what exceptions are already set, compare them to the exceptions that should be set, and if there is a mismatch, set the exception.  New exceptions are new property instances that can be added in the Property Dictionary manually through the gui or via the blcli.

 

A few ideas:

  • Around line 80 instead of getting all the components for the template, just get the components created in the last few days and only process those.  If the default exception list is static that should reduce the processing time, especially in larger environments.  This could be done w/ the creation of a smart component group based on the template and the DATE_CREATED property.
  • Read the exceptions out of a file instead of pulling the exceptions out of the Property Dictionary.  Might work as an integration point with another compliance tracking system or CMDB.
  • Parameterize the script so it can either process just the newly components or all and then you have a script that can be used to only handle newly created components or push out a new exception across the existing components.
  • Put this in a batch job that runs Discovery, this NSH Script and then Compliance so you know the Components always have the exceptions set before Compliance runs.

 

 

#!/bin/nsh

blcli_setjvmoption -Dcom.bladelogic.cli.execute.quietmode.enabled=true
blcli_setoption serviceProfileName defaultProfile
blcli_setoption roleName BLAdmins
blcli_connect

exceptionClass="Class://SystemObject/ApprovedExceptions"
exceptionFile="/tmp/exceptions.csv"
templateGroup="/CIS Compliance Content"


# get all the templates in the group
blcli_execute TemplateGroup groupNameToDBKey "${templateGroup}"
blcli_storeenv templateGroupKey
blcli_execute Template findAllByGroup ${templateGroupKey} true
blcli_execute Template getDBKey
blcli_execute Utility setTargetObject
blcli_execute Utility listPrint
blcli_execute Utility setTargetObject
blcli_storeenv templateKeys
templateKeys="$(awk 'NF' <<< "${templateKeys}")"

blcli_execute Utility getCurrentRole
blcli_storeenv rbacRole
blcli_execute Utility getCurrentUserName
blcli_storeenv rbacUser


# for each template, get the exception name and other info
while read templateKey
        do
        blcli_execute Template findByDBKey ${templateKey}
        blcli_execute Template getName
        blcli_storeenv templateName
        blcli_execute Template getGroupId
        blcli_storeenv templateGroupId
        blcli_execute Group getQualifiedGroupName 5008 ${templateGroupId}
        blcli_storeenv templateGroupPath
        echo "Processing ${templateGroupPath}/${templateName}..."
        blcli_execute PropertyClass isPropertyClassDefined "${exceptionClass}/${templateName}"
        blcli_storeenv classExists

        if [[ "${classExists}" = "false" ]]
                then
                echo "   Cannot find exception class ${exceptionClass}/${templateName}, will not set exceptions for ${templateName}..."
                break
        fi

        # get the instance (exception) names in the class (for the template)
        blcli_execute PropertyClass listAllInstanceNames "${exceptionClass}/${templateName}"
        blcli_storeenv instanceNames
        instanceNames="$(awk 'NF' <<< "${instanceNames}")"

        typeset -a exceptionNames
        typeset -A exceptionDescriptions
        typeset -A exceptionComments
        typeset -A exceptionRuleNames

        if [[ -n ${instanceNames} ]]
                then
                while read i
                        do
                        echo "   Getting exception information for: ${i}..."
                        blcli_execute PropertyInstance getPropertyValue "${i}" "exceptionName"
                        blcli_storeenv exceptionName
                        exceptionNames+=("${exceptionName}")
                        blcli_execute PropertyInstance getPropertyValue "${i}" "exceptionDescription"
                        blcli_storeenv exceptionDescription
                        exceptionDescriptions[${exceptionName}]="${exceptionDescription}"
                        blcli_execute PropertyInstance getPropertyValue "${i}" "exceptionComment"
                        blcli_storeenv exceptionComment
                        exceptionComments[${exceptionName}]="${exceptionComment}"
                        blcli_execute PropertyInstance getPropertyValue "${i}" "exceptionRuleName"
                        blcli_storeenv exceptionRuleName
                        exceptionRuleNames[${exceptionName}]="${exceptionRuleName}"
                done <<< "${instanceNames}"

                # get all the components for a template
                blcli_execute Component getAllComponentKeysByTemplateKey ${templateKey}
                blcli_storeenv componentKeys
                componentKeys="$(awk 'NF' <<< "${componentKeys}")"

                while read componentKey
                        do
                        blcli_execute Component componentKeyToName ${componentKey}
                        blcli_storeenv componentName
                        echo "   Processing exceptions on ${componentName}..."
                        blcli_execute Component findComponentExceptions ${componentKey}
                        blcli_execute ComponentException getName
                        blcli_execute Utility setTargetObject
                        blcli_execute Utility listPrint
                        blcli_storeenv componentExceptions
                        for i in {1..${#exceptionNames}}
                                do
                                exceptionName="${exceptionNames[${i}]}"
                                echo "    Checking ${componentName} for ${exceptionName}..."
                                hasException="false"
                                while read componentException
                                        do
                                        if [[ "${exceptionName}" = "${componentException}" ]]
                                                then
                                                hasException="true"
                                                echo "     ${componentName} has exception: ${exceptionName}..."
                                                break
                                        fi
                                done <<< "${componentExceptions}"
                        
                                if [[ "${hasException}" = "false" ]]
                                        then
                                        echo "     ${componentName} needs exception: ${exceptionName}, setting..."
                                        echo "      Adding exception: ${exceptionName} ${exceptionDescriptions[${exceptionName}]} ${exceptionComments[${exceptionName}]} on rule: ${exceptionRuleNames[${exceptionName}]}..."
                                        blcli_execute ComponentException createComponentExceptionWithOneRule ${componentKey} "${exceptionName}" "${exceptionDescriptions[${exceptionName}]}" "${rbacRole}" "${rbacUser}" "${exceptionComments[${exceptionName}]}" "${templateGroupPath}" "${templateName}" "${exceptionRuleNames[${exceptionName}]}"
                                        blcli_storeenv componentKey
                                fi
                        done 
                done <<< "${componentKeys}"

                unset exceptionNames exceptionDescriptions exceptionComments exceptionRuleNames
        else    
                echo "   No exceptions for ${templateName}..."
        fi
done <<< "${templateKeys}"

 

When I was building the script above I realized I wanted a way to create the Property Dictionary instances by processing a text file so I had something to work with in the other script, instead of creating the instances by hand.  So that script is below.  It reads a csv file (further below) and creates the instances.

 

#!/bin/nsh

blcli_setjvmoption -Dcom.bladelogic.cli.execute.quietmode.enabled=true
blcli_setoption serviceProfileName defaultProfile
blcli_setoption roleName BLAdmins
blcli_connect

exceptionClass="Class://SystemObject/ApprovedExceptions"
typeset -A exceptionProperties
exceptionProperties=(exceptionName Primitive:/String exceptionDescription Primitive:/String exceptionComment Primitive:/String exceptionRuleName Primitive:/String)
exceptionFile="/tmp/testExceptions.csv"
bulkPropertyFile="/tmp/bulkSetPSIProps.csv"
templateGroup="/CIS Compliance Content"

[[ -f "${bulkPropertyFile}" ]] && rm -f "${bulkPropertyFile}"

blcli_execute PropertyClass isPropertyClassDefined "${exceptionClass}"
blcli_storeenv classExists

if [[ "${classExists}" = "false" ]]
        then
        blcli_execute PropertyClass createSubClass "${exceptionClass%/*}" "${exceptionClass##*/}" ""
fi

for i in ${(k)exceptionProperties}
        do
        blcli_execute PropertyClass isPropertyDefined "${exceptionClass}" "${i}"
        blcli_storeenv propertyExists

        if [[ "${propertyExists}" = "false" ]]
                then
                blcli_execute PropertyClass addProperty "${exceptionClass}" "${i}" "${i}" "${exceptionProperties[${i}]}" true false ""
        fi
done

# get all templates in the group, will create a subclass for each one to hold the exceptions
blcli_execute TemplateGroup groupNameToDBKey "${templateGroup}"
blcli_storeenv templateGroupKey
blcli_execute Template findAllByGroup ${templateGroupKey} true
blcli_execute Template getDBKey
blcli_execute Utility setTargetObject
blcli_execute Utility listPrint
blcli_execute Utility setTargetObject
blcli_storeenv templateKeys
templateKeys="$(awk 'NF' <<< "${templateKeys}")"

while read templateKey
        do
        blcli_execute Template findByDBKey ${templateKey}
        blcli_execute Template getName
        blcli_storeenv templateName
        blcli_execute PropertyClass isPropertyClassDefined "${exceptionClass}/${templateName}"
        blcli_storeenv subClassExists
        if [[ "${subClassExists}" = "false" ]]
                then
                blcli_execute PropertyClass createSubClass "${exceptionClass}" "${templateName}" ""
        fi
done <<< "${templateKeys}"

# read the csv file that lists the env-wide exceptions for each template and create the PSI and set the property values for each one.
if [[ -f "${exceptionFile}" ]]
        then
        while IFS=, read templateName exceptionName exceptionDescription exceptionComment exceptionRuleName
                do
                blcli_execute PropertyClass isPropertyClassInstanceDefined "${exceptionClass}/${templateName}" "${exceptionName}"
                blcli_storeenv instanceExists

                if [[ "${instanceExists}" = "false" ]]
                        then
                        blcli_execute PropertyInstance createInstance "${exceptionClass}/${templateName}" "${exceptionName}" ""
                else
                        echo "\"${exceptionClass}/${templateName}/${exceptionName}\",\"exceptionName\",\"${exceptionName}\"" >> "${bulkPropertyFile}"
                        echo "\"${exceptionClass}/${templateName}/${exceptionName}\",\"exceptionDescription\",\"${exceptionDescription}\"" >> "${bulkPropertyFile}"
                        echo "\"${exceptionClass}/${templateName}/${exceptionName}\",\"exceptionRuleName\",\"${exceptionRuleName}\"" >> "${bulkPropertyFile}"
                        echo "\"${exceptionClass}/${templateName}/${exceptionName}\",\"exceptionComment\",\"${exceptionComment}\"" >> "${bulkPropertyFile}"
                fi
        done < "${exceptionFile}"
        if [[ -s "${exceptionFile}" ]]
                then
                blcli_execute PropertyInstance bulkSetPropertyValues "${bulkPropertyFile%/*}" "${bulkPropertyFile##*/}"
        fi
else
        echo "Cannot find ${exceptionFile}..."
        exit 1
fi

 

Sample input file for the Property Instance creation script:

Template Name, exception name, exception description,comment,rule path/name.

CIS - Windows Server 2008 R2,exception-1,description-1,comment-1,/1.1.1.1 System Services/1.1.1.1.3 Configure IIS Admin Service
CIS - Windows Server 2008 R2,exception-2,description-2,comment-2,/1.1.1.1 System Services/1.1.1.1.4 Configure Windows Installer
CIS - Windows Server 2012 R2,exception-1a,description-1a,comment-1a,/1 Account Policies/1.1.1 Set Enforce password history to 24 or more passwords
CIS - Windows Server 2012 R2,exception-2a,description-2a,comment-2a,/1 Account Policies/1.1.3 Set Minimum password age to 1 or more days

 

This was a couple hours of hacking at it.  There are likely some performance improvements and error handling to be added and it hasn't been tested at any large scale.

Share:|

I am very pleased to announce that BSA 8.9 Service Pack 1 is now available.

 

Here are some highlights of the release:

1> Support for TLS 1.2

BMC Server Automation now supports version 1.2 of the Transport Layer Security (TLS) protocol for session layer security across the various communications legs between BMC Server Automation components.

After installing or upgrading to BMC Server Automation 8.9.01, TLS version 1.2 is the default protocol for communication between the Application Servers and the RSCD Agents.

2> Windows 2016 Support:

Support for Microsoft Windows Server 2016 was added in the following components and functionalities in BMC Server Automation:

  • Unified Product Installer (UPI)
  • Application Server
  • Patch management
  • Bare-metal provisioning

    Virtualization for the following types of virtual servers:

  • VMware vSphere, vCenter Server 5.5, 6.0, and 6.5
  • Microsoft Hyper-V 3.0. SCVMM 2012 R2

3> Support for Native YUM:

BMC Server Automation now supports the use of native YUM as the patching tool for deploying patches to target Linux servers. Prior to this release of BMC Server Automation, native YUM was supported only on Red Hat Enterprise Linux version 7, and it is now supported on the following types and versions of Linux servers:

  • RHEL 5 and 6
  • OEL 5, 6, and 7

4> RHEL patch analysis optimizations for include and exclude:

BMC Server Automation can automatically select the appropriate rpm version or versions while including or excluding an rpm package in an RHEL patch analysis job. To enable this version optimization, select the By Package Name Only option while including or excluding patches.

5> Database cleanup enhancement:

During online database cleanup, various cleanup-related log messages are saved in the DBM_RUN_LOG table. This table is now cleaned up as well during subsequent cleanups of historical data (any cleanup mode that runs the Delete cleanupHistoricalData command) or cleanups of the database (any cleanup mode that runs the Delete cleanupDatabase command). Log messages from the last 4 cleanup runs of each object type are retained.

6> n-3 direct upgrade:

Support for direct upgrade from 8.6.x to 8.9 Service Pack 1 has been added to make upgrade process simpler and faster.

 

New Support:

Platform Support

Patching Support

Compliance Support

  • Red Hat 6-7.x support for zSeries
  • Red Hat 7.x Little Endian
  • SUSE 11 and 12 for zSeries
  • Windows 2016
  • VMware vCenter 6.5
  • Oracle Enterprise Linux 7.x
  • CentOS 7

 

  • IBM AIX 7.2
  • Red Hat 6-7.x for zSeries
  • SuSE 11 and 12 for zSeries
  • Microsoft Windows 2016
  • Oracle Enterprise Linux 7.x

 

  • CIS Microsoft Windows Server 2008 R2
  • CIS Red Hat Enterprise Linux 7
  • CIS Red Hat Enterprise Linux 6
  • DISA STIG for Windows 2016 (early drop released)

 

 

More details on What’s New here:

https://docs.bmc.com/docs/ServerAutomation/89/release-notes-and-notices/8-9-00-enhancements-and-updates/service-pack-1-version-8-9-01

Share:|

[now with updated compliance policy template]

 

So, CVE-2017-0144 https://nvd.nist.gov/vuln/detail/CVE-2017-0144, a vulnerability that was identified about two months ago (published Mar 16 2017), is now being widely exploited in the wild, most visibly impacting hospitals in the UK’s National Health Service to the point that they’ve had to redirect incoming patients to other facilities.

This vulnerability is addressed by Microsoft Bulletin MS17-010, which is also included in OS-specific Security Bulletin (roll-ups) SB17-002, SB17-003, SB17-004.  MS17-010 applies to Server 2003 and Server 2008, while SB17-002 applies to Server 2008 R2, SB17-003 applies to Server 2012 R2 and SB17-004 applies to Server 2012 (thanks to Joe Schuler)

 

Part of what makes the vulnerability so serious is that it doesn’t require direct action by the user, simply having the vulnerability and being on the same network as an infected host can expose your system to the ransomware.

 

Wana Decrypt0r screenshot.png

(source: Wikipedia)

 

 

The good news is that the required hotfix is easily identified and applied using either BMC Server Automation or SecOps Response Service.  We’ve shown three routes you can use to apply this fix: Patching Job, a Compliance Job, and Vulnerability Remediation (SecOps Response).

We'll shortly post a video for the Patching approach.

 

For the patching approach, first start by updating your Windows Patch Catalog if it hasn’t updated since at least May 13.  Microsoft has released updates for now-unsupported operating systems including Windows XP and Server 2003, that are included in this update.  To do this, right-click on your Windows Patch Catalog, and select “Update Catalog”

 

 

This task may take a few minutes to complete.

While that is running, you can create a Patch Smartgroup that identifies any “Windows Bulletin” where name equals “MS17-010”.  You may also want to include the OS-specific security bulletins SB17-002, SB17-003, SB17-004, which include MS17-010.

And once your catalog update completes, you can run a Patch Analysis (Patching Job) to identify which servers need this patch.  To do that, right-click on the new Patch Smart Group, and select “Analyze Patch(s)”:

 

This will create a Patching Job that focuses on this particular bulletin.

 

Note that we are analyzing in List mode, using the MS17-010 Patch Smartgroup we just created.  Click Next.

 

 

If you want to create remediation artifacts (packages and jobs to deploy the patches), click the checkbox at top.

 

 

You may need to select appropriate places to save the Packages and Batch/Deploy Jobs if you have not used this feature before, otherwise they will default to the same values used previously.  To have the remediation jobs execute either immediately or on a schedule for a later time, click on Deploy Job Options…”,

 

 

And select either “Execute job now” or select a time for “Execute selected phases” as appropriate.  Here I’ve set this up to start executing at 10PM, or “22:00”.

Click ok, and on the next screen, select the appropriate Server Smartgroup to audit, and click Next.

 

 

On the Notification page, I like to send notifications to either the client that owns these systems, or to myself, so I’ve put in my own email address, selected all three conditions: Success, Failed, Aborted, append results to email, and asked it to keep it under 1MB.  Click next.

Because I want these results right away, I click “Execute Job Now”, and click Finish:

 

 

I will shortly have patching results indicating where I have already applied, and where I still need to apply MS17-010.

Here I see that I have 4 servers that need this patch.

 

 

Since I selected to create the remediation objects, and to schedule them, they will run tonight at 10PM.

Once they've executed, we can look at the deploy status of these patches:

In this example, two servers did successfully remediate, but still need a reboot (this is another option in the job options, to reboot after deployment).  Some teams will avoid scheduling the reboot to allow them to deploy patches as fast as possible, then reboot when the maintenance window becomes available.

In this case, I'm going to sort by the servers that need the manual reboot, and reboot them directly from BSA by right-clicking on the green checkmarks, and selecting "reboot".  (There's also an NSH Script that will reboot servers at scale, and you can always right-click, advanced, Reboot").

Now they'll reboot, and come back up fully patched shortly:

 

To audit this using Compliance, simply build a rule that looks for whether MS17-010 is installed (This policy is attached as a version-neutral export).  Unfortunately, since there's no registry key to inspect for this bulletin, we can't use registry-keys to check.

Let's see how easy this is in SecOps Response first let's login to our tenant using BSA:

I imported my latest scan info, then went over to the Operator Dashboard.  Filter by "CVE-2017-0144", and it shows me exactly which systems have this vulnerability detected on, and that the oldest detection is 22 days old (and now in violation of SLA, being a critical vulnerability):

 

I scroll down and see all the systems that I can remediate.

Click remediate:

 

 

I'm going to deselect one server, but continue with the rest:

Select "Execute Now":

Select some notifications, then hit execute now.

 

Isn't that easy?

Share:|

We are happy to announce the availability of HotFix on top of 8.9GA release.

With this hotfix we can do patching and virtual provisioning of windows 2016 targets.

VC6.5 has been qualified.

 

Download the 8.9.00.HotFix1 from here (two zip files) Linux.zip and Windows.zip.

 

Applying 8.9.00 HotFix1 for Virtual Provisioning:

  1. Stop all the processes
  2. Extract linux.zip/Windows.zip (depending upon your AppServer OS Flavour) files to a temporary location on all appservers.
  3. Once we extract check we have all the below list of files:
    1. bladelogic_blvmware.jar
    2. bladelogic_infra.jar
    3. bladelogic_rcp-ui.jar
    4. bladelogic_virtualization.jar
    5. blncdll.dll (not applicable/available in linux.zip)
    6. libncdll.dll (not applicable/available in linux.zip)
    7. jpavmware.zip
  4. Make a backup of the original jars at same location (eg /original) on Application Servers(all) andRCP Consoles from the following location:
    1. Application Server:
      1. <appserver install location>/NSH/br/stdlib/bladelogic_blvmware.jar  to (eg /original/bladelogic_blvmware.jar)
      2. <appserver install location>/NSH/br/stdlib/bladelogic_infra.jar to (eg /original/bladelogic_infra.jar)
      3. <appserver install location>/NSH/br/stdlib/bladelogic_rcp-ui.jar to (eg /original/bladelogic_rcp-ui.jar)
      4. <appserver install location>/NSH/br/stdlib/bladelogic_virtualization.jar to (eg /original/bladelogic_virtualization.jar)
      5. <appserver install location>/NSH/bin/blncdll.dll  to (eg/original/blncdll.dll) (Not applicable to AppServers on Non-Windows OS)
      6. <appserver install location>/NSH/bin/libncdll.dll to (eg /original/libncdll.dll) (Not applicable to AppServers on Non-Windows OS)

          B. RCP Console:

      1. <RCP install location>/CM/rcp/plugins/com.bladelogic.client.jars_1.0.0/lib/bladelogic_blvmware.jar to (eg /original_rcp/bladelogic_blvmware.jar)

      2. <RCP Install Location>/NSH/br/stdlib/bladelogic_blvmware.jar to (eg /original_rcp/bladelogic_blvmware.jar)
      3. <RCP install location>/CM/rcp/plugins/com.bladelogic.client.jars_1.0.0/lib/bladelogic_infra.jar to (eg /original_rcp/bladelogic_infra.jar)
      4. <RCP Install Location>/NSH/br/stdlib/bladelogic_infra.jar to (eg /original_rcp/bladelogic_infra.jar)
      5. <RCP install location>/CM/rcp/plugins/com.bladelogic.client.jars_1.0.0/lib/bladelogic_rcp-ui.jar to (eg /original_rcp/bladelogic_rcp-ui.jar)
      6. <RCP Install Location>/NSH/br/stdlib/bladelogic_rcp-ui.jar to (eg /original_rcp/bladelogic_rcp-ui.jar)
      7. <RCP install location>/CM/rcp/plugins/com.bladelogic.client.jars_1.0.0/lib/bladelogic_virtualization.jar to (eg /original_rcp/bladelogic_virtualization.jar)
      8. <RCP Install Location>/NSH/br/stdlib/ bladelogic_virtualization.jar to (eg /original_rcp/bladelogic_virtualization.jar)
  1. Copy downloaded files to required Appserver & RCP Console locations and replace the existing files
    1. Application Server:
      1. <download location>bladelogic_blvmware.jar to <appserver install location>/NSH/br/stdlib/bladelogic_blvmware.jar

      2. <download location>/bladelogic_infra.jar to <appserver install location>/NSH/br/stdlib/bladelogic_infra.jar

      3. <download location>/bladelogic_rcp-ui.jar to <appserver install location>/NSH/br/stdlib/ bladelogic_rcp-ui.jar

      4. <download location>/bladelogic_virtualization.jar to <appserver install location>/NSH/br/stdlib/bladelogic_virtualization.jar

      5. <download location>/blncdll.dll to <appserver install location>/NSH/bin/blncdll.dll (Not applicable AppServers on Non-Windows OS)

      6. <download location>/libncdll.dll to <appserver install location>/NSH/bin/libncdll.dll (Not applicable AppServers on Non-Windows OS)

          B. RCP Console:

      1. <download location>bladelogic_blvmware.jar to <RCP install location>/CM/rcp/plugins/com.bladelogic.client.jars_1.0.0/lib/bladelogic_blvmware.jar

      2. <download location>bladelogic_blvmware.jar to <RCP Install Location>/NSH/br/stdlib/bladelogic_blvmware.jar
      3. <download location>/bladelogic_infra.jar to <RCP install location>/CM/rcp/plugins/com.bladelogic.client.jars_1.0.0/lib/bladelogic_infra.jar

      4. <download location>/bladelogic_infra.jar to <RCP Install Location>/NSH/br/stdlib/bladelogic_infra.jar
      5. <download location>/bladelogic_rcp-ui.jar to <RCP install location>/CM/rcp/plugins/com.bladelogic.client.jars_1.0.0/lib/bladelogic_rcp-ui.jar

      6. <download location>/bladelogic_rcp-ui.jar to <RCP Install Location>/NSH/br/stdlib/bladelogic_rcp-ui.jar
      7. <download location>/bladelogic_virtualization.jar to <RCP install location>/CM/rcp/plugins/com.bladelogic.client.jars_1.0.0/lib/bladelogic_virtualization.jar

      8. <download location>/bladelogic_virtualization.jar to <RCP Install Location>/NSH/br/stdlib/bladelogic_virtualization.jar
  1. Start the appserver, RSCD and all the bladelogic processess.
  2. Check the appserver.log and ensure all things are progressing.

 

Steps to enable Online Patch Catalog for Windows 2016:

1.    Go to PGC (Patch Global Configuration)

2.    Navigate to Windows Tab

3.    Navigate Windows Filter Configuration File and download Product Categories XML File

4.    Open Product Categories XML File in any text editor(Notepad++)

5.    Add Below content at the end of the file and before </windows_product_categories> tag.

               <product_category name="Microsoft Windows Server 2016" type="category">

<product name="WINDOWS SERVER 2016 DATACENTER">  </product>

<product name="WINDOWS SERVER 2016 STANDARD">  </product>

<product name="WINDOWS SERVER 2016 ESSENTIALS">  </product>

<product name="WINDOWS STORAGE SERVER 2016 STANDARD">  </product>

<product name="WINDOWS STORAGE SERVER 2016 WORKGROUP">  </product>

<product name="WINDOWS SERVER 2016 NANO">  </product>

</product_category>

6.    Save the file

7.    Upload the edited Product Categories XML File in Windows Filter Configuration under PGC in Windows Tab

8.    Click Apply

Now you will be able to see Microsoft Windows Server 2016 filter for windows catalog

 

Steps to enable Offline Patch Catalog for Windows 2016:

  1. Download All-OS-Patch-Downloaders-<OS TYPE>-build-8.9.00.tar.gz
  2. Extract zip file and go to lib folder of the extracted location
    1. ex: <Extracted Location>\All-OS-Patch-Downloaders-linux-build-8.9.00\All-OS-Patch-Downloaders-linux-build-8.9.00\libs
  3. Open Support-files-1.0.SNAPSHOT.jar using appropriate extract tool (winzip/winrar) and extract the file product_categories.xml to your local drive ex: D:\Installers\com\bmc\sa\patchfeed\windows
  4. Open product_categories.xml with a text editor and add the below content at the end of the file and before </windows_product_categories> tag.

            <product_category name="Microsoft Windows Server 2016" type="category">

<product name="WINDOWS SERVER 2016 DATACENTER">  </product>

<product name="WINDOWS SERVER 2016 STANDARD">  </product>

<product name="WINDOWS SERVER 2016 ESSENTIALS">  </product>

<product name="WINDOWS STORAGE SERVER 2016 STANDARD">  </product>

<product name="WINDOWS STORAGE SERVER 2016 WORKGROUP">  </product>

<product name="WINDOWS SERVER 2016 NANO">  </product>

</product_category>

     5. Save the file and add it back to jar file Support-files-1.0.SNAPSHOT.jar

     6. For detailed information on how to use Offline Patch Downloader please refer our docs wiki here.   

 

Steps to import new CO jpavmware.zip for VC6.5 supportability:-

  1. Login to BSA Console as an administrator (eg: BLAdmins role)
  2. Go to configuration -->Config Object Dictionary View
  3. Click on + button
  4. Under Specify Configuration Object, select Server Object and Click Next
  5. Browse to jpvmware.zip file which is extracted with this hotfix and click next and click finish. This will import the new CO.
  6. If VMWare vCenter Server is not configured in AMO mode then configure VMWare vCenter Server mode as given as mentioned here.
  7. To upgrade custom configuration objects follow steps from step 1 to 5 mentioned here
Share:|

BMC is coming to Boston!  Join technical experts and peers to learn about the latest BMC products and solutions that enable their innovation, their careers, and their companies. 

 

Attend the Datacenter Automation and Security track to hear the latest news.  We will talk about what is happening with our products, Security Operations and Service Process Orchestration. 

 

You will hear from Brian Alexander, Daniel Nelson, Dan Reed, and Neil Parisi

 

If you are interested in speaking and telling your Automation or IT Security story, please comment below.

 

Details for BMC Day

Date: May 9, 2017

Where: Hotel Commonwealth, 500 Commonwealth Avenue, Boston, MA 02215

When: 9:00am - 7:00pm

 

Register today!

 

 

North East User Group SecOps Response Service BladeLogic

 

Heather Tomlinson Ina Shpak

Share:|

By Yechezkel Schatz

 

GettyImages-81711999-1.jpgIn a new walkthrough, I demonstrate how to set up and use the patch analysis solution for Solaris 11 in BMC Server Automation.  You will learn:

  • How to set up or update a patch repository on an Oracle Solaris 11 IPS server
  • How to analyse your Solaris 11 systems for outdated IPS packages and deploy packages to the target servers where they were detected as outdated.

View: Walkthrough: Performing Solaris 11 patch analysis

 

I would like to hear your feedback on this walkthrough. Leave a comment or rate this blog post. Also, if you have other suggestions for walkthroughs, please leave them in the comment section.

Kim Wharton

BMC Day Chicago - March 2

Posted by Kim Wharton Employee Feb 23, 2017
Share:|

chicago.jpg

 

Next week (March 2), BMC is taking over the Loews Hotel in Chicago to talk about digital transformation at the BMC Day Chicago.  If you are in the MidWest (or feel like coming to Chicago), we would love to see you. 

 

During the event, you will learn, connect with others and also understand the latest trends and technologies.

 

In the Datacenter Automation, Cloud and Security track, we will have the following sessions:

  • Innovate, Automate, and Secure Your Digital Transformation
  • Secure Delivery - Ensuring Compliance and Control for Application Pipelines
  • Secure Operations - New Approaches to Secure Your Enterprise in the App Economy
  • Bridging the Gap with Service Process Orchestration

 

View the full agenda and abstracts.  We are track 4 on the agenda.

 

Register today to let us know you are joining us.

 

It will be a fun day - learning and also, visiting with our sponsors, and of course networking.  Come join us!

 

BladeLogic Discovery / ADDM SecOps Response Service Midwest User Group

Share:|

Well, as you can tell I am a little late in posting part 4 of our Hack and Defend video series.

 

Watch the final video in our series - Hacker Goes for the Crown Jewels

 

 

Other videos you may have missed:

Part 1 - Intro to Hack and Defend

Part 2 - Hacker Breaches the Perimeter

Part 3 - Hacker Goes for the Gooey Center

 

BladeLogic Discovery / ADDM Network Automation

Share:|

Welcome to Week 3 of our BladeLogic Threat Director video series.

 

Part 3 of the Hack and Defend video series focuses on hackers exposing more vulnerabilities than you were expecting.  Check out Hack and Defend: Hacker goes for the Gooey Center.

 

 

Other videos you may have missed:

Part 1 - Intro to Hack and Defend

Part 2 - Hacker Breaches the Perimeter

Part 4 - Hacker Goes for the Crown Jewels

 

Discovery / ADDM BladeLogic Network Automation

Share:|

Another week has gone by and we are heading in to an exciting weekend.  I am going to digress here - I live in Houston and Super Bowl 51 (American football) is coming here 5 Feb.  This weekend, they start all the fun activities - concerts, food trucks, NFL Experience, a NASA virtual reality ride (I will be in line for this), and so much more.

 

OK, back to work for now.

 

As we move in to week 2 of our video series, I want to share with you the next video: Hack and Defend - Hacker Breaches the Perimeter.   In this video, GuidePoint Security will show us how easy it is for hackers to download free tools to breach the enterprise.

 

 

 

 

Other videos you may have missed:

Part 1 - Intro to Hack and Defend

Part 3 - Hacker goes for the Gooey Center

Part 4 - Hacker goes for the Crown Jewels

 

Discovery / ADDM BladeLogic Network Automation

Share:|

This week I am beginning a 4 week video series showcasing BladeLogic Threat Director.

 

BladeLogic Threat Director provides IT operations and security teams the data they need to prioritize and remediate threats based on potential impact to the business.

 

In the video series, BMC partners with GuidePoint Security to share how hackers can get in to your systems and how you can prevent the vulnerabilities.

 

Part 1 is an introduction to our Hack and Defend demo.

 

 

 

Other videos:

Part 2 - Hacker Breaches the Perimeter

Part 3 - Hacker goes for the Gooey Center

Part 4 - Hacker goes for the Crown Jewels

 

 

 

Discovery / ADDM  BladeLogic Network Automation

Share:|

BMC Cover.jpg

 

 

Today, BMC and Forbes Insights release their 2nd Annual Security survey titled "Enterprises Re-Engineer Security in the Age of Digital Transformation".  The report is focused on how organizations are continuing to invest in protecting themselves and their customers from cyber attacks.

 

Hear from Bill Berutti, president of data center automation & security and compliance, on his viewpoints from the report and read the full report.

 

As Bill says, "the frequency and severity of attacks will continue to rise until something significant is done.  The time for action is now."

 

Upcoming webinar to hear BMC's thoughts on the 2nd Annual Security Survey:

Tuesday, February 21, 2017 - Join BMC and Forbes Insights to explore the latest thinking around security for public clouds, big data, mobile apps, and more.  The webinar will begin at 12:00pm CST/10:00am PST/1:00pm EST.  Register today.

 

Upcoming event:

March 2, 2017 - BMC Day Chicago, Chicago, IL - Learn, connect, and advance.  Join BMC to get insider access to BMC solutions and executives.  Register today.

 

 

BladeLogic, Discovery / ADDM

Share:|

By Dave Wicinas and Lisa Greene

 

Lisa Greene and I have produced two new videos that demonstrate how to use Threat Director to address vulnerabilities detected in a server environment. The videos are companions to existing walkthroughs that describe the same processes in more detail.

Walkthrough for Threat Director: Mapping vulnerability scan results to a server environment links to a video that shows how to map server assets in a scan report to servers managed with BMC Server Automation. The video also demonstrates how to map vulnerabilities in the scan to BladeLogic remediation content.

 

Walkthrough for Threat Director: Remediating server issues detected in a vulnerability scan links to another video that shows how to filter results on the Security and Operator Dashboards and launch remediation operations to correct vulnerabilities.

 

Let us know if the videos are helpful by rating or commenting on the blog post.

Filter Blog

By date:
By tag: