Skip navigation

Server Automation

11 Posts authored by: Ton Machielsen
Share:|

Goose: It's the bottom of the 9th, the score is tied. It's time for the big one.

Iceman: You up for this one, Maverick?

Maverick: Just a walk in the park, Kazansky

 

In my previous blog post i expressed my excitement about how easy it is to install BMC Server Automation 8.6 using the Unified Product Installer. In that blog post i called it the One-Click Installer, i was asked to call it by its official name, Unified Product Installer, i will try to do that from now on.

 

Today i am going to jot down my experiences upgrading one of my current 8.5 demo environments to 8.6 using the One-Click Unified Product Installer. I will not do anything to my environment except a database cleanup, which you need to do one in a while anyway, and a backup of my current environment in case something goes wrong. Ah, and i didn't RTFM either...

 

Let's go! I am using the same download as i did for the fresh install and click in the setup.exe which launches the setup.jar file.

 

12-20-2014 10-58-21 AM.png

The Unified Product Installer apparently detected that i already have a running BSA instance on my server, so it automatically goes into upgrade mode.

12-20-2014 10-59-22 AM.png

Next. Accept License agreement. Next.

The Unified Product Installer now asks for my BSA admin credentials because it wants to do an environment discovery.

12-20-2014 11-01-48 AM.png

This environment discovery is going to detect how many appservers you have running and where, where your fileserver resides, if they are up and running, which agent versions are running, database connections, etc. Everything needed to do the database, application server and agent upgrades.

12-20-2014 11-04-34 AM.png

Yes, i know, my environment is not huge, but it is what it is. One appserver and one fileserver. Both are up and running and agent versions are 8.5 SP1 and 8.5 SP1 Patch 1. Always keep your agents on the latest version, Ton!! Anyway, maybe this is a good opportunity to also fix that.

Obviously the BSA instance will be brought down during the upgrade. After all, we are replacing the agents, the appserver software, migrating the database, etc. Just to not make mistakes, the Installer asks us to agree upon the fact that the appserver will be down during upgrade.

12-20-2014 11-05-33 AM.png

And that's basically all. I cannot make it more complicated than that. Just press Install and the Unified Product Installer will do it's thing.

12-20-2014 11-09-02 AM.png

On my environment this "thing", the upgrade, took appx 20 minutes to complete. Don't be misguided by the progress bar in the screenshot above showing 79% done and still 17 minutes to go. We all know progress bars cheat on us.

So after 20 minutes i got the following screen and was ready to do the post-install activities like upgrading the consoles, agents on the targets, etc.

12-20-2014 11-31-13 AM.png

As i said, 20 minutes for an environment upgrade. The actual download of the installer takes longer than the upgrade itself. And everything in a controlled fashion. I will not forget the DB upgrade scripts anymore. I will not forget the upgrade the agent. I will not forget to run the post-upgrade scripts, etc.

 

After my positive experience with using the Unified Product Installer for a fresh install somehow i was not surprised that an environment upgrade using the same installer would also work fabulously.

Again i want to thank the various development teams for making this possible. I am managing quite a few demo environments and performing infrastructure upgrade is always something i look up to.

Well, not anymore.

Share:|

BladeLogic Server Automation 8.6 is out. And a lot of things have changed. I just downloaded the HUGE installer from EPD, installed it and started working with it. And i must say, i am particularly impressed with the new One-Click Installer, the Welcome screen and the now integrated Health and Value Dashboards.

For me as a person that builds demo environments, and i guess the same counts for people running PoC's or rolling out BSA at a customer, these three new features are making our life so much easier.

In the Pre-8.6 era, installation of BSA more or less was as follows:

  • Download several installers and scripts
  • Install an agent to the appserver machine
  • Create the OM database
  • Create the OM schema
  • Install the appserver
  • Create the fileserver
  • Install at least one RCP client
  • Install the Compliance Content
  • Install a couple of ZipKits
  • Install the Dashboards
  • Configure the Unified Agent Installer
  • Roll out agents to targets

A lot of steps. When you do it a lot you do it on the automatic pilot, but still...

 

Now, with the arrival of BSA 8.6, everything has changed. This is how it works now:

  • Make sure you have your database setup
  • Download the BSA 8.6 and agent installers. Yes it's big but hold on, the best is yet to come.
  • Run Setup.exe and follow the Wizard

12-12-2014 12.24.37 PM.png

  • Enter your database details

12-12-2014 12.27.29 PM.png

  • And click Install

12-12-2014 12.28.37 PM.png

 

That's all there is to it. Three steps! Now, according to the installer, the installation will take more or less one hour. On my modest test environment on the BDC Demo Cloud (Windows 2008R2, SQL Server 2008R2, 2CPU, 8Gb memory, all virtual) the complete installation took less than 30 minutes and this is what you get:

  • Application Server, File Server, PXE and TFTP Servers and RCP client installed and configured.
  • Compliance Content installed for all policies and platforms.
  • Sample ZipKits installed for application deployment, common operations and compliance on Windows and Linux.
  • Unified Agent Installer configured for all platforms. You will still need to configure the credentials, but you can do that from the Welcome page. I'll get to that later.
  • Health and Value Dashboards installed, configured and running.

 

And then there is the Welcome page.

12-12-2014 1.09.42 PM.png

It has never been so easy to start with BSA.

To enroll your servers? Just click the Enroll Servers link and import your CSV file. I use a CSV file i exported from my DNS server.

Deploy Agents? The Install RSCD Agents link launches a Wizard that configures the Unified Agent Installer and deploys your agents. I still remember when you had to go into Infrastructure Management and create Remote Host Authentications, etc. All gone. You can still do it that way, but i prefer the Quick Start wizard on the Welcome Page.

To run Compliance? Just click the Discover and Compliance links for the template you want.

Deploy Applications? The ZipKits are already loaded and linked on the Welcome Page.

Direct links to the Heath and value Dashboards.

 

And i can go on and on for Provisioning, Patching, etc. As late Michael Levey, host of the "Amazing Discoveries" infomercial series would say: "This is AMAZING!!"

 

I remember when i was running Proof of Concepts working for BladeLogic we used to have a test case where we stated that the product needed to be up and running in less than 2 hours. We did this because the compatition back then usually needed a couple of days and and a handful of engineers to make their product work. Now, with the One-Click Installer and the Welcome Page, we can change that test criteria to "The product needs to be up and running in less than 45 minutes".

 

Big, big kudos to the team that made this possible.

Share:|

Today i hit the so-many-st time where i needed to install the Bladelogic Compliance Content on a Linux appserver. Normally these Linux appservers are minimal servers, hence no graphical display is installed.

 

The Content installer is a graphical Java application so we need to do something here:

  • We have the option to install the X11 libraries on the appserver. Probably we don't want that, otherwise we would have done this already when we were installing the server.
  • We can install an X server on our local machine and redirect all X traffic from that machine to our local X server. This means that we need to installl and configure software like Xming on our Windows desktop, configure it, configure X forwarding on our SSH client and configure the correct $DISPLAY environment variable on our appserver. And then we cross our fingers. If all is configured correctly this setup should work. And i have to admit, this is the way we normally want to set this up. But there are way too many variables in this setup that can go wrong.
  • The simplest and fastest way to set up the Compliance Content on an appserver without X is to use the Silent Install option. And that's simpler than you thought.

 

Here is how it works:

First we need to write a response file that tells the installer what to install, where and which account to be used for the installation. This response file can look something like this:

 

-P installLocation=/opt/bmc/content

-J BLADELOGIC_PROFILES=defaultProfile

-J BLADELOGIC_PROFILES_TYPES=SRP

-J USER_PROFILE_NAME=defaultProfile

-J USER_ACCOUNT_NAME=BLAdmin

-J USER_ACCOUNT_PASSWORD=KVLXMUZWABUTZBAWMQZTMMEOBNAUVEZUNLBQBPUUZNWBNBEVQTTVPMLOELUQZVEL  (generate this encrypted password using the blenc utility)

-J USER_ROLE=BLAdmins

-J CONTENT_APPSERVER_LIST=appserver1

-J INSTALL_DISA_TEMPLATES=true

-J INSTALL_CIS_TEMPLATES=true

-J INSTALL_HIPAA_TEMPLATES=true

-J INSTALL_PCI_TEMPLATES=true

-J INSTALL_PCIv2_TEMPLATES=true

-J INSTALL_SOX_TEMPLATES=true

-J installs the full set of templates for a specific policy. If you don't want the full set of templates you can use the -A option instead. Example:

-A featureSoxAixTemplate

-A featureSoxLinuxTemplate

 

Let's store this file in the same directory where the content installer is and call it response.txt.

 

Next you launch the content installer using the following options: ./Content85-LIN.bin -i silent -DOPTIONS_FILE=response.txt

 

The installation will run for a while. Normally i open a second terminal windows where i run a tail -f /tmp/content_install_log.txt to see what the installer is doing and if there are errors or warnings.

 

I have used the Silent install a couple of times now and i found out that it's much faster and easier to set up than to fiddle around installing X libraries and trying to get my local and remote X server and X forwarding configuration right.

 

I hope this article will help people doing the right thing for their situation. More information on the silent content install and all the available options can be found in the Bladelogic Server Automation manuals on Docs.bmc.com.

Share:|

Hi all!

 

Here is a small video on how to deploy and configure the RSCD agent to listen on a non-standard port.

 

 

Let me know what you think,

 

Ton.

Share:|

This is a 5 minute video explaining how BladeLogic Portal leverages BladeLogic Server Automation's Role-Based Access Control to enable specific operations on a selected set of targets.

 

Meet Joe!

 

Share:|

Hi all!

 

I just uploaded another BladeLogic Portal video to the BMC Communities channel on YouTube. This time it's a video on how to create and execute NSH Script operations using BladeLogic Portal.

The video shows how to search and select an existing NSH Script Job, a simple netstat scripts, but it shows the functionality of Portal well, run it against multiple servers and how to read out the results of the script.

Please show this video to anyone who is interested in it and make the BladeLogic Portal wave big!

Share:|

This is the second video in a series of how-to videos which show how to perform the typical BladeLogic Server Automation use cases using the new BladeLogic Portal.

In this video i am showing how to perform a Windows 2008 Patch Analysis and how to deploy missing patches (remediate patch incompliance) using the Portal.

 

Share:|

This is a first of a series of videos on how to execute certain Server Automation use cases using Bladelogic Portal. In this video i am showing how to perform a PCI Compliance Audit against RedHat Linux server and how to remediate any compliance violations.

Further videos will be published as Portal develops.

 

Share:|

Today we are going to talk about the Configuration Object Dictionary and specifically about the Configuration Files section.

 

In almost every PoC or implementation we find that we need to add, modify or remove entries from a text file. Many times these are just standard files like /etc/hosts files or .ini files for which we have standard grammars, but what if we need to add new Configuration Files entries into the Configuration Object Database and we don't have the grammar? Then we can do two things:

1. Try to satisfy the customer with a presentation of the file using one of the standard grammars.

2. Write a new grammar.

 

We tend to go for option 1, but you know, writing a grammar is not as difficult as it looks.

 

So let's do it!

 

First let's take a look how grammar actually work by taking a very simple one like the generic.gm. Generic.gm basically parses a file line by line and displays it as Configuration File entries.

 

#

# generic.gm - generic line pass through grammar that allows

# parameter substitution.

#

# derek@bladelogic.com - 7/31/03

#

# Each line is parsed as one field

#

%%

COMMENT #.*

LINE .*

%%

comment COMMENT $save_as_comment $0

record LINE $new_field $0 $save_record $0 $set_add_rule $0

%%

add $write_field $0

 

 

So what do we see here?

 

Each grammar file contains four sections separated by the %% symbols:

  • Variable definitions
  • Token definitions
  • Rules
  • Add-Rules

 

Variables

Variables are simple to understand. Just like in any other programming language you specify a variable name and an assigned value. In this example we don't use variables. We will introduce them later.

 

Tokens

Tokens are defined by regular expressions. In our example the token COMMENT stands for everything that comes after a # and the LINE token contains everything that doesn't have a # in front of it.

These regex are very basic. In a further example we will use more complex regular expressions. Regular expressions look very cryptic, but they aren't. Learn them. You need them everywhere. When you are writing a shell or Perl scripts, when you write grammars in BSA, when you work with BNA, etc. etc.

If you need a good tutorial on regular expressions, look here: http://www.regular-expressions.info

 

Rules

Rules define if a line in a textile gets displayed or not. Rules have a name and contain Tokens, actions and arguments. In our example we have two rules called "comment" and "record". These two rule get checked in sequential order and if a match is found, the line will be displayed.

 

Let's look at the first rule called "comment". This rule says that if the COMMENT token exists, so if the regex returns a value, then save the token value ($0) as comment ($save_as_comment). So if you had comment lines (lines that start with a #, remember?) in the configuration file, BSA will save those.

 

The second rule called "record" says: If the LINE token exists then create a new field ($new_field) and give this field the value of token 0 ($0). Now this needs a bit of explanation. You can have multiple tokens in a rule and each of the tokens can have a value. Those values are number-indexed starting at 0. Think of it like an array of values. So the value assigned to the first token is value $0.

 

So again, the rule "record" says: If the token LINE exists (the regex return a value), then create a new field and give this field the value $0 of token LINE (LINE $new_field $0), then save record $0 and when you deploy this record, then apply add-rule $0 ($set_add_rule $0). As you can already see, we can also have multiple add-rules and add-rules are also indexed starting at $0.

 

Add-Rules

The add-rules are used to describe what BSA needs to do when a record of this type is deployed using a BSA Deploy Job.

In our example the add-rule is called "add" and on deployment of this object BSA will write field $0 into the configuration file.

 

As you can see, writing grammars is not as mysterious as you might think. As a good practice exercise you can take a look at the example in the https://docs.bmc.com/docs/display/NP/BladeLogic+Server+Automation+Grammars

 

I hope, the next time a customer wants you to manage a configuration file for which we don't have a grammar yet, you will write your own grammar and show your customer what BSA is capable of.

 

Ah, and once you have that grammar file, send it here so your colleagues will benefit from your efforts as well.

 

Have fun writing grammars!!

Share:|

When we go into a customer doing a pre-sales presentation or a PoC, a regular question comes up: "We currently have a whole range of scripts currently in production, can we re-use these scripts?" and the answer is "Yes, you can". And this is true, we can just copy the existing scripts into a BLPackage as external commands and 9 out of 10 times this will work.

But there is a huge disadvantage of doing this.

First of all there is the readability of the BLPackage. Usually only OS experts are able to understand what the commands in the scripts mean.

Then there is the matter of roll-back. Usually the script gets added as an external command, but the Undo Commands are never created. So what happens if the package get deployed to the wrong server? In the worst case you have to restore a backup.

 

So what do we do? While importing scripts in a BLPackage works we better "migrate" the script into a BLPackage. How do we do that? Here is an example. Let's take the following piece of a script that installs JBoss on a RedHat server:

 

;copy JBoss source files to target server

1. scp <jboss source dir> <target server>

 

;install JBoss

2. groupadd www

3. useradd -g www -d /opt/jboss jboss

4. chmod -R 755 /opt/jboss

5. cp -r /opt/jboss/* /opt/jboss

6. cd /opt/jboss/bin

7. sed 's/#JAVA_HOME="\/usr\/java\/jdk1.6.0"/JAVA_HOME="\/opt\/java\/jdk1.6.2"/g' run.conf >> run_variable.conf

8. chmod 755 /opt/jboss/bin/run_variable.conf

9. mv /opt/jboss/bin/run.conf /opt/jboss/bin/run.conf_original

10. cp /opt/jboss/bin/run_variable.conf /opt/jboss/bin/run.conf

11. cd /opt/jboss/server

12. rm -rf  web standard minimal all

 

<snip>

 

;add permissions to logs

32. mkdir /opt/jboss/logs

33. cd /opt/jboss/logs

34. touch nohup.out

35. chmod 755 /opt/jboss/logs/nohup.out

36. mkdir /opt/jboss/server/default/log

37. chmod 755 /opt/jboss/server/default/log/

 

<snip>

 

and the script goes on and on. A total of over 100 lines.

 

So how do we translate this into a package BSA-style? The first thing we need to do is to let go of the idea that BSA has to perform the actions that we want. This is script-thinking. What we want is to describe the desired state of our server after deployment of JBoss. And we don't really care what operations BSA uses to accomplish that.

 

Let's break the script up line-by-line.

 

2,3: Unix Users and Groups are native BSA objects, so we will create and entry in the BLPackage with the user and group.

user.jpg
group.jpg

1,4,5: Instead of copying the source files to temporary storage on the target server and then copying it to the right directory, we can include the files directly in the BLPackage AND with the right permissions.

dir.jpg

7-10: To manage configuration files we can create a Configuration File object in BSA's Configuration Object Dictionary.

cod.jpg

After creating this Configuration File in the Configuration Object Dictionary we can access and package any configuration parameter in this configuration file as an individual object.
conf file.jpg

cfinblp.jpg

11-12: As i explained before, BLPackages describe the desired state of a server. If, for some reason, the directories mentioned in line 12 of the script are present on the target server, you can explicitly tell BSA to remove those directories by selecting the Delete option for these directory objects.

fd.jpg

32-35 and 36,37: This can all be done by just adding the file nohup.out to the BLPackage with the correct path and permissions.

nohup.jpg

The script is much larger than this, but i think i made my point.

Using BSA's BLPackage technology i have been able to convert 37 lines (so far) of scripting into 6 BLPackage objects. I could have made it into 4 if i had included the nohup.out file and the /opt/jboss/server/default/log into the initial jboss file structure i created in step 2.

fullpack.jpg

Now the BLPackage is perfectly readable for anyone who has basic knowledge of BSA no matter if he or she is a Unix expert or not. Also we have perfect visibility on what will happen to the server when we deploy this package. Only native BSA objects are used, so automatically we have the ability to roll-back the deployment if needed.

 

So yes, we can leverage existing scripts in BSA, but think about the advantages of using native BSA technology.

Share:|

Last night, together with my 11-year old son and his best friend, i went to see a friendly soccer match of our favorite team. It was a beneficial match to help the poor children in the world. The theme was called "In our game nobody will be left off-side". We had good fun and at the end the organization collected 400.000 euros. You can imagine that we got home tired but in a very good mood.

 

I went to bed and then i had this dream…

I went back to the time when i was a Software Consultant running a Proof of Concept at a customer. Most of us have been there: You show up, install the software and then begin to work. But this time it was different. The software was different. We were doing things different. It was only a 2 day PoC where as normally we would do the same PoC in 5 days. What had changed?

We loaded up the software and started the DCA Portal. A nice looking interface from where all common actions like OS provisioning, software deployments, audit and compliance jobs were to be launched. Very nice, but that was not the most important difference. There was a link on the DCA portal that said: Up/Download Content. I click the link and a menu shows up with various items: OS Packages, Software Packages, Application Release, Regulatory Compliance and more. I was planning to load the standard Out-of-the-Box content, but i couldn't find it on EPD. Now, that's why! It's all online now!! So i click on Compliance Content and get a huge list of policies to select. Policies i am looking for, but also policies i have never heard of. I guess they must be used in other parts of the world.

At the top of the list there are three filter buttons: BMC provided, Partner provided and Community provided. Also there is a button with the text Upload. This means that i can upload the content i create this week and use it for my next PoC's. So that's how you perform 5 day PoC's in only 2 days!!

 

And then i had this dream…

I was an Account Manager creating an economical proposal for a new Cloud customer. The licensing part is easy, we have price lists for that. The hard part is the consulting Services part. Once the software is installed, what does the customer want us to do and how much do i charge for that? I had a list of activities the customer wants us to do, the content he wants us to create for his CLM project. Service Offerings. A huge list varying from simple Windows OS deployment to complex fully stacked virtual datacenters. How do i calculate all this?

But this time things were different. In my quoting tool i could select the Service Offering components i wanted and with every selection i make the tool tells me if this content is already available or if this content needs to be created. I can see that there is a lot of content available as Community provided content and that it takes 0 hours to build.

I don't know how we do this, but this certainly keeps the total price for the project down and makes it easier for the customer to decide in favor of BMC.

 

And then i had this dream…

I was a CS Project Manager building a planning for a new project a customer just signed for. This time i am short on resources, but what needs to be done needs to be done. The main part of the project is, again, creating content for this DCA project.

Focus for the customer is on rapid deployment of application updates while keeping their environment legally compliant.

They are in the grass-growing business and the grass-growing business has a regulation called TGGSCR (The Grass-Growing Society Compliance Regulation) which i have never heard from. Also they are looking for integration into the GGSDT (Grass-Growing Software Delivery Tool) which i have never heard from. How to quote this in time and effort if i don't even know what the regulation is about?

Having a look at other customers in the GG business, i have found that a project has been done in South America with a GG company and that the integration code and compliance content has been uploaded and made available on the BMC Content Repository.

Now, that saves me a bunch of headache! Project plan done, minimal resources required, price acceptable for the customer and time to deliver approximately 2 weeks!!

 

And then i had this dream…

I am a new System Automation Engineer in a company that has BMC Cloud Lifecycle Management as their main platform to deliver application infrastructure to our customers. We just had a new customer on board who i need to deliver new infrastructure for before the end of this week. This is a non-standard customer for us. Most of our customers are using WebSphere and Weblogic and this customer is using JBoss.

I just came back from the mandatory BMC CLM training, so i know how that works, but i have no clue about JBoss. What's is it? What do i need to get this deployed? And all this before the end of this week? That's not going to be easy!!

In the CLM training i learned that there is a Service Offering Download option which connects to the BMC Content Repository. Let's see if someone already has done something with JBoss. Maybe i'm lucky.

I open the CLM console, Click on Content Download, CLM, Application Stacks and yes, there is JBoss! I click on JBoss and select the "With Dependencies" checkbox and press Download.

Two minutes later JBoss shows up in my Available Service Offerings panel. WOW!! That saved me a lot of headache!! What do you mean "before the end of this week"? It's done!!

 

And then i had this dream…

I am a small software integrator and times are hard. Projects, if any at all, are under time and money pressure. I have my people sitting on a bench. What do i do?

There is a lot of buzz in the partner community, specially around some ecosystem BMC has created around their products. What's up with that?

Looking on the BMC Partner portal i see a link to the BMC Content Store where Certified Content Partners can sell content related to BMC products. So now i can dedicate my people with spare cycles to creating content for BMC products and upload it to the Store. And the good thing is, i only create the content once, upload it and after that moment every time my content gets downloaded i get a major piece of the money pie! Now, that's nice! Work once and collect many.

Now wait, there is this content rating system where users can give feedback on the quality of my content, so it'll better be good!

 

.... And then i woke up…

I took a shower, my breakfast and sat down at my laptop writing this blog post.

 

What if we could make this dream a reality?

Filter Blog

By date:
By tag: