12 Replies Latest reply on Jan 20, 2016 3:01 PM by Hal DeVore

    Tagging all over the place!

    Daric Smith

      So, tags definitely need some serious rethinking.  I currently see two types of "tags":


      1. Tags that are applied to agents via installers or policies.

      2. Tags that can be applied from the Admin Console to almost anything under the General Administration tab.


      First: Tags are powerful.  They can be used for grouping, policies, permissions, event rules, views and probably many other things I haven't touched on yet.  They are pretty much the key to doing things to CIs.


      Unfortunately, v9.5 of ProactiveNet still appears to be in a hybrid state between the CMA and Admin Console:


      • Tags created in the CMA don't show up in the Admin console and tags in the Admin Console are not valid for use in policies.
      • There isn't really a "repository of tags" that are selectable via a drop-down.  You have to maintain a goofy spreadsheet so you know how things are tagged.
      • BMCs best practice is to apply tags to agent installers and use scripts to modify tags on host operating systems.  I think this is an older way to go.
      • You can also tag things within a CMA Policy based on agent selection criteria.  I believe this function should be improved upon greatly as it offers the potential for the most dynamic tagging capabilities.


      I believe (in v9.5's current state) the best practice is to have two primary types of policies: A host tagging and grouping policy followed by the monitoring policies (using precedence for one over the other).  I don't think tags should be applied directly to agent installers at all.


      The current v9.5 downside to this is that you still have to manually update Agent Selectors but hopefully there is a future integration with either ADDM Discovery or the CMDB that allows us to get dynamic data from those sources.  I've also found this is easier to maintain instead of working through a bunch Agent Installers.


      Once you have established your Tagging Policy, you now have the tags available to establish any monitoring policies.  And, to top it off, it is so easy to add/remove tags from the policy instead of working on the agent host itself running a silly script.


      The other aspect to tags is the automatic Grouping of CIs from a CMA policy along with assigning permissions to CIs from a CMA policy.  Both of these still need a little refining but that little Server Configuration screen in the CMA policy offers a ton of power for organizing your CIs in the Operations Console.


      Does anyone else have some experience with Tagging and Grouping CIs in v9.5?  How do you leverage the power of Tags without making it too manual?

        • 1. Re: Tagging all over the place!
          Daric Smith

          We improved on the original tagging process (as you'll see in the screenshot).  We now have 3 types of Policies: Tagging, CI Grouping/Permissions and Monitoring and I'll explain the reasons below:


          • Tagging Policy:
            • We decided to use "dumb" agents (The installer applies zero tags).  This forces us to leverage the Agent Configuration in CMA policies for tagging and makes grouping and monitoring a lot more dynamic instead of static configurations on the Patrol agent. 
            • The two down-sides for the tagging policy:
              • We have to maintain a static list of hosts (as mentioned above) but it is easier to maintain than agent installers or host-side tagging scripts and the efficiency is regained on the grouping and monitoring policies. 
              • The other caveat is only ONE tagging policy can be applied to a host as it is an agent-side policies and has an overwrite affect. 
            • Precedence note: All of the tagging policies have the exact same precedence.  Because tagging is agent-side, the settings would get overwritten and we only apply one host to one tagging policy, the precedence doesn't matter.
            • Description Note: I use the description field to list out all of the tags applied to the agent so you don't have to open the policy.
          • Grouping/Permissions Policy:
            • This policy leverages the Server Configuration only and allows for grouping of CIs together for viewing in the Operations Console: Using the tags from above, we could automatically group things like "Pilot SharePoint Farm0", or "Prod SQL", etc.
            • The Permissions functionality has potential but I think BMC will find it is a little buggy ( have a ticket open for them).  Ideally, entering a BMC Security group to that field automatically grants full control to those CIs and adds the group tree to their Operations Console.  Currently, I have it working for a couple policies but only partially (I get Full Control of the devices, but don't get the Group tree).
            • Precedence note: All of these have the same precedence as well.  Because this policy is a server configuration, the settings are cumulative: I can have 2 policies with the same host going to two different groups.
            • Name / Description Note: I use the name to show the operations console group tree and the description to list the BMC Security group that has permission to those objects.
          • Finally, you get the Monitoring Policies.
            • These are your standard monitoring policies and leverage the tags created above and get automatically applied once a host is added to a tagging policy.
            • Precedence note: We do establish precedence blocks (Windows, Linux, SQL, Apps, etc) to help track the type of policy applied.


          Below are a few screenshots showing what our Policy screen looks like in the CMA and what the Operations Console then looks like because of the policies.  Sorry for all the black-outs but hopefully it gives you a flavor.  PM me if you have questions on our naming conventions.


          • 2. Re: Re: Tagging all over the place!
            Jon Trotter

            Do you have any screenshots you can provide of where you enter the tag?  I'm working with this for 9.5 SP1 and when I have set the tag to be App:WinOS, the tag shows up as App, not WinOS.  I have tried "App:WinOS", App:"WinOS", only WinOS, "WinOS" (quotes are literal) and all of those have returned error messages when saving the policy, effectively not allowing the policy to be saved.  The screen hint states the format should be tag:tag description and notes that double quotes should be used for the description, assuming it means tag:"tag description".


            Pulling the config shows the entry to be /Tags/tag/App with a value of WinOS.


            "/AgentSetup/Identification/Tags/tag/App" = { REPLACE = "WinOS" }


            I'm new to policies via 9.5, but I'd like to utilize the functionality appropriately.

            • 3. Re: Re: Tagging all over the place!
              Jon Trotter

              Nevermind, I'm an idiot.  After reading and looking, I'm formatting mine wrong.  I still think the double quotes are not working, but going to test a new policy and see if that corrects it.  Maybe it's a bug that you can't edit a tag with double quotes or something. 

              • 4. Re: Re: Tagging all over the place!
                Daric Smith

                This is what I have found to be our best practice on tagging in its current state because the overall system needs to be cleaned up.


                • Only put ONE OS Specific tag in the Agent Installer.  ALL of the rest of our tagging is done through the CMA Policy.  This makes it so we only have a very small number of installer packages in our repository.  The only time we have a different installer is if we have a different Patrol Account (for DCs).  We also just put all of the KMs in the installer instead of doing a Patrol Agent installer then a KM installer.  The space difference is negligible but more importantly your installer packages are pretty clear-cut.


                • We initially used the description to add flavor text (WinOS:"Windows Operating System") But quickly realized it was tedious to maintain the descriptions and our tags are sort of self-explanatory.  So we just duplicate the tag in the description.
                • We still use a similar scheme for policies above and can't emphasize enough how important ONE Host to ONE Tagging Policy is.  But, we are liberal with our tags in the policy.  For example, our Pilot SQL Servers will have the tag list: eDelSQL, eDelPilotSQL (on top of the WinOS Policy applied from the agent installer).
                • Policy Automatic Grouping is probably the second most powerful side of ProactiveNet's configurations and efficiency.  There are so many things that leverage good grouping: Event Rules, Views, Reports, Filters, etc.  Because grouping is server-side you can be liberal with them.
                  • There are a couple annoying interface issues with grouping:
                    • The group created through a CMA Policy does NOT automatically add the devices.  You have to manually do that through the Admin Console.  I've raised a Community Idea for this.
                    • Do not use group trees.  For example, if I wanted my agents to be in a group tree "Corporate/Exchange/Pilot" and another group of agents in a group tree "Corporate/Exchange/Production", The Administration Console takes each level of the tree as a separate object.  So it looks great and clean in the Group view of the operations console, but in the Administration Console, Views, Reports, Event rules and anywhere else that uses groups, it doesn't maintain that nice looking tree.
                • In my top screenshot, I show that we have Tagging, Monitoring and Grouping Policies.  We actually separated things a little more by creating "Threshold:" policies.  Because of the interface limitations and as we've added more policies we found it is easier to separate configurations.  This way I don't have to dig through monitoring policies to find where I made this one threshold setting.


                There are other interface issues that need to be cleaned up, but once we got the relationships between our policies established, it has just become a bunch of mouse-clicks.  Here is our "quick" check list to get application systems fully monitored:


                1. Done in the CMA:
                  • Tag It
                  • Group It
                  • Monitor It
                  • Threshold It
                2. Done in the Administration Console
                  • Add Device(s) to Object Group(s).
                    1. All <Base_OS> Servers.
                    2. App-Specific Groups.
                  • User Group granted permission to object group(s).
                  • User group granted permission to individual objects:
                    1. Full Control: App Owners
                    2. Read Only: Interested Parties.
                3. Add Event Rules
                  • Basic Event Rule for critical events to cover system outages.
                  • Advanced Event Rule for Disk, CPU any other customizations.
                • 5. Re: Re: Tagging all over the place!
                  Jon Trotter

                  Daric, thanks for sharing the information!  We're ramping up with 9.5 and working to implement some new policies using tags, so this will help.  Need to digest what you've written and review the previous posts, but definitely some good information here.


                  Would you mind providing the link for the community idea?  I'd like to follow it and review any others that are out there.  I've noticed that some of the KM packages (if not all of them) except for the base installation do not allow providing a tag, which I feel would be more than helpful.  This, of course, is when building the package in the CMA package repository.  There may be reasons why they don't allow this or haven't configured the installer to allow for a tag per KM package in CMA, but the "work around" you mentioned by building a package all inclusive would obviously suffice for a base installation.


                  Thanks again for sharing how you have been using this!

                  • 6. Re: Tagging all over the place!
                    Jon Trotter

                    Just wanted to share since I have been pining over this until I remembered (re-read what Daric wrote) the option to set variables in CMA.   I was trying to understand the best way to implement tags as needed without managing a large customized repository.  Just wanted to share that I have a monitoring policy in place to set a tag based on the server type since we have our servers named this way, i.e., servers that have SQL have this designated in the name of the server, Oracle ORA and so on. The monitoring policy does nothing more than set the tag based on whatever criteria we set.  The order is set lower than the monitoring policy so it will be applied first, then the monitoring policy based on the tag we just set.


                    I feel setting tags in general should be a staging step where we can set a tag in one policy and set another tag in another policy for location and assign the PATROL agent to an Integration Service.  Really I think the staging policies should allow for either directing the agent to the same staging Integration Service or none at all, just so it can apply whatever tags are desired as many times as are required, if that makes sense.  Anyways, just thought I would share.

                    • 7. Re: Re: Re: Tagging all over the place!
                      Jon Trotter
                      The group created through a CMA Policy does NOT automatically add the devices.  You have to manually do that through the Admin Console.  I've raised a Community Idea for this.

                      Daric, can you expound on devices not being added in the group when using policies?  I think I understand what you're saying, but I see the devices in the BPPM admin console (local admin console, not CMA) I'm testing with that have a policy applied to them.  Granted, this is only a few managed systems, but they is there and the objects they are collecting data against per the monitoring policy seem to be there.  I did notice that when I was doing my testing and upgrading the agents to 9.5, I failed to include the 4.7.00 version for the Windows Server KM.  Once that was implemented, the three agents I'm testing showed in the respective consoles and showed that the policies were applied correctly in CMA.  Maybe I'm misunderstanding what you are saying.

                      • 8. Re: Re: Re: Tagging all over the place!
                        Daric Smith

                        Yes, comparing your pictures to mine, the missing information is an object called "Device".  By not having that, any external events that come in won't be viewed in that group unless you manually add the "Device' from the Admin Console.  This involves, clicking the computer object on the left and simply clicking "Add".  You will then see a "Device" object in your list:


                        By doing that, we now have an extra tab show up for the groups called "Devices" when viewing a group in the Operations Console


                        That does not show up unless you manually add it in the Admin Console:


                        The real problem is that External Events assigned to a "Device" won't show up for that group.  So you may look at all of your groups that are Green, but there is an external event indicating a problem.

                        • 9. Re: Re: Tagging all over the place!
                          Damien Poudret


                          double quotes seem to be working if tag description actually contains spaces. If there is not space, the tag format is reported as invalid. Strange, though.



                          • 10. Re: Tagging all over the place!
                            Hal DeVore

                            Whenever I hit something in CMA that's not working as expected, I go dig thru your posts, Daric Smith.


                            Trying to beat Groups into submission now.  This is another one where I'm coming to the conclusion that the old tools are still the place to do this.


                            Doing it by policy results in incomplete groups.  Missing the device entry as you note and authorizing CIs seems to just not work.  Which means that creating groups gets split across two tools/places: policy plus hand work in the Admin Console.

                            Monitor Group creation in the Admin Console is pretty simple.  User Group, ditto although having to authorize CIs seems redundant given that they're in a Monitor Group.



                            • 11. Re: Tagging all over the place!
                              Daric Smith

                              This is true using two tools but:


                              1. Admin Console does not leverage tags so dynamically adding monitors to the group then adding the device is easier.

                              2. A vast majority of our events are intelligent so are monitor based.  The only reason the "device" is needed is because of: self-monitoring disconnect events, event log events and log file monitoring events.  This makes it much more dynamic than thick client groups with tags.

                              3. For some reason BMC still hasn't fixed the dissociation of devices and monitors in TrueSight v10.1  I have been bringing it up for the last two years.  If they ever fix such a simple design flaw this conversation would be moot.


                              I really wish BMC would realize the power they have in tags but it is a slow boat to turn.

                              • 12. Re: Tagging all over the place!
                                Hal DeVore

                                For our situation

                                • Groups (monitor groups, user groups, authorization groups) are used to give application owners a view of "their" servers.
                                • Tags are used to select specific monitoring


                                The servers/objects selected for each case isn't the same since not all servers for a given application get the same monitoring (aside from the common base OS monitoring).


                                So both those things come down to building or selecting a list of server names.