13 Replies Latest reply on Jan 15, 2019 1:37 PM by Stephane Guedon

    Create a dynamic blackout policy by command?

    Pablo Lobarede

      Hi guys!

       

      I need to create a dynamic blackout policy but by command. And then import a .csv file with the devices that I need to leave in maintenance.

       

      I saw the MPOSTER command but I can not create the policy or load data.

       

      ######command######

      mposter -n <name_cell> -q -d -a DDBL_MATCH_TABLE -b "input_match=['*','*','<mc_host>','*','*','*'];tag='BT_BlackOut_PortalGestion';active_timeframes=[timeframe_test];ref_instances_classes=[EVENT]"

       

       

      I appreciate your help.

        • 1. Re: Create a dynamic blackout policy by command?
          Brendan Murray

          Hi Pablo,

           

          Based on your example command, it looks like you have already seen this post by Daniel Vallejo on BMC Communities. The mposter command you show in your example will only create one record in the DDBL_MATCH_TABLE that will black out all events for a device whose host name is 'mc_host'. It won't even do that until you create a dynamic policy that uses the table.

           

          Unfortunately, you can't load a .CSV file directly into a table with mposter. We provide a utility called mcsvtodde that allows you to load .CSV files for dynamic enrichment policies. Unfortunately, it does not work for dynamic blackout policies. The post I refer to above describes a workaround that involves writing the output of mcsvtodde to a file and then replacing the references to DDEN_MATCH_TABLE with DDBL_MATCH_TABLE. You can then use the file as input to the mposter command and that will populate your dynamic blackout policy data table.

           

          That only solves part of your problem, though. You still need to create a a dynamic blackout policy, a selector and a time frame. This can also be done with the mposter command, although I am not sure why you would want to do this. Normally the goal is to create one (or a few) dynamic blackout policies and then use the DDBL_MATCH_TABLE to control the details of which events or devices get blacked out. I can see why you might want to create time frames from the command line, but you should try to use the DDBL_MATCH_TABLE to control the other blackout parameters as much as possible. That said, I will show you how to create everything from the command line: the policy, the selector and the time frame(s).

           

          The easiest way to do this is to create a dynamic blackout policy in the Administration Console first, export the data to files, modify the files to your needs and then mposter the new policy data back into the cell. You don't have to use BAROC files as the inputs to your mposter command. You can include the slot values in the command line if you prefer. My examples use files. I also provide one example of how to create a dynamic policy completely from the command line, without using an input file.

           

          Here are the steps:

           

          1. Create a dynamic blackout policy and an associated selector and time frame in the TSIM Administration Console. I am going to assume you already know how to do this.

           

          2. Using the mquery command, export the policy, selector and time frame data to baroc files so you can see how the data is represented internally. Note that my example below uses a Suppression Policy, rather than a Dynamic Blackout policy. The principle is the same, regardless of policy type.

           

          mquery -n <your TSIM cell name> -d -a IM_POLICY -w "name: equals 'My_Suppression_Policy'" -f BAROC -q > policy.baroc

          mquery -n <your TSIM cell name> -d -a SELECTOR -w "name: equals 'My_Selector'" -f BAROC -q > selector.baroc

          mquery -n <your TSIM cell name> -d -a TIME_FRAME -w "name: equals 'My_Timeframe'" -f BAROC -q > timeframe.baroc

           

          I recommend you exclude the cell-managed slots in your query output.  The commands above give you all slots for the data record. You want to exclude the slots in red below and export only the minimum slots required to create the policy. The cell will create the other slots automatically when you mposter the new data back in.

           

          IM_SUPPRESSION_POLICY;

          data_handle=18;

          mc_udid='mc.plsasevm16.e3319dc.17';

          mc_creation_time=1311971804;

          mc_modification_time=1311971804;

          mc_modification_requestor='';

          publish_env_id='';

          name=My_Suppression_Policy;

          description='Suppress unwanted events';

          enabled=0;

          active_timeframes=[My_Timeframe];

          except_timeframes=[];

          active_global_timeframes=[];

          except_global_timeframes=[];

          selector_name=My_Selector;

          selector_class=EVENT;

          selector_ecf='*Test_Events ($EV)';

          ordinal=0;

          during_schedule=YES;

          END

           

          Here are some example mquery commands with the exclusion argument:

           

          mquery -n <your TSIM cell name> -d -a IM_POLICY -w "name: equals 'My_Suppression_Policy'" -x "data_handle, mc_udid, mc_creation_time, mc_modification_time, mc_modification_requestor, publish_env_id" -f BAROC -q > policy.baroc

           

          mquery -n <your TSIM cell name> -d -a SELECTOR -w "name: equals 'My_Selector'" -x "data_handle, mc_udid, mc_creation_time, mc_modification_time, mc_modification_requestor, publish_env_id" -f BAROC -q > selector.baroc

           

          mquery -n <your TSIM cell name> -d -a TIME_FRAME -w "name: equals 'My_Timeframe'" -x "data_handle, mc_udid, mc_creation_time, mc_modification_time, mc_modification_requestor, publish_env_id" -f BAROC -q > timeframe.baroc

           

          3. Edit the policy, selector and time frame BAROC files to define your new dynamic blackout policy. You do not need to assign values to every slot. You may need to experiment to identify which slots are required and which are optional.

           

          4. Upload the contents of the files to your TSIM cell with the mposter command:

           

          mposter -n <your TSIM cell name> -d timeframe.baroc

          mposter -n <your TSIM cell name> -d selector.baroc

          mposter -n <your TSIM cell name> -d policy.baroc

           

          NOTE: The order in which you upload the files is important. The policy records contain references to the time frames and selectors. You need to upload the time frames and selector first so they exist in the cell when you upload the policy, otherwise you will get errors.

           

          5. (Optional) Once you are satisfied that you are able to create new policies, selectors and time frames using BAROC files as inputs to mposter, try including the policy details in the command line. Here is an example. (Note that I have not tested this particular command line example, but it should work.):

           

          mposter -n <your TSIM cell name> -d -a IM_POLICY -b "name='My_Suppression_Policy';description='Suppress unwanted events';enabled=0;active_timeframes=[My_Timeframe];selector_name='My_Selector';selector_class=EVENT;selector_ecf='*Test_Events ($EV)';ordinal=0;during_schedule=YES"

          2 of 2 people found this helpful
          • 2. Re: Create a dynamic blackout policy by command?
            Pablo Lobarede

            Hi Brendan, thank you very much for your help! I can already create the policies, time frame and selector by command.

            Now I need to be able to import the devices from the .csv

            I explain what I need:

            From REMEDY CHANGE 8.5, they enter a maintenance request for various devices where they define:

            * Start time and end maintenance

            * Designated user

            * And .csv with the devices / nodes

            Currently the designated users read that information and manually blackout the devices one by one.
            I want to automate and when they enter REMEDY the change request sends the .csv file to the TSIM server and runs a .bat with the commands to create the policy, selector and time (according to the time and data that comes in the. csv).

            It's possible?

            Thank you very much

            • 3. Re: Create a dynamic blackout policy by command?
              Brendan Murray

              Hi Pablo,

               

              You didn't mention in your original post that you were using Remedy Change Management. Are you aware that there is already a standard integration between Remedy Change Management and BPPM/TSIM that is designed to do exactly what you are trying to do? Here is a link to the documentation that explains what the integration does. This link will take you to the list of product versions for which this integration is supported against TSIM 11. (Note that there is no mention of Remedy Change Management 8.5. I can find no record of an 8.5 version of the Remedy ITSM Suite or Remedy Change. Are you sure your version number is correct?) The change management integration is available in previous versions of BPPM/TSIM back to BPPM 9.5.

               

              If I were you, I would first investigate whether the standard, supported integration between Remedy Change Management and BPPM/TSIM will work for your requirement. I am pretty sure it will not be able to use your .CSV file. I believe you will have to link the change request and the related maintenance window to all the CIs (devices) for which you want to black out events. I am not a Remedy expert, so you will need to consult your Remedy support team to determine if this is possible.

               

              If you are not able to use the standard, fully-supported integration and you want to proceed with your original approach, I would offer you the following guidance:

               

              1. As I mention in my initial response to your post, you should not need to constantly create new dynamic policies and selectors. We call them 'dynamic' for a reason. Their behavior changes based on the contents of an associated data table. The specifics of which devices get blacked out during which time frames should be managed in the data (.CSV), not in the policy. You should be able to create a single, permanent dynamic blackout policy whose selector is very general. The details of the blackouts are controlled by updating the data file and, if necessary, the time frames. You should not need to change the policy, or constantly create new policies.

               

              2. Yes, you can run a .BAT file or a .CMD file (on Windows) to automatically load the .CSV data into the Dynamic Blackout data table. However, you will need to use the workaround I mentioned in my initial response. It is described in this Communities post. It uses the mcsvtodde utility that comes with BPPM/TSIM to convert the .CSV data into a format that can be understood by mposter and write it to a file. You will then need to run the Windows command file provided by Bhuvaneswaran Ekanathan, or some other similar tool to modify the output of mcsvtodde to replace all references to DDEN_MATCH_TABLE with DDBL_MATCH_TABLE. This step is required because mcsvtodde is designed to update Dynamic Enrichment Policies, not Dynamic Blackout Policies. Unfortunately, the table name is hard-coded in mcsvtodde.

               

              Review the documentation for mcsvtodde to see examples of the syntax. In particular, take note of the -r argument. It allows you to replace the existing contents of the data table rather than adding the new data.

               

              Once you have replaced the table name in the mcsvtodde data, you can run mposter to load the data into the cell:

               

              mposter -n <your TSIM cell name> -d mcsvtodde_output_file.baroc.

               

              Again, my advice is to try to use the standard, supported integration before spending the time and effort to build (and support) your custom solution.

               

              Regards,

               

              Brendan

              2 of 2 people found this helpful
              • 4. Re: Create a dynamic blackout policy by command?
                Pablo Lobarede

                Hi Brendan!

                 

                I am using mposter to create the policy of Dynamic Blackout from Linux but I get an error and can not create it but if I run it in windows it works perfect. Do you have any idea ? Thanks

                 

                Command used in Linux:

                /bmc/bin/mposter -n @10.10.10.11/2020#mc -d -a IM_DDBL_POLICY -b "name=TestBlackout;description='';enabled=1;active_timeframes=[Prueba];selector_name='Blackout_Dinamico';selector_class='EVENT';selector_ecf='*Blackout_Dinamico ($EV)';lookup_table_tag=Prueba;match_fields=[ipaddress];data_file_separator=';';last_import_type=replace;"

                 

                Error:

                [root@rsso bin]# /bmc/bin/mposter -n @10.10.10.11/2020#mc -d -a 'IM_DDBL_POLICY' -b "name=TestBlackout;description='';enabled=1;active_timeframes=[Prueba];selector_name='Blackout_Dinamico';selector_class='EVENT';selector_ecf='*Blackout_Dinamico ($EV)';lookup_table_tag=Prueba;match_fields=[ipaddress];data_file_separator=';';last_import_type=replace;"

                BMC Impact mposter 9.6.00.06 (Build 241242464 - 12-Dec-2014) [l2]

                Copyright 1998-2014 BMC Software, Inc. as an unpublished work.  All rights reserved.

                20180710 134119.743785 mposter: CONFIG: BMC-IMC023005E {/etc/mclient.conf}

                20180710 134119.743878 mposter: TRACE: BMC-IMC021003W {/etc/locale/mclient.load}

                20180710 134119.743895 mposter: TRACE: BMC-IMC021004W {/etc/mclient.trace}

                20180710 134119.743914 mposter: CONFIG: BMC-IMC023005E {/etc/mclient.conf}

                20180710 134119.743926 mposter: TRACE: BMC-IMC021003W {/etc/locale/mclient.load}

                20180710 134119.743935 mposter: TRACE: BMC-IMC021004W {/etc/mclient.trace}

                20180710 134119.744442 mposter: SVRDIR: BMC-IMC051001E {/etc/mcell.dir}

                20180710 134119.961823 mposter: SERVICE: BMC-IMC050051W {#1.1,BMC-IMC080012E {BMC-IMC032091W {1254735,,BMC-IMC031001E {1,22,BMC-IMC031142E {$}},IM_DDBL_POLICY;

                        mc_udid='';

                        mc_bad_slot_names=[];

                        mc_bad_slot_values=[];

                        name=TestBlackout;

                        description='';

                        enabled=1;

                        active_timeframes=[Prueba];

                        except_timeframes=[];

                        active_global_timeframes=[];

                        except_global_timeframes=[];

                        selector_name=Blackout_Dinamico;

                        selector_class=EVENT;

                        selector_ecf='*Blackout_Dinamico ()';

                        ordinal=0;

                        during_schedule=YES;

                        execution_order=1;

                        event_class='';

                        lookup_table='';

                        lookup_table_tag=Prueba;

                        match_fields=[ipaddress];

                        trace=NO;

                        data_file='';

                        data_file_separator=';';

                        pmep_file='';

                        last_import_time='';

                        last_import_type=replace;

                        num_of_total_records=0;

                        num_of_failed_records=0;

                END}}}

                20180710 134119.961950 mposter: MCLI: BMC-IMC300016E {BMC-IMC080012E {BMC-IMC032091W {1254735,,BMC-IMC031001E {1,22,BMC-IMC031142E {$}},IM_DDBL_POLICY;

                        mc_udid='';

                        mc_bad_slot_names=[];

                        mc_bad_slot_values=[];

                        name=TestBlackout;

                        description='';

                        enabled=1;

                        active_timeframes=[Prueba];

                        except_timeframes=[];

                        active_global_timeframes=[];

                        except_global_timeframes=[];

                        selector_name=Blackout_Dinamico;

                        selector_class=EVENT;

                        selector_ecf='*Blackout_Dinamico ()';

                        ordinal=0;

                        during_schedule=YES;

                        execution_order=1;

                        event_class='';

                        lookup_table='';

                        lookup_table_tag=Prueba;

                        match_fields=[ipaddress];

                        trace=NO;

                        data_file='';

                        data_file_separator=';';

                        pmep_file='';

                        last_import_time='';

                        last_import_type=replace;

                        num_of_total_records=0;

                        num_of_failed_records=0;

                END}}}

                20180710 134119.962109 mposter: MPOSTER: BMC-IMC332034E {dnew + IM_DDBL_POLICY;name=TestBlackout;description='';enabled=1;active_timeframes=[Prueba];selector_name='Blackout_Dinamico';selector_class='EVENT';selector_ecf='*Blackout_Dinamico ()';lookup_table_tag=Prueba;match_fields=[ipaddress];data_file_separator=';';last_import_type=replace;END}

                BMC-IMC332034E {dnew + IM_DDBL_POLICY;name=TestBlackout;description='';enabled=1;active_timeframes=[Prueba];selector_name='Blackout_Dinamico';selector_class='EVENT';selector_ecf='*Blackout_Dinamico ()';lookup_table_tag='Prueba';match_fields=[ipaddress];data_file_separator=';';last_import_time='';last_import_type=replace;END}

                [root@rsso bin]#

                 

                • 5. Re: Create a dynamic blackout policy by command?
                  Alvaro Paronuzzi

                  Hi Pablo Lobarede

                   

                  could you please provide an high level description of your scenario?

                   

                  In my use case I will receive events from a third-party tool which will contain the details of the blackout timeframe so I should try to create dynamically the timeframe and the blackout policy starting from the events slot values.

                   

                  I am trying to understand:

                  1. Whether it is possible to do it with a reasonable effort

                  2. If there is any point of contact with the scenario you had to manage

                   

                  Thank you in advance for your help.

                   

                  Regards,

                  Alvaro

                  • 6. Re: Create a dynamic blackout policy by command?
                    Garland Smith

                    I think what you might consider is to create an MRL rule that does the equivalent of a blackout policy using a dynamic data table.  Maybe someone who has attempted something like this could provide the basic steps and possibly a prototype for the data table and corresponding rule.

                     

                    Garland Smith

                    • 7. Re: Create a dynamic blackout policy by command?
                      Stephane Guedon

                      Hi Alvaro

                       

                      Quite agree with Garland... You may need to write your own rules (mrl).

                      You can also use ProactivePack for Event Management with Blackout Module.

                       

                      Check ProactivePack web site (www.proactivepack.com) or videos here

                       

                      Note that all you can do useing web interfce can be also done using CLI / REST APIs

                       

                       

                      Regards

                      1 of 1 people found this helpful
                      • 8. Re: Create a dynamic blackout policy by command?
                        Brendan Murray

                        Hi Alvaro,

                         

                        It is definitely possible to do what you describe. In fact, the ability to create blackouts from events was added to TrueSight Infrastructure Management several releases ago (I am not sure exactly when). There is now an event class called BLACKOUT_EV and a set of rules that will take blackout settings from event slots and turn them into a blackout policy. However, there are some limits to the out-of-the-box solution. For example, your BLACKOUT_EV event must use either an existing event selector or an entire event class as the scope of the blackout. If you want the blackout to apply to just one device, or a set of devices in a data table, you will need to create your own rules. If you want to create the selector and the time frame dynamically using events, you will also have to write your own rules. If you are using a recent version of TrueSight Infrastructure Management, you can use the out-of-the-box rules as a template for creating your own solution.

                         

                        Here is a link to the documentation for Managing blackouts using an event.

                         

                        Regards,

                         

                        Brendan

                        • 9. Re: Create a dynamic blackout policy by command?
                          Stephane Guedon

                          Hi Alvaro,

                           

                          I've just cheked. This feature is already present in 10.7.

                           

                          Used rule is blackout_conf.mrl one

                          • 10. Re: Create a dynamic blackout policy by command?
                            Alvaro Paronuzzi

                            Hi Brendan, Stephane,

                            Thank you for your input.

                            I am currently on BPPM 9.0 but soon on TSIM 10.7. According to the information you shared, I can try writing my own rules taking as an example the 10.7 ootb rule.

                            My main concern is that the requirement for this mechanism is not the device or the class but the host+metric combination.

                             

                            In other words this is an example of how it should work:

                             

                            Event 1 on host 1 for metric 1 --> Blackout 1

                            Event 2 on host 2 for metric 1 --> Blackout 2

                            Event 3 on host 3 for metric 2 --> Blackout 3

                             

                            Potentially I may need to create a different blackout for any host/metric couple.

                            If you have any suggestion or concern about this approach please feel free to share it.

                             

                            Thank you in advance for your help.

                            Alvaro

                            • 11. Re: Create a dynamic blackout policy by command?
                              Brendan Murray

                              Hi Alvaro,

                               

                              There are many ways you could approach this.  A lot depends on the number and frequency of the blackouts you need to manage. You could try to dynamically create and delete selectors, time frames and blackout policies from MRL rules, however it might be easier to create a custom table-driven approach using a dynamic data table.

                               

                              Here's how it could work:

                               

                              1. Each record in the table would have two slots: hostname and metric name. You would write a rule that would select a broad class of events, so that Event 1, Event 2 and Event 3 would all trigger the rule to scan the table looking for matching hostname and metric name for the current event.

                               

                              2. If a matching record is found in the table, the rule would put the event in BLACKOUT status. In other words, the mere presence of a matching record in the table causes the event to be blacked out.

                               

                              3. You would use Timer Rules to create and delete records in the table when blackouts start and end.

                               

                              4. You would need an MRL rule that would process "create blackout" events containing the blackout parameters: hostname, metric name, blackout start time and blackout end time. The rule would add the hostname-metric name combination to the table and start a timer that, when it expires, deletes the record from the data table and updates the status of the events that were blacked out based on the record.

                               

                              NOTE: If you use the approach I describe above, you should put an index on the table for best performance.  Also, I should make it clear that I have not tested this approach. My advice is purely theoretical.

                               

                              Regards,

                               

                              Brendan

                              1 of 1 people found this helpful
                              • 12. Re: Create a dynamic blackout policy by command?
                                Alvaro Paronuzzi

                                Hi Brendan,

                                 

                                Thank you for your inputs.

                                In your opinion, does TSIM 10.7 blackout_conf.mrl file contain everything needed to create my own set of rules for the creation of selector, policy and timer for BPPM 9.0 that will meet my specific requirement? I mean: is it a good example from which I could start for managing my use case or it would be better to start  from scratch?

                                In addition, as per my understanding, I will have to create and manage the dynamic table which will be populated with the host/metric of the incoming event (I will check how to do it, but any example is highly appreciated).

                                 

                                Thank you,

                                Alvaro

                                • 13. Re: Create a dynamic blackout policy by command?
                                  Stephane Guedon

                                  Hi Alvaro

                                   

                                  I don't know how fluent you are with the 'cell' but here are some basics:

                                  1) Events are defined by Classes with heritage.

                                  2) Rules apply on events by phase (11 phases) and can apply on specific class or at another level (example: PATROL_EV for PATROL only or EVENT that include PATROL_EV, SNMP .... and so on.)

                                  3) Rules can use tables (usual name: DDA) in order to perform the job

                                   

                                  In you case, the blackout_conf.mrl use defined classes in classes/im_policies.baroc and blackout_conf.baroc

                                   

                                  For example, you may check the " BMC Knowledge Base Development Reference Guide" documentation