1 2 Previous Next 17 Replies Latest reply on Apr 19, 2012 8:38 AM by Sebastian C.

    cleanupDatabase does not work

      Hi,

       

      I try to cleanup our Database, but this doesn't work. It makes no difference if I execute it as an nsh script job, or "locally" from one of the AppServers (blcli -v defaultProfile Delete cleanupDatanbase). Other commands like for example "Server listAllServers" working without problems. Is there a major difference between these two commands, or what could be the problem? It seems that we have some VMWare based performance problems in our env.

      Can this cause such problems?

       

      Thanks,

      Sebastian

       

      P.S.: We're using 8.1SP3 and we've got 4 AppServers in our env which are running on RHEL 5.x. DB is Oracle 10g on HP-Ux.

        • 1. cleanupDatabase does not work

          Sorry, I just realized that it's not clear what happens when I execute the command. The output looks like this:

           

          # blcli -v defaultProfile Delete cleanupDatabase

           

          Available Roles are :

          -----------------------

          1 ) RBACAdmins

          2 ) GlobalReportAdmins

          3 ) GlobalReportViewers

          4 ) BLAdmins

          5 ) GLOB_Win_Patch_Primary_Contact

          6 ) AMER_Unix_Regional_System_Admin

          7 ) AMER_Win_Regional_System_Admin

          8 ) AMER_Win_Software_Installer

          9 ) GLOB_BL_Operator

          10 ) GLOB_Unix_Software_ESD_Tester

          11 ) GLOB_Unix_Patch_Primary_Contact

          12 ) GLOB_Win_Software_Release_Manager

          13 ) APAC_Win_Software_Installer

          14 ) EMEA_Win_Regional_System_Admin

          15 ) GLOB_Win_Software_Packaging

          16 ) AMER_Unix_Software_Installer

          17 ) GLOB_BLAdmins

          18 ) GLOB_RBAC_RoleAdmins

          19 ) APAC_Unix_Software_Installer

          20 ) EMEA_Unix_Regional_System_Admin

          21 ) EMEA_Unix_Software_Installer

          22 ) EMEA_Win_Software_Installer

          23 ) GLOB_RBACAdmins

          24 ) GLOB_RBAC_UserAdmins

          25 ) GLOB_Unix_Patch_Catalog_Admin

          26 ) GLOB_Unix_Software_Packaging

          27 ) GLOB_Win_Patch_Catalog_Admin

          28 ) GLOB_Add_Server

          29 ) GLOB_Win_Software_ESD_Tester

          30 ) APAC_Unix_Regional_System_Admin

          31 ) GLOB_Unix_Software_Release_Manager

          32 ) APAC_Win_Regional_System_Admin

          Enter role number : 4

           

          and that's it. There's no more activity.

          • 2. cleanupDatabase does not work

            The reason you cant run this command is when you have to acquire credentials to do so. run blcred and specify the role and run the command, it should work.

            refer to the blcli documentation for the complete syntax of the command.

            • 3. cleanupDatabase does not work

              Check blcli.log

              You can also increase the debug level in the file called blcli.cf ( in your /NSH/br install directory or/and in your home directory $HOME on Unix and %APPDATA%/BladeLogic on windows)

              • 4. cleanupDatabase does not work

                No, no - I've got a vaild session credential. As you can see in my second posting, I was asked as which role I want to execute the command.

                • 5. cleanupDatabase does not work

                  I changed the log level to DEBUG and this is the output when the command starts:

                   

                  19 Apr 2012 12:48:50,111 INFO [MESSAGE] Starting CLI session for single command [/MESSAGE]

                  19 Apr 2012 12:48:50,216 DEBUG [MESSAGE] Creating instance of bean 'ucsIsUCSProvisioningAbstractCommandInfo' [/MESSAGE]

                  19 Apr 2012 12:48:50,216 DEBUG [MESSAGE] Creating instance of bean 'ucsUcsServerBeanHelper' [/MESSAGE]

                  19 Apr 2012 12:48:50,217 DEBUG [MESSAGE] Finished creating instance of bean 'ucsUcsServerBeanHelper' [/MESSAGE]

                  19 Apr 2012 12:48:50,217 DEBUG [MESSAGE] Finished creating instance of bean 'ucsIsUCSProvisioningAbstractCommandInfo' [/MESSAGE]

                  19 Apr 2012 12:48:50,217 DEBUG [MESSAGE] Creating instance of bean 'provisioningIsProvisionCommandInfo' [/MESSAGE]

                  19 Apr 2012 12:48:50,217 DEBUG [MESSAGE] Creating instance of bean 'provisioningProvisionHelperBean' [/MESSAGE]

                  19 Apr 2012 12:48:50,217 DEBUG [MESSAGE] Finished creating instance of bean 'provisioningProvisionHelperBean' [/MESSAGE]

                  19 Apr 2012 12:48:50,217 DEBUG [MESSAGE] Finished creating instance of bean 'provisioningIsProvisionCommandInfo' [/MESSAGE]

                  19 Apr 2012 12:48:50,225 DEBUG [MESSAGE] Creating instance of bean 'ucsIsUCSProvisioningAbstractCommandInfo' [/MESSAGE]

                  19 Apr 2012 12:48:50,225 DEBUG [MESSAGE] Creating instance of bean 'ucsUcsServerBeanHelper' [/MESSAGE]

                  19 Apr 2012 12:48:50,225 DEBUG [MESSAGE] Finished creating instance of bean 'ucsUcsServerBeanHelper' [/MESSAGE]

                  19 Apr 2012 12:48:50,225 DEBUG [MESSAGE] Finished creating instance of bean 'ucsIsUCSProvisioningAbstractCommandInfo' [/MESSAGE]

                  19 Apr 2012 12:48:50,225 DEBUG [MESSAGE] Creating instance of bean 'provisioningIsProvisionCommandInfo' [/MESSAGE]

                  19 Apr 2012 12:48:50,225 DEBUG [MESSAGE] Creating instance of bean 'provisioningProvisionHelperBean' [/MESSAGE]

                  19 Apr 2012 12:48:50,225 DEBUG [MESSAGE] Finished creating instance of bean 'provisioningProvisionHelperBean' [/MESSAGE]

                  19 Apr 2012 12:48:50,225 DEBUG [MESSAGE] Finished creating instance of bean 'provisioningIsProvisionCommandInfo' [/MESSAGE]

                  19 Apr 2012 12:48:50,236 DEBUG [MESSAGE] Creating instance of bean 'ucsIsUCSProvisioningAbstractCommandInfo' [/MESSAGE]

                  19 Apr 2012 12:48:50,236 DEBUG [MESSAGE] Creating instance of bean 'ucsUcsServerBeanHelper' [/MESSAGE]

                  19 Apr 2012 12:48:50,236 DEBUG [MESSAGE] Finished creating instance of bean 'ucsUcsServerBeanHelper' [/MESSAGE]

                  19 Apr 2012 12:48:50,236 DEBUG [MESSAGE] Finished creating instance of bean 'ucsIsUCSProvisioningAbstractCommandInfo' [/MESSAGE]

                  19 Apr 2012 12:48:50,236 DEBUG [MESSAGE] Creating instance of bean 'provisioningIsProvisionCommandInfo' [/MESSAGE]

                  19 Apr 2012 12:48:50,236 DEBUG [MESSAGE] Creating instance of bean 'provisioningProvisionHelperBean' [/MESSAGE]

                  19 Apr 2012 12:48:50,236 DEBUG [MESSAGE] Finished creating instance of bean 'provisioningProvisionHelperBean' [/MESSAGE]

                  19 Apr 2012 12:48:50,237 DEBUG [MESSAGE] Finished creating instance of bean 'provisioningIsProvisionCommandInfo' [/MESSAGE]

                  19 Apr 2012 12:48:50,237 DEBUG [MESSAGE] Looking for method com.bladelogic.om.infra.app.api.DeleteServiceClientStubImpl.cleanupDatabase [/MESSAGE]

                  19 Apr 2012 12:48:50,239 DEBUG [MESSAGE] New request, id = 38 [/MESSAGE]

                  19 Apr 2012 12:48:50,239 DEBUG [MESSAGE] Sending request, id = 38 : cleanupDatabase():DeleteServiceClientStubImpl.java:659 [/MESSAGE]

                  19 Apr 2012 12:48:50,239 DEBUG [MESSAGE] Connection pool size : 1 [/MESSAGE]

                  19 Apr 2012 12:48:50,239 DEBUG [MESSAGE] Used connetions : 1 [/MESSAGE]

                  19 Apr 2012 12:48:50,239 DEBUG [MESSAGE] registering client handler for cid = 2002 object = com.bladelogic.om.infra.message.app.APIMessageRequest@10bcc8f4 [/MESSAGE]

                  19 Apr 2012 12:48:50,239 DEBUG [MESSAGE] to write (org)865 bytes [/MESSAGE]

                  19 Apr 2012 12:48:50,239 DEBUG [MESSAGE] reqID = 38 [/MESSAGE]

                  19 Apr 2012 12:48:50,239 DEBUG [MESSAGE] cid = 2002 [/MESSAGE]

                  19 Apr 2012 12:48:50,239 DEBUG [MESSAGE] dest =  [/MESSAGE]

                  19 Apr 2012 12:48:50,239 DEBUG [MESSAGE] Will send frame of size 865 [/MESSAGE]

                   

                   

                  And that's it. There's no more reaction.

                  • 6. cleanupDatabase does not work

                    Do you see anything in the appserver.log ?

                    • 7. cleanupDatabase does not work

                      Sorry, my bad. I had not seen the second post.


                      So you enter 4 as BLAdmins and then nothing happens? I believe this is where its captured:

                       

                      19 Apr 2012 12:48:50,237 DEBUG [MESSAGE] Looking for method com.bladelogic.om.infra.app.api.DeleteServiceClientStubImpl.cleanupDatabase [/MESSAGE]

                      19 Apr 2012 12:48:50,239 DEBUG [MESSAGE] New request, id = 38 [/MESSAGE]

                      19 Apr 2012 12:48:50,239 DEBUG [MESSAGE] Sending request, id = 38 : cleanupDatabase():DeleteServiceClientStubImpl.java:659 [/MESSAGE]

                      19 Apr 2012 12:48:50,239 DEBUG [MESSAGE] Connection pool size : 1 [/MESSAGE]

                      19 Apr 2012 12:48:50,239 DEBUG [MESSAGE] Used connetions : 1 [/MESSAGE]

                      19 Apr 2012 12:48:50,239 DEBUG [MESSAGE] registering client handler for cid = 2002 object = com.bladelogic.om.infra.message.app.APIMessageRequest@10bcc8f4 [/MESSAGE]

                      19 Apr 2012 12:48:50,239 DEBUG [MESSAGE] to write (org)865 bytes [/MESSAGE]

                      19 Apr 2012 12:48:50,239 DEBUG [MESSAGE] reqID = 38 [/MESSAGE]

                      19 Apr 2012 12:48:50,239 DEBUG [MESSAGE] cid = 2002 [/MESSAGE]

                      19 Apr 2012 12:48:50,239 DEBUG [MESSAGE] dest =  [/MESSAGE]

                      19 Apr 2012 12:48:50,239 DEBUG [MESSAGE] Will send frame of size 865 [/MESSAGE]

                      • 8. cleanupDatabase does not work

                        Also, have you setup and enabled the retention policy already before cleanup ?

                        You then need to execute the retention policy as well.

                        After this done, you should try the cleanup command

                        • 9. cleanupDatabase does not work

                          appserver.log:

                           

                          [19 Apr 2012 12:58:38,459] [Client-Connections-Thread-3] [INFO] [bmc:BLAdmins:168.124.226.192] [Client] User 'bmc' assumed the role 'BLAdmins'

                           

                          I couldn't find anything in the job instance log.

                          • 10. Re: cleanupDatabase does not work

                            You may want to check if you have correctly set the retention policy. I think its mentioned in the guide to do that.

                            • 11. cleanupDatabase does not work

                              from an article:

                               

                               

                               

                              Problem

                               

                               

                              The database stores all the information related to jobs (job runs, job run logs) as well as information about the servers (audit trail, for ex).

                               

                               

                              This can make the database grow significantly.

                               

                               

                              Using the retention policy utility lets you manage the amount of physical space the database requires and avoid potential performance issues resulting from your database getting too large.

                               

                               

                              procedure

                               

                               

                              This is a 6 steps process:

                               

                               

                              1) Run the following queries on the OM DB to set the default retention time for server audit trail and job run events.

                               

                               

                              Server audit trails is the information you see when you view the properties of a server and check the right tab.

                              Job run events are all the job run log lines you see when you view the job logs. This is usually the biggest table.

                               

                               

                              If you want to retain 2 months of information for both, use:

                               

                              UPDATE HK_CLEANUP_PROCEDURE SET DEFAULT_RETENTION_TIME ='60' WHERE name= 'AuditTrail';

                               

                              UPDATE HK_CLEANUP_PROCEDURE SET DEFAULT_RETENTION_TIME ='60' WHERE name='JobRunEvent';

                               

                              commit;

                               

                               

                               

                               

                               

                               

                              The default if you don't change these values is 14 days.

                               

                               

                              This needs to be changed directly in SQL because there is no hard relationship between Job Run Events and Job Runs in the current code. When Job Runs are marked for deletion, their job run events are not.

                              For Audit trail, there is simply nothing to delete these logs.

                               

                               

                               

                               

                              2) read page 184 of the 7.4.5 Admin guide "Marking Data for Deletion":

                              . enable the retention policy utility:

                              % blasadmin set cleanup EnableRetentionPolicy true

                               

                               

                               

                               

                              . set the autogenerated retention time in days:

                              % blasadmin set cleanup AutoGeneratedRetentionTime 60

                               

                               

                               

                               

                              restart the appserver

                               

                               

                              3) set the RESULT_RETENTION_TIME (in days) property on:

                              - the JOB class in the dictionary

                              - any specific JOB type in the dictionary

                              - any specific jobs properties

                               

                               

                              4) execute the retention policy:

                               

                               

                              % blcli Delete executeRetentionPolicy

                               

                               

                              This will mark all objects older than the values you specify for the retention time for deletion.

                               

                               

                              5) read page 154 of the 7.4.5 installation guide for configuring the update_delete_dependencies.nsh script.

                               

                               

                              update_delete_dependencies.nsh

                              This script checks if deleted objects have any dependencies. If a deleted object has dependencies, the script marks the dependent objects for deletion. For example, it marks the components of a deleted template for deletion. Running this script ensures that no deleted objects will be loaded into the reporting data warehouse the next time popuplate_reports.nsh is run.

                               

                               

                              Procedure

                              - 1. Using Configuration Manager, add the update_delete_dependencies.nsh script to the Depot as a Network Shell script. Assign any name to the script. When specifying the Script Type, choose the option that uses the runscript command. You do not have to set up any parameters when adding this script to the depot.

                              - 2. Create a Network Shell script job based on the script you added to the Depot in the previous step. The job should run against the reporting server—that is, the server where BladeLogic Reports is installed. This job should be run with the same permissions as the role that executes the clean-up utility. This job should be run between running the clean-up utility and a subsequent popuplate_reports.nsh script. See the BladeLogic Administration Guide for more information on running the clean-up utility.

                               

                               

                              Run that script.

                               

                               

                              6) run the database cleanup to permanently delete all the objects that were marked for deletion:

                               

                               

                              % blcli Delete cleanupDatabase

                               

                               

                              This is an example, you can modify based on what all do you want to cleanup.

                              • 12. cleanupDatabase does not work

                                Thanks for that article. I checked if the parameters are setup correctly in blasadmin, which looks okay. When I try to execute the command:

                                 

                                blcli Delete executeRetentionPolicy

                                 

                                it behaves excatly like the "cleanupDatabase" command, which means that it doesn't do anything. We've got a test environment, which is configured exactly the same way like our productive environment (on which this problem occures) and in this test env, the cleanupDatabase command runs perfectly.

                                • 13. cleanupDatabase does not work
                                  Bill Robinson

                                  update_delete_dependencies is not needed in 8.x.  the last version that was available was 7.6.

                                   

                                  if you have an oracle back end, can you run your cleanup command let it sit for 30 minutes or an hour, then have the DBA generate an AWR report from just before you started the command till the present.  There is something equivlant for mssql but i'm not as familar w/ it.  we need to see what's happening in the database when you run the cleanup command.  it's also possible now that you've maybe run this a few times that you've got some blocking sessions or some concurrent runs of the cleanup procedures that are causing problems w/ each other.

                                   

                                  what prabhat mentions is that there is a sequence of cleanup commands you should be running, not only the cleanupDatabase command.  I would not go changing the retention policies yet in HK_CLEANUP_PROCEDURE, and the cleanupDatabase does not touch the tables specified in the HK_CLEANUP_PROCEDURE anyway (cleanupHistorical does).

                                   

                                  there's another KA that mentions the sequnce of commands, so definately look for that.

                                   

                                  take the AWR and the blcli log and open a support ticket.

                                  • 14. Re: cleanupDatabase does not work
                                    Bill Robinson

                                    There was an issue w/ the cleanupRetentionPolicy that is fixed in SP4 where it would hang.  Getting an AWR when that runs would also be helpful.

                                     

                                    Also, just a note – just because you kill the blcli command does not mean the db procedures were killed…

                                    1 2 Previous Next