6 Replies Latest reply on Aug 7, 2016 5:54 PM by Wayne Barnes

    Repository Cleaning

      Share This:

      Hello,

       

      IN our BCO 10.0 environment as we have the repository cleaner task working as it should be but I think we have a lot of data/files/reports that we can remove from the App/ETL  server which are old as old as 2013. Please let me know which could be removed or is there a best practice which I can use. And also the Plots directory under Repository folder, does that have files which cannot be removed at all?

       

      Regards,

      Rajith.K

        • 1. Re: Repository Cleaning

          Hi Rajith ,

           

          We have the same issue. The file system cleaner is set to keep plots for 1 day and statprod stat for 330 days. However both of these directories have files going back up to 2 years. I also am seeking an answer on how to effectively manage the purging of obsolete data in these two folders.

           

          Thanks

          Wayne

          • 2. Re: Repository Cleaning
            Stefano Omini

            Cleaning of BCO outputs is performed by a set of system tasks...

            • On database side, this is performed by Database Cleaner (for "audit-like" tables) and by DB Space Manager (for warehouse data).
            • On file system side, this is performed by FileSystem Cleaners (normally used for log folders and temporary user session items, including temporary chart images and dataset) and by Repository Cleaner task (which ensure consistency between configured analyses/models/reports and related objects on the repository).

             

            "Repository cleaner" system task is designed to cleanup objects which are no more referenced within the BCO console.

            For example:

            • reports results for a specific result date which have been overwritten
            • analyses charts which have been refreshed at a later time
            • objects referred to analyses/models/reports which have been removed from the console
            • and so on...

             

            The 1 day aging setting for plots, configured within file system cleaner, only applies to user session items.

            This is how my configuration (probably the default for this plots folder) appears. Note that the match is inverted (so everything which is NOT like PLOT* or PR_RUN* is considered.... in other words, user session folders):

             

            However, if you have a configured analysis (or report, or model, ...) whose last refresh occurred 1 year ago, you still need to keep images/files available on the repository, otherwise users would experience broken links or missing objects when browsing the console.

             

            Additional info available here:

             

            Hope this helped to clarify how cleaning is performed.

             

            Regards

            Stefano

            1 of 1 people found this helpful
            • 3. Re: Repository Cleaning
              Lorenzo Lissoni

              Hi Rajith and Wayne

               

              The Repository cleaner simply delete old files from the filesystem that are no more referenced from the console (e.g when you run an analysis twice, the application keep track of the last run plots, the first ones can be deleted by the repository cleaner, you get the same behavior when you manually delete an analysis).

               

              About the report aging please look at this page

              KnowledgeArticle - BMC

              to know how to configure the database cleaner to remove old results from the console, after that the repository cleaner will delete no more referenced reports. If you need more structured aging rules please ask the support.

               

              @Wayne: By default plot aging in the filesystem cleaner have the option Complement matching pattern for plots?= true. that means the task remove all except plots (that are managed by repository cleaner as told before)

               

              Hope it helps

              Best regards

              Lorenzo

              1 of 1 people found this helpful
              • 4. Re: Repository Cleaning

                Hi Stefani and Lorenzo,

                 

                Thanks for the clear descriptions.

                 

                However I am seeing the situation where we have a number of Golden models producing charts and stats that end up in the statprod and plots directories in the BCO Repository. The default value for the ${BASE_DIR}/repository/statprod - recurse into subdirs was false. I changed this to True as there a large number of statprod values in the subdirs that were significantly older than the aging value we had specified. This assisted in cleaning up the old items. A question here. As we run the golden models on a monthly basis and the last set of results effectively replace any previous runs as only the last run execution summary can be seen, how should the previous run results be cleaned up?

                 

                thanks

                Wayne

                • 5. Re: Repository Cleaning
                  Stefano Omini

                  Hi Wayne,

                   

                  You're right...

                  I checked on a 9.5 instance I'm currently working with, and I confirm that some old "statprod" results (even if the corresponding database rows for this "EST_OUTPUT_IMAGE" stats have been deleted from the related SYS_DATA_CUST_STAT table) still appear on the repository.

                   

                  This might suggest one of the following:

                  • "Repository cleaner" task is not perfectly working with golden models results (bug)
                  • Cleaning of golden model results is not supported by such task (design choice)

                   

                  Did you open any ticket to BMC support to check this?

                   

                  regards,

                  Stefano

                  • 6. Re: Repository Cleaning

                    Hi Stefano,

                     

                    I have not raised this with support yet. Only recently came to this conclusion also.

                     

                    Regards

                    Wayne