Overview of the nightly TSCO Gateway Manager/BPA console data processing process activity

Version 5
    Share This:

    This document contains official content from the BMC Software Knowledge Base. It is automatically updated when the knowledge article is modified.


    PRODUCT:

    TrueSight Capacity Optimization


    COMPONENT:

    Capacity Optimization


    APPLIES TO:

    TrueSight Capacity Optimization (Gateway Server) 10.7, 10.5, 10.3, 10.0 ; BMC Performance Assurance 9.5



    PROBLEM:

     

    Overview of the nightly TrueSight Capacity Optimization (TSCO) Gateway Manager or BMC Performance Assurance (BPA) console data processing process activity
     

     


    SOLUTION:

     

    Legacy ID:KA424847

      


    In general the most active locations on the file system will be:

      
       
    • During nightly data transfer (usually 00:04 through ~01:00 or later): The Console Data Repository directory (where the UDR data resides on the console)
    •  
    • During nightly data processing (usually 00:15 through ~04:00 or later): The /[Manager Output Directory] and UDR Console Data Repository
    •  
    • Throughout the day: The $BEST1_HOME/local/manager/status, $BEST1_HOME/local/manager/log directories
      


    Note that the TSCO Agent/BPA agent will also be running on the console and it will be accessing logs in the following directory:
    $BEST1_HOME/bgs/log

    Here is an overview of the nightly TSCO Gateway Server BPA console data processing process activity.

    == Every 1 minute ==

    Every minute throughout the day the $BEST1_HOME/bgs/bin/pcron command is executed by cron. It determines which TCCO Gateway Server/BPA commands need to be executed that minute and executes them as necessary.

    Output files:

      
       
    • Log file: $BEST1_HOME/bgs/log/pcron/[hostname]-[user].log
      

    Input/Output Files:

      
       
    • $BEST1_HOME/bgs/pcron/repository/[hostname]-[user].pcron
    •  
    • $BEST1_HOME/bgs/pcron/repository/[hostname]-[user].counter
      


    == 23:30 or 23:45 (vcmds:COLLECT_LEAD_TIME minutes before vcmds:DAILY_BEGIN) ==

    The /[Manager Output Directory]/[begin]-[end].Manager script is executed by pcron. It calls the $BEST1_HOME/bgs/scripts/best1manager script which in turn calls the $BEST1_HOME/bgs/bin/best1manager.awk script.

    The TSCO Application Server Gateway Manager default COLLECT_LEAD_TIME value is 30 minutes (so Manager scripts are scheduled to run at 23:30 by default).  In earlier releases of the product (where runs were started via the TSCO Gateway Server X-Windows console) the default COLLECT_LEAD_TIME was 15 minutes so runs were scheduled at 23:45 to issue data collection requests starting at 00:00.

    When using AutoNodeDiscovery the first step of the Manager script execution uses the $BEST1_HOME/bgs/bin/AutoNodeDiscovery command to refresh the domain (*.dmn) file (which defines which computers will be managed by this run) based upon the master computer list source file defined for the run.

    Input files:

      
       
    • The run's *.vcmds file
    •  
    • The run's *.dmn file
      


    Output files:

      
       
    • Log file: /[Manager Output Directory]/best1manager.log
    •  
    • Created: /[Manager Output Directory]/[date]-[date].ProcessDay
    •  
    • Created: /[Manager Output Directory]/[date]-[date].Collect
    •  
    • Created: /[Manager Output Directory]/[date]-[date].XferData
      


    Executed:

      
       
    • $BEST1_HOME/bgs/bin/udrCollectMgr process which sends collection request to all of the remote agents (throttled based upon the $BEST1_HOME/local/setup/collectManager.cfg COLREQ_CONCURRENT_RUN and COLREQ_CONCURRENT_GLOBAL parameters -- which, by default, allow 10 concurrent start collection request threads at a time).
      

    The udrCollectMgr process would then continue to run for the rest of the day (during the whole collect and transfer period of the run).

    udrCollectMgr files:

      
       
    • Log file: $BEST1_HOME/local/manager/log/[hostname]-[RunName]-MM-DD-YYYY-############.log
    •  
    • $BEST1_HOME/local/manager/run/[RunName]-MM-DD-YYYY-############.mrc
    •  
    • $BEST1_HOME/local/manager/run/[hostname]-MRQ
    •  
    • Semaphore: $BEST1_HOME/local/manager/run/colreq_concurrent_global.key_##
      


    == Every 15 minutes throughout the day ==

    Every 15 minutes the udrCollectMgr will query the status of each remote agents (throttled based upon the $BEST1_HOME/local/setup/collectManager.cfg COLREQ_CONCURRENT_RUN and COLREQ_CONCURRENT_GLOBAL parameters -- which, by default, allow 10 concurrent start collection request threads at a time) and update the UCM Status Reports (if vcmds:COLLECT_STATUS_REPORTING set to YES).

    Output files:

      
       
    • $BEST1_HOME/local/manager/status/ UCMStat-[RunName]-MM-DD-YYYY-############.xml
    •  
    • $BEST1_HOME/local/manager/status/ UCMStat-MM-DD-YYYY.xml
      


    == 23:58 (and every 15 minutes afterwards at 13, 28, 43, and 58 minutes past the hour for the next 24 hours) ==

    The /[Manager Output Directory]/[date]-[date].Collect script is executed.

    The [date]-[date].Collect script executes a udrCollectMgr process for the run. The udrCollectMgr process checks the $BEST1_HOME/local/manager/run/[hostname]-MRQ and if there is already a udrCollectMgr process running for the run it exits. If there isn't one currently running the newly executed udrCollectMgr process takes ownership of the run.

    The following steps happen 24 hours after the start of the collection for the day (these are the TRANSFER and PROCESSING stages of the run):

    == [Next day] 00:04 (collectManager.cfg:COLMGR_TRANSFER_DELAY minutes after vcmds:DAILY_END) ==

      

    The running udrCollectMgr process transitions from COLLECT mode to TRANSFER mode and begins initiating data transfer request to the remote agents (throttled based upon the $BEST1_HOME/local/setup/collectManager.cfg TRANSFER_CONCURRENT_RUN and TRANSFER_CONCURRENT_GLOBAL parameters -- which, by default, allow 10 concurrent transfer request threads at a time).

    == [Next day] 00:09 (vcmds:DELAY_RUN_BY minutes after vcmds:DAILY_END) ==

    The /[Manager Output Directory]/[date]-[date].XferData script is executed by pcron.  The [date]-[date].XferData script executes a udrCollectMgr process for the run.  The udrCollectMgr process checks the $BEST1_HOME/local/manager/run/[hostname]-MRQ and checks the status of the run.  If the run status indicates that all of the computers in the run have successfully transferred or the run is beyond the vcmds:TRANSFER_DURATION time then the [date]-[date].XferData script executes the [date]-[date].ProcessDay script to begin processing.  If not, the [date]-[date].XferData script goes into a 15 minute wait cycle waiting for one of those conditions to become true.

    Output files:

      
       
    •   Log file: /[Manager Output Directory]/[date]-[date].manager.log
      


    == [Next day] After the transfer period (So usually between 00:15 and 01:15 or later) ==

    The /[Manager Output Directory]/[date]-[date].ProcessDay is executed by the [date]-[date].XferData script.  This is the script that manages data processing in the environment.

    Output files:

      
       
    •   Log file: /[Manager Output Directory]/[date]-[date].ProcessDay.out
    •  
    •   Many other files in the /[Manager Output Directory]
      


    The [date]-[date].ProcessDay script manages data processing.  That means that for each vcmds:VISUAL_INTERVAL of the day it is going to do the following:
       (1)    Execute a $BEST1_HOME/bgs/bin/bgsanalyze process that will read the UDR data for the computers defined in the domain and create a model (*.md) file and Visualizer (*.vis) file that contains the processing results
       (2)    [If vcmds:RUN_PREDICT  set to YES]: Execute a $BEST1_HOME/bgs/bin/bgsbest1 (Predict) process that will read the model (*.md) file and generate a Predict Visualizer (*.vis) file.

    Then once all of the intervals have been processed it will:
       (3)    Run the $BEST1_HOME/bgs/bin/combine.pl script that will combine the individual interval Visualizer files into a single combined Visualizer file (with a name like MDDh[domain].vis).
       (4)    [If vcmds:POPULATE set to YES]: Run the $BEST1_HOME/bgs/scripts/mpopulate.sh which will populate the Visualizer file into the defined Oracle database
       (5)    [If vcmds:BEGIN_CLEANUP_OUTPUT set to YES]: Clean up the temporary files created during the Manager execution (based upon file extensions selected in the vcmds:FILETYPES list)
       (6)    [If vcmds:BEGIN_DATA_COMPRESSION set to YES]: Find and compress the UDR data older than vcmds:BEGIN_DATA_COMPRESSION -> DAYSOLD
       (7)    [If vcmds: BEGIN_DATA_DELETION set to YES]: Find and delete UDR data older than vcmds: BEGIN_DATA_DELETION -> DAYSOLD
    NOTE:
    Point 7 is applicable till to 10.5.00 GW version only.

      
      New in 10.7.00  we are using "collectManager.cfg" file for UDR DATA   RETENTION ,   This setting specifies the number of days to keep UDR data for all the repositories in the console.
    # This value should be considerable greater then the delete and compression intervals specified in the manager scripts.
    # The primary purpose is to delete nodes that are no longer collected and will never be deleted by any manager runs
    # (Default : 0 (Not Deleted))
     
      UDR_GLOBAL_DATA_RETENTION = 30
      


       (8)    Run $BEST1_HOME/bgs/bin/managerExceptions.pl at the end of the run to generate the Manager Exception files in the $BEST1_HOME/local/manager/status/MM-DD-YYYY directory.

      

     

      
    Related Products:  
       
    1. TrueSight Capacity Optimization
    2.  
    3. BMC Performance Assurance for Servers

     


    Article Number:

    000095622


    Article Type:

    Solutions to a Product Problem



      Looking for additional information?    Search BMC Support  or  Browse Knowledge Articles