This document contains official content from the BMC Software Knowledge Base. It is automatically updated when the knowledge article is modified.
TrueSight Capacity Optimization
Capacity Optimization 11.5.01
TrueSight Capacity Optimization 11.5, 11.3.01
After upgrading to TSCO 11.5.01 the Automatic Forecast Service (AFS) log starts to repeat the following message after about 6 hours:
The message repeats until the TSCO Datahub where the AFS is running is restarted without the "last activity date" ever changing.
On AFS startup the log reports:
WARN - Could not identify exactly AFS status during startup. Consider it enabled.
INFO - AFS START process.
<-- no further messages until the repeating BCO_AFS_FAIL102 message -->
Environment recently upgraded from TSCO 11.0 or earlier to TSCO 11.5
One possible source of this behavior is a cleanup of old Indicators step that is performed by the TSCO Automatic Forecast Service (AFS) on startup. This cleanup will restructure the old AFS records into their new table location in TSCO 11.5. In some environments this SQL may have experience performance issues that cause the AFS to hang on startup and never be able to begin processing INDICATORS.
SolutionThe AFS hang problem during the old indicator cleanup is being tracked via Defect DRCOZ-23375 and a fix will be available in TSCO 11.5.01 CHF (11.5.01.000.C00004) and later to avoid the AFS thread handing when the cleanOldIndicators phase is executed.
WorkaroundHere is a potential workaround for the AFS cleanOldIndicators workflow hanging:
(1) Set the 'afs.cleanOldIndicators' property value to 'false'.
-- Check if the afs.cleanOldIndicators value is currently set:
-- If not set:
-- If set:
(2) Restart the TSCO Datahub
(3) Wait 30 minutes or so
(4) Run 'kill -3 [PID of datahub]'
Note : kill -3 does not terminate the process. It is a thread dump that will list all the Java threads that are currently active in Java Virtual Machine (JVM).
(5) Capture the logGrabber output from the TSCO Datahub AS.
If this workaround addresses the AFS hang problem the AFS can be run with the 'afs.cleanOldIndicators = false' configuration as long as necessary.
There was a change in TSCO 11.3.01 which moved the Days Idle Indicator metric from the DETAIL to HOUR level. So, for TSCO 11.3.01 and later it is good to have the afs.cleanOldIndicators run once successfully to cleanup the old data but once that cleanup is complete the task is done and doesn't need to be run again. Without the cleanup being run it just means some old indicators will remain at the DETAIL level so the impact is just a small amount of database space consumption (so some old rows that could be cleaned up will continue to live in the database without any other visible symptoms of the afs.cleanOldIndicators functionality being disabled.
If the errors are not resolved using above steps, collect the following information and reach out to customer engineering :
- Log Grabber output
- Oracle AWR report
- Diagnostic Data flow report