TrueSight Capacity Optimization (TSCO) -  Web failure due to, "BCO_WEB_ERR015: Console exhausted the connection pool to the database"

Version 2
    Share This:

    This document contains official content from the BMC Software Knowledge Base. It is automatically updated when the knowledge article is modified.


    PRODUCT:

    TrueSight Capacity Optimization


    COMPONENT:

    Capacity Optimization


    APPLIES TO:

    BMC Capacity Optimization



    PROBLEM:

     

    Problem symptoms include:
      (A) The TSCO Web UI is failing.  It may be totally inaccessible (giving a '404 not found' error) or may allow users to enter their password via RSSO but then fail on the redirect back to TSCO.
      (B) A Diagnostic Alert e-mail saying, "BCO_WEB_ERR015: Console exhausted the connection pool to the database" is being generated

    The Diagnostic Alert e-mail contains:

    BCO_WEB_ERR015: Console exhausted the connection pool to the database

    com.neptuny.cpit.entity.ExceptionContainer:
    com.neptuny.cpit.querymanager.QueryException - message: [com.neptuny.cpit.database.DatabaseConnectionManagerException] - Unable to borrow datasource connection:
    "Cannot get a connection, pool error Timeout waiting for idle object" - [borrowed=31]
    com.neptuny.cpit.entity.EntityException - message: SELECT_ALL_FAILED
    Class --> com.neptuny.cpit.entity.base.VHostTypes
    Attributes --> hostid=null;name=null;type=null;schedulerid=null;deploymenttaskid=null;schedulerstatus=null;

    at com.neptuny.cpit.entity.EntityDBHelper.selectAll(EntityDBHelper.java:258)
    at com.neptuny.cpit.entity.EntityDBHelper.selectAll(EntityDBHelper.java:210)
    at com.neptuny.cpit.entity.DatabaseFacade.getAll(DatabaseFacade.java:102)
    at com.neptuny.cpit.coreclient.CoreClient.getAll(CoreClient.java:1237)
    at com.neptuny.cpit.coreclient.CoreClient$$FastClassBySpringCGLIB$$4db26b8e.invoke(<generated>)
    at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:746)
    <-- cut a long backtrace -->

     

     


    SOLUTION:

     

    Typically the "BCO_WEB_ERR015: Console exhausted the connection pool to the database" error means that there is a problem on the TSCO database side.  The error may be associated with Oracle error messages being reported in the TSCO log files.  Often a database error affecting the TSCO Web component will also affect the TSCO Scheduler and there may be clearer error messages in the /opt/bmc/BCO/scheduler.log which may report "ORA-#####" errors that better indicate what is failing on the database side.

    If there is an Oracle error being generated in the logs the following KA describes common Oracle errors associated with the TSCO database instance and how to resolve them:
      000096715: Common Oracle errors in TSCO: how detect and debug (https://bmcsites.force.com/casemgmt/sc_KnowledgeArticle?sfdcid=000096715)

      

    Potential Workaround

       If the problem was transient (there was an environmental or database problem but it has been corrected, but the TSCO connection pool is still failing) restarting the TSCO components may work around the problem.

    Restart the TSCO processes on the TSCO Application Server:

      cd /[TSCO Installation Directory] # By default the /opt/bmc/BCO directory
      ./cpit restart all

    If the problem was a database side issue or environmental problem that has been fixed that will correct the problem.  If the problem still exists on the database side there should be a clear error message generated in the /opt/bmc/BCO/scheduler/log/cpit.log file that indicates why the TSCO Scheduler couldn't connect to the database.  You can then work with your DBA to correct the problem.

       

    Debugging

       You can capture the logGrabber output from the command line of the TSCO Application Server:

    (1) Log on the machines from which you need to gather data via ssh.

    (2) Locate the TSCO installation directory (by default, /opt/bmc/BCO) and access the 'tools' sub-folder 

    (3) Run the script logGrabber.sh. This script creates a .tar.gz archive file containing the logs of all TSCO services installed on the local machine.  This file will be created in the same directory as the logGrabber.sh script itself.

    (4) Open a case with TSCO Technical Support and associate the log as attachment with the case via the BMC Support site (http://www.bmc.com/support/support-central.html) -> Manage Your Cases (http://www.bmc.com/support/resources/issue-defect-management.html) -> Submit and Manage Cases (http://www.bmc.com/available/submit-new-issue.html) -> Open the case and add the attachment to it.

     


    Article Number:

    000293814


    Article Type:

    Solutions to a Product Problem



      Looking for additional information?    Search BMC Support  or  Browse Knowledge Articles