How to prevent failures when upgrading AtriumCore from 7.5 to 8.1 and 9.1.

Version 7
    Share This:

    This document contains official content from the BMC Software Knowledge Base. It is automatically updated when the knowledge article is modified.


    PRODUCT:

    BMC Atrium Core - CMDB


    COMPONENT:

    BMC Atrium Core


    APPLIES TO:

    BMC Atrium Core - CMDB 7604, 8.1, 9.0 through 9.1.04



    PROBLEM:

     

    Upgrading from versions prior to the 7.6.04 release can be tricky, but fairly simple to acheive if you know the steps covered below.

      

    There are various KA's on this topic already. This KA rolls up all the known causes into one article.

      

    Starting point : BMC AtriumCore 7.5 CMDB, AIS, Product Catalog and plugins.

      

    Finish: BMC AtriumCore 9.1.04

      

    NOTE: This KA does not take AtriumCore WSC into account because we've changed the Tibco WSC to Axis 2 with Java UDDI (JUDDI). This change alone virtually removed all install issues of the Atrium Core Web Services.

     


    CAUSE:

    Upgrade can fail when the application isn't in good condition before upgrading it.


    SOLUTION:

     

    Legacy ID:KA406262

      

    The following areas are known to cause issues at some time during the installer: 

      
       
    • 1. Prechecker Utility of the CMDB Installer
    •  
    • 2. Disable Escalations
    •  
    • 3. Best Practice Conversion Utility - BPCU
    •  
    • 4. Custom Workflow and Indexes for cmdb fields
    •  
    • 5. Attachment Size Limit
    •  
    • 6. Warning suppression
    •  
    • 7. SIM Extensions out of synch with ARS Metadata
    •  
    • 8. 32 bit CMDB vs. 64 bit ARS Server
    •  
    • 9. Keeping Windows Explorer opened in the AtriumCore folder during the upgrade
    •  
    • 10. Expected component services not running/started during the upgrade
    •  
    • 11. Invalid "Midtier" password that causes "AtriumCore console flyout error" when accessing it.
    •  
    • 12. ZDT (Zero Down Time) Roll back (9104 exclusive)
    •  
    • 13. Atrium Web Services fails because the Tomcat could not be started during upgrade. 
      

     

      
    1. Prechecker Utility of the CMDB Installer
      

    Part of the installation includes a CMDB Pre-Checker that validates field ids and names. This prechecker should be only executed if the version of CMDB is already 7604 and not executed if the version is prior to 7603 SP1, however it is part of the installer and therefore it will run anyway and add ERRORs to the install log. This is OK and can be ignored.

      

    Here is some background on this. If you run the prechecker before 7603 SP1 then the prechecker will have some failure exit codes that may also show up in the arerror.log. These errors are there because there are Field IDs for Company that get checked in the following forms: 

      

    PCT:VersionCompanyAssocStatusFlags
    PCT:PatchCompanyAssocStatusFlags

      

    The installer needs to change these Field IDs from value "420000165" to this value "1000000001". Since this is part of the installer and the prechecker runs before that can happen and hence these errors are ignored.

      

    The log will show an exit code of 1025 after running this command: 

      

    LOG EVENT {Description=[AR ChangeID command],Detail=[C:UsersE020156AppDataLocalTempUtilitiesrikarchgid.exe -x ARSALIAS -u Action Request Installer Account -p ****** -t 0 -c 10003 -N 7200 -L 14400 -X 21600 -F C:UsersCURENTUSERAppDataLocalTempUtilitiespcchangeFiledIDs.txt
     

      

    Since there isn't a way to easily remove the Pre-Checker from the available AtriumCore Installer program then all you need to do is to understand that this is a false positive result.

      

    2. Disable Escalations

      

    Upgrade takes a while to complete and some of the changes during the install impact CMDB data that trigger AR Escalations or qualify escalations to run.
    This is can halt a stage from completing and cause a failure of the install. Usually this type of failure is not obvious and causes the installer to fail at different stages.
    Please always remember to Disable AR Escalations before the upgrade or install.

     

      

    3. BPCU (Best Practice Conversion Utility)

      

    BMC has produced a customization conversion utility that converts custom workflow into Overlays. For customers who have Asset Management and other applications where the CMDB is providing data structure service and have customized workflow this utility is best used AFTER the AtriumCore Upgrade is completed. Althoug the BPCU utility does not overlay any CMDB schemas and workflow even if customizations are found, however running this utility sets the expectation that even if the CMDB was overlaid than this would be corrected by the BPCU. This is not true.

      

    Please see https://selfservice.bmc.com/casemgmt/sc_KnowledgeArticle?sfdcid=000096586 for reference.

      

    Bottom line for this point is - run the BPCU after the upgrade of CMDB is completed, and successful and was not impacted by overlay issues. We've done these upgrades now with 100% success rate and are confident to support this practice.

      

     

      

     

      

    4. Custom Workflow for cmdb fields

      

    This item is related to the previous one, but more specific to data that is being inserted into BMC.CORE CDM forms.

      

    The installer inserts various template data into the BMC.CORE forms and this step can be blocked if additional data restrictions or attribute requirements are enforced by custom workflow. These fields may include MarkAsDeleted, Company, Region, Site and so on. If you have customizations in place that require a value in a field that is not already required out of the box then you should disable the workflow for those fields before you start the installer.

      

    The install log would have captured such failure with exit code of 1025 and log entry would look like this: 

      

    C:UsersCurrentUserAppDataLocalTempUtilitiesrikrik.exe" loadapp -x ARSALIAS -t 0 -u "Action Request Installer Account" -p <not_displayed> -l "D:Program FilesBMC SoftwareAtriumCore" -n CMDB-RIK_Install -f "D:Program FilesBMC SoftwareAtriumCorecmdbenworkflowupgrade764patch000wf-RIK-CMDB.xml" -L -C

      

    Where the name of the log that actually captured the real failure codes is "CMDB-RIK_Install.log"

      

    This log would then show an error like this

      

    INFO  - Error importing record 1: ERROR (806201): ; Please supply a Category, Type and Item for this Configuration Item. Those fields require and entry to create or modify a CI.
    DATA "" "" "" "" "" "" "" "BMC_BUSINESSSERVICE" "0;" "" "" "" "" 1282162748 "BMC.ASSET" "" "" "" "" 0 "" "STANDARD" "BMC_GLOBAL_DEFAULT_SRVC" "" "" "Demo" "" "" "" "" "" "" "" "" "" "" 1285101154 "BMC_GLOBAL_DEFAULT_SRVC" "" 30 "" "" "" "" "" 0 "BMC_GLOBAL_DEFAULT_SRVC" "" "" "" "000000000000011|000000000000383" "" 10 10 "Default Service" "" "" "" "" "" "" "" "" "" "" 0 "1284595003Demo" "Demo" 0 "" "0" "" "" "" "" "" 0 
    INFO  - Import Completed in 8.237 seconds. 0 records were imported to BMC.CORE:BMC_BusinessService; 1 Records were not. From File D:Program FilesBMC SoftwareAtriumCorecmdbenworkflowupgrade764patch000.USM_BusinessService_DefaultData.arx

      

    [ERROR] LoadComponent- Data Import failed with code 1025 for file D:Program FilesBMC SoftwareAtriumCorecmdbenworkflowupgrade764patch000.USM_BusinessService_DefaultData.arx

      

    And there would additional records like this that fail. Exit code of 1025 can almost always be associated with a data collision of some type. Either the record could not be imported, or it could not be deleted (code 1024) for what ever reason.

      

    Please disable workflow that would prevent the installer from managing records in the CMDB. No customer records are deleted during the upgrade, only CMDB template or default loads like the BusinessService data will be impacted.

      

     

      

    5. Attach Size Limit

      

    We've found that the ARDBC setting of "Db-Max-Attach-Size: 0" has impact on attachments during upgrades. If you have this setting in the ar.cfg (ar.conf if Unix/Linux) then change the value to 200000000 (~200 MB) or just remove it from the config file altogether.

      

    ARS also has a maximum attachment size limit that can conflict with the Data Visualization Components that need to be loaded into the Data Visualization forms during the install. The size of these attachments varies from few kilobytes to several megabytes.
    Rule of thumb is to remove the attachment size restriction for the installer to complete successfully. This means that you should set the size limit to 0 (unlimited).

      

    Set Attachment Size Maximum to 0 for the ARDBC (ARS Data Base Configuration values a.k.a. the ARDBC metadata).

      

    You can also run a test before the upgrade by attaching a sample file that's no more than 100 MB to see if the attachment will work. This setting is in the AR Server Configuration Panel. You can leave it at 0 if it already is set that way.

      

    The following settings in the ar.conf file will imdepe imports of files larger than the set limit: 

      

    RPC-Max-Record-Size: 2000000

      

    DB-Max-Attach-Size: 2000000

      

    AR-Max-Attach-Size: 2000000

      

    2000000 is 2 MBs, so that is too small. If you run into attachment size issues like this then you'll need make sure that after the change these size limits are removed.

      

    Remove these while the install activity is taking place or increase it to allow for attachments beyond 100 MB (100000000).

      

     

      

     6. Warning Supression (ARS)

      

    There are several Warnings interpreted by the installer as failures. These warning can be ingored by supressing them during the time of the installation.

      

    You can add the following the Warning supressions: 

      

    See : https://selfservice.bmc.com/casemgmt/sc_KnowledgeArticle?sfdcid=000039883

      

    Suppress-Warnings: 9936

      

    Additional warnings maybe there already, so just add 9936 to the list for the upgrade to complete successfully. This should no longer be an issue in 8.1 or even 8.0 upgrades, however you'd be better off to see that done to prevent this from being a false positive failure of the install.

      

    Our expectation and coding of the Installer should not have this issue surface at all. However we'd like to mention it here for those customers that are using this KA for upgrading to 7.6.04 SP4 of AtriumCore where this issue has manifested itself.

      

    Please see https://selfservice.bmc.com/casemgmt/sc_KnowledgeArticle?sfdcid=000074621 for additional information.

      

     

      

    7. SIM Extensions being out of synch with ARS Metadata

      

    If you installed SIM extensions and the installation did not go succesfully then your CMDB metadata could be out of synch.

      

    Fortunatelly we have a new utility that you can get from the Utilities folder. The cdmchecker. It will check the metadata of CMDB and compare it wiht ARS when you run this utility with the -a option.

      

    Syntax: 

      

    cdmchecker -a -u <ARS USER> -p <ARPASSWORD> -s <ARHOST_ALIAS> -t "<ARPORT>" -o "missing_fields.log" -i "BMC_MAINFRAME*OB000F1F66BB3CX44aQQCfftAAMgAA|BMC_PRINTER*OB000F1F66BB3CX44aQQCfftAAMgAA|BMC_FEDERATEDDATAINTERFACE*OB005056BE6784SK1zSAZnC9gwRocG"

      

    The -i option gives a list of known attributes to be out of synch and hence they need to be included in the command arguments exactly as you see it above.

      

    The "missing_fields.log" will then show any discrepancies between CDM and ARS that will need to be resolved before you start the installer.

      

    This utility is available in 8.1 release of the CMDB and can be found in the ATRIUMCORE_HOMEcmdbserverbin folder and also in the %TEMP%Utilitiescmdb folder during installation.

      

    A copy of this binary that can be run on 64 bit Windows has been put here: 

      

    ftp://ftp.bmc.com/pub/BSM/AtriumCore/8.1/CMDChecker81.zip

      

    It can be ran from Windows even if the AR Server is on UNIX. However if you need to run it from non-Windows systems then start the installer and let it create the /tmp/Utilities directory. The command will be found in the cmdb folder of that utility. You can then run this command and fix any issues before you continue with the installer.

      

    Using this utility before running the installer is recommended. You may run into a defect Defect SW00465091: Upgrade to 8.1.x fails on Dimensions_pool if the CDM was extended with new class as child of ComputerSystem where install fails while running cdmchecker:

      
       com.bmc.smbu.install.common.rule.engine.CommandExecutionException: com.bmc.smbu.install.common.rule.engine.CommandExecutionException: java.lang.Exception: CDM Checker found missing fields on underlying AR forms. Please contact BMC Support with the file : "<C:?Program Files?BMC Software?AtriumCore?Logs? missing_fields.log>" CDM checker Return code =132 Output message=No response  
      
        
      
      missing_fields.log shows: 
      
        
      
      [ERROR]  Error during ARGetMultipleFields in FindMissingFields Status: Field does not exist on current form. Details: BMC.CORE:BMC_ComputerSystem : <301185300> 
      
      [ERROR]  In class BMC.CORE:BMC_ComputerSystem, for attribute Dimensions_pool with attribute id OB000F1F66BB3CX44aQQCfftAAMgAA, could not find corresponding AR field with ID=301185300 on form BMC.CORE:BMC_ComputerSystem. 
      
        
      
      1. Class: BMC.CORE:BMC_ComputerSystem  Attribute: Dimensions_pool 
      
      2. Class: PNC.CORE:Custom_Desktop  Attribute: Dimensions_pool 
      
        
      
      However even If you run the command by itself, it will not ignore it if you add it to the ignore list. As shown: 
      
        
      
      cdmchecker -a -u Demo -p Demo -s MYARSHOSTNAME -t "0" -o "missing_fields2.log" -i "BMC_MAINFRAME*OB000F1F66BB3CX44aQQCfftAAMgAA|BMC_PRINTER*OB000F1F66BB3CX44aQQCfftAAMgAA|BMC_FEDERATEDDATAINTERFACE*OB005056BE6784SK1zSAZnC9gwRocG|BMC_COMPUTERYSTEM*OB000F1F66BB3CX44aQQCfftAAMgAA" 
      
        
      
      Deleting the record "Dimensions_pool" from the AttributeDefinitions form before proceeding with the Install will work and pass the cdmchecker. 
      
        
      

    8. 32 bit AtriumCore CMDB vs. 64 bit ARS Server

      

    If you've decided to upgrade to ARS Server 7.6.04 and later and do so on a 64 bit Operating System such as Windows or Linux, then you will need to upgrade to 7604 AtriumCore that is also 64 bit. These modes can not be mixed. The ARS Server will need to remain 32 bit if you want to stay with 32 bit CMDB (i.e. CMDB 7.5 SP5). Ignoring this compatibility requirement will result in Win32 error in the arerror.log while loading cmdbsvr75.dll.

      

    For example you'd see an error in arerror.log immediately after starting the ARS Server: 

      

    Thu Feb 27 23:25:02 2014 : Action Request System(R) Server x64 Version 7.6.04 SP5 201308012025
    (c) Copyright 1991-2011 BMC Software, Inc.
    Thu Feb 27 23:25:02 2014  ProcessMain : Error encountered while loading the shared library (ARERR 9750)
    Thu Feb 27 23:25:02 2014     cmdbsvr75.dll: %1 is not a valid Win32 application.

      

    Please make sure you maintain the same bit width when upgrading. This is also why the AtriumCore installer first makes changes to configuration files and shuts down the server to replace the binaries before starting it back up.

      

     

      

     

      

    9. Keeping Windows Explorer opened in the AtriumCore folder during the upgrade

      

    You should not have any command line interface (CLI) windows/sessions opened or browse any directories within the AtriumCore directories during the upgrade.If you do so during the ugprade you'll see install failures with the following errors: 

      

    Installation skipped since deletion of the following folders failed:D:\Program Files\BMC Software\AtriumCore\cmdb

      

    Feature Web Services Registry failed

      

    Installation skipped since deletion of the following folders failed:D:\Program Files\BMC Software\AtriumCore\cmdb

      

    Installation skipped since deletion of the following folders failed:D:\Program Files\BMC Software\AtriumCore\cmdb

      

    Feature Web Services Infrastructure failed

      

    Installation skipped since deletion of the following folders failed:D:\Program Files\BMC Software\AtriumCore\cmdb

      

    Feature Atrium Core Web Services failed

      

    Installation skipped since deletion of the following folders failed:D:\Program Files\BMC Software\AtriumCore\cmdb

      

    Installation skipped since deletion of the following folders failed:D:\Program Files\BMC Software\AtriumCore\cmdb

      

    Installation skipped since deletion of the following folders failed:D:\Program Files\BMC Software\AtriumCore\cmdb

      

    Feature BMC Atrium CMDB failed

      

    Installation skipped since deletion of the following folders failed:D:\Program Files\BMC Software\AtriumCore\cmdb

      

     

      

    Logs directory is an exception. You can have your browser or CLI in the AtriumCore\Logs dir.

      

     

      

    10. Expected component services not running/started during the upgrade

      

    Components that run as services (Windows) or deamons (Unix/Linux) should be running during the upgrade.

      

    For example, AIS, AIE, WebServices Apache Tomcat and Java API's like NORM should be running before the upgrade is started. Installer will try to check for them and will result in failure if their process or listening port can not be found.

      

    Related log entry: 

      

    [Atrium Impact Simulator not running],Detail=[ Output message=20140704 213243.103000 mcstat: MCLI: BMC-IMC300051W: Message #1 canceled (destination Cell disconnected)

      

    Please make sure that all AtriumCore componets you've selected to upgrade have their related services/deamons running before you start the installer.

      

     

      

    11. Invalid "Midtier" password that causes "AtriumCore or AtriumIntegrator console Authentication error" when accessing it.

      

      

    Deployment of the Data Visualization System Components depends on having the correct password in the midtier config panel. Make sure this password passes the test by validating it in Midtier Configuration before the upgrade or before trying to access the AtriumCore console for the first time.
    To test it enter the password you think is correct with the "Validate Password" box checked. As shown below in green highlight. If you get "ERROR (623): Authentication Failed" then the password you have on file is not correct and needs to be updated.

      

    Midtier Config Panel

      

     

      

    To fix this go to the AR Server Administration panel for "Connection Settings" and update the password shown below in green highlight.
    Restart of AR is required and you will need to empty your browser cache if you're tried accessing the AtriumCore console before and got the error even just once.
    It will be cached in the Adobe player's  Flash cache and that must be purged (flushed).

      

     

      

    Restart ARS and retest the password in the Midtier config console. It must pass before you can get the AtriumCore console data. 
    You can also test it through the AtriumIntegrator console under the AR Server type datastore. It is setup with the Remedy Application Service user and password. 

      

    If the error continues then the best course of action is to stop the midtier and go to the AR Server/Midtier/WEB-INF/  directory and delete the folder that has the hostname of the AR Server in it.

      

      

    Restart the midtier and make sure the folder is rebuilt after accessing the AtriumCore or AtriumIntegrator at least once.

      

    Above is a screen capture of an example. In this configuration the server name "clm-aus-000442" is the AR Server.
    In this use case we'd delete the folder with the hostname of the AR Server with hostname "clm-aus-000442".
    Midtier will rebuild it from Data Visualization System Files attachments. (See Attachment Size Limit info issues in this Knowledge Article)  

      

     

      

    12. ZDT (Zero Down Time) Roll back (9104 exclusive)

      

    This feature has been problematic for customers and hence the 1805 release of the product will feature an option to prevent the roll back from occurring. However, in the meantime you should know that 9104 and 1802 (recent product versioning change that reflects the year and month of its release) will roll back everything even if other components have succeeded. For example if AR Server and AtriumCore succeed, but AI Server fails to upgrade then AtriumCore and ARServer both will be rolled back. There is a work around for this. Remove the ZDT folders from AR Server and AtriumCore directories before the upgrade. If there is any failure then AR and AtriumCore will not fail. 
     

      

    13. Atrium Web Services fails because the Tomcat could not be started during upgrade. 

    If you have AtriumWebServices (AWS) on the same host as AR Server then the chances of it failing is high. We have a defect filed for this now, however the best way to avoid this is to install the AtriumCoreWebservices on a different host. Use the bundled version of Tomcat and then upgrade it. The failure is related to starting and stopping the Tomcat service during the upgrade. Publishing the UDDI registry takes about 6 to 10 minutes and this makes it look like the installer is stalled. You may be able to detect this in the log and restart the Tomcat services manually, however this is not always reliable.  

     


    Article Number:

    000097520


    Article Type:

    Solutions to a Product Problem



      Looking for additional information?    Search BMC Support  or  Browse Knowledge Articles