REMEDY 9.x Release - CONFIGURATION CHECKLIST from the BMC R&D PERFORMANCE TEAM

Version 3
    Share:|

    This document contains official content from the BMC Software Knowledge Base. It is automatically updated when the knowledge article is modified.


    PRODUCT:

    Remedy AR System Server


    COMPONENT:

    AR System


    APPLIES TO:

    All Remedy AR System 9.x Releases



    QUESTION:

     
       What are the best practice recommendations to configure the Out of the Box AR System platform for Remedy 9.x? 


    ANSWER:

     

       

    Introduction
    This document is a configuration checklist for the OOTB AR platform install configuration for Remedy 9.1 SP2 release.  IT is applicable to SP1 as well.  Much of the configuration is the same.  Where they differ, a note has been included.  The document may be updated as more testing and observation is complete.  So please check periodically for updates.  Check this link for more info on SP2 changes.
    For more information on Performance Tuning for Remedy, check this link.
    For information on Hardware Requirements, check this
    link.

      
      

    This configuration is specific to small, medium and large model defined as follows:

                                                                                                                              
    EnvironmentBMC Remedy ITSM (concurrent users)BMC Remedy Smart Reporting (concurrent users) Number of managed devices
    Small800502,000 or fewer
    Medium2,00010010,000 or fewer
    Large5,00025030,000 or fewer
      

    This checklist is for the configurations in the Midtier, RSSO, AR Server, Smart Reporting, Smart IT/MyIT server and database server. There’s also a section for the settings of the web-infrastructure (web proxy, tomcat, JVM parameters), the Midtier, and ITSM application stack.


    Web Proxy or Load-Balancer for The Remedy SSO Service
    There are 2 network segments: From the browser proxy (or from the actual client if there’s no proxy) to the F5; from the F5 to the SSO service. Since the interaction between the browser and the SSO service is brief, there is no need to activate HTTP keep-alive for either of the network segments.

    For the F5, the recommended load-balancing scheme is round-robin no session binding.

    Notes:
    1- The client HTTP protocol version for HTTP monitoring must be set to HTTP 1.1. The return HTTP code for alive is HTTP 200.
    2- For performance testing, the granularity of ping interval was 5 seconds with 3 consecutive non-HTTP 200 indicating the node is down. For deployment, you may configure this to a lower threshold as required.



    JVM Options in tomcat/bin/setenv.sh for Remedy SSO

                                                                                                                                                          
    ComponentSmallMediumLarge
    BMC Remedy SSOOne Server
    2 CPU core
    8 GB RAM
    200 GB disk space
    Two Servers
    2 CPU core
    8 GB RAM
    200 GB disk space
    Four Servers
    2 CPU core
    8 GB RAM
    200 GB disk space
    Java 1.8-XX:MaxMetaspaceSize=512m
    -Xms4096m
    -Xmx4096m
    -XX:+UseCompressedOops
    -XX:+UseConcMarkSweepGC
    -XX:+UseParNewGC
    -XX:NewRatio=2
    -XX:MaxMetaspaceSize=512m
    -Xms4096m
    -Xmx4096m
    -XX:+UseCompressedOops
    -XX:+UseConcMarkSweepGC
    -XX:+UseParNewGC
    -XX:NewRatio=2
    -XX:MaxMetaspaceSize=512m
    -Xms4096m
    -Xmx4096m
    -XX:+UseCompressedOops
    -XX:+UseConcMarkSweepGC
    -XX:+UseParNewGC
    -XX:NewRatio=2
    Tomcat/conf/server.xmlmaxThreads="1600"maxThreads="3000"maxThreads="3000"
    Tomcat/conf/logging.propertiesChange all log level from INFO to SEVEREChange all log level from INFO to SEVEREChange all log level from INFO to SEVERE
      



    Web Proxy or Load-Balancer for Midtier
    There are 2 network segments:
                  - From the client browser proxy to the F5
                  - From the F5 to the Midtier Tomcat cluster.

    1. For both network segments, HTTP keep-alive must be turned on.
    If the keep-alive feature also supports the optional values as in tomcat, the recommended values are given by the following table:

                                                                    
    Keep Alive PropertyValue
    Keep-alive count5,000
    Keep-alive timeout60 seconds
      

    2. For the F5, the recommended load-balancing scheme is round-robin with HTTP session binding cookie insert. (Note: Client HTTP session binding via the SSL session ID is to be tested.)
    3. For detection whether a node is alive, use HTTP monitoring and issue a GET on the following URL:  /arsys/shared/images/bkgd_image.gif

    Notes:
    a) The client HTTP protocol version for HTTP monitoring must be set to HTTP 1.1. The return HTTP code for alive is HTTP 200.
    b)For performance testing, the granularity of ping interval was 15 seconds with 3 consecutive non-HTTP 200 indicating the node is down. For deployment, you may configure this to a lower threshold as required.


    JVM Options in tomcat/bin/startup.sh For The Midtier Tomcat
    Important: The following settings assume that each tomcat instance in the cluster hosts only the Midtier and the RSSO agent.

    BMC AR Mid-Tier Server configuration settings
    Note
    :  For Windows, use the “Configure Tomcat’ tool and select the ‘Java’ tab to set these options:

                                                                                                                                                                                                                  
    ComponentSmallMediumLarge
    BMC Remedy AR System mid tier serversTwo servers:
    2 CPU core
    8 GB RAM
    60 GB disk space
    Two servers:
    4 CPU core
    12 GB RAM
    60 GB disk space
    Five servers:
    4 CPU core
    16 GB RAM
    60 GB disk space
    Java 1.8-XX:MaxMetaspaceSize=512m
    -Xms4096m
    -Xmx4096m
    -XX:+UseCompressedOops
    -XX:+UseConcMarkSweepGC
    -XX:+UseParNewGC
    -XX:NewRatio=2
    -XX:MaxMetaspaceSize=512m
    -Xms8192m
    -Xmx8192m
    -XX:+UseCompressedOops
    -XX:+UseConcMarkSweepGC
    -XX:+UseParNewGC
    -XX:NewRatio=2
    -XX:MaxMetaspaceSize=512m
    -Xms12288m
    -Xmx12288m
    -XX:+UseCompressedOops
    -XX:+UseConcMarkSweepGC
    -XX:+UseParNewGC
    -XX:NewRatio=2
    Tomcat/conf/server.xmlacceptCount="250"
    connectionTimeout="60000"
    maxKeepAliveRequests=”5000
    maxSpareThreads="100"
    maxThreads="1200"
    minSpareThreads="50"
    acceptCount="250"
    connectionTimeout="60000"
    maxKeepAliveRequests=”5000
    maxSpareThreads="100"
    maxThreads="2000"
    minSpareThreads="50"
    acceptCount="250"
    connectionTimeout="60000"
    maxKeepAliveRequests=”5000
    maxSpareThreads="100"
    maxThreads="2000"
    minSpareThreads="50"
    Tomcat/conf/logging.propertiescatalina.org.apache.juli.FileHandler.level = SEVERE
    localhost.org.apache.juli.FileHandler.level = SEVERE
    catalina.org.apache.juli.FileHandler.level = SEVERE
    localhost.org.apache.juli.FileHandler.level = SEVERE
    catalina.org.apache.juli.FileHandler.level = SEVERE
    localhost.org.apache.juli.FileHandler.level = SEVERE
    MIDTIER/WEB-INF/classes/config.propertiesarsystem.pooling_max_connections_per_server=150
    arsystem.log_category=INTERNAL
    arsystem.use_loadbalanceinterval=true
    arsystem.use_loadbalanceinterval_limit=15 
    arsystem.pooling_max_connections_per_server=350
    arsystem.log_category=INTERNAL
    arsystem.use_loadbalanceinterval=true
    arsystem.use_loadbalanceinterval_limit=15 
    arsystem.pooling_max_connections_per_server=350
    arsystem.log_category=INTERNAL
    arsystem.use_loadbalanceinterval=true
    arsystem.use_loadbalanceinterval_limit=15 
    Tomcat/conf/logging.propertiesChange all log level from INFO to SEVEREChange all log level from INFO to SEVEREChange all log level from INFO to SEVERE
      


    JVM Options in tomcat/bin/startup.sh For The Midtier+RSSO Tomcat
    Important:
    The following settings assume that each tomcat instance in the cluster hosts both the Midtier (with the RSSO agent) and the RSSO web app.

    All of the settings in the previous section (where tomcat hosts only the Midtier and the RSSO agent) applies. There’s only 1 change given by the following:

                                          
    Tomcat/conf/server.xmlmaxThreads="1600"maxThreads="3000"maxThreads="3000"
      



    BMC AR System Server (user facing) configuration settings

                                                                                                                                                                                                                               
    ComponentSmallMediumLarge
    BMC Remedy AR System serversTwo servers:
    2 CPU core
    12 GB RAM
    60 GB disk space
    Two servers:
    4 CPU core
    16 GB RAM
    60 GB disk space
    Three servers:
    8 CPU core
    32 GB RAM
    60 GB disk space
    Java 1.8
    (arserver.config)

    jvm.minimum.heap.size=6442450944
    jvm.maximum.heap.size=6442450944

    -XX:+DisableExplicitGC
    -XX:MaxMetaspaceSize=512m
    -XX:MetaspaceSize=384m
    -XX:+UseCompressedOops
    -XX:+UseConcMarkSweepGC
    -XX:+UseParNewGC
    -XX:NewRatio=2

    jvm.minimum.heap.size=8589934592
    jvm.maximum.heap.size=8589934592

    -XX:+DisableExplicitGC
    -XX:MaxMetaspaceSize=512m
    -XX:MetaspaceSize=384m
    -XX:+UseCompressedOops
    -XX:+UseConcMarkSweepGC
    -XX:+UseParNewGC
    -XX:NewRatio=2

    jvm.minimum.heap.size=21474836480
    jvm.maximum.heap.size=21474836480

    -XX:+DisableExplicitGC
    -XX:MaxMetaspaceSize=512m
    -XX:MetaspaceSize=384m
    -XX:+UseCompressedOops
    -XX:+UseConcMarkSweepGC
    -XX:+UseParNewGC
    -XX:NewRatio=2
    Note: For jdk/jre up to 1.7, use PermSize in place of MetaspaceSize, i.e., use -XX:MaxPermSize=512m -XX:PermSize=384m
    Fast queuePrivate-RPC-Socket:  390620   4   6Private-RPC-Socket:  390620   8  12Private-RPC-Socket:  390620   20   24
    Loop back or private  socket queuePrivate-RPC-Socket:  390626   5   5Private-RPC-Socket:  390626   5   7Private-RPC-Socket:  390626   5  9
    List queuePrivate-RPC-Socket:  390635   6  10Private-RPC-Socket:  390635   16 20Private-RPC-Socket:  390635   24  30
     CMDB-Inline-Normalization: TCMDB-Inline-Normalization: TCMDB-Inline-Normalization: T
      
       



    BMC AR System Server (Backend Integration) configuration settings

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    
    ComponentSmallMediumLarge
    BMC Remedy AR System serversTwo servers:
    4 CPU core
    12 GB RAM
    60 GB disk space
    Two servers:
    4 CPU core
    16 GB RAM
    60 GB disk space
    Two servers:
    8 CPU core
    32 GB RAM
    60 GB disk space
           
           Java 1.8     
           
           (arserver.config)     

    jvm.minimum.heap.size=6442450944
    jvm.maximum.heap.size=6442450944

    -XX:+DisableExplicitGC
    -XX:MaxMetaspaceSize=512m
    -XX:MetaspaceSize=384m
    -XX:+UseCompressedOops
    -XX:+UseConcMarkSweepGC
    -XX:+UseParNewGC
    -XX:NewRatio=2

    jvm.minimum.heap.size=8589934592
    jvm.maximum.heap.size=8589934592

    -XX:+DisableExplicitGC
    -XX:MaxMetaspaceSize=512m
    -XX:MetaspaceSize=384m
    -XX:+UseCompressedOops
    -XX:+UseConcMarkSweepGC
    -XX:+UseParNewGC
    -XX:NewRatio=2

    jvm.minimum.heap.size=21474836480
    jvm.maximum.heap.size=21474836480

    -XX:+DisableExplicitGC
    -XX:MaxMetaspaceSize=512m
    -XX:MetaspaceSize=384m
    -XX:+UseCompressedOops
    -XX:+UseConcMarkSweepGC
    -XX:+UseParNewGC
    -XX:NewRatio=2
    Note: For jdk/jre up to 1.7, use PermSize in place of MetaspaceSize, i.e., use -XX:MaxPermSize=512m -XX:PermSize=384m
    FTS QueuePrivate-RPC-Socket: 390602   2 2Private-RPC-Socket: 390602   2 2Private-RPC-Socket: 390602   2 4
    Reconciliation QueuePrivate-RPC-Socket:  390698   2 4Private-RPC-Socket:  390698   2 4Private-RPC-Socket:  390698   2 8
    CMDB RPC Normalization QueuePrivate-RPC-Socket:  390699   2 4Private-RPC-Socket:  390699   2 4Private-RPC-Socket:  390699   2 8
    AR RPC Normalization QueuePrivate-RPC-Socket:  390681   2 2Private-RPC-Socket:  390681   2 2Private-RPC-Socket:  390681   2 2
    List queuePrivate-RPC-Socket:  390635   6 10Private-RPC-Socket:  390635   16  20*Private-RPC-Socket:  390635   24  30
    Fast queuePrivate-RPC-Socket:  390620   4 6Private-RPC-Socket:  390620   8  12*Private-RPC-Socket:  390620   20   24
    Loop back or private  socket queuePrivate-RPC-Socket:  390626   5  5Private-RPC-Socket:  390626   5  7*Private-RPC-Socket:  390626   5  9
     CMDB-Inline-Normalization: TCMDB-Inline-Normalization: TCMDB-Inline-Normalization: T
    Alert queuePrivate-RPC-Socket:  390601   1  1Private-RPC-Socket:  390601   1   1Private-RPC-Socket:  390601   1  1
    Escalation queuePrivate-RPC-Socket:  390603   3  3Private-RPC-Socket:  390603   6   6Private-RPC-Socket:  390603   9  9
    ApprovalPrivate-RPC-Socket:  390680   2  2Private-RPC-Socket:  390680   2  2Private-RPC-Socket:  390680   2  2
     Plugin-ARDBC-Threads: 4    12Plugin-ARDBC-Threads: 4    12Plugin-ARDBC-Threads: 4    12
     Plugin-AREA-Threads: 4    12Plugin-AREA-Threads: 4    12Plugin-AREA-Threads: 4    12
     Plugin-Filter-API-Threads: 4    12Plugin-Filter-API-Threads: 4    12Plugin-Filter-API-Threads: 4    12
      
      
       

    * Denotes new default value in 9.1 SP2. If this value is unset, the new default is used.


    BMC AR System Server – Common configuration settings for User facing and backend server

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
    SmallMediumLarge
    Max-Entries-Per-Query: 2000*Max-Entries-Per-Query: 2000*Max-Entries-Per-Query: 2000*
    Next-ID-Block-Size: 100Next-ID-Block-Size: 100Next-ID-Block-Size: 100
    Server-Side-Table-Chunk-Size: 1000Server-Side-Table-Chunk-Size: 1000Server-Side-Table-Chunk-Size: 1000
    Allow-Unqual-Queries: F*Allow-Unqual-Queries: F*Allow-Unqual-Queries: F*
    Cache-Mode: 0*Cache-Mode: 0*Cache-Mode: 0*
    Submitter-Mode: 1*Submitter-Mode: 1*Submitter-Mode: 1*
    Authentication-Chaining-Mode: 1*Authentication-Chaining-Mode: 1*Authentication-Chaining-Mode: 1*
    Server-Plugin-Default-Timeout: 120*Server-Plugin-Default-Timeout: 120*Server-Plugin-Default-Timeout: 120*
    External-Authentication-Return-Data-Capabilities: 31*External-Authentication-Return-Data-Capabilities: 31*External-Authentication-Return-Data-Capabilities: 31*
    RE-RPC-Socket: 390698RE-RPC-Socket: 390698RE-RPC-Socket: 390698
    Debug-mode: 131120  Debug-mode: 131120  Debug-mode: 131120  
    Max-Log-History: 8Max-Log-History: 8Max-Log-History: 8
    Max-Log-File-Size: 134217728Max-Log-File-Size: 134217728Max-Log-File-Size: 134217728
    Plugin-Log-Level: 1000Plugin-Log-Level: 1000Plugin-Log-Level: 1000
    VersionControl-Object-Modification-Log-Mode: 10VersionControl-Object-Modification-Log-Mode: 10VersionControl-Object-Modification-Log-Mode: 10
    VersionControl-Object-Modification-Log-Save-Definition-Files: 0VersionControl-Object-Modification-Log-Save-Definition-Files: 0VersionControl-Object-Modification-Log-Save-Definition-Files: 0
      
      

    * Denotes new default value in 9.1 SP2. If this value is unset, the new default is used.


    AR Dir/pluginsvr/pluginsvr_config.xml
    There are 2 things to do:
    1-Add or alter this file to:
                  <numCoreThreads>30</numCoreThreads>
                  <numSelectorThreads>2</numSelectorThreads>

      

    2- Create arsys_api.xml in the current plug-in server directory with the following content.
                  <?xml version="1.0" encoding="UTF-8" ?>
                    <javaapi_config>
                     <maxProxiesPerServer>80</maxProxiesPerServer>
                    </javaapi_config>



    AR Dir/pluginsvr/fts/secondary/pluginsvr_config.xml
    NOTE: Configure for primary FTS only if in server group setup. 
    There are 2 things to do:
    1- Add or alter this file to:
                  <numCoreThreads>10</numCoreThreads>
                  <numSelectorThreads>2</numSelectorThreads>


    2- Create arsys_api.xml in the current plug-in server directory with the following content:
                  <?xml version="1.0" encoding="UTF-8" ?>
                  <javaapi_config>
                  <maxProxiesPerServer>80</maxProxiesPerServer>
                  </javaapi_config>



    <atrium core install dir>/cmdb/plugins/shared/pluginsvr_config.xml
    There are 2 things to do:
    1- Add or alter this file to
                  <numCoreThreads>10</numCoreThreads>
                  <numSelectorThreads>2</numSelectorThreads>


    2- Create arsys_api.xml in the current plug-in server directory with the following content:
                  <?xml version="1.0" encoding="UTF-8" ?>
                  <javaapi_config>
                  <maxProxiesPerServer>80</maxProxiesPerServer>
                  </javaapi_config>



    <atrium core install dir>/cmdb/plugins/ne/pluginsvr_config.xml
    There are 2 things to do:
    1- Add or alter this file to:
                  <numCoreThreads>5</numCoreThreads>
                  <numSelectorThreads>2</numSelectorThreads>


    Note: For versions prior to 8.0, make sure the following was added if changes were done through the NE configuration GUI.
    For versions 8.0+, no need to have this content:
                  <com_bmc_ne_system_CMDBRpcQueue>390699</com_bmc_ne_system_CMDBRpcQueue>
                  <com_bmc_ne_system_ARRpcQueue>390681</com_bmc_ne_system_ARRpcQueue>

      

    2- Create arsys_api.xml in the current plug-in server directory with the following content:
                  <?xml version="1.0" encoding="UTF-8" ?>
                  <javaapi_config>
                  <maxProxiesPerServer>80</maxProxiesPerServer>
                  </javaapi_config>




    Database Server

                                                                                                                                                                                                                                                                                                                                                                                          
    ComponentSmallMediumLarge
    BMC Remedy AR Database server8 CPU core
    16 GB RAM
    200 GB disk space
    16 CPU core
    32 GB RAM
    200 GB disk space
    32 CPU core
    64 GB RAM
    200 GB disk space
    Oracle Database   
    memory_target (>11g)81632
    sga_target (10g)51020
    Db_cache_size3715
    Shared_pool_size235
    Pga_aggregate_target3612
    Cursor_sharingEXACTEXACTEXACT
    _b_tree_bitmap_plansFalseFalseFalse
    SQL Server Database   
    PARAMETERIZATIONSimpleSimpleSimple
    READ_COMMITTED_SNAPSHOTOnOnOn
      



    Tomcat/conf/server.xml for smart reporting

                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
    ComponentSmallMediumLarge
    BMC Remedy Smart Reporting ServerOne Server
    4 CPU core
    12 GB RAM
    200 GB disk space
    Two Servers
    4 CPU core
    16 GB RAM
    200 GB disk space
    Two Servers
    8 CPU core
    32 GB RAM
    200 GB disk space
    Concurrent users supported50100250
    Java 1.8-XX:MaxMetaspaceSize=512m
    -Xms6144m
    -Xmx6144m
    -XX:+UseG1GC
    -XX:MaxMetaspaceSize=512m
    -Xms8192m
    -Xmx8192m
    -XX:+UseG1GC
    -XX:MaxMetaspaceSize=512m
    -Xms16384m
    -Xmx16384m
    -XX:+UseG1GC
    Accept Count100200500
    Max Threads60012002400
    Connection Timeout900009000090000
    Max Spare Threads100100100
    Min Spare Threads252525
    max Http Header Size819281928192
    CompressionONONON
    Compression Min Size204820482048
    Compressable Mime Typetext/html,text/xml,text/plaintext/html,text/xml,text/plaintext/html,text/xml,text/plain
    noCompressionUserAgentsgozilla, traviatagozilla, traviatagozilla, traviata
    disableUploadTimeouttruetruetrue
    enableLookupsfalsefalsefalse
      
       



    Connection Pool
    Log on to Remedy home page, navigate to:
    Smart Reporting home page->Administration->Admin Console->Data Sources->AR System

                                                                                                                                                              
    SmallMediumLarge
    Min Connections   50Min Connections   50Min Connections   50
    Max Connections   100Max Connections   100Max Connections   100
    Refresh Time 3 hrsRefresh Time 3 hrsRefresh Time 3 hrs
    Timeout 6400 secondsTimeout 6400 secondsTimeout 6400 seconds
       



    <Smart-Reporting-Dir>/BMCReporting/SmartReporting/appserver/webapps/ROOT/WEB-INF/
    File: web.xml
    Settings:

                                                                                                                                  
    SmallMediumLarge
    JDBCMinCount 50JDBCMinCount 50JDBCMinCount 50
    JDBCMaxCount 100JDBCMaxCount 200JDBCMaxCount 300
    JDBCTimeout  1800JDBCTimeout  1800JDBCTimeout  1800
      
      
       



    Smart-IT/My-IT

                                                                                                                                                                                                                                                                
    ComponentSmallMediumLarge
    BMC Remedy Smart IT/ My IT ServerOne Server
    4 CPU core
    12 GB RAM
    200 GB disk space
    Two Servers
    4 CPU core
    16 GB RAM
    200 GB disk space
    Three Servers
    8 CPU core
    16 GB RAM
    200 GB disk space
    Java 1.8-XX:MaxMetaspaceSize=512m
    -Xms6144m
    -Xmx6144m
    -XX:+UseCompressedOops
    -XX:+UseConcMarkSweepGC
    -XX:+UseParNewGC
    -XX:NewRatio=2
    -XX:MaxMetaspaceSize=512m
    -Xms8192m
    -Xmx8192m
    -XX:+UseCompressedOops
    -XX:+UseConcMarkSweepGC
    -XX:+UseParNewGC
    -XX:NewRatio=2
    -XX:MaxMetaspaceSize=512m
    -Xms8192m
    -Xmx8192m
    -XX:+UseCompressedOops
    -XX:+UseConcMarkSweepGC
    -XX:+UseParNewGC
    -XX:NewRatio=2
    Mongo DB (3.0 and above, recommended version 3.2)Two Servers
    2 CPU core
    12 GB RAM
    Two Servers
    4 CPU core
    12 GB RAM
    Two Servers
    4 CPU core
    16 GB RAM + 120 GB

    SSD
    ulimitulimit -n 21000ulimit -n 21000ulimit -n 21000
    atimechattr -R +A /var/mongodata/chattr -R +A /var/mongodata/chattr -R +A /var/mongodata/
    NUMAnumactl --interleave=all /opt/mongo/mongodb-linux-x86_64-2.4.8/bin/mongodnumactl --interleave=all /opt/mongo/mongodb-linux-x86_64-2.4.8/bin/mongodnumactl --interleave=all /opt/mongo/mongodb-linux-x86_64-2.4.8/bin/mongod
      

     


    Article Number:

    000114508


    Article Type:

    FAQ/Procedural



      Looking for additional information?    Search BMC Support  or  Browse Knowledge Articles