Remedy - Server - ERROR: "The specified public key encryption algorithm is not supported by the encryption library (ARERR 9006). Encryption disabled" on Premium Security Encryption / Performance Encryption in v.19.02 with Java 11 and 12

Version 12
    Share This:

    This document contains official content from the BMC Software Knowledge Base. It is automatically updated when the knowledge article is modified.


    PRODUCT:

    Remedy AR System Server


    COMPONENT:

    AR System


    APPLIES TO:

    All versions of Remedy AR System Server with OpenJDK 11+ or Oracle Java 11+



    PROBLEM:

    When installing and enabling Encryption Premium or Performance Security there is a error in armonitor.log and arerror.log:

    publicKeyAlg=6 is not support. errorCode = 9006
    The specified public key encryption algorithm is not supported by the encryption library (ARERR 9006). Encryption disabled.

    The following error might be found in ardebug as well:

    com.bmc.arsys.common.encrypt.AREncryptionException: null

    Because of this error the undesired result is that encryption is disabled, and AR System Server is started in a non-encrypted state.  Other observed behavior could include the inability to communicate and connect with other plugin servers that have encryption set to forced.

    Note: If you set the encryption policy to "0" or Optional you might not notice a issue, but traffic between Remedy components will not be encrypted because it is optional and not required.


    CAUSE:

    In the Java version 9+, it has changed how extensions are loaded into a JVM. No longer will it load entire folders, instead you must call out the specific .jar that you want to include.


    SOLUTION:

    No hotfix is needed to fix this issue.  Please make the following configuration changes based on your situation:

    To Fix AR System Server:

    In the command used to start the process (in this example we’ll use AR System Server) you need to call the class path parameters of the specific jar’s used by BMC Encryption Security.  For AR System Server this is done with the arserver.config file.  You’ll add the parameters to the section called “JVM Classpaths”  This is what a potential section would look like for a Remedy 19.08 system:

    # JVM classpaths (number indicates classpath order)
    jvm.classpath.1=./lib/com.bmc.arsys.boot-9.1.08-SNAPSHOT.jar
    jvm.classpath.2=./lib/jmx_prometheus_javaagent-0.11.1.jar
    jvm.classpath.3=C:/Oracle/JDK/lib/bmcext/cryptojce.jar
    jvm.classpath.4=C:/Oracle/JDK/lib/bmcext/cryptojcommon.jar
    jvm.classpath.5=C:/Oracle/JDK/lib/bmcext/jcmFIPS.jar

    Once you’ve updated the arserver.config file to include these parameters, then save the file and restart the AR System Server.  This should resolve the error found in AR Monitor log.

    Steps to Resolve Client's and Plugin Servers:
    To resolve Java based processes encryption issues, you must call out the .jar files in the command used to launch the JVM.  See below for 2 examples:

    ***** Example 1: Tomcat *****

    This must be done for all Mid-Tiers as the Premium Security installer does not modify these settings in Tomcat by default.
    When launching Tomcat, a new set of classpaths must be included in addition to the others already present.  In this example I had these before:
    C:\Apache\TomcatMidTier\bin\bootstrap.jar;
    C:\Apache\TomcatMidTier\bin\tomcat-juli.jar

    Step 1: I had to add the following 3 encryption .jar’s so Tomcat knew to use them when launching (The direct location will vary based on where you installed Java):
    C:/Oracle/JDK/lib/bmcext/jcmFIPS.jar;
    C:/Oracle/JDK/lib/bmcext/cryptojcommon.jar;
    C:/Oracle/JDK/lib/bmcext/cryptojce.jar

    Step 2: Once you do this, restart the process, and Tomcat should now be able to communicate with Remedy AR System with Premium Security enabled.

    ***** Example 2: Java CLI Process *****

    The BMC Premium Security installer does add this parameter to all out of the box plugin servers.  So this step is not needed for out of the box installations, but is required for any upgraded installations coming from Oracle Java 1.8 or custom plugin servers.

    A good example is the Default Java Plugin Server that comes with AR System Server.  This can be found by looking in your armonitor.cfg file.  A proper command to start the Java Plugin Server is below:

    "%BMC_JAVA_HOME%\java" -Xmx1024m -classpath "C:\BMC\ARSystem\pluginsvr;C:\BMC\ARSystem\pluginsvr\arpluginsvr91_build007.jar; C:\BMC\ARSystem\arserver\api\lib\arcmnapp91_build007.jar" com.bmc.arsys.pluginsvr.ARPluginServerMain -x clm-aus-tt8f8w -i "C:\BMC\ARSystem" -m

    Step 1: We have to add the following to this command so it knows to load the .jar’s needed for encryption:

    C:\Oracle\JDK\lib\bmcext\*;

    This will create a command line like so:

    "%BMC_JAVA_HOME%\java" -Xmx1024m -classpath "C:\BMC\ARSystem\pluginsvr;C:\BMC\ARSystem\pluginsvr\arpluginsvr91_build007.jar;C:\Oracle\JDK\lib\bmcext\*;C:\BMC\ARSystem\arserver\api\lib\arcmnapp91_build007.jar" com.bmc.arsys.pluginsvr.ARPluginServerMain -x clm-aus-tt8f8w -i "C:\BMC\ARSystem" -m

    The * used in the CLI example is a "wildcard" to call out each file in that folder.  Either using the * or calling out the 3 files individually will work the same, with the only difference being that ALL files will be loaded with the * not just the specific ones.


    Article Number:

    000171362


    Article Type:

    Solutions to a Product Problem



      Looking for additional information?    Search BMC Support  or  Browse Knowledge Articles