10 Replies Latest reply on Feb 14, 2018 5:43 AM by Attachment Scanner

    SSO MidTier Agent Integration Failure 9.0 Patch001

    Sankeerth Jalapu
      Share:|

      Hello

       

      I am stuck with Mid tier Agent integration failing with errors that are consistent all the same.

       

      Environment:

      Windows 2012 Server (running on a Microsoft Hyper-V)

      AR/CMDB/ITSM/SRM/SLM MidTier all on 9.0 Patch 001.

      2 MidTier Servers (Load Balanced-F5);2 App Servers in server group load balanced (F5), 1 SSO Server, 1 DB server (Oracle 11G)

       

      Steps Taken So Far for SSO 9.0 patch 001 Setup:

       

      1. Installed SSO on External Tomcat.
      2. I am able to open and login to the SSO Admin Console
      3. Certificates are loaded on the SSO Box, Mid Tier Box and the F5. Verified that 8443 is accessible.
      4. Installed the Agent on both AR Servers; configured the Server info EA tab and Verified the SSO tab.
      5. Installing Agent on MidTier Failing.
      6. Added the Trust-Store as JKS
      7. Changed the Keystore to JKS
      8. On Mid TIer
        1. added Java Home (JDK),Catalina Home, JRE home (1.7.55)
        2. Added the Provider information in the Java Security File (Initially i dint but since it was complaining)
      9. Server.XML 

      <Connector port="8443" protocol="org.apache.coyote.http11.Http11NioProtocol" SSLEnabled="true"

                     maxThreads="600" scheme="https" secure="true"

                     clientAuth="false" sslProtocol="TLS"

                     maxHttpHeaderSize="49152"

                     connectionTimeout="90000"

                     keepAliveTimeout="-1"

                     acceptCount="100"              

                     keystoreFile="E:\Program Files\Apache Software Foundation\Tomcat 7.0_Tomcat7_MidTier\conf\keystore.p12"

                     keystorePass="internal4bmc"

                     keystoreType="PKCS12"

                     />

      1. I also tried adding the trustore information in the Server.XML it dint matter.
      2. I tried putting in Cipher's still exact same result.

       

      10. Ran the MidTier integration utility still no go.

       

      Installer is consistently giving one error

      FINE: No RSA provider found for type: {0}

      1. java.lang.ClassNotFoundException: com.rsa.jsafe.provider.JsafeJCE

       

      Any help is appreciated.

       

      Thanks

        • 1. Re: SSO MidTier Agent Integration Failure 9.0 Patch001
          Shrihari Salem

          Hi Keeth,

           

          Can you help in providing some more details

           

          1. Can you attach the atsso.0.log file . (From temp folder)

          2. Can you also share the mid-tier integration utility logs.

          3. Also, can I ask why is MT being configured with SSL as you are already load balancing it using F5. So you can offload all SSL traffic at the LB level and keep the traffic from LB to MT non-SSL. This would help avoiding encryption and decryption for the traffic behind LB.

           

          Thanks

          Shrihari

          1 of 1 people found this helpful
          • 2. Re: SSO MidTier Agent Integration Failure 9.0 Patch001
            Shrihari Salem

            Hi,

             

            I just had a look at the attached server.xml and I see that the <Connector> is not defined correctly.

            For an OOTB MT, I see that the default <Connector> looks similar to as below

            <!-- A "Connector" represents an endpoint by which requests are received

                     and responses are returned. Documentation at :

                     Java HTTP Connector: /docs/config/http.html (blocking & non-blocking)

                     Java AJP  Connector: /docs/config/ajp.html

                     APR (HTTP/AJP) Connector: /docs/apr.html

                     Define a non-SSL HTTP/1.1 Connector on port 8080

                -->

                  <Connector URIEncoding="UTF-8"

                             acceptCount="250"

                             connectionTimeout="90000"

                             disableUploadTimeout="true"

                             enableLookups="false"

                             maxHttpHeaderSize="8192"

                             maxKeepAliveRequests="-1"

                             maxThreads="300"

                             minSpareThreads="50"

                             port="80"

                             protocol="HTTP/1.1"

                             redirectPort="8443"/>

             

            Also, can you try accessing the MT URL using the direct hostname (not the LB URL), this way you can eliminate if the issue is with MT tomcat or with LB. Try accessing the MT direct URL http://mthostname.abc.com:8443/arsys

             

            If this is accessible then we can debug what could be the issue when accessing the LB URL.

             

            Thanks

            Shrihari

            • 3. Re: SSO MidTier Agent Integration Failure 9.0 Patch001
              Sankeerth Jalapu

              I will try the connector as specified and see if that fixes it.

               

              Also, as with the current configuration, i am able to do the below.

               

              https://<F5FQDN>:8443/ -- Works-- displays Apache Home Page

               

              https://<F5FQDN>:8443/arsys -- Wont work -- no response received error-- more like a page cannot be displayed

               

              First Web Server:

               

              http://<HostFQDN>:8080/arsys -- Works -- displays Login Page

               

              http://<HostFQDN>:8443/arsys -- Works-- displays Login Page

               

              http://<HostFQDN>:8445/arsys -- Works-- displays Login Page

               

              Second Web Server:

               

              http://<HostFQDN>:8080/arsys -- Works-- displays Login Page

               

              http://<HostFQDN>:8443/arsys -- Works-- displays Login Page

               

              http://<HostFQDN>:8445/arsys -- Works-- displays Login Page

               

              https://<F5FQDN>:8443/arsys -- doesn’t work


              Let me know what you think?


              Thanks

              S

              • 4. Re: SSO MidTier Agent Integration Failure 9.0 Patch001
                Sankeerth Jalapu

                Hello,

                 

                I think now i am able to get the SSL Offloading to work.

                 

                F5 Settings:

                 

                the F5 VIP should ONLY have a client SSL profile configured. NO server SSL profile


                Apache Tomcat Settings:


                Server.XML

                <Connector   

                                 port="8443"

                                 protocol="HTTP/1.1"

                                 connectionTimeout="20000"

                                 compression="on"

                                compressionMinSize="32"

                                  noCompressionUserAgents="gozilla, traviata"

                                  compressableMimeType="text/html,text/xml,text/javascript,application/x-javascript,text/css"

                                  redirectPort="8443"

                                 maxHttpHeaderSize="49000"

                                 maxKeepAliveRequests="-1"

                                 maxThreads="600"

                                 minSpareThreads="50" 

                                 proxyPort="8443"

                                 proxyName="F5 FQDN"

                                 scheme="https"

                                 secure="true"   />

                 

                 

                What remains is to start the agent installer on the Mid Tier and see if it works. Will keep you posted.

                 

                Thanks

                S

                1 of 1 people found this helpful
                • 5. Re: SSO MidTier Agent Integration Failure 9.0 Patch001
                  Sankeerth Jalapu

                  Ok. So i re-did the Agent install after i was able to do the Server.XML modifications and testing that i was able to get to the URL's OK.

                   

                  IT is still the same issue after all of this?

                   

                  I see the process reading all my Certs properly (from the SSO Cacerts) and then from what I understand it is trying to load these certs into the local cacerts store on the Mid-Tier box for some reason it is not able to do so. I looked at the Java Security file on Mid-Tier it says the keystore type by default is JKS.

                   

                  FINE: Selecting keystore provider for keystore type: JKS

                  Sep 02, 2015 9:48:49 AM com.bmc.atrium.sso.common.Obfuscator getKeyStoreInstance

                  FINE: No RSA provider found for type: {0}

                  1. java.lang.ClassNotFoundException: com.rsa.jsafe.provider.JsafeJCE

                   

                  Sep 02, 2015 9:48:49 AM com.bmc.atrium.sso.common.Obfuscator getKeyStoreInstance

                  FINE: No BouncyCastle provider found for type: {0}

                  1. java.security.KeyStoreException: JKS not found

                  at java.security.KeyStore.getInstance(Unknown Source)

                   

                  Sep 02, 2015 9:48:52 AM com.bmc.atrium.sso.agents.web.deployer.Main main

                  SEVERE: BMCSSG1319E: Failed deployer execution. Cause: AtriumSSOException [BMCSSG1422E: Failed call to Atrium SSO server, return code: 500.]. Contact BMC Software, Inc.

                  Sep 02, 2015 9:48:52 AM com.bmc.atrium.sso.common.AtssoLogger logStackTrace

                  SEVERE: BMCSSG1422E: Failed call to Atrium SSO server, return code: 500.

                  AtriumSSOException [BMCSSG1422E: Failed call to Atrium SSO server, return code: 500.]


                  So i am thinking, the issue is revolving around loading the certs from SSO Box to Local MT cacerts-- but question is if this is labeled as a optional step on the installer and the fact that i am not using a SSL set up in the Local box why is it still trying to load it and failing?



                  thanks

                  S

                  • 6. Re: SSO MidTier Agent Integration Failure 9.0 Patch001
                    Shrihari Salem

                    Hi,

                     

                    Are you using the AR/MT integration utility to integrate with SSO? You do not integrate with SSO Server manually.

                    You can refer following documentation for performing the same

                    https://docs.bmc.com/docs/display/public/ars9000/Integrating+BMC+Remedy+Single+Sign-On+with+BMC+Remedy+AR+System

                     

                    Also, make sure that SSO Server is accessible from the individual MT nodes.

                     

                    Thanks

                    Shrihari

                    • 7. Re: SSO MidTier Agent Integration Failure 9.0 Patch001
                      Sankeerth Jalapu

                      Yes. I did try the Integration utility also did the same thing- I mean the exact same error on the SSOintegrationutility log

                       

                      I am following this link

                       

                      https://docs.bmc.com/docs/display/public/sso90/Running+the+BMC+Atrium+Single+Sign-On+Installer+on+BMC+Remedy+Mid+Tier

                       

                      thanks

                      S

                      • 8. Re: SSO MidTier Agent Integration Failure 9.0 Patch001
                        Eugen Degraf

                        Hi Keeth,

                         

                        can you please try following.

                         

                        1.     Log in to SSO Admin Console

                        2.     Open "Edit Server Configuration"

                        3.     Go to Tab "Certificate" and choose "Truststore" from Dropdown Menu.

                        4.     Is your Midtier Certificate available in this Truststore ?

                        5.     Try to remove midtier Certificate (where SSO Agent will be installed) of the truststore, it should be added during installation.

                         

                         

                        Please check also if filter of ASSO is uncommented in web.xml within folder  <DISK>:\Program Files\BMC Software\ARSystem\midtier\WEB-INF

                         

                        Remove above and below of ASSO Filter the comment sign <!-- and -->

                         

                         

                         

                        Is the SSO Agent shown in Atrium SSO Adminconsole within "Agent Details" after Midtier Installation ?

                         

                        Regards,

                        Eugen

                        • 9. Re: SSO MidTier Agent Integration Failure 9.0 Patch001
                          Sankeerth Jalapu

                          Thank you Eugen

                           

                          I validated your steps.

                           

                          1. I was able to login to the ASSO Admin Console looked up the Truststore -- Mid Tier Cert was PRESENT, so i removed it from the truststore as advised.

                           

                          2. I also verified that JVM cacerts on  the Mid Tier box dont have Mid Tier certs loaded.

                           

                          3. Ran the installer, it presented the same issue.

                           

                          Sep 16, 2015 2:53:37 PM com.bmc.atrium.sso.sdk.impl.BaseREST sendRequest

                          WARNING: java.io.IOException: Message is blocked due to security concerns

                                  at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

                                  at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)

                                  at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)

                                  at java.lang.reflect.Constructor.newInstance(Unknown Source)

                                  at sun.net.www.protocol.http.HttpURLConnection$6.run(Unknown Source)

                                  at sun.net.www.protocol.http.HttpURLConnection$6.run(Unknown Source)

                                  at java.security.AccessController.doPrivileged(Native Method)

                                  at sun.net.www.protocol.http.HttpURLConnection.getChainedException(Unknown Source)

                                  at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)

                                  at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(Unknown Source)

                                  at com.bmc.atrium.sso.sdk.impl.BaseREST.sendRequest(Unknown Source)

                                  at com.bmc.atrium.sso.sdk.impl.BaseREST.send(Unknown Source)

                                  at com.bmc.atrium.sso.sdk.impl.BaseREST.sendRequest(Unknown Source)

                                  at com.bmc.atrium.sso.sdk.impl.BaseREST.sendPost(Unknown Source)

                                  at com.bmc.atrium.sso.sdk.impl.CertREST.putCertificate(Unknown Source)

                                  at com.bmc.atrium.sso.sdk.impl.IdentityImpl.addIdentity(Unknown Source)

                                  at com.bmc.atrium.sso.agents.web.deployer.Generic$Installer.uploadAgentCert(Unknown Source)

                                  at com.bmc.atrium.sso.agents.web.deployer.Generic$Installer.execute(Unknown Source)

                                  at com.bmc.atrium.sso.agents.web.deployer.Tomcat6$Installer.execute(Unknown Source)

                                  at com.bmc.atrium.sso.agents.web.deployer.BaseWorker.run(Unknown Source)

                                  at com.bmc.atrium.sso.agents.web.deployer.Main.run(Unknown Source)

                                  at com.bmc.atrium.sso.agents.web.deployer.Main.main(Unknown Source)

                          Caused by: java.io.IOException: Message is blocked due to security concerns

                                  at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)

                                  at java.net.HttpURLConnection.getResponseCode(Unknown Source)

                                  at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(Unknown Source)

                                  at com.bmc.atrium.sso.sdk.impl.BaseREST.sendRequest(Unknown Source)

                                  ... 11 more

                           

                           

                          Sep 16, 2015 2:53:37 PM com.bmc.atrium.sso.agents.web.deployer.Main main

                          SEVERE: BMCSSG1319E: Failed deployer execution. Cause: AtriumSSOException [BMCSSG1422E: Failed call to Atrium SSO server, return code: 500.]. Contact BMC Software, Inc.

                          Sep 16, 2015 2:53:37 PM com.bmc.atrium.sso.common.AtssoLogger logStackTrace

                          SEVERE: BMCSSG1422E: Failed call to Atrium SSO server, return code: 500.

                          AtriumSSOException [BMCSSG1422E: Failed call to Atrium SSO server, return code: 500.]

                                  at com.bmc.atrium.sso.sdk.impl.BaseREST.sendRequest(Unknown Source)

                                  at com.bmc.atrium.sso.sdk.impl.BaseREST.send(Unknown Source)

                                  at com.bmc.atrium.sso.sdk.impl.BaseREST.sendRequest(Unknown Source)

                                  at com.bmc.atrium.sso.sdk.impl.BaseREST.sendPost(Unknown Source)

                                  at com.bmc.atrium.sso.sdk.impl.CertREST.putCertificate(Unknown Source)

                                  at com.bmc.atrium.sso.sdk.impl.IdentityImpl.addIdentity(Unknown Source)

                                  at com.bmc.atrium.sso.agents.web.deployer.Generic$Installer.uploadAgentCert(Unknown Source)

                                  at com.bmc.atrium.sso.agents.web.deployer.Generic$Installer.execute(Unknown Source)

                                  at com.bmc.atrium.sso.agents.web.deployer.Tomcat6$Installer.execute(Unknown Source)

                                  at com.bmc.atrium.sso.agents.web.deployer.BaseWorker.run(Unknown Source)

                                  at com.bmc.atrium.sso.agents.web.deployer.Main.run(Unknown Source)

                                  at com.bmc.atrium.sso.agents.web.deployer.Main.main(Unknown Source)

                           

                           

                          Sep 16, 2015 2:53:37 PM com.bmc.atrium.sso.agents.web.deployer.Main main

                          INFO: BMCSSG1318I: Deployer execution completed.

                           

                          4. I looked at the contents of the Trusstore on the Mid Tier box after i ran the deployer and saw that the process has created a cert with alias asso -- however the cert is my CA server details.

                           

                          What i am thinking is that the install process is having troubles with loading the certs received from the ASSO server into the truststore on the Mid Tier box.

                           

                          Thank you for your time and advise..

                          • 10. Re: SSO MidTier Agent Integration Failure 9.0 Patch001
                            Attachment Scanner

                            UPDATE: Based on File attachment policies, attached files were removed, see FAQ for more