DWP/DWP Catalog- SSL Troubleshooting

Version 221
    Share This:

    This document contains official content from the BMC Software Knowledge Base. It is automatically updated when the knowledge article is modified.


    PRODUCT:

    BMC Digital Workplace Advanced


    COMPONENT:

    MyIT Service Broker


    APPLIES TO:

    DWP Catalog and Advanced 18.00, 19.00, 20.00



    DETAILS:



    BEST PRACTICE:

    Offload SSL through loadbalancer instead.


    POSSIBLE ROOT CAUSES:

       
    • Bad certificate: missing signature: root/intermediate certificate not present, DNS/hosts entries are missing: Please, make sure you have at least hostname, fqdn in the dns entries, if you have an alias you should have aliashostname and aliasfqdn.
    •  
    • Expired certificate
    •  
    • When using wildcard certificates confirm it has all the san entries, i.e.     
           
      • My company has the following domains: mycompany.dwp.com, mycompany.dwp.mx, mycompany.dwp.org, alter.mycompany.dwp.com.. dwp.mycompany.alternative.com,
      •    
      • Then multiple wildcard have to be declared when issuing a wildcard certificate: *.dwp.com, *.dwp.mx, *.dwp.org, *.mycompany.dwp.com, *.mycompany.alternative.com
      •   
    •  
    • If the  DWP/DWPC Loadbalancer has a cert, make sure to import the same to DWP and DWP C Servers. That will also apply for RSSO, Midtier, SmartIT  loadbalancers.
    •  
    • Server A cannot resolve dns entries from Server B or/and viceversa: This can be managed vi DNS Server or by adding entries under server's hosts file
    •  
    • SSL port blocked
    •  
    • If you have a DWP Catalog Cluster, make sure all the DNS entries are present in the DWPCatalog.jks/Keystore. Best practice would be creating the keystore as follows:
      
      keytool -genkey -alias DWPC -keyalg RSA  -keystore DWPCatalog.jks  -dname  "cn=ServergroupName/LoadBalancerName"  -storepass keystorepass -deststoretype JKS -ext san=dns:loadbalancerfqdn,dns:server1,dns:server1fqdn,dns:server2,dns:server2fqdn,dns:server3,dns:server3fqdn
      
       
    • Loadbalancer redirection issue
    •  
    • Non root user trying to start a process on port 80/443 which can only be used by root users. Non root users cannot run applications on port 80/443. 
    •  
    • Several steps from the below documentation were skipped. Make sure that the DWP Catalog .crt file is imported into the DWP Adv keystore, as mentioned here:
         
    RESOLVING THE ISSUE:  
       
    • In order to fix the PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target error on  the DWP Advanced - Enhanced Catalog  Section, do the following     
           
      • NOTE: Had one case on which, even though the certs were right, VIP and LB balancer settings were off, and customer was still getting above errors when using DWP Catalog LB URL under DWP Admin Enhanced Catalog section. As a workaround, we had to use DWP C primary server URL instead.
      •   
    •  
    • Get the cert from DWP Catalog by using Firefox
      
      User-added image
      
       
    • If you have DWP Advanced Cluster, make sure all the server group nodes have the DWP Catalog Cert. Make sure you do the same on DWP, AR Server, SmartIT and RSSO servers.
    •  
    • Import the certificate to the cacerts by using below commands
      
       
    1. openssl x509 -outform der -in catalogcert.pem -out certificate.der
    2.  
    3. keytool -import -alias DWPCatalog -trustcacerts cacerts -file certificate.der
      

     Note: It can be java cacerts or a custom one

      
       
    • Restart Applications Tomcat / AR server service
    •  
    • Clear browser cache
    •  
    •  Log into the DWP Admin Console.
    •  
    • Go to the Enhanced Catalog section and update the DWP Catalog url with the right port and protocol.     
           
      • Do the same from the DWP Advanced LB URL and from each node individually to confirm this works.
      •   
    •  
    • Do the same on the AR Server's SB:RemoteApprovalConfiguration and Centralized Configuration com.bmc.itsm.sbe
      
      User-added image
      
       
    • Click on Save
    •  
    • Confirm that you are not getting the error anymore
    •  
    •  If you get the error proceed with SSL Poke Troubleshooting section below
      


    SSL POKE

      
       
    • Download SSL poke and run it on DWP Advanced Server. SSL Poke is attached to this KA.
      
      Consider you will have to repeat this from AR Server, RSSO, SmartIT Servers to DWP Catalog and viceversa.
      
       
    • On a cmd/terminal Windows Run:
      
      java SSLPoke DWPCatalogServerFQDN SSLPort
    Example:
    java SSLPoke dwpcatalogservername.com 8443
      
       
    • If Firewall allows and you have imported a good cert to DWP Server, you should get a message like "Connection was successful" 
    •  
    • If it fails, it should give you the reason why this is not working. See below:
    •  
    • SSL Port is blocked
    •  
    • Certificate does not exist in the keystore/truststore. Check it exists by running
      
                   keytool -list -v -keystore keystore.jks  
      
       
    • Certificate chain/Root-intermediate cert is missing
      
      Check this with your CA/Security Team or go through below doc.
    https://www.sslshopper.com/article-most-common-java-keytool-keystore-commands.html
      
       
    • Server alias/dns entry for this catalog does not exist in the certificate
      
        keytool -list -v -keystore keystore.jks -alias mydomain 
        mydomain should be replaced with dwpcatalogFQDN
      
       
    • Firwewall/Loabalancer settings need to be revisited
    •  
    • Certificate is expired. Check validity
      

                keytool -list -v -alias <alias> -keystore <keystore> -storepass <passwd> alias mydomain | grep "Valid from:"

      

    DWP Catalog checks.

    You can make use of attached .ppt file to crosscheck your config vs ppt.

       
      a) Make sure that you used FQDN where it tells you to use FQDN and not the hostname on <installLocation>/sb/rxscripts/bin/setenv.sh
    b) Make sure that you used hostname where it tells you to use hostname and not FQDN, in <installLocation>/sb/env/set_script_variables.sh
    c) Do not use localhost in the files that we just mentioned above.
    d) Make sure that you update the below section on <installLocation>/jetty/etc/jetty-http.xml  and put the right/actual keystore and trusttore.
          
       
    <New id="sslContextFactory" class="org.eclipse.jetty.util.ssl.SslContextFactory"> <Set name="KeyStorePath">/opt/bmc/digitalworkplace/certs/keystore</Set> <Set name="KeyManagerPassword">OBF:1v2j1uum1xtv1zej1zer1xtn1uvk1v1v</Set> <Set name="KeyStorePassword">OBF:1v2j1uum1xtv1zej1zer1xtn1uvk1v1v</Set> <Set name="TrustStorePath">/opt/bmc/digitalworkplace/truststore/cacerts</Set> <Set name="TrustStorePassword">OBF:1v2j1uum1xtv1zej1zer1xtn1uvk1v1v</Set> <Set name="IncludeCipherSuites">
      
      
     
     e) Make sure ar.conf has below values 
      
       
    • Jetty-Port: 443
    •  
    • Jetty-Protocol: HTTPS
      
     
    f)If any changes, restart dwpcontroller and confirm this is now working by running below on another terminal:  
      
      
       netstat -tulpn | grep LISTEN | grep 443;    lsof -i :443
      
     
    You should be able to see something like  
     
    [root@xxxx work]#     netstat -tulpn | grep LISTEN  | grep 443
      
      tcp6       0      0 :::443                  :::*                    LISTEN      17353/java 
      
     
    [root@xxxxxxxx]#   lsof -i :443
      
      COMMAND   PID USER   FD   TYPE  DEVICE SIZE/OFF NODE NAME  
    java    17169 root   16u  IPv6 4871373      0t0  TCP localhost:41076->localhost:https (CLOSE_WAIT)  
    java    17169 root   17u  IPv6 4871157      0t0  TCP localhost:41070->localhost:https (CLOSE_WAIT)  
    java    17169 root   18u  IPv6 4872364      0t0  TCP localhost:41082->localhost:https (CLOSE_WAIT)  
    java    17169 root   19u  IPv6 4872436      0t0  TCP localhost:41122->localhost:https (CLOSE_WAIT)  
    java    17353 root   32u  IPv6 4855123      0t0  TCP *:https (LISTEN) 
      
     
    And you can also confirm these processes are running by taking the PID and using below command  
      
      
      [root@xxxxx]#    ps -aux | grep 17169 
     
    [root@xxxxxxx]#    ps -aux | grep 17169;ps -aux | grep 17353  
     
    root     17169  0.2  0.7 6910620 118908 pts/1  Sl   12:42   0:08 /apps/java/bin/java -jar /apps/dwpc/bin/healthcheck-1.0-shaded.jar /apps/dwpc/bin/monitor.properties  
    root     27997  0.0  0.0 112716   972 pts/1    S+   13:32   0:00 grep --color=auto 17169 
      


               Jetty <DWPCatalogInstallDir>/sb/jetty/work/jersey.0.0.log should show the below entries

               cat jersey.0.0.log | grep -i ssl

     

      

            at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:426)
            at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:320)
            at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:158)

      

                <DWPCatalogInstallDir>//sb/rxscripts/trace.log will give you details too
     

      

    ./sb/rxscripts/trace.log:1:== Info: About to connect() to xxxxxx port 443 (#0)
    ./sb/rxscripts/trace.log:3:== Info: Connected to xxx (127.0.1.1) port 443 (#0)
    ./sb/rxscripts/trace.log:8:== Info:     subject: CN=cxxxx
    ./sb/rxscripts/trace.log:11:== Info:    common name: xxxxx
    ./sb/rxscripts/trace.log:12:== Info:    issuer: CN=xxxxx
    ./sb/rxscripts/trace.log:19:00e0: Host: xxxxxx
    ./sb/rxscripts/trace.log:58:== Info: Added cookie _cacheId="89" for domain xxxx, path /, expire 0

      




    Now, both servers should be able to communicate to each other with no issues.


    For user group sync issues not working over SSL,
    KA 000133376  DWP Catalog - User Group sync script doesn't work with SSL

    For AD  over SSL issues
    KA 000164451  DWP Catalog - Configuring AD Connector on DWP Catalog. Includes AD over SSL

     


    Article Number:

    000229680


    Article Type:

    Product/Service Description



      Looking for additional information?    Search BMC Support  or  Browse Knowledge Articles