With the release of True Sight 11.0 (September 2017) came support for Remedy Single Sign-On and more authentication methods. In Atrium Single Sign-on True Sight was limited to using local and LDAP authentication. With Remedy Single Sign-On you now have the option of using Kerberos, SAML and Certificate Based Authentications.
In this post we'll take a look at configuring both Kerberos and SAML ADFS to Authenticate Trusight users. We'll also take a brief look at Certificate Based Authentication but this will be covered in more detail in a separate post. We'll also take a look at how to troubleshooting both Kerberos and SAML Authentications.
Versions being used in this post
Kerberos KDC/LDAP/DC Windows 2012
We'll start with a high level description of Kerberos, SAML and Certifcate Based Authentications
Kerberos since Windows 2000 is the default authentication protocol for windows (NTLM was the default before this), every release of Windows desktop or server since includes a Kerberos client provider. One of the main benefits of using Kerberos as an authentication method besides being more secure than NTLM is the ability to seamlessly authenticate with applications, meaning once a user has authenticated with the domain and the KDC (Key Distribution Center) provides the user with a ticket to prove identity, the user will not need to authenticate themselves to any Kerberos enabled applications. Generally Kerberos will not work unless a user is logged into a domain within the intranet or VPN.
SAML Authentication (ADFS)
SAML (Security Assertion Mark up Language) is a is an Extensible Markup Language (XML) standard that allows a user to log on once for affiliated but separate web applications.
The purpose of SAML is to provide centralised management of userid and password to allow access to applications from different vendors. SAML is a standard for logging users into applications based on their sessions in another context. This single sign-on (SSO) login standard has significant advantages over logging in using a username/password, SAML can be and is usually configured for cross domain authentication. Microsoft's implementation of SAML is Microsoft Active Directory Federation Services (ADFS) is is an extension of Active Directory. While active directory serves to contain user identification, authentication and authorisation within its own organisation and domain boundaries, its extension Federation Services can be used to cross these boundaries. ADFS can be configured to make use of Kerberos seamless authentication also.
Certificate Based Authentication
Certificate-based authentication uses a Digital certificate (X.509 certificates) to identify a user, machine, or device before granting access to a resource, network, application, etc. it is often deployed in coordination with traditional methods such as username and password. Certificates are installed on a user's device and is only valid for that user on the device. A user can not move a certificate form one device to another, if they want to use a different device they will need to request a new user certificate for that device.
Which Authentication Method Should I use?
The simple answer is you won't generally have a choice. This will be dictated by the network and security admins and whatever authentication method they have put in place would be the one you use. We'll go through a few pro's and cons of these authentication methods. Kerberos out of the three would probably be the easiest and quickest one to configure, in terms of Remedy Single Sign-On all you will need is the KDC server name, a principle account and password. Kerberos although can be configured for cross domain authentication it usually is not, and will only work if you have logged in to the domain either via the internal network or VPN. SAML and certificate based authentication can be configured for cross domain authentication, but are more difficult to configure.
NOTE: The above Auth methods are only supported by the web components of TS. The java admin UI and the TS mobile app do not support these authentication protocols, therefore the Local Authentication option will need to be used. See comment section below for further updates regarding the authentication for the admin console.
NOTE: If you have configured BMC applications such as Remedy MT or MyIT with one of the authentication methods above, you would have noticed all you need to do is configure the Kerberos/SAML/Certificate authentication to be able to login to the application. True Sight works differently, with these authentication methods you will need to configure a separate authentication along side LDAP. Why is this the case? In a nutshell the answer is Groups and Authorisation, RSSO does Authentication not Authorisation.
RSSO goes to the IDP for authentication of a user, it then creates the related tokens and sessions and passes this over to the calling application, it is at the application Authorisation is done and the user is given the privileges they are entitled to for that particular application i.e. Operator for Truesight, Asset Viewer for Remedy ITSM to give two examples.
For Truesight we'll need to search the LDAP server to determine which groups the user belongs to.
1. The the initial Authentication (Kerberos, SAML, Certificate Based Authentications) will authenticate the user and provide the ability for users to login to applications without a user name and password once logged in.
2. LDAP provides the users groups
Taking the need to have LDAP configured we will take a look at configuring this first and then move on to do the configurations for the kerberos/SAML(ADFS) and Certificate Based Authentications. We will touch lightly on certificate Authentication in the post. A more detailed post will be published to delve deeper into configuring and troubleshooting Certificate Based Authentication.
Our first step is to create an LDAP authentication in RSSO, this will predominantly be used to fetch group membership of the users after the initial authentication has been done with the primary authentication method Kerberos/ADFS or Certificate Base Authentication. A more detailed post of LDAP configuration can be found at ConfiguringRSSOLDAP
1. Login to the RSSO admin console edit the realm you want to configure for Authentication (in this instance we'll be using the default " * " realm)
2. For the application domain we are going to add "clm-aus-021983.bmc.com" which is the FQDN to our TSOM server i.e. http://clm-aus-021983.bmc.com:444
3. On the Authentication tab select "LDAP" from the "Authentication Type" drop down list, select "Active Directory" from the "Preset" drop down list, fill out the rest of the LDAP server information and check "Enable Group Retrieval" check box. Hit the test tab and confirm connection to the LDAP server is successful(fig1)
Fig1. Shows the LDAP configuration and successful test connection to the server. For a detailed description of LDAP configuration see ConfiguringRSSOLDAP
4. Once we have LDAP configured we need to do a test login to make sure that we are able to login to TSOM and ensure the user's group list is also being returned successfully. We'll enable
the RSSO server logs to see if the groups are returning successfully. Go to the General tab---> Basic-->Set Server log level to "DEBUG" (Don't forget to put this setting back to "INFO" after)
5. Do a test login to the TSOM server by entering the url in a browser in this case http://clm-aus-021983.bmc.com:444 if everything is working you will be redirected to the RSSO login page
6. Enter in an LDAP user name and password, if successful you will then login to the TSOM dasboard.
At login if you see:
"Authentication Failed" This means either the user name or password is incorrect or the user does not exist in the LDAP. Check the user name and password and the LDAP config in RSSO admin console, see ConfiguringRSSOLDAP for more information on RSSO LDAP configuration
"You are not Authorised to view this page" This means RSSO has been able to authenticate the user. The group the user belongs to is either not being brought back by the LDAP query, the group configuration in RSSO is incorrect or user profile mapping in TS has not been configured, see ConfiguringRSSOLDAP for more information on RSSO LDAP configuration which goes through the users and group searches for LDAP in RSSO.
To confirm if the group is being brought back from the LDAP server, login to the RSSO server and open the RSSO server log file "tomcat/logs/rsso.0.log" search the log for:
"Searching groups by user '<UserName>' with filter" i.e " Searching groups by user 'JCKER' with filter "
Further down in the log you will see a list of groups the user belongs to in LDAP
"All returned entries: " i.e. "All returned entries: [Domain Admins, Operators, Administrators]"
Note: You might see a partial result exception message in the log, this is fine and normally indicates a possible referral in LDAP
Now that we have LDAP configured and confirmed login to TSOM. We will take a look at configuring Kerberos, SAML ADFS & Certificate Based Authentications with RSSO
Summary of of Kerberos configuration with RSSO
1. Create the service principle on the KDC server (Domain Admin function)
2. Enable chaining mode for RSSO
3. Add Kerberos Authentication type
4. Configure browser for automatic login
Create The Service Principle
The first step in configuring Kerberos with Remedy Single Sign-On is to create the Service Principle User. This normally would be done by the Domain Administrator ask the admin to create a user account and map the RSSO service it or add the RSSO service to a pre-existing Service Principle Account. To do this they will need to use the setspn command from the command line.
Example: We will create a user in AD called "RSSOPRIN" and will map our RSSO service will be the FQDN of the RSSO server can also be a Load Balancer url, in this example or RSSO servers are behind an LB with the following url http://rssolb.bmc.com
To create the SPN in this example we will run the following from the command line on the Domain Controller (KDC) (fig2)
setspn -S HTTP/rssolb.bmc.com RSSOPRIN
Other useful setspn commands:
setspn -X - Check to see wether there are duplicate services in the. Useful when you get a message that duplicates are found when creating services
setspn -d <servicename> <USER - Deletes a service that is mapped to a user
setspn -L <USER> - Lists service registered to the Service Principle User
RSSO has the ability to connect to the KDC directly by way if Service Principle user name and password, however some Domain Controllers(KDC) do not allow direct connection, if this is the case then you can use a keytab file to load into RSSO.
To create a keytab file after you have created the Service Principle User and mapped the service to it run the following command from the command line of the DC
ktpass -out <file> -mapuser <user> -princ HTTP/<host>@<DOMAIN> -pass <password> -ptype KRB5_NT_PRINCIPAL -target <DOMAIN> -knvo 0
In our example we run the following (fig 3) where the key tab file will be created in c:\temp directory. Move the keytab file in to a location on the RSSO server you will need this path and file name later.
ktpass -out c:\temp\RSSOKeytab -mapuser RSSOPRIN -princ HTTPemail@example.com -pass Mypassword -ptype KRB5_NT_PRINCIPAL -target sso.bmc.com
Now that we have created the Service Principle User we'll continue to configure RSSO to authenticate with Kerberos
Configuring RSSO for Kerberos
1. Login to the RSSO admin console and edit the realm created earlier that has the LDAP configuration
2. Go to "Authentication" tab
3. Click on "Enable Chaning Mode"
4. Click "Add Authentication"
5. From the Authentication Type select "Kerberos"
6. Enter the FQDN name of the KDC server
7. Enter the Service Principle user name
8. If you are using a keytab file enter the path to and file name of the keytab file. If you are using direct connection to the server select the "SPN Password" radio button and enter the password
9. Leave User ID format & User ID Transformation (We'll cover this some more below)
10. Test the connection (fig4)
If successful you will see a "Kerberos Connection Successful message" if the connection fails review KA000130918 in BMC Knowledge base to troubleshoot further. If you are sure the connection details are correct review the RSSO server logs and ask the domain admin to check the Windows event log on the Domain Controller, this will usually give a clue on why the connection failed.
11. Save the Authentication configuration (top right of screen)
12. In the List of Authentication Screen move Kerberos Authentication up to first in the list(Fig 5) (otherwise the save button will be greyed out)
13. Click save (bottom right of screen)
Fig5. Shows the chaining mode Kerberos Authentication needs to be first on the list
NOTE ON USER TRANSFORMATIONS: For Trusesight you won't need to worry about having the user transformation, however if this realm is going to also serve other applications such as Remedy and MyIT you will need to select and option for "USERID Format" & "USER ID Transformation" from the drop down list, depending on how your users are configured in the application.
Before we can test the login we will need to configure the browsers to use auto login. This is normally done by group Windows policy where the configuration is sent to each end user's browsers. We will take a look at how to configure the browsers for testing purposes
Configure Browser For automatic Login
Internet Explorer & Chrome: Chrome will take is configurations from internet explorer. Once you have configured IE there is no need to do it on Chrome.
1. Open IE and go to "Internet Options" -->Security-->Select Intranet--->Custom Level-->Scroll all the way down to the bottom and select "Automatically Logon for Intranet Zone" (Fig6)
2. Press OK
3. Clear the browser cache (not really necessary for end user but testing its good practice) & close the browser
4. Open the browser and go to the application URL in our case TSOM "https://clm-aus-021983.bmc.com:444/"
5. If everything is working as expected you will not be prompted for user name and password and you should see the TSOM dashboard with the user name at the top right of the screen (Fig8)
1. Open the FF browser
2. In the url bar type "about:config" click "Accept the Risk" on the warning message
3. In the Search bar type "network.negotiate-auth.trusted-uris"
4. Add the domain name to the trusted uri list (fig7)
5. Clear the browser cache (not really necessary for end user but testing its good practice) & close the browser
6. Open the browser and to the application URL in our case TSOM "https://clm-aus-021983.bmc.com:444/"
7. If everything is working as expected you will not be prompted for user name and password and you should see the TSOM dashboard with the user name at the top right of the screen (Fig)
If you are getting a windows prompt for user name and password while trying to login this generally means your browser is not configured correctly, review the browser configuration and perhaps speak to the network/domain administrator to see if there are any specific group policies on the browsers.
Summary of of ADFS configuration with RSSO
1. Create signing keystore using Java keytool, Sign certificate, Import the certificate into the keystore (optional only needs to be done if your ADFS IDP needs signing speak to the ADFS admin)
2. Enter the Service Provider information (The service provider in this instance is RSSO)
3. Enable chaining mode for RSSO
4. Import the ADFS IDP meta data file
5. Send the SP meta data to the ADFS admin to create the relying party trust and the claim rules
Creating The Signing Keystore
This step is optional depending if requests that goes to the IDP needs signing. Speak to the ADFS admins to confirm this.
1. On the RSSO server ensure that you have "java/jre/bin" in the environment variable path and you are able to run the "keytool" command from any location on the system.
Open a command prompt, "cd" to the location where you want to create the keystore. We can create the signing keystore in any directory in this case we will create it in the "tomcat/conf/" directory
2. To create the keystore we will use the keytool command
keytool -keystore <keystorefile> -genkey -alias <aliasname> -keyalg RSA -sigalg SHA256withRSA -keysize 2048 -validity 730
keytool -keystore saml.jks -genkey -alias sp-signing -keyalg RSA -sigalg SHA256withRSA -keysize 2048 -validity 730
Fill in the details requested (fig9)
The above command creates a keystore file named saml.jks that contains a keypair with the alias as "sp-signing"
3. Export the certificate signing request from the keystore (csr) with the following command
keytool -certreq -keyalg RSA -alias myalias -file certreq.csr -keystore c:\yoursite.mykeystore
keytool -certreq -keyalg RSA -alias sp-signing -file certreq.csr -keystore saml.jks
This command will create a file called certreq.csr in the directory (fig10)
4. Send the .csr file to the Certificate Authority for Signing
5. When the signed certificate comes back it will either be in .cer or .p7b format. Use the following keytool command to import the signed certificate into the signing keystore
keytool -importcert -alias <alias_name> -keyalg RSA -keystore <keystore_file> -storepass <keystore_password> -file <signed_cert_file>
keytool -importcert -alias sp-signing -keyalg RSA -keystore saml.jks -storepass internal4bmc -file certnew.p7b
Fig11. Shows the import of the signed certificate
Configure The Service Provider Information
6. We now need to create the Service Provider configuration (RSSO) login to the RSSO admin console -->Advance tab. In the "SAML Service Provider" section fill out the entries (Fig12)
SP Entity ID: This can be anything, best practice here is to name it something that describes the what the service provider is. Don't uses space since the IDP (ADFS) will use this EntityID
External URL: This is the service url to RSSO with the "/rsso" at the end. It will either be the FQDN of the RSSO server or the FQDN of the LB that is servicing RSSO
KeyStore File: File and path to the keystore that holds the signing certificate. Include both the path and the keystore name.
Keystore Password: Keystore password for the keystore
Signing Key Alias: This is the alias name created when creating the keystore
Fig12. Shows the configuration of the example we are using to configure the SAML service provider
7. Restart the RSSO tomcat service to ensure the keystore get loaded correctly before moving on. If everything is OK with loading the keystore you will see the following messages in the RSSO server log.
com.bmc.rsso.core.cert.KeystoreManager.getKeystore(): Keystore type is not specified. Analyzing keystore file extension: saml.jks
com.bmc.rsso.core.cert.KeystoreManager.getKeystore(): Keystore type defined as: JKS
INFO Thread_23 com.bmc.rsso.core.cert.KeystoreManager.getKeystore(): Found keystore: C:\Program Files\Apache Software Foundation RSSO\apache-tomcat8.5.23\webapps\rsso\WEB-INF\classes\saml.jks
If there are any errors with loading the keystore you won't be able to move on to the next portion of the configuration. The error message in the log should be enough to tell you why the keystore has failed to load correctly i.e. "Invalid keystore password" , "Keystore Not found"
Importing ADFS (IDP) Meta Data
8. The next step is to import the IDP meta data in the RSSO admin console click Realm--->Edit the Realm that has LDAP configured
9. Click "Enable Chaining Mode" Click "Add Authentication"
10. Select "SAML" from the "Authentication Type" drop down list
11. At this stage we will need to either use the URL to the IDP meta data or the .xml file of the meta data. Depending on how ADFS has been configured you can get to the IDP meta data by
using the url ""https://ADFSserver.FQDN.com/FederationMetadata/2007-06/FederationMetadata.xml " or have the ADFS Admin send you the .xml file of the IDP meta data
12. In the "Import IDP Provider" screen select either to "Import from URL" in which case give the meta Data URL or "Import from Local File" in which case browse to the meta data .xlm file (fig13)
13. Click Import. If the meta data was able to import correctly you should get a successful message (fig14)
If the meta data fails to import correctly, then check with the ADFS admin to see if there are any issues connecting to the meta data url (if using the URL method) if you can get to the meta data url from a browser with url using the "https://ADFSserver.FQDN.com/FederationMetadata/2007-06/FederationMetadata.xml " url, then save the meta data locally and use the import from file options (some ADFS servers are configured to not allow direct url access)
14. After the importing of the IDP data, you will need to check some fields are correct before saving the configuration
User ID Attribute: This is the LDAP attribute used for the user name default for ADFS is "sAMAccountName"
Sign Request: If the IDP does not require signing then unccheck this box. If the IDP requires signing ensure that this box is checked and that you have configured the keystore configuration as above.
User Transformation: If this realm is also being used by another application such as Remedy and MyIT, you might need to select one of the user transformations depending how the usernames are configured in the Remedy ITSM people form.
15. Save the configuration (Top right hand of the screen)
16. In the List of Authentication screen push "SAML" to the top of the Authentication list and hit "Save" (fig14)
17. We now have to provide the Service Provider (RSSO) meta data to the IDP admins. Edit the SAML configuration
18. Click on "View Meta Data" this will show the meta data for the service provider (RSSO). You can either send the IDP admin the url or save the page to an .xml file and send that to them.
Creating The Relying Party Trust
19. The next step in the process is a ADFS admin function and should be done by the ADFS Admin. After either receiving the SP meta data url or .xml file the ADFS admin will create the relying party trust and create the claims rule. Send the ADFS admin the following url to follow Configuring Relying Party Trust (Valid for ADFS 2.0 & 3.0)
20. After the IDP has been configured you can now test the login. Open a browser and go to the TSPS url. Depending on gow ADFS is configured you will either login automatically (no username/password required) or you will get the ADFS login screen, put in your username and password to login (fig15)
Troubleshooting ADFS login problem will be discussed in more details in a future post, but there are a few places we can have a look at to see where login issues might be
1. RSSO server log. On the RSSO admin console go to the General tab--->Basic and set "Server Log Level" to "Debug" reproduce this issue again, check the logs for any error messages and contact BMC support with the logs if you still can not figure out what the issue is.
2. Check the RSSO user sessions list, if the user is listed there, this mean RSSO Authentication was successful with ADFS, further troubleshooting needs to be done on the application end (in this case TSPS), check the application logs.
3. If there are no error messages in RSSO server log, this generally means ADFS has not returned back to RSSO. In this case ask the ADFS admin you check the ADFS Admin logs to see if that give a clue to what the problem is, do a filter search for the relying party trust of RSSO (fig16)
CERTIFICATE BASED AUTHENTICATION
Certificate based authentication will be configured in the same process order as Kerberos and SAML above i.e. configure LDAP, Test and then configure the cert based authentication. The whole process will be covered in more details in a later post. But in summary what needs to be done is
1. Create the truststore and sign the certificate
2. Configure RSSO to use Certificate based information you can find details at Certificate-based authentication process
3. Depending on the device being used, Common Access Card or key you will need to deploy the user certificate to this device
4. You can also use software based Certificate Access, in this case the user certificate will be installed on user's local certificate store
5. Test the login you should be prompted to select a certificate before login can occur (fig17)
For more information on end to end configuration of certificate base information see Certificate-based authentication process
When troubleshooting the login process, start with LDAP authentication first and confirm that you can login with an LDAP user, see ConfiguringRSSOLDAP for detailed troubleshooting notes
Once LDAP is confirmed then continue with the other authentication method (Kerberos, SAML Certificate Based Authentication). Turn on the RSSO server logs to DEBUG and check the logs.
Trying to figure out where the failure is can be quite confusing, but here are a few things to remember
1. If a user Session for the user exists in the RSSO admin sessions list then the Authentication process has been successful. The next step would be to check the application logs
2. For ADFS Authentication, if you see a call to ADFS in the RSSO server logs like
INFO Thread_33 com.bmc.rsso.servlet.saml.AuthnRequestServletSaml.processRequest():  User is redirected to IdP login url:https://clm-aus-022742.sso.bmc.com/adfs/ls/?SAMLRequest=nZNPj9owEMW%2FSuR7CHFSYC1AoqCqSNtuRNIe9rJynKFryX9Sj7O7%2FfZ1AiwcEAd8HI9f3vvNZI5cq5atOv9qdvC3A%2FTRh1YG2XCxIJ0zzHKUyAzXgMwLVq5%2BPDI6GrPWWW%2BFVSTabhbkZTKlPEvhIaYPkMU5pSKejRse75usrid0NsvzmkS%2FwaG0ZkGCAomunC1iB1uDnhsfusbpNE5pnM4qStmXlOWTUZ5mzyTaBK%2FScD%2BIvXrfIk
And there is no response in the log like
com.bmc.rsso.commons.XMLUtil.removeNodes(): SAML document after removing nodes: <?xml version="1.0" encoding="UTF-8" standalone="no"?>
<samlp:Response xmlns:samlp="urn:oasis:names:tc:SAML:2.0:protocol" Consent="urn:oasis:names:tc:SAML:2.0:consent:unspecified" Destination="https://rssolb.bmc.com:8443/rsso/receiver" ID="_4e6f17a5-d653-4550-a82f-03e4fb38d9e5" InResponseTo="_672a31e9-29e3-422c-80da-fd3bb628844b" IssueInstant="2017-12-18T22:53:23.298Z" Version="2.0">
This mean that RSSO has sent the user over to SAML, but no response is coming back. Further troubleshooting will need to be done on the SAML IDP end, for ADFS check the ADFS admin event viewer.
3. For kerberos check if the ticket is being obtained from the KDC in the RSSO server logs
DEBUG Thread_29 com.bmc.rsso.core.kerberos.KerberosHelper.logSubject(): Kerberos subject obtained: Subject:
Private Credential: Kerberos Principal RSSOLBSPN@SSO.BMC.COMKey Version 0key EncryptionKey: keyType=17 keyBytes (hex dump)=
0000: 6D 63 E8 B0 CA F3 8E AF C5 1B BA 6F 7B AB 75 9A mc.........o..u.
f the ticket is not obtained, have the Admins check the Event Viewer logs on the KDC server, this will give you a clue on why the ticket was not granted
4. If this is a Dev/Test environment and you are in an LB configuration, shutdown one of the servers of RSSO and the application if possible, this will make troubleshooting easier
Contacting BMC support.
If you have been unable to find and fix the problem then contact BMC support with the following information
1. RSSO version
2. Authentications in use
3. Set RSSO server logs to debug reproduce the problem and send in the logs
4. Document one of the user name that is having the problem and a date/time stamp the problem was reproduced
5. Document the steps you are going through screen shots and Videos will help here if you can provide them.
6. The applications being used i.e. TSPS, Remedy, BAO, etc..etc..