TestHttpClient - Command line tool to test HTTP(S) services

Version 14
    Share This:

    This document contains official content from the BMC Software Knowledge Base. It is automatically updated when the knowledge article is modified.


    PRODUCT:

    Remedy AR System Server


    COMPONENT:

    AR System Mid Tier


    APPLIES TO:

    Web applications that need to be tested and troubleshoot the HTTP traffic and SSL Handshake



    DETAILS:

    Prerequisites

       
    • Java 8 or above is needed  (should be available on any host that runs Tomcat or Remedy AR server). Java 7 will not work. The utility has been tested with Oracle Java 8 and OpenJDK java 14
    •  
    • Text editor to edit plain text files
    •  
    • Command line access to run java and read the sample files
       Simple command can be run as  
       
    1. java -jar TestHttpClient.jar <TextFile>
      
      Debug logs 
    There are 2 flags to get verbose logging   
    --debugHTTP will display full http wire log  
    and   
    --debugSSL will display javax.net.debug=all logs  
    each of them can be used independently and in any sequence   
     
    For most examples I would use debugHTTP 
      
       
    1. java -jar TestHttpClient.jar <TextFile> --debugHTTP
      
      Additional options can be set in the java log such as keystores, truststore or proxyServers 
      
    How to write debug log to a file  
      Logs will be written to console.  If a file is preferred, use this syntax 
      
       
    1. java -jar TestHttpClient.jar <TextFile> --debugHTTP 2> log.txt
      
      Simple GET request  
      
       
    1. GET http://hostname:8008/api/arsys/v1.0/swagger/ars-jwt.json
      
      If this file is run with debug options the following logging will be returned 
      
       
    1. ...log truncated for brevity
    2.  
    3. ... >> "GET /api/arsys/v1.0/swagger/ars-jwt.json HTTP/1.1[\r][\n]"
    4.  
    5. ... >> "Host: clm-aus-tnkxn4:8008[\r][\n]"
    6.  
    7. ... >> "Connection: Keep-Alive[\r][\n]"
    8.  
    9. ... >> "User-Agent: Apache-HttpClient/4.5.3 (Java/14.0.1)[\r][\n]"
    10.  
    11. ... >> "Accept-Encoding: gzip,deflate[\r][\n]"
    12.  
    13. ... >> "[\r][\n]"
    14.  
    15. ... << "HTTP/1.1 200 OK[\r][\n]"
    16.  
    17. ... << "Date: Tue, 16 Jun 2020 22:10:24 GMT[\r][\n]"
    18.  
    19. ... << "X-Frame-Options: SAMEORIGIN[\r][\n]"
    20.  
    21. ... << "Content-Security-Policy: frame-ancestors 'self'[\r][\n]"
    22.  
    23. ... << "Content-Type: application/json[\r][\n]"
    24.  
    25. ... << "Cache-Control: no-store[\r][\n]"
    26.  
    27. ... << "Content-Length: 2248[\r][\n]"
    28.  
    29. ... << "[\r][\n]"
    30.  
    31. ... log truncated for brevity
      
     
    The log explains the actual HTTP Request sent to the server (lines including >>) and the HTTP Response ( lines including << )   
    There are several sample files attached in a single zip file. 
      
     
      Text File Structure for Requests   
       The structure can be simple with only one request or multiple requests can be specified one after another  
      
       Sample files

    FILE: 01_get.txt  
      Defines variables in first 3 lines.  
    Line 4 performs a GET request with the url defined reusing predefined variables 
      
       
    1. @hostname=arservername:8008
    2.  
    3. @schema=http
    4.  
    5. @url = {{schema}}://{{hostname}}/api/arsys/v1.0/swagger/ars-jwt.json
    6.  
    7. GET {{url}}
      
     
    FILE: 02_login_getmetadata.txt 
      
        
      
       
    1. @username = Demo
    2.  
    3. @password = remedy
    4.  
    5. @hostname=arservername:8008
    6.  
    7. @schema=http
    8.  
    9. POST {{schema}}://{{hostname}}/api/jwt/login
    10.  
    11. Content-Type: application/x-www-form-urlencoded
    12.  
    13.  
    14.  
    15. username={{username}}&password={{password}}
    16.  
    17. <<<
    18.  
    19. @token = AR-JWT {{responseText}}
    20.  
    21. ###
    22.  
    23. //get metadata of a known field reusing token
    24.  
    25. GET {{schema}}://{{hostname}}/api/arsys/v1.0/fields/User/?field_ids=1
    26.  
    27. Authorization: {{token}}
      
      Lines 1-4 define variables  
    Line 5 defines a POST request to Remedy REST login url  
    Line 6 defines an http header for Content-Type  
    Line 7 is empty but required. (The empty line is needed between the HTTP headers and the HTTP body)  
    Line 8 defines the username and password payload  
    Line 9 <<< marks the end of the request  
    Line 10 reads one of the reserved variables responseText , preceeded by "AR-JWT" and stores it as @token for later use  
    Line 11 ### marks end of the request context and a new one starts on line 12  
    Line 12 // marks a comment  
    Line 13 - 14 build a new request to get Metadata and uses the token created in line 10 
      
    FILE: 03_login_create_entry.txt  
      Note:  only new concepts are highlighted 
      
       
    • //this is line 16
    •  
    •  
    •  
    • POST {{schema}}://{{hostname}}/api/arsys/v1/entry/zzTestAttachment
    •  
    • Authorization: {{token}}
    •  
    • content-type: application/json
    •  
    •  
    •  
    • {"values":{"Submitter":"Demo",
    •  
    • "Short Description":"testing 123"}}
       
      Line 18 defines a POST request reusing variables defined earlier in that same file  
    Line 22-23 defines json payload 
      
      FILE: 04_login_create_entry_with_attachment.txt  
       
    • //create entry line 16
    •  
    • @boundary= 854185817597954740109521
    •  
    • POST {{schema}}://{{host}}/api/arsys/v1/entry/zzTestAttachment
    •  
    • Authorization: {{token}}
    •  
    • Content-Type: multipart/form-data; boundary={{boundary}}
    •  
    •  
    •  
    • --{{boundary}}
    •  
    • Content-Transfer-Encoding: 8bit
    •  
    • Content-Type: application/json; charset=UTF-8
    •  
    • Content-Disposition: form-data; name="entry"
    •  
    •  
    •  
    • {"values":{"Submitter":"Demo",
    •  
    • "Short Description":"testing 123",
    •  
    • "Attachment":"file.png"}}
    •  
    •  
    •  
    • --{{boundary}}
    •  
    • Content-Disposition: form-data; name="attach-Attachment"; filename="file.png"
    •  
    • Content-Type: application/octet-stream
    •  
    • Content-Transfer-Encoding: binary
    •  
    •  
    •  
    • <<< C:\tmp\rainbow.png
    •  
    •  
    •  
    • --{{boundary}}--
      
      Line 36 defines that a file will be read in binary mode and appended to the HTTP Request Body 
    <<< C:\tmp\rainbow.png  
      If the file is not found a FileNotFoundException will be thrown and printed in the logs but the program will continue 
      
       
    1. java.io.FileNotFoundException: C:\Users\Administrator\.....png (The system cannot find the file specified)
    2.  
    3. at java.io.FileInputStream.open0(Native Method)
    4.  
    5. at java.io.FileInputStream.open(FileInputStream.java:195)
    6.  
    7. at java.io.FileInputStream.<init>(FileInputStream.java:138)
    8.  
    9. at com.bmc.sample.restapi.basic.FileStreamEntity.<init>(FileStreamEntity.java:37)
    10.  
    11. at com.bmc.sample.restapi.basic.RequestContext.isBodyLine(RequestContext.java:486)
      
    FILE 05_login_create_multiple_entries.txt  
      There are 2 variables to repeat a request   
    @repetition = number  
    and  
    @repetitionOffset = number  
     
    if a request should be repeated, simply specify @repetition = n variable.  {{repetitionCounter}} will be available in every repetition (starting in repetitionOffset+1)   
    {{repetitionCounter}} is only available from the URL Line and headers till end of Body   
     
    Line 15 defines the repetition value as 10  
    Line 23-25 define AR headers using the repetition counter 
      
       
    1. //create entry line 14
    2.  
    3. @repetition =10
    4.  
    5. @repetitionOffset= 9
    6.  
    7. @boundary= 854185817597954740109521
    8.  
    9. @clientType = 2831853071
    10.  
    11. @CoreId = 3.14159265
    12.  
    13. POST {{schema}}://{{host}}/api/arsys/v1/entry/zzTestAttachment
    14.  
    15. Authorization: {{token}}
    16.  
    17. X-AR-Client-Type: {{clientType}}
    18.  
    19. X-AR-TR-Core-Id: {{CoreId}}-{{repetitionCounter}}
    20.  
    21. X-AR-TR-Counter: {{repetitionCounter}}
    22.  
    23. Content-Type: multipart/form-data; boundary={{boundary}}
    24.  
    25.  
    26.  
    27. --{{boundary}}
    28.  
    29. Content-Transfer-Encoding: 8bit
    30.  
    31. Content-Type: application/json; charset=UTF-8
    32.  
    33. Content-Disposition: form-data; name="entry"
    34.  
    35.  
    36.  
    37. {"values":{"Submitter":"Demo",
    38.  
    39. "Short Description":"testing 123",
    40.  
    41. "Attachment":"file.png"}}
    42.  
    43.  
    44.  
    45. --{{boundary}}
    46.  
    47. Content-Disposition: form-data; name="attach-Attachment"; filename="file.png"
    48.  
    49. Content-Type: application/octet-stream
    50.  
    51. Content-Transfer-Encoding: binary
    52.  
    53. <<< C:\tmp\rainbow.png
    54.  
    55. --{{boundary}}--
      
     
    Logs show headers using the {{repetittionCounter}} variable 
      
       
    1. >> "POST /api/arsys/v1/entry/zzTestAttachment HTTP/1.1[\r][\n]"
    2.  
    3. >> "X-AR-TR-Core-Id: 3.14159265-19[\r][\n]"
    4.  
    5. >> "X-AR-TR-Counter: 19[\r][\n]"
    6.  
    7. ...
    8.  
    9. << "HTTP/1.1 201 Created[\r][\n]"
    10.  
    11. << "X-AR-TR-Core-Id: 3.14159265-19[\r][\n]"
    12.  
    13. << "X-AR-TR-Counter: 20[\r][\n]"
      

    Other variables  
      Once a request has been issued and a response is available, these variables are created to read the content  
      
       
    • {{responseText}} includes the response Body (useful to reuse the JWT token)
    •  
    • {{responseHeader[header-name]}} useful to retrieve the transaction counter
    •  
    • {{responseStatus}} eg: 200, 201, etc
       Using TestHttpClient as SSL Poke  
       
    • Create a text file with a request for an SSL host such as (eg: ssl_test_file.txt)
      
       
    1. GET https://clm-aus-tnkxn4:8443/api/jwt/login
    Execute the utility with the text file in debug mode  
       
    1. java -jar TestHttpClient.jar ssl_test_file.txt --debugSSL
      
       
    • Review the log 
      
       
    1. javax.net.ssl|DEBUG|01|main|2020-06-18 16:39:10.577 CDT|CertificateMessage.java:357|Consuming server Certificate handshake message (
    2.  
    3. "Certificates": [
    4.  
    5. "certificate" : {
    6.  
    7. "version" : "v3",
    8.  
    9. "serial number" : "10 9D F9 06",
    10.  
    11. "signature algorithm": "SHA256withRSA",
    12.  
    13. "issuer" : "CN=clm-aus-tnkxn4, OU=Customer Support, O=BMC Software, L=Houston, ST=Texas, C=US",
    14.  
    15. "not before" : "2020-05-07 17:15:24.000 CDT",
    16.  
    17. "not after" : "2020-08-05 17:15:24.000 CDT",
    18.  
    19. "subject" : "CN=clm-aus-tnkxn4, OU=Customer Support, O=BMC Software, L=Houston, ST=Texas, C=US",
    20.  
    21. "subject public key" : "RSA",
    22.  
    23. "extensions" : [
    24.  
    25. {
    26.  
    27. ObjectId: 2.5.29.14 Criticality=false
    28.  
    29. SubjectKeyIdentifier [
    30.  
    31. KeyIdentifier [
    32.  
    33. 0000: D9 1E EC A7 7B 01 D0 13 A2 AC 8F 81 3C BA E3 69 ............<..i
    34.  
    35. 0010: 72 15 D4 71 r..q
    36.  
    37. ]
    38.  
    39. ]
    40.  
    41. }
    42.  
    43. ]}
    44.  
    45. ]
    46.  
    47. )
    48.  
    49. javax.net.ssl|ERROR|01|main|2020-06-18 16:39:10.643 CDT|TransportContext.java:317|Fatal (CERTIFICATE_UNKNOWN): PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target (
    50.  
    51. "throwable" : {
    52.  
    53. sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
      
       
    • Specify keystore and truststore if needed as standard JVM  arguments
      
       
    1. java -Djavax.net.ssl.keyStore=myKeystore.jks -Djavax.net.ssl.keyStorePassword=changeit -Djavax.net.ssl.trustStore=myKeystore.jks -Djavax.net.ssl.trustStorePassword=changeit -jar TestHttpClient.jar sslPoke.txt --debugSSL --debugHTTP 2> log.txt
      
    Limitations  
       
    1. Do not set content-length as a header, the utility will set it automatically and it crashes if the header is set in a request
    2.  
    3. Retrieving an attachment is feasible but it is not saved as a file, therefore the only way at this point to check if an attachment was stored and downloaded fine is to download with another client such as Mid-tier, curl, wget, etc

     


    Article Number:

    000369988


    Article Type:

    Product/Service Description



      Looking for additional information?    Search BMC Support  or  Browse Knowledge Articles