1 2 Previous Next 21 Replies Latest reply: May 3, 2012 6:17 AM by Emmanuel TELECHEA RSS

TMART HTTP VirtualHost testing

Emmanuel TELECHEA

Hi all,

 

I'm facing a little tricky issue with tmart script :

 

I need to test an URL that is hosted as virtualhost on different nodes.

 

ex : https://toto.world.com/application/index.php

 

toto.world.com is an http virtualhost (IIS)  that is running on several nodes. Let's say nodeA , nodeB.

 

If directly pass the following url to my http monitor :  https://nodeA/application/index.php I obtain a 404 error, what is normal.

 

My question is the following : How to test in a bdl script the : https://toto.world.com/application/index.php URL, but specifying that I want to reach nodeA and then test it on nodeB ?

 

I found the following function is bdl script reference : WebAddToHNC() that allow me to overload the IP address of toto.world.com, but with no

positive result...

 

 

Any help would be greatly appreciate.

 

Thanks & regards.

 

Emmanuel.

  • 1. TMART HTTP VirtualHost testing
    Hal DeVore

    Emmanuel,

     

    If you know the IP address, you should substitute that into the URL.  For example, if nodea has an IP address of 10.1.1.10, then usee https://10.1.1.10/application/index.php

     

    TM ART Workbench also has a feature called "Standard Host".  You can find some examples in the Workbench Help.  With "Standard Host", you use the name "standardhost" for the hostname in URLs.  The replay engine will substitute a value that you provide.  Example https://standardhost/application/index.php

     

    You can provide a value for standardhost by using the WebSetStandardHost(...) function call or by setting it in the Workbench project profile.  For the project profile, open the project in Workbench and select the Settings menu, then Active Profile, click the Internet icon in the icon list on the left of the Settings dialog, then click the Hosts tab.

     

    If you set the value using the WebSetStandardHost() function call, however, you can combine this with the substitution capabilities of Project Attributes and set the value of standardhost when you create the monitor on Central and avoid creating multiple scripts to access different hosts.

     

    --Hal

  • 2. TMART HTTP VirtualHost testing
    Emmanuel TELECHEA

    Hello Hal, and thanks for your answer.

     

    I know how to change the atrget IP address using the WebSetStandardHost() function but it does not fit my need.

     

    My issue is a bit more tricky :

     

    let say I have 2 http server : nodeA , nodeB

    each server is hosting a virtualhost corresponding to toto.world.com service.  (http virtual host with a differnet IP address)

     

    What I want to achieve is to test http://toto.world.com/application/index.php on each node.

    BUT : as toto.world.com is a virtual http host, if I try to test directly http://nodeA/application/index.php or nodeB/application/index.php I obtain a 404 HTTP error code. Which is normal, cause application web site is hosted on toto.world.com http vhost on each server.  There is no application/index.php at the root of http://nodeA or http://nodeB.

     

    So my concern is to be able to test the service URL, but on each target node.

     

    I don't know if it's more clear...

     

    regards.

     

    Emmanuel.

  • 3. TMART HTTP VirtualHost testing
    cpriddy

    Hi Emmanuel,

          sounds like you've got a scenario where the virtual http host is acting as a load balancer and a firewall at the same time. It sounds like it chooses node A or B for each session and redirects to those.  But the actual node A and B are not directly accessible from the user point of view.

     

    First off, if you can't launch internet explorer from the execution server and type in a URL that works and the execution server has network access to,  that should be your first action.  Until you can accomplish your exact workflow in a browser session manually, you'll need to figure out different urls or execution server locations behind the firewall that can access nodeA and B directly.

     

    Second,  TMART is typically used to monitor user experience.   Part of the user experience is the overhead and balancing provided by the virtual interface/load balancing.  To monitor customer experience (including the value of the virtual interface hiding when node B or A is down and proving your HA config is working or not),  you should really monitor through that virtual interface as your primary "user experience".  

     

    If you still want to go directly to test node A and B (I'd only do it in addition to, not instead of the above) -- I'd typically consider that diagnostic monitoring not user experience monitoring only useful for the purpose of identifying root cause metrics to try and isolate different infrastructure.    Nothing wrong doing both with TMART but a lot of customers often get confused between monitoring UEM vs. what they're used to monitoring at the hardware/infrastructure level.   So just a suggestion to understand your overall UEM monitoring strategy vs. infrastructure/component monitoring typically done with KMs etc.

     

    If you can't see node A and B from your location(s) you are wanting to get your UEM data from,  you might need to create a different location that can see node A, B or both and have the diagnostic scripts deployed to those locations.

     

    -Chuck

  • 4. TMART HTTP VirtualHost testing
    Emmanuel TELECHEA

    Hi Chuck,

     

    Many thanks for your very interresting answer.

     

    In fact I manage to find out a start of solution to my issue by using the WebModifyHttpHeader() function.

     

    I'm not facing firewall or direct access issue. In fact it is just that when configuring a virtual host on your http server,

    the applications that are hosted on it are not accessible when using physical node name. (let say nodeA for instance).

     

    Previously we used a curl script to perform the test operations :

    ex:

     

    if I try : http://virtual/application/index.html it works

    but http://nodeA/application/index.html it return a 404 what is normal.

     

    Using curl you can perform the following:

     

    curl --header "Host: virtual" http://nodeA/application/index.html

     

    This way you access physical node and by replacing "Host:" in HTTP header you are routed to the virtual host.

    This way you can test directly the virtualhost on nodeA and be able to check which server in the farm is not responding.  You can perform the test on nodeB, nodeC , etc ..

     

    I find the function WebModifyHttpHeader() function in bdl reference guide but it's not working

    as it add the header and not replace it.. Event if I use the WEB_MODIFY_OPT_RemoveAdd option in function.

     

    I think that I will open a CASE with support as it's not working as design it seems.

     

    By that way thanks again guys !

     

    Regards.

     

    Emmanuel.

  • 5. TMART HTTP VirtualHost testing
    Hal DeVore

    Emmanuel,

     

    The Web...Header functions are used internally in various places so it is unlikely they are not working.

     

    For a simpler call, you can look at WebSetHttpHeader(...).

     

    Also, the headers are retained for all Web calls from the point that they are set until the script ends or you change them.  So you would typically put the Web...HttpHeader...() call in the Tinit section of your code.

     

    Can you share some code snippet from your script?

     

    --Hal

  • 6. Re: TMART HTTP VirtualHost testing
    Emmanuel TELECHEA

    Hi Hal,

     

    Thanks for your clue, I will give it a try.

     

    Here the bdf script I use :

     

    benchmark BenchmarkName

     

    use "WebAPI.bdh"

    use "MeasureBounds.bdh"

     

    dcluser

      user

        VirtUser

      transactions

        TINIT                : begin;

        NoTRT_UrlCheck       : 1;

     

    var

      sURL                : string;

      sProxy              : string;

      nProxyPort          : number;

      sParsedTitle        : string;

      sCheckTitle         : string;

      sTitleError         : string;

      sCheckText          : string;

      sTextError          : string;

      sSuppressDomains    : string;

      bBypassLocal        : boolean;

      sBypassProxy        : string;

      fServerBusyLBound   : float;

      fServerBusyUBound   : float;

      fDocDownloadLBound  : float;

      fDocDownloadUBound  : float;

      fPagetimeLBound     : float;

      fPagetimeUBound     : float;

     

      bLinkChecking       : boolean;

      bDetailedPageTimers : boolean;

      sReturnedTitle      : string;

     

      sVirtualHost        : string;

     

     

    dclfunc

     

    //----------------------------------------------------------------------------------

      function trimString(toTrim : string) : string

      var

        output : string;

        trimBegin : number;

        trimEnd  : number;

        revToTrim : string;

        trimChars : string;

      begin

     

        trimChars := " " + chr(9) + chr(13) + chr(10);

        trimBegin := Strspn(toTrim, trimChars);

     

        //print("trimBegin " + string(trimBegin));

     

        revToTrim := toTrim;

        Strrev(revToTrim);

        trimEnd := Strlen(revToTrim) - Strspn(revToTrim, trimChars);

     

        //print("trimEnd " + string(trimEnd));

     

     

        if (trimBegin < trimEnd) then

          Substr(toTrim, output, trimBegin, trimEnd - trimBegin + 2); 

        else

          output := "";

        end;

        trimString := output;

      end TrimString;

     

     

    dcltrans

     

      transaction TINIT

      begin

     

        AttributeGetString("URL", sURL);

        AttributeSetString("#Measure1.Name", sURL);

     

        AttributeGetString("Virtual Host Name", sVirtualHost);

     

     

        if (StrLen(sProxy) > 0) then

          WebSetProxy(WEB_PROXY_HTTP, sProxy, nProxyPort);

          WebSetHttpHeader("pragma: no-cache");

          if (AttributeGetBoolean("Use HTTP 1.0 through proxy connections")) then

            WebSetHttpVersion("HTTP/1.0");

          end;

          bBypassLocal := AttributeGetBoolean("Bypass proxy server for local addresses");

          AttributeGetString("Do not use proxy server for", sBypassProxy);

          if (Strlen(sBypassProxy) > 0) then

            WebSetProxyBypass(bBypassLocal, sBypassProxy);

          else

            WebSetProxyBypass(bBypassLocal);

          end;

        end;

     

        // Set title and content verification

     

        AttributeGetString("Verify if the title is", sCheckTitle);

        AttributeGetString("Raise error if the title is", sTitleError);

     

        AttributeGetString("Verify if the content contains", sCheckText);

        AttributeGetString("Raise error if the content contains", sTextError);

     

        AttributeGetString("Suppress domains", sSuppressDomains);

        if (Strlen(sSuppressDomains) > 0) then

          WebSetDomainSuppress(sSuppressDomains);

        end;

     

        // Enable Link Checking

     

        bLinkChecking := AttributeGetBoolean("Link Checking");

        if (bLinkChecking) then

          WebSetOption(WEB_OPT_LINK_CHECK, 1);

        else

          WebSetOption(WEB_OPT_LINK_CHECK, 0);

        end;

     

        // Enable Link Checking

     

        bDetailedPageTimers := AttributeGetBoolean("Detailed Page Timers");

        if (bDetailedPageTimers) then

          WebSetOption(WEB_OPT_STAT_TO_TSD, STATFLAG_All);

        end;

     

        // Set time bounds

     

        InitMeasureBounds();

      end TINIT;

     

      transaction NoTRT_UrlCheck

      begin

     

        WebSetVerification(true, 0, true);

     

        WebParseResponseData(sParsedTitle, sizeof(sparsedTitle),"<title>","</title>");

     

        if (Strlen(sCheckText) > 0) then

          WebVerifyContent(sCheckText, 0);

        end;

        if (Strlen(sTextError) > 0) then

          WebVerifyContent(sTextError, 1, 0, WEB_VER_FLAG_CONTENT_SMALLER);

        end;

     

     

       // Set Host: Header to vHost value

        WebModifyHttpHeader("Host",sVirtualHost,WEB_MODIFY_OPT_RemoveAdd);

        // Open URL

        WebPageUrl(sURL);    

     

        WebThreadWait();

     

     

        //print(sParsedTitle);

        //print(string(Strlen(sParsedTitle)));

     

        sParsedTitle := trimString(sParsedTitle);

     

        //print(sParsedTitle);

        //print(string(Strlen(sParsedTitle)));

     

     

        if (Strlen(sTitleError) > 0) then

          if (Strnicmp(sTitleError, sParsedTitle, Strlen(sTitleError)) = 0) AND (Strnicmp(sTitleError, sParsedTitle, Strlen(sParsedTitle)) = 0) then

            RepMessage("Error on Title Validation: " + sTitleError, SEVERITY_ERROR);

          end;

        end;

     

        if (Strlen(sCheckTitle) > 0) then

          if (Strnicmp(sParsedTitle, sCheckTitle, Strlen(sParsedTitle)) <> 0) OR (Strnicmp(sParsedTitle, sCheckTitle, Strlen(sCheckTitle)) <> 0)  then

            RepMessage("Error on Title Validation: " + sParsedTitle, SEVERITY_ERROR);

            end;

        end;

     

      end NoTRT_UrlCheck;

     

     

    Thanks again.

     

    Emmanuel.

  • 7. Re: TMART HTTP VirtualHost testing
    Emmanuel TELECHEA

    Hal,

     

    unfortunatly it still not works.... Here is the header passed by monitor :

     

    GET / HTTP/1.1

    Host: srvparmut11

    Connection: Keep-Alive

    Accept: */*

    Accept-Encoding: gzip, deflate

    User-Agent: BMC Transaction Management Application Response Time Central/4.1.0+simulating+Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.0)

    UA-CPU: x86

    Host: regulator.fr.world.socgen

     

    As you can see Host: regulator.fr.world.socgen has been added instead of being replaced.... I still have Host: srvparmut11 in header

     

    Regards.

     

    Emmanuel.

  • 8. Re: TMART HTTP VirtualHost testing
    Hal DeVore

    The code looks like it should be fine as is.

     

    If you haven't already done so, run a Try Script from the Workbench and examine the headers in the TrueLog Explorer to see if they are what you expect them to be.

     

    If not, I suggest first changing from WebModifyHttpHeader to WebSetHttpHeader.

     

    --Hal

  • 9. Re: TMART HTTP VirtualHost testing
    Emmanuel TELECHEA

    Hal,

     

    in fact I tried both methods; WebSetHttpHeader() and WebmodifyHttpHeader() with the same result. (calls are made in Tinit section)

     

    Here the headers in Truelogs explorer :

     

    GET / HTTP/1.1

    Host: srvparmut11

    Connection: Keep-Alive

    Accept: */*

    Accept-Encoding: gzip, deflate

    User-Agent: BMC Transaction Management Application Response Time Central/4.1.0+simulating+Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.0)

    Host: regulator.fr.world.socgen

     

    So the header has been added at the end... Make the monitor running in a HTTP 400 error Bad Header. what is normal.

     

    Thanks & regards.

     

    Emmanuel.

  • 10. Re: TMART HTTP VirtualHost testing
    Hal DeVore

    The "host" header isn't a normal header in that it is a mandatory header and normally is parsed out of the URL presented to the browser.  So there is some chance that modifying it isn't supported.

     

     

     

    OK, let's back up to the question that Chuck had. Ignoring TM ART for a moment, what exactly does a user of your website do to access the applications?

     

    The script you posted looks like it was hand-coded or based on the URL Checker.

     

    Have you tried recording a normal browser accessing your website?  It would be interesting to see the Recorded TrueLog from using a browser to access the site.

     

    --Hal

  • 11. Re: TMART HTTP VirtualHost testing
    Emmanuel TELECHEA

    Here is the request from my internal customer :

     

    "Dear Web team,

    Please set up a monitoring task which will monitor each 6 servers on webfarm for the following link :

    https://lemans.fr.world.socgen/WS_SCOW/DREF_OnBoardingClient_v1.0.5/DREF_OnBoardingClient/onBoardingService_v.0.1.php

     

    We would like to check every 5 minutes that page is availble, and if connection times out to raise an alert to lon-itec-pcdev@sgcib.com

     

    This is an end point for a critical web-service connecting SCOW to LeMANs and we require constant availability of this end-point.

    We would like to monitor each server (PARMUT 11, 12, 13, 21, 22, 23) individually, and alert should mention which server failed to respond.

    (in past few days, we noticed that this link was timing out without any indication on iis)"

     

     

    We used to address this need by directly testing each server of the web farm, but what is sightly different in this case, is that the https://lemans.fr.world.socgen is a http vHost. so if you try for instance :

     

    https://srvparmut11/WS_SCOW/DREF_OnBoardingClient_v1.0.5/DREF_OnBoardingClient/onBoardingService_v.0.1.php  you will encounter a 404 error.

     

    I know that i can create complex scenario using recorder and so on, but I'm facing an "Availability" concern.

     

    The service URL might be available indeed, but they would like to know if one server is the farm is not running as expected.

     

    But I think that you're right, and that http header Host: might be hardcoded in WebpageUrl() function or WebUrl() functions...

     

    Emmanuel.

  • 12. Re: TMART HTTP VirtualHost testing
    Hal DeVore

    No need to create a complex scenario with the Recorder, just use the recorder to access the https://lemans.fr.world.socgen/WS_SCOW/DREF_OnBoardingClient_v1.0.5/DREF_OnBoardingClient/onBoardingService_v.0.1.php page. 

     

    Then examine what was recorded.  Let's see what sort of headers are being passed around.

     

    Try two different recordings, do one recording as a normal "Web Business Transaction" and a second one as "Web Low Level".  Let's see what you get in the HTTP headers in both cases.

     

    The goal here is to get the script to make a TCP connection to lemans.fr.world.socgen but to pass one of the real hosts in the host header.  My thought is this may require giving up the nice simple page-oriented functions and using the lower level functions in the Workbench.  Scripting that would be much simpler if you started from a recording.

     

    --Hal

  • 13. Re: TMART HTTP VirtualHost testing
    Emmanuel TELECHEA

    Ok I will try this way.

     

    Once again many many thanks for your kind help on this ! 

     

     

    Emmanuel.

  • 14. TMART HTTP VirtualHost testing
    cpriddy

    As I mentioned earlier,  I'd suggest bringing up internet explorer just by itself on the execution server and find a way using just IE to test your scenario.   Once you can do that,  then I'd run monitor workbench from that ES and record/customize the script until it repeatedly runs trys successfully.

     

    Other alternatives I can suggest if you can't get the above or similar to work:

    1. If you have a PATROL/BPPM environment, I believe that you can configure the web application server KM to track individual servlet response times on each of your application servers.   The approach here would be to use TMART to monitor end user experience as intended by the application, but allow the WAS KM to track servlet responses on the app server.  That way the KM can set thresholds on all app servers for that servlet and you'll know if and which server is having a problem.
    2. Similar to #1, but using the AppSight product for diagnostics.   If you start getting period problems 1 of N responses from your virtual query returning problems, you can analyze the AppSight results for all app servers to see which had unusual response times and drill down into whether it is an application code problem, or a dependency on another component (e.g. database query or MQ request etc) slowing the processing.
      • What's more if you have AppSight and BPPM TMART Adapter,  you can use the TMART detailed diagnostic for a particular execution performance problem and let AppSight figure out which server was involved processing that query and drill down into the diagnostic details - assuming you turn on the "tagging" config in the TMART script for AppSight.   The benefit of this approach over a separate monitor for TMART or even the KM servlet tracking is that you don't have the overhead of continuously running those queries, storing, aggregating, and deleting all that data just in case.   Instead you let AppSight manage a circular queue of history and can query/run diagnostics after the fact and eliminate all the per server monitor data overhead and capacity impacts to the monitoring infrastructure of both TMART and BPPM.
    3. Since you can use cURL to do what you are wanting, create a profile that will launch cURL directly or a batch, perl, or other script that invokes cURL.   Then do a recording that captures the results of that cURL execution.   After tweaking/customizing that resulting script to run repeatedly, you can then deploy that to TMART execution servers (assuming that both cURL and any surrounding script like Perl has already been deployed and setup on the execution servers in the same location etc.   This uses a very similar approach to how we often suggest testing web services directly using the SOAPUI utility instead of the default TMART mechanism.   If nothing else, you can see how the capture/basic script generated by MWB differs from your own above and merge the key changes into your script.

     

     

    Rather than fighting the default behavior,  I'd suggest going with #3.   Or do so initially until you can replace it with a different script long term if you prefer.

     

    Chuck

1 2 Previous Next