8 Replies Latest reply on Apr 23, 2019 12:03 PM by Andrew Waters

    discovery.restfulGet example

    Chris Cusack
      Share:|

      Does anyone have a working example of discovery.restfulGet?  Trying to incorporate it into a TPL to use vsphere6.5 API (via REST) to get VM's tags under vsphere 6.5.  Found the "restfulGet" doc for 11.3 at https://docs.bmc.com/docs/display/DISCO113/discovery.restfulGet

      discovery.restfulGet - BMC Discovery 11.3 - BMC Documentation

       

      I'm trying to figure out if the first parameter in discovery.restfulGet(), which is called "target", is a string representation of the discovered host's name or does it have to be the discovered object in TPL?

        • 1. Re: discovery.restfulGet example
          David Heydecker

          Hello, Chris-

          There are good examples in the TKU- have a look at module EMC-ECSPortal.tpl, where a number of functions are defined. Look for the definition of function "login" and you'll see that the first parameter is a Host node, "host", called from within the main pattern.

          I hope that this helps- please let us know how you get on!

          3 of 3 people found this helpful
          • 2. Re: discovery.restfulGet example
            Andrew Waters

            As the docs say, target is the device against which the request it to be performed. You need to pass it a node.

             

            Assuming you are using a vCenter then this could be the Host representing the vCenter appliance or machine running the vCenter server or a DDD node for the current scan (which assumes it discovered the vCenter). So you need a node.

             

            So, for example for vCenter appliance, given the trigger

            triggers

              on host := Host created, confirmed where os_type = "VMware vCenter Appliance";

            end triggers;

            you could use host as the target.

            4 of 4 people found this helpful
            • 3. Re: discovery.restfulGet example
              Chris Cusack

              Thank you David and Andrew, these were very helpful.  I ended up getting it working against a discovered host as you said, however I ended up using a different host.  I could not discover the vCenter host in BMC Discovery, but we had a CloudForms host which overlays vSphere and has the field values I am looking for also.  The CloudForms host was (1) discoverable and therefore (2) used as my trigger.  With this in the TPL, I was able to find VMs in CloudForms and extract the field values I was looking for.  For this, I ended up using the discovery.restfulGet call against the CloudForms REST API (similar to the vSphere API).  Interestingly, the key that is common to both BMC Discovery VM Objects and CloudForms VM Objects is the (BMC Discovery field) "UUID" within a VM Host, which appears to hold the same value as the (CloudForms field) "uid_ems".  Using this, I am able to bridge between systems for each discovered VM, and augment the Discovery data with more information from CloudForms' fields.

              2 of 2 people found this helpful
              • 4. Re: discovery.restfulGet example
                David Heydecker

                Thank you for feeding back, Chris- it seems that you've been able to extract useful data to augment the standard discovery. Would you consider sharing the pattern that you wrote, as the restfulGet method is fairly new, and others might benefit from your experiences?

                • 5. Re: discovery.restfulGet example
                  Chris Cusack

                  Hi David,

                  Here is what I've found via working on REST in TPLs, as example per your last post.

                   

                  Giving credit where it is due, the below was pieced together via examples and documentation on bmc.com, information found on the web and of course many iterations of trial and error. 

                   

                  Prerequisites on the API Host

                  So lets say, for example, that the host I've tagged is named "mywebhost" and the API is running on http://mywebapi/api

                   

                  The host "mywebhost" should:

                  1. have credentials setup.  In my case, the credentials are of type "RESTful Web API with basic authentication", but these may vary depending on how you intend to connect to the API
                  2. be discovered in BMC Discovery

                   

                  Also, I recommend you download and use the client called "Postman" which is for running REST API calls against the API server.  Although its only useful for narrowing down your exact REST API calls, it allowed me to iterate much faster vs. the waiting time it takes to upload the TPL to BMC Discovery and execute it. It will also give you a view of the returned object when the API call is run. 

                   

                  The TPL

                  First for my trigger, I both specify the host's model type in the trigger, and, also I have separately tagged the host in BMC Discovery.  This is prerequisite so that when I manually run the pattern, the host which the TPL executes on is controllable.

                   

                  triggers

                     on host := Host created, confirmed where model = 'VMware Virtual Platform';

                  end triggers;

                   

                   

                  Since I wanted to iterate on a set of discovered hosts in BMC Discovery, I also did a query in the Body of the TPL to get all the VM's I want to run this API call against. 

                   

                  search_results := search(Host where model has substring 'VMware Virtual Platform' and name has substring 'testVM' order by name);

                   

                  ...and create a loop to iterate on this list...

                   

                  for search_results_item in search_results do

                   

                  Within the loop, making the rest call, would look like this:

                   

                  rest_call_results := discovery.restfulGet(host, 'basic_auth', '/api/foo?name=\'%search_results_item.name%\', use_http:=true);

                   

                  1. From above, the API I would be hypothetically calling the API endpoint http://mywebapi/api/foo?name=testVM1 which is broken down in the above discovery.restfulGet call
                  2. 'host' is from the trigger, not the search_results.  Its the host where the API resides, in this example, that host is "mywebhost"
                  3. 'basic_auth' eludes to the credentials I've defined for 'host' in BMC Discovery (username/password) for accessing the API on "mywebhost"
                  4. Note the API call URL portion '/api....' has no protocol, nor hostname, since these are derived from "host" and "use http:=true" in the other sections of this call
                  5. escape the single quotes in the API URL with backslashes
                  6. Percent signs enclose the TPL variable name for search_results_item
                  7. use HTTP=true if your API server has issue connecting via HTTP, but only at your own risk since HTTP is insecure in transport as compared to HTTPS.

                   

                  And, after some checking to make sure you did not get a 'null' or an error back from your API, I convert the response to a JSON object because my particular API example hypothetically returns JSON objects:

                   

                  my_response_item := json.decode(rest_call_results.response_body);

                   

                  And from there you can address it in TPL using the JSON structure (again using "Postman" will show you this JSON structure easily so you can get an idea of what the object structure of the response truly is):

                   

                  some_item := my_response_item["item_name"];

                   

                  ...where 'item_name' would be a name of a name/value pair from your returned JSON, and 'some_item' would be assigned the value of 'item_name'.

                  2 of 2 people found this helpful
                  • 6. Re: discovery.restfulGet example
                    Andrew Waters

                    This is really not a good idea. You should not be doing a big search in an individual pattern and processing everything. It makes patterns take a long time and can create large memory usage. Plus if you are doing as you described you run against everything whenever a single VMware Virtual Platform Host is found. Also you require that all the Hosts be in the same scan range otherwise you will get blocked scans.

                    1 of 1 people found this helpful
                    • 7. Re: discovery.restfulGet example
                      Chris Cusack

                      Yes sir, I guess that's all true.  You know, I looked at this issue I don't know how many times, over and over again, figuring maybe I'd see something that shouldn't be there.  Then suddenly it occurred to me, I had it backwards.  Maybe there was something that should be there but wasn't.  And that's what led me to use the search list of specific VMs as the iteration for the master loop. 

                       

                      See, if it were me, I'd position things so I could have some control and see what is going on.  My mind is kind of disinclined to hurry up and it takes me some time to wade through everything. 

                       

                      So I'd iterate through a searchable list with criteria I can control, and have a particular scanner appliance assigned to each unique set of scan ranges (ex. internal dev, internal production, customer facing, etc.) to ensure scanning performance, so this would work well in that kind of setup. 

                       

                      And actually with performance, the REST API calls in TPL run much faster than the TPL calls for reading the host's registry, to get similar data.  And its OS-agnostic.  I've spent considerable time on improving performance and saw no degradation of performance on our scanner host or TPL run-times.  Also you might have missed this, but the trigger is not as broad as it appears because I have one API host tagged for manual running of the TPL against that single API host.  So we could presumably remove the trigger altogether since it is run on demand, and as you point out, does quite a bit of work.

                       

                      I'll keep my eye out for blocked scans, thank you for the heads-up!

                      2 of 2 people found this helpful
                      • 8. Re: discovery.restfulGet example
                        Andrew Waters

                        You need to try to change the way you approach this otherwise you are going to get yourself into lots of problems with how Discovery works.