7 Replies Latest reply on Dec 9, 2019 6:33 AM by Bob Anderson

    Get Host out of DiscoveredProcess

    Rémi Chaffard
      Share This:



      When developping custom patterns, I often use this kind of query to get the process I'm interested in with the host names:


      search DiscoveredProcess where ( condition ) 
      show cmd, #Member:List:List:ProcessList.#DiscoveryResult:DiscoveryAccessResult:DiscoveryAccess:DiscoveryAccess.#Associate:Inference:InferredElement:Host.name


      So I'm trying to go from process to Host through 3 relationships:




      Today I tried the same with some process, and host name column is almost always "not set". I tried to understand, if I do the same search traversing only to DiscoveryAccess, it's ok. If I click on DA, I see the Host in the "Inferred entity".

      I tried to change the last relationship by :::Host but same result.


      So basically in the UI I see the DA is correctly linked to the Host, but using query it seems it is not working.

      Is there anything I miss here ?


      Thanks for help


        • 1. Re: Get Host out of DiscoveredProcess
          Rémi Chaffard

          I guess I'm too tired, this is because the DA is associated to the Host only for the last discovery.


          Using include_destroyed flags works:


          search FLAGS(include_destroyed) DiscoveredProcess where ( condition ) 
          show cmd, #Member:List:List:ProcessList.#DiscoveryResult:DiscoveryAccessResult:DiscoveryAccess:DiscoveryAccess.#Associate:Inference:InferredElement:Host.name




          • 2. Re: Get Host out of DiscoveredProcess
            Andrew Waters

            What are you actually trying to achieve? This is searching a lot of nodes.

            • 3. Re: Get Host out of DiscoveredProcess
              Rémi Chaffard

              It is just a one-shot analysis when having to create custom patterns. I know what processes I need to search for, then I want to search those processes and have the hostname where they are running.

              Then I'm doing some pivot table in excel to check if all processes are running on all hosts, or check the exact repartition. It helps to understand the software architecture, then to choose correctly triggers and so on in the custom patterns.


              In general I'm using windows_cmd or unix_cmd shortcuts so it makes the query running faster. If not I agree it searches for a lot of nodes, but only once.




              1 of 1 people found this helpful
              • 4. Re: Get Host out of DiscoveredProcess
                Bob Anderson



                When you are searching DiscoveredProcess, you are searching DirectlyDiscoveredData or 'ddd'. There is a history of ddd depending on your ddd aging.

                Typically, only the last DiscoveryAccess (and its related ddd) for an endpoint will result in a relationship to a 'Host' node.


                It appears [ we cannot tell by your masked ( condition ) ] you are searching ALL the DiscoveredProcesse's


                When you search ddd without including the '_last_marker' or '_last_interesting' attribute on the parent DiscoveryAccess, you are searching ALL ddd nodes and getting many results that do not have the DiscoveryAccess relationship to the inferred 'Host' node.


                Your ( condition ) should include something like either: #Member:List:List:ProcessList.#DiscoveryResult:DiscoveryAccessResult:DiscoveryAccess:DiscoveryAccess._last_marker




                to include only the DiscoverdProcess's that are associated with DA's that could be providers for the 'Host' node.


                This will greatly reduce the set of nodes searched and will more than likely include just those that actually have the related 'Host' node





                2 of 2 people found this helpful
                • 5. Re: Get Host out of DiscoveredProcess
                  Rémi Chaffard

                  Hi Bob,


                  Thanks for this tip, I will for sure test it next time I need.



                  • 6. Re: Get Host out of DiscoveredProcess
                    Duncan Grisby

                    Trying to filter to only include last-marker nodes in that way does not actually reduce the amount of processing required. It greatly increases it!  That is because to evaluate it, it first builds a set of all DiscoveredProcess nodes, then does all the traversals to get to their DiscoveryAccess nodes so it can filter them. That is substantially more work than accessing the state of all the DiscoveredProcess nodes. In fact, if you combine that kind of condition with a simple condition on the node state (such as equal conditions or has subword conditions), the system will process the state-based part of the query first, before doing the filtering on related DAs, because it knows it is more efficient that way.


                    If you have a complex condition on the process state (involving complex regular expressions, for example), you do benefit from filtering to only the most recent processes, but you have to do it in a way that reduces the traversals, not increases them:


                    search DiscoveryAccess where _last_marker defined

                    traverse DiscoveryAccess:DiscoveryAccessResult:DiscoveryResult:ProcessList

                    traverse List:List:Member:DiscoveredProcess where (process condition)


                    That way, you start from a set of just the most recent DiscoveredProcess nodes before you start retrieving anything about them.

                    3 of 3 people found this helpful
                    • 7. Re: Get Host out of DiscoveredProcess
                      Bob Anderson



                      Thank you for pointing out how the perfomance can vary so greatly when selecting good or poor starting points for the searches