Customers frequently ask for help with custom patterns in the Community.
Here is some general information to help you get started.
See this YouTube video: How to add or modify BMC discovered data and synchronize to CMDB? - YouTube
TPL - Template Pattern Language. Some users use the terms "TPL" and "pattern" interchangeably.
A module (contained inside a TPL file) contains one or more pattern or syncmapping statements.
There are 2 types of TPL customization for the BMC Discovery product:
1) Discovery patterns typically add and/or modify Discovered information.
There are many OOTB Discovery patterns (defined by the TKU).
There may also be custom Discovery patterns.
The keyword "pattern" is used to define a pattern.
2) syncmappings define the behavior of the CMDB Sync process.
There are many OOTB syncmappings (defined by the TKU).
There may also be custom syncmappings.
The keyword "syncmapping" is used to define a syncmapping.
Should I edit the OOTB patterns and syncmappings?
If you edit the OOTB pattern, the edits will be lost with the next TKU update.
How can I change the behavior since I should not edit the OOTB patterns and syncmappings?
1) To modify the behavior of Discovery patterns, there are 2 methods:
- Preferred method: add an additional (custom) Discovery pattern
Each discovery pattern has a trigger statement. If the trigger succeeds, then the pattern body is executed.
Example of Discovery pattern:
- Discovery Override pattern
Sometimes, an Override pattern is required to modify the OOTB Discovery pattern behavior.
Only use this method when absolutely required.
An Override pattern is only to be used when an Extension can not provide the desired functionality.
An Override pattern redefines the entire OOTB pattern, with special additional syntax to Override the base pattern.
2) To modify the behavior of the CMDB syncmappings, you can add one of these 2 types of custom patterns:
- Syncmapping Extension (also called "augment")
Syncmapping extensions can add new data to the CMDB, and/or modify data in the CMDB.
Example: if you wish to add the age_count information for a Host to the CMDB, use an Extension.
Features of Syncmapping Extensions:
- Extensions can add new attributes or modify attribute values in the CMDB. However, they can not delete attributes.
- Extensions can add new relationships to the CMDB. However, they can not delete or modify relationships.
Limitations of Syncmapping Extensions:
- Extensions can not delete attributes from the CMDB.
- Extensions can not delete or modify relationships in the CMDB.
Examples of Syncmapping Extensions:
- Syncmapping Override
Sometimes, a Syncmapping Override is required to modify the OOTB behavior because of limitations of Syncmapping Extensions.
Always use a Syncmapping Extension when possible.
An Override is only to be used when an Extension can not provide the desired functionality.
An Override redefines the entire OOTB syncmapping, with special additional syntax to Override the base pattern.
Example of CMDB Syncmapping Override pattern:
Writing a custom pattern is like writing a small software program.
Here are some best practice steps for writing/testing a pattern:
1) Write the pattern on your laptop using a text editor.
Or, try the experimental IDE: Experimental development IDE for TPL
Hint: It is less confusing to always edit the pattern from your text editor or IDE instead of editing the pattern directly in the UI.
Hint: Add log.info statements while you are debugging your pattern. Later, you can change those from log.info to log.debug.
Hint: Put an identifier (such as your name) in each log.info statement so that you can easily identify that it is your log statement.
Hint: It may be helpful to number each log statement so you can easily find it in the pattern.
log.info("LISA 1: Pattern was triggered. Here I am in the pattern body. Host name=%host.name%");
log.info("LISA 2: Inside the 'for each fsmount' loop. fsmount=%fsmount.mount%");
2) Upload the pattern from the Manage->Knowledge page
The Upload action checks for syntax errors in the pattern, and if there are no syntax errors, then it Activates the pattern.
3) Fix syntax errors in your text editor or IDE, and then Upload again in the UI until there are no more syntax errors
4) If you added new attributes (which are needed by your CMDB Syncmapping) to your CMDB,
then do this after adding the new attributes one time: Restart the Discovery Services
5) Test the pattern / Fix the pattern / Test / Fix until you are happy with the pattern
2 ways to test Discovery patterns:
A) Create a Manual Group and the "Run Pattern" feature:
With "Run Pattern", you will see the log.info and log.debug statements on the UI page easily.
The "Run Pattern" tells you if any of the nodes in the Manual Group triggered your pattern.
If your pattern triggers on a SoftwareInstance, then you must have a SoftwareInstance in your Manual Group.
If your pattern triggers on a Host, then you must have a Host node in your Manual Group.
And so on.
B) Run the Discovery of the Host/Device which should trigger your pattern.
Look for your log statements in this log file: /usr/tideway/log/tw_svc_eca_patterns.log
Make sure your Discovery pattern gets triggered
The Discovery patterns have a "triggers" statement. The trigger statement is very important to define correctly.
If the trigger statement does not succeed, then the pattern body will not be executed.
Example of a trigger statement:
on process := DiscoveredProcess created, confirmed where
cmd matches regex "(?i)CouchDB\S+\\w?erl\.exe" or cmd matches unix_cmd 'beam'
4 ways to test CMDB Syncmappings:
A) Pick a pertinent Device and choose Actions->CMDB Sync (from the Consolidator)
B) With Continuous Sync running, Scan the Device (from the Scanner)
C) Perform a Resync from the Consolidator (This is long-running. Only do this when necessary).
D) Pick a Host/Device and choose Actions->CMDB Sync Preview (from the Consolidator)
If your syncmapping syncs to a new class such as BMC_FileSystem, then check the preview visualization for the new class.
If your syncmapping only changes/adds attributes, you will not see any change in the preview visualization.
All of the above actions will log data to this log file: tw_svc_cmdbsync_transformer.log
Actions A,B,C will change the data in the CMDB.
Action D will not change the data in the CMDB. It is only a Preview. But, it logs the messages to the log file.
Hint: If the CMDB Sync Preview does not work, then there is something very wrong with your pattern.
Resources to help with custom patterns.
1) The Discovery UI:
The Discovery UI has some information about creating SoftwareInstance patterns magically through the UI:
At the top, there is specific information about modeling a SoftwareInstance in Discovery.
If you wish to have a custom pattern to create an SI, be sure to read that section, as well as the documentation that it points to.
Also, in the Discovery UI are some sample templates:
Sample SI pattern templates:
Sample "location" pattern templates:
Sample pattern template to create a SQL Integration Point:
Sample pattern template which adds addition SQL calls:
Sample Mainframe pattern templates:
Sample External Event pattern template:
Sample CMDB Syncmapping templates:
To utilize one of these pattern templates, perform the following steps:
A) Download the pattern, and save it to your laptop
B) Use an editor on your laptop to edit and save the pattern.
Names surrounded by double dollar signs like $$pattern_name$$ should all be replaced with values suitable for the pattern.
C) Upload your pattern on the Manage-Knowledge page to see if there are syntax errors.
If not, Edit / Upload until the compile errors are gone.
D) Test your Discovery pattern using a Manual Group and the "Run Pattern" feature: Executing patterns manually - Documentation for BMC Discovery 11.3 - BMC Documentation
With "Run Pattern", you will see the log.info and log.debug statements for your pattern.
If you run Discovery without "Run Pattern", you will need to look for your log statements in this log file: tw_svc_eca_patterns.log
E) To test your CMDB Syncmapping, you can add log.info statements into the pattern, upload the pattern, run CMDB Sync,
and then check for the log statements in this log file: tw_svc_cmdbsync_transformer.log
2) The Community
Visit the Discovery community link: Discovery
Click on "Patterns" as seen below:
You will find sample patterns which are made freely available by other customers, and by the BMC Discovery support team and developers.
3) Look directly at the OOTB patterns that are found in the TKU
You can look at the patterns in the UI.
And/Or, you can unzip the TKU zip file, and review the abundance of patterns.
To view the patterns in the UI, you can look on the Manage->Knowledge page.
You will find the Syncmapping patterns here on the page: (Under BMC Discovery Operation -> CMDB Sync):
The Advanced Discovery training course has information about Discovery patterns and custom Discovery patterns.
To this date, the course does not teach about the CMDB Syncmappings.
5) Customer Support can help with certain questions
Support will not write custom patterns or custom syncmappings for you. But, Support may be able to help with specific questions or problems.
Support has access to some samples that may not be in the community. Certain sample patterns are attached to internal KA's.