There have been several enhancements in recent releases of BMC Remedy IT Service Management Suite, BMC Atrium CMDB Suite, and BMC Remedy AR System Server that change where data is stored, how to customize the user interface (UI), and preserve those changes during application upgrades. These changes include:
- BMC Remedy IT Service Management moved asset management lifecycle attributes out of the CMDB to the AST:Attributes form
- BMC Remedy AR System Server introduced granular overlays, see The Pulse: Using granular overlays to manage customizations
- BMC Atrium CMDB Suite addressed issues with the Synchronize UI feature, also known as Sync UI to Asset or cmdb2asset
In this post, I will discuss how these changes impact the guidelines for successfully extending the data model and customizing the user interface. I will also try to put some issues and experiences in context of their role and best usage of these features.
Adding data to AST:Attributes in relation to CIs
In Asset Management version 8.0, asset lifecycle attributes such as those that track status and costs, those were moved out of the CMDB to the AST:Attributes form. The Data structure change is described in more detail in the BMC IT Service Management architecture documentation. The relationship between the CI and the AST:Attributes record is stored as a foreign key relationship where:
- If the CI is not identified, Reconciliation Identity on AST:Attributes = Instance Id of the CI
- If the CI is identified, Reconciliation Identity on AST:Attributes = Reconciliation Identity of the CI
The first case is temporary - once the CI is identified in the CMDB, the AST:Attributes record is updated to contain the Reconciliation Identity of the CI and becomes available to the Asset viewer through the Asset Management application.
Below are different scenarios of how data is populated to AST:Attributes.
- ITSM is upgraded from version 7.6.04 to 8.x
Once you upgrade to ITSM 8.x, the upgrade process moves the data from the CMDB into AST:Attributes records and removes the attributes from the CMDB. Any workflow or integration referencing them can be updated after phase 2 of the ITSM upgrade which uses workflow to sync the legacy attribute values to AST:Attributes. The third phase of the ITSM upgrade removes the legacy attributes from the CMDB.
- An ITSM user adds a new CI from the Manage CIs feature, using an Asset Sandbox. (BMC.ASSET.SANDBOX)
An entry is created in AST:Attributes with the instanceid of the CI. This is a temporary foreign key relationship until the CI is identified. When the CI is identified by Reconciliation, workflow updates the AST:Attributes entry to have the correct value of Reconciliation Identity of the CI.
- An ITSM user adds a new CI from the Manage CIs console, without using an Asset Sandbox.
The CI and AST:Attributes entry are created with a value of Reconciliation Identity to relate them.
- A data provider like ADDM populates data to the CMDB.
The AST:Attributes entry is created when the CI is reconciled.
However, an AST:Attributes record is not created for all classes. Workflow checks if the class is in one of the classes which tracks asset lifecyle attributes. This list is stored in the AST:ClassAttributes form and you can add new entries to the form if there are new classes for which asset life cycle attributes should be tracked.
- Data is loaded to Asset Management application from a legacy data import or third party integration.
Two different methods can be used - either the Data Management Tool with Transactional_CI spreadsheet or the process in knowledge article KA400491.
- In another scenario, data is loaded and reconciled in CMDB 8.1 and ITSM 8.1 is installed later on the system. How are the AST:Attributes records created in this situation?
This can be done several ways. Either create a one-time escalation to create entries in AST:Attributes with default values, or export data from CMDB to get the Reconciliation Identity and Instance IDs for the CIs, and use one of the above methods to import it with AST:Attribute data.
Determining the proper location of new attributes
Before defining a new attribute, consider where is the correct place to store it. Since there are two possible locations – in the CMDB vs. on the AST:Attributes form - you have to ask the question:
"Should it be added as a field on AST:Attributes or should it be added as an attribute in CMDB?"
Below are some criteria to make that decision:
- Is the attribute discoverable from the CI itself?
- Is it a property of the CI itself, or an attribute which is being track about the CI?
- Will the data be primarily accessed using the BMC Remedy IT Service Management applications?
Using this criteria, if the attribute will be added to the CMDB, the process works the same as in previous versions - add it in Atrium Class Manager and run Sync UI to update the AST form to make it visible to users of the Asset Management application.
If the field will be added to the AST:Attributes form, the process is essentially to add a new field to the form, and update the AST forms to expose it in the join forms. The use of additive view overlays is recommended so these changes are preserved when you upgrade the ITSM applications. In the example below, I added the field MyNewLifecycleAttribute to AST:Attributes and used Remedy Developer Studio to update the Computer System form to make it visible on the AST form.
Best Practices on making changes to the data model – in CMDB or AST:Attributes
Following these guidelines will avoid introducing issues when making data model changes:
- Extend or add to the data model, rather than changing, removing, or making attribute/field properties more restrictive.
When a new class or attribute is added, it is certain that no existing applications or use cases will be affected. This is because the new object did not exist – so nothing could be using it. The only concern when adding an attribute is making a new attribute or field required – because there may be a significant amount of existing data without a value specified, so adding a required attribute of field can introduce issues.
Modifying an existing attribute – for example, to make the size smaller or to make it a required attribute – can introduce many types of issues. There may be application features or data use cases which do not meet the new restrictions and issues may occur well after making the changes.
- Back up the original definitions and data before making the changes. See the recent Connect with Atrium webinar on the Common Data Model for suggestions on how to do this for changes to the CDM.
- Consider the impact on existing data. If a new attribute or field is added – even with a default value – existing records will not have this value populated. When adding a new attribute or field with a default value, existing CMDB data import methods can be used to update existing data. If the field was added to AST:Attributes form, either use the Data Management Tool (DMT) to load the data or see knowledge article KA400491 for a method of loading data using other import mechanisms.
Maintaining a consistent UI
After the class or attribute is added to the data model, it needs to be included in the user interface. This happens automatically for extension of the CMDB data model because the class forms are updated to include the attribute – at least in the CMDB user interface forms. There are more details in the documentation on Control of the layout of class forms.
If BMC Remedy IT Service Management applications are installed on the server, it is recommended to also run the Sync UI process to update the Asset Management UI with these changes so users can access the new classes and attributes.
This feature is accessible in accessible from the Application Administration Console, by choosing from the menu:
Custom Configuration > Asset Management > Advanced Options > Sync Asset UI with CMDB
This Sync UI feature – also known as Synchronize UI, also known as cmdb2asset – performs much of the work of updating forms and attaching workflow. However, the new attribute is added to a Custom page which is not visible. It is expected that the administrator will make the decision to either make the page visible, or perhaps move the attribute to another page on the form. These manual changes are made in BMC Remedy Developer Studio. For changes made in Remedy Developer Studio, the recommendation to preserve customizations is to use overlays, so we’ll explore the impact of doing that in the paragraphs below.
Changes made in both tools - Atrium Class Manager and Sync UI – drive workflow changes to AR System workflow base objects. They do not directly update or interact with overlays. This does not mean they are completely independent, as we will get into below.
Retaining changes during upgrades
Changes made through the CMDB API – using Class Manager or the CMDB API – store metadata which describes the changes to the data model. During CMDB upgrades, CMDB understands and preserves changes and additions to the data model. Changes in the AR System workflow and database objects are driven from the CMDB metadata, so upgrading or making additional changes in Class Manager preserve the changes to the workflow objects.
As for the AST:Attributes form, changes made in BMC Remedy Developer Studio should always use the Best Practice Customization Mode so the workflow changes are stored on overlays. When ITSM upgrades update forms or workflow, they update the base objects and the overlay objects are retained – except in the rare case where the base object is deleted during the upgrade. You can view the details of the workflow affected in the upgrade, which is included with the ITSM release.
Overlays and CMDB forms and workflow
As noted above, changes to CMDB classes should be made through the CMDB Class Manager or CMDB API calls. This process updates the CMDB metadata and all related AR System forms and workflow. Changes to the functional behavior of the CMDB should never be made directly in the AR System tools such as BMC Remedy Developer Studio because this introduces inconsistencies between the CMDB metadata and the AR System objects. This is unsupported, can cause problems, and is contrary to best practices. The guideline to understand “functional behavior” is as follows:
If the change can be made in Class Manager, then it should be made in Class Manager.
Customizing the user interface by moving attributes around on the form does not affect the functional behavior of the CMDB, nor is it a task that can be performed in Class Manager. As this is not a change to the functional behavior of the CMDB it can be made in BMC Remedy Developer Studio. All the standard guidelines to performing Remedy workflow customization apply here including testing on development servers, documenting changes, backing up existing workflow, and acquiring the necessary training and knowledge to understand how to customize Remedy workflow.
Now let’s consider an example of using overlays to make these changes, first using AR System version 7.6.04 and then with AR System version 8.1.
In this example, let’s consider the case of making changes to the user interface by modifying views to move attributes on the form BMC.CORE:BMC_ComputerSystem using BMC Remedy Developer Studio. For servers with BMC Remedy IT Service Management installed there the Asset Management forms perform the role of user interface so this example is likely only applicable to servers without ITSM installed.
In AR System version 7.6.04, BMC Remedy Developer Studio is launched, switched to Best Practice Customization mode, and the BMC.CORE:BMC_ComputerSystem form is edited. An overlay of the form is created. This makes a copy of all of the properties of the form – the fields on the form, the field properties, and the views. Changes are made to the views according to the requirements and the changes are saved. At this point everything works fine – AR System is using the overlay instead of the definition of the BMC.CORE:BMC_ComputerSystem form, but the only difference between the two are the changes to the view. Next, let’s consider the case that a new attribute is added to the class in Class Manager. This process updates the underlying AR System forms and workflow to include this attribute – it updates the base objects not the overlay. This is the designed behavior – because BMC code changes are made in the base. However, note that we have just introduced a discrepancy between the CMDB metadata and the AR System objects! The CMDB metadata includes the new attribute, and it expects the AR System forms also have this field. However, since AR System is using the overlay instead of the base object, AR System believes the field does NOT exist on the form.
This is sometimes explained as "overlays being incompatible” with CMDB forms, and sometimes explained as they “conflict” with CMDB changes. But in reality, the challenge is that the use of overlays introduces data model inconsistencies between the CMDB and AR System objects. This is usually where errors would be encountered. One way to fix this is to delete the overlay and re-create it. This is essentially making another, more recent, copy of the form properties to address the data model inconsistency, until another attribute is added via CMDB Class Manager. Ironically, the use of overlays to adjust the view is mostly unnecessary because the CMDB data model changes used an inheritance model to update the forms, so customizations to the view are generally preserved when classes are updated. So on CMDB versions 7.6.04 – the best advice is still “do not use overlays on CMDB forms”.
Creating an overlay on AR System 7.6.04 required creating an Overwrite overlay for the entire form – data model properties and view properties alike. In AR System 8.1, the concept of overlays was extended to support more types of overlays – additive, overwrite, inherit – and to be more granular. So on AR System 8.1, you can create an additive overlay and create it on only the views of a form. This allows making changes to the view without impacting the functional behavior of the data model. So in AR System 8.1 or later overlays can be used on CMDB forms if proper care is taken to only create additive overlays and only on view properties.
Sync UI and overlays
The role of Sync UI to Asset is to update the Asset Management user interface forms to reflect additional classes or attributes added to the data model in the CMDB. It leverages metadata in the SHR:SchemaNames form, plus an inheritance model that leverages existing forms and workflow for parent classes to either build or update the form. Sync UI creates and updates base objects – which is appropriate for BMC-made workflow changes.
The SynchronizeUI log generated on the server gives a good picture of the activity it performs – checking if the form already exists, looking for new attributes, and attaching workflow which is related to parent class forms to ensure the form works consistently. This log is also a good reference while syncing to determine when the process is complete. Sync UI updates the existing form rather than removing and recreating it, so existing changes to the form are retained. Sync UI does not have metadata to tell it where to add new attributes so it follows a standard routine of adding new attributes in the next available position on a set of custom tabs. This avoids the need to manually edit each AST form in the class hierarchy to add the attribute as described in KA404401 for adding an attribute via AST:Attributes.
How does this behavior impact the use of overlays?
In AR System 7.6.04, all overlays are what we now call “overwrite” overlays. When you create an overlay of a form, for example AST:ComputerSystem, AR System uses your overlay instead of the base object. Sync UI updates the base object, so the expected behavior is that changes made via Sync UI are not visible on the form – only the changes in your overlay exist. Since the fields are propagated to the base object, BMC Remedy Developer Studio could be used to simply add them to the overlay – but this is a manual step. Creating an overlay of workflow which governs the behavior of the AST form also causes problems on AR System 7.604. When you create a new class in CMDB and run Sync UI, it updates many active links and filters to also run on the new AST form generated. If there is an overlay on the workflow, it supersedes the base object and the workflow does not execute on the new form. To avoid problems like this, it is better to generate all the classes in CMDB, use Sync UI to build the AST forms, and only then create overlays on the AST forms.
In AR System 8.1 and later, granular overlays allow overlaying smaller components of the form and the can be designated as Additive overlays. This allows the overlaid objects to be maintained separately and the both base and overlaid attributes are displayed on the form. As long as granular additive overlays are used and fields are not positioned in a way to conflict with the usual placement of new attributes added by Sync UI, the overlays should work fine with Sync UI. But it means you may need to review your coordinates of fields on the form.
Sync UI fixes
A number of fixes have been made to Sync UI in recent CMDB service packs. The expected behavior is for Sync UI to bypass AR System configurable options meant to restrict end user behavior. For example, the Max-Entries-Returned-by-Getlist setting should not block Sync UI. This is considered a defect because the automated tool Sync UI should not be bound by this restriction. This defect is fixed in CMDB 7.6.04 SP5 and will be fixed in the upcoming CMDB 8.1 Service Pack 1, along with an issue described in knowledge article KA399521 regarding Sync UI updating the Status attribute to be invisible. See the release notes for specific issues fixed in Sync UI. I only mention it here because some of these issues are sometimes considered evidence of a conflict with overlays, when in fact, they are just defects fixed in the latest service pack.
How Auditing and Archiving changes with data stored in AST:Attributes
Asset lifecycle attributes are often the ones which would be the most interested in auditing changes since they are made by users instead of discovered automatically. Since these attributes are no longer stored in the CMDB, there is a slightly different way of enabling auditing, but both mechanisms ultimately use the same AR System feature to perform the auditing. See my colleague Jared’s recent blog post – The Pulse: Auditing your Assets for more details on this topic.
AST:Attributes only stores current attribute values so consider whether this data needs to be archived when archiving data from the CMDB, if the goal will be to capture data at a particular point in time.
The names of the forms AST:Attributes and AST:ClassAttributes are similar in name to CMDB metadata forms which perform completely different roles, so for clarity, I always refer to these forms by their explicit full names. Below is a quick summary of forms, whether they belong to CMDB or Asset Management, and their purpose.
|AST:Attributes||Asset Management form that contains data instances|
|Attribute Definitions (OBJSTR:Attributes)||CMDB definiton of attributes and their properties|
|AST:ClassAttributes||Asset Management form to hold list of forms which use AST:Attributes|
|Class Attributes||CMDB Definition of attributes on the class|
Below is a summary of the key points addressed in the article:
Both CMDB data model and Sync UI changes to the interface forms act on the AR System base workflow objects. Neither is aware of or directly interacts with overlays in AR System. AR System overlays can be used effectively with CMDB and AST forms by using additive overlays specifically to views – to adjust the user interface but not change the functional behavior. On AR System versions prior to version 8.1, it was not possible to create such granular or additive overlays, so using overlays if you plan to afterwards run Sync UI on those versions is not recommended.
When adding an attribute, consider whether it is a discoverable attribute of the CI itself or a non-discoverable attribute which is used to track the lifecycle of the asset. This will help determine if the attribute should be added to the CMDB or to AST:Attributes. See Knowledge article KA404401 regarding steps for adding to AST:Attributes.
Always use additive view overlays for changes made in BMC Remedy Developer Studio, including changes to the AST forms after running Sync UI.
When auditing changes for attributes stored on AST:Attributes, make the changes via BMC Remedy Developer Studio.
Archiving, deleting or resetting ReconciliationIdentity data in BMC.ASSET dataset – It was never a good idea to do this, because it orphans ITSM relationships which refer to the CI’s by ReconciliationIdentity. Now data in AST:Attributes is added to that list of data that would be lost or orphaned. When data is removed or reset using cmdbdiag, it operates at the database level so filters which would remove or update data in ITSM forms do not fire.
I hope this summary provides clarity on how to use these features effectively, and make sense of past experiences and issues encountered. Please use comments below to provide feedback. See BMC Remedy Pulse Blogs for similar posts.