How to Normalize data successfully.

Version 8
    Share This:

    This document contains official content from the BMC Software Knowledge Base. It is automatically updated when the knowledge article is modified.


    PRODUCT:

    BMC Atrium Core - CMDB


    COMPONENT:

    BMC Atrium Core


    APPLIES TO:

    CMDB Normalization using AtriumCore 8.1 with SW00495466 Hotfix and also 9.1 version.



    PROBLEM:

     

    NOTE: This article does not replace documentation. For specifics of features not discussed in this article please consult http://docs.bmc.com

    Points covered in this article:

    1. CMDB-Inline-Normalization
    2. What triggers Normalization?
    3. Normalization CMDB data in AR Server group
    4. Why does Normalization matter?
    5. Normalization and Multitenancy
    6. "- Global -" Company - What does it mean?
    7. Unknown Versions and Patches
    8. Normalization of objects acquired from discovery sources
    9. Alias Mapping
    10. Market Version default values
    11. Normalization Defects

    ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------


    1. CMDB-Inline-Normalization

    Let's begin by explaining the CMDB-Inline-Normalization setting in ar.cfg (ARDBC).

    This setting only effects manual changes by an end user. Typically someone like an Asset Operator or perhaps even CMDB Administrator that is testing Normalization changes with Dataset level Inline Normalization .
    It is used with the setting of "Preserve Manual Edits" in the Normalization configuration.
    Setting the dataset level "Inline Normalization" is what causes CI's (Configuration Items) to be normalized during a manual update or anytime a CI is created or modified.

    Typically normalization is done via batch job or job running in Continuous mode, but it can be used with dataset level Inline Normalization
    Running batch or continuous jobs will also include normalization of Relationships. Please see BMC documentation for listing of available features.
    Uncheck this setting and Normalization will only normalize in a scheduled batch job or continuous job.

    Visual aid

    The CMDB-Inline-Normalization setting must always be set to True (T). Please don't confuse this setting with "dataset level Inline Normalization".


    2. What actually triggers Normalization?

    Normalization is triggered by saving any changes or adding values into the fields of BMC.CORE forms. These are defined as ConfigurationItem (CI) records where the CMDB class ID is included in the "Normalization Attributes Information" form. For example BMC_Person is not normalized because it is not listed in that form.

    Fields "Model" and "ManufacturerName" are required for normalization to pass.

    These fields (CMDB data Attributes) are as follows:

    - Category (Field Id 200000003. Owned by BMC.CORE:BMC_BaseElement class)
    - Type (Field Id 200000004. Owned by BMC.CORE:BMC_BaseElement class)
    - Item (Field Id 200000005. Owned by BMC.CORE:BMC_BaseElement class)
    - Model (Field Id 240001002. Owned by BMC.CORE:BMC_BaseElement class)
    - ManufacturerName (Field Id 240001003. Owned by BMC.CORE:BMC_BaseElement class)
    - VersionNumber (Field Id 240001005. Owned by BMC.CORE:BMC_BaseElement class)
    - PatchNumber (Field Id 260131107. This one has 64 characters and is owned by BMC.CORE:BMC_ApplicationSystem class . Also field Id 301137300  with 254 characters and owned by BMC.CORE:BMC_Software )
    - MarketVersion (Field Id 530062400. Owned by BMC.CORE:BMC_BaseElement class)
    - Company (Field ID 1000000001. Owned by BMC.CORE:BMC_BaseElement class)
    - AccountId (Field ID 301186800. Owned by BMC.CORE:BMC_BaseElement class) If Company is specified then AccountId will also be considered.

      


    3. Normalizing CMDB data using NORM plugin running in AR Server group:

    Prior to 9.x release the normalization plugin was triggered and could run on any system in the server group without any control. 
    In those versions Normalization could run on ARServer that is "closest" to the call that has received a change to an altered CI.
    By closest we mean that the CI is processed on the server that is currently accessed via Midtier. The actions requested will be performed on "this" server.
    AR Server Dispatcher Service has no specific instructions to run the Normalization Jobs on a specific server.
    Inline settings discussed above will apply in this context. Removing the NORM plugin from configuration can lead to "AR ERROR 8755 Plugin" errors.

    After the 9.x release of AR Server NORM plugin can be set to only run on a specific server by configuring it in the "AR System Failover Ranking" form.
    Please see documentation on AR Server and how to implement Normalization Service using this form.

    4. What impact is there if CIs are not normalized? Why does normalization of data matter?  

    Normalization of data makes sure that all records are in a format that every business unit can consume in the same way. It validates approved and normalized products.

    "What makes Normalization work?" or "How do I get the 'Normalized and Approved' status?"

    - CIs get reconciled by the Reconciliation Engine. CMDB data Reconciliation has a toggle that will only process CIs that are Normalized. This setting is listed as "Process normalized CIs only".

    - Another impact is on running reports on data where CI's have not been normalized. A product might be imported as "Latitude" or "Dell Latitude" and running a report that looks for
      Dell Latitude laptop would exclude all "Latitude" CIs even though they are the same product. Product Catalog is very effective when data is normalized to common and matching values.

    - Software licensing will use and associate MarketVersion with cost of a software package. This helps with cost analysis of "my" production environment.

     


    CAUSE:

    Configuration of Normalization features depends on configuration of AR Server Group


    SOLUTION:

     

    First lets begin with the CMDB Class and what you can do to see what fields need to have a value to be Normalized.
    Each class you want normalized must be listed in the Class Configuration panel of the Normalization console. Each of the fields that have a check in them must have a value to pass normalization.

    This screen capture shows an example using the BMC.CORE:BMC_Product class.

    Normalization Configuration Editor:
    Class Configuration

    In the Normalization Configuration Editor screen capture above you can see that PatchNumber is specified for BMC_Product class. Using the correct class when look for reasons of Failed Normalization is important.
    Below you can see that PatchNumber attribute is visible in CI classes that are children of BMC_BaseElement, but not in BaseElement itself.

    PatchNumber in Attribute Definitions
    PatchNumber in Attribute Definitions form
    This means that looking at data from BaseElement container will not give you the information you need. If PatchNumber has a value and caused the Product CI to fail, then this is not the view that will give you that.
    Please use the class container (form, schema, view) that is closest to the ClassID. For example use BMC.CORE:BMC_Product  instead of BMC.CORE:BMC_BaseElement .

    No PatchNumber in BaseElement

    Patch, Version and Model can vary in Associated Company. This are is often one that is overlooked by data admins. Please make sure you add this to your investigation of CIs that have failed to normalize.
    Once you specify one PatchNumber for a Model then all Versions without a Patch will need to have Unknown PatchNumber associated with it. Otherwise they will fail to normalize.
    Again, here the software is doing exactly what it is told. This creates lots of extra work for the CMDB Admin so please make informed decisions on patch tracking via the CMDB.


    5. Normalization and Multi-tenancy

    Multi-tenancy applies when a service provider uses BMC applications to provide services to multiple clients.
    Company example like ACME Co. and Calbro Inc. may both be providing services using the same application managed on the same AR Server instance.
    Multi-tenancy setting impacts directly into what is put into the Company field of a CI.
    If the AR Server hosted Asset Management application is set to "single tenant" then all values in the Company field should be the same. With that setting all company values default to that company.
    A product (or a model) typically has a - Global - association and be used as such by all Companies consuming the CMDB data. However, one or more Companies may have an exclusive version of PatchNumber that is not common for other Companies.
    These Version(s)/PatchNumber(s) would then be only associated with these Companies. A Model should almost always be associated with - Global -, but the version can change. Most or all of your Models will be associated with - Global - Manufacturer.

    In the Product Catalog this translates into "- Global -" Company and Product association.
    If the AR Server hosted Asset Management application is set to multi-tenants as seen here:

    System Setting


    6. -Global- Company - What does it mean?
    The values of Model and Manufacturer must be found in the Product Catalog (PCT) and associated with the "- Global -" Company:

    PCT Setup

    It is not necessary to have the Normalized Product checked to Yes for Model and Company Association.

    The same thing applies to Patches. Once a  patch is specified then from that point on that Model will require a Patch number to be set or set as Unknown.





    7. Unknown Versions and Patches

    If a Version is being used with a product and no Version is specified in the CI's Version field then a new Version must be added with Unknown.

    Unknown Version


    Conversely if any "known" Versions are associated with a Company then each Company must have this version associated with it. Otherwise it is all a - Global - Company.
    Consider that a product model is used by everyone and hence this will usually always be '- Global -'.

    Products that are exclusively used by one Company are the only products that would not be Global. Such products would not be visible to anyone else.
    Product that are common, such as Microsoft Windows operating system, will always be Global and only the Version could be associated with a non-Global Company but only if the other companies don't use it.
    Please keep in mind that you will not be able to use the '- Global -' Company once you associate a Version to a unique company. These two relationships can not coexist. A product is either Global or not.

    Model Version and Company

    8. Impact on data found by overlapping Discovery

    This setting is important to understand because you might find that the data discovery sources are discovering CIs with Version values, but not all the time.
    Some of the discovered items may not provide Version and since our goal is to have the data normalized without a value then this does not make logical sense. 
    The Version is either important to your Organization or it isn't. To say that we're tracking Versions of a Model but then accept a value that indicates UNKNOWN makes no sense to the program.
    Once a version is added to Version and Company association then that Model, Manufacturer and Company (even if - Global -) must have a valid value to normalize. We've added the UNKNOWN keyword to our code to solve this.
    Using "Unknown" is a workaround for the software code to be able to accept CIs without the version. We had to do this because Version is not a guaranteed discovered item and the data needs to be sorted. A NULL value does not do well while used in data sort situation. It's a feature.

    Best practice would be to allow the Normalization Status to fail and then work a case using Change Management to find out why the Version is missing. It's definition and assignment should be documented for reference.

    9. Model and Manufacturer Alias and Alias Mapping

    This area tends to confuse quite a lot of users because of the two apparent alias mappings by PCT and Norm. 
    Product Catalog Mapping and Normalization Alias Mapping will only work in the following sequence.

    NE Alias Mapping explained in pictures


    Only use the Normalization Alias from the Product Catalog Console for Model or Manufacturer mapping:

    User-added image

    Through this form you can setup mapping for Manufacturer values to map to actual manufacturer name. Also Model Name to actual model name.
    This works rather well and it populates the CTI values as well. However, the sequence is important.

    For example here is a screen capture of what a BMC_Product CI looks like before normalization:

    User-added image


    And here it is after Normalization. No Product Catalog Alias Mapping is necessary to have this work.
    In fact it actually makes it more complicated than it needs to be. Please only use the Normalization Alias to get the data normalized.

    User-added image


    10. Market Version and default values

    You may have noticed that Market Version is populated with a value that is the same as Version when Normalization is set to Dataset Inline mode (see point 1) or after a Normalization Job was completed.
    This is set automatically be the following rule found in Normalization Features known as "Rulename98". This rule is configured to populate the Market Version field when InstanceId and Version is not empty.
    If you've configured your Market Version in the Product Catalog setup to be a different value then this rule could override that. You can change this rule to only show the "Major" part of a Version.
    For example 10.1.001 is broken down to 10(Major).1(Minor).001(no designation). Update the rule to show only the Major part of the string value.

    Note: Since MarketVersion only matters to Licensing where the cost associated with the product release rather than patch, using Major is most common for all products.

    Market Version Rule

    Please note that there is a know defect (SW00520230) while searching the Normalization Features console for rules by their name.
    The search feature does not work and you'll have to scroll through the list to find the rule. Also scrolling does not work if you've tried to search once and you will need to reopen the interface to find it.


    11. Normalization and code defects

    Some product defects do exist. Please review the release notes of each product release and its related Service Pack to see if you're encountering defects that do match the features described in this article.
    Some known issues that were released with the 8.1 code line have already been fixed with a hotfix which will normalize data as described above (per design specification).
    Additional work is being done to address multiple matches in the PCT. These only impact products where Manufacturer is Unknown.

     


    Article Number:

    000119958


    Article Type:

    Solutions to a Product Problem



      Looking for additional information?    Search BMC Support  or  Browse Knowledge Articles