Remedy - FTS - Full Text Search (FTS) indexing: The magic with ft_pending table and what its contents mean

Version 13
    Share:|

    This document contains official content from the BMC Software Knowledge Base. It is automatically updated when the knowledge article is modified.


    PRODUCT:

    Remedy AR System Server


    COMPONENT:

    AR System


    APPLIES TO:

    Remedy AR System v.9.x - Full Text Search (FTS)



    QUESTION:

    What does all the information in the ft_pending table mean?


    ANSWER:

     

    This document gives a summarized view of most of the commonly seen ft_pending record details when indexing request is performed.
    This article is also posted on the BMC Communities

    - Negative things are highlighted in red just for differentiating with others.
    - "+" indicates valid positive value according to the request.

      

    All others are actual values (e.g. operationType)

                                                                                                                                                                                   
    Use caseft_pending columnsComments
    operationTypeschemaIdentryIdupdateTimeseqNum
    CREATE ENTRY0++++ 
    ft_pending record picked up for processing0+-entryId-1+ 
    If entry is successfully processed or form was deleted or entry was deleted from T tableIndexer enqueues entry for delayed commit and delete.Committer thread bulk deletes the entries enqueued every 1 minute.
                                                                                                                                                                                    
    Use caseft_pending columnsComments
    operationTypeschemaIdentryIdupdateTimeseqNum
    SET ENTRY1++++ 
    ft_pending record picked up for processing1+-entryId-1+ 
    If entry is successfully processed or form was deleted or entry was deleted from T tableIndexer enqueues entry for delayed commit and delete.Committer thread bulk deletes the entries enqueued every 1 minute.
                                                                                                                                                                                    
    Use caseft_pending columnsComments
    operationTypeschemaIdentryIdupdateTimeseqNum
    DELETE ENTRY
    2

    -schemaId

    +

    +

    +
    All other ft_pending records for this entry are deleted before insert.
    Schema ID is negative to avoid unique index violation if Merge Entry call comes in for the same entry ID again after deletion.
    ft_pending record picked up for processing2-schemaId+++No change.
    If entry is successfully processedIndexer enqueues entry for delayed commit and delete. 
                                                                                                                                                                                    
    Use caseft_pending columnsComments
    operationTypeschemaIdentryIdupdateTimeseqNum
    INDEX ENTRY3++++Only AR upgrade installer can create such entries while optimizing ft_pending entries.
    ft_pending record picked up for processing3+-entryId-1+ 
    If entry is successfully processed or form was deleted or entry was deleted from T tableIndexer enqueues entry for delayed commit and delete.Committer thread bulk deletes the entries enqueued every 1 minute.
                                                                                                                                                                                    
    Use caseft_pending columnsComments
    operationTypeschemaIdentryIdupdateTimeseqNum
    FORM DELETE INDEX
    - When a form is deleted

    - When last FTS enabled field is deleted

    - When last FTS enabled field is modified to remove index property

    - Similar changes in overlay layer

    4

    +

    +uniqueNum

    +

    +

    All other ft_pending records for this form are deleted before insert.
    ft_pending record picked up for processing4++uniqueNum++No change
    If entry is successfully processed or form was deleted or entry was deleted from T tableIndexer enqueues entry for delayed commit and delete.Committer thread bulk deletes the entries enqueued every 1 minute.
                                                                                                                                                                                                                                                                          
    Use caseft_pending columnsComments
    operationTypeschemaIdentryIdupdateTimeseqNum
    FORM CREATE / RE-INDEX
    - When form’s FTS properties are changed (title/keyword/environment)

    - When boost field (177) is deleted from the form

    - When global reindex is performed on server

    - Upgrade installer creates this for all vendor join forms when upgrading from a pre-90 version

    - When application command “Application-FTS-Reindex-Form” is executed without updatetime

    - SSI 341 for form(s)

    - Similar metadata changes in overlay layer

    5

    +

    +uniqueNum

    +

    +

    All other ft_pending records for this form are deleted before insert.
    ft_pending record picked up for processing-5++uniqueNum++ 
    After 1000 entries are processed by schema reindex-5++uniqueNum++No change in ft_pending record, but explicit Lucene commit() happens every 1000 entries.
    After first 5000 entries are processed by schema reindex-5++uniqueNum5000+ft_pending.updateTime value is updated after every 5000 records i.e. 5000, 10000, 15000, 20000, …
    If entry is successfully processed or form was deletedIndexer enqueues entry for delayed commit and delete.Committer thread bulk deletes the entries enqueued every 1 minute.
                                                                                                                                                                                                           
    Use caseft_pending columnsComments
    operationTypeschemaIdentryIdupdateTimeseqNum
    PERIODIC SCANNo ft_pending record is created in this case for operation Type = 6  
    “Application-FTS-Reindex-Form” with 2nd parameter of timestamp after form name6++uniqueNum++Also ft_pending.fieldId is set to value of 2nd argument i.e. timestamp
    ft_pending record picked up for processing-6++uniqueNum++ 
    If entry is successfully processed or form was deletedIndexer enqueues entry for delayed commit and delete.There is also an explicit commit on Lucene, and committer thread bulk deletes the entries enqueued every 1 minute.
                                                                                                                                                                                                                               
    Use caseft_pending columnsComments
    operationTypeschemaIdentryIdupdateTimeseqNum
    FIELD REINDEX
    - When field property for FTS index is changed from None to FTS+MFS or MFS-Only

    - When field property for MFS Category Name is changed from empty to some value

    - When field is created on a join/view/vendor form with FTS index or category name

    - Similar changes in overlay layer


    7


    +


    +uniqueNum


    +


    +


    All other ft_pending records for this form are deleted before insert.
    Field reindex
    - When MFS category name is changed or deleted

    7

    +
    Old Category Name
    +

    +
     
    ft_pending record picked up for processing7+<No change>++ 
    If entry is successfully processed or form was deletedIndexer enqueues entry for delayed commit and delete.Committer thread bulk deletes the entries enqueued every 1 minute.
                                                                                                                                                                                    
    Use caseft_pending columnsComments
    operationTypeschemaIdentryIdupdateTimeseqNum
    FIELD DELETE INDEX
    - When FTS index / category enabled field is deleted

    - When FTS index / category name is removed from field properties

    - Similar changes in overlay layer

    8

    +
    Old Category Name, if exists, else +uniqueNum
    +

    +
     
    ft_pending record picked up for processing8+<No change>++ 
    If entry is successfully processed or form was deletedIndexer enqueues entry for delayed commit and delete.Committer thread bulk deletes the entries enqueued every 1 minute.
                                                                                                           
    Use caseft_pending columnsComments
    operationTypeschemaIdentryIdupdateTimeseqNum
    GLOBAL REINDEX
    - When reindex checkbox is used from Server Info page

    - SSI 234

    - FTS config change
    (e.g. case sensitivity change)
    No ft_pending record is created in this case for operation Type = 9All records from ft_pending are cleaned before insert.
    But this results in creation of many ft_pending records with operation type = 5, one record for every form that has at least 1 FTS enabled field
                                                         
    In all above cases, if there is a failure in processing an entry due to Lucene issue<no change><no change><no change>-2<no change> 
      

      

     


    Article Number:

    000152517


    Article Type:

    FAQ/Procedural



      Looking for additional information?    Search BMC Support  or  Browse Knowledge Articles