1 2 Previous Next 21 Replies Latest reply on Sep 14, 2018 9:44 AM by young so

    ServerGroup installation: FTS does not work anymore

      Hi teams,

       

      I ask you because since the establishment of a second server on our infra, FTS no longer works. Exemple: Want to search an incident on global search returns 0 entries+> Impossible to attach parent/children incidents,...


      Note that we don't want use FTS on SERVER2!


      On the server1 (used as primary FTS server):


      Here is the armonitor.conf extract:


      # FTS Primary

      /usr/java6_64/jre/bin/java -Xmx2048m -classpath /its1/ARSystem/pluginsvr/fts/primary:/its1/ARSystem/pluginsvr/

      fts/core:/its1/ARSystem/pluginsvr:/its1/ARSystem/pluginsvr/arpluginsvr81_build001.jar: com.bmc.arsys.pluginsvr

      .ARPluginServerMain -x SERVER1 -i /its1/ARSystem -m

      # FTS Secondary

      /usr/java6_64/jre/bin/java -Xmx1024m -classpath /its1/ARSystem/pluginsvr/fts_itsprd2/secondary:/its1/ARSystem/

      pluginsvr/fts_itsprd2/core:/its1/ARSystem/pluginsvr:/its1/ARSystem/pluginsvr/arpluginsvr81_build001.jar: com.b

      mc.arsys.pluginsvr.ARPluginServerMain -x SERVER1 -i /its1/ARSystem -m

       

      Here is the ar.conf extract:

       

      Server-Plugin-Alias: ARSYS.ARDBC.FTSCONFIG ARSYS.ARDBC.FTSCONFIG SERVER1:9980

      Server-Plugin-Alias: ARSYS.ARF.FTS ARSYS.ARF.FTS SERVER1:9998

      Full-Text-Configuration-Directory: /its1/ARSystem/ftsconfiguration/conf

      Full-Text-Collection-Directory: /its1/ARSystem/ftsconfiguration/collection

      Full-Text-Mode: Server-Group-primary

      Full-Text-Disable-Indexing: F

       

      FTS1.png

      FTS1_2.png

       

       

      On the server2 (not used for FTS):


      Here is the armonitor.conf extract (note that process lines are commented, then, no process FTS are running).


      # FTS Primary

      #/usr/java6_64/jre/bin/java -Xmx3072m -classpath /its1/ARSystem/pluginsvr/fts/primary:/its1/ARSystem/pluginsvr

      /fts/core:/its1/ARSystem/pluginsvr:/its1/ARSystem/pluginsvr/arpluginsvr81_build001.jar: com.bmc.arsys.pluginsv

      r.ARPluginServerMain -x SERVER2 -i /its1/ARSystem -m

      # FTS Secondary

      #/usr/java6_64/jre/bin/java -Xmx3072m -classpath /its1/ARSystem/pluginsvr/fts/secondary:/its1/ARSystem/plugins

      vr/fts/core:/its1/ARSystem/pluginsvr:/its1/ARSystem/pluginsvr/arpluginsvr81_build001.jar: com.bmc.arsys.plugin

      svr.ARPluginServerMain -x SERVER2 -i /its1/ARSystem -m

       

      Here is the ar.conf extract:

       

      Server-Plugin-Alias: ARSYS.ARF.FTS ARSYS.ARF.FTS SERVER1:10000

      Server-Plugin-Alias: ARSYS.ARDBC.FTSCONFIG ARSYS.ARDBC.FTSCONFIG SERVER2:9980

      Full-Text-Configuration-Directory: /its1/ARSystem/ftsconfiguration/conf

      Full-Text-Collection-Directory: /its1/ARSystem/ftsconfiguration/collection

      Full-Text-Mode: Server-Group-secondary

      Full-Text-Disable-Searching: T

       

      FTS2.png

       

      FTS2_2.png

       

      There is absolutely no error message on any logs (FTS, arerror, armonitor,...).

       

      We already did (KB article https://kb.bmc.com/infocenter/index?page=content&id=KA392482&actp=sear):

       

      >> Remove the FTS ranking for other servers in server group. (Keep them unranked Except the primery FTS server )

      >> Stop the AR Server service.on which the FTS indexer is running.

      >> Comment the FTS process, from the armonitor.conf file.

      >> Start the AR server service,

      >> Remove the Records from the FT_pending form + empty collection directory.

      >> make sure all the other Servers in Server group are unranked for FTS.

      >> Stop other servers in server group.

      >> Make sure the Primary server only is FTS ranked as one.

      >> Stop the Primary AR server,

      >> uncommnent the FTS plugin from armonitor.cfg file.

      >> Start the AR server service for primary + start reindex fully completed with success

      >> Start the AR server service for rest of the servers one by one.

       

       

      Have someone an idea? Don't hesitate if any question!

       

      Thank you!

       

      Sam

        • 1. Re: ServerGroup installation: FTS does not work anymore
          Andreas Mitterdorfer

          # FTS Secondary

          /usr/java6_64/jre/bin/java -Xmx1024m -classpath /its1/ARSystem/pluginsvr/fts_itsprd2/secondary:/its1/ARSystem/

          is somehow odd. Imho it should read

          /usr/java6_64/jre/bin/java -Xmx1024m -classpath /its1/ARSystem/pluginsvr/fts/secondary:/its1/ARSystem/..

           

          Check the armonitor.log file whether the secondary was able to start or not.

          • 2. Re: ServerGroup installation: FTS does not work anymore

            Hi Andreas,

             

            It should be a problem on copy/paste. I confirm that all is on the same line. Both of the process well run on SERVER1. On SERVER2, no FTS process are running=> as commented on armonitor.conf.

             

            Here is the armonitor.conf extract (note that process lines are commented, then, no process FTS are running).


            SERVER2:

            # FTS Primary

            #/usr/java6_64/jre/bin/java -Xmx3072m -classpath /its1/ARSystem/pluginsvr/fts/primary:/its1/ARSystem/plugins/fts/core:/its1/ARSystem/pluginsvr:/its1/ARSystem/pluginsvr/arpluginsvr81_build001.jar:com.bmc.arsys.pluginsvr.ARPluginServerMain -x SERVER2 -i /its1/ARSystem -m

            # FTS Secondary

            #/usr/java6_64/jre/bin/java -Xmx3072m -classpath /its1/ARSystem/pluginsvr/fts/secondary:/its1/ARSystem/plugins

            vr/fts/core:/its1/ARSystem/pluginsvr:/its1/ARSystem/pluginsvr/arpluginsvr81_build001.jar: com.bmc.arsys.plugin

            svr.ARPluginServerMain -x SERVER2 -i /its1/ARSystem -m

            • 3. Re: ServerGroup installation: FTS does not work anymore
              Andreas Mitterdorfer

              If both java processes run on server1 be able to do a fts search there.

              You stated that the index was created correctly on server1 so that shouldn't be the problem either.

               

              Enable api/sql log on server1, access server1 via web/user tool  issue an fts search and check the logs. There should be an indication why it is not working.

              • 4. Re: ServerGroup installation: FTS does not work anymore
                Carl Wilson

                Hi,

                to set up FTS for a Server Group, you need to follow the steps below:

                 

                1.  Open the FTS Configuration forms on both servers and set the parameters e.g. "FTS Enable", "FTS Server Group Primary/Secondary", Memory, etc.  This will set up the FTS plugin's. Note:  You should not be making manual changes to the files, they are controlled through the FTS Configuration forms.

                2.  Open the Server Group Rankings form and set the FTS Indexer to the Primary server configured for FTS above.  Set the secondary as failover.

                3.  Restart both ARS Servers to allow the servers to pick up the FTS Configuration (plugin) changes [necessary as the armonitor file is updated through the configuration]

                4.  Navigate to the AR System Administration console.  Update the settings to enable / disable FTS on that server as required.

                5.  Restart both ARS Servers after the enable / disable

                6.  Optional:  Remove all the records in the ft_pending and delete current indexing files.

                7.  Re-index the servers participating in the FTS Indexing.

                 

                Dependent on your ARS version i.e. before 8.1 SP1 or after, the FTS will run as a Primary and Secondary setup (post SP1) where there are 2 or more servers indexing simultaneously, or as a Primary with the Secondary disabled (pre SP1) where you will see the FTS commented in the armonitor file.

                 

                Cheers

                Carl

                 

                http://www.missingpiecessoftware.com/

                1 of 1 people found this helpful
                • 5. Re: ServerGroup installation: FTS does not work anymore
                  young so

                  Samuel,

                   

                  After following Carl's steps, if you still have issue.  Please try turning off the FTS and doing a re-index option.  After that you turn it back on.  I remember having similar issue and that's what support ask me to do.

                  1 of 1 people found this helpful
                  • 6. Re: ServerGroup installation: FTS does not work anymore

                    Hi guys!

                     

                    I will review your recommandations today or during the week-end and keep you aware!

                     

                    Thanks for your advice!

                     

                    Sam

                    • 7. Re: ServerGroup installation: FTS does not work anymore
                      young so

                      Samuel,

                       

                      Looking forward to your feedback.  Thanks for posting.

                      • 8. Re: ServerGroup installation: FTS does not work anymore

                        Hi,

                         

                        Just for information, an extract from SQL/API, catching an FTS research (sorry, i don't know why communities doesn't like copy/paste):

                         

                         

                        <API > <TID: 0000001820> <RPC ID:
                        0002814916> <Queue: Fast      >
                        <Client-RPC: 390620   > <USER:
                        MidTier Service                              >
                        <Overlay-Group: 1         > /* Fri
                        Dec 12 2014 07:43:07.3073 */+GLG    ARGet

                         

                         

                        ListGroup -- user B4909 from Mid-tier (protocol 20) at IP
                        address 10.172.236.213  // :q:0.0s

                         

                         

                        <API > <TID: 0000001820> <RPC ID:
                        0002814916> <Queue: Fast      >
                        <Client-RPC: 390620   > <USER:
                        MidTier Service                              >
                        <Overlay-Group: 1         > /* Fri
                        Dec 12 2014 07:43:07.3096 */-GLG

                         

                         

                          OK

                         

                         

                        <API > <TID: 0000012871> <RPC ID:
                        0002814917> <Queue: List      >
                        <Client-RPC: 390620   > <USER: B7522

                         

                         

                        <Overlay-Group: 1         > /* Fri
                        Dec 12 2014 07:43:07.3216 */+GLEWF  ARGet

                         

                         

                        ListEntryWithFields -- schema AR System Multi-Form Search
                        from Mid-tier (protocol 20) at IP address XXX.XX.XX.XX  // :q:0.0s

                         

                         

                        <SQL > <TID: 0000012871> <RPC ID:
                        0002814917> <Queue: List      >
                        <Client-RPC: 390620   > <USER:
                        B7522

                         

                         

                                                        >
                        <Overlay-Group: 1         > /* Fri
                        Dec 12 2014 07:43:07.3358 */FTS: Request

                         

                         

                        for multi-schema search (phase one)

                         

                         

                        <SQL > <TID: 0000012871> <RPC ID:
                        0002814917> <Queue: List      >
                        <Client-RPC: 390620   > <USER:
                        B7522

                         

                         

                                                        >
                        <Overlay-Group: 1         > /* Fri
                        Dec 12 2014 07:43:07.4295 */FTS: Found 0

                         

                         

                        entries and returned 0

                         

                         

                        <API > <TID: 0000001820> <RPC ID:
                        0002814916> <Queue: Fast      >
                        <Client-RPC: 390620   > <USER:
                        MidTier Servi

                         

                         

                        ce                              >
                        <Overlay-Group: 1         > /* Fri
                        Dec 12 2014 07:43:07.3073 */+GLG    ARGet

                         

                         

                        ListGroup -- user B4909 from Mid-tier (protocol 20) at IP
                        address 10.172.236.213  // :q:0.0s

                         

                         

                        <API > <TID: 0000001820> <RPC ID:
                        0002814916> <Queue: Fast      >
                        <Client-RPC: 390620   > <USER:
                        MidTier Servi

                         

                         

                        ce                              >
                        <Overlay-Group: 1         > /* Fri
                        Dec 12 2014 07:43:07.3096 */-GLG

                         

                         

                             OK

                         

                         

                        <API > <TID: 0000012871> <RPC ID:
                        0002814917> <Queue: List      >
                        <Client-RPC: 390620   > <USER: B7522

                         

                         

                                                        >
                        <Overlay-Group: 1         > /* Fri
                        Dec 12 2014 07:43:07.3216 */+GLEWF  ARGet

                         

                         

                        ListEntryWithFields -- schema AR System Multi-Form Search
                        from Mid-tier (protocol 20) at IP address 10.172.236

                         

                         

                        .212  // :q:0.0s

                         

                         

                        <SQL > <TID: 0000012871> <RPC ID:
                        0002814917> <Queue: List      >
                        <Client-RPC: 390620   > <USER:
                        B7522

                         

                         

                                                        >
                        <Overlay-Group: 1         > /* Fri
                        Dec 12 2014 07:43:07.3358 */FTS: Request

                         

                         

                        for multi-schema search (phase one)

                         

                         

                        <SQL > <TID: 0000012871> <RPC ID:
                        0002814917> <Queue: List      >
                        <Client-RPC: 390620   > <USER:
                        B7522

                         

                         

                                                        >
                        <Overlay-Group: 1         > /* Fri
                        Dec 12 2014 07:43:07.4295 */FTS: Found 0

                         

                         

                        entries and returned 0


                        Extarct from FTS.log:

                         

                         

                         

                        <FTI > <TID: 0000000772> <RPC ID:
                        0002814883> <Queue: Admin     >
                        <Client-RPC: 390600   > <USER:
                        Remedy Application Service                   > <Overlay-Group:
                        1         > /* Fri Dec 12 2014
                        07:43:00.3709 */FT Indexer Tr

                         

                         

                        ace Log -- ON (AR Server 8.1.00 201301251157

                         

                         

                        <FTI > <TID: 0000017240> <RPC ID:
                        0000000000> <Queue: FTDispatch> <Client-RPC: 000000   > <USER:

                         

                         

                                                        >
                        <Overlay-Group: 0         > /* Fri
                        Dec 12 2014 07:43:12.4825 */Staged 1 pending entry for indexing (limit 16)

                         

                         

                        <FTI > <TID: 0000016983> <RPC ID:
                        0000000000> <Queue: FTIndexer > <Client-RPC: 390602   > <USER: Full Text Indexer                            >
                        <Overlay-Group: 1         > /* Fri
                        Dec 12 2014 07:43:12.4827 */Processing en

                         

                         

                        try started (seq number 31222 from server itsprd1) for
                        schema 1654, field 301389614, entry INC000000250939

                         

                         

                        <FTI > <TID: 0000016983> <RPC ID:
                        0000000000> <Queue: FTIndexer > <Client-RPC: 390602   > <USER: Full Text Indexer                            >
                        <Overlay-Group: 1         > /* Fri
                        Dec 12 2014 07:43:12.4912 */Indexing data

                         

                         

                        for
                        schema 1654, table field 301389614, entry INC000000250939 [WLG000000227644 X

                        ......

                         

                        So it really seems that FTS works fine.

                         

                        Regards,


                        Sam

                        • 9. Re: ServerGroup installation: FTS does not work anymore

                          Hi,

                           

                          We proceed some tests:

                           

                          1 another recommandation from Jan Sierens:

                          1 only server1 defined as single-server, diabled agent on server2 and disabled FTS on server information + remove collection + reindex: not works

                           

                          CARL WILSON recommandations:

                          2 servers defined as 1 primary and 1 secondary, all processes activated, both agent enabled, FTS activated on server 1 and 2 + remove collection + reindex: not works

                           

                          I can finally see in arftsplugin.log an error:

                           

                          2014-12-14 07:55:00,769 ERROR [pool-2-thread-1] com.bmc.arsys.pluginsvr.plugins.ARFilterAPIPluginRepository (?:?) - ARFilterApiCall FAILs in plugin: ARSYS.ARF.FTS

                          1. java.lang.NullPointerException

                                 at org.apache.lucene.analysis.CharTokenizer.incrementToken(CharTokenizer.java:77)

                                 at org.apache.lucene.analysis.LowerCaseFilter.incrementToken(LowerCaseFilter.java:38)

                                 at org.apache.lucene.index.DocInverterPerField.processFields(DocInverterPerField.java:138)

                                 at org.apache.lucene.index.DocFieldProcessorPerThread.processDocument(DocFieldProcessorPerThread.java:244)

                                 at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:831)

                          at org.apache.lucene.index.DocumentsWriter.addDocument(DocumentsWriter.java:807)

                          at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:2477)

                                 at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:2451)

                                 at com.bmc.arsys.fts.impl.lucene.LuceneFTSService.indexEntryFieldsAll(Unknown Source)

                                 at com.bmc.arsys.fts.impl.lucene.LuceneFTSService.indexEntryFields(Unknown Source)

                                 at com.bmc.arsys.plugins.ftsplugin.FTSPlugin.createUpdateOperation(Unknown Source)

                                 at com.bmc.arsys.plugins.ftsplugin.FTSPlugin.filterAPICall(Unknown Source)

                                 at com.bmc.arsys.pluginsvr.plugins.ARFilterAPIPluginRepository.ARFilterApiCall(Unknown Source)

                                 at com.bmc.arsys.pluginsvr.a.ArEsFilterApi_5(Unknown Source)

                                 at com.bmc.arsys.pluginsvr.a.ArEsFilterApi_4(Unknown Source)

                                 at com.bmc.arsys.arrpc.ARPluginServerDispatcher.dispatchOncRpcCall(Unknown Source)

                                 at com.bmc.arsys.arrpc.nio.ArRpcCallHandler.dispatchCall(Unknown Source)

                                 at com.bmc.arsys.arrpc.nio.ArRpcCallHandler.if(Unknown Source)

                                 at com.bmc.arsys.arrpc.nio.ArRpcCallHandler.processRpcCall(Unknown Source)

                                 at com.bmc.arsys.arrpc.nio.ArRpcCallHandler$Processor.run(Unknown Source)

                                 at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)

                                 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)

                                 at java.lang.Thread.run(Thread.java:761)


                          It seems that config is perhaps out of cause..

                           

                          But no idea if it is related to our problem. This is a critical production incident and BMC support is so slow...There is more result here with 2 posts here than 3 weeks with bmc support..

                           

                          Thanks for your advices!

                           

                          Sam

                          • 10. Re: ServerGroup installation: FTS does not work anymore
                            Carl Wilson

                            Hi,

                            the Java Null Pointer error is of concern, this means the plugin is not starting due to not being able to find a dependency file.  This could be due to server pathing or a missing/incomplete configuration in the FTS plugin server configuration (XML), incorrect reference, etc, so you need work out what it is not liking.  Also check permissions on the directories to ensure you can write to these to allow updates.

                            You can attempt to start the plugin manually using the command line in the armonitor file and observe the errors and logged errors.

                            You can also change the logging level to "error" from "warn" in the associated log4j file to get detailed information (if you have not already set this).

                             

                            You should have something similar to the following in your "pluginsvr_config.xml" files in the main directory as well as the Primary and Secondary directories on the server [C:\Program Files\BMC Software\ARSystem\pluginsvr\fts, C:\Program Files\BMC Software\ARSystem\pluginsvr\fts\primary, C:\Program Files\BMC Software\ARSystem\pluginsvr\fts\secondary]:

                             

                            --><pluginsvr_config>

                               <port>9998</port>

                               <regPortMapper>false</regPortMapper>

                               <encryptionPolicy>2</encryptionPolicy>

                               <publicKeyAlg>4</publicKeyAlg>

                               <publicKeyExpiry>86400</publicKeyExpiry>

                               <dataEncryptionAlg>1</dataEncryptionAlg>

                               <dataKeyExpiry>2700</dataKeyExpiry>

                               <numCoreThreads>5</numCoreThreads>

                               <numSelectorThreads>2</numSelectorThreads>

                               <workQueueMonitorLogInterval>0</workQueueMonitorLogInterval>

                               <workQueueTaskThreshold>5</workQueueTaskThreshold>

                             

                             

                              <plugins>

                               <plugin>

                                   <name>ARSYS.ARF.FTS</name>

                                   <pathelement type="location">C:/Program Files/BMC Software/ARSystem/pluginsvr/fts/ftsplugin80_build001.jar</pathelement>

                                   <pathelement type="location">C:/Program Files/BMC Software/ARSystem/pluginsvr/fts/tika-app-0.6.jar</pathelement>

                                   <classname>com.bmc.arsys.plugins.ftsplugin.FTSPlugin</classname>

                                   <userDefined>

                                     <ftCollectionDir>C:\Program Files\BMC Software\ARSystem\ftsconfiguration\collection</ftCollectionDir>

                                     <ftConfigDir>C:\Program Files\BMC Software\ARSystem\ftsconfiguration\conf</ftConfigDir>

                                     <ftCaseSensitivity>false</ftCaseSensitivity>

                                     <ftStopFile>C:\Program Files\BMC Software\ARSystem\ftsconfiguration\conf\arsfts.stp</ftStopFile>

                                     <ftLangCode>en</ftLangCode>

                                     <ftSearchThreshold>10000</ftSearchThreshold>

                                      <ftOptimizeThreshold>1000</ftOptimizeThreshold>

                                     </userDefined>

                                 </plugin>

                               </plugins>

                            </pluginsvr_config>

                             

                             

                            Cheers

                            Carl

                             

                            http://www.missingpiecessoftware.com/

                            • 11. Re: ServerGroup installation: FTS does not work anymore

                              Hi Carl,

                               

                              Thank for your answer!

                               

                              I checked the xml files on SERVER1.

                              On ARSystem\pluginsvr\fts\, the pluginsvr_config.xml file is like that:

                               

                              --><pluginsvr_config>

                                        <port>9998</port>

                                        <regPortMapper>false</regPortMapper>

                                        <encryptionPolicy>2</encryptionPolicy>

                                        <publicKeyAlg>4</publicKeyAlg>

                                        <publicKeyExpiry>86400</publicKeyExpiry>

                                        <dataEncryptionAlg>1</dataEncryptionAlg>

                                        <dataKeyExpiry>2700</dataKeyExpiry>

                                        <numCoreThreads>5</numCoreThreads>

                                        <numSelectorThreads>2</numSelectorThreads>

                                        <workQueueMonitorLogInterval>0</workQueueMonitorLogInterval>

                                        <workQueueTaskThreshold>5</workQueueTaskThreshold>

                               

                                      <plugins>

                                                <plugin>

                                                            <name>ARSYS.ARF.FTS</name>

                                                            <pathelement type="location">/its1/ARSystem/pluginsvr/fts/core/ftsplugin81_build

                              001.jar</pathelement>

                                                            <pathelement type="location">/its1/ARSystem/pluginsvr/fts/core/tika-app-0.6.jar<

                              /pathelement>

                                                            <classname>com.bmc.arsys.plugins.ftsplugin.FTSPlugin</classname>

                                                            <userDefined>

                                                                      <ftCollectionDir>/its1/ARSystem/ftsconfiguration/collection</ftCollect

                              ionDir>

                                                                      <ftConfigDir>/its1/ARSystem/ftsconfiguration/conf</ftConfigDir>

                                                                      <ftCaseSensitivity>false</ftCaseSensitivity>

                                                                      <ftStopFile>/its1/ARSystem/ftsconfiguration/conf/arsfts.stp</ftStopFil

                              e>

                                                                      <ftLangCode>en</ftLangCode>

                                                                      <ftSearchThreshold>10000</ftSearchThreshold>

                                                               <ftOptimizeThreshold>1000</ftOptimizeThreshold>

                                       </userDefined>

                                                  </plugin>

                               

                              On ARSystem\pluginsvr\fts\primary, the pluginsvr_config.xml file is like that:

                               

                               

                              --><pluginsvr_config>

                                      <port>9998</port>

                                      <regPortMapper>false</regPortMapper>

                                      <encryptionPolicy>2</encryptionPolicy>

                                      <publicKeyAlg>4</publicKeyAlg>

                                      <publicKeyExpiry>86400</publicKeyExpiry>

                                      <dataEncryptionAlg>1</dataEncryptionAlg>

                                      <dataKeyExpiry>2700</dataKeyExpiry>

                                      <numCoreThreads>5</numCoreThreads>

                                      <numSelectorThreads>2</numSelectorThreads>

                                      <workQueueMonitorLogInterval>0</workQueueMonitorLogInterval>

                                      <workQueueTaskThreshold>5</workQueueTaskThreshold>

                               

                                      <plugins>

                                              <plugin>

                                                      <name>ARSYS.ARF.FTS</name>

                                                      <pathelement type="location">/its1/ARSystem/pluginsvr/fts/core/ftsplugin81_build001.ja

                              r</pathelement>

                                                      <pathelement type="location">/its1/ARSystem/pluginsvr/fts/core/tika-app-0.6.jar</pathe

                              lement>

                                                      <classname>com.bmc.arsys.plugins.ftsplugin.FTSPlugin</classname>

                                                      <userDefined>

                                                              <ftCollectionDir>/its1/ARSystem/ftsconfiguration/collection</ftCollectionDir>

                                                              <ftConfigDir>/its1/ARSystem/ftsconfiguration/conf</ftConfigDir>

                                                              <ftCaseSensitivity>false</ftCaseSensitivity>

                                                              <ftStopFile>/its1/ARSystem/ftsconfiguration/conf/arsfts.stp</ftStopFile>

                                                              <ftLangCode>en</ftLangCode>

                                                              <ftSearchThreshold>10000</ftSearchThreshold>

                                                      <ftOptimizeThreshold>1000</ftOptimizeThreshold>

                              </userDefined>

                                              </plugin>

                               

                              On ARSystem\pluginsvr\fts\secondary, the pluginsvr_config.xml file is like that:

                               

                              --><pluginsvr_config>

                                      <port>9977</port>

                                      <regPortMapper>false</regPortMapper>

                                      <encryptionPolicy>2</encryptionPolicy>

                                      <publicKeyAlg>4</publicKeyAlg>

                                      <publicKeyExpiry>86400</publicKeyExpiry>

                                      <dataEncryptionAlg>1</dataEncryptionAlg>

                                      <dataKeyExpiry>2700</dataKeyExpiry>

                                      <numCoreThreads>5</numCoreThreads>

                                      <numSelectorThreads>2</numSelectorThreads>

                                      <workQueueMonitorLogInterval>0</workQueueMonitorLogInterval>

                                      <workQueueTaskThreshold>5</workQueueTaskThreshold>

                               

                                      <plugins>

                                              <plugin>

                                                      <name>ARSYS.ARF.FTS</name>

                                                      <pathelement type="location">/its1/ARSystem/pluginsvr/fts/core/ftsplugin81_build001.ja

                              r</pathelement>

                                                      <pathelement type="location">/its1/ARSystem/pluginsvr/fts/core/tika-app-0.6.jar</pathe

                              lement>

                                                      <classname>com.bmc.arsys.plugins.ftsplugin.FTSPlugin</classname>

                                                      <userDefined>

                                                              <ftCollectionDir>/its1/ARSystem/ftsconfiguration/collection</ftCollectionDir>

                                                              <ftConfigDir>/its1/ARSystem/ftsconfiguration/conf</ftConfigDir>

                                                              <ftCaseSensitivity>false</ftCaseSensitivity>

                                                              <ftStopFile>/its1/ARSystem/ftsconfiguration/conf/arsfts.stp</ftStopFile>

                                                              <ftLangCode>en</ftLangCode>

                                                              <ftSearchThreshold>10000</ftSearchThreshold>

                                                      <ftOptimizeThreshold>1000</ftOptimizeThreshold>

                              </userDefined>

                                              </plugin>

                               

                              I checked the xml files on SERVER2:

                               

                              On ARSystem\pluginsvr\fts\,there is no file, probably possible

                               

                              On ARSystem\pluginsvr\fts\primary, the pluginsvr_config.xml file is like that:

                               

                              <pluginsvr_config>

                                      <port>9998</port>

                                      <regPortMapper>false</regPortMapper>

                                      <encryptionPolicy>2</encryptionPolicy>

                                      <publicKeyAlg>4</publicKeyAlg>

                                      <publicKeyExpiry>86400</publicKeyExpiry>

                                      <dataEncryptionAlg>1</dataEncryptionAlg>

                                      <dataKeyExpiry>2700</dataKeyExpiry>

                                      <numCoreThreads>5</numCoreThreads>

                                      <numSelectorThreads>2</numSelectorThreads>

                                      <workQueueMonitorLogInterval>0</workQueueMonitorLogInterval>

                                      <workQueueTaskThreshold>5</workQueueTaskThreshold>

                               

                                      <plugins>

                                              <plugin>

                                                      <name>ARSYS.ARF.FTS</name>

                                                      <pathelement type="location">/its1/ARSystem/pluginsvr/fts/core/ftsplugin81_build001.ja

                              r</pathelement>

                                                      <pathelement type="location">/its1/ARSystem/pluginsvr/fts/core/tika-app-0.6.jar</pathe

                              lement>

                                                      <classname>com.bmc.arsys.plugins.ftsplugin.FTSPlugin</classname>

                                                      <userDefined>

                                                              <ftCollectionDir>/its1/ARSystem/ftsconfiguration/collection</ftCollectionDir>

                                                              <ftConfigDir>/its1/ARSystem/ftsconfiguration/conf</ftConfigDir>

                                                              <ftCaseSensitivity>false</ftCaseSensitivity>

                                                              <ftStopFile>/its1/ARSystem/ftsconfiguration/conf/arsfts.stp</ftStopFile>

                                                              <ftLangCode>en</ftLangCode>

                                                              <ftSearchThreshold>1000000</ftSearchThreshold>

                                                      </userDefined>

                                              </plugin>

                               

                              On ARSystem\pluginsvr\fts\secondary, the pluginsvr_config.xml file is like that:

                               

                              <pluginsvr_config>

                                      <port>9977</port>

                                      <regPortMapper>false</regPortMapper>

                                      <encryptionPolicy>2</encryptionPolicy>

                                      <publicKeyAlg>4</publicKeyAlg>

                                      <publicKeyExpiry>86400</publicKeyExpiry>

                                      <dataEncryptionAlg>1</dataEncryptionAlg>

                                      <dataKeyExpiry>2700</dataKeyExpiry>

                                      <numCoreThreads>5</numCoreThreads>

                                      <numSelectorThreads>2</numSelectorThreads>

                                      <workQueueMonitorLogInterval>0</workQueueMonitorLogInterval>

                                      <workQueueTaskThreshold>5</workQueueTaskThreshold>

                               

                                      <plugins>

                                              <plugin>

                                                      <name>ARSYS.ARF.FTS</name>

                                                      <pathelement type="location">/its1/ARSystem/pluginsvr/fts/core/ftsplugin81_build001.ja

                              r</pathelement>

                                                      <pathelement type="location">/its1/ARSystem/pluginsvr/fts/core/tika-app-0.6.jar</pathe

                              lement>

                                                      <classname>com.bmc.arsys.plugins.ftsplugin.FTSPlugin</classname>

                                                      <userDefined>

                                                              <ftCollectionDir>/its1/ARSystem/ftsconfiguration/collection</ftCollectionDir>

                                                              <ftConfigDir>/its1/ARSystem/ftsconfiguration/conf</ftConfigDir>

                                                              <ftCaseSensitivity>false</ftCaseSensitivity>

                                                              <ftStopFile>/its1/ARSystem/ftsconfiguration/conf/arsfts.stp</ftStopFile>

                                                              <ftLangCode>en</ftLangCode>

                                                              <ftSearchThreshold>1000000</ftSearchThreshold>

                                                      </userDefined>

                                              </plugin>

                               

                               

                              I remider the actual plugin definition on server1

                               

                              Server-Plugin-Alias: ARSYS.ARF.FTS ARSYS.ARF.FTS SERVER1:9998

                              Server-Plugin-Alias: ARSYS.ARDBC.FTSCONFIG ARSYS.ARDBC.FTSCONFIG SERVER1:9980

                               

                              I remider the actual plugin definition on server2

                               

                              Server-Plugin-Alias: ARSYS.ARF.FTS ARSYS.ARF.FTS SERVER1:9977

                              Server-Plugin-Alias: ARSYS.ARDBC.FTSCONFIG ARSYS.ARDBC.FTSCONFIG SERVER2:9980

                               

                              i reminder because they changed since your last recommendation and testing.

                               

                              I checked permissions and all is ok. I already changed the value from warn to debug on log4j.

                               

                              Do you see something strange?

                               

                              Thank you

                               

                              Sam

                              • 12. Re: ServerGroup installation: FTS does not work anymore
                                Carl Wilson

                                Hi,

                                you can see from the armonitor entry that the following is the pathing:

                                 

                                /usr/java6_64/jre/bin/java -Xmx2048m -classpath /its1/ARSystem/pluginsvr/fts/primary:/its1/ARSystem/pluginsvr/

                                fts/core:/its1/ARSystem/pluginsvr:/its1/ARSystem/pluginsvr/arpluginsvr81_build001.jar

                                 

                                So all the necessary files are required to be present in these directories - this is on both servers.

                                There is also this BMC this article which seems related to memory allocation although a different version of FTS:

                                 

                                https://kb.bmc.com/infocenter/index?page=content&id=KA385494&actp=search&viewlocale=en_US&searchid=1418593981840

                                 

                                From that and you above information, it appears your search threshold is rather large which would require a great deal of memory:

                                 

                                <ftSearchThreshold>1000000</ftSearchThreshold>

                                 

                                Try putting them back to the default and allocating 3GB Ram for the writer on both servers and increase if still having issues.  I have seen memory up to 6-8GB dedicated to indexing alone on some setups.

                                 

                                Also, a re-index may take many hours dependent on the volume of requests and information in the system, so allow for this.

                                 

                                Cheers

                                Carl

                                 

                                http://www.missingpiecessoftware.com/

                                 

                                • 13. Re: ServerGroup installation: FTS does not work anymore

                                  Hi,

                                   

                                  I will answer for Samuel (working together)

                                   

                                  we had change threshold and set heap size to 5Gb each process but nothing change global search is still failing

                                   

                                  BMC support are not giving any solution and we play at "try & error" for weeks with our customers that complain

                                   

                                  As someone an idea of what could be wrong ?

                                  • 14. Re: ServerGroup installation: FTS does not work anymore
                                    Carl Wilson

                                    Hi,

                                    not without seeing what your ARS configuration and FTS configuration along with the log files.

                                    As mentioned, start with the basics (power of elimination) and go back to a single FTS server with the defaults (as supplied OOB) and check log files. 

                                     

                                    This means going back to the standard configuration for FTS and plugin server configuration files, clearing out any indexing files in the collection directories, clearing out the fts tables in the DB, fresh logging, etc.

                                     

                                    Once the basic FTS is working, increasing the memory will help with indexing speeds and introducing HA can be done.

                                     

                                    Cheers

                                    Carl

                                     

                                    http://www.missingpiecessoftware.com/

                                    1 2 Previous Next