10 Replies Latest reply on Jan 12, 2018 1:56 AM by Andreas Mitterdorfer

    REMEDY.ARDBC.SERVER.ADMINISTRATION : RPC: Program not registered

    Pravashini Nayak

      Hi All

       

      We are facing one issue, i.e "ARERR [8760] Cannot establish a network connection to the AR System Plug-In server : <servername>  REMEDY.ARDBC.SERVER.ADMINISTRATION : RPC: Program not registered" while clicking on Server information.

       

      Please fines the screenshot for the same.

       

      We have analysed the arerror logs and found the following error:-

       

      Mon Jan  8 15:27:03 2018  0 : AR System server terminated ▒ fatal error occurred in ARSERVER (ARNOTE 21)

      Mon Jan  8 15:27:03 2018  390620 : Cannot establish a network connection to the AR System Plug-In server (ARERR 8760)

      Mon Jan  8 15:27:03 2018     <servername>  REMEDY.ARDBC.SERVER.ADMINISTRATION : RPC: Program not registered.

       

      Remedy version 7.6.04

      OS :- SunOS 5.10

       

      Please help.

        • 1. Re: REMEDY.ARDBC.SERVER.ADMINISTRATION : RPC: Program not registered
          Sergey Smirnov

          If use portmapper:

               - сheck if you have reserved ports for 390600-390695 arserverd in the file /etc/rpc;

               - check launched whether you have a rpc service;

               - for UNIX systems in preparation for the installation is required to run the service with the key "i":

          service rpcbind stop

          rpcbind -i

          service rpcbind status

          • 2. Re: REMEDY.ARDBC.SERVER.ADMINISTRATION : RPC: Program not registered
            Pravashini Nayak

            Hi Sergey Smirnov,

             

            We didn't upgrade or install new server.

            I couldn't understand  why AR server is getting terminated. And I have checked in ar.conf we dont have any entry for "REMEDY.ARDBC.SERVER.ADMINISTRATION".

             

            Regards

            Pravashini

            • 3. Re: REMEDY.ARDBC.SERVER.ADMINISTRATION : RPC: Program not registered
              Andreas Mitterdorfer

              Is arplugin process running at all?

              If yes, try to configure a fixed plugin server port if you haven't one already, and check wheter the problem persists.

              • 4. Re: REMEDY.ARDBC.SERVER.ADMINISTRATION : RPC: Program not registered
                Pravashini Nayak

                Hi Anderas

                 

                Tried using a fixed plugin server port as 2600. The error got changed as "ARERR [8760] Cannot establish a network connection to the AR System Plug-In server : <servername> REMEDY.ARDBC.SERVER.ADMINISTRATION : RPC: Miscellaneous tli error - An event requires attentionError 0".

                 

                Regards

                Pravashini

                • 5. Re: REMEDY.ARDBC.SERVER.ADMINISTRATION : RPC: Program not registered
                  Andreas Mitterdorfer

                  Please can you attach the ar.conf and armonitor.log?

                  • 6. Re: REMEDY.ARDBC.SERVER.ADMINISTRATION : RPC: Program not registered
                    Pravashini Nayak

                    Hi Andreas

                     

                    I checked, and found the monitor.log is also not getting generated.

                     

                    Could you please help with this.

                     

                    Regards

                    Pravashini

                    • 7. Re: REMEDY.ARDBC.SERVER.ADMINISTRATION : RPC: Program not registered
                      Fred Grooms

                      I remember seeing something like this and in the Plugin log I could see something where the Server Administration plugin was not loading correctly.  The plugin was trying to connect to a server called <servername>.dummy.host.com

                      To see the error in the plugin log we had to add to the ar.conf

                           Plugin-Log-Level: 100

                       

                      I am guessing in one of the modules used in that c plugin there was logic where if it couldn't determine the server it would just add dummy.host.com to the short server name and use that.

                       

                      The quick workaround when that happened was to just kill the arplugin process and let armonitor restart it.

                      • 8. Re: REMEDY.ARDBC.SERVER.ADMINISTRATION : RPC: Program not registered
                        Pravashini Nayak

                        Hi Fred

                         

                        I have killed the arplugin process and armonitor is restaring the process but still the issue persists. while analysing the arerror we are getting one issue 

                         

                        AR System server terminated when a signal/exception was received by the server (ARNOTE 20)

                        Mon Jan  8 15:27:03 2018     11

                        Operation: ARDBCGetEntry

                        VendorName: REMEDY.ARDBC.SERVER.ADMINISTRATION

                        Library: ServerAdmin.so

                        Stacks:

                        /RemedyUatApp/bmc/ARSystem/bin/arplugin:DumpStackTrace+0x390

                        /RemedyUatApp/bmc/ARSystem/bin/arplugin:SignalTrapProc+0x108

                        /lib/libc.so.1:0xcb024

                        /lib/libc.so.1:0xbf6d4

                        /lib/libc.so.1:0xbf8bc

                        /lib/libc.so.1:fwrite+0x8 [ Signal 11 (SEGV)]

                        /RemedyUatApp/bmc/ARSystem/bin/ServerAdmin.so:0xcc028

                        /RemedyUatApp/bmc/ARSystem/bin/ServerAdmin.so:ARDBCGetEntry+0xee0

                        /RemedyUatApp/bmc/ARSystem/bin/arplugin:aresardbcget_4_svc+0x274

                        /RemedyUatApp/bmc/ARSystem/bin/arplugin:HandleRPCs+0x1ac8

                        /RemedyUatApp/bmc/ARSystem/bin/arplugin:WorkerThread+0x1d4

                        /RemedyUatApp/bmc/ARSystem/bin/arplugin:RestartableThreadMain+0x68

                        /RemedyUatApp/bmc/ARSystem/bin/arplugin:UnixThreadStartRoutine+0x128

                        /lib/libc.so.1:0xcaef8

                        Mon Jan  8 15:27:03 2018  0 : AR System server terminated ▒ fatal error occurred in ARSERVER (ARNOTE 21)

                        Mon Jan  8 15:27:03 2018  390620 : Cannot establish a network connection to the AR System Plug-In server (ARERR 8760)

                        Mon Jan  8 15:27:03 2018     <servername>  REMEDY.ARDBC.SERVER.ADMINISTRATION : RPC: Program not registered

                        Mon Jan  8 15:27:03 2018  390620 : Cannot establish a network connection to the AR System Plug-In server (ARERR 8760)

                         

                        Regards

                        Pravashini

                        • 9. Re: REMEDY.ARDBC.SERVER.ADMINISTRATION : RPC: Program not registered
                          Pravashini Nayak

                          Hi All

                           

                          Please find the ar.conf file for the affected server.

                           

                           

                          Server-Name: <servername>

                          Server-Connect-Name: <Server FQDN>

                          Server-directory: /RemedyUatApp/bmc/ARSystem/db

                          Multiple-ARSystem-Servers: T

                          Register-With-Portmapper: T

                          TCD-Specific-Port: 0

                          Plugin-Port:

                          Alternate-Approval-Reg: T

                          Currency-Ratio-Client-Refresh-Interval: 60

                          Locale-List: en

                          Required-Field-Identifier: *

                          Required-Field-Identifier-Location: 1

                          Db-name: RDATA

                          Db-user: ARAdmin

                          Oracle-Clob-Storage-In-Row: T

                          Db-Character-Set: UTF-8

                          Dbhome-directory: /u01/app/oracle/client/11.2.0.3

                          Oracle-Two-Task: <name>

                          Db-password: *********

                          Remedy-App-Service-Password: ************

                          Mid-Tier-Service-Password: **********

                          Application-Enable: T

                          RE-RPC-Socket: 390698

                          Approval-RPC-Socket: 390680

                          AE-Log: OFF

                          AE-Trace: OFF

                          Next-ID-Block-Size: 100

                          Allow-Guest-Users: F

                          Load-Shared-Library-Path: /RemedyUatApp/bmc/AtriumCore/cmdb/server/bin

                          Plugin-Loopback-RPC-Socket: 390626

                          Plugin-Path: /RemedyUatApp/bmc/ARSystem/bin

                          Plugin-Path: /RemedyUatApp/bmc/ARSystem/pluginsvr

                          Plugin-Path: /RemedyUatApp/bmc/ARSystem/arealdap

                          Plugin-Path: /RemedyUatApp/bmc/ARSystem/ardbcldap

                          Plugin-Path: /RemedyUatApp/bmc/AtriumCore/cmdb/server/bin

                          Load-Shared-Library: libcmdbsvr7604_solsp64.so

                          CMDB-Inline-Normalization: T

                          CMDB-Install-Directory: /RemedyUatApp/bmc/AtriumCore/cmdb

                          RE-Log-File-Location: /RemedyUatApp/bmc/AtriumCore/Logs

                          AINT-Enabled: T

                          Delay-Recache-Time: 180

                          VersionControl-Object-Modification-Log-Save-Definition-Files: 10

                          Plugin: ardbcconf.so

                          Plugin: reportplugin.so

                          Plugin: ServerAdmin.so

                          Plugin: FlashboardObject.so

                          Plugin: /RemedyUatApp/bmc/ARSystem/arealdap/arealdap.so

                          Plugin: /RemedyUatApp/bmc/ARSystem/ardbcldap/ardbcldap.so

                          Plugin: /RemedyUatApp/bmc/ARSystem/approval/bin/arapprove.so

                          Server-Plugin-Default-Timeout: 600

                          Assignment-Engine-Suspended: F

                          Suppress-warnings: 300

                          Server-Plugin-Alias: ARSYS.ARF.REGISTRY ARSYS.ARF.REGISTRY RGCOMAPPUAT:9999

                          Server-Plugin-Alias: ARSYS.ARDBC.REGISTRY ARSYS.ARDBC.REGISTRY RGCOMAPPUAT:9999

                          Server-Plugin-Alias: ARSYS.ARDBC.ARREPORTENGINE ARSYS.ARDBC.ARREPORTENGINE RGCOMAPPUAT:9999

                          Server-Plugin-Alias: ARSYS.ARF.QUERYPARSER ARSYS.ARF.QUERYPARSER RGCOMAPPUAT:9999

                          Server-Plugin-Alias: ARSYS.ALRT.WEBSERVICE ARSYS.ALRT.WEBSERVICE RGCOMAPPUAT:9999

                          Server-Plugin-Alias: ARSYS.ARF.WEBSERVICE ARSYS.ARF.WEBSERVICE RGCOMAPPUAT:9999

                          Server-Plugin-Alias: ARSYS.ARF.FTS ARSYS.ARF.FTS RGCOMAPPUAT:9998

                          Server-Plugin-Alias: BMC.FILTERAPI.NORM.ENGINE BMC.FILTERAPI.NORM.ENGINE RGCOMAPPUAT:9555

                          Server-Plugin-Alias: AIS.FILTERAPI AIS.FILTERAPI RGCOMAPPUAT:9556

                          Server-Plugin-Alias: BMC.ARDBC.ATRIUM.API BMC.ARDBC.ATRIUM.API RGCOMAPPUAT:9556

                          Server-Plugin-Alias: DSM.FILTER DSM.FILTER RGCOMAPPUAT:9556

                          Server-Plugin-Alias: BMC.ARDBC.DEPRECATION.PLUGIN BMC.ARDBC.DEPRECATION.PLUGIN RGCOMAPPUAT:9556

                          Server-Plugin-Alias: BMC.FILTERAPI.NOE BMC.FILTERAPI.NOE RGCOMAPPUAT:9556

                          Server-Plugin-Alias: BMC.ARDBC.ATRIUM.NGIE BMC.ARDBC.ATRIUM.NGIE RGCOMAPPUAT:9556

                          Server-Plugin-Alias: BMC.FILTERAPI.EVENT_ENGINE BMC.FILTERAPI.EVENT_ENGINE RGCOMAPPUAT:9556

                          Server-Plugin-Alias: REMEDY.ARF.CAI REMEDY.ARF.CAI RGCOMAPPUAT:9999

                          Server-Plugin-Alias: REMEDY.ARDBC.APPQUERY REMEDY.ARDBC.APPQUERY RGCOMAPPUAT:9999

                          Server-Plugin-Alias: REMEDY.ARF.ITSMUtil REMEDY.ARF.ITSMUtil RGCOMAPPUAT:9999

                          Server-Plugin-Alias: REMEDY.ARF.CBDATA REMEDY.ARF.CBDATA RGCOMAPPUAT:9999

                          Server-Plugin-Alias: NextId NextId RGCOMAPPUAT:9999

                          Server-Plugin-Alias: RMDY.ITSM.ASSET.SOFTWAREUSAGE RMDY.ITSM.ASSET.SOFTWAREUSAGE RGCOMAPPUAT:9999

                          Server-Plugin-Alias: RMDY.ITSM.ARL RMDY.ITSM.ARL RGCOMAPPUAT:9999

                          Server-Plugin-Alias: RMDY.ITSM.RLE RMDY.ITSM.RLE RGCOMAPPUAT:9999

                          Full-Text-Disable-Indexing: F

                          External-Authentication-Return-Data-Capabilities:31

                          Plugin-Path: /RemedyUatApp/bmc/ServiceLevelManagement/bin

                          Plugin: /RemedyUatApp/bmc/ServiceLevelManagement/bin/libomfobjiefilapi.so

                          Plugin: /RemedyUatApp/bmc/ServiceLevelManagement/bin/libarfslasetup.so

                          Registry-Admin-User: admin

                          Registry-Admin-Password: OWKDl5e4Maew9MRqQU20M4PzDUTIH3iwBJJAq9/yoPr6xTA7qVOASMxsGRBGJkfA4DjJq3ByyZPzIxF4pRJsXZViDE8hscrn63iGhvMn0MFKMc0AcCVvmw==

                          Registry-Location: http://GCXAPPDEV:8080/uddi

                          System-Messages-Displayed: 1

                          Filter-Api-Timeout: 300

                          Cache-Mode: 0

                          Homepage-Form: Home Page 7_6

                          Private-RPC-Socket:  390601   1   4

                          Private-RPC-Socket:  390603   7   7

                          Private-RPC-Socket:  390620   2  20

                          Private-RPC-Socket:  390621   5  16

                          Private-RPC-Socket:  390635   8  30

                          Private-RPC-Socket:  390680   2   15

                          Private-RPC-Socket:  390698   6   6

                          Private-RPC-Socket:  390626   2   2

                          Default-Web-Path: http://10.185.4.151:8081/arsys/

                          Authentication-Chaining-Mode: 0

                          Full-Text-Disable-Searching: T

                          Filter-Max-Stack: 25

                          Arfork-Log-File: /RemedyUatApp/bmc/ARSystem/db/arfork_1.log

                          VersionControl-Object-Modification-Log-Mode: 0

                          Record-Server-Events: 1;

                          Distrib-Log-File: /RemedyUatApp/bmc/ARSystem/db/DSO_200715_2.log

                          DSO-User-Password: EQTovWI0vy1iOylBdqG1zaaKXJOxty6x6+ju0atrZBB1UOOmxojWKvv9E79n4vjzJi7vwxEm0PkQVvclI4S7ir+GoGV2qwaXkGBX0nqtnIh1uME0s9wGjQ==

                          Admin-Only-Mode: F

                          Plugin-Log-Level: 100

                          User-Log-File: /RemedyUatApp/bmc/ARSystem/db/aruser_30122015.log

                          Use-FTS-In-Workflow: F

                          Arsignald-Log-File: /RemedyUatApp/bmc/ARSystem/db/UATRebootARsignald2

                          Thread-Log-File: /RemedyUatApp/bmc/ARSystem/db/UATRebootThread2

                          External-Authentication-RPC-Socket: 390695

                          Max-Entries-Per-Query: 200000

                          Server-Side-Table-Chunk-Size: 10000

                          License-Usage-Recording: 0

                          Set-Process-Timeout: 60

                          Filter-Max-Total: 999999999

                          Disable-Alerts: F

                          AE-RPC-Socket: 390627

                          Disable-Escalations: F

                          Plugin-Log-File: /RemedyUatApp/bmc/ARSystem/db/arpluginError.log

                          SQL-Log-File: /RemedyUatApp/bmc/ARSystem/db/t.log

                          Filter-Log-File: /RemedyUatApp/bmc/ARSystem/db/t.log

                          Escalation-Log-File: /RemedyUatApp/bmc/ARSystem/db/t.log

                          API-Log-File: /RemedyUatApp/bmc/ARSystem/db/t.log

                          Log-Form-Selected: 0

                           

                          Regards

                          Pravashini

                          • 10. Re: REMEDY.ARDBC.SERVER.ADMINISTRATION : RPC: Program not registered
                            Andreas Mitterdorfer

                            The plugin (ServerAdmin.so)  is defined in ar.conf and obviously it gets loaded but crashes when AR server make an api call to the plugin. Best is to open a case with BMC.

                             

                            Edit:

                            Try to set plugin log level all in ar.conf manually (Debug-mode: 131072, Plugin-Log-Level: 100) then restart the arplugin process and see wheter there are any error messages, when the plugin gets loaded.