1 2 Previous Next 16 Replies Latest reply on Oct 9, 2014 7:36 PM by Bill Robinson

    Unable to Decommission Server

    Rahul Boraste

      Hi All,

       

      I am trying to Verify Server but it is giving error that  "java.lang.NullPointerException".

       

      So I tried to Decommission Server through Console but it is giving Error that  "The following server(s) failed to decommission Reason Null".

       

      Then I tried to decommission server through BLCLI  but the output was "Command execution failed.  java.lang.NullPointerException"

       

      Then I thought of running USP Job for same Server & again the output was "Error java.lang.NullPointerException."

       

      I am using BBSA Version 8.3.

       

      Please let me know what might be causing this issue.


      Regards,

      Rahul

        • 1. Re: Unable to Decommission Server

          What does App server and Console Logs (Users/<username/>App data/ Roaming/Bladelogic) says when these exceptions are thrown  ?

          • 2. Re: Unable to Decommission Server
            Rajeev Gupta

            Usually this error happens when there is no data, but the command interprets that the data is present.

             

            So, looks like the server does not exists but has some dependency..Could you check for any dependency?

            • 3. Re: Unable to Decommission Server
              Rahul Boraste

              @Swapnil Patil  I have checked AppServer logs, there is some Database issue But i am not getting exact solution . I have added appserver logs below:

               

              [19 Sep 2014 16:00:02,840] [Client-Connections-Thread-6] [WARN] [BLAdmin:BLAdmins:172.24.4.10] [Client] User session did not properly release a database connection.

              [19 Sep 2014 16:00:39,973] [Scheduled-System-Tasks-Thread-15] [INFO] [System:System:] [Memory Monitor] Total JVM (B): 274419712,Free JVM (B): 82116240,Used JVM (B): 192303472,VSize (B): 399745024,RSS (B): 329662464,Used File Descriptors: 2357

              [19 Sep 2014 16:01:39,970] [Scheduled-System-Tasks-Thread-19] [INFO] [System:System:] [Memory Monitor] Total JVM (B): 274419712,Free JVM (B): 132222080,Used JVM (B): 142197632,VSize (B): 399790080,RSS (B): 331362304,Used File Descriptors: 2366

              [19 Sep 2014 16:02:39,966] [Scheduled-System-Tasks-Thread-6] [INFO] [System:System:] [Memory Monitor] Total JVM (B): 274419712,Free JVM (B): 117295896,Used JVM (B): 157123816,VSize (B): 399790080,RSS (B): 331370496,Used File Descriptors: 2363

              [19 Sep 2014 16:03:39,962] [Scheduled-System-Tasks-Thread-6] [INFO] [System:System:] [Memory Monitor] Total JVM (B): 274419712,Free JVM (B): 102255280,Used JVM (B): 172164432,VSize (B): 399745024,RSS (B): 331350016,Used File Descriptors: 2359

              [19 Sep 2014 16:04:39,954] [Scheduled-System-Tasks-Thread-1] [INFO] [System:System:] [Memory Monitor] Total JVM (B): 274419712,Free JVM (B): 87662064,Used JVM (B): 186757648,VSize (B): 399745024,RSS (B): 331358208,Used File Descriptors: 2359

              [19 Sep 2014 16:05:39,955] [Scheduled-System-Tasks-Thread-12] [INFO] [System:System:] [Memory Monitor] Total JVM (B): 274419712,Free JVM (B): 147847912,Used JVM (B): 126571800,VSize (B): 399794176,RSS (B): 331436032,Used File Descriptors: 2365

              [19 Sep 2014 16:06:39,953] [Scheduled-System-Tasks-Thread-18] [INFO] [System:System:] [Memory Monitor] Total JVM (B): 274419712,Free JVM (B): 132533288,Used JVM (B): 141886424,VSize (B): 400056320,RSS (B): 331444224,Used File Descriptors: 2365

              [19 Sep 2014 16:07:39,948] [Scheduled-System-Tasks-Thread-17] [INFO] [System:System:] [Memory Monitor] Total JVM (B): 274419712,Free JVM (B): 117801400,Used JVM (B): 156618312,VSize (B): 400056320,RSS (B): 331452416,Used File Descriptors: 2363

              [19 Sep 2014 16:08:39,939] [Scheduled-System-Tasks-Thread-5] [INFO] [System:System:] [Memory Monitor] Total JVM (B): 274419712,Free JVM (B): 104005992,Used JVM (B): 170413720,VSize (B): 400007168,RSS (B): 331427840,Used File Descriptors: 2359

              [19 Sep 2014 16:09:20,288] [Client-Connections-Thread-9] [INFO] [BLAdmin:BLAdmins:172.24.4.10] [Client] DBKey:SComponentModelKeyImpl:5-1-2019916 marked 0 Psis as deleted

              [19 Sep 2014 16:09:21,027] [Client-Connections-Thread-9] [INFO] [BLAdmin:BLAdmins:172.24.4.10] [Client] DBKey:SComponentModelKeyImpl:11-1-2020009 marked 0 Psis as deleted

              [19 Sep 2014 16:09:21,196] [Client-Connections-Thread-9] [INFO] [BLAdmin:BLAdmins:172.24.4.10] [Client] DBKey:SComponentModelKeyImpl:24-1-2020083 marked 0 Psis as deleted

              [19 Sep 2014 16:09:21,333] [Client-Connections-Thread-9] [INFO] [BLAdmin:BLAdmins:172.24.4.10] [Client] DBKey:SComponentModelKeyImpl:27-1-2020102 marked 0 Psis as deleted

              [19 Sep 2014 16:09:21,475] [Client-Connections-Thread-9] [INFO] [BLAdmin:BLAdmins:172.24.4.10] [Client] DBKey:SComponentModelKeyImpl:28-1-2020108 marked 0 Psis as deleted

              [19 Sep 2014 16:09:21,623] [Client-Connections-Thread-9] [INFO] [BLAdmin:BLAdmins:172.24.4.10] [Client] DBKey:SComponentModelKeyImpl:30-1-2020191 marked 0 Psis as deleted

              [19 Sep 2014 16:09:21,777] [Client-Connections-Thread-9] [INFO] [BLAdmin:BLAdmins:172.24.4.10] [Client] DBKey:SComponentModelKeyImpl:37-1-2020413 marked 0 Psis as deleted

              [19 Sep 2014 16:09:21,926] [Client-Connections-Thread-9] [INFO] [BLAdmin:BLAdmins:172.24.4.10] [Client] DBKey:SComponentModelKeyImpl:38-1-2020417 marked 0 Psis as deleted

              [19 Sep 2014 16:09:22,088] [Client-Connections-Thread-9] [INFO] [BLAdmin:BLAdmins:172.24.4.10] [Client] DBKey:SComponentModelKeyImpl:39-1-2020421 marked 0 Psis as deleted

              [19 Sep 2014 16:09:22,240] [Client-Connections-Thread-9] [INFO] [BLAdmin:BLAdmins:172.24.4.10] [Client] DBKey:SComponentModelKeyImpl:41-1-2020495 marked 0 Psis as deleted

              [19 Sep 2014 16:09:22,407] [Client-Connections-Thread-9] [INFO] [BLAdmin:BLAdmins:172.24.4.10] [Client] DBKey:SComponentModelKeyImpl:47-1-2021655 marked 0 Psis as deleted

              [19 Sep 2014 16:09:22,603] [Client-Connections-Thread-9] [INFO] [BLAdmin:BLAdmins:172.24.4.10] [Client] DBKey:SComponentModelKeyImpl:48-1-2021663 marked 0 Psis as deleted

              [19 Sep 2014 16:09:22,753] [Client-Connections-Thread-9] [INFO] [BLAdmin:BLAdmins:172.24.4.10] [Client] DBKey:SComponentModelKeyImpl:49-1-2021671 marked 0 Psis as deleted

              [19 Sep 2014 16:09:22,901] [Client-Connections-Thread-9] [INFO] [BLAdmin:BLAdmins:172.24.4.10] [Client] DBKey:SComponentModelKeyImpl:50-1-2021675 marked 0 Psis as deleted

              [19 Sep 2014 16:09:23,058] [Client-Connections-Thread-9] [INFO] [BLAdmin:BLAdmins:172.24.4.10] [Client] DBKey:SComponentModelKeyImpl:51-1-2021680 marked 0 Psis as deleted

              [19 Sep 2014 16:09:23,227] [Client-Connections-Thread-9] [INFO] [BLAdmin:BLAdmins:172.24.4.10] [Client] DBKey:SComponentModelKeyImpl:52-1-2021690 marked 0 Psis as deleted

              [19 Sep 2014 16:09:23,395] [Client-Connections-Thread-9] [INFO] [BLAdmin:BLAdmins:172.24.4.10] [Client] DBKey:SComponentModelKeyImpl:56-1-2021721 marked 0 Psis as deleted

              [19 Sep 2014 16:09:23,572] [Client-Connections-Thread-9] [INFO] [BLAdmin:BLAdmins:172.24.4.10] [Client] DBKey:SComponentModelKeyImpl:61-1-2021779 marked 0 Psis as deleted

              [19 Sep 2014 16:09:23,732] [Client-Connections-Thread-9] [INFO] [BLAdmin:BLAdmins:172.24.4.10] [Client] DBKey:SComponentModelKeyImpl:62-1-2021786 marked 0 Psis as deleted

              [19 Sep 2014 16:09:23,883] [Client-Connections-Thread-9] [INFO] [BLAdmin:BLAdmins:172.24.4.10] [Client] DBKey:SComponentModelKeyImpl:63-1-2021793 marked 0 Psis as deleted

              [19 Sep 2014 16:09:24,019] [Client-Connections-Thread-9] [INFO] [BLAdmin:BLAdmins:172.24.4.10] [Client] DBKey:SComponentModelKeyImpl:64-1-2021800 marked 0 Psis as deleted

              [19 Sep 2014 16:09:24,156] [Client-Connections-Thread-9] [INFO] [BLAdmin:BLAdmins:172.24.4.10] [Client] DBKey:SComponentModelKeyImpl:65-1-2053948 marked 0 Psis as deleted

              [19 Sep 2014 16:09:24,315] [Client-Connections-Thread-9] [INFO] [BLAdmin:BLAdmins:172.24.4.10] [Client] DBKey:SComponentModelKeyImpl:66-1-2053952 marked 0 Psis as deleted

              [19 Sep 2014 16:09:24,462] [Client-Connections-Thread-9] [INFO] [BLAdmin:BLAdmins:172.24.4.10] [Client] DBKey:SComponentModelKeyImpl:75-2-2054530 marked 0 Psis as deleted

              [19 Sep 2014 16:09:24,602] [Client-Connections-Thread-9] [INFO] [BLAdmin:BLAdmins:172.24.4.10] [Client] DBKey:SComponentModelKeyImpl:77-1-2054563 marked 0 Psis as deleted

              [19 Sep 2014 16:09:24,759] [Client-Connections-Thread-9] [INFO] [BLAdmin:BLAdmins:172.24.4.10] [Client] DBKey:SComponentModelKeyImpl:80-1-2055929 marked 0 Psis as deleted

              [19 Sep 2014 16:09:24,915] [Client-Connections-Thread-9] [INFO] [BLAdmin:BLAdmins:172.24.4.10] [Client] DBKey:SComponentModelKeyImpl:83-1-2056348 marked 0 Psis as deleted

              [19 Sep 2014 16:09:25,049] [Client-Connections-Thread-9] [INFO] [BLAdmin:BLAdmins:172.24.4.10] [Client] DBKey:SComponentModelKeyImpl:84-1-2056412 marked 0 Psis as deleted

              [19 Sep 2014 16:09:25,210] [Client-Connections-Thread-9] [INFO] [BLAdmin:BLAdmins:172.24.4.10] [Client] DBKey:SComponentModelKeyImpl:85-1-2056677 marked 0 Psis as deleted

              [19 Sep 2014 16:09:25,384] [Client-Connections-Thread-9] [WARN] [BLAdmin:BLAdmins:172.24.4.10] [Client] Could not decommission server with id 6 due to error:null

              com.bladelogic.om.infra.mfw.util.BlException: java.lang.NullPointerException

                  at com.bladelogic.om.infra.app.db.TransactionalOperation.execute(TransactionalOperation.java:145)

                  at com.bladelogic.om.infra.model.typesystem.value.BlValueServiceImpl.save(BlValueServiceImpl.java:859)

                  at com.bladelogic.om.infra.model.property.PropertySetInstanceServiceImpl$13.doDBOperation(PropertySetInstanceServiceImpl.java:1435)

                  at com.bladelogic.om.infra.app.db.TransactionalOperation.execute(TransactionalOperation.java:132)

                  at com.bladelogic.om.infra.model.property.PropertySetInstanceServiceImpl.update(PropertySetInstanceServiceImpl.java:1438)

                  at com.bladelogic.om.infra.model.server.ServerImpl.update(ServerImpl.java:2641)

                  at com.bladelogic.om.infra.model.server.ServerFactory$6.doDBOperation(ServerFactory.java:1180)

                  at com.bladelogic.om.infra.app.db.TransactionalOperation.execute(TransactionalOperation.java:132)

                  at com.bladelogic.om.infra.model.server.ServerFactory.update(ServerFactory.java:1183)

                  at com.bladelogic.om.infra.model.server.ServerServiceImpl$1.doDBOperation(ServerServiceImpl.java:840)

                  at com.bladelogic.om.infra.app.db.TransactionalOperation.manageTransaction(TransactionalOperation.java:174)

                  at com.bladelogic.om.infra.app.db.TransactionalOperation.execute(TransactionalOperation.java:136)

                  at com.bladelogic.om.infra.model.server.ServerServiceImpl.decommission_internal(ServerServiceImpl.java:855)

                  at com.bladelogic.om.infra.model.server.ServerServiceImpl.decommission(ServerServiceImpl.java:751)

                  at com.bladelogic.om.infra.model.server.ServerServiceRequestHandlerImpl.decommission_45(ServerServiceRequestHandlerImpl.java:1921)

                  at com.bladelogic.om.infra.model.server.ServerServiceRequestHandlerImpl.execute(ServerServiceRequestHandlerImpl.java:483)

                  at com.bladelogic.om.infra.app.api.manage.APIMethodInvocationProvider.executeRequest(APIMethodInvocationProvider.java:54)

                  at com.bladelogic.om.infra.app.api.manage.APIMethodInvocationProvider.invoke(APIMethodInvocationProvider.java:39)

                  at com.bladelogic.om.infra.app.service.routing.RoutingServiceImpl.invokeMethodLocally(RoutingServiceImpl.java:317)

                  at com.bladelogic.om.infra.app.service.routing.RoutingServiceImpl.invoke(RoutingServiceImpl.java:244)

                  at com.bladelogic.om.infra.app.api.manage.APIRequestManager.executeRequest(APIRequestManager.java:30)

                  at com.bladelogic.om.infra.message.app.frame.APIMessageFrame.service(APIMessageFrame.java:113)

                  at com.bladelogic.om.infra.mfw.net.ClientWorkerThread.processRequest(ClientWorkerThread.java:385)

                  at com.bladelogic.om.infra.mfw.net.ClientWorkerThread.processClientRequest(ClientWorkerThread.java:282)

                  at com.bladelogic.om.infra.mfw.net.ClientWorkerThread.handleClientRequest(ClientWorkerThread.java:182)

                  at com.bladelogic.om.infra.mfw.net.ClientWorkerThread.execute(ClientWorkerThread.java:101)

                  at com.bladelogic.om.infra.mfw.net.ClientWorkerThread.execute(ClientWorkerThread.java:27)

                  at com.bladelogic.om.infra.app.service.thread.BlBlockingThread.run(BlBlockingThread.java:95)

              Caused by: java.lang.NullPointerException

                  at com.bladelogic.om.infra.model.property.LazyPropertySetInstanceImpl.save(LazyPropertySetInstanceImpl.java:722)

                  at com.bladelogic.om.infra.model.typesystem.value.BlValueServiceImpl$1.doDBOperation(BlValueServiceImpl.java:856)

                  at com.bladelogic.om.infra.app.db.TransactionalOperation.execute(TransactionalOperation.java:132)

                  ... 27 more

               

              Regards,

              Rahul

              • 4. Re: Unable to Decommission Server

                I see some messages like com.infra.app.service.routing.RoutingServiceImpl.invoke

                 

                Does this server added using SOCKS proxy ?  Check the Proxy routing rules and Server properties. If this server is added using Proxy routing rules then ensure Verify server is able out reach out server. Then Try decommissioning the server.

                • 5. Re: Unable to Decommission Server
                  Rahul Boraste

                  @Rajeev Gupta

                   

                  But during Verify also it is giving error as "java.lang.NullPointerException".

                   

                  What could be the reason for this ?

                   

                  Regards,

                  Rahul

                  • 6. Re: Unable to Decommission Server
                    Rahul Boraste

                    @

                     

                    No i have not added this server using SOCKS proxy.

                     

                    Regards,

                    Rahul

                    • 7. Re: Unable to Decommission Server

                      when you try verify or agentinfo command. What message do you get in rscd.log file on that server ?

                      • 8. Re: Unable to Decommission Server
                        Rahul Boraste

                        @

                         

                        RSCD Logs when I verify server :

                         

                        06ca532df372d2cb187d 0000001152 09/24/14 15:05:07.515 INFO1rscd -  172.24.4.10 5728 BladeLogicRSCD@M0607->Administrator@M0607:PrivilegeMapped (BLAdmins:BLAdmin): CM: > [Client] Retrieving property values
                        59e352167c2fd13abef8 0000001153 09/24/14 15:05:07.517 INFO rscd -  M0607 5728 BladeLogicRSCD (Not_available): (Not_available): The operation completed successfully. 
                        91eb9fa09941f2f86aaf 0000001154 09/24/14 15:05:07.540 INFO rscd -  M0607 5728 BladeLogicRSCD (Not_available): (Not_available): The operation completed successfully. 
                        d93d5574c0720a7ef57c 0000001155 09/24/14 15:05:07.541 INFO rscd -  M0607 5728 BladeLogicRSCD (Not_available): (Not_available): The operation completed successfully. 

                         

                         

                        And when I give AgentInfo Command it shows following result :

                         

                          Agent Release   : 8.3.00.115

                          Hostname        : M0607

                          Operating System: WindowsNT 6.0

                          User Permissions: BladeLogicRSCD@M0607->Administrator@M0607:PrivilegeMapped (I

                        dentity via trust)

                          Security        : Protocol=5, Encryption=TLS1

                          Host ID         : F6804AED

                          # of Processors : 1

                          License Status  : Licensed for NSH/CM

                         

                        RSCD logs of AgentInfo :


                        62e105503131c4b00f72 0000001156 09/24/14 15:08:39.499 INFO rscd -  172.24.4.10 4224 BladeLogicRSCD@M0607->Administrator@M0607:PrivilegeMapped (Administrator): agentinfo: agentinfo m0607
                        23373de0696f681e5b66 0000001157 09/24/14 15:08:39.501 INFO rscd -  M0607 4224 BladeLogicRSCD (Not_available): (Not_available): The operation completed successfully. 
                        • 9. Re: Unable to Decommission Server

                          I think database doesn't have consistent records for this server.. I again seen logs carefully; From log it looks like blvalue table returns no rows for this server and it throws NPE.

                           

                          Try refreshing the server group or reconnect client one more time. If you still see the server then I would recommend contact BMC Support. 

                           

                          at com.bladelogic.om.infra.app.db.TransactionalOperation.execute(TransactionalOperation.java:145)

                              at com.bladelogic.om.infra.model.typesystem.value.BlValueServiceImpl.save(BlValueServiceImpl.java:859)

                              at com.bladelogic.om.infra.model.property.PropertySetInstanceServiceImpl$13.doDBOperation(PropertySetInstanceServiceImpl.java:1435)

                              at com.bladelogic.om.infra.app.db.TransactionalOperation.execute(TransactionalOperation.java:132)

                              at

                          1 of 1 people found this helpful
                          • 10. Re: Unable to Decommission Server
                            Bill Robinson

                            can you run a couple db queries:

                             

                            select * from device where name = 'your server name';

                            select * from device where device_id = 6;

                            • 11. Re: Re: Unable to Decommission Server
                              Rahul Boraste

                              Hi Bill,

                               

                              select * from device where name = 'hostname'

                               

                              it results with two records (one with the changed IP address and old IP address)

                              Do I need to remove the record with old IP address from the table ?

                               

                              Please find attached result.

                              • 12. Re: Re: Unable to Decommission Server
                                Bill Robinson

                                in RBAC / Audit Trail you can pull in 'object type' and filter for 'job' and then 'object name' but that won't get you the group path.  i think in reports studio you could take that query and do a join w/ job activity where you can get the folder path. 

                                • 13. Re: Re: Unable to Decommission Server
                                  Bill Robinson

                                  device id 1 is just an older version of the same name, i don't think that's causing an issue.  i'd open a ticket w/ support.  there's probably some related object that has a messed up dependency.

                                  • 14. Re: Unable to Decommission Server
                                    Rahul Boraste

                                    Hi Bill,

                                     

                                    Did u got any updates on ticket.

                                     

                                     

                                     

                                    Thanks,

                                    Rahul

                                    1 2 Previous Next