Remedy - Server - v9.1.00 - ARSystem cannot start. Errors: "...Error creating bean with name 'localBroker' defined in OSGi resource[bundleentry:...." and/or "...Detected missing/corrupt journal files..."

Version 2
    Share:|

    This document contains official content from the BMC Software Knowledge Base. It is automatically updated when the knowledge article is modified.


    PRODUCT:

    Remedy AR System Server


    COMPONENT:

    AR System


    APPLIES TO:

    BMC Remedy AR System Server 9.1.00



    PROBLEM:

    ARSystem 9.1.00 does not start.

    The arerror log and armonitor log shows the following.

    arerror.log
    ....
    Tue Jul 05 19:49:14.138 2016 updating statistics configuration: enabled=false aggregation interval (sec)=3600
    Tue Jul 05 20:31:32.761 2016 Failed to start Apache ActiveMQ ([broker_0, null], {})
    Tue Jul 05 20:31:33.312 2016 Error encountered in bundle: com.bmc.arsys.messaging
    Tue Jul 05 20:31:33.316 2016 Application context refresh failed (OsgiBundleXmlApplicationContext(bundle=com.bmc.arsys.messaging, config=bundleentry://160.fwk1456208737/OSGI-INF/blueprint/application_context.xml))
    Tue Jul 05 20:31:33.321 2016 Unable to create application context for [com.bmc.arsys.messaging], unsatisfied dependencies: none
    Tue Jul 05 20:31:33.346 2016 Stopserver.await: accept
    Tue Jul 05 20:31:33.362 2016 TID 65 initTransport  Normal ARServer shutdown process Initiated.
    ....


    armonitor.log.
    ....
    > /* Tue Jul 05 16 20:31:13.0759  */ ProcessMonitor starting process [ ARSERVER ].
    > /* Tue Jul 05 16 20:31:13.0760  */ Process commands List - [/usr/java/jdk1.8.0_60/jre/bin/java, -jar, /opt/bmc/ARSystem/bin/arserver.jar, -s, {AR Server Name}, -i, /opt/bmc/ARSystem, -l, /etc/arsystem/{AR Server Name}]
    > /* Tue Jul 05 16 20:31:18.0772  */ Server Statup Notification Port [ARSERVERPORT] :39678
       > /* Tue Jul 05 16 20:31:18.0774  */ Server port socket listener initializing..
    > /* Tue Jul 05 16 20:31:23.0962  */ Tue Jul 05 20:31:23.957 2016 RestApiActivator:starting bundle...
    > /* Tue Jul 05 16 20:31:23.0990  */ Tue Jul 05 20:31:23.987 2016 Restapi bundle started.
    > /* Tue Jul 05 16 20:31:25.0547  */ Registering CMDB REST API @ /cmdb ...
    > /* Tue Jul 05 16 20:31:25.0559  */ Registered CMDB REST API
    > /* Tue Jul 05 16 20:31:32.0781  */ Tue Jul 05 20:31:32.761 2016 Failed to start Apache ActiveMQ ([broker_0, null], {})
    > /* Tue Jul 05 16 20:31:32.0781  */ java.io.IOException: Detected missing/corrupt journal files. 3081 messages affected.
    > /* Tue Jul 05 16 20:31:32.0782  */     at org.apache.activemq.store.kahadb.MessageDatabase.recoverIndex(MessageDatabase.java:854) ~[na:na]
    > /* Tue Jul 05 16 20:31:32.0782  */     at org.apache.activemq.store.kahadb.MessageDatabase$5.execute(MessageDatabase.java:625) ~[na:na]
    > /* Tue Jul 05 16 20:31:32.0782  */     at org.apache.activemq.store.kahadb.disk.page.Transaction.execute(Transaction.java:779) ~[na:na]
    > /* Tue Jul 05 16 20:31:32.0782  */     at org.apache.activemq.store.kahadb.MessageDatabase.recover(MessageDatabase.java:622) ~[na:na]
    > /* Tue Jul 05 16 20:31:32.0782  */     at org.apache.activemq.store.kahadb.MessageDatabase.open(MessageDatabase.java:400) ~[na:na]
    > /* Tue Jul 05 16 20:31:32.0783  */     at org.apache.activemq.store.kahadb.MessageDatabase.load(MessageDatabase.java:418) ~[na:na]
    > /* Tue Jul 05 16 20:31:32.0783  */     at org.apache.activemq.store.kahadb.MessageDatabase.doStart(MessageDatabase.java:262) ~[na:na]
    > /* Tue Jul 05 16 20:31:32.0783  */     at org.apache.activemq.store.kahadb.KahaDBStore.doStart(KahaDBStore.java:194) ~[na:na]
    > /* Tue Jul 05 16 20:31:32.0783  */     at org.apache.activemq.util.ServiceSupport.start(ServiceSupport.java:55) ~[bundlefile:5.10.0]
    > /* Tue Jul 05 16 20:31:32.0783  */     at org.apache.activemq.store.kahadb.KahaDBPersistenceAdapter.doStart(KahaDBPersistenceAdapter.java:215) ~[na:na]
    > /* Tue Jul 05 16 20:31:32.0783  */     at org.apache.activemq.util.ServiceSupport.start(ServiceSupport.java:55) ~[bundlefile:5.10.0]
    > /* Tue Jul 05 16 20:31:32.0784  */     at org.apache.activemq.broker.BrokerService.doStartPersistenceAdapter(BrokerService.java:640) ~[bundlefile:5.10.0]
    > /* Tue Jul 05 16 20:31:32.0784  */     at org.apache.activemq.broker.BrokerService.startPersistenceAdapter(BrokerService.java:629) ~[bundlefile:5.10.0]
    > /* Tue Jul 05 16 20:31:32.0784  */     at org.apache.activemq.broker.BrokerService.start(BrokerService.java:594) ~[bundlefile:5.10.0]
    > /* Tue Jul 05 16 20:31:32.0784  */     at com.bmc.arsys.messaging.data.MessageBroker.init(MessageBroker.java:106) [bundlefile:9.1.00-SNAPSHOT]
    > /* Tue Jul 05 16 20:31:32.0784  */     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_60]
    > /* Tue Jul 05 16 20:31:32.0784  */     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_60]
    > /* Tue Jul 05 16 20:31:32.0785  */     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_60]
    > /* Tue Jul 05 16 20:31:32.0785  */     at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_60]
    ....etc.....
    > /* Tue Jul 05 16 20:31:33.0318  */ Tue Jul 05 20:31:33.316 2016 Application context refresh failed (OsgiBundleXmlApplicationContext(bundle=com.bmc.arsys.messaging, config=bundleentry://160.fwk1456208737/OSGI-INF/blueprint/application_context.xml))
    > /* Tue Jul 05 16 20:31:33.0319  */ org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'localBroker' defined in OSGi resource[bundleentry://160.fwk1456208737/OSGI-INF/blueprint/application_context.xml|bnd.id=160|bnd.sym=com.bmc.arsys.messaging]: Invocation of init method failed; nested exception is java.io.IOException: Detected missing/corrupt journal files. 3081 messages affected.
    > /* Tue Jul 05 16 20:31:33.0319  */     at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1568) ~[bundlefile:na]
    > /* Tue Jul 05 16 20:31:33.0319  */     at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:540) ~[bundlefile:na]
    > /* Tue Jul 05 16 20:31:33.0319  */     at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:476) ~[bundlefile:na]
    ...etc...
    > /* Tue Jul 05 16 20:31:33.0322  */     at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60]
    > /* Tue Jul 05 16 20:31:33.0322  */ Caused by: java.io.IOException: Detected missing/corrupt journal files. 3081 messages affected.
    > /* Tue Jul 05 16 20:31:33.0328  */     ... 14 common frames omitted
    > /* Tue Jul 05 16 20:31:33.0328  */ Tue Jul 05 20:31:33.321 2016 Unable to create application context for [com.bmc.arsys.messaging], unsatisfied dependencies: none
    > /* Tue Jul 05 16 20:31:33.0328  */ org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'localBroker' defined in OSGi resource[bundleentry://160.fwk1456208737/OSGI-INF/blueprint/application_context.xml|bnd.id=160|bnd.sym=com.bmc.arsys.messaging]: Invocation of init method failed; nested exception is java.io.IOException: Detected missing/corrupt journal files. 3081 messages affected.
    > /* Tue Jul 05 16 20:31:33.0329  */     at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1568) ~[na:na]
    > /* Tue Jul 05 16 20:31:33.0329  */     at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:540) ~[na:na]
    > /* Tue Jul 05 16 20:31:33.0329  */     at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:476) ~[na:na]
    ...etc...
    > /* Tue Jul 05 16 20:31:33.0340  */     at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60]
    > /* Tue Jul 05 16 20:31:33.0341  */ Caused by: java.io.IOException: Detected missing/corrupt journal files. 3081 messages affected.
    > /* Tue Jul 05 16 20:31:33.0346  */     ... 14 common frames omitted
    > /* Tue Jul 05 16 20:31:33.0349  */ Tue Jul 05 20:31:33.346 2016 Stopserver.await: accept
    > /* Tue Jul 05 16 20:31:33.0349  */ java.net.SocketException: Socket closed
    > /* Tue Jul 05 16 20:31:33.0349  */     at java.net.PlainSocketImpl.socketAccept(Native Method) ~[na:1.8.0_60]
    > /* Tue Jul 05 16 20:31:33.0350  */     at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:409) ~[na:1.8.0_60]
    > /* Tue Jul 05 16 20:31:33.0350  */     at java.net.ServerSocket.implAccept(ServerSocket.java:545) ~[na:1.8.0_60]
    > /* Tue Jul 05 16 20:31:33.0350  */     at java.net.ServerSocket.accept(ServerSocket.java:513) ~[na:1.8.0_60]
    > /* Tue Jul 05 16 20:31:33.0350  */     at com.bmc.arsys.shutdown.ShutdownListener.run(ShutdownListener.java:103) ~[bundlefile:9.1.00-SNAPSHOT]
    > /* Tue Jul 05 16 20:31:33.0350  */     at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60]
    > /* Tue Jul 05 16 20:31:33.0363  */ Tue Jul 05 20:31:33.362 2016 TID 65 initTransport  Normal ARServer shutdown process Initiated.
    > /* Tue Jul 05 16 20:31:33.0383  */ Unregistering CMDB REST API...
    > /* Tue Jul 05 16 20:31:33.0383  */ Canceling scheduled tasks of CMDB..
    > /* Tue Jul 05 16 20:31:33.0384  */ Cancelled scheduled tasks of CMDB.
    > /* Tue Jul 05 16 20:31:33.0384  */ Unregistered CMDB REST API.
    > /* Tue Jul 05 16 20:31:34.0037  */ Process Stopped with exit value: 1
    > /* Tue Jul 05 16 20:31:34.0038  */ Tue Jul 05 20:31:14.787 2016 Launching Action Request System(R) Server Version 9.1.00 201512160229
    > /* Tue Jul 05 16 20:31:34.0038  */ Tue Jul 05 20:31:24.930 2016 Removal of previously generated PID file successful
    > /* Tue Jul 05 16 20:31:34.0038  */
    > /* Tue Jul 05 16 20:32:18.0835  */ Process [ ARSERVER ] stopped with exit value - [ 0 ].

    .......

     


    SOLUTION:

    Please perform the following:

    1. Take a backup of all the files in
    Windows - {installation directory}
    \ARSystem\activemq-data\broker_0\KahaDB
    UNIX - {installation directory}/ARSystem/bin/activemq-data/broker_0/KahaDB
    2. Delete all files from this directory.
    3. Restart ARSystem.


    Article Number:

    000116900


    Article Type:

    Solutions to a Product Problem



      Looking for additional information?    Search BMC Support  or  Browse Knowledge Articles