Skip navigation

Discovery UI should provide useful message when crontab problems prevent UI from coming up

score 25
You have not voted. Not Planned

previous Idea:  Include HTTPD & Cron status when checking tideway status

 

My Idea is to have the UI give an informative message if all "tideway" services are up, but something else such as a crontab problem is preventing the UI from being available.

 

If a customer directly edits the crontab using crontab -e (not the correct way to update the crontab for the Discovery appliance), nothing happens of bad consequence until days or weeks later when the services are restarted for some reason, such as the customer updates the TKU or customer takes action from the UI to restart services or perform a reboot.

 

At that point, the services will start, but the UI will report "The appliance has been shut down" forever, until the problem is solved.

 

There is no indication of the problem that is easy to find unless you know what you are looking for.

The problem will be seen in log/tw_svc_control.out, and in log/tw_cron_update.log if you happen to look there.

Since all services are running, those 2 files will not be the most recently updated log files by the time you might look.

And, a grep for ERROR: or CRITICAL: will not help.  Actually, there is no helpful keyword to grep for in tw_cron_update.log

 

If the failure were to occur immediately after editing crontab with crontab -e, then customer might realize the problem.  But, no failure occurs at the time of the edit.

Weeks later, customer is unlikely to connect the two events.

 

 

There might be some similar problem with baseline?

 

===================================================

If interested, here is what you will see in the 2 log files mentioned:

 

[tideway@host log]$ cat tw_cron_update.log

Merging .cron files into new crontab file...

Comparing current cron with expected cron

 

Since tw_cron_update was last run, the crontab for tideway

has been changed. The differences are:

30a31

> #

The expected cron is contained in the file

      /usr/tideway/etc/cron/tw_cron.previous

Once the differences are accounted for, remove the file

and run tw_cron_update again.

[tideway@clm-aus-spil1s log]$

 

 

 

[tideway@host log]$ cat tw_svc_control.out

Starting local BMC Discovery application services

    Starting Security service: [  OK  ]

    Starting Model service: [  OK  ]

    Starting Vault service: [  OK  ]

    Starting Discovery service: [  OK  ]

    Starting Mainframe Provider service: [  OK  ]

    Starting SQL Provider service: [  OK  ]

    Starting CMDB Sync (Exporter) service: [  OK  ]

    Starting CMDB Sync (Transformer) service: [  OK  ]

    Starting Reasoning service: [  OK  ]

    Starting Tomcat: [  OK  ]

    Starting Reports service: [  OK  ]

    Starting External API service: [  OK  ]

    Starting Application Server service: [  OK  ]

    Updating cron tables:

Since tw_cron_update was last run, the crontab for tideway

has been changed. The differences are:

30a31

> #

 

The expected cron is contained in the file

      /usr/tideway/etc/cron/tw_cron.previous

 

Once the differences are accounted for, remove the file

and run tw_cron_update again.

 

[FAILED]

[tideway@clm-aus-spil1s log]$

Comments

Vote history