Broadcasts are background tasks that run on a schedule and are used to send reports to users or an FTP server. It is a useful way to make sure the right people can access reports quickly, at the right time and with the latest data. However, when badly organized, they are also a common reason for performance issues in the environment. For this reason, I’ll be discussing here the best practices when scheduling a broadcast, ways to identify when the broadcasts are the cause of a performance issue and how to fix it.
Here are some best practices we can follow when scheduling a broadcast:
1. Amount of broadcast.
Coordinate the people that have access to schedule a broadcast. All users with “content writer” role will have the option to set up a broadcast. Is important to keep a limit to the amount of broadcast that are scheduled. The amount of broadcast that are supported will depend on how big the reports are, and the amount of computing resources allocated in the Smart Reporting and AR System servers.
2. Broadcast frequency.
It is possible to have a report being sent every 5 minutes, however, that would have the report refreshing every 5 minutes, this will generate a constant load to the servers that will affect the performance of the console. It is recommended to run a broadcast one or two times a day, not more.
3. Broadcast concurrency.
When selecting the time of broadcast execution, is important to keep in mind other broadcast that might be running at the same time, if there is already a broadcast running at that time, then space it out by fifteen minutes. That way, it will give it time to allocate enough resources to each task.
4. Scheduling time frame.
It's recommended to schedule broadcast out of office times, for example, having the broadcasts run early in the morning, before the standard users start accessing the console.
When there is a broadcast performance issue in an environment, the first thing to check is the Schedule Management panel in the Administration menu of the Smart Reporting Console. This panel shows all scheduled tasks, including the broadcasts.
By clicking a broadcast, you can see the details. This information can help to check if the best practices are being followed. Also, if the broadcast fails, the status will show a general error message and the clock icon will show in red.
Another useful panel is the Background Execution panel, it will show all running and waiting tasks.
The task scheduler limits the number of tasks that can run by default. It will check every minute and look for tasks that need to be run. If it’s the right time to run a task, it gets submitted to the queue. If there are free threads, the tasks get removed from the queue and start running. There are 5 threads that can run at the same time and the queue size is 20.
These values can be changed in the Task Scheduler Values -- in the web.xml file located in the following directory:
- Bundled Tomcat: //webapps/ROOT/WEB-INF
- External Tomcat: /Tomcat/webapps//WEB-INF
Note: The Smart Reporting service needs to be stopped before modifying the web.xml file.
Finally, when a broadcast runs, it creates a call to the data source for the data of the report, it does this by using the tomcat service in the Smart Reporting server. It is important to make sure there is enough memory allocated in the service and the proper Java parameters for the size of the environment. As well as a capable AR Server with enough resources to handle the requests of data from the Smart Reporting console, if there is a requirement for a large number of broadcasts, it is recommended to have a dedicated AR Server to Smart Reporting outside of the server group connected to a replicated AR database.
If a broadcast related performance issue has been identified, the tasks can be momentarily disabled by adding the following script in Servlet block in the web.xml file mentioned above while the service is stop.
<init-param> <param-name>DisableTaskScheduler</param-name> <param-value>TRUE</param-value> </init-param>
This will give the opportunity to set the service back up without the performance issues to be able to evaluate if the best practices are being followed and review the possible limits in resources or parameter values.
Related Knowledge Articles and documents: