Advanced Repeater servers have numerous advantages over Traditional BL Repeaters but are also a little more complex to configure and troubleshoot. The goal of this blog post is to assist with the troubleshooting of issues which users may encounter when using Advanced Repeater servers during BSA deployments.
BMC Server Automation (BSA) Deploy Jobs can be configured to deploy directly to the selected Target Servers or indirectly via Repeater Servers.
With an indirect deploy, BSA Deploy Jobs use Repeater Servers to stage the payload to an intermediate server before copying it to the Target Servers and completing the deployment. See here for some advantages of performing indirect deploys.
There are two types of Repeaters in BSA:
- Traditional Repeater servers (aka BL Repeaters - uses a combination of BSA NSH/Agent) and,
- Advanced Repeater servers. (uses a combination of BCA/Marimba and BSA Agent)
Below is control flow diagram of Advanced Repeater enabled BSA environment
Introduction to BSA Advanced Repeater servers:
The main advantages of using BSA Advanced Repeaters over Traditional BL Repeaters include:
- File Compression,
- Byte Level Differentiating and
- Network Throttling.
Advanced Repeaters can also be used to control the network bandwidth usage and throughput between the BSA File Server and the Targets.
Closely related to the Advanced Repeater in BSA is the Advanced File Server. See here for a discussion of the relationship between the Advanced Repeater and the Advanced File Server.
See this section of the BSA documentation for full details on installing and configuring Advanced Repeater servers in BSA.
Advanced Repeater Servers - Frequently Encountered Issues
This section discusses some issues which users frequently encounter when using Advanced Repeater Servers during their BSA Deployments.
Problem 1: "Remote host is unkown" error during staging.
The following error is encountered in the BSA Job Run Log:
Failed to stage on repeater at //<AdvRepeaterHostName>/C/tmp/ars_staging: JNI fileExists '//AdvRepeaterHostName/C/tmp/ars_staging' failed:: Remote host is unknown //AdvRepeaterHostName/C/tmp/ars_staging
The Advanced Repeater Host is not reachable from the AppServer Host or the FileServer Host to initiate the copy of source files.
- From Infrastructure Management -> Repeaters node, right click on the Advanced Repeater Host and select "Update Repeater Server Status" to verify the connectivity to Advanced Repeater.
- If the connection has issues, check application server logs of configuration App server.
- From the Configuration Application Server host check below:
- Ping AdvRepeaterHostName (ping may be disabled in some customer environments)
- Telnet AdvRepeaterHostname 7717 (this is RPC listener port)
- Check whether the BMC BladeLogic Advanced Repeater Tuner Service is running or not.
- On Windows: check services menu (services.msc) for above service status.
- On Linux: using command: ps -ef | grep "tuner"
- On Windows: C:\Users\IMATTA> netstat -A | find "7717"
- On Linux: $ netstat -anp | grep "7717"
Solution: Fix the issues accordingly and ensure the BSA Application Server can connect to the Advanced Repeater causing the issue and re-run the deploy job.
Problem 2: "No such file or directory" error during staging.
The following error is encountered in the BSA Job Run Log::
java.io.ioexception: JNI file copy from //FileServerHostName/opt/bmc/bladelogic/NSH/storage/depotfiles/255a3abd04348bbfddd6f8242f883aea to
//AdvRepeaterHostName/temp/stage/73db53773f64340b8496450314f60695/255a3abd04348bbfddd6f8242f883aea No such file or directory
This error can occur for a number of reasons including incorrect agent ACLs, connectivity issues, disk space and incorrect server properties.
- Ensure the Advanced Repeater host is reachable from the File Server host network (ping & telnet commands).
- Check the rscd.log on the Advanced Repeater for the presence of any error messages from the time of the failed staging. Also ensure the local user which is being mapped to is as expected e.g. root, Administrator etc.
- Check user mappings defined in the "users.local" file:
- Default local OS user for unix advanced repeaters is "root"
- Default local OS user for windows advanced repeaters is "Administrator"
- Example of users.local file entry is below:
- BLAdmins:BLAdmin rw,map=root (Unix)
- BLAdmins:BLAdmin rw,map=Administrator (Windows)
- Ensure there is sufficient free disk space on the Advanced Repeater host.
- Run an "Update Server Properties" BSA job against the Advanced Repeater Host.
Problem 3: "Replicator: 19012 Publish-sdk" error during staging.
The following error is encountered in the BSA Application Server Log:
[20/Jul/2011:14:52:39 -0400] Replicator: 19010 Publishing C:\Program Files\BMC Software\BladeLogic\8.0\NSH\storage\blpackages\softlinked-Copy-of-blp-8.1 to http://AdvancedFileServerHostName:5282/blpackages/softlinked-Copy-of-bl_jmx_monitor.properties-blp-8.1 version: .v12
[20/Jul/2011:14:52:40 -0400] Replicator: 19012 Publish-sdk Exception: java.io.IOException: commit failed: C:\Program Files\BMC Software\BladeLogic\8.0\NSH\storage\blpackages\softlinked-Copy-of-bl-8.1\.rep\pub_internal\.castanet\publish.mrb
This error is usually accompanied by the following NullPointerException in the BSA Appserver log:
[20 Jul 2011 15:08:29,359] [WorkItem-Thread-15] [ERROR] [BLAdmin:BLAdmins:] [Deploy] Error to execute ProxyAdminOpImpl on host 'AdvancedFileServerHost'
This error is usually caused by network issues (possible intermittent) between the BSA Application Server and the Advanced File Server host.
- From Infrastructure Management -> Advanced File Server node, right click on the Advanced File Server Host and select "Update Advanced File Server Status" to verify the connectivity between the BSA Configuration Appserver and the Advanced File Server.
- Go to the Advanced File Server properties window from Infrastructure Management and see If the Advanced File Server uses a different host for the Transmitter Host and verify if that host is also accessible from the BSA Application Server.
- Open a web browser and check the below URLs for status of the transmitter host (use https if SSL enabled) and verify "storage" parameter values for any space issues:
- Re-run the Deploy Job again.
Problem 4: Common errors during the Installation and Configuration of Advanced Repeaters.
Here are some issues which users often encounter when installing and configuring Advanced Repeaters in BSA:
"Installer User Interface mode not supported" error while installing on Windows 2012 environment
Please refer to knowledge article# KA407406
"Error loading jvm.dll" error while enabling Advanced File Server / Advanced Repeater
Please refer to knowledge article # KA412884
"Could not connect to http://AdvRepeaterServerHostorIP:7717/proxy/admin"
Please referto knowledge article # KA383534
I hope this post provides useful information about how to investigate issues with Advanced Repeater Servers in BSA Deploy Jobs. Please rate my blog below to let me know if it was useful. You can find more content like this in BMC BladeLogic Pulse Blogs.