We've been getting some odd behaviour that may be related to DNS. Our appservers are Windows and have a dns suffix order of x.y.z.com and z.com (in this order). Some of our rebuilds have required moving networks and when this happens the names usually don't match: server.x.y.z.com will resolve to the new IP, server.z.com will resolve to the old IP. This has caused a few problems: during the provisioning, sometimes (not always) the agent will fail to enroll at the end of the provisioning process. Eventually we're able to run a verify on the server object and move on. The other problem comes from our qa job and occurs even long after any dns names cached (by Windows at least) would have expired. It also occurs when live browsing the hardware information node for affected servers:
Error Mar 26, 2011 7:49:35 PM com.bladelogic.app.collector.AssetCollectionException: com.bladelogic.app.service.agentservice.AgentConnectionException: com.bladelogic.app.remote.BlRemoteException: org.apache.xmlrpc.XmlRpcException: Failed to connect to SERVER:4750(component=QA_TEMPLATE (SERVER), selector=LogicalStorageDevices:/Hardware/StorageDevices/LogicalStorageDevices)
Jobs run against SERVER, i can live browse other nodes, extended objects run, but when i try to access the Hardware Information Node I get essentially this same error. I ran a network trace while attempting to live browse the hardware information node and saw that traffic was being sent to the old IP: i.e. the one to which server.z.com resolves. Running nslookup, ping, etc on the appserver all return the IP for server.x.y.z.com
Does Blade have any sort of separate name caching? Or is there any reason it would be resolving names differently than windows? All of our server objects use a single host name (unqualified). The server object has the fqdn right in its fq_host property and its ip_address property is also correct (i.e. the new IP). Is there any way blade could be using an old IP from the previous server object of the same name (the previous server object was decommed prior to reprovisioning the server).