-
1. Re: TrueSight v10.7 Agent Deployments
Patrick Mischler Jun 8, 2017 12:43 AM (in response to Daric Smith)Hi Daric
We have successfully upgraded several PATROL Agents running on Windows.
Are there multiple agents started?
Regards,
Patrick
-
2. Re: TrueSight v10.7 Agent Deployments
Amitai BarJun 11, 2017 2:30 AM (in response to Daric Smith)
Hi
Can you please open a support ticket for this?
Thanks
Amitai
-
3. Re: TrueSight v10.7 Agent Deployments
Maneesh Kumar Jun 15, 2017 4:17 AM (in response to Daric Smith)Hi Daric,
I am getting the same errors while upgrading the agents on many Windows servers? I have applied the repository fix pack 001. Tried installing the old package, then created a new package, but all failed with same errors.
Along with the errors you mentioned above, we are also getting errors on some servers saying C:\Program Files\BMC Software\Patrol3\bin\msvcr100.dll file is in use. Uninstall does not work either. These files cannot be deleted until the server is restarted, which is not the solution.
Have you found a solution for it? Did BMC provide a solution for this?
Thanks,
Maneesh
-
4. Re: TrueSight v10.7 Agent Deployments
Daric Smith Jun 15, 2017 8:50 AM (in response to Amitai Bar)I mentioned that I have an open support ticket with BMC in the original post.
-
5. Re: TrueSight v10.7 Agent Deployments
Daric Smith Jun 15, 2017 8:55 AM (in response to Maneesh Kumar)1 of 1 people found this helpfulI am doing a webex with development early next week to troubleshoot. Following are all of the things I have tried (and am going to try). Perhaps you have other troubleshooting that you can add in case I've missed something.
- Ran ProcMon during
install. You can see the PML file in the root of D. I could see the
FILE LOCK error but I couldn’t see what was locking it. - Ran Process Explorer
checking for open handles to patrolagent.exe - I ran sysinternals
“handles64.exe” utility BUT I was getting access denied viewing the
handles owned by SYSTEM. Maybe you could get that to work.- The package we create doesn't leverage an account so only uses local system. I have read some things about the SYSTEM account keeping handles open even when the process is dead. Because it is SYSTEM, you can't view or kill them.
- I read stuff about
the Application Experience service needing to be set to Automatic. Didn’t
help. - On <servername> I stopped
and disabled the following to no avail:- DynamicCache Service
- Enstart64
- Networker
- OfficeScan
- VMTools
- Windows Defender
- I also created a new
Dev VM and successfully ran the installer just to make sure it worked.- I have not yet installed the v10 agent then done an
upgrade on a new VM.
- I have not yet installed the v10 agent then done an
- I was able to
successfully deploy the same package to the TDMZ servers (2012).
The error seen is:
[15:42:03]
global_BackupRestoreInstall command is
[C:\Users\\Desktop\WinX64_Agent_EDEL_v10_7\bmc_products\Install\instbin\getDiskSpace.bat
"C:\Program Files\BMC Software\\Uninstall" "C:\Program Files\BMC
Software\Patrol3\tmp" "C:\Program Files\BMC Software" "Patrol3"][15:42:03]
RedirectSTDinoutAndCreateProcess:Created Process
[C:\Users\\Desktop\WinX64_Agent_EDEL_v10_7\bmc_products\Install\instbin][15:42:05]
ExecuteScriptNT:external process output = [OUTPUT]C:\Program Files\BMC
Software\Patrol3/bin\PatrolAgent.exe is in use[15:42:05] C:\Program Files\BMC
Software\Patrol3/bin\PatrolAgent.exe is in use[15:42:05]
ExecuteScriptNT:external process output = [OUTPUT] The PATROL installer upgrade
process failed while installing due to existing files being used by active
processes on the system.[15:42:05] The PATROL
installer upgrade process failed while installing due to existing files being
used by active processes on the system.Path to installer log files: C:\Program
Files\BMC Software\log - Ran ProcMon during
-
6. Re: TrueSight v10.7 Agent Deployments
Maneesh Kumar Jun 16, 2017 8:26 AM (in response to Daric Smith)1 of 1 people found this helpfulHi Daric,
Thanks for sharing the details.
I tried to install the agent, but some files including patrolagent.exe were not removed because they were in use. PatrolAgent.exe was in use by DHCP Client process. I stopped the DHCP client process to release the lock but did not work. I tried some other ways to release the lock on the file but nothing worked. Eventually, I had to restart one of the servers, which released the lock on the files, then reinstalled the agent. But its not possible to restart the production servers just to upgrade the agent.
I faced similar issue with msvcr100.dll file being in use.
The issue seems related to how the upgrade is handled in the installation utility of 10.7 repository. I have never faced these issues with agent upgrade before.
Thanks,
Maneesh
-
7. Re: TrueSight v10.7 Agent Deployments
Daric Smith Jun 16, 2017 8:53 AM (in response to Maneesh Kumar)Oh, that happened to us: It said the DHCP Client was locking the process. We found it to be a red herring because when we stopped the DHCP Client, it would go on to another service and stopping that would tell us another service.
There is definitely something on the OS side that is causing the lock.
I have built a 2008 R2 VM and so far have installed a bunch of our utilities such as backup, forensic and A\V but have not run across the problem.
The final step is to fully Windows Patch the server to see if that breaks it.
-
8. Re: TrueSight v10.7 Agent Deployments
Daric Smith Jun 16, 2017 1:27 PM (in response to Daric Smith)What I am really curious about: How many deployments have been done by BMC in their on corporate network? I know they QAed it in the development but are they actually using v10.7 in their production environment that has thousands of nodes? It can't just be us that are experiencing this in our production environments.
-
9. Re: TrueSight v10.7 Agent Deployments
Andy AbramsJun 16, 2017 2:38 PM (in response to Daric Smith)
Daric,
BMC-IT has been running TrueSight 10.7 in Production since April 20th (~24 days after it was released GA). I will check with the Assurance team manager to see if they have had any of the issues upgrading PATROL Agents as described in this thread.
-Andy
-
10. Re: TrueSight v10.7 Agent Deployments
Daric Smith Jun 19, 2017 1:10 PM (in response to Andy Abrams)Thank you very much for the response Andy. If you could provide some more information on how your agents are handled as it will help the BMC support and engineers to figure out why Maneesh and I have this issue:
- Are 100% of the agents deployed in your environment created through the TrueSight Repository (so not using the old Patrol installer utility)?
- Have you converted most of your agent configurations over to Infrastructure Policies or is it still mostly done through PCM?
- Have you attempted v10.7 agent deployments through TrueSight to a large number of production systems?
- What deployment methods to you leverage for your agents (manual install / Blade Logic)?
With the exception of Monitoring Studio that still requires some level of PCM and that is only on about 4 nodes out of 1200, everything we configure is through TrueSight.
Again, I am trying to gather as much data as possible to determine the differences and why you environment with thousands more nodes wouldn't experience this at all.
-
11. Re: TrueSight v10.7 Agent Deployments
Daric Smith Jun 28, 2017 9:32 AM (in response to Daric Smith)1 of 1 people found this helpfulNew information breakthrough for this as I've worked with support.
I learned something new for running Process Explorer as Local System: Using psexec with the switches -i -s -d will run the target process as Local System.
So, executing: psexec -i -s -d procexp.exe will open it and now we can see all of the information for open handles.
Now, I can see that svchost -k LocalServiceNetworkRestricted is what keeps the handle open:
This also matches up with the "services" that were keeping the process open: DHCP, Event Log and TCP/IP Helper:
Can you see if you get the same results on a troubled server?
We are now waiting for BMC to provide more information on why patrolagent.exe handle gets locked for that svchost.
-
12. Re: TrueSight v10.7 Agent Deployments
Maneesh Kumar Jun 29, 2017 1:48 AM (in response to Daric Smith)Hi Daric,
Thank you very much for sharing this information. I can confirm that we have the exact scenario which you have described.
Maneesh Kumar
-
13. Re: TrueSight v10.7 Agent Deployments
David Kokes Jul 2, 2017 6:59 AM (in response to Daric Smith)Hi Daric,
Thank you for the useful information on this. We are also experiencing same issue as you (installation fails because patrol process is being used).
I have also some other thing. If I run the update on more agents (for example 100) it will start with first batch of 40 and once that is done it will continue with another 40 and then it will do the 20. It will not run simultaneously on all of them. Info from support is that it should be possible to run 500 simultaneously. Have you seen this?
Thank you
-
14. Re: TrueSight v10.7 Agent Deployments
Patrick Mischler Jul 3, 2017 12:00 AM (in response to David Kokes)Hi all,
The maximum numbers of paralell deployments can be configured in the tssh properties. For example 'tssh properties list' does return this value:
MAX_PARALLEL_PA_DEPLOYMENT 40
A value of '40' seems to be the default.
You can set this to another value witht the following command:
tssh properties set MAX_PARALLEL_PA_DEPLOYMENT 500
But I am not sure what BMC recommends. To be sure, ask the support.
Regards,
Patrick