1 2 Previous Next 16 Replies Latest reply on Sep 20, 2011 1:18 PM by Hemal Shah

    2008 R2 Erratic Failures on Step14

    Jim Campbell

      We have had a longstanding and frustrating issue with some provisions of 2008 R2.  On step 14, where the server maps a drive to a windows share on the datastore server and attempts to install the agent and call back to the appserver, sometimes (not always, but fairly frequently especially with physical boxes) the newly provisioned server will fail to map to the datastore share.  This does not occur with 2003 or 2008 (non-R2) provisions using exactly the same process.  We also have a post-install command to map to the same share that also fails in these situations.  The security log files on the datastore server indicate the password is wrong but its the same one we use everywhere else - the same error shows up 3 times in succession for the attempt to map to the share for the agent install, the post-command we include, and the bmiwin call.  The password actually appears in cleartext in the runonce log (since it is used in the manually defined post-install steps) and is accurate.  In fact, we've included a script to parse the runonce file and pull out all of the failed commands and rerun them - it always works when executed manually.


      Has anyone else seen this problem?  I can't say for certain but it seems to mostly affect our builds on physical servers - virtual 2008 R2 builds often get past this point without any issue.

        • 1. 2008 R2 Erratic Failures on Step14
          Bill Robinson

          btw, s: should be mapped to the data-store in the 'post-install' bit of the system package so you shouldn't need to map it again.


          can you check to see if that is mounted? or is that what you are saying isn't mounted - i'm not sure because you say you are also manually mounting the datastore.


          could there be a control character getting inserted into the passwd?


          are you using a different datastore instance for r2 than your other system packages?

          • 2. 2008 R2 Erratic Failures on Step14
            Jim Campbell

            We probably don't strictly need to remap the drive.  The sequence for us goes like:


            map r: to datastore share (blade automagic)

            install agent (blade automagic)

            map t: to datastore share (post-install step we specify with share password in cleartext)

            run script from share (vbs script that changes admin account name)

            map s: to datastore share (blade automagic)

            run bmiwin.exe (blade automagic)


            When these fail, all three attempts to map the drive fail in rapid succession (can see this from the windows security logs on the datastore server).  We can then log in, pick up the password in cleartext from the runonce.log, and map the drive and carry on using precisely the same commands that failed.


            We only have two datastore instances and we use the same one on probably 95%+ of our provisions.  I don't know that its ever happened on the other one but we may just not have done enough (or possibly any) r2 provisions on it to know.

            • 3. 2008 R2 Erratic Failures on Step14
              Bill Robinson

              try re-entering the passwd in the datastore instance and make sure there are no extra characters.  it looks like you have a bad password. (even though the cleartext is ok)

              • 4. 2008 R2 Erratic Failures on Step14
                Jim Campbell

                I don't see how.  We use the same datastore for provisioning 2003, 2008, and 2008 R2 servers.  2003 and 2008 servers never have problems and 2008 R2 failures are not consistent.  If the password were entered wrong, we would not be seeing only sporadic failures with only one type of OS.

                • 5. Re: 2008 R2 Erratic Failures on Step14

                  Have you tried passing in “net use * /delete /all” in your script before it re-maps the drives.

                  • 6. Re: 2008 R2 Erratic Failures on Step14
                    Bill Robinson

                    I know - it's weird that it happens only in R2.  what's the error in the event log on the share say? 


                    does the passwd have any special characters in it ?

                    • 7. Re: 2008 R2 Erratic Failures on Step14
                      Jim Campbell

                      When the problem occurs, it doesn't re-map drives so I don't think deleting mapped drives first would help.  All of the mappings fail at once.  Then when you log in to the server and try it, all of the mappings always succeed.


                      The password has numbers, letters, and a # sign.  Could it be somehow interpolating that character?


                      The event log error (in the Security log) is:


                      Logon Failure:

                        Reason:  Unknown user name or bad password

                        User Name: local_account_for_mapping

                        Domain:  Server_With_Datastore

                        Logon Type: 3

                        Logon Process: NtLmSsp

                        Authentication Package: NTLM

                        Workstation Name: Server_Being_Provisioned

                        Caller User Name: -

                        Caller Domain: -

                        Caller Logon ID: -

                        Caller Process ID: -

                        Transited Services: -

                        Source Network Address: IP_Of_Server_Being_Provisioned

                        Source Port: 0




                      Logon attempt by: MICROSOFT_AUTHENTICATION_PACKAGE_V1_0

                      Logon account: local_account_for_mapping

                      Source Workstation: Server_Being_Provisioned

                      Error Code: 0xC000006A


                      (these 2 events in the log are from the same time)

                      • 8. Re: 2008 R2 Erratic Failures on Step14
                        Bill Robinson

                        # should be ok, but just for kicks take it out of the passwd and see what happens.


                        also, take out any net use commands that you manually put in, and see if s and/or r get mounted

                        • 9. 2008 R2 Erratic Failures on Step14
                          Bill Robinson

                          did you ever resolve this?

                          • 10. 2008 R2 Erratic Failures on Step14
                            Jim Campbell

                            No, we've just learned to live with it.

                            • 11. 2008 R2 Erratic Failures on Step14

                              Jim, if net use fails for wrong password and you are sure the password provided is right, there are couple of thing to check.

                              One is the timezone difference on the two servers, one that you are provisoning i.e., your physical server and the DataStore you are using. I guess you have a Windows datastore. I have seen such issues with net use in WinPE due to datetime difference.

                              Also, if you are having a script of your own and that too is not working for net use, try removing the domain name in the command and I suppose you might have already tried using IP instead of hostname if at all.


                              Can you attach the runonce.log file here, I would like to take a look at it.  

                              • 12. 2008 R2 Erratic Failures on Step14
                                Jim Campbell

                                I'll get one the next time this occurs - it is a sporadic problem and unfortunately does not generally occur on our test boxes that I can keep around to troubleshoot.  I had considered the possibility that time skew was a problem as we've seen it cause errors in the provisioning process before, but I would think that the windows event logs would have a different error code if this were the case.

                                • 13. 2008 R2 Erratic Failures on Step14
                                  Bill Robinson

                                  what is your datastore server - samba, 2003, 2008 ?  was the account ever getting locked? 


                                  could this be some kind of UAC issue on the outbound side?

                                  • 14. 2008 R2 Erratic Failures on Step14
                                    Jim Campbell

                                    The datastore is 2003x86.  The account is not being locked.


                                    By outbound, you mean on the server that is being provisioned?  We disable UAC but not until the first step of the post job so it would still be enabled on the server at the time the drive mapping is performed.

                                    1 2 Previous Next