1 2 Previous Next 20 Replies Latest reply: Feb 2, 2012 6:31 AM by Kate Fell RSS

TFTP Servers for Provisioning

Jim Campbell

We've been provisioning in one datacenter using a PXE/TFTP setup and it works well enough.  We're looking at provisioning in a separate datacenter and while we can transfer the OS images at pretty close to the same speed, the extra latency is causing the boot image to take forever to download.  From looking at a network monitor capture and reading about TFTP, it seems that it sends a single packet of 1468 bytes and then waits for a response before sending the next one, which would explain the slowness.  Is there a way to change the block size for the blade TFTP service?

 

If necessary, we could also set up a blade TFTP server in the other datacenter was well, but there does not appear to be a place to specify a second TFTP server on the appserver?  How/where would we designate which TFTP server is to be used?

  • 1. Re: TFTP Servers for Provisioning
    Bill Robinson

    I would setup a 2nd pxe/tftp server.  You can override what’s in the db by editing the br/pxe.conf on the pxe server and set “use_config_file=true” and then filling in all of the options.  you will still need to run blpxeconf to setup the db connection though.

  • 2. Re: TFTP Servers for Provisioning
    Jim Campbell

    The provisioning process is not my forte, but if we set up a second PXE/TFTP server (and probably in most cases a second datastore?) then we would have DHCP relay set up on the remote datacenter's network to point to the new PXE server, which we can configure to point to the new TFTP server using pxe.conf?  Do the PXE and TFTP servers get their information about which clients to service (i.e. which MAC addresses to respond to) directly from the database?  My concern was that if i redirect remote PXE clients to the new PXE server, it would still communicate with the appservers (where we've specificed our old TFTP server in the UI) and try to use the old TFTP server.

     

    If the PXE server can be configured to communicate with the database/appserver to pull back the MAC addresses to service while overriding the behaviour of pulling back the TFTP server to use, then we shouldn't have any problems.

  • 3. Re: TFTP Servers for Provisioning
    Paul Seager-Smith

    Jim,

     

    you can only have a single PXE server defined in the database. If you use multiple PXE servers, you need to configure PXE to use the pxe.conf file. This is an option in the PXE configuration utility.

     

    You then need to have a deploy job that sends out the correct pxe.conf files to each PXE server (or you can do it by hand if you really want to).

     

    Regards,

     

    Paul

  • 4. Re: TFTP Servers for Provisioning
    Jim Campbell

    With only two provisioning servers I just edited the file by hand.  It appears to be working as PXE clients on the same subnet are getting the WinPE images.

     

    For one last question, i see entries for broadcast responses to PXE requests like:

     

    [07 Jan 2011 09:27:53,685] [Thread-19] [INFO] [::] [] RPacketProcessor Thread id= 56 started

    [07 Jan 2011 09:27:53,810] [Thread-19] [INFO] [::] [] It is a proxy DHCP packet from /255.255.255.255:68

    [07 Jan 2011 09:27:53,810] [Thread-19] [INFO] [::] [] Packet type is DHCP Discover setting up DHCP Offer packet for reply

    [07 Jan 2011 09:27:53,810] [Thread-19] [INFO] [::] [] makeBootMenu : type 1

    [07 Jan 2011 09:27:54,013] [Thread-19] [INFO] [::] [] Replying to /255.255.255.255:68

    [07 Jan 2011 09:27:54,013] [Thread-19] [INFO] [::] [] RPacketProcessor Thread id= 56 Finished [07 Jan 2011 09:27:53,685] [Thread-19] [INFO] [::] [] RPacketProcessor Thread id= 56 started
    [07 Jan 2011 09:27:53,810] [Thread-19] [INFO] [::] [] It is a proxy DHCP packet from /255.255.255.255:68
    [07 Jan 2011 09:27:53,810] [Thread-19] [INFO] [::] [] Packet type is DHCP Discover setting up DHCP Offer packet for reply
    [07 Jan 2011 09:27:53,810] [Thread-19] [INFO] [::] [] makeBootMenu : type 1
    [07 Jan 2011 09:27:54,013] [Thread-19] [INFO] [::] [] Replying to /255.255.255.255:68
    [07 Jan 2011 09:27:54,013] [Thread-19] [INFO] [::] [] RPacketProcessor Thread id= 56 Finished

     

    The DHCP Offer packet is just a packet that directs the client to the TFTP server right?  Its not a normal DHCP offer that would be sent from a DHCP server?  My understanding is that a DHCP client will discard a second DHCP offer packet on a subnet with competing DHCP services, but in this case they can coexist because the one sent by the Blade PXE server is not a 'real' DHCP Offer?

  • 5. Re: TFTP Servers for Provisioning
    Bill Robinson

    the dhcp server is responding to the dhcp request from the target w/ the ip, gateway, etc, 211, 212.  the pxe server is responding to the same request (i think) w/ options for 'next-server' and 'filename' - which would be the tftp server ip from either the db or the config file, and 'pxelinux.0' as the boot file name. 

     

    the target bootstraps off of that image, then reads files in /tftproot/X86PC/pxelinux/pxelinux.cfg/ to figure out what image to boot into - gentoo, rhel, sles, winpe, etc.

  • 6. Re: TFTP Servers for Provisioning
    Bill Robinson

    Well – that’s not entirely true ☺  there is a hack that will let you monitor the pxe instances and have them show up separately in the infra mgr, but you still need to use the pxe.conf for the configuration.  I believe there is a rfe to properly support multiple pxe instances but I’m not sure when that’s targeted for release.

  • 7. Re: TFTP Servers for Provisioning
    Jim Campbell

    Is there any problem with running two PXE/TFTP setups without being able to monitor them separately in the infrastructure manager (outside of inconvenience?).  This setup seems to be working, but for now only for servers on the same subnet as it takes a while to get changes to network services done.

     

    I was curious about how the PXE server responded on its own subnet because it appears to be responding to PXE requests without anything being configured on the subnet's DHCP relay.

  • 8. Re: TFTP Servers for Provisioning
    Jim Campbell

    About datastores:

     

    I know we can manually designate the datastore to use when creating the job, but is there any way to automatically use a certain datastore based on the IP address of the PXE client (akin to how Marimba subnet repeater policy directs clients to repeaters based on their subnet)?

  • 9. Re: TFTP Servers for Provisioning
    Bill Robinson

    There is no issue not being able to monitor them via infra mgr.  the pxe server is a dhcp server that responds to the broadcasts, but only responds w/ the ‘next-server’ and ‘filename’ options.

  • 10. Re: TFTP Servers for Provisioning
    Kate Fell

    Hi I have the second PXE server configured but when I try run a provisioning job the windows files are still being transferred from the first PXE server which is taking for ever..The 2nd PXE server (in the remote network) is pointing to the Live DB and the pxe.conf is set to use_config_file=true and the TFTP is set to the Build tftp server not the remote one...

     

    Where am I going wrong?

  • 11. Re: TFTP Servers for Provisioning
    Bill Robinson

    in the pxe.conf on the 2nd pxe server the tftp ip should be for the remote system. 

  • 12. Re: TFTP Servers for Provisioning
    csdale

    Bill ,

     

    Can you point me in the right direction to get the "hack" to see mulpiple PXE Servers....I will be having 10 to 15 pxe servers based on region for performance and to limit PXE boot problems over slow wan links and would like to be able to see thim in Infra MGR.

     

    Is this something i can get from support for 8.0 sp10

     

    Chris

  • 13. Re: TFTP Servers for Provisioning
    Bill Robinson

    Iirc you need to use blasadmin on each pxe box and set the appservername parameter – so like:

     

    Blasadmin –s _pxe set app appservername blah

     

    And restart the pxe service.  It might be ‘displayname’.

  • 14. Re: TFTP Servers for Provisioning
    Kate Fell

    HI, I am still having some issues here....

     

    Now have 2 PXE servers in the same stretched Build vLan covering 2 Data Centers.... The 1st PXE server is working fine, the 2nd PXE server is configured, the pxe.conf is updated to use_config_file=true and the TFTP is set to the 2nd PXE server.... When the server PXE boots it always picks up the first PXE server and tries to push the files from the 1st PXE server which takes for ever.... How do I configure it to pick up 2nd PXE server when Provisioning in the remote DC...

1 2 Previous Next