Hi

we have just set up SCCM 2012 and have configured OSD. It works well apart form the length of time it takes to download the boot.wim on a PXE boot. 

We took some network traces and noticed that TFTP was behaving as if the Windowing was set to 1 so every block was being ACK'ed before another was sent.

I checked the Windowing size on the bcd file of the boot image and the setting was

ramdisktftpwindowsize   4

Any ideas why TFTP doesn;t seem to be picking this up?



Hi. May be this posts help you:

https://netecm.netree.ch/blog/Lists/Posts/Post.aspx?ID=57#.U606pPl_uCQ

http://blog.danovich.com.au/2011/07/28/slow-sccm-osd-tftp-pxe/

Thanks for the reply

I've already read both of these posts. I've not checked for the Jumbo Frames setting yet but it doesn't really explain why the TFTP windowing isn't working.

Word of warning to anyone who is tempted to increase the block size. Bear in mind from '08 onwards the default TFTP block size is 1456. We found OSD deployment fell over or ground to a halt if we went above this. I guessing that this is because the 1500 byte MTU limit on our network.

Cheers

Alex

To my knowledge, the BCD of the boot disk in not used during the TFTP transfer in ConfigMgr. ConfigMgr (from memory) generates an alternate BCD that is used. This is also briefly discussed at http://adventuresinosd.blogspot.com/2011/03/troubleshooting-pxe-in-sccm-osd-part-3.html

Have you tested this setting at least though as EminM has suggested? change the value on your WDS server? We also had sluggish downloads and once we changed the below value to 8192, performance went from pretty poor to brilliant instantly (https://netecm.netree.ch/blog/Lists/Posts/Post.aspx?ID=57#.U6995fmSx8F)

The only downside is older hardware can error out (when I say older I mean old old, like 6 - 7 - 8 years ago)

RamDiskTFTPBlockSize

This value is the most important setting to boost the download speed over TFTP. It increases the block size and reduces the count of needed acknowledge packages.

Create a DWord Value with:

Name: RamDiskTFTPBlockSize

Key: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\SMS\DP

Values: 1024(Default), 2048, 4096, 8192 or 16384

Yeh, I have changed that value. Performance was terrible until I reached 1024.

The default value for it is actually 512 or 1456 for server 08 onwards. :-)

We have our value to set 8192 - Is there a reason why you cant go this high? I agree that the default is fairly slow from what I've seen.

If you can get it set to this value I think you will see a speed improvement assuming you have newish computer hardware and no other mitigating network constraints.

Hi 

thanks for all your input but we're getting away from my original question.

I want to understand why the TFTP isn't windowing. From server '08 onwards TFTP should send 4 blocks (window size of 4) before the client ACK's, instead of giving an ACK for every block. Given the round trip time between the DP and the clients is around 10ms increasing the windowing will significantly increase the speed.

Cheers

Alex

Hi 

thanks for all your input but we're getting away from my original question.

I want to understand why the TFTP isn't windowing. From server '08 onwards TFTP should send 4 blocks (window size of 4) before the client ACK's, instead of giving an ACK for every block. Given the round trip time between the DP and the clients is around 10ms increasing the windowing will significantly increase the speed.

Cheers

Alex

TFTP Window is not supported by the TFTP client used by the SCCM PXE client. This TFTP client only supports one block at a time.

Thanks for that Kerwin. Can you point to anywhere that this is documented?

It's not that I don't believe you, it's just that I've read so many articles that suggest increasing the Window size will improve the performance of WDS and SCCM OSD from a PXE boot. Many of them  have graphs showing significant improvment with block size increase!!

The Microsoft article would suggest that the Window Size options should be available in the latest implementations of PXE

http://technet.microsoft.com/en-us/library/hh974416.aspx

TFTP enhancements

What value does this change add?

Trivial File Transfer Protocol (TFTP) enhancements result in improved performance.

What works differently?

TFTP (Trivial File Transfer Protocol) has been enhanced and delivers improved results in performance.

You use the Windows Deployment Services Trivial File Transfer Protocol (TFTP) server to download the files that are needed to do a network boot using the Pre-Boot Execution Environment (PXE). PXE technology is a standard created by Intel that establishes a common and consistent set of pre-boot services within the boot firmware. The end goal is to enable a client to do a network boot and receive a network boot program (NBP) from a network boot server.

TFTP enhancements include:

• Scalable buffer management Provides support for a shared client buffer; allows buffering an entire file instead of a fixed size buffer for each client. Scalable TFTP buffer feature allows maintaining a single buffer per file in the server. When the server is buffering a file in shared mode, different sessions can read from the same shared buffer.
• Scalable port management Ability to use a dynamic or a fixed range of UDP ports to service clients with shared UDP port allocation. Sharing the same server port among different TFTP sessions improves scalability because there are sufficient ports when more clients are actively using the server.
• Variable-size transmission window Allows the client and server to determine the largest workable window size, resulting in improved TFTP performance. Provides the ability to dynamically determine the optimal window size.
• Maximum TFTP block size Previously implemented as a registry setting, this is now exposed to users through WDSUTIL and the WDS MMC snap-in.

• Edited by Mr P Wednesday, July 02, 2014 10:16 AM mis type

Thanks for that Kerwin. Can you point to anywhere that this is documented?

It's not that I don't believe you, it's just that I've read so many articles that suggest increasing the Window size will improve the performance of WDS and SCCM OSD from a PXE boot. Many of them  have graphs showing significant improvment with block size increase!!

I don't think this is documented anywhere.

SCCM has a TFTP client that is different from the TFTP client that comes with the Windows OS.

You will get performance increase by increasing the block size, but that is different from Windowing. The SCCM TFTP client can only handle one block at at time. The only way to improve the performance is to increase that block size.

Hi Kerwin

the exert above is taken from a document detailing improvements for WDS in Server 2012. The only time WDS uses TFTP is in the PXE phase. If you're saying that this isn't how TFTP works when the client is using the network boot program it's not really an improvement is it?

Cheers

Alex

Right I think I've found the answer, or least least found out why SCCM can't take advantage of the improvements in TFTP.

If you are using WDS PXEBoot.com will download the default.BCD store which will allow you to set the TFTP windowing size using BCDEdit

SCCM seems to do thing a little different. It creates a BCD store on the fly in RemoteInstall\SMSTemp. Looking at one of the BCD stores it doesn't seem to have the options for Block Size or Windowing.

The question is then is it possible to change the settings in these temporary BCD files that SCCM creates? The TFTP client is capable of using windowing (as explained in the WDS technet article above) but the BCD stores that SCCM creates don't set it.

By the way we can't increase the block size as the MTU of the network link is 1500

another update

Apparently was well as adding the 

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\SMS\DP\RamdiskTFTPBlockSize

key to increase the block size you can also add

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\SMS\DP\RamdiskTFTPWindowSize 

to increase the window size.

Someone has even created a tool to make the changes for you 

http://www.techygeekshome.co.uk/2014/05/dppxespeedup.html

I've not tried it yet though so I can't guarantee it'll work.

I'll post back with the results.

First, I already told you the above in my first reply.

Next, Kerwin specifically pointed out to you that OSD sets its own custom tftp client so the tftp client used by weds is irrelevant.

Last, know that Kerwin was the lead OSD dev and probably wrote the code being discussed here.

Hi Kerwin

the exert above is taken from a document detailing improvements for WDS in Server 2012. The only time WDS uses TFTP is in the PXE phase. If you're saying that this isn't how TFTP works when the client is using the network boot program it's not really an improvement is it?

Cheers

Alex

Hi Alex,

The WDS server supports Windowing. It is the SCCM TFTP client that does not support Windowing.

What will happen is that the WDS server will send multiple blocks. The SCCM TFTP client does not know that multiple blocks are being sent. The client continues to use the default incoming buffer size which is not enough to hold multiple blocks. Only the first few blocks will be processed (one-at-a-time). The last trailing blocks will be lost and the WDS server will have to resend them.

If the TFTP download speed is pretty bad in your environment and if you think that your network will benefit from TFTP Windowing, then please file a DCR.

Happy 4th...

Kerwin

• Marked as answer by Mr P Friday, July 11, 2014 9:22 AM

Hi Kerwin

I'm not questioning your expertise here I'm just trying to understand how things work. I'm like most techies I want to know what's going on.

From my understanding WDS and OSD use exactly the same Network Boot Program, which contains the TFTP client.  The only difference I can see between the two is the BCD.

What am I missing?

Anyway I'm going to give the registry Window size change an go and I'll post up the results good or bad.

Cheers

Alex

Hi

as promised reporting back....

I added the WindowSize key and it had no impact at all. All the blocks required an ACK before another was sent. Running procmon shows that the RamdiskTFTPBlockSize is looked for when SCCM is creating the BCD but there is no mention of the RamdiskTFTPWindowSize. The same goes for when you restart the WDS service, so I'm not sure if that key is valid at all. :-(

I'm sure I'm not the only person who has an offsite DP and limited to a 1500 MTU and it seems a shame not to leverage the latest updates to TFTP. could you point me in the right direction for logging a DCR?

Cheers

Alex

http://connect.Microsoft.com

Did you ever find a solution here or any feedback from Connect? I also am having this problem of being unable to set a WindowSize.

• Edited by bcehr Wednesday, December 10, 2014 8:01 PM
• Proposed as answer by _Pat Friday, December 12, 2014 2:33 PM
• Unproposed as answer by _Pat Friday, December 12, 2014 2:33 PM

http://blogs.technet.com/b/pingpawan/archive/2014/01/12/deep-dive-pxe-boot-flow-for-sccm-2007-2012.aspx

From the link we see the SCCM client process transfers the bootmgr.exe which TFTP loads the bcd then the Boot.wim

The bootmgr.exe does have the code for dealing with the "windowsize" TFTP option (look for the word "windowsize" with a hex editor)  but the client only proposes its use to the TFTP server if the corresponding parameter was added to the just retrieved BCD.

WDS uses an static BCD while SCCM uses a dynamically created one. It is \BIN\I386\smspxe.dll or \BIN\X64\smspxe.dll the component that creates the SCCM's BCD .
smspxe.dll  parses RamDiskTFTPBlockSize but it never parses RamdiskTFTPWindowSize.  I can clearly see the code that injects RamDiskTFTPBlockSize in the BCD but the code that injects RamdiskTFTPWindowSize is obviously missing.

I do not have a running SCCM handy right now then if you guys have a minute just try this experiment:
1) back-up your BIN\X64\smspxe.dll
2) open it with a hex editor like Hex Workshop (stop the corresponding service if locked)
3) search the "hexadecimal" sequence 07000035, (it should appear only once 5.0.7804.1000 ).
4) replace the byte "07" with "08" and save the change (it should look like 08000035)
5) Set the RamDiskTFTPBlockSize registry key to 8 or 4
6) Boot a client and a Wireshark capture will tell you if the TFTP transfer sends 8 or 4 consecutive blocks before to stop for an ACK.

What we just did was a dirty patch of the dll that will read RamDiskTFTPBlockSize from the registry but it should inject its value in the BCD as the corresponding RamdiskTFTPWindowSize parameter.
If it works the client will receive in the BCD RamdiskTFTPWindowSize=8 and itll take a default for the missing RamDiskTFTPBlockSize value.
If anything goes wrong just replace the patched dll with the back-up and delete the RamDiskTFTPBlockSize registry key. No big deal.

Let me know if it works. BTW using RamDiskTFTPBlockSize for improving speed is usually not good as it leads to IP fragmentation.

Best,
Patrick

Edit:
OK it works, then please consider this is just for testing purposes absolutely not for production. The idea was just to prove that the SCCM client TFTP windowsize capabilities "are" really there; it is eventually MS who should add the windowsize handling to smspxe.dll in the correct way.

Best,
Patrick

• Proposed as answer by bcehr Friday, December 12, 2014 4:42 PM
• Edited by _Pat Saturday, December 13, 2014 3:08 PM

I can confirm Patrick's solution works perfectly! If you are using ConfigMgr 2012 R2 (5.0.7958.1401), the Hex to search for is BA07000035 (still changing the 7 to an 8).  Making this change speeds up TFTP transfers by 3x+.  Thanks Patrick!

Brent can you confirm your values for RamDiskTFTPBlockSize and RamdiskTFTPWindowSize?

And the DLL in ConfigMgr 2012 R2 CU3 is at <letter>:\SMS_DP$\SMS\BIN\SMSPXE.DLL right?

Thanks!

Chris, yes, I have confirmed the values.

I ran bcdedit.exe /enum all /store X:\RemoteInstalSMSTemp\<file name of latest BCD file>.bcd.  This outputs the data from the ConfigMgr generated BCD file (this file name changes on each PXE boot, so there will be a new bcd file each time).  At the bottom of the output I see ramdisktftpwindowsize 8. There is no entry for ramdisktftpblocksize because the DLL change doesn't add a second entry to the BCD file, but instead causes the sms registry key to write a different BCD value (windowsize instead of blocksize).  However, in Server 2008 R2 and later, the default ramdisktftpblocksize is 1432, which combined with a windowsize of 8, is perfect and that value takes affect when ramdisktftpblocksize is not specified. (1432 is the biggest possible without IP Fragmentation.)

Correct, the DLL is X:\SMS_DP$\SMS\BIN\SMSPXE.DLL, except if PXE is on site server, then it is X:\Program Files\Microsoft Configuration Manager\bin\X64\SMSPXE.DLL.

Hope this answers your question.

sorry for bumping a thread this old, but i cant find any answers.

We are running configmgr 2007 R3. After migrating confmgr to co-location and wmvare hosts the tftp transfer takes forever.

Im trying to set RamDiskTFTPBlockSize to 1432 (thats what supported in vmware) in the registry and then I am changing smspxe.dll (X:\sms_ccm\x64\smspxe.dll) from BA07000035 to BA08000035 but it doesnt go any faster.

And if I check one of the autogenerated bcd files in smstemp it says this:

Setup Ramdisk Options
---------------------
identifier              {ramdiskoptions}
ramdisksdidevice        boot
ramdisksdipath          \SMSBoot\boot.sdi
ramdisktftpwindowsize   1432

Any help whit this?

(i have also tried to set blocksize to 8)

• Edited by solopcv 18 hours 26 minutes ago

sorry for bumping a thread this old, but i cant find any answers.

We are running configmgr 2007 R3. After migrating confmgr to co-location and wmvare hosts the tftp transfer takes forever.

Im trying to set RamDiskTFTPBlockSize to 1432 (thats what supported in vmware) in the registry and then I am changing smspxe.dll (X:\sms_ccm\x64\smspxe.dll) from BA07000035 to BA08000035 but it doesnt go any faster.

And if I check one of the autogenerated bcd files in smstemp it says this:

Setup Ramdisk Options
---------------------
identifier              {ramdiskoptions}
ramdisksdidevice        boot
ramdisksdipath          \SMSBoot\boot.sdi
ramdisktftpwindowsize   1432

Any help whit this?

(i have also tried to set blocksize to 8)

• Edited by solopcv Monday, September 07, 2015 1:16 PM

I haven't tested with ConfigMgr 2007, but I would recommend trying to set RamDiskTFTPBlockSize to 4 or 6, as that will set that WindowSize in the BCD file to 4 or 6. In our environment 8 gets even slower, 4 is most reliable and fast, 6 is slightly less reliable but fastest.