I'm in the process of upgrading our domain infrastruture to 2008R2 from 2003. We have a child domain which replicates to the forest root using IPSEC encapsulation through a firewall. This was setup primarily based on this document, and has worked well in the years we've used it. I've now been setting up the same policy on the 2008R2 servers, and even though the policy is assigned, the R2 box will not connect to the root, and I cannot even ping let alone replicate. I've checked all firewall rules, and they are exactly correct. I even exported the policy from the 2003 server and imported it into the 2008R2 box (which worked ok), but still nothing. I've installed the MS IPSEC Diagnostic Tool, and when running the IKE debug, nothing is reported. I've also looked at the IP Security Monitor, and this shows no attempt at all to negotiate security, no IKE connections or anything. All I see is the Quick Mode 'Pending Key Operations' value increase, along with the 'Key Deletions' value. Normally, I'll see the 'Negotiating Security' message appear on a 2003 box if I'm ping'ing the root DC, but I don't see anything like this on the 2008R2 box. I've double and triple checked all settings, and they are correct as far as what works for 2003. Have I missed anything? Is there anything else you have to do in 2008 if assigning a security policy like this?

I've just put a network monitor on either machine at the end of the tunnel, and interestingly, can see IKE phase 1 traffic being sent between the 2008R2 and 2003 DC's. It appears they have negotiated phase 1, but I never see any phase 2 traffic. The NIC in the 2008R2 server is fully functional, but has all the TCP Offload/Receive Side scaling implemented. Could this cause any problems with IPSEC? The (older) 2003 hosts have much more basic NIC's.

Hi, Thank you for your post here. Can you verify the connectivity between two DCs if you configure the IPSec policy to secure other traffic such as RDP? If you cannot connect either, could you please search the Event log to find out any related IPSec errors? Could you please export the IPSec policies from both DC and paste it here (mask the IP address)?