SMB2 Metadata Caching and Directory Change Notifications
I reposted my question here on the advice of Niki Han: http://social.msdn.microsoft.com/Forums/en-US/os_fileservices/thread/b08c890c-90d2-4e39-af18-051abdcf6f82
And again on the advice of Hongwei Sun: http://social.msdn.microsoft.com/Forums/en/windowssdk/thread/7d4bdddb-b9a8-4040-908e-39bafd4b0d3e
I have been researching SMB2 Metadata Caching for quite a while now. I work with networked multi-user software, so we require a high level of metadata consistency across clients. We have been working toward a solution that would not involve modifying
the workstation-wide cache settings to 0.
http://technet.microsoft.com/en-us/library/ff686200(v=ws.10).aspx
http://social.technet.microsoft.com/Forums/en-US/windowsserver2008r2networking/thread/2b4b34f3-b2e3-47f1-9859-b5fd7db36d3e
In my online research I have often seen suggested that programs register for Directory Change Notifications:
http://msdn.microsoft.com/en-us/library/aa365261(v=vs.85).aspx
http://blogs.msdn.com/b/dhawan/archive/2009/07/10/file-exists-access-getfileattributes-findfirstfile-findnextfile-stat-behavior-over-smb-2-0.aspx
It wasn't until I watched the channel9 presentation on SMB2 caching and reviewed another PowerPoint PDF that it became clear to me that our program needn't do anything with the notifications. Apparently it is enough for the metadata cache manager
to see a Directory Change Notification come down the wire.
http://channel9.msdn.com/posts/Windows-Client-Metadata-Caching-MS-SMB2-2010
http://www.snia.org/events/storage-developer2010/presentations/wednesday/DavidKruse_Analyzing_Metadata_Caching_Windows_SMB2_Client.pdf
So that was easy to implement, but the program behaved inconsistently. The Directory Change Notifications came down the wire exactly as they should, but the cache didn't always seem to be invalidated. Long story short, I discovered that if
I call FindFirstFile before every call to GetFileAttributes or _access, then those functions behaved consistently.
I believe from my observations that the three caches are not invalidated equally.
Contentions:
GetFileAttributes requests can be satisfied from either the Directory cache or the File Not File / File Info caches.
FindFirstFile requests can be satisfied from the Directory cache only. FindFirstFile only populates the Directory cache. GetFileAttributes populates both. Directory Change Notifications only invalidate the Directory cache.
I believe (5) to be a bug. Both the channel9 presentation and the SNIA presentation notes state that the Directory Change Notifications should invalidate all three caches, but this is not the behavior observed.
For example:
Register for Directory Change Notifications Call GetFileAttributes on a known file name that does not exist Signal, via an out-of-band mechanism, for the server or another client to create the file
Loop on GetFileAttributes waiting for a successful result - The Directory Change Notification is received after a few milliseconds - The loop continues for five seconds
However:
Register for Directory Change Notifications Call FindFirstFile on a known file name that does not exist Signal, via an out-of-band mechanism, for the server or another client to create the file
Loop on FindFirstFile waiting for a successful result - The Directory Change Notification is received after a few milliseconds - The loop continues until the notification is received
It gets even more wierd when the two functions are used in conjunction. It is possible to be in a state where FindFirstFile says that the file exists and GetFileAttributes says the file does not exist. I was able to duplicate this using a simple
program that employed a third-party computer's file share to signal the server to create the file. The results were not always consistent, but often looked like this:
Calling _access to populate the File Not Found cache.
Invoking server to create a file in our directory.
Beginning main loop...
Access: -1, FindFirst: 0
Access: -1, FindFirst: 1
Access: -1, FindFirst: 1
Access: -1, FindFirst: 1
Access: -1, FindFirst: 1
Access: -1, FindFirst: 1
Access: -1, FindFirst: 1
Access: -1, FindFirst: 1
Access: -1, FindFirst: 1
Access: -1, FindFirst: 1
Access: 0, FindFirst: 1
Invoking server to delete the file in our directory.
Test Finished.
The main loop waits 500 ms between iterations. FindFirstFile locates the created file after the first 500 ms, but _access requires 10 iterations which comes out to about 5 second (which is the default timeout of the File Not Found cache).
If I comment out the code that starts the Directory Change Notification thread. Then the results are consistent, but take longer to expire:
Calling _access to populate the File Not Found cache.
Invoking server to create a file in our directory.
Beginning main loop...
Access: -1, FindFirst: 0
Access: -1, FindFirst: 0
Access: -1, FindFirst: 0
Access: -1, FindFirst: 0
Access: -1, FindFirst: 0
Access: -1, FindFirst: 0
Access: -1, FindFirst: 0
Access: -1, FindFirst: 0
Access: -1, FindFirst: 0
Access: -1, FindFirst: 0
Access: -1, FindFirst: 0
Access: -1, FindFirst: 0
Access: -1, FindFirst: 0
Access: -1, FindFirst: 0
Access: -1, FindFirst: 0
Access: -1, FindFirst: 0
Access: -1, FindFirst: 0
Access: -1, FindFirst: 0
Access: -1, FindFirst: 0
Access: -1, FindFirst: 0
Access: 0, FindFirst: 1
Invoking server to delete the file in our directory.
Test Finished.
It takes 20 loops 500 ms apart to finally acknowledge the file. That's 10 seconds which is the timeout of the Directory cache. These two examples are meant to demonstrate that Directory Change Notifications do not flush the File Not Found cache.
Furthermore, it is possible to receive conflicting indications from FindFirstFile and _access.
Is there anyway to flush the File Not Found cache on receipt of Directory Change Notifications?
February 23rd, 2011 11:46pm