Please provide oplock usage exmaple in next releases

[snarb]

There is no any kernel mode oplock code exmaple not in windows drivers sample, not on MSDN, not even sigle line of code in all github or internet.
Please provide oplock usage sample. Better for minifilter.

[cgallred]

Are you asking for examples of how a minifilter can provide oplocks, i.e. act like a file system? Or are you asking for examples of how a minifilter can use oplocks provided by an underlying file system?

[snarb]

More generic(and my case) is consuming oplock provided by an underlying file system. According to the: Granting Oplocks : FltFsControlFile and ZwFsControlFile can be used to request Windows 7 oplocks from kernel mode. And minifilters should use FltFsControlFile to get the oplock to the specific file. But it does not appear that FltFsControlFile is sufficient for obtaining oplocks. Because we can't provide complition callback to get notified on oplock break. So how to do this? Maybe we should use provided by the filter manager post-operation callback functionality? How to find that my oplock was breaked and not some-one other oplock? After searches... Maybe we should use
FltRequestOperationStatusCallback because as we can find in the Filter Manager Overview it is : Necessary if you need to know if an oplock was granted or not.
But how to use it if for this? Not clear..
I have asked on OSR - even for the most expirienced driver guys like Tony Mason this is not clear how to use FltFsControlFile with oplocks and to handle breaks. Maybe you can answer this, looks like you are the one who should know? I will be grateful.

In any case it would be greate to have oplock examples for minifilter devs.. This mechanism isn't similar to others. And such simple example would save a lot of time for devs.

IMHO good example will be something like if you will make Metadata Manager File System Minifilter Driver sample CSV-volumes aware with possible concurrent writes from different nodes during failover. This will be requare oplocks and will be generic example.

[cgallred]

I replied to the OSR thread. Unfortunately the MSDN documentation that refers to FltFsControlFile is incorrect. I've made a note to have it fixed.

Tony's answer is right; you need to use FltPerformAsynchronousIo to request oplocks from a minifilter, since as you've noticed you can't use FltFsControlFile for an asynchronous FSCTL.

[jmrossy]

Pavel, do you need additional clarification on this topic? Otherwise I'll close out this issue in a few days.

[snarb]

Thank you, Christian!
Now it is clear. Sure it will be more convinient to have some API for oplock consuming, but at least now all clear.

Nevertheless I wote up for some real oplock example, like syncronyzation of driver metadata on the CSV volume, maybe in the context of Metadata Manager File System Minifilter. I am assuming that many of CSV aware drivers will need to syncronyze their metadata and oplock usage is very different then other syncronyzation types. For driver devs it will be very good to have more advanced example then in Windows SDK, example with inter-nodes orchestration. Because this will simple save time for such devs and will help to not do errors. Also there are a lot of CSV specific nuances that are also not very documented.

Sure that is only my opinion.

[snarb]

More example of what qustions arise when triing to use oplocks(all operations are for the same thread):

• Why read break routine is called when requesting R oplock when having R oplock now (R->R) ?
  Oplocks Overview
  A Read (R) oplock (shared) indicates that there are multiple
  readers of a stream and no writers
  Breaking oplocks, IRP_MJ_READ
  Read.
  The oplock is not broken, no acknowledgment is required, and the operation
  proceeds immediately.
• Why for RW -> RW write break routing is called without REQUEST_OPLOCK_OUTPUT_FLAG_ACK_REQUIRED flag?
• RW(I am having now)->R( I want to have = downgrading): read break routine is called! And not write break as
  expected. Why so? And flags looks incorrect: OriginalOplockLevel = 0x1 (should
  be 5 = RW), NewOplockLevel = 0x1.
  Then FltPerformAsynchronousIo returns finally STATUS_OPLOCK_NOT_GRANTED status.
  An acknowledgment must be received before the operation continues.
  Breaking oplocks, IRP_MJ_READ

[cgallred]

I replied to the OSR thread. In short, I think it would help you to read the entire Oplock Semantics document starting at the Overview. So far you've only done things that are covered in the "Granting Oplocks" part of that doc, but you're incorrectly expecting behaviour from the "Breaking Oplocks" part.

[snarb]

CAllredMSFT,
Thanks. But if we can downgrade oplock, then orchestration looks unclear.
Let talk in the context of Metadata manager. We have metadata file.
We opening this file on two nodes A and B with R.
Then node A upgrades the oplock to RW and make changes in metadata file.
Oplock downgrade is not possible, so how Node B will read changed metadata? And node B can't request R oplock while there is RW oplock-
Granting Oplocks Read-Write, Read-Write-Handle: STATUS_OPLOCK_NOT_GRANTED is returned.. Vicious circle.
Maybe I can close file for this, but that is inconvinient and looks strange as for me.

An other crytical thing. I see that the only way for Node B to be notified when node A will finish the process of metadata file modification is to request RW oplock instead of R and wait for acknowledgment, but it nead only read access and other nodes should have read access at the same time and RW does not allow this.
Myabe I can request RW from B and after acknowledgment close file and request for the R and A should close file after sending acknowledgment and ask for R also. But all that looks like magic and workarounds.
Can you explaine plese what is the right strategy?

[cgallred]

You can't downgrade an oplock. As the oplock requester you can upgrade an oplock. An oplock only "downgrades" when it is broken by an I/O operation.

We opening this file on two nodes A and B with R.
Then node A upgrades the oplock to RW

Node A can't upgrade its oplock to RW if the file is open by somebody else at the same time. You can only have RW and RWH oplocks if your handle is the only one open to the file. Besides, if node A is updating the file directly (i.e. not locally caching the writes to send to the file later) it doesn't need an RW oplock.

I think you might be somewhat misunderstanding what oplocks are for. Oplocks are a notification and signalling mechanism, not an actual resource locking mechanism, designed for use in two scenarios. The first, and the reason they were originally invented, is for cache coherency. An example application is the SMB/CIFS server and client. The second scenario is when you want to be unobtrusive, i.e. you need to be informed when you should close your handle to get out of the way. An example application is the Windows search indexer.

Scenario 1 (Cache Coherency): The SMB client ("C1") requests an oplock that describes the kind of caching it wants to do. For example, it opens a file on the server and requests an RW oplock. This allows C1 to cache the data it reads from the server, and modify its cache without writing the changes back to the server, which reduces network traffic. If another client ("C2") opens the file on the server, then C1's RW oplock breaks to R and C2's open blocks. The oplock break tells C1 that it must write its cached changes back to the server, then acknowledge the oplock break. Acknowledging the break unblocks C2's open. C1 can continue to cache the data it read from the server. If C2 then writes to the file, C1's R oplock breaks to None. This tells C1 that its cache is no longer current, so it should discard the cache and re-fetch the data from the server. If C1 can't get an RW (or R) oplock when it opens the file, that means that it cannot cache but must read/write across the network directly on the server. The client periodically re-requests the oplock so that it can start caching again. As long as the oplock is not granted the client must send all its I/O across the network. Note that C1 is not directly notified when C2 is finished making its modifications. C1 has to periodically request an oplock until it gets one.

Scenario 2 (Unobtrusive Service): The second scenario is for a service such as the Windows indexer. The indexer opens files for read access to read and index the contents. If an application, for example Word, comes along and wants to write to the file, it is possible that the presence of the indexer would cause Word to get a sharing violation. The indexer wants to be invisible, so while it is reading it takes out an RH oplock on the file. When Word comes along its open causes a sharing violation. However because of the indexer's RH oplock the sharing violation is not returned to Word. Instead Word's open operation blocks and the indexer's RH oplock breaks. That tells the indexer that it should close its handle and try again later. When the indexer closes its handle that serves to acknowledge the RH oplock break. Word's open proceeds and it gets to use the file, blissfully unaware that something else was working on it.