Skip to content

Reshape block interfaces for a "pull" model#53

Closed
pfmooney wants to merge 3 commits into
oxidecomputer:masterfrom
pfmooney:block-reshape
Closed

Reshape block interfaces for a "pull" model#53
pfmooney wants to merge 3 commits into
oxidecomputer:masterfrom
pfmooney:block-reshape

Conversation

@pfmooney

@pfmooney pfmooney commented Oct 7, 2021

Copy link
Copy Markdown
Contributor

This temporarily excludes the NVMe emulation from the build as parallel
efforts to update its internals are underway.

This temporarily excludes the NVMe emulation from the build as parallel
efforts to update its internals are underway.

@jmpesp jmpesp left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 plus a few comments

Comment thread propolis/src/block/mod.rs Outdated
Comment thread propolis/src/block/file.rs Outdated
Comment thread server/src/lib/config.rs Outdated
Comment thread standalone/src/config.rs Outdated
use tokio::sync::Semaphore;

// XXX: completely arb for now
const MAX_WORKERS: usize = 32;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add as an optional paramter in the toml for block dev nodes?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps the configured worker count, but probably not the max

@pfmooney

Copy link
Copy Markdown
Contributor Author

Merged in e8db4b0

@pfmooney pfmooney closed this Oct 11, 2021
@pfmooney pfmooney deleted the block-reshape branch October 11, 2021 16:24
luqmana added a commit that referenced this pull request Oct 13, 2021
This updates the NVMe device to use the new pull based model from #53.

We had to drop namespace support for now as the design there complicates the IO dispatching: the new model is that the block device pulls IO requests from the NVMe Submission Queues. Multiple namespaces may use the same Submission Queues and hence we can't simply associate a particular block device with a given namespace without some intermediate buffering. Buffering we'd rather avoid to simplify the live migration story by trying to keep as much device state in guest memory as possible.

Some other changes here include making pushing to the Completion Queues infallible. We do this by returning a new "permit" type whenever we successfully pop a command off the Submission Queue. The permit's existence guarantees there's at least one space available on the Completion Queue, hence reserving it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants