Skip to content
This repository was archived by the owner on Sep 12, 2018. It is now read-only.

refactoring Rc->Arc#656

Closed
mmacedoeu wants to merge 2 commits into
mozilla:masterfrom
mmacedoeu:master
Closed

refactoring Rc->Arc#656
mmacedoeu wants to merge 2 commits into
mozilla:masterfrom
mmacedoeu:master

Conversation

@mmacedoeu

Copy link
Copy Markdown

Refactoring to Arc will get Send and ability for use on multi-threaded environments like webapps

@rnewman

rnewman commented Apr 24, 2018

Copy link
Copy Markdown
Collaborator

I agree that this needs to be addressed. However:

We almost certainly don't want to do all of these — e.g., query parser internals shouldn't escape across thread boundaries, so they don't need to be Arc.

This is also an opportunity to do better encapsulation; we shouldn't be leaking that strings and keywords are Rc or Arc. Laziness on my part!

More broadly, I'm not convinced that directly switching Rc to Arc in a bunch of places is the right thing to do: it's the easiest way to add that marker, yes, but at significant performance cost.

We use TypedValues in four main ways:

  • As part of query and transact parsing. These don't cross thread boundaries, and so don't need to be Send.
  • In the cache. This needs to be Send and/or Sync in order for stores to be shared across threads, but I think we really want stores to be Sync, because we realistically don't want to be passing stores by value.
  • In query results, which need to be handed back to callers.
  • In query inputs, which need to be passed from the caller into the query engine, where they will be cloned around and chewed on. (This line item generalizes to Add variable bindings to transact #655.)

The reason we introduced Rc in the first place was for the latter (see #395 / #397). We will also need cheap cloning/reuse of values in query results when I finish building pull (#638).

My suspicion is that we need to introduce some kind of context against which TypedValues are valid. We'd have one for QueryInputs, one for QueryResult, and one for AttributeCaches.

Comment thread sql/src/lib.rs
// in order to dedupe. We'll add these to the regular argument vector later.
byte_args: HashMap<Vec<u8>, String>, // From value to argument name.
string_args: HashMap<Rc<String>, String>, // From value to argument name.
string_args: HashMap<Arc<String>, String>, // From value to argument name.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see no reason why this would need to change. Did you just replace every Rc in the project, or did you make the changes necessary to pass some particular test?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some changes to Arc simplified the refactoring as most code with clone or cloned could remain the same instead of implementing some convert from Arc->Rc which would require to clone the inner value, so just cloning Arc potentially is cheaper then cloning the inner value into a new Rc.

@rnewman

rnewman commented Apr 25, 2018

Copy link
Copy Markdown
Collaborator

I think I managed to boil this PR down to a one-line change in types.rs.

I'd like to get a little better understanding of the implications before I make that one-line change, and some other work needs to happen before Mentat is truly usable across threads (#660, for example).

ncalexan added a commit to ncalexan/mentat that referenced this pull request May 3, 2018
)

@mmacedoeu did a good deal of work to show that Arc instead of Rc
wasn't too difficult in mozilla#656, and @rnewman pushed the refactoring
across the line in mozilla#659. However, we didn't flip the switch at that
time. For mozilla#673, we'd like to include TypedValue instances in errors,
and with error-chain (and failure) error types need to be 'Sync +
'Send, so we need Arc.

This builds on mozilla#659 and should also finish mozilla#656.
rnewman pushed a commit that referenced this pull request May 3, 2018
) r=rnewman

@mmacedoeu did a good deal of work to show that Arc instead of Rc
wasn't too difficult in #656, and @rnewman pushed the refactoring
across the line in #659. However, we didn't flip the switch at that
time. For #673, we'd like to include TypedValue instances in errors,
and with error-chain (and failure) error types need to be 'Sync +
'Send, so we need Arc.

This builds on #659 and should also finish #656.
@rnewman

rnewman commented May 9, 2018

Copy link
Copy Markdown
Collaborator

Fixed in #678.

@rnewman rnewman closed this May 9, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants