Long term proposal for ObjectId coercion

[superkhau]

At the moment, we do not support ObjectId in model definitions. I am creating this issue to gather some opinions on how to move forward for a bigger fix than the one at loopbackio/loopback-datasource-juggler#778.

[superkhau]

The aforementioned fix depends on the native mongodb driver's ObjectId implementation (simply delegates to it for the time being). The following are some interesting items:

Dependency on the the native mongodb impl

@bajtos @ritch and @superkhau do not like this solution for the long term. My suggestion is to probably use our own class impl that uses the strategy pattern to allow swappable ObjectId impl's. If none is specified, it will fall back to our own default impl, which is probably going to be a simple subset of Mongo's impl.

@ritch suggested that we have a registry for datatypes

See https://github.com/strongloop-internal/scrum-loopback/issues/534.

Basically, we should have an app level registry for all datatypes that are currently valid in the system. It should be updateable during runtime. Something along the lines of:

app.registry.dataTypes = [
  'String',
  'ObjectId',
   ...
];

With regards to impl, I'm not sure how to approach this yet. Maybe we can have LoopBack boot register dataTypes based on the ones read in from model definitions at startup. Ultimately though, I think Juggler should be the one with the main list of data types and the app level is just a link to the one maintained in Juggler.

Juggler.dataTypes = [
  ...
];

app.registry.dataTypes = Juggler.dataTypes;

[superkhau]

How to persist the data consistently across SQL and NoSQL databases

This is an interesting question because we need to support persistence in both types of databases. For NoSQL dbs that support ObjectIds, the solution is simple. For SQL databases, do we persist them as String PKs when marshalling/unmarshalling or do we use another format?

[superkhau]

@bajtos @ritch @raymondfeng Please add your insights (especially @raymondfeng since you know database architecture/design pretty well).

[superkhau]

We should allow the custom ObjectId type to be required for typeof/instanceof of comparison

This is pretty simple in that we can just do something like:

module.exports = ObjectId;
exports.ObjectId = ObjectId;
exports.ObjectID = ObjectId;

function ObjectId {
}

Note we should also alias Id and ID defensively (saw this in mongo's native impl).

[superkhau]

How do we handle relations with ObjectId

This is interesting and probably dependent on how we choose to serialize/deserialize the ObjectId type. We could do deep comparison or simple string comparison via the hex string, etc. Not sure what is the best route here. In addition, do we support multi-level relations, etc.

[superkhau]

Ultimately, I think the long term proposal would involve this major items:

• Create a registry in Juggler
• Expose the registry in the app object which delegates to the registry in Juggler
• Add impl to Boot in order to populate registry with data types based on model definitions
• Create ObjectId class impl
  • Figure out algo for hex strings
  • Figure out how to make them unique (timestamp + algo)
• Impl functions for adding and removing data types during runtime (ie. reg.remove/reg.add)
• Figure out how relations will work

[bajtos]

One more thing to consider: how to support models with ObjectId properties when running in a browser, where we don't have any mongodb connector?

1. We should not include the full mongodb module in the browser bundle, this is something we should probably fix in the current short-term fix ASAP.

2. Since there is no mongodb module in the browser that could contribute the ObjectId type, we don't have any other choice than to implement our own ObjectId implementation that would be mapped to MongoDB's ObjectId by the MongoDB connector.

That way it should be reasonable easy to implement custom mapping for SQL and foreign keys too.

Thoughts?

[superkhau]

1. We should not include the full mongodb module in the browser bundle, this is something we should probably fix in the current short-term fix ASAP.

How do we fix this? Come up with our own impl now?

2. Since there is no mongodb module in the browser that could contribute the ObjectId type, we don't have any other choice than to implement our own ObjectId implementation that would be mapped to MongoDB's ObjectId by the MongoDB connector.

Not sure how to do this in the browser. It sounds like if the browser is a requirement, we need to impl our own ObjectId type that is not dependent on the mongodb module. The short term-fix doesn't take this use case into consideration at all since it just uses the mongodb module directly. Is the browser requirement a must have at this point in time? Or can it be patched in a future update?

[bajtos]

Not sure how to do this in the browser. It sounds like if the browser is a requirement, we need to impl our own ObjectId type that is not dependent on the mongodb module. The short term-fix doesn't take this use case into consideration at all since it just uses the mongodb module directly. Is the browser requirement a must have at this point in time? Or can it be patched in a future update?

We must allow users to write universal/isomorphic LoopBack apps that can run in browser (backed by localStorage/indexedDB) and on the server (backed e.g. by MongoDB).

At the moment, we require sync-ed models to use id: { type: 'string', defaultFn: 'guid' } and it seems to work reasonably well. So for short-term, all we need is to exclude mongodb from the browser bundle, e.g. by providing a browser-specific implementation of ObjectID that can e.g. throw an exception in the constructor. We already do something similar for depd, see https://github.com/strongloop/loopback-datasource-juggler/blob/5cf37aae5b217a92ed213c89884156dd42c1c0df/package.json#L22-L24

I don't know what are the implications of using GUID-like string ids in models stored in MongoDB. If using ObjectID instead of string would be more efficient, or perhaps if using ObjectId instead of string in browser models would make things easier, then I think we should create a new user story to investigate these options.

It makes me really wonder: what do we actually need from ObjectId, besides the type information that it's a string in a specific format? Why can't we simply implement our ObjectId as a thin wrapper around string with a bit of extra validation?

[bajtos]

Possibly related: #1994

I realized that Loopback use toString to "hide" objects, which makes it very complicated to debug/understand what is happening. He are some examples:

console.log(accessToken.uid) gonna display MyUserId. One might think that accessToken.uid is a string when it actually is an object using toString() to return the actual value, which makes the following fail: accessToken.uid === "MyUserId"