Calling populate on embedded doc of embedded doc

[dmmalam]

Hey,
I have the following schemas ( simplified for issue)

User = new Schema
login:String

A = new Schema
emb: [B]
comments: [Comments]

//and following embedded schemas

B = new Schema
comments: [Comments]

Comments = new Schema
creator:
type: ObjectId
ref: 'User'
message: String

Now while populate works on A
eg A.find().populate('comments.creator')

it doesn't work on the double nested embedded doc
eg A.find().populate('emb.comments.creator')

Any ideas?

[eneko]

The way I am doing that currently is populating the nested items and then iterating through them to populate the children. I was planing on implementing a solution for populating multiple nested levels, but it get very tricky.

[paulshapiro]

Hi eneko,

How are you doing that?

When you do your first populate, you get returned the nested items but when you run through them to populate their children, are you able to save the populated children? It never sticks when I try to do it.

Some code would be awesome.

Thanks,
Paul

[paulshapiro]

@eneko nevermind, just needed .toObject() ;)

[eneko]

Actually Paul, I figured populate() works on queries that return multiple elements, so there is no need to iterate through the child objects. But yes, you are right, you need to call .toObject if you want the properties to stick.

Here is an example:

// Ideally should be Parent.findOne().populate('children').populate('children.grandchildren').run();

function loadParentWithChildrenAndGrandChildren(parentId) {

  // Load parent without children references (children is array of ObjectId)
  Parent.findOne({ id: parentId }, { children: 0 }, function(err, parent) {
    if (err || !parent) return next(new Error("Parent not found: " + parentId));

    // Load children for this parent, populating grandchildren (no need to load parent reference)
    Children.find({ parent: parent._id }, { parent: 0 })
      .populate('grandchildren', [], { })
      .run(function(err, children) {
        if (err) return next(new Error("Could not load children: " + parentId));

        var result = parent.toObject();
        result.children = children;
        next(null, result);
      });
  });

}

[paulshapiro]

Ah, thanks for your detailed answer. Your mongoose calls are more specific than mine and I could learn from your approach. As it happens my particular application required a recursive populate, so for now I may be stuck doing this manually.

I end up doing something like this.

var viewWithId_forDisplay = function(id, callback) {
if ( !id || typeof id === 'undefined' || id.toString().match(app_utils.emptyReg) ) {
    throw new Error('Bad user ID.');
}

View.findById(id)
.run(function(err, obj) {
    if ( err || !obj || !obj.subviews || !obj.subviews.length ) return callback(err, obj);

    obj = obj.toObject();

    // recursive subview fetch
    async.map( obj.subviews, viewWithId_forDisplay, function(err, results) {
        obj.subviews = results;

        return callback(err, obj);
    } );

})
}

[dmmalam]

Does this now work in the latest mongoose release?

[dboundz]

So I think this is the issue I'm experiencing. Each Comment object has an embedded User object. Comments are embedded as an array in Activity. A query for activities (as below) does not populate the User object on each comment.

var get_activities = function (callback) {
  var _this = this
  Activity.find({}, [], {sort:{ _id: -1 }})
  .populate('user')
  .populate('comments')
  .run(function (error, activities) {
    callback(error, activities)
  })
}

var User = new Schema({
    id: { type: String, required: true, lowercase: true, index: { unique: true } }
  , email_address: { type: String, required: true, index: true }
  , name: { type: String, required: true }
  , first_name: { type: String,  required: true } 
  , last_name: { type: String, required: true }
  , user_name : { type: String, required: true, lowercase: true, index: { unique: true } }
  , avatar_url: { type: String, required: true }
  , bio: String
  , following: [{ type: Schema.ObjectId, ref: 'User', index: { unique: true } }]
  , followers: [{ type: Schema.ObjectId, ref: 'User', index: { unique: true } }]
})

var Activity = new Schema({
    id: { type: Number, required: true, index: { unique: true } }
  , user: { type: Schema.ObjectId, ref: 'User', required: true }
  , recipients: [{ type: String, index: { unique: true } }]
  , type: String  
  , body: { type: String, required: true }
  , timestamp: { type: Date, required: true }
  , likes: [{ type: Schema.ObjectId, ref: 'User', index: { unique: true } }]
  , comments: [{ type: Schema.ObjectId, ref: 'Comment' }]
})

var Comment = new Schema({
    id: { type: String, required: true, index: { unique: true } }
  , timestamp: { type: Date, required: true }
  , body: { type: String, required: true }
  , user: { type: Schema.ObjectId, ref: 'User', required: true } 
}) 

[dethe]

Is there any plan for populate to support nested paths to handle cases like this?

[aheckmann]

yes we'd like to at some point. it could quickly become a performance issue but yeah we should support it.

[dethe]

For nested objects I need to iterate over them manually, loading them, which seems like more of a performance issue and clutters up my server code. So I'm +1 for making my job easier ;-)

In the meantime I will continue, and also address it by splitting sections of the page into separately loadable objects that don't require such deeply nested data on each call.

[Qard]

I'd be all for something like this;

Parent.findById(1).populate('child').run(function (err, parent) {
  parent.child.populate('grandchild', function (err) {
    console.log(parent.child.grandchild)
  })
})

It'd actually be pretty handy to have populate available on the document like that. You could keep diving as far as you need.

[aheckmann]

@Qard you read my mind.

[eneko]

@Qard that is how mongoose-relationships works. I used that module for a while at first, but ended up doing things manually. Maybe Mongoose should integrate it into the core, or at least parts of it like calling populate on arrays.