Questions upgrading from mongoose 3.8.x to 4.9.0

[xeroxoid]

Hi all.

It was high time we upgraded our mongoose version from 3.8.x to the latest version (4.9.0), so taking into account (and adapting) all breaking changes listed here and here we went ahead and updated our codebase.

After clearing some problems like incompatibilities with other libraries such as agenda we could finally start the server and everything looked fine at first.

Then we run our tests and many were failing...

Main question is why properties inside virtual getters now return

[ '$__', '_doc', 'linkedin', 'facebook' ] // '$__' and '_doc' were not there before

were as in mongoose 3.8.x it only returned

[ 'linkedin', 'facebook' ]

So this caused some issues in code such as the following:

UserSchema
.virtual('publicProfile')
.get(function () {
  return {
    '_id': this._id,
    'name': this.name,
    'avatar': this.avatar,
    'role': this.role,
    ... // ommited for simplicity
    'status': this.status || constants.STATUS.OFFLINE,
    'socialNetworks': _.compact(_.map(this.socialAccounts, function(accountData, accountName) { // Issue 1
      return !_.isEmpty(accountData) && accountName
    })),
    'donation': utils.isDeepEmpty(this.donation) ? undefined : this.donation, // Issue 2
  };
});

the utils.isDeepEmpty implementation is as follows:

function isDeepEmpty(obj) {
  var cleanObj = _.reject(obj, function (v) {
    return _.isFunction(v);
  });
  return _.reduce(cleanObj, function (r, v) {
    return r && (_.isObject(v) ? isDeepEmpty(v) : (_.isEmpty(v) && !_.isNumber(v)));
  }, true);
}

In issue 1, this.socialAccounts returns the extra _doc and $__ thus essentially returning an array with 2 extra items. The solution to that is to apply .toObject() to this.socialAccounts, but I do not seem to find why/how this changed and if it may affect other parts of the application, as I can not seem to find it documented.

Issue 2 is similar, donation is and object described in the schema as follows:

  donation: {
    donates: Boolean,
    percentage: Number,
    beneficiary: String,
    beneficiaryBankAccountDetails: {
      beneficiaryName: String,
      iban: String,
      accountNumber: String,
      bicOrSwift: String,
    },
    beneficiaryPayPalEmail: String
  }

thus we are checking if the object .isDeepEmpty() before returning it
The problem there again is that while with previous version _.rejecting just functions would suffice, we now have to obj.toObject() in order for it not to land in an endless recursion i.e. RangeError: Maximum call stack size exceeded.

We have patched the issues by using .toObject() were appropriate, but for reference I would like to know why this changed.

Thanks in advance and sorry for the long issue submission.

NodeJS: 4.2.6
Mongoose: 4.9.0
MongoDB: 3.4.2

[vkarpov15]

Sorry we missed this issue for so long. I don't quite recall why we made this change, we'll investigate making $__ and _doc non-enumerable for a future release. However, in general, if you're going to be iterating over object properties (_.map(), etc.) then I recommend you use .toObject() first, like _.map(this.socialAccounts.toObject()). Mongoose documents have a lot of internal state and that tends to confuse lodash.

[xeroxoid]

Thanks!