MongoDB

Dealing With Unique Index of Mongoose and Mongodb

Mongoose provide a unique attribute for schema types. By setting it as true, you can easily create uniqe index for an attribute.

However there are some issues you must know.

Mongodb won’t ensure one item to be unique from others in the same array.

Say, you want to have a email list for your user schema, and you must ensure all members in the email list are unique from anyone else, whether they are in the same array or not. So you set this.

1
2
3
4
5
6
7
var userSchema = new Schema({
emailList: {
type: [String],
unique: true,
},
});
var User = Mongoose.model('User', userSchema);

And now, if you try this

1
2
3
4
var user1 = new User({emailList: ['john@doe.com', 'foo@bar.com']});
var user2 = new User({emailList: ['foo@bar.com']});
user1.save();
user2.save();

You’ll get an E11000 error as desired, everything seems to be fine. But try this,(If you don’t get this error please see section below)

1
2
var user3 = new User({emailList: ['john@doe.com', 'john@doe.com']});
user3.save();

It will pass the unique test… That’s definitely not what we want.

The solution is simple, create one your own validator. In this situation:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
var chkArrayDuplicate = {
validator: function (arr) {
var sorted = arr.slice();
sorted.sort();

var i;
for (i = 1; i < sorted.length; ++i) {
if (sorted[i] === sorted[i-1]) {
return false;
}
}
return true;
},
msg: 'Some items are duplicate'
};

And note that, emails are case-insensitive, so you should add a lowercase setter as well. Mongoose will apply setter first and then the validator, so that will be enough. It goes like below,

1
2
3
4
5
6
function arrToLowerCase(arr) {
arr.forEach(function (str, index, array) {
array[index] = str.toLowerCase();
});
return arr;
}

Sometimes unique index won’t work as you desired.

It’s normal and often that we’ll change our schema during the development & testing. Sometimes when we changed our schema and added the unique qualifier, Mongoose and Mongodb won’t reflect our changes. The unique index won’t be generated, thus you won’t ensure the uniqueness.

Sometimes, that happens when you have stored some duplicates in collections before.

But fix this problem won’t be as simple as droping the collections or the databases, or even you restart the mongod instance. Because the problem may lie in your codes.

You can first check your collections index, by this command in mongo shell, (I’ll use previous user collections as an example)

use your_testing_db
db.users.getIndexes()

If the index was created, then something terribly may have happened. Or most likely, you won’t have that index. You can add this listener to your model,

1
2
3
4
5
User.on('index', function (err) {
if (err) {
console.error(err);
}
});

This listener will listen the index event created by ensureIndex().

When your application starts up, Mongoose automatically calls ensureIndex for each defined index in your schema.

See Indexes and Model.ensureIndexes.

And keep in mind that, everything in Node.js are asynchronous, so the ensureIndex() is as well. So everytime when you tried to fix the problem, you first drop the collection or the database, and the index will also be dropped. And then you run your code again, node fired ensureIndex() and before it has done its job, save() got fired. So you’re messed up as well.

So the best you can do is to keep index created before you write some data to mongo. On production, you’d better create your collections on the database first.

When you test your code, better use another database. And because we need to constantly drop something when testing, you should always listen the index event and make sure you write something after that.

Empty array will be counted

That’s a weired feature of Mongodb, if you use unique index, you can’t have two documents have empty array. It will be count as duplicates, Even if you use sparse. You can check this SERVER-3934.

That’s it.

Adding Users for Mongodb

Today after messing around with MongoDB for hours. I finally figured out one thing… adding a user with password for a specifc database of MongoDB. :<

So I better write a blog to note this down…

My situation was simple, there is only one single instance. So I don’t need to touch those complex ‘replica set’ stuff.

So, few things to note:

First enable the auth mechanism, if you use the config file, modify it. By default, config lies in /etc/mongod.conf. Uncomment this line,

auth = true

Or you can start mongod with --auth option.

Then use mongo admin to connect to server and switch to the admin database, in where you’ll create the Admin user, and use this admin user to create a user for our db.

If you only want this user to have the minimum privileges to create other user. You can make its role as userAdminAnyDatabase;

However, this role is very limited. So for development convenience, I used root. So the commands are,

mongo admin

db.createUser( {
    user: 'userNameHere',
    pwd: 'passwordHere',
    roles: [
        { role: 'root', db: 'admin'}
    ]
})

If you got db.createUser() is not a function, please check your mongo’s version. This method was not introduced until 2.6. And you can use db.addUser() in previouse versions.

Then switch to the database you want, and create another user with readWrite role.

use yourDB

db.createUser( {
    user: 'userNameHere',
    pwd: 'passwordHere',
    roles: [
        { role: 'readWrite', db: 'yourDB'}
    ]
})

That’s it, by now you can use this user to connect mongo. You can test it by using db.auth('username', 'password')