can not use external volume

[mattma]

Several people reported this problem on Docker mongo repo.

I experience the same issue. I am running in vagrant latest with docker v1.5.0, my system is Linux ubuntu-14 3.13.0-24-generic #46-Ubuntu SMP Thu Apr 10 19:11:08 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux.

I downloaded the latest mongo build v3.0.0 via docker pull mongo. Then I run docker run -it -v ~/shared/mongodb/data:/data/db mongo:latest, I got an error below

2015-03-19T05:04:59.871+0000 I CONTROL  [initandlisten] MongoDB starting : pid=1 port=27017 dbpath=/data/db 64-bit host=715c517cfbb7
2015-03-19T05:04:59.872+0000 I CONTROL  [initandlisten]
2015-03-19T05:04:59.872+0000 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
2015-03-19T05:04:59.872+0000 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2015-03-19T05:04:59.872+0000 I CONTROL  [initandlisten]
2015-03-19T05:04:59.872+0000 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.
2015-03-19T05:04:59.872+0000 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2015-03-19T05:04:59.872+0000 I CONTROL  [initandlisten]
2015-03-19T05:04:59.872+0000 I CONTROL  [initandlisten] db version v3.0.0
2015-03-19T05:04:59.873+0000 I CONTROL  [initandlisten] git version: a841fd6394365954886924a35076691b4d149168
2015-03-19T05:04:59.873+0000 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.1e 11 Feb 2013
2015-03-19T05:04:59.873+0000 I CONTROL  [initandlisten] build info: Linux ip-10-171-101-151 3.2.0-4-amd64 #1 SMP Debian 3.2.46-1 x86_64 BOOST_LIB_VERSION=1_49
2015-03-19T05:04:59.873+0000 I CONTROL  [initandlisten] allocator: tcmalloc
2015-03-19T05:04:59.873+0000 I CONTROL  [initandlisten] options: {}
2015-03-19T05:04:59.887+0000 I STORAGE  [initandlisten] exception in initAndListen: 98 Unable to create/open lock file: /data/db/mongod.lock errno:13 Permission denied Is a mongod instance already running?, terminating
2015-03-19T05:04:59.887+0000 I CONTROL  [initandlisten] dbexit:  rc: 100

If I do not run with volume, it works fine.

I have talked to the MongoDB support tech support. They said the issue is You do not have the permission to write to volume "/data/db" inside container. It is definitely a permission issue. How could I solve smartly??

[niahmiah]

I have a similar problem with volumes running this on OS X via Kitematic:

2015-03-20T01:33:28.224+0000 I CONTROL [initandlisten] MongoDB starting : pid=1 port=27017 dbpath=/data/db 64-bit host=d392a5642411
2015-03-20T01:33:28.225+0000 I CONTROL [initandlisten]
2015-03-20T01:33:28.225+0000 I CONTROL [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
2015-03-20T01:33:28.225+0000 I CONTROL [initandlisten] ** We suggest setting it to 'never'
2015-03-20T01:33:28.225+0000 I CONTROL [initandlisten]
2015-03-20T01:33:28.225+0000 I CONTROL [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.
2015-03-20T01:33:28.225+0000 I CONTROL [initandlisten] ** We suggest setting it to 'never'
2015-03-20T01:33:28.225+0000 I CONTROL [initandlisten]
2015-03-20T01:33:28.225+0000 I CONTROL [initandlisten] db version v3.0.0
2015-03-20T01:33:28.225+0000 I CONTROL [initandlisten] git version: a841fd6394365954886924a35076691b4d149168
2015-03-20T01:33:28.225+0000 I CONTROL [initandlisten] OpenSSL version: OpenSSL 1.0.1e 11 Feb 2013
2015-03-20T01:33:28.225+0000 I CONTROL [initandlisten] build info: Linux ip-10-171-101-151 3.2.0-4-amd64 #1 SMP Debian 3.2.46-1 x86_64 BOOST_LIB_VERSION=1_49
2015-03-20T01:33:28.225+0000 I CONTROL [initandlisten] allocator: tcmalloc
2015-03-20T01:33:28.225+0000 I CONTROL [initandlisten] options: {}
2015-03-20T01:33:28.235+0000 I STORAGE [initandlisten] exception in initAndListen: 98 Unable to create/open lock file: /data/db/mongod.lock errno:13 Permission denied Is a mongod instance already running?, terminating
2015-03-20T01:33:28.235+0000 I CONTROL [initandlisten] dbexit: rc: 100

[cmoro-deusto]

Reproduces as well with docker 1.6.0 on OS X 10.10.3 (14D131) and Kitematic:

Kitematic Version 0.5.15

docker version
Client version: 1.6.0
Client API version: 1.18
Go version (client): go1.4.2
Git commit (client): 4749651
OS/Arch (client): darwin/amd64
Server version: 1.6.0
Server API version: 1.18
Go version (server): go1.4.2
Git commit (server): 4749651
OS/Arch (server): linux/amd64

docker images --digests=true
REPOSITORY          TAG                 DIGEST              IMAGE ID            CREATED             VIRTUAL SIZE
mongo               latest              <none>              be2c71632559        6 days ago          255.9 MB

Launched without volume

2015-04-17T09:25:33.274+0000 I JOURNAL  [initandlisten] journal dir=/data/db/journal
2015-04-17T09:25:33.275+0000 I JOURNAL  [initandlisten] recover : no journal files present, no recovery needed
2015-04-17T09:25:33.481+0000 I JOURNAL  [initandlisten] preallocateIsFaster=true 2.2
2015-04-17T09:25:33.607+0000 I JOURNAL  [durability] Durability thread started
2015-04-17T09:25:33.607+0000 I JOURNAL  [journal writer] Journal writer thread started
2015-04-17T09:25:33.610+0000 I CONTROL  [initandlisten] MongoDB starting : pid=1 port=27017 dbpath=/data/db 64-bit host=ceb569dd4784
2015-04-17T09:25:33.610+0000 I CONTROL  [initandlisten] 
2015-04-17T09:25:33.610+0000 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
2015-04-17T09:25:33.610+0000 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2015-04-17T09:25:33.610+0000 I CONTROL  [initandlisten] 
2015-04-17T09:25:33.610+0000 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.
2015-04-17T09:25:33.610+0000 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2015-04-17T09:25:33.610+0000 I CONTROL  [initandlisten] 
2015-04-17T09:25:33.610+0000 I CONTROL  [initandlisten] db version v3.0.2
2015-04-17T09:25:33.610+0000 I CONTROL  [initandlisten] git version: 6201872043ecbbc0a4cc169b5482dcf385fc464f
2015-04-17T09:25:33.610+0000 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.1e 11 Feb 2013
2015-04-17T09:25:33.610+0000 I CONTROL  [initandlisten] build info: Linux ip-10-171-120-232 3.2.0-4-amd64 #1 SMP Debian 3.2.46-1 x86_64 BOOST_LIB_VERSION=1_49
2015-04-17T09:25:33.610+0000 I CONTROL  [initandlisten] allocator: tcmalloc
2015-04-17T09:25:33.610+0000 I CONTROL  [initandlisten] options: {}
2015-04-17T09:25:33.611+0000 I INDEX    [initandlisten] allocating new ns file /data/db/local.ns, filling with zeroes...
2015-04-17T09:25:33.688+0000 I STORAGE  [FileAllocator] allocating new datafile /data/db/local.0, filling with zeroes...
2015-04-17T09:25:33.688+0000 I STORAGE  [FileAllocator] creating directory /data/db/_tmp
2015-04-17T09:25:33.693+0000 I STORAGE  [FileAllocator] done allocating datafile /data/db/local.0, size: 64MB,  took 0.001 secs
2015-04-17T09:25:33.700+0000 I NETWORK  [initandlisten] waiting for connections on port 27017

Launched with volume (mapped /data/db to directory inside /Users/)

2015-04-17T09:29:25.132+0000 I STORAGE  [initandlisten] exception in initAndListen: 98 Unable to create/open lock file: /data/db/mongod.lock errno:13 Permission denied Is a mongod instance already running?, terminating

[yosifkit]

For those using Kitematic, the issue is identical to boot2docker/boot2docker#581. I would assume that there is a similar issue when sharing host directories into a vagrant VM.

[cmoro-deusto]

@yosifkit you are totally right. It's a file ownership/permission issue (not related to this docker image), either using boot2docker with VB or a vagrant box with VB.

Nevertheless, I managed to hack the ownership, remounting the /Users shared volume inside boot2docker to uid 999 and gid 999 (which are what mongo docker image uses) and got it to start:

$ boot2docker ssh
$ sudo umount /Users
$ sudo mount -t vboxsf -o uid=999,gid=999 Users /Users

only to find that mongod crashes due to filesystem type not being supported (mmap not working on vboxsf).

I will just store the data inside the boot2docker vm image for development.

Thanks very much for your help.

PS: I log the start crash on vboxsf just in case somebody finds it useful

2015-04-20T10:07:49.017+0000 I JOURNAL  [initandlisten] journal dir=/data/db/journal
2015-04-20T10:07:49.019+0000 I JOURNAL  [initandlisten] recover : no journal files present, no recovery needed
2015-04-20T10:07:49.022+0000 I STORAGE  [initandlisten]     WARNING: This file system is not supported. For further information see:
2015-04-20T10:07:49.022+0000 I STORAGE  [initandlisten]             http://dochub.mongodb.org/core/unsupported-filesystems
2015-04-20T10:07:49.022+0000 I STORAGE  [initandlisten]         Please notify MongoDB, Inc. if an unlisted filesystem generated this warning.
2015-04-20T10:07:49.038+0000 I JOURNAL  [durability] Durability thread started
2015-04-20T10:07:49.038+0000 I JOURNAL  [journal writer] Journal writer thread started
2015-04-20T10:07:49.040+0000 I CONTROL  [initandlisten] MongoDB starting : pid=1 port=27017 dbpath=/data/db 64-bit host=74088274f847
2015-04-20T10:07:49.040+0000 I CONTROL  [initandlisten] 
2015-04-20T10:07:49.040+0000 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
2015-04-20T10:07:49.040+0000 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2015-04-20T10:07:49.040+0000 I CONTROL  [initandlisten] 
2015-04-20T10:07:49.040+0000 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.
2015-04-20T10:07:49.040+0000 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2015-04-20T10:07:49.040+0000 I CONTROL  [initandlisten] 
2015-04-20T10:07:49.040+0000 I CONTROL  [initandlisten] db version v3.0.2
2015-04-20T10:07:49.040+0000 I CONTROL  [initandlisten] git version: 6201872043ecbbc0a4cc169b5482dcf385fc464f
2015-04-20T10:07:49.040+0000 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.1e 11 Feb 2013
2015-04-20T10:07:49.040+0000 I CONTROL  [initandlisten] build info: Linux ip-10-171-120-232 3.2.0-4-amd64 #1 SMP Debian 3.2.46-1 x86_64 BOOST_LIB_VERSION=1_49
2015-04-20T10:07:49.040+0000 I CONTROL  [initandlisten] allocator: tcmalloc
2015-04-20T10:07:49.040+0000 I CONTROL  [initandlisten] options: { storage: { mmapv1: { smallFiles: true } } }
2015-04-20T10:07:49.043+0000 I INDEX    [initandlisten] allocating new ns file /data/db/local.ns, filling with zeroes...
2015-04-20T10:07:49.167+0000 E CONTROL  [initandlisten]   mmap() failed for /data/db/local.ns len:16777216 errno:22 Invalid argument
2015-04-20T10:07:49.167+0000 F INDEX    [initandlisten] error couldn't open file /data/db/local.ns terminating
2015-04-20T10:07:49.167+0000 I -        [initandlisten] Invariant failure false src/mongo/db/storage/mmap_v1/catalog/namespace_index.cpp 249
2015-04-20T10:07:49.173+0000 I CONTROL  [initandlisten] 
 0xf6b4b9 0xf0bcb1 0xef0762 0xd2390f 0xd4e91c 0xd51b7f 0x93e8b5 0x94b406 0x82307a 0x82516b 0x7f0794 0x7f1b5d189ead 0x821909
----- BEGIN BACKTRACE -----
{"backtrace":[{"b":"400000","o":"B6B4B9"},{"b":"400000","o":"B0BCB1"},{"b":"400000","o":"AF0762"},{"b":"400000","o":"92390F"},{"b":"400000","o":"94E91C"},{"b":"400000","o":"951B7F"},{"b":"400000","o":"53E8B5"},{"b":"400000","o":"54B406"},{"b":"400000","o":"42307A"},{"b":"400000","o":"42516B"},{"b":"400000","o":"3F0794"},{"b":"7F1B5D16B000","o":"1EEAD"},{"b":"400000","o":"421909"}],"processInfo":{ "mongodbVersion" : "3.0.2", "gitVersion" : "6201872043ecbbc0a4cc169b5482dcf385fc464f", "uname" : { "sysname" : "Linux", "release" : "3.18.11-tinycore64", "version" : "#1 SMP Thu Apr 16 17:46:31 UTC 2015", "machine" : "x86_64" }, "somap" : [ { "elfType" : 2, "b" : "400000", "buildId" : "FE06CE34D0A8EA7A70E5E8682BFCDB31D065F341" }, { "b" : "7FFD684B9000", "elfType" : 3, "buildId" : "96A32D413E7F959881EC4D55256BA257F953C228" }, { "b" : "7F1B5E6F9000", "path" : "/lib/x86_64-linux-gnu/libpthread.so.0", "elfType" : 3, "buildId" : "FEF281218797AD6AE726DD5FCEDECADD9E9F51DC" }, { "b" : "7F1B5E499000", "path" : "/usr/lib/x86_64-linux-gnu/libssl.so.1.0.0", "elfType" : 3, "buildId" : "AEE5F3A05E87AFA440FCF6352C568A0F08584119" }, { "b" : "7F1B5E0A1000", "path" : "/usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0", "elfType" : 3, "buildId" : "37084B8E55653C947BA6295814D850D6AA0C561D" }, { "b" : "7F1B5DE99000", "path" : "/lib/x86_64-linux-gnu/librt.so.1", "elfType" : 3, "buildId" : "F58D5DE3E7A2989E915422BA4203FE53DBA449A0" }, { "b" : "7F1B5DC95000", "path" : "/lib/x86_64-linux-gnu/libdl.so.2", "elfType" : 3, "buildId" : "5D1CA3A3D93ED5B6C6462FFA03E787FDBE4013A3" }, { "b" : "7F1B5D98E000", "path" : "/usr/lib/x86_64-linux-gnu/libstdc++.so.6", "elfType" : 3, "buildId" : "8711429397A5AF8B6269B867D830EDF6E0225B8D" }, { "b" : "7F1B5D70C000", "path" : "/lib/x86_64-linux-gnu/libm.so.6", "elfType" : 3, "buildId" : "7F58D6664571941C86B2D969701A572AD4D7BF1D" }, { "b" : "7F1B5D4F6000", "path" : "/lib/x86_64-linux-gnu/libgcc_s.so.1", "elfType" : 3, "buildId" : "F980B1188708F8D8B5C35D185444AF4CB939AA1E" }, { "b" : "7F1B5D16B000", "path" : "/lib/x86_64-linux-gnu/libc.so.6", "elfType" : 3, "buildId" : "A745EBA2C16BA80AE1EF1A7A7B70740C2CF1B363" }, { "b" : "7F1B5E915000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "9B23F2A44CC8CA6175CBD8D64584B1C7EA5FD18C" }, { "b" : "7F1B5CF54000", "path" : "/lib/x86_64-linux-gnu/libz.so.1", "elfType" : 3, "buildId" : "1EFEB71FD4999C2307570D673A724EA4E1D85267" } ] }}
 mongod(_ZN5mongo15printStackTraceERSo+0x29) [0xf6b4b9]
 mongod(_ZN5mongo10logContextEPKc+0xE1) [0xf0bcb1]
 mongod(_ZN5mongo15invariantFailedEPKcS1_j+0xB2) [0xef0762]
 mongod(_ZN5mongo14NamespaceIndex4initEPNS_16OperationContextE+0x5AF) [0xd2390f]
 mongod(_ZN5mongo26MMAPV1DatabaseCatalogEntryC1EPNS_16OperationContextERKNS_10StringDataES5_bb+0x13C) [0xd4e91c]
 mongod(_ZN5mongo12MMAPV1Engine23getDatabaseCatalogEntryEPNS_16OperationContextERKNS_10StringDataE+0x1AF) [0xd51b7f]
 mongod(_ZN5mongo14DatabaseHolder6openDbEPNS_16OperationContextERKNS_10StringDataEPb+0x105) [0x93e8b5]
 mongod(_ZN5mongo17AutoGetOrCreateDbC1EPNS_16OperationContextERKNS_10StringDataENS_8LockModeE+0x96) [0x94b406]
 mongod(+0x42307A) [0x82307a]
 mongod(_ZN5mongo13initAndListenEi+0x1A5B) [0x82516b]
 mongod(main+0x134) [0x7f0794]
 libc.so.6(__libc_start_main+0xFD) [0x7f1b5d189ead]
 mongod(+0x421909) [0x821909]
-----  END BACKTRACE  -----
2015-04-20T10:07:49.173+0000 I -        [initandlisten] 

***aborting after invariant() failure

[reharik]

Man I wish I had checked here first. I had these issues trying to use a data container with --volumes-from datacontainer. Only I had never done that before, or much of anything so I figured I was just doing it wrong. Well, now i know a lot more than I did.
My work around was to just create my own dockerfile from ubuntu and install mongo from apt-get.

[cosminonnet]

I have the same error trying to add a volume to the mongo container (I'm on OSX):

2015-05-22T10:13:34.375+0000 I STORAGE [initandlisten] exception in initAndListen: 98 Unable to create/open lock file: /data/db/mongod.lock errno:13 Permission denied Is a mongod instance already running?, terminating
2015-05-22T10:13:34.375+0000 I CONTROL [initandlisten] dbexit: rc: 100

Can you please provide a solution for this?
Thanks.

[PiTiLeZarD]

So after 2 days, I'm almost giving up here hehe

I'm on OSx with boot2docker and I tried all the workarounds I could find, namely:

• busybox (meh...)
• VBoxManage hack to mount a folder outside /Users with uid 999 in virtualbox
• Dockerfile with usermod -u 1000 mongodb
• all of those combined in different orders...

In my container I see the folder being mounted all fine, I can touch files and see it syncs up great. The mongod.lock is created... seems like it should work!

But: I'm stuck at:
exception in initAndListen: 13651 Couldn't fsync directory '/data/db': errno:22 Invalid argument, terminating

Which, unlike the message states, means that mongo will not fsync over vboxfs no matter how hard you try!

I'm left with very few solutions here and I'd be happy to hear some more:

• try mounting my folder with sshfs, apparently that would work somehow
• have a bigger virtualbox disk and not persisting my data
• ... After that I'm out, you have other workarounds?

So much pain and time wasted with those things...

[wolf0403]

/data/db/mongod.lock errno:13 Permission denied Is a mongod instance already running?, terminating

Discussion: boot2docker/boot2docker#581
Solution: change "useradd" line in mongo Dockerfile to

RUN groupadd -r mongodb && useradd -r -g mongodb -u 1000 mongodb

I'm hitting

E CONTROL [initandlisten] mmap() failed for /data/db/local.ns len:16777216 errno:22 Invalid argument

And is probably because of vboxsf bug https://www.virtualbox.org/ticket/819

Conclusion:
DO NOT try mounting OS X folders into boot2docker hosted Docker containers now, until sshfs / other workarounds available.

[yosifkit]

While changing the user id will not work for mongo because of the memory mapped files, we are looking into making each of the official image databases able to run as a different user id. I think the current idea is something like docker run -e GOSU_AS=1000:50 image....

[Hokutosei]

Greetings,

hi is there a workaround for this already? I'm having this issue, I just upgraded to 1.8 using boot2docker

2015-09-08T07:34:36.248Z I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=8G,session_max=20000,eviction=(threads_max=4),statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),
2015-09-08T07:34:36.262Z E STORAGE  [initandlisten] WiredTiger (22) [1441697676:262178][17:0x7f6b0172ebc0], connection: /data/db/WiredTiger.wt: fsync: Invalid argument
2015-09-08T07:34:36.263Z I -        [initandlisten] Fatal Assertion 28561
2015-09-08T07:34:36.263Z I -        [initandlisten]

***aborting after fassert() failure

-> % docker version
Client:
 Version:      1.8.1
 API version:  1.20
 Go version:   go1.4.2
 Git commit:   d12ea79
 Built:        Thu Aug 13 02:49:29 UTC 2015
 OS/Arch:      darwin/amd64

Server:
 Version:      1.8.1
 API version:  1.20
 Go version:   go1.4.2
 Git commit:   d12ea79
 Built:        Thu Aug 13 02:49:29 UTC 2015
 OS/Arch:      linux/amd64

docker run -d -e "REPL_SET=testa" -p 27021:27017 -v "$(pwd)"/data/:/data/db/ mongodb:3.1

[Hokutosei]

after docker-machine ssh dev (which is in virtualbox), then i tried running mongodb container on it, it worked.. I guess i have some issues mounting dir from osx..

[yosifkit]

@Hokutosei, yeah the mounting to OSX will not work for MongoDB because of the way that virtualbox shared folders work.