Mutagen - symlink issues

[matthew-gill]

• I have tried with the latest version of my channel (Stable or Edge)
• I have uploaded Diagnostics
• Diagnostics ID:

Expected behavior

File sharing resource can be marked as cached

Actual behavior

Error

Information

• macOS Version: 10.14.6 (18G103)

Trying out this new sync option as I've used Mutagen in the past with great results (👋@havoc-io)
When marking the path as cached, I simply see Error I can't find the logs anywhere.

I did some digging and it seemed as if within the Symfony commmunity it's been determined as symlink issues. When removing symlinks the problem goes away.

I know mutagen has the following: https://mutagen.io/documentation/synchronization/symbolic-links

How do we get this to work with symlinks?

[matthew-gill]

Screenshot:

[maniator]

Restart docker -- then it works (I had the same issue)

[xenoscopic]

Hey @matthew-gill! I'm not sure exactly what the configuration looks like for the Docker for Mac synchronization sessions, but it does sound as if it might be defaulting to the portable symlink mode and then encountering either a relative link that points outside of the synchronization root or an absolute link. Since this is only shipping with Docker for Mac, it might be possible to use the posix-raw symlink mode for these synchronization sessions, which should alleviate the problem.

[matthew-gill]

Restart docker -- then it works (I had the same issue)

no joy i'm afraid

[djs55]

Thanks for the report! I think @havoc-io 's is exactly right -- it's defaulting to the portable symlink mode. I've just created a development build which re-enables support for the global mutagen config file which hopefully could be used to change the symlink mode. If you want to try it the build is here: https://omakase.docker.paris/#/stage/mac/edge/45418 . Perhaps a global config file with:

sync:
  defaults:
    symlink:
      mode: "posix-raw"

Although we're not currently bundling the mutagen CLI (perhaps we should?) it's possible to use an existing one to talk to the daemon like this:

$ MUTAGEN_DISABLE_AUTOSTART=1 MUTAGEN_DATA_DIRECTORY=~/.docker/mutagen ./cmd/mutagen/mutagen list --long
--------------------------------------------------------------------------------
Name: desktop0175412122967d70d55c8cacc8c1b9cb
Identifier: sync_jxpSK8QElcUAzcwmkOFy3TnM6fhH6QnQ3bLOOaBz9u3
Labels: None
Configuration:
	Synchronization mode: Two Way Resolved
	Maximum allowed entry count: Default (2⁶⁴−1)
	Maximum staging file size: Default (18 EB)
	Symbolic link mode: Default (Portable)
	Ignore VCS mode: Default (Propagate)
	Ignores:
		DockerDesktop.vhdx
		Docker.raw
		Docker.qcow2
Alpha configuration:
	URL: /Users/djs/workspace/my-app
	Watch mode: Default (Portable)
	Watch polling interval: Default (10 seconds)
	Probe mode: Default (Probe)
	Scan mode: Default (Accelerated)
	Stage mode: Default (Mutagen Data Directory)
	File mode: 0644
	Directory mode: 0755
	Default file/directory owner: id:501
	Default file/directory group: id:20
Beta configuration:
	URL: ssh://desktop@localhost:desktop0175412122967d70d55c8cacc8c1b9cb
	Watch mode: Default (Portable)
	Watch polling interval: Default (10 seconds)
	Probe mode: Default (Probe)
	Scan mode: Default (Accelerated)
	Stage mode: Default (Mutagen Data Directory)
	File mode: 0644
	Directory mode: 0755
	Default file/directory owner: id:501
	Default file/directory group: id:20
Alpha:
	Connection state: Connected
Beta:
	Connection state: Connected
Status: Watching for changes
--------------------------------------------------------------------------------

(although note in the edge build it currently has v0.11.3 while in the build I linked above it's now v0.11.4)

[romaricdrigon]

Same issue here (Error in the GUI), and it may be the same cause.
I deleted all symlinks I could find, pruned data and restart Docker app, but I still have an error message. Is there any way to get some logs of what's happening?

[djs55]

I've made a further build which should change the default symlink mode to posix-raw (in addition to supporting the global mutagen config file): https://desktop-stage.docker.com/mac/edge/45494/Docker.dmg . If you have time to try it, let me know how it goes! (It may be necessary to turn the caching off, hit apply, then turn it on again)

[yano3nora]

I was getting same error, but #4595 (comment) build works for me now. thanks.

• macOS Version: 10.15.4 (19E287)

[romaricdrigon]

I've made a further build which should change the default symlink mode to posix-raw (in addition to supporting the global mutagen config file): https://desktop-stage.docker.com/mac/edge/45494/Docker.dmg . If you have time to try it, let me know how it goes! (It may be necessary to turn the caching off, hit apply, then turn it on again)

I just tried it, the Error is gone, thank you very much!
And Mutagen sync is definitely impressive performance-wise, that's really great.

[matthew-gill]

@djs55 yes this build seems to work, thank you!
My hesitation now is that it seems to take approx 10 seconds for local changes to be reflected in the container, which isn't the best for local development!

I think it would be worth enabling a mutagen config file in the root of the sync'd directory. For example, I'd probably opt to exclude vendor and node_modules from mutagen and maintain them as separate volumes within my container, I'd imagine that would speed up sync times.

When I used mutagen historically, it would normally take ~200ms for changes to be reflected.

Also, excluding any git repo from the sync should be the default behaviour if it's not already (https://mutagen.io/documentation/synchronization/version-control-systems)

[romaricdrigon]

@djs55 yes this build seems to work, thank you!
My hesitation now is that it seems to take approx 10 seconds for local changes to be reflected in the container, which isn't the best for local development!

Weird, I don't have the same issue 🤔

I think it would be worth enabling a mutagen config file in the root of the sync'd directory. For example, I'd probably opt to exclude vendor and node_modules from mutagen and maintain them as separate volumes within my container, I'd imagine that would speed up sync times.

Definitely! Configuring Mutagen caching via a mechanism similar to :delegated, plus being able to exclude some subfolders would be great. It is especially useful when doing IO intensive operations such as npm install, node_modules being usually huge.