copy-node-modules.sh fails on Linux during LMS provisioning due to permission error

[timmc-edx]

Developers with a previous devstack installation cannot re-provision due to file permission issues.

This has only been observed on Linux; it may or may not occur on Mac, since we've observed inconsistent behavior in whether files end up as root-owned there.

Acceptance criteria

• Short-term: Communicate this issue to devs so they know what to do if they run into it (add to troubleshooting guide, with link back to ticket so we know when to remove the section)
  • Docs change: docs: Add copy-node-modules.sh note to troubleshooting page #1148
• More permanently: Fix the script so that the copying happens as root (maybe a change to paver to explicitly call the copy script rather than having npm do it)
  • Fix npm installation (permissions, caching) openedx/openedx-platform#32946 -- the fix proposed here is to just delete the files as root before any copying is done

Symptoms

When the provisioning script runs npm ci, npm ends up running the postinstall script scripts/copy-node-modules.sh (as of PR openedx/openedx-platform#32767). On Linux, this fails with the following:

Copying studio-frontend JS & CSS from node_modules into vendor directories... 
+ read -r -d '' src_file
++ find node_modules/@edx/studio-frontend/dist -type f -print0
+ [[ node_modules/@edx/studio-frontend/dist/accessibilityPolicy.min.css = *.css ]]
+ cp --force node_modules/@edx/studio-frontend/dist/accessibilityPolicy.min.css common/static/common/css/vendor
cp: cannot remove 'common/static/common/css/vendor/accessibilityPolicy.min.css': Permission denied

Cause

The files in common/static/common/css/vendor are owned by the user root, whereas the script is running as the user app (as determined by adding a call to whoami.) This still happens if I just run npm install from inside make lms-shell -- even though I'm running that command as root, by default.

npm performs this change-of-user for all of its scripts, with no apparent workaround:

When npm is run as root, scripts are always run with the effective uid and gid of the working directory owner.

As far as I can tell, there's no workaround (see npm/rfcs#546) so maybe we can just switch to calling the script explicitly after the npm ci call completes.

Workaround

In edx-platform:

sudo rm -rf common/static/common/css/
sudo rm -rf common/static/common/js/

(More thorough version that will also clear out root-owned pycache files that keep cropping up: sudo find -user 0 -group 0 -prune -exec rm -rf '{}' \;)

Once this is done, npm install inside lms-shell will succeed, and the re-created files in those directories will have the UID and GID of the desktop user.

[timmc-edx]

Should be able to reproduce the issue this way:

• sudo chown -R root: common/static/common/css/ in edx-platform to simulate lingering root-owned files
• Add a newline to the front of package.json in edx-platform to bust the cache
• make dev.destroy dev.pull.lms dev.provision