Recent Changes - Search:

edit SideBar

npm5 Considered Harmful

Summary

npm 5 deletes files in the node_modules directory.

This is a terrible design choice, a tool should NEVER delete files that the tool itself did not create.

Why does this matter

This matters because Accessors are using the node_modules directory in a somewhat novel manner.

In particular, in our source code repository, we check in a node_modules directory that contains required modules. We do this for the following reasons

  • Ease of development: make edits and the edits are automatically appear in the next invocation of node.
    • The worst alternative is to bump up the version number (as required by npm) and upload each change to the npm mothership
    • A different alternative is to have multiple copies of the module
  • Reliability
    • We want to be sure that the set of modules is the right set of modules and is available whether or not the user is connected to network

Example

Sadly, npm 5, which is what is shipped with Node.js 8.x has a feature that manages the node_modules directory by helpfully removing directories from it.

Below, I run npm install in web/hosts/node and the contents of node_modules is helpfully removed.

bash-3.2$ pwd
/Users/cxh/src/ptII11.0.devel/org/terraswarm/accessor/accessors/web/hosts/node
bash-3.2$ node --version
v8.4.0
bash-3.2$ npm --version
5.3.0
bash-3.2$ svn update
Updating '.':
At revision 2171.
bash-3.2$ $PTII/bin/npm install node-localstorage
/Users/cxh/src/ptII11.0.devel/bin/npm: Running /Users/cxh/src/ptII11.0.devel/vendors/node/node-v8.4.0-darwin-x64/bin/npm istall node-local storage
+ node-localstorage@1.3.0
removed 22 packages and updated 1 package in 0.875s
bash-3.2$ svn update
Updating '.':
Restored 'node_modules/ws'
Restored 'node_modules/ws/index.js'
Restored 'node_modules/ws/package.json'
Restored 'node_modules/ws/.travis.yml'
Restored 'node_modules/ws/lib'
Restored 'node_modules/ws/lib/WebSocketServer.js'
Restored 'node_modules/ws/lib/Extensions.js'
Restored 'node_modules/ws/lib/PerMessageDeflate.js'
Restored 'node_modules/ws/lib/Sender.js'
...

Unfortunately, this sort of lack of backward compatibility suggests that the npm development process is horribly broken and anyone with any sense on the team has taken the last bus for the coast and left. I've seen this in open source projects time and time again where crappy developers take over and people who know better leave.

One workaround is to use npm 4. I'll have to track down which version of Node includes npm 4.x. I'm not sure if we can use Node 6.11.3 LTS, I'll have to check.

I really cannot recommend Node 8.x at this time, and I question whether we should be using npm at all in the future, which means we should question using Node.

It looks like the workaround is to upload all of our packages to the npm host and install them. Sadly, npm uses yet another file called package-lock.json which is a database of what has been installed. In theory, I could look in to maintaining that file. Previously, it just worked to have directories in a node_modules directory. With npm 5.x, those days of milk and honey are over.

We had a reasonably nice working system and then someone made it "better" so that it removes our work. This is so wrong.

Root Cause Analysis

Clearly, this is the product of developers who have not used other peoples tools that much.

http://blog.npmjs.org/post/161081169345/v500 says

"This release marks months of hard work for the young, scrappy, and hungry CLI team, and includes some changes we’ve been hoping to do for literally years."

The key word is "young"

'nuf said.

What Should Have Happened

Removing Files

Never remove files that the new version of the tool did not place there.

If npm5 really wanted to remove files, then it should keep track of what files npm5 created and act accordingly. Keeping backward compatibility with npm4 is essential

Backward Compatibility

npm5 has a bunch of great and useful features.

npm5 should have used a directory other than node_modules and Node8.x should have been modified to look in that directory.

Workarounds

See Also

Edit - History - Print - Recent Changes - Search
Page last modified on October 06, 2017, at 09:02 AM