WebPack. Different assets’ hashes on different machines. The Problem. The Solution.

We found that when we started using a load balancer we met a very serious problem with our frontend assets.

We had different assets’s hashes for the same files for the same release but on different machines with the same OS.

That is how it looked.

On the picture above you can see that some files just have different hashes. But the content of the files is absolutely the same (see files’ size).

We had to find a reason. Some guys blamed WebPack. I wanted to figure out what exactly is wrong.

I found some relevant articles and an issue on github.

Chunk hash are not consistent on different machines · Issue #2215 · webpack/webpack

Long-term caching of static assets with Webpack

Avoid using webpack-md5-hash when using long-term caching

Some people think that his shit happens because WebPack does something wrong inside a method which calculates MD5 hash for modules during a compilation. Spoiler: It’s not a true. Everything is fine inside WebPack.

First of all I had to figure out why hashes are different if a content looks identical. I checked a source of files and yes, content was absolutely identical. But why hashes are different then?

I tried to install webpack-md5-hash how it was recommended in an artical above. This plugin is very simple. That is in short what it does :

compilation.plugin("chunk-hash", function(chunk, chunkHash) {
  ...
  var chunk_hash = md5(source)
  chunkHash.digest = function () { return chunk_hash }
})

So. I’ve found that this plugin uses source variable. It means I can install this plugin and check which difference in the source I have on different machines.

I used Mac OS and Debian. A teammate of mine helped me with testing.

We cleaned up our webpack.config.js and removed almost all entry points except the only which had a difference in hashes on different machines.

We saved source variable on Mac and on Debian. We found the following:

Oh! My! Looks like we found the issue. When we use JADE template engine require method uses an absolute path. Crazy thing! How is it possible?

But it was only the first step. After that on another file we found a similar thing.

3 Steps to Fix the Issue

Finally we found 3 reasons of our problem:

• JADE uses absolute paths in requires
• script-loader plugin uses absolute paths in requires
• expose-loader plugin uses absolute paths in requires

We even found some PRs for this issues. Unfortunately they were not merged.

Fix issue with absolute paths by jimedelstein · Pull Request #49 · webpack-contrib/script-loader

Fix: Absolute paths being used when expose-loader exposes more than one global for a library by…

There is an official recommendation about loaders writing.

webpack/docs

To fix the issue we had to:

• Update JADE to PUG
• Use patched script-loader plugin #patch
• Use patchedexpose-loader plugin #patch

The Story Is Not Ended Yet

We fixed a few most obvious cases, but that is not the end.

During the last check we found a few additional cases when hashes may be different.

This very odd case happens on different operation systems. Currently we think that when we use the same OS this problem doesn’t happen. Looks so, but I’m not 100% sure. Time will show how it goes.

A one more interesting thing here — we think this issue is happened after we replaced JADE with PUG.

One more problem we faced looks so:

I don’t know what may be wrong here. Probably something with source maps. I faced this problem once. I wanted to find a reason of this issue, but when I back to my investigation this issue is disappeared. Probably we updated something. I don’t know. But the issue just disappeared.

It’s ok that we don’t have it now, bit I keep it in mind and I have concerns about it.

What have I learned?

1. WebPack’s ecosystem has some plugins that provide problems. Some of them are not being updated for a long time, even if they are very popular and being at webpack-contrib repo.
2. If you want to write a loader you always MUST use relative path into a module code.
3. There are some undefined problems with WebPack and Plugins. These problems may affect assets’s hashes and it may be a critical problem for the production environment. That is why currently I have concerns about it.
4. WebPack has a bit odd way to define hashes for modules.

UPDATE: 2019 May

In another project I found that time to time linter’s rules are being changed. Spaces are being added and removed randomly after yarn install. Do you recognize a case from `The Story Is Not Ended Yet`? Looks like linter may be a potential cause of an issue. But this chance is extra small and we can just skip it.

A random change in the code. Time to time space is being added or removed.