Skip to content

Commit d1c1dad

Browse files
jouni-kantolaJouni Kantola
authored andcommitted
Deterministic chunk hashes with Webpack 2
1 parent 1e91bcf commit d1c1dad

File tree

1 file changed

+115
-103
lines changed

1 file changed

+115
-103
lines changed

content/guides/caching.md

Lines changed: 115 additions & 103 deletions
Original file line numberDiff line numberDiff line change
@@ -3,118 +3,128 @@ title: Caching
33
sort: 16
44
contributors:
55
- okonet
6+
- jouni-kantola
67
---
78

89
To enable long-term caching of static resources produced by webpack:
910

1011
1. Use `[chunkhash]` to add a content-dependent cache-buster to each file.
11-
2. Use compiler stats to get the file names when requiring resources in HTML.
12-
3. Generate the chunk-manifest JSON and inline it into the HTML page before loading resources.
13-
4. Ensure that the entry point chunk containing the bootstrapping code doesn’t change its hash over time for the same set of dependencies.
12+
2. Extract the webpack manifest into a separate file.
13+
3. Ensure that the entry point chunk containing the bootstrapping code doesn’t change hash over time for the same set of dependencies.
14+
15+
For even more optimized setup:
16+
4. Use compiler stats to get the file names when requiring resources in HTML.
17+
5. Generate the chunk manifest JSON and inline it into the HTML page before loading resources.
18+
1419

1520
## The problem
1621

1722
Each time something needs to be updated in our code, it has to be re-deployed on the server and then re-downloaded by all clients. This is clearly inefficient since fetching resources over the network can be slow. This is why browsers cache static resources.
1823

19-
The way it works has a pitfall: if we don’t change filenames of our resources when deploying a new version, browser might think it hasn’t been updated and client will get a cached version of it.
24+
The way it works has a pitfall: If we don’t change filenames of our resources when deploying a new version, the browser might think it hasn’t been updated and client will get a cached version of it.
2025

21-
Probably the simplest way to tell the browser to download a newer version is to alter asset’s file name. In a pre-webpack era we used to add a build number to the filenames as a parameter and then increment it:
26+
A simple way to tell the browser to download a newer version is to alter the asset’s file name. In a pre-webpack era we used to add a build number to the filenames as a parameter and then increment it:
2227

2328
```bash
2429
application.js?build=1
2530
application.css?build=1
2631
```
2732

28-
It is even easier to do with webpack: each webpack build generates a unique hash which can be used to compose a filename. The following example config will generate 2 files (1 per entry point) with a hash in filenames:
33+
It is even easier to do with webpack. Each webpack build generates a unique hash which can be used to compose a filename, by including output [placeholders](/concepts/output/#options).
34+
The following example config will generate 2 files (1 per entry) with hashes in filenames:
2935

3036
```js
3137
// webpack.config.js
32-
const path = require('path');
38+
const path = require("path");
3339

3440
module.exports = {
3541
entry: {
36-
vendor: './src/vendor.js',
37-
main: './src/index.js'
42+
vendor: "./src/vendor.js",
43+
main: "./src/index.js"
3844
},
3945
output: {
40-
path: path.join(__dirname, 'build'),
41-
filename: '[name].[hash].js'
46+
path: path.join(__dirname, "build"),
47+
filename: "[name].[hash].js"
4248
}
4349
};
4450
```
4551

4652
Running webpack with this config will produce the following output:
4753

4854
```bash
49-
Hash: 55e783391098c2496a8f
50-
Version: webpack 1.10.1
51-
Time: 58ms
52-
Asset Size Chunks Chunk Names
53-
main.55e783391098c2496a8f.js 1.43 kB 0 [emitted] main
54-
vendor.55e783391098c2496a8f.js 1.43 kB 1 [emitted] vendor
55-
[0] ./src/index.js 46 bytes {0} [built]
56-
[0] ./src/vendor.js 40 bytes {1} [built]
55+
Hash: 2a6c1fee4b5b0d2c9285
56+
Version: webpack 2.2.0
57+
Time: 62ms
58+
Asset Size Chunks Chunk Names
59+
vendor.2a6c1fee4b5b0d2c9285.js 2.58 kB 0 [emitted] vendor
60+
main.2a6c1fee4b5b0d2c9285.js 2.57 kB 1 [emitted] main
61+
[0] ./src/index.js 63 bytes {1} [built]
62+
[1] ./src/vendor.js 63 bytes {0} [built]
5763
```
5864

59-
But the problem here is that, *each* time we create a new build, all filenames will get altered and clients will have to re-download the whole application code again. So how can we guarantee that clients always get the latest versions of assets without re-downloading all of them?
65+
But the problem here is, builds after *any file update* will update all filenames and clients will have to re-download all application code. So how can we guarantee that clients always get the latest versions of assets without re-downloading all of them?
6066

6167
## Generating unique hashes for each file
6268

63-
What if we could produce the same filename if the contents of the file did not change between builds? For example, the file where we put all our libraries and other vendor stuff does not change that often.
64-
65-
T> Separate your vendor and application code with [CommonsChunkPlugin](/plugins/commons-chunk-plugin) and create an explicit vendor chunk to prevent it from changing too often.
69+
What if we could produce the same filename, if the contents of the file did not change between builds? For example, it would be unnecessary to re-download a vendor file, when no dependencies have been updated, only application code.
6670

67-
Webpack allows you to generate hashes depending on the file contents. Here is the updated config:
71+
webpack allows you to generate hashes depending on file contents, by replacing the placeholder `[hash]` with `[chunkhash]`. Here is the updated config:
6872

69-
```js
70-
// webpack.config.js
73+
```diff
7174
module.exports = {
7275
/*...*/
7376
output: {
7477
/*...*/
75-
filename: '[name].[chunkhash].js'
78+
- filename: "[name].[hash].js"
79+
+ filename: "[name].[chunkhash].js"
7680
}
7781
};
7882
```
7983

8084
This config will also create 2 files, but in this case, each file will get its own unique hash.
8185

8286
```bash
83-
main.155567618f4367cd1cb8.js 1.43 kB 0 [emitted] main
84-
vendor.c2330c22cd2decb5da5a.js 1.43 kB 1 [emitted] vendor
87+
Hash: cfba4af36e2b11ef15db
88+
Version: webpack 2.2.0
89+
Time: 66ms
90+
Asset Size Chunks Chunk Names
91+
vendor.50cfb8f89ce2262e5325.js 2.58 kB 0 [emitted] vendor
92+
main.70b594fe8b07bcedaa98.js 2.57 kB 1 [emitted] main
93+
[0] ./src/index.js 63 bytes {1} [built]
94+
[1] ./src/vendor.js 63 bytes {0} [built]
8595
```
8696

8797
T> Don’t use [chunkhash] in development since this will increase compilation time. Separate development and production configs and use [name].js for development and [name].[chunkhash].js in production.
8898

89-
W> Due to this [issue in Webpack](https://github.com/webpack/webpack/issues/1315), this method of generating hashes still isn’t deterministic. To ensure hashes are generated based on the file contents, use [webpack-md5-hash plugin](https://github.com/erm0l0v/webpack-md5-hash). Here is an example how to integrate it into your project: https://github.com/okonet/webpack-long-term-cache-demo/pull/3/files
90-
9199
## Get filenames from webpack compilation stats
92100

93-
When working in development mode, you just reference a JavaScript file by entry point name in your HTML.
101+
When working in development mode, you just reference JavaScript files by entry point name in your HTML.
94102

95103
```html
104+
<script src="vendor.js"></script>
96105
<script src="main.js"></script>
97106
```
98107

99108
Although, each time we build for production, we’ll get different file names. Something, that looks like this:
100109

101110
```html
102-
<script src="main.155567618f4367cd1cb8.js"></script>
111+
<script src="vendor.50cfb8f89ce2262e5325.js"></script>
112+
<script src="main.70b594fe8b07bcedaa98.js"></script>
103113
```
104114

105-
In order to reference a correct file in the HTML, we’ll need some information about our build. This can be extracted from webpack compilation stats by using this simple plugin:
115+
In order to reference a correct file in the HTML, we’ll need information about our build. This can be extracted from webpack compilation stats by using this plugin:
106116

107117
```js
108118
// webpack.config.js
109-
const path = require('path');
119+
const path = require("path");
110120

111121
module.exports = {
112122
/*...*/
113123
plugins: [
114124
function() {
115125
this.plugin("done", function(stats) {
116126
require("fs").writeFileSync(
117-
path.join(__dirname, "", "stats.json"),
127+
path.join(__dirname, "build", "stats.json"),
118128
JSON.stringify(stats.toJson()));
119129
});
120130
}
@@ -127,7 +137,7 @@ Alternatively, just use one of these plugins to export JSON files:
127137
* https://www.npmjs.com/package/webpack-manifest-plugin
128138
* https://www.npmjs.com/package/assets-webpack-plugin
129139

130-
A sample output of webpack-manifest-plugin for our config looks like:
140+
A sample output when using `webpack-manifest-plugin` in our config looks like:
131141

132142
```json
133143
{
@@ -136,30 +146,36 @@ A sample output of webpack-manifest-plugin for our config looks like:
136146
}
137147
```
138148

139-
The rest depends on your server setup. There is a nice [walk through for Rails-based projects](http://clarkdave.net/2015/01/how-to-use-webpack-with-rails/#including-precompiled-assets-in-views). For serverside rendering in Node.js you can use [webpack-isomorphic-tools](https://github.com/halt-hammerzeit/webpack-isomorphic-tools). Or, if your application doesn’t rely on any server-side rendering, it’s often enough to generate a single `index.html` file for your application. To do so, just use these 2 amazing plugins:
149+
## Deterministic hashes
140150

141-
* https://github.com/ampedandwired/html-webpack-plugin
142-
* https://github.com/szrenwei/inline-manifest-webpack-plugin
151+
To minimize the size of generated files, webpack uses identifiers instead of module names. During compilation, identifiers are generated, mapped to chunk filenames and then put into a JavaScript object called *chunk manifest*.
152+
To generate identifiers that are preserved over builds, webpack supply the `NamedModulesPlugin` (recommended for development) and `HashedModuleIdsPlugin` (recommended for production).
143153

144-
It will simplify the setup dramatically.
154+
> TODO: When exist, link to `NamedModulesPlugin` and `HashedModuleIdsPlugin` docs pages
145155
146-
We’re done, you might think. Well, almost.
156+
> TODO: Describe how the option `recordsPath` option works
147157
148-
## Deterministic hashes
158+
The chunk manifest (along with bootstrapping/runtime code) is then placed into the entry chunk and it is crucial for webpack-packaged code to work.
149159

150-
To minimise the size of generated files, webpack uses identifiers instead of module names. During compilation identifiers are generated, mapped to chunk filenames and then put into a JavaScript object called *chunk manifest*. It is (along with some bootstrapping code) then placed into the entry chunk and it is crucial for webpack-packaged code to work.
160+
T> Separate your vendor and application code with [CommonsChunkPlugin](/plugins/commons-chunk-plugin) and create an explicit vendor chunk to prevent it from changing too often. When `CommonsChunkPlugin` is used, the runtime code is moved to the *last* common entry.
151161

152-
The problem with this is the same as before: whenever we change any part of the code, it will, even if the rest of its contents wasn’t altered, update our entry chunk to include the new manifest. This, in turn, will lead to a new hash and dismiss the long-term caching.
162+
The problem with this, is the same as before: Whenever we change any part of the code it will, even if the rest of its contents wasn’t altered, update our entry chunk to include the new manifest. This in turn, will lead to a new hash and dismiss the long-term caching.
153163

154-
To fix that, we should use [chunk-manifest-webpack-plugin](https://github.com/diurnalist/chunk-manifest-webpack-plugin) which will extract that manifest to a separate JSON file. Here is an updated webpack.config.js which will produce chunk-manifest.json in our build directory:
164+
To fix that, we should use [chunk-manifest-webpack-plugin](https://github.com/diurnalist/chunk-manifest-webpack-plugin), which will extract the manifest to a separate JSON file. This replaces the chunk manifest with a variable in the webpack runtime. But we can do even better; we can extract the runtime into a separate entry by using `CommonsChunkPlugin`. Here is an updated `webpack.config.js` which will produce the manifest and runtime files in our build directory:
155165

156166
```js
157167
// webpack.config.js
158-
var ChunkManifestPlugin = require('chunk-manifest-webpack-plugin');
168+
var ChunkManifestPlugin = require("chunk-manifest-webpack-plugin");
159169

160170
module.exports = {
161-
// your config values here
171+
/*...*/
162172
plugins: [
173+
/*...*/
174+
new webpack.optimize.CommonsChunkPlugin({
175+
name: ["vendor", "manifest"], // vendor libs + extracted manifest
176+
minChunks: Infinity,
177+
}),
178+
/*...*/
163179
new ChunkManifestPlugin({
164180
filename: "chunk-manifest.json",
165181
manifestVariable: "webpackManifest"
@@ -168,14 +184,14 @@ module.exports = {
168184
};
169185
```
170186

171-
Since we removed the manifest from entry chunk, now it’s our responsibility to provide webpack with it. You have probably noticed the `manifestVariable` option in the example above. This is the name of the global variable where webpack will look for the manifest JSON and this is why *it should be defined before we require our bundle in HTML*. This is easy as inlining the contents of the JSON in HTML. Our HTML head section should look like this:
187+
As we removed the manifest from the entry chunk, now it’s our responsibility to provide webpack with it. The `manifestVariable` option in the example above is the name of the global variable where webpack will look for the manifest JSON. This *should be defined before we require our bundle in HTML*. This is achieved by inlining the contents of the JSON in HTML. Our HTML head section should look like this:
172188

173189
```html
174190
<html>
175191
<head>
176192
<script>
177193
//<![CDATA[
178-
window.webpackManifest = {"0":"main.3d038f325b02fdee5724.js","1":"1.c4116058de00860e5aa8.js"}
194+
window.webpackManifest = {"0":"main.5f020f80c23aa50ebedf.js","1":"vendor.81adc64d405c8b218485.js"}
179195
//]]>
180196
</script>
181197
</head>
@@ -184,84 +200,80 @@ Since we removed the manifest from entry chunk, now it’s our responsibility to
184200
</html>
185201
```
186202

187-
So the final webpack.config.js should look like this:
203+
At the end of the day, the hashes for the files should be based on the file content. For this use [webpack-chunk-hash](https://github.com/alexindigo/webpack-chunk-hash) or [webpack-md5-hash](https://github.com/erm0l0v/webpack-md5-hash).
204+
205+
So the final `webpack.config.js` should look like this:
188206

189207
```js
190-
var path = require('path');
191-
var webpack = require('webpack');
192-
var ManifestPlugin = require('webpack-manifest-plugin');
193-
var ChunkManifestPlugin = require('chunk-manifest-webpack-plugin');
194-
var WebpackMd5Hash = require('webpack-md5-hash');
208+
var path = require("path");
209+
var webpack = require("webpack");
210+
var ChunkManifestPlugin = require("chunk-manifest-webpack-plugin");
211+
var WebpackChunkHash = require("webpack-chunk-hash");
195212

196213
module.exports = {
197214
entry: {
198-
vendor: './src/vendor.js',
199-
main: './src/index.js'
215+
vendor: "./src/vendor.js", // vendor reference file(s)
216+
main: "./src/index.js" // application code
200217
},
201218
output: {
202-
path: path.join(__dirname, 'build'),
203-
filename: '[name].[chunkhash].js',
204-
chunkFilename: '[name].[chunkhash].js'
219+
path: path.join(__dirname, "build"),
220+
filename: "[name].[chunkhash].js",
221+
chunkFilename: "[name].[chunkhash].js"
205222
},
206223
plugins: [
207224
new webpack.optimize.CommonsChunkPlugin({
208-
name: "vendor",
225+
name: ["vendor", "manifest"], // vendor libs + extracted manifest
209226
minChunks: Infinity,
210227
}),
211-
new WebpackMd5Hash(),
212-
new ManifestPlugin(),
228+
new webpack.HashedModuleIdsPlugin(),
229+
new WebpackChunkHash(),
213230
new ChunkManifestPlugin({
214231
filename: "chunk-manifest.json",
215232
manifestVariable: "webpackManifest"
216-
}),
217-
new webpack.optimize.OccurenceOrderPlugin()
233+
})
218234
]
219235
};
220236
```
221237

222-
T> If you're using [webpack-html-plugin](https://github.com/ampedandwired/html-webpack-plugin), you can use [inline-manifest-webpack-plugin](https://github.com/szrenwei/inline-manifest-webpack-plugin) to do this.
223-
224-
Using this config the vendor chunk should not be changing its hash unless you change its code or dependencies. Here is a sample output for 2 runs with `moduleB.js` being changed between the runs:
238+
Using this config the vendor chunk should not be changing its hash, unless you update its code or dependencies. Here is a sample output for 2 runs with `moduleB.js` being changed between the runs:
225239

226240
```bash
227-
> webpack
228-
229-
Hash: 92670583f688a262fdad
230-
Version: webpack 1.10.1
231-
Time: 65ms
232-
233-
Asset Size Chunks Chunk Names
234-
chunk-manifest.json 68 bytes [emitted]
235-
vendor.6d107863983028982ef4.js 3.71 kB 0 [emitted] vendor
236-
1.c4116058de00860e5aa8.js 107 bytes 1 [emitted]
237-
main.5e17f4dff47bc1a007c0.js 373 bytes 2 [emitted] main
238-
239-
[0] ./src/index.js 186 bytes {2} [built]
240-
[0] ./src/vendor.js 40 bytes {0} [built]
241-
[1] ./src/moduleA.js 28 bytes {2} [built]
242-
[2] ./src/moduleB.js 28 bytes {1} [built]
243-
244-
> webpack
245-
246-
Hash: a9ee1d1e46a538469d7f
247-
Version: webpack 1.10.1
248-
Time: 67ms
249-
250-
Asset Size Chunks Chunk Names
251-
chunk-manifest.json 68 bytes [emitted]
252-
vendor.6d107863983028982ef4.js 3.71 kB 0 [emitted] vendor
253-
1.2883246944b1147092b1.js 107 bytes 1 [emitted]
254-
main.5e17f4dff47bc1a007c0.js 373 bytes 2 [emitted] main
255-
256-
[0] ./src/index.js 186 bytes {2} [built]
257-
[0] ./src/vendor.js 40 bytes {0} [built]
258-
[1] ./src/moduleA.js 28 bytes {2} [built]
259-
[2] ./src/moduleB.js 28 bytes {1} [built]
241+
> node_modules/.bin/webpack
242+
243+
Hash: f0ae5bf7c6a1fd3b2127
244+
Version: webpack 2.2.0
245+
Time: 102ms
246+
Asset Size Chunks Chunk Names
247+
main.9ebe4bf7d99ffc17e75f.js 509 bytes 0, 2 [emitted] main
248+
vendor.81adc64d405c8b218485.js 159 bytes 1, 2 [emitted] vendor
249+
chunk-manifest.json 73 bytes [emitted]
250+
manifest.d41d8cd98f00b204e980.js 5.56 kB 2 [emitted] manifest
260251
```
252+
```bash
253+
> node_modules/.bin/webpack
254+
255+
Hash: b5fb8e138b039ab515f3
256+
Version: webpack 2.2.0
257+
Time: 87ms
258+
Asset Size Chunks Chunk Names
259+
main.5f020f80c23aa50ebedf.js 521 bytes 0, 2 [emitted] main
260+
vendor.81adc64d405c8b218485.js 159 bytes 1, 2 [emitted] vendor
261+
chunk-manifest.json 73 bytes [emitted]
262+
manifest.d41d8cd98f00b204e980.js 5.56 kB 2 [emitted] manifest
263+
```
264+
265+
Notice that **vendor chunk has the same filename**, and **so does the manifest** since we’ve extracted the manifest chunk!
266+
267+
## Manifest inlining
268+
269+
Inlining the chunk manifest and webpack runtime (to prevent extra HTTP requests), depends on your server setup. There is a nice [walkthrough for Rails-based projects](http://clarkdave.net/2015/01/how-to-use-webpack-with-rails/#including-precompiled-assets-in-views). For server-side rendering in Node.js you can use [webpack-isomorphic-tools](https://github.com/halt-hammerzeit/webpack-isomorphic-tools).
261270

262-
Notice that vendor chunk has the same filename!
271+
T> If your application doesn’t rely on any server-side rendering, it’s often enough to generate a single `index.html` file for your application. To do so, use i.e. [webpack-html-plugin](https://github.com/ampedandwired/html-webpack-plugin) in combination with [script-ext-html-webpack-plugin](https://github.com/numical/script-ext-html-webpack-plugin) or [inline-manifest-webpack-plugin](https://github.com/szrenwei/inline-manifest-webpack-plugin). It will simplify the setup dramatically.
263272

264273
## References
265274

266275
* https://medium.com/@okonetchnikov/long-term-caching-of-static-assets-with-webpack-1ecb139adb95#.vtwnssps4
267276
* https://gist.github.com/sokra/ff1b0290282bfa2c037bdb6dcca1a7aa
277+
* https://github.com/webpack/webpack/issues/1315
278+
* https://github.com/webpack/webpack.js.org/issues/652
279+
* https://presentations.survivejs.com/advanced-webpack/

0 commit comments

Comments
 (0)