Skip to content

Commit a24289e

Browse files
[Docs] - How Flutter implements golden tests (Resolves #45) (#50)
1 parent 79d403d commit a24289e

File tree

17 files changed

+1080
-34
lines changed

17 files changed

+1080
-34
lines changed

doc/website/source/_data.yaml

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,10 @@ navigation:
4141
tag: scenes
4242
sortBy: navOrder
4343

44-
- title: Golden Metadata
45-
tag: metadata
44+
- title: How it Works
45+
tag: how-it-works
4646
sortBy: navOrder
47+
48+
- title: Flutter's Implementation
49+
tag: flutter-implementation
50+
sortBy: navOrder

doc/website/source/golden-metadata/_data.yaml

Lines changed: 0 additions & 1 deletion
This file was deleted.

doc/website/source/golden-metadata/payload.md

Lines changed: 0 additions & 19 deletions
This file was deleted.
Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
---
2+
title: Failures
3+
navOrder: 40
4+
---
5+
`flutter_test_goldens` improves the review experience for failing golden tests.
6+
7+
Golden failures typically generate four files: the golden image, the new screenshot, an isolated
8+
diff, and a masked diff. These images are useful, but viewing them across four files is tedious.
9+
10+
Instead of generating four separate image files upon failure, `flutter_test_goldens` paints
11+
all four of those failure images as a single image, making it easy to view side-by-side.
12+
13+
Moreover, if a Golden Scene contains more than one golden image, all failure images for all
14+
goldens in the scene, are painted to the same image. This provides the most holistic view
15+
possible for a reviewing developer.
16+
17+
TODO: show examples
Lines changed: 14 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,16 @@
11
---
22
title: Filmstrip
3-
navOrder: 2
3+
navOrder: 30
44
---
5-
A filmstrip is a golden scene, which displays a widget at different points in
5+
A filmstrip is a Golden Scene, which displays a widget at different points in
66
time, and possibly across various user interactions.
77

88
```dart
9-
await FilmStrip(tester)
9+
await FilmStrip(
10+
tester,
11+
goldenName: "button_elevated_interactions",
12+
layout: SceneLayout.row,
13+
)
1014
.setupWithPump(() {
1115
return FlutterWidgetScaffold(
1216
goldenKey: goldenKey,
@@ -16,13 +20,13 @@ await FilmStrip(tester)
1620
),
1721
);
1822
})
19-
.takePhoto(find.byKey(goldenKey), "idle")
23+
.takePhoto("idle")
24+
// Screenshot the hovered state.
2025
.hoverOver(find.byType(ElevatedButton))
21-
.takePhoto(find.byKey(goldenKey), "hover")
26+
.takePhoto("hover")
27+
// Screenshot the pressed state.
2228
.pressHover()
23-
.takePhoto(find.byKey(goldenKey), "pressed")
24-
.renderOrCompareGolden(
25-
goldenName: "button_elevated_interactions",
26-
layout: SceneLayout.row,
27-
);
29+
.takePhoto("pressed")
30+
// Render or compare the scene.
31+
.renderOrCompareGolden();
2832
```

doc/website/source/golden-scenes/gallery.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
---
22
title: Gallery
3-
navOrder: 1
3+
navOrder: 20
44
---
5-
A gallery is a golden scene, which includes a variety of different widgets
5+
A gallery is a Golden Scene, which includes a variety of different widgets
66
and/or widget configurations.
77

88
```dart
Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
---
2+
title: What is a Golden Scene?
3+
navOrder: 10
4+
---
5+
Golden scenes are one of `flutter_test_golden`'s major innovations. A Golden Scene
6+
is a single image file that contains any number of individual golden images within it.
7+
8+
> Note: If you're familiar with `golden_toolkit` you might think that Golden Scenes are just
9+
> another name for `golden_toolkit` `GoldenBuilder`s, which have existed for years. This isn't the
10+
> case. We address this in its own section below.
11+
12+
Historically, Flutter paints a single golden image to a file. One file means one golden,
13+
and one golden means one file. One file per golden is very cumbersome when working with
14+
a large number of golden images - especially when multiple goldens are related, such as
15+
different visual states of the same widget.
16+
17+
A Golden Scene with independent golden images, such as different configurations of the same
18+
widget, is called a [Gallery](/golden-scenes/gallery).
19+
20+
A Golden Scene with golden images that take place over time, such as the idle, hovered, and
21+
pressed state of a button, is called a [Film Strip](/golden-scenes/filmstrip).
22+
23+
## What makes Golden Scenes different?
24+
Fans of golden tests are no doubt familiar with `golden_toolkit`, a package by eBay Motors,
25+
which has long been the most popular package for writing golden tests.
26+
27+
> Note: The `golden_toolkit` package is now discontinued, and no longer supported.
28+
29+
The `golden_toolkit` package included the concept of a `GoldenBuilder`, which painted multiple
30+
widget configurations to the same image file. The purpose and visual output of `golden_toolkit`
31+
`GoldenBuilder`s is similar to `flutter_test_golden` Golden Scenes, but we've taken the concept to
32+
a new level of capability and value.
33+
34+
### Extracting Golden Images from a Scene
35+
Unlike `golden_toolkit`, `flutter_test_goldens` doesn't do a pixel comparison of an entire file.
36+
`flutter_test_goldens` tracks the position of every golden image in the Golden Scene file and
37+
extracts just the golden images for comparison.
38+
39+
When comparing an entire file, you end up comparing decorations, too. You compare the text labels
40+
beneath each golden image. You compare borders around each golden image. But none of these decorations
41+
are relevant to your test. These decorations are for human reviewers, not for the computer that's
42+
running the comparison. `flutter_test_goldens` ignores these decorations, leaving you free to
43+
paint whatever details you want, around the golden images in the scene.
44+
45+
### Capturing Goldens over Time
46+
`golden_toolkit` made it possible to capture multiple widget configurations in a single golden file,
47+
but each of those widget configurations were independent widget trees. There was no concept of time.
48+
What if you want to capture the idle state, the hover state, and the pressed state of the same button?
49+
`golden_toolkit` couldn't help you.
50+
51+
`flutter_test_goldens` includes the concept of a `GoldenCamera`, which allows developers to take screenshots
52+
over a period of time. Those screenshots can then be stitched together into a Golden Scene, such as
53+
with a [`FilmStrip`](/golden-scenes/filmstrip).
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
tags: flutter-implementation

0 commit comments

Comments
 (0)