Skip to content

Commit e4f7ead

Browse files
committed
docs: reformat plugin-design.md for semantic line breaks
I'm hoping to pick this idea back up soon.
1 parent 7f36398 commit e4f7ead

File tree

1 file changed

+74
-106
lines changed

1 file changed

+74
-106
lines changed

docs/plugin-design.md

Lines changed: 74 additions & 106 deletions
Original file line numberDiff line numberDiff line change
@@ -4,38 +4,28 @@
44

55
## Objective
66

7-
Rearchitect imageproxy to use a plugin-based system for most features like
8-
transformations, security, and caching. This should reduce build times and
9-
binary sizes in the common case, and provide a mechanism for users to easily
10-
add custom features that would not be added to core for various reasons.
7+
Re-architect imageproxy to use a plugin-based system for most features like transformations, security, and caching.
8+
This should reduce build times and binary sizes in the common case,
9+
and provide a mechanism for users to easily add custom features that would not be added to core for various reasons.
1110

1211
## Background
1312

14-
I created imageproxy to [scratch a personal itch](https://wjn.me/b/J_), I
15-
needed a simple way to dynamically resize images for my personal website. I
16-
published it as an open source projects because that's what I do, and I'm happy
17-
to see others finding it useful for their needs as well.
18-
19-
But inevitably, with more users came requests for additional features because
20-
people have different use cases and requirements. Some of these requests were
21-
relatively minor, and I was happy to add them. But one of the more common
22-
requests was to support different caching backends. Personally, I still use the
23-
on-disk cache, but many people wanted to use redis or a cloud provider like
24-
AWS, Azure, or GCP. For a long time I was resistant to adding support for
25-
these, mainly out of concern for inflating build times and binary sizes. I did
26-
eventually relent, and
27-
[#49](https://github.com/willnorris/imageproxy/issues/49) tracked adding
28-
support for the most common backends.
29-
30-
Unfortunately my concerns proved true, and build times are _significantly_
31-
slower (TODO: add concrete numbers) now because of all the additional cloud
32-
SDKs that get compiled in. I don't personally care too much about binary size,
33-
since I'm not running in a constrained environment, but these build times are
34-
really wearing on me. Additionally, there are a number of outstanding pull
35-
requests for relatively obscure features that I don't really want to have to
36-
support in the main project. And quite honestly, there are a number of obscure
37-
features that did get merged in over the years that I kinda wish I could rip
38-
back out.
13+
I created imageproxy to [scratch a personal itch](https://wjn.me/b/J_), I needed a simple way to dynamically resize images for my personal website.
14+
I published it as an open source projects because that's what I do, and I'm happy to see others finding it useful for their needs as well.
15+
16+
But inevitably, with more users came requests for additional features because people have different use cases and requirements.
17+
Some of these requests were relatively minor, and I was happy to add them.
18+
But one of the more common requests was to support different caching backends.
19+
Personally, I still use the on-disk cache, but many people wanted to use redis or a cloud provider like AWS, Azure, or GCP.
20+
For a long time I was resistant to adding support for these, mainly out of concern for inflating build times and binary sizes.
21+
I did eventually relent, and [#49] tracked adding support for the most common backends.
22+
23+
Unfortunately my concerns proved true, and build times are _significantly_ slower (TODO: add concrete numbers) now because of all the additional cloud SDKs that get compiled in.
24+
I don't personally care too much about binary size, since I'm not running in a constrained environment, but these build times are really wearing on me.
25+
Additionally, there are a number of outstanding pull requests for relatively obscure features that I don't really want to have to support in the main project.
26+
And quite honestly, there are a number of obscure features that did get merged in over the years that I kinda wish I could rip back out.
27+
28+
[#49]: https://github.com/willnorris/imageproxy/issues/49
3929

4030
### Plugin support in Go
4131

@@ -44,114 +34,94 @@ TODO: talk about options like
4434
- RPC (<https://github.com/hashicorp/go-plugin>)
4535
- pkg/plugin (<https://golang.org/pkg/plugin/>)
4636
- embedded interpreter (<https://github.com/robertkrimen/otto>)
47-
- custom binaries (<https://github.com/mholt/caddy>,
48-
<https://caddy.community/t/59>)
37+
- custom binaries (<https://github.com/mholt/caddy>, <https://caddy.community/t/59>)
4938

50-
Spoiler: I'm planning on following the Caddy approach and using custom
51-
binaries.
39+
Spoiler: I'm planning on following the Caddy approach and using custom binaries.
5240

5341
## Design
5442

55-
I plan to model imageproxy after Caddy, moving all key functionality into
56-
separate plugins that register themselves with the server, and which all
57-
compile to a single statically-linked binary. The core project will provide a
58-
great number of plugins to cover all of the existing functionality. I also
59-
expect I'll be much more open to adding plugins for features I may not care as
60-
much about personally. Of course, users can also write their own plugins and
61-
link them in without needing to contribute them to core if they don't want to.
43+
I plan to model imageproxy after Caddy, moving all key functionality into separate plugins that register themselves with the server,
44+
and which all compile to a single statically-linked binary.
45+
The core project will provide a great number of plugins to cover all of the existing functionality.
46+
I also expect I'll be much more open to adding plugins for features I may not care as much about personally.
47+
Of course, users can also write their own plugins and link them in without needing to contribute them to core if they don't want to.
6248

6349
I anticipate providing two or three build configurations in core:
6450

65-
- **full** - include all the plugins that are part of core (except where they
66-
may conflict)
67-
- **minimal** - some set of minimal features that only includes basic caching
68-
options, limited transformation options, etc
69-
- **my personal config** - I'll also definitely have a build that I use
70-
personally on my site. I may decide to just make that the "minimal" build
71-
and perhaps call it something different, rather than have a third
72-
configuration.
51+
- **full** - include all the plugins that are part of core (except where they may conflict)
52+
- **minimal** - some set of minimal features that only includes basic caching options, limited transformation options, etc
53+
- **my personal config** - I'll also definitely have a build that I use personally on my site.
54+
I may decide to just make that the "minimal" build and perhaps call it something different, rather than have a third configuration.
7355

74-
Custom configurations beyond what is provided by core can be done by creating a
75-
minimal main package that imports the plugins you care about and calling some
76-
kind of bootstrap method (similar to [what Caddy now
77-
does](https://caddy.community/t/59)).
56+
Custom configurations beyond what is provided by core can be done by creating a minimal main package that imports the plugins you care about
57+
and calling some kind of bootstrap method (similar to [what Caddy now does](https://caddy.community/t/59)).
7858

7959
### Types of plugins
8060

81-
(Initially in no particular order, just capturing thoughts. Lots to do here in
82-
thinking through the use cases and what kind of plugin API we really need to
83-
provide.)
61+
(Initially in no particular order, just capturing thoughts.
62+
Lots to do here in thinking through the use cases and what kind of plugin API we really need to provide.)
8463

8564
See also issues and PRs with [label:plugins][].
8665

8766
[label:plugins]: https://github.com/willnorris/imageproxy/issues?q=label:plugins
8867

8968
#### Caching backend
9069

91-
This is one of the most common feature requests, and is also one of the worst
92-
offender for inflating build times and binary sizes because of the size of the
93-
dependencies that are typically required. The minimal imageproxy build would
94-
probably only include the in-memory and on-disk caches. Anything that talked to
95-
an external store (redis, cloud providers, etc) would be pulled out.
70+
This is one of the most common feature requests, and is also one of the worst offender for inflating build times
71+
and binary sizes because of the size of the dependencies that are typically required.
72+
The minimal imageproxy build would probably only include the in-memory and on-disk caches.
73+
Anything that talked to an external store (redis, cloud providers, etc) would be pulled out.
9674

9775
#### Transformation engine
9876

99-
Today, imageproxy only performs transformations which can be done with pure Go
100-
libraries. There have been a number of requests (or at least questions) to use
101-
something like [vips](https://github.com/DAddYE/vips) or
102-
[imagemagick](https://github.com/gographics/imagick), which are both C
103-
libraries. They provide more options, and (likely) better performance, at the
104-
cost of complexity and loss of portability in using cgo. These would likely
105-
replace the entire transformation engine in imageproxy, so I don't know how
106-
they would interact with other plugins that merely extend the main engine (they
107-
probably wouldn't be able to interact at all).
77+
Today, imageproxy only performs transformations which can be done with pure Go libraries.
78+
There have been a number of requests (or at least questions) to use something like [vips] or [imagemagick], which are both C libraries.
79+
They provide more options, and (likely) better performance, at the cost of complexity and loss of portability in using cgo.
80+
These would likely replace the entire transformation engine in imageproxy,
81+
so I don't know how they would interact with other plugins that merely extend the main engine (they probably wouldn't be able to interact at all).
82+
83+
[vips]: https://github.com/DAddYE/vips
84+
[imagemagick]: https://github.com/gographics/imagick
10885

10986
#### Transformation options
11087

111-
Today, imageproxy performs minimal transformations, mostly around resizing,
112-
cropping, and rotation. It doesn't support any kind of filters, brightness or
113-
contrast adjustment, etc. There are go libraries for them, they're just outside
114-
the scope of what I originally intended imageproxy for. But I'd be happy to
115-
have plugins that do that kind of thing. These plugins would need to be able to
116-
hook into the option parsing engine so that they could register their URL
117-
options.
88+
Today, imageproxy performs minimal transformations, mostly around resizing, cropping, and rotation.
89+
It doesn't support any kind of filters, brightness or contrast adjustment, etc.
90+
There are go libraries for them, they're just outside the scope of what I originally intended imageproxy for.
91+
But I'd be happy to have plugins that do that kind of thing.
92+
These plugins would need to be able to hook into the option parsing engine so that they could register their URL options.
11893

11994
#### Image format support
12095

121-
There have been a number of requests for imge format support that require cgo
122-
libraries:
96+
There have been a number of requests for image format support that require cgo libraries:
12397

124-
- **webp encoding** - needs cgo
125-
[#114](https://github.com/willnorris/imageproxy/issues/114)
126-
- **progressive jpegs** - probably needs cgo?
127-
[#77](https://github.com/willnorris/imageproxy/issues/77)
128-
- **gif to mp4** - maybe doable in pure go, but probably belongs in a plugin
129-
[#136](https://github.com/willnorris/imageproxy/issues/136)
130-
- **HEIF** - formate used by newer iPhones
131-
([HEIF](https://en.wikipedia.org/wiki/High_Efficiency_Image_File_Format))
98+
- **webp encoding** - needs cgo [#114](https://github.com/willnorris/imageproxy/issues/114)
99+
- **progressive jpegs** - probably needs cgo? [#77](https://github.com/willnorris/imageproxy/issues/77)
100+
- **gif to mp4** - maybe doable in pure go, but probably belongs in a plugin [#136](https://github.com/willnorris/imageproxy/issues/136)
101+
- **HEIF** - formate used by newer iPhones ([HEIF](https://en.wikipedia.org/wiki/High_Efficiency_Image_File_Format))
132102

133103
#### Option parsing
134104

135-
Today, options are specified as the first component in the URL path, but
136-
[#66](https://github.com/willnorris/imageproxy/pull/66) proposes optionally
137-
moving that to a query parameter (for a good reason, actually). Maybe putting
138-
that in core is okay? Maybe it belongs in a plugin, in which case we'd need to
139-
expose an API for replacing the option parsing code entirely.
105+
Today, options are specified as the first component in the URL path, but [#66] proposes optionally moving that to a query parameter (for a good reason, actually).
106+
Maybe putting that in core is okay?
107+
Maybe it belongs in a plugin, in which case we'd need to expose an API for replacing the option parsing code entirely.
108+
109+
[#66]: https://github.com/willnorris/imageproxy/pull/66
140110

141111
#### Security options
142112

143-
Some people want to add a host blacklist
144-
[#85](https://github.com/willnorris/imageproxy/pull/85), refusal to process
145-
non-image files [#53](https://github.com/willnorris/imageproxy/issues/53)
146-
[#119](https://github.com/willnorris/imageproxy/pull/119). I don't think there
147-
is an issue for it, but an early fork of the project added request signing that
148-
was compatible with nginx's [secure link
149-
module](https://nginx.org/en/docs/http/ngx_http_secure_link_module.html).
113+
Some people want to add a host blacklist [#85], refusal to process non-image files [#53] [#119].
114+
I don't think there is an issue for it,
115+
but an early fork of the project added request signing that was compatible with nginx's [secure link module](https://nginx.org/en/docs/http/ngx_http_secure_link_module.html).
116+
117+
[#85]: https://github.com/willnorris/imageproxy/pull/85
118+
[#53]: https://github.com/willnorris/imageproxy/issues/53
119+
[#119]: https://github.com/willnorris/imageproxy/pull/119
150120

151121
### Registering Plugins
152122

153-
Plugins are loaded simply by importing their package. They should have an
154-
`init` func that calls `imageproxy.RegisterPlugin`:
123+
Plugins are loaded simply by importing their package.
124+
They should have an `init` func that calls `imageproxy.RegisterPlugin`:
155125

156126
```go
157127
type Plugin struct {
@@ -160,9 +130,8 @@ type Plugin struct {
160130
func RegisterPlugin(name string, plugin Plugin)
161131
```
162132

163-
Plugins hook into various extension points of imageproxy by implementing
164-
appropriate interfaces. A single plugin can hook into multiple parts of
165-
imageproxy by implementing multiple interfaces.
133+
Plugins hook into various extension points of imageproxy by implementing appropriate interfaces.
134+
A single plugin can hook into multiple parts of imageproxy by implementing multiple interfaces.
166135

167136
For example, two possible interfaces for security related plugins:
168137

@@ -196,6 +165,5 @@ type ImageTransformer interface {
196165
}
197166
```
198167

199-
Plugins are additionally responsible for registering any additional command
200-
line flags they wish to expose to the user, as well as storing any global state
201-
that would previously have been stored on the Proxy struct.
168+
Plugins are additionally responsible for registering any additional command line flags they wish to expose to the user,
169+
as well as storing any global state that would previously have been stored on the Proxy struct.

0 commit comments

Comments
 (0)