44
55## Objective
66
7- Rearchitect imageproxy to use a plugin-based system for most features like
8- transformations, security, and caching. This should reduce build times and
9- binary sizes in the common case, and provide a mechanism for users to easily
10- add custom features that would not be added to core for various reasons.
7+ Re-architect imageproxy to use a plugin-based system for most features like transformations, security, and caching.
8+ This should reduce build times and binary sizes in the common case,
9+ and provide a mechanism for users to easily add custom features that would not be added to core for various reasons.
1110
1211## Background
1312
14- I created imageproxy to [ scratch a personal itch] ( https://wjn.me/b/J_ ) , I
15- needed a simple way to dynamically resize images for my personal website. I
16- published it as an open source projects because that's what I do, and I'm happy
17- to see others finding it useful for their needs as well.
18-
19- But inevitably, with more users came requests for additional features because
20- people have different use cases and requirements. Some of these requests were
21- relatively minor, and I was happy to add them. But one of the more common
22- requests was to support different caching backends. Personally, I still use the
23- on-disk cache, but many people wanted to use redis or a cloud provider like
24- AWS, Azure, or GCP. For a long time I was resistant to adding support for
25- these, mainly out of concern for inflating build times and binary sizes. I did
26- eventually relent, and
27- [ #49 ] ( https://github.com/willnorris/imageproxy/issues/49 ) tracked adding
28- support for the most common backends.
29-
30- Unfortunately my concerns proved true, and build times are _ significantly_
31- slower (TODO: add concrete numbers) now because of all the additional cloud
32- SDKs that get compiled in. I don't personally care too much about binary size,
33- since I'm not running in a constrained environment, but these build times are
34- really wearing on me. Additionally, there are a number of outstanding pull
35- requests for relatively obscure features that I don't really want to have to
36- support in the main project. And quite honestly, there are a number of obscure
37- features that did get merged in over the years that I kinda wish I could rip
38- back out.
13+ I created imageproxy to [ scratch a personal itch] ( https://wjn.me/b/J_ ) , I needed a simple way to dynamically resize images for my personal website.
14+ I published it as an open source projects because that's what I do, and I'm happy to see others finding it useful for their needs as well.
15+
16+ But inevitably, with more users came requests for additional features because people have different use cases and requirements.
17+ Some of these requests were relatively minor, and I was happy to add them.
18+ But one of the more common requests was to support different caching backends.
19+ Personally, I still use the on-disk cache, but many people wanted to use redis or a cloud provider like AWS, Azure, or GCP.
20+ For a long time I was resistant to adding support for these, mainly out of concern for inflating build times and binary sizes.
21+ I did eventually relent, and [ #49 ] tracked adding support for the most common backends.
22+
23+ Unfortunately my concerns proved true, and build times are _ significantly_ slower (TODO: add concrete numbers) now because of all the additional cloud SDKs that get compiled in.
24+ I don't personally care too much about binary size, since I'm not running in a constrained environment, but these build times are really wearing on me.
25+ Additionally, there are a number of outstanding pull requests for relatively obscure features that I don't really want to have to support in the main project.
26+ And quite honestly, there are a number of obscure features that did get merged in over the years that I kinda wish I could rip back out.
27+
28+ [ #49 ] : https://github.com/willnorris/imageproxy/issues/49
3929
4030### Plugin support in Go
4131
@@ -44,114 +34,94 @@ TODO: talk about options like
4434- RPC (< https://github.com/hashicorp/go-plugin > )
4535- pkg/plugin (< https://golang.org/pkg/plugin/ > )
4636- embedded interpreter (< https://github.com/robertkrimen/otto > )
47- - custom binaries (< https://github.com/mholt/caddy > ,
48- < https://caddy.community/t/59 > )
37+ - custom binaries (< https://github.com/mholt/caddy > , < https://caddy.community/t/59 > )
4938
50- Spoiler: I'm planning on following the Caddy approach and using custom
51- binaries.
39+ Spoiler: I'm planning on following the Caddy approach and using custom binaries.
5240
5341## Design
5442
55- I plan to model imageproxy after Caddy, moving all key functionality into
56- separate plugins that register themselves with the server, and which all
57- compile to a single statically-linked binary. The core project will provide a
58- great number of plugins to cover all of the existing functionality. I also
59- expect I'll be much more open to adding plugins for features I may not care as
60- much about personally. Of course, users can also write their own plugins and
61- link them in without needing to contribute them to core if they don't want to.
43+ I plan to model imageproxy after Caddy, moving all key functionality into separate plugins that register themselves with the server,
44+ and which all compile to a single statically-linked binary.
45+ The core project will provide a great number of plugins to cover all of the existing functionality.
46+ I also expect I'll be much more open to adding plugins for features I may not care as much about personally.
47+ Of course, users can also write their own plugins and link them in without needing to contribute them to core if they don't want to.
6248
6349I anticipate providing two or three build configurations in core:
6450
65- - ** full** - include all the plugins that are part of core (except where they
66- may conflict)
67- - ** minimal** - some set of minimal features that only includes basic caching
68- options, limited transformation options, etc
69- - ** my personal config** - I'll also definitely have a build that I use
70- personally on my site. I may decide to just make that the "minimal" build
71- and perhaps call it something different, rather than have a third
72- configuration.
51+ - ** full** - include all the plugins that are part of core (except where they may conflict)
52+ - ** minimal** - some set of minimal features that only includes basic caching options, limited transformation options, etc
53+ - ** my personal config** - I'll also definitely have a build that I use personally on my site.
54+ I may decide to just make that the "minimal" build and perhaps call it something different, rather than have a third configuration.
7355
74- Custom configurations beyond what is provided by core can be done by creating a
75- minimal main package that imports the plugins you care about and calling some
76- kind of bootstrap method (similar to [ what Caddy now
77- does] ( https://caddy.community/t/59 ) ).
56+ Custom configurations beyond what is provided by core can be done by creating a minimal main package that imports the plugins you care about
57+ and calling some kind of bootstrap method (similar to [ what Caddy now does] ( https://caddy.community/t/59 ) ).
7858
7959### Types of plugins
8060
81- (Initially in no particular order, just capturing thoughts. Lots to do here in
82- thinking through the use cases and what kind of plugin API we really need to
83- provide.)
61+ (Initially in no particular order, just capturing thoughts.
62+ Lots to do here in thinking through the use cases and what kind of plugin API we really need to provide.)
8463
8564See also issues and PRs with [ label: plugins ] [ ] .
8665
8766[ label:plugins ] : https://github.com/willnorris/imageproxy/issues?q=label:plugins
8867
8968#### Caching backend
9069
91- This is one of the most common feature requests, and is also one of the worst
92- offender for inflating build times and binary sizes because of the size of the
93- dependencies that are typically required. The minimal imageproxy build would
94- probably only include the in-memory and on-disk caches. Anything that talked to
95- an external store (redis, cloud providers, etc) would be pulled out.
70+ This is one of the most common feature requests, and is also one of the worst offender for inflating build times
71+ and binary sizes because of the size of the dependencies that are typically required.
72+ The minimal imageproxy build would probably only include the in-memory and on-disk caches.
73+ Anything that talked to an external store (redis, cloud providers, etc) would be pulled out.
9674
9775#### Transformation engine
9876
99- Today, imageproxy only performs transformations which can be done with pure Go
100- libraries. There have been a number of requests (or at least questions) to use
101- something like [ vips] ( https://github.com/DAddYE/vips ) or
102- [ imagemagick] ( https://github.com/gographics/imagick ) , which are both C
103- libraries. They provide more options, and (likely) better performance, at the
104- cost of complexity and loss of portability in using cgo. These would likely
105- replace the entire transformation engine in imageproxy, so I don't know how
106- they would interact with other plugins that merely extend the main engine (they
107- probably wouldn't be able to interact at all).
77+ Today, imageproxy only performs transformations which can be done with pure Go libraries.
78+ There have been a number of requests (or at least questions) to use something like [ vips] or [ imagemagick] , which are both C libraries.
79+ They provide more options, and (likely) better performance, at the cost of complexity and loss of portability in using cgo.
80+ These would likely replace the entire transformation engine in imageproxy,
81+ so I don't know how they would interact with other plugins that merely extend the main engine (they probably wouldn't be able to interact at all).
82+
83+ [ vips ] : https://github.com/DAddYE/vips
84+ [ imagemagick ] : https://github.com/gographics/imagick
10885
10986#### Transformation options
11087
111- Today, imageproxy performs minimal transformations, mostly around resizing,
112- cropping, and rotation. It doesn't support any kind of filters, brightness or
113- contrast adjustment, etc. There are go libraries for them, they're just outside
114- the scope of what I originally intended imageproxy for. But I'd be happy to
115- have plugins that do that kind of thing. These plugins would need to be able to
116- hook into the option parsing engine so that they could register their URL
117- options.
88+ Today, imageproxy performs minimal transformations, mostly around resizing, cropping, and rotation.
89+ It doesn't support any kind of filters, brightness or contrast adjustment, etc.
90+ There are go libraries for them, they're just outside the scope of what I originally intended imageproxy for.
91+ But I'd be happy to have plugins that do that kind of thing.
92+ These plugins would need to be able to hook into the option parsing engine so that they could register their URL options.
11893
11994#### Image format support
12095
121- There have been a number of requests for imge format support that require cgo
122- libraries:
96+ There have been a number of requests for image format support that require cgo libraries:
12397
124- - ** webp encoding** - needs cgo
125- [ #114 ] ( https://github.com/willnorris/imageproxy/issues/114 )
126- - ** progressive jpegs** - probably needs cgo?
127- [ #77 ] ( https://github.com/willnorris/imageproxy/issues/77 )
128- - ** gif to mp4** - maybe doable in pure go, but probably belongs in a plugin
129- [ #136 ] ( https://github.com/willnorris/imageproxy/issues/136 )
130- - ** HEIF** - formate used by newer iPhones
131- ([ HEIF] ( https://en.wikipedia.org/wiki/High_Efficiency_Image_File_Format ) )
98+ - ** webp encoding** - needs cgo [ #114 ] ( https://github.com/willnorris/imageproxy/issues/114 )
99+ - ** progressive jpegs** - probably needs cgo? [ #77 ] ( https://github.com/willnorris/imageproxy/issues/77 )
100+ - ** gif to mp4** - maybe doable in pure go, but probably belongs in a plugin [ #136 ] ( https://github.com/willnorris/imageproxy/issues/136 )
101+ - ** HEIF** - formate used by newer iPhones ([ HEIF] ( https://en.wikipedia.org/wiki/High_Efficiency_Image_File_Format ) )
132102
133103#### Option parsing
134104
135- Today, options are specified as the first component in the URL path, but
136- [ # 66 ] ( https://github.com/willnorris/imageproxy/pull/66 ) proposes optionally
137- moving that to a query parameter ( for a good reason, actually). Maybe putting
138- that in core is okay? Maybe it belongs in a plugin, in which case we'd need to
139- expose an API for replacing the option parsing code entirely.
105+ Today, options are specified as the first component in the URL path, but [ # 66 ] proposes optionally moving that to a query parameter (for a good reason, actually).
106+ Maybe putting that in core is okay?
107+ Maybe it belongs in a plugin, in which case we'd need to expose an API for replacing the option parsing code entirely.
108+
109+ [ #66 ] : https://github.com/willnorris/imageproxy/pull/66
140110
141111#### Security options
142112
143- Some people want to add a host blacklist
144- [ # 85 ] ( https://github.com/willnorris/imageproxy/pull/85 ) , refusal to process
145- non-image files [ # 53 ] ( https://github.com/willnorris/imageproxy/issues/53 )
146- [ # 119 ] ( https://github.com/willnorris/imageproxy/pull/119 ) . I don't think there
147- is an issue for it, but an early fork of the project added request signing that
148- was compatible with nginx's [ secure link
149- module ] ( https://nginx.org/en/docs/http/ngx_http_secure_link_module.html ) .
113+ Some people want to add a host blacklist [ # 85 ] , refusal to process non-image files [ # 53 ] [ # 119 ] .
114+ I don't think there is an issue for it,
115+ but an early fork of the project added request signing that was compatible with nginx's [ secure link module ] ( https://nginx.org/en/docs/http/ngx_http_secure_link_module.html ) .
116+
117+ [ #85 ] : https://github.com/willnorris/imageproxy/pull/85
118+ [ #53 ] : https://github.com/willnorris/imageproxy/issues/53
119+ [ #119 ] : https://github.com/willnorris/imageproxy/pull/119
150120
151121### Registering Plugins
152122
153- Plugins are loaded simply by importing their package. They should have an
154- ` init ` func that calls ` imageproxy.RegisterPlugin ` :
123+ Plugins are loaded simply by importing their package.
124+ They should have an ` init ` func that calls ` imageproxy.RegisterPlugin ` :
155125
156126``` go
157127type Plugin struct {
@@ -160,9 +130,8 @@ type Plugin struct {
160130func RegisterPlugin (name string , plugin Plugin )
161131```
162132
163- Plugins hook into various extension points of imageproxy by implementing
164- appropriate interfaces. A single plugin can hook into multiple parts of
165- imageproxy by implementing multiple interfaces.
133+ Plugins hook into various extension points of imageproxy by implementing appropriate interfaces.
134+ A single plugin can hook into multiple parts of imageproxy by implementing multiple interfaces.
166135
167136For example, two possible interfaces for security related plugins:
168137
@@ -196,6 +165,5 @@ type ImageTransformer interface {
196165}
197166```
198167
199- Plugins are additionally responsible for registering any additional command
200- line flags they wish to expose to the user, as well as storing any global state
201- that would previously have been stored on the Proxy struct.
168+ Plugins are additionally responsible for registering any additional command line flags they wish to expose to the user,
169+ as well as storing any global state that would previously have been stored on the Proxy struct.
0 commit comments