"Noise" aka "Variegation"

[sobotka]

Thought I'd drop a couple of references that support a high level understanding of "noise", and why it might be prudent to think deeply about how and where variegation is applied.

For those who are unfamiliar with the research out there^1,2, the general rule of thumb is that variegation along regions that our cognition tries to disentangle as "boundaries" is a Bad Idea, while variegation along spatially continuous form regions is a Good Idea. Part of the deeper reasoning is that the nature of our visual cognition apparatus is driven primarily by gradient domain relationships of the neurophysiological signals, and that these signals are used to "construct" what we think we "see".

Rapid rates of change along what could be considered of a neurophysiological second derivative signal nature, are plausibly implicated in our cognitions of form boundaries. Conversely, variegation in the same direction of first and second derivatives, thresholds depending, might be part of cognizing "part of" an existing form, and the variegation is then interpreted as "belonging to" surface material characteristics. Ergo, film grain can elevate cognitions of "sharpness".

All of this is to say that it might be worth exploring techniques to detect luminance and chrominance gradients, and reduce the variegation where high probabilities of boundaries could have a high probability of being cognitively constructed, and permit variable variegation where geometric regions of a high probability of "sameness" might be cognized?

---

There is a good body of research on the subject of variegation, but these two papers are pretty reasonable entry points, and could serve as decent entry points to the discussion.

¹ Wan X, Aoki N, Kobayashi H. Improving the Perception of Image Sharpness Using Noise Addition. Preprint posted online December 31, 2014. doi:10.11454/ephotogrst.24.2_19
² Wan X, Kobayashi H, Aoki N. Improvement in perception of image sharpness through the addition of noise and its relationship with memory texture. In: Rogowitz BE, Pappas TN, De Ridder H, eds. 2015:93941B. doi:10.1117/12.2082922

[rymuelle]

A very interesting though. Thank you for the contribution.

From Wan et a., I thought Fig 12 was an interesting visual:

[sobotka]

Be warned, this is roughly like walking in a dark room only to slip down a ramp into an incredibly deep ocean that folks weren’t even aware existed.

The General Rule of Thumb seems simplistic at first, but will rapidly descend into madness as folks realize that algorithmic deduction of “boundaries” is as elusive as Santa Claus.

Anderson is a deity on this front, if one is keen to try and get a broad understanding of the complexities. The TL;DR is that we ought to be careful with what we label “noise”, and realize that any concept of “deleterious variegation” is related to cospecification of the neurophysiological signal fields, as Marlow and Anderson explored in a paper¹. Anderson also wrote a remarkable paper that zooms out to a forest-through-trees level, that one might find fascinating when thinking about variegation in presented pictorial stimuli².

There are some fascinating lines of thinking that these two papers hint at regarding how we might exploit variegation in stimuli for elevated “quality”. Might be challenging in terms of the black box machine learning approach, and the implicit metrics employed within the model, but the subject is at least worth thinking about in the context of software designed around “noise”.

——
¹ Marlow, Phillip J., and Barton L. Anderson. “The Cospecification of the Shape and Material Properties of Light Permeable Materials.” Proceedings of the National Academy of Sciences 118, no. 14 (April 6, 2021): e2024798118. https://doi.org/10.1073/pnas.2024798118.
² Anderson, Barton L. “Mid-Level Vision.” Current Biology 30, no. 3 (February 2020): R105–9. https://doi.org/10.1016/j.cub.2019.11.088.