Merge tag 'v1.45.0rc1' into develop

Synapse 1.45.0rc1 (2021-10-12)
==============================

**Note:** We are aware of [a performance issue](#11049) introduced in Synapse v1.44.0, as well as [a bug](#11025) with the user directory when using application services. While this release candidate doesn't fix either of those issues, a second release candidate is expected to come out in a few days to address them.

Media storage providers module that read from Synapse's configuration need changes as of this version, see the [upgrade notes](https://matrix-org.github.io/synapse/develop/upgrade#upgrading-to-v1450) for more information.

Features
--------

- Add [MSC3069](matrix-org/matrix-spec-proposals#3069) support to `/account/whoami`. ([\#9655](#9655))
- Support autodiscovery of oEmbed previews. ([\#10822](#10822))
- Add a `user_may_send_3pid_invite` spam checker callback for modules to allow or deny 3PID invites. ([\#10894](#10894))
- Add a spam checker callback to allow or deny room joins. ([\#10910](#10910))
- Include an `update_synapse_database` script in the distribution. Contributed by @Fizzadar at Beeper. ([\#10954](#10954))
- Include exception information in JSON logging output. Contributed by @Fizzadar at Beeper. ([\#11028](#11028))

Bugfixes
--------

- Fix a minor bug in the response to `/_matrix/client/r0/voip/turnServer`. Contributed by @lukaslihotzki. ([\#10922](#10922))
- Fix a bug where empty `yyyy-mm-dd/` directories would be left behind in the media store's `url_cache_thumbnails/` directory. ([\#10924](#10924))
- Fix a bug introduced in Synapse v1.40.0 where the signature checks for room version 8 and 9 could be applied to earlier room versions in some situations. ([\#10927](#10927))
- Fix a long-standing bug wherein deactivated users still count towards the monthly active users limit. ([\#10947](#10947))
- Fix a long-standing bug which meant that events received over federation were sometimes incorrectly accepted into the room state. ([\#10956](#10956))
- Fix a long-standing bug where rebuilding the user directory wouldn't exclude support and deactivated users. ([\#10960](#10960))
- Fix [MSC2716](matrix-org/matrix-spec-proposals#2716) `/batch_send` endpoint rejecting subsequent batches with unknown batch ID error in existing room versions from the room creator. ([\#10962](#10962))
- Fix a bug that could leak local users' per-room nicknames and avatars when the user directory is rebuilt. ([\#10981](#10981))
- Fix a long-standing bug where the remainder of a batch of user directory changes would be silently dropped if the server left a room early in the batch. ([\#10982](#10982))
- Correct a bugfix introduced in Synapse v1.44.0 that would catch the wrong error if a connection is lost before a response could be written to it. ([\#10995](#10995))
- Fix a long-standing bug where local users' per-room nicknames/avatars were visible to anyone who could see you in the user directory. ([\#11002](#11002))
- Fix a long-standing bug where a user's per-room nickname/avatar would overwrite their profile in the user directory when a room was made public. ([\#11003](#11003))
- Work around a regression, introduced in Synapse v1.39.0, that caused `SynapseError`s raised by the experimental third-party rules module callback `check_event_allowed` to be ignored. ([\#11042](#11042))
- Fix a bug in [MSC2716](matrix-org/matrix-spec-proposals#2716) insertion events in rooms that could cause cross-talk/conflicts between batches. ([\#10877](#10877))

Improved Documentation
----------------------

- Change wording ("reference homeserver") in Synapse repository documentation. Contributed by @maxkratz. ([\#10971](#10971))
- Fix a dead URL in development documentation (SAML) and change wording from "Riot" to "Element". Contributed by @maxkratz. ([\#10973](#10973))
- Add additional content to the Welcome and Overview page of the documentation. ([\#10990](#10990))
- Update links to MSCs in documentation. Contributed by @dklimpel. ([\#10991](#10991))

Internal Changes
----------------

- Improve type hinting in `synapse.util`. ([\#10888](#10888))
- Add further type hints to `synapse.storage.util`. ([\#10892](#10892))
- Fix type hints to be compatible with an upcoming change to Twisted. ([\#10895](#10895))
- Update utility code to handle C implementations of frozendict. ([\#10902](#10902))
- Drop old functionality which maintained database compatibility with Synapse versions before v1.31. ([\#10903](#10903))
- Clean-up configuration helper classes for the `ServerConfig` class. ([\#10915](#10915))
- Use direct references to config flags. ([\#10916](#10916), [\#10959](#10959), [\#10985](#10985))
- Clean up some of the federation event authentication code for clarity. ([\#10926](#10926), [\#10940](#10940), [\#10986](#10986), [\#10987](#10987), [\#10988](#10988), [\#11010](#11010), [\#11011](#11011))
- Refactor various parts of the codebase to use `RoomVersion` objects instead of room version identifier strings. ([\#10934](#10934))
- Refactor user directory tests in preparation for upcoming changes. ([\#10935](#10935))
- Include the event id in the logcontext when handling PDUs received over federation. ([\#10936](#10936))
- Fix logged errors in unit tests. ([\#10939](#10939))
- Fix a broken test to ensure that consent configuration works during registration. ([\#10945](#10945))
- Add type hints to filtering classes. ([\#10958](#10958))
- Add type-hint to `HomeserverTestcase.setup_test_homeserver`. ([\#10961](#10961))
- Fix the test utility function `create_room_as` so that `is_public=True` will explicitly set the `visibility` parameter of room creation requests to `public`. Contributed by @AndrewFerr. ([\#10963](#10963))
- Make the release script more robust and transparent. ([\#10966](#10966))
- Refactor [MSC2716](matrix-org/matrix-spec-proposals#2716) `/batch_send` mega function into smaller handler functions. ([\#10974](#10974))
- Log stack traces when a missing opentracing span is detected. ([\#10983](#10983))
- Update GHA config to run tests against Python 3.10 and PostgreSQL 14. ([\#10992](#10992))
- Fix a long-standing bug where `ReadWriteLock`s could drop logging contexts on exit. ([\#10993](#10993))
- Add a `CODEOWNERS` file to automatically request reviews from the `@matrix-org/synapse-core` team on new pull requests. ([\#10994](#10994))
- Add further type hints to `synapse.state`. ([\#11004](#11004))
- Remove the deprecated `BaseHandler` object. ([\#11005](#11005))
- Bump mypy version for CI to 0.910, and pull in new type stubs for dependencies. ([\#11006](#11006))
- Fix CI to run the unit tests without optional deps. ([\#11017](#11017))
- Ensure that cache config tests do not share state. ([\#11019](#11019))
- Add additional type hints to `synapse.server_notices`. ([\#11021](#11021))
- Add additional type hints for `synapse.push`. ([\#11023](#11023))
- When installing the optional developer dependencies, also include the dependencies needed for type-checking and unit testing. ([\#11034](#11034))
- Remove unnecessary list comprehension from `synapse_port_db` to satisfy code style requirements. ([\#11043](#11043))

Author: babolivier