Skip to content

Commit bab655e

Browse files
committed
Prepare for release
1 parent 78adb11 commit bab655e

18 files changed

+100
-64
lines changed

docs/changelog/0.3.0.rst

Lines changed: 99 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,99 @@
1+
0.3.0 (2025-12-15)
2+
==================
3+
4+
Release of Data.SyncMaster 0.3.0 brings up Iceberg support and Spark-on-K8s support,
5+
as well as some UX improvements.
6+
7+
Breaking Changes
8+
----------------
9+
10+
- Allow using SyncMaster worker image as ``spark.kubernetes.container.image``. (:issue:`295`)
11+
12+
Worker container command should be changed from ``--queues 123-myqueue`` to ``worker --queues 123-myqueue``.
13+
14+
- Introduce ``config.yml`` file which is used to store settings of all components - server, scheduler, worker, frontend (:issue:`289`).
15+
16+
This is the main way to configure application now. It's still possible to use environment variables, but it is not recommended for security reasons.
17+
18+
This also implies:
19+
* Environment variable ``SYNCMASTER__ENTRYPOINT__SUPERUSERS`` is renamed to ``SYNCMASTER__SUPERUSERS``.
20+
* Logging format is configured explicitly via ``config.yml`` instead of having a set of preset configuration files.
21+
22+
- Move ``server.session`` middleware settings to ``auth`` block.
23+
Also rename some fields in ``auth.keycloak`` settings block.
24+
25+
Before:
26+
27+
.. code:: yaml
28+
29+
auth:
30+
provider: ...
31+
keycloak:
32+
server_url: ...
33+
redirect_url: ...
34+
35+
server:
36+
session:
37+
enabled: true
38+
secret_key: ...
39+
40+
Now:
41+
42+
.. code:: yaml
43+
44+
auth:
45+
provider:
46+
keycloak:
47+
api_url: ...
48+
ui_callback_url: ...
49+
cookie:
50+
secret_key: (:issue:`304`)
51+
52+
- Make S3 connection ``region`` a mandatory option.
53+
54+
55+
Features
56+
--------
57+
58+
- Added Iceberg support (:issue:`282`, :issue:`284`, :issue:`294`, :issue:`297`).
59+
60+
Iceberg connection currently supports only Iceberg REST Catalog with S3 warehouse.
61+
62+
- Allow passing default Spark session config via worker settings:
63+
64+
.. code-block:: yaml
65+
:caption: config.yml
66+
67+
worker:
68+
spark_session_default_config:
69+
spark.master: local
70+
spark.driver.host: 127.0.0.1
71+
spark.driver.bindAddress: 0.0.0.0
72+
spark.sql.pyspark.jvmStacktrace.enabled: true
73+
spark.ui.enabled: false (:issue:`291`)
74+
75+
- Added OAuth2GatewayProvider (:issue:`283`).
76+
77+
This allows using Data.SyncMaster under OAuth2 Gateway. Implementation is similar to DummyAuthProvider.
78+
79+
- Allow disabling ``SessionMiddleware``, as it only required by ``KeycloakAuthProvider``.
80+
81+
- Add hooks support to worker classes (TransferController, Handler) (:issue:`279`).
82+
- Add transfer name and group name to TransferController (:issue:`308`).
83+
84+
Improvements
85+
------------
86+
87+
- Hide ``database_name`` from Clickhouse and MySQL connection pages.
88+
- Add placeholders to connection params, like host, port and so on.
89+
- Sync frontend and backend checks for some field patterns, e.g. table name should be in format ``schema.table``.
90+
- Improve OpenAPI schema fields description.
91+
92+
Bug Fixes
93+
---------
94+
95+
Fix some file format options were ignored by SyncMaster worker:
96+
* ``XML(root_tag, row_tag)``
97+
* ``Excel(start_cell, include_header)``
98+
* ``CSV(include_header, line_sep)``
99+
* ``JSON(line_sep)``, ``JSONLine(line_sep)``

docs/changelog/index.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@
33
:caption: Changelog
44

55
DRAFT
6+
0.3.0
67
0.2.5
78
0.2.4
89
0.2.3

docs/changelog/next_release/+.breaking.rst

Lines changed: 0 additions & 1 deletion
This file was deleted.

docs/changelog/next_release/+.bugfix.rst

Lines changed: 0 additions & 5 deletions
This file was deleted.

docs/changelog/next_release/+.feature.rst

Lines changed: 0 additions & 1 deletion
This file was deleted.

docs/changelog/next_release/279.feature.rst

Lines changed: 0 additions & 2 deletions
This file was deleted.

docs/changelog/next_release/282.feature.rst

Lines changed: 0 additions & 1 deletion
This file was deleted.

docs/changelog/next_release/283.feature.rst

Lines changed: 0 additions & 2 deletions
This file was deleted.

docs/changelog/next_release/284.feature.rst

Lines changed: 0 additions & 1 deletion
This file was deleted.

docs/changelog/next_release/289.breaking.1.rst

Lines changed: 0 additions & 2 deletions
This file was deleted.

0 commit comments

Comments
 (0)