|
1 | 1 | --- |
2 | 2 | myst: |
3 | 3 | html_meta: |
4 | | - "description": "" |
5 | | - "property=og:description": "" |
6 | | - "property=og:title": "" |
7 | | - "keywords": "" |
| 4 | + "description": "Zope Object Database (ZODB)" |
| 5 | + "property=og:description": "Zope Object Database (ZODB)" |
| 6 | + "property=og:title": "Zope Object Database (ZODB)" |
| 7 | + "keywords": "Plone, ZODB, Zope Object Database, RelStorage, ZEO, ZODB Extension Objects" |
8 | 8 | --- |
9 | 9 |
|
| 10 | +% TODO: Diátaxis conceptual guide |
| 11 | + |
10 | 12 | (backend-zodb-label)= |
11 | 13 |
|
12 | | -# ZODB |
| 14 | +# Zope Object Database (ZODB) |
13 | 15 |
|
14 | | -The ZODB (Zope Object Database) is a Python-native, object-oriented database designed for direct persistence of Python objects. |
| 16 | +The {term}`Zope Object Database` (ZODB) is a Python native, object-oriented database designed for direct persistence of Python objects. |
15 | 17 | Unlike traditional relational databases that rely on tables and SQL queries, ZODB allows developers to work directly with Python objects, persisting them without the need for object-relational mapping (ORM). |
16 | 18 |
|
17 | 19 |
|
18 | | -## Core Features of ZODB |
| 20 | +## Core features of ZODB |
| 21 | + |
| 22 | +Transparent persistence |
| 23 | +: Objects stored in ZODB automatically persist. |
| 24 | + In a Plone request/response cycle, they do not require an explicit save or commit operation, since this is done automatically. |
19 | 25 |
|
20 | | -- Transparent Persistence: Objects stored in ZODB automatically persist. |
21 | | - In a Plone request/response cycle they do not require an explicit save or commit operation, since this is done automatically when the response is ready but before it is delivered. |
22 | | -- No schema constraints: Unlike relational databases, ZODB does not require predefined schemas, allowing for flexible and dynamic data structures. |
| 26 | +No schema constraints |
| 27 | +: Unlike relational databases, ZODB does not require predefined schemas, allowing for flexible and dynamic data structures. |
23 | 28 | The attributes of the Python objects themselves are stored. |
24 | | -- ACID Compliance: ZODB ensures data consistency through transactions that support Atomicity, Consistency, Isolation, and Durability. |
25 | | -- Automatic Conflict Resolution: With its multi-version concurrency control (MVCC), ZODB can handle concurrent access efficiently. |
26 | | -- Built-in Versioning and Undo: The database allows versioning, enabling rollback to previous states if needed. |
27 | | -- Scalability: ZODB can be used in standalone applications or scaled with ZEO (ZODB Extension Objects) for distributed storage. |
28 | | - Additionally, ZODB supports the RelStorage adapter, which allows it to use relational databases like PostgreSQL, MySQL, or Oracle as a backend, providing flexibility for integration with existing database infrastructures. |
29 | | - |
30 | | -## How ZODB Works |
31 | | - |
32 | | -At its core, ZODB operates as an object store, maintaining pickled Python objects with some metadata in a hierarchical structure. |
33 | | - |
34 | | -- Storage: Handles how objects are stored. |
35 | | - The default storage is FileStorage, which writes data to `.fs` files Filestorage does not scale, as only one process can work with it. |
36 | | - ZEO (Zope Enterprise Objects) storage is used for distributed multi-client access to the same database, introducing scalability. |
37 | | - Another highly scalable option is RelStorage, which allows ZODB to use relational databases like PostgreSQL, MySQL, or Oracle as backend storage, combining object persistence with traditional database infrastructure. |
38 | | - RelStorage is used often in containerized deployment environment. |
39 | | - |
40 | | -- Connection: Acts as the interface between Python applications and the database. |
41 | | - With ZEO for each active Zope thread one connection is established. |
42 | | - Connection are pooled and re-used. |
43 | | - |
44 | | -- Transaction Manager: Manages transactional operations, ensuring data integrity. |
45 | | - A transaction normally start with the requests and ends with the response. |
46 | | - However, in long running requests with lots of database writes, the programmer can decide to commit actively in between. |
47 | | - |
48 | | -- Append only behavior: |
49 | | - The ZODB in general appends new transactions and keeps the old version of the object. |
50 | | - Even on delete only the reference to the object is removed, the object itself is not touched. |
51 | | - Exception here is RelStorage in history free mode, where actual data is overridden. |
52 | | - |
53 | | -- Binary Large Objects (BLOBs) are files, images or similar larger objects. |
54 | | - Those are handled special in ZODB. |
55 | | - Either those are stored in a own tree of folders or in the database. |
56 | | - Latter is recommended only for RelStorage, since FileStorage gets bloated and slow with many large objects. |
57 | | - If the blobs are stored in the filesystem and ZEO is used, each ZEO client needs to mount the blob-storage folder-tree as a shared folder. |
58 | | - There is an option to not configure ZEO with shared folders, but it is not recommended because of performance reasons. |
59 | | - |
60 | | -- Any object that inherits from `Persistent` can be stored. |
61 | | - Python objects are stored as a whole. |
62 | | - To not store one large object they need to be divided. |
63 | | - If an objet has an attribute which itself inherits from `Persistent` it is stored as a separate object in the ZODB. |
64 | | - To divide lists and dicts into an objet for each value, there are `PersistentDict` and `PersistentList` classes. |
65 | | - For larger structures the `BTree` package with its different optimized tree- and set-classes is generally used. |
66 | | - Blobs are handled separately |
67 | | - |
68 | | -- Low Level Indexing and Caching: Optimizes read and write operations for better performance. |
69 | | - Tuning the ZODB-cache sizes to hardware-environment and the kind of data stored may help to speed up the application. |
70 | | - |
71 | | -- Packing is the process of removing old versions of an object. |
72 | | - This includes deleted objects. |
73 | | - So even with the above mentioned history free RelStorage packing is needed to get rid of the deleted objects. |
74 | | - The packing process needs to be started actively, either using the web interface (ZMI) or by starting a pack script. |
75 | | - Usually this is done as a cron job. |
| 29 | + |
| 30 | +ACID compliance |
| 31 | +: ZODB ensures data consistency through transactions that support atomicity, consistency, isolation, and durability (ACID). |
| 32 | + |
| 33 | +Automatic conflict resolution |
| 34 | +: With its multi-version concurrency control (MVCC), ZODB can handle concurrent access efficiently. |
| 35 | + |
| 36 | +Built-in versioning and undo |
| 37 | +: The database allows versioning, enabling rollback to previous states if needed. |
| 38 | + |
| 39 | +Scalability |
| 40 | +: ZODB can be used in standalone applications or scaled with {term}`ZODB Extension Objects` (ZEO) for distributed storage. |
| 41 | + Additionally, ZODB supports the {term}`RelStorage` adapter, which allows it to use relational databases, including PostgreSQL, MySQL, or Oracle, as a backend, providing flexibility for integration with existing database infrastructures. |
| 42 | + |
| 43 | + |
| 44 | +## How ZODB works |
| 45 | + |
| 46 | +At its core, ZODB operates as an object store, maintaining serialized Python objects with some metadata in a hierarchical structure. |
| 47 | + |
| 48 | +Storage |
| 49 | +: Handles how objects are stored on disk or in-memory. |
| 50 | + |
| 51 | + The default storage is {term}`FileStorage`, which writes data to `.fs` files. `FileStorage` does not scale, as only one process can work with it. |
| 52 | + |
| 53 | + {term}`ZEO` storage is used for distributed multi-client access to the same database, introducing scalability. |
| 54 | + |
| 55 | + {term}`RelStorage` is another highly scalable option. |
| 56 | + It allows ZODB to use relational databases like PostgreSQL, MySQL, or Oracle as backend storage, combining object persistence with traditional database infrastructure. |
| 57 | + RelStorage is used often in a containerized deployment environment. |
| 58 | + |
| 59 | +Connection |
| 60 | +: Acts as the interface between Python applications and the database. |
| 61 | + With ZEO for each active Zope thread, one connection is established. |
| 62 | + Connections are pooled and re-used. |
| 63 | + |
| 64 | +Transaction manager |
| 65 | +: Manages transactional operations, ensuring data integrity. |
| 66 | + A transaction normally starts with the request, and ends with the response. |
| 67 | + However, in long-running requests with lots of database writes, transactions can be committed in between. |
| 68 | + |
| 69 | +Indexing and caching |
| 70 | +: Optimizes read and write operations for better performance. |
| 71 | + Tuning the ZODB cache sizes to hardware environment and the kind of data stored may help to speed up the application. |
| 72 | + |
76 | 73 |
|
77 | 74 | ## Further reading |
78 | 75 |
|
79 | | -More information can be found at the official [ZODB website](https://zodb.org) |
| 76 | +More information can be found at the official [ZODB website](https://zodb.org/en/latest/). |
0 commit comments