Skip to content

Commit 44c5cfb

Browse files
authored
Merge pull request ceph#64172 from zdover23/wip-doc-2025-06-25-tr70758-radosgw-bucket_logging
doc/radosgw: line edit bucket_logging.rst Reviewed-by: Yuval Lifshitz <[email protected]>
2 parents 0fe0789 + c01eb37 commit 44c5cfb

File tree

1 file changed

+68
-45
lines changed

1 file changed

+68
-45
lines changed

doc/radosgw/bucket_logging.rst

Lines changed: 68 additions & 45 deletions
Original file line numberDiff line numberDiff line change
@@ -6,60 +6,74 @@ Bucket Logging
66

77
.. contents::
88

9-
Bucket logging provides a mechanism for logging all access to a bucket. The
10-
log data can be used to monitor bucket activity, detect unauthorized
11-
access, get insights into the bucket usage and use the logs as a journal for bucket changes.
12-
The log records are stored in objects in a separate bucket and can be analyzed later.
13-
Logging configuration is done at the bucket level and can be enabled or disabled at any time.
14-
The log bucket can accumulate logs from multiple buckets. It is recommended to configured
15-
a different "prefix" for each bucket, so that the logs of different buckets will be stored
16-
in different objects in the log bucket.
9+
Bucket logging provides a mechanism for logging all access to a bucket. The log
10+
data can be used to monitor bucket activity, detect unauthorized access, get
11+
insights into bucket usage, and use the logs as a journal for bucket
12+
changes. The log records are stored in objects in a separate bucket and can be
13+
analyzed later. Logging configuration is done at the bucket level and can be
14+
enabled or disabled at any time. The log bucket can accumulate logs from
15+
multiple buckets. It is recommended to configure a different "prefix" for each
16+
bucket, so that the logs of different buckets will be stored in different
17+
objects in the log bucket.
1718

1819
.. note::
1920

2021
- The log bucket must be created before enabling logging on a bucket
2122
- The log bucket cannot be the same as the bucket being logged
2223
- The log bucket cannot have logging enabled on it
23-
- The log bucket cannot have any encryption set on in (including SSE-S3 with AES-256)
24+
- The log bucket cannot have any encryption set on it (including SSE-S3
25+
with AES-256)
2426
- The log bucket cannot have any compression set on it
2527
- The log bucket must not have RequestPayer enabled
26-
- Source and log bucket must be in the same zonegroup
27-
- Source and log buckets may belong to different accounts (with proper bucket policy set)
28+
- Source and log buckets must be in the same zonegroup
29+
- Source and log buckets may belong to different accounts (with proper
30+
bucket policy set)
2831
- The log bucket may have object lock enabled with default retention period
29-
- The 16 byte unique ID part of the log object name is a lexicographically ordered random string,
30-
that consists of a 10 bytes counter, and a 6 bytes random alphanumeric string.
31-
or a random, alphanumeric string if the counter is not available
32+
- The 16-byte unique ID part of the log object name is a lexicographically
33+
ordered random string that consists of a 10-byte counter and a 6-byte
34+
random alphanumeric string (or a random alphanumeric string if the
35+
counter is not available)
3236

3337

3438
.. toctree::
3539
:maxdepth: 1
3640

3741
Logging Reliability
3842
-------------------
39-
For performance reasons, even though the log records are written to persistent storage, the log object will
40-
appear in the log bucket only after some configurable amount of time (or if the maximum object size of 128MB is reached).
41-
This time (in seconds) could be set per source bucket via a Ceph extension to the REST API,
42-
or globally via the `rgw_bucket_logging_obj_roll_time` configuration option. If not set, the default time is 5 minutes.
43-
Adding a log object to the log bucket is done "lazily", meaning, that if no more records are written to the object, it may
44-
remain outside of the log bucket even after the configured time has passed.
45-
To counter that, you can flush all logging objects on a given source bucket to log them,
46-
regardless if enough time passed or if no more records are written to the object.
47-
Flushing will happen automatically when logging is disabled on a bucket, its logging configuration is changed, or the bucket is deleted.
43+
For performance reasons, even though the log records are written to persistent
44+
storage, the log object will appear in the log bucket only after some
45+
configurable amount of time (or if the maximum object size of 128MB is
46+
reached). This time (in seconds) can be set per source bucket via a Ceph
47+
extension to the REST API, or globally via the
48+
`rgw_bucket_logging_obj_roll_time` configuration option. If not set, the
49+
default time is 5 minutes. Adding a log object to the log bucket is done
50+
"lazily", meaning that if no more records are written to the object, it may
51+
remain outside of the log bucket even after the configured time has passed. To
52+
counter that, you can flush all logging objects on a given source bucket to log
53+
them, regardless if enough time passed or if no more records are written to the
54+
object. Flushing will happen automatically when logging is disabled on a
55+
bucket, or its logging configuration is changed, or the bucket is deleted.
4856

4957
Standard
5058
````````
51-
If logging type is set to "Standard" (the default) the log records are written to the log bucket after the bucket operation is completed.
52-
This means that there are the logging operation may fail, with no indication to he client.
59+
If the logging type is set to "Standard" (the default) the log records are
60+
written to the log bucket after the bucket operation is completed. This means
61+
that the logging operation may fail with no indication to the client.
5362

5463
Journal
5564
```````
56-
If logging type is set to "Journal", the records are written to the log bucket before the bucket operation is completed.
57-
This means that if the logging action fails, the operation will not be executed, and an error will be returned to the client
58-
Some exceptions exists to that rule, the "Fails Operation" columns in the table below indicate by "No" which operations will not fail even if logging failed.
59-
Journal mode supports filtering out records based on matches of the prefixes and suffixes of the logged object keys. Regular-expression matching can also be used on these to create filters.
60-
Note that it may happen that the log records were successfully written, but the bucket operation failed, since the logs are written.
61-
62-
The following operation are supported in journal mode:
65+
If the logging type is set to "Journal", the records are written to the log
66+
bucket before the bucket operation is completed. This means that if the logging
67+
action fails, the operation is not executed and an error is returned to the
68+
client. Some exceptions to that rule exist: the "Fails Operation" columns in
69+
the table below indicate by "No" which operations will not fail even if logging
70+
failed. Journal mode supports filtering out records based on matches of the
71+
prefixes and suffixes of the logged object keys. Regular-expression matching
72+
can also be used on these to create filters. Note that it may happen that the
73+
log records were successfully written but the bucket operation failed, since
74+
the logs are written.
75+
76+
The following operations are supported in journal mode:
6377

6478
+-------------------------------+-------------------------------------+-----------------+
6579
| Operation | Operation Name | Fails Operation |
@@ -87,16 +101,21 @@ The following operation are supported in journal mode:
87101

88102
Multisite
89103
`````````
90-
In a multi-zone deployment, each zone will use its own log object before the log object is added to the log bucket.
91-
After the log object is added to the log bucket (e.g after being flushed) it will be replicated to other zones.
92-
This means that for a given time period there could be more than one log object holding relevant log records.
104+
In a multi-zone deployment, each zone uses its own log object before the
105+
log object is added to the log bucket. After the log object is added to the
106+
log bucket (that is, after being flushed) it is replicated to other zones.
107+
This means that for a given time period there can be more than one log object
108+
holding relevant log records.
93109

94110
Bucket Logging Policy
95111
---------------------
96-
On the source bucket, only its owner is allowed to enable or disable bucket logging.
97-
For a bucket to be used as a log bucket, it must have bucket policy that allows that (even if the source bucket and the log bucket are owned by the same user or account).
98-
The bucket policy must allow the `s3:PutObject` action for the log bucket, to be performed by the `logging.s3.amazonaws.com` service principal.
99-
It should also specify the source bucket and account that are expected to write logs to it. For example:
112+
On the source bucket, only its owner is allowed to enable or disable bucket
113+
logging. For a bucket to be used as a log bucket, it must have a bucket policy
114+
that allows that (even if the source bucket and the log bucket are owned by the
115+
same user or account). The bucket policy must allow the `s3:PutObject` action
116+
for the log bucket, to be performed by the `logging.s3.amazonaws.com` service
117+
principal. The bucket policy should also specify the source bucket and account
118+
that are expected to write logs to it. For example:
100119

101120
::
102121

@@ -126,9 +145,11 @@ It should also specify the source bucket and account that are expected to write
126145

127146
Bucket Logging Quota
128147
--------------------
129-
Bucket and user quota are applied on the log bucket. Quota is checked every time a log record is written,
130-
and updated when the log object is added to the log bucket. In "Journal" mode, if the quota is exceeded, the logging operation will fail
131-
and as a result the bucket operation will also fail. In "Standard" mode, the logging operation will be skipped, but the bucket operation will continue.
148+
Bucket and user quota are applied on the log bucket. Quota is checked every
149+
time a log record is written, and is updated when the log object is added to
150+
the log bucket. In "Journal" mode, if the quota is exceeded, the logging
151+
operation fails and as a result the bucket operation also fails. In "Standard"
152+
mode, the logging operation is skipped, but the bucket operation continues.
132153

133154

134155
Bucket Logging REST API
@@ -170,14 +191,16 @@ For example:
170191
Log Records
171192
~~~~~~~~~~~
172193

173-
The log records are space separated string columns and have the following possible formats:
194+
The log records are space-separated string columns and have the following
195+
possible formats:
174196

175197
Journal
176198
```````
177-
minimum amount of data used for journaling bucket changes (this is a Ceph extension).
199+
minimum amount of data used for journaling bucket changes (this is a Ceph
200+
extension).
178201

179202
- bucket owner (or dash if empty)
180-
- bucket name (or dash if empty). in the format: ``[tenant:]<bucket name>``
203+
- bucket name (or dash if empty), in the format: ``[tenant:]<bucket name>``
181204
- time in the following format: ``[day/month/year:hour:minute:second timezone]``
182205
- object key (or dash if empty)
183206
- operation in the following format: ``WEBSITE/REST.<HTTP method>.<resource>``

0 commit comments

Comments
 (0)