Merge branch 'main' into add-email-contact

Author: Shraddha Agrawal

doc/radosgw/account.rst

@@ -177,8 +177,8 @@ An existing user can be adopted into an account with ``user modify``::
 .. warning:: Ownership of the user's notification topics will not be
    transferred to the account. Notifications will continue to work, but
    the topics will no longer be visible to SNS Topic APIs. Topics and
-   their associated bucket notifications should be removed before migration
-   and recreated within the account.
+   their associated bucket notifications can be migrated as described below
+   in `Migrating Notification Topics`_.
 
 Because account users have no permissions by default, some identity policy must
 be added to restore the user's original permissions.
@@ -187,6 +187,44 @@ Alternatively, you may want to create a new account for each existing user. In
 that case, you may want to add the ``--account-root`` option to make each user
 the root user of their account.
 
+Migrating Notification Topics
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Account topics are supported only when the ``notification_v2`` feature is enabled,
+as described in `Bucket Notifications`_ and `Supported Zone Features`_.
+
+1. ``Migration Impact``: When a non-account user is migrated to an account, the
+the existing notification topics remain accessible through the RadosGW admin API,
+but the user loses access to them via the SNS Topic API. Despite this, the topics
+remain functional, and bucket notifications will continue to be delivered as expected.
+
+2. ``Re-creation of Topics``: The account user should re-create the topics using
+the same names. The old topics (now inaccessible) and the new account-owned topics
+will coexist without interference.
+
+3. ``Updating Bucket Notification Configurations``: Buckets that are subscribed to
+the old user-owned topics should be updated to use the new account-owned topics.
+To prevent duplicate notifications, maintain the same notification IDs.
+For example, if a bucket's existing notification configuration is:
+
+    .. code-block:: json
+
+        {"TopicConfigurations": [{ "Id": "ID1", "TopicArn": "arn:aws:sns:default::topic1", "Events": ["s3:ObjectCreated:*"]}]}
+
+The updated configuration would be:
+
+    .. code-block:: json
+
+        {"TopicConfigurations": [{ "Id": "ID1", "TopicArn": "arn:aws:sns:default:RGW00000000000000001:topic1", "Events": ["s3:ObjectCreated:*"]}]}
+
+In this example, `RGW00000000000000001` is the account ID, `topic1` is the
+topic name and `ID1` is the notification ID.
+
+4. ``Removing Old Topics``: Once no buckets are subscribed to the old user-owned topics,
+they can be removed by an admin::
+
+	$ radosgw-admin topic rm --topic topic1
+
 Account Root example
 --------------------
 
@@ -252,3 +290,5 @@ This example uses `awscli`_ to create an IAM user for S3 operations.
 .. _Evaluating policies within a single account: https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_evaluation-logic.html#policy-eval-basics
 .. _Cross-account policy evaluation logic: https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_evaluation-logic-cross-account.html
 .. _awscli: https://docs.aws.amazon.com/cli/latest/
+.. _Bucket Notifications: ../notifications/
+.. _Supported Zone Features: ../zone-features/#supported-features

doc/radosgw/oidc.rst

@@ -45,7 +45,7 @@ Example::
   POST "<hostname>?Action=Action=CreateOpenIDConnectProvider
     &ThumbprintList.list.1=F7D7B3515DD0D319DD219A43A9EA727AD6065287
     &ClientIDList.list.1=app-profile-jsp
-    &Url=http://localhost:8080/auth/realms/quickstart
+    &Url=http://localhost:8080/auth/realms/quickstart"
 
 
 DeleteOpenIDConnectProvider
@@ -63,7 +63,7 @@ Request Parameters
 
 Example::
   POST "<hostname>?Action=Action=DeleteOpenIDConnectProvider
-    &OpenIDConnectProviderArn=arn:aws:iam:::oidc-provider/localhost:8080/auth/realms/quickstart
+    &OpenIDConnectProviderArn=arn:aws:iam:::oidc-provider/localhost:8080/auth/realms/quickstart"
 
 
 GetOpenIDConnectProvider
@@ -81,7 +81,7 @@ Request Parameters
 
 Example::
   POST "<hostname>?Action=Action=GetOpenIDConnectProvider
-    &OpenIDConnectProviderArn=arn:aws:iam:::oidc-provider/localhost:8080/auth/realms/quickstart
+    &OpenIDConnectProviderArn=arn:aws:iam:::oidc-provider/localhost:8080/auth/realms/quickstart"
 
 ListOpenIDConnectProviders
 --------------------------
@@ -95,3 +95,50 @@ None
 
 Example::
   POST "<hostname>?Action=Action=ListOpenIDConnectProviders
+
+AddClientIDToOpenIDConnectProvider
+----------------------------------
+
+Add a client id to the list of existing client ids registered while creating an OpenIDConnectProvider.
+
+Request Parameters
+~~~~~~~~~~~~~~~~~~
+
+``OpenIDConnectProviderArn``
+
+:Description: ARN of the IDP which is returned by the Create API.
+:Type: String
+
+``ClientID``
+
+:Description: Client Id to add to the existing OpenIDConnectProvider.
+:Type: String
+
+Example::
+  POST "<hostname>?Action=Action=AddClientIDToOpenIDConnectProvider
+    &OpenIDConnectProviderArn=arn:aws:iam:::oidc-provider/localhost:8080/auth/realms/quickstart
+    &ClientID=app-jee-jsp"
+
+UpdateOpenIDConnectProviderThumbprint
+-------------------------------------
+
+Update the existing thumbprint list of an OpenIDConnectProvider with the given list.
+This API removes the existing thumbprint list and replaces that with the input thumbprint list.
+
+Request Parameters
+~~~~~~~~~~~~~~~~~~
+
+``OpenIDConnectProviderArn``
+
+:Description: ARN of the IDP which is returned by the Create API.
+:Type: String
+
+``ThumbprintList.member.N``
+
+:Description: List of OpenID Connect IDP's server certificates' thumbprints. A maximum of 5 thumbprints are allowed.
+:Type: Array of Strings
+
+Example::
+  POST "<hostname>?Action=Action=UpdateOpenIDConnectProviderThumbprint
+    &OpenIDConnectProviderArn=arn:aws:iam:::oidc-provider/localhost:8080/auth/realms/quickstart
+    &&ThumbprintList.list.1=ABCDB3515DD0D319DD219A43A9EA727AD6061234"

src/client/Client.cc

@@ -15836,6 +15836,10 @@ int Client::ll_read(Fh *fh, loff_t off, loff_t len, bufferlist *bl)
     return -CEPHFS_ENOTCONN;
   }
 
+  /* We can't return bytes written larger than INT_MAX, clamp len to that */
+  len = std::min(len, (loff_t)INT_MAX);
+
+  std::scoped_lock lock(client_lock);
   if (fh == NULL || !_ll_fh_exists(fh)) {
     ldout(cct, 3) << "(fh)" << fh << " is invalid" << dendl;
     return -CEPHFS_EBADF;
@@ -15847,10 +15851,6 @@ int Client::ll_read(Fh *fh, loff_t off, loff_t len, bufferlist *bl)
   tout(cct) << off << std::endl;
   tout(cct) << len << std::endl;
 
-  /* We can't return bytes written larger than INT_MAX, clamp len to that */
-  len = std::min(len, (loff_t)INT_MAX);
-  std::scoped_lock lock(client_lock);
-
   int r = _read(fh, off, len, bl);
   ldout(cct, 3) << "ll_read " << fh << " " << off << "~" << len << " = " << r
 		<< dendl;
@@ -15981,6 +15981,10 @@ int Client::ll_write(Fh *fh, loff_t off, loff_t len, const char *data)
     return -CEPHFS_ENOTCONN;
   }
 
+  /* We can't return bytes written larger than INT_MAX, clamp len to that */
+  len = std::min(len, (loff_t)INT_MAX);
+
+  std::scoped_lock lock(client_lock);
   if (fh == NULL || !_ll_fh_exists(fh)) {
     ldout(cct, 3) << "(fh)" << fh << " is invalid" << dendl;
     return -CEPHFS_EBADF;
@@ -15993,10 +15997,6 @@ int Client::ll_write(Fh *fh, loff_t off, loff_t len, const char *data)
   tout(cct) << off << std::endl;
   tout(cct) << len << std::endl;
 
-  /* We can't return bytes written larger than INT_MAX, clamp len to that */
-  len = std::min(len, (loff_t)INT_MAX);
-  std::scoped_lock lock(client_lock);
-
   int r = _write(fh, off, len, data, NULL, 0);
   ldout(cct, 3) << "ll_write " << fh << " " << off << "~" << len << " = " << r
 		<< dendl;
@@ -16010,12 +16010,11 @@ int64_t Client::ll_writev(struct Fh *fh, const struct iovec *iov, int iovcnt, in
     return -CEPHFS_ENOTCONN;
   }
 
+  std::scoped_lock cl(client_lock);
   if (fh == NULL || !_ll_fh_exists(fh)) {
     ldout(cct, 3) << "(fh)" << fh << " is invalid" << dendl;
     return -CEPHFS_EBADF;
   }
-
-  std::scoped_lock cl(client_lock);
   return _preadv_pwritev_locked(fh, iov, iovcnt, off, true, false);
 }
 
@@ -16026,12 +16025,11 @@ int64_t Client::ll_readv(struct Fh *fh, const struct iovec *iov, int iovcnt, int
     return -CEPHFS_ENOTCONN;
   }
 
+  std::scoped_lock cl(client_lock);
   if (fh == NULL || !_ll_fh_exists(fh)) {
     ldout(cct, 3) << "(fh)" << fh << " is invalid" << dendl;
     return -CEPHFS_EBADF;
   }
-
-  std::scoped_lock cl(client_lock);
   return _preadv_pwritev_locked(fh, iov, iovcnt, off, false, false);
 }
 
@@ -16054,18 +16052,24 @@ int64_t Client::ll_preadv_pwritev(struct Fh *fh, const struct iovec *iov,
       return retval;
     }
 
+    retval = 0;
+    std::unique_lock cl(client_lock);
+
     if(fh == NULL || !_ll_fh_exists(fh)) {
       ldout(cct, 3) << "(fh)" << fh << " is invalid" << dendl;
       retval = -CEPHFS_EBADF;
+    }
+
+    if (retval != 0) {
       if (onfinish != nullptr) {
+        cl.unlock();
         onfinish->complete(retval);
+        cl.lock();
         retval = 0;
       }
       return retval;
     }
 
-    std::scoped_lock cl(client_lock);
-
     retval = _preadv_pwritev_locked(fh, iov, iovcnt, offset, write, true,
                                     onfinish, bl, do_fsync, syncdataonly);
     /* There are two scenarios with each having two cases to handle here
@@ -16086,9 +16090,9 @@ int64_t Client::ll_preadv_pwritev(struct Fh *fh, const struct iovec *iov,
     if (retval < 0) {
       if (onfinish != nullptr) {
         //async io failed
-        client_lock.unlock();
+        cl.unlock();
         onfinish->complete(retval);
-        client_lock.lock();
+        cl.lock();
         /* async call should always return zero to caller and allow the
         caller to wait on callback for the actual errno/retval. */
         retval = 0;

src/rgw/rgw_auth_s3.cc

@@ -495,6 +495,8 @@ bool is_non_s3_op(RGWOpType op_type)
   case RGW_OP_DELETE_OIDC_PROVIDER:
   case RGW_OP_GET_OIDC_PROVIDER:
   case RGW_OP_LIST_OIDC_PROVIDERS:
+  case RGW_OP_ADD_CLIENTID_TO_OIDC_PROVIDER:
+  case RGW_OP_UPDATE_OIDC_PROVIDER_THUMBPRINT:
   case RGW_OP_PUBSUB_TOPIC_CREATE:
   case RGW_OP_PUBSUB_TOPICS_LIST:
   case RGW_OP_PUBSUB_TOPIC_GET:

src/rgw/rgw_iam_policy.cc

@@ -160,6 +160,8 @@ static const actpair actpairs[] =
  { "iam:DeleteOIDCProvider", iamDeleteOIDCProvider},
  { "iam:GetOIDCProvider", iamGetOIDCProvider},
  { "iam:ListOIDCProviders", iamListOIDCProviders},
+ { "iam:AddClientIdToOIDCProvider", iamAddClientIdToOIDCProvider},
+ { "iam:UpdateOIDCProviderThumbprint", iamUpdateOIDCProviderThumbprint},
  { "iam:TagRole", iamTagRole},
  { "iam:ListRoleTags", iamListRoleTags},
  { "iam:UntagRole", iamUntagRole},
@@ -1550,6 +1552,12 @@ const char* action_bit_string(uint64_t action) {
   case iamListOIDCProviders:
     return "iam:ListOIDCProviders";
 
+  case iamAddClientIdToOIDCProvider:
+    return "iam:AddClientIdToOIDCProvider";
+
+  case iamUpdateOIDCProviderThumbprint:
+    return "iam:UpdateOIDCProviderThumbprint";
+
   case iamTagRole:
     return "iam:TagRole";
 

src/rgw/rgw_iam_policy.h

@@ -143,6 +143,8 @@ enum {
   iamDeleteOIDCProvider,
   iamGetOIDCProvider,
   iamListOIDCProviders,
+  iamAddClientIdToOIDCProvider,
+  iamUpdateOIDCProviderThumbprint,
   iamTagRole,
   iamListRoleTags,
   iamUntagRole,

src/rgw/rgw_op_type.h

@@ -161,5 +161,7 @@ enum RGWOpType {
   RGW_OP_DELETE_OIDC_PROVIDER,
   RGW_OP_GET_OIDC_PROVIDER,
   RGW_OP_LIST_OIDC_PROVIDERS,
+  RGW_OP_ADD_CLIENTID_TO_OIDC_PROVIDER,
+  RGW_OP_UPDATE_OIDC_PROVIDER_THUMBPRINT,
 };
 

src/rgw/rgw_rest_iam.cc

@@ -45,6 +45,8 @@ static const std::unordered_map<std::string_view, op_generator> op_generators =
   {"ListOpenIDConnectProviders", [](const bufferlist& bl_post_body) -> RGWOp* {return new RGWListOIDCProviders;}},
   {"GetOpenIDConnectProvider", [](const bufferlist& bl_post_body) -> RGWOp* {return new RGWGetOIDCProvider;}},
   {"DeleteOpenIDConnectProvider", [](const bufferlist& bl_post_body) -> RGWOp* {return new RGWDeleteOIDCProvider;}},
+  {"AddClientIDToOpenIDConnectProvider", [](const bufferlist& bl_post_body) -> RGWOp* {return new RGWAddClientIdToOIDCProvider;}},
+  {"UpdateOpenIDConnectProviderThumbprint", [](const bufferlist& bl_post_body) -> RGWOp* {return new RGWUpdateOIDCProviderThumbprint;}},
   {"TagRole", [](const bufferlist& bl_post_body) -> RGWOp* {return new RGWTagRole(bl_post_body);}},
   {"ListRoleTags", [](const bufferlist& bl_post_body) -> RGWOp* {return new RGWListRoleTags;}},
   {"UntagRole", [](const bufferlist& bl_post_body) -> RGWOp* {return new RGWUntagRole(bl_post_body);}},