Which authentication type to use

[shaileshkm]

Hi,
I'm working on code where I need to read/write data in customer's own Google account. I need to get account credentials from user when these routines execute.
I'm using C++ SDK for my application.
I need help in deciding which authentication type to use.

• App key: I did not find any API in C++ SDK. This also does not look secure from customer pov as it never expires.
• Oauth2:
  ** API to use is CreateAuthorizedUserCredentialsFromJsonContents
  ** I need to accept from user: Client_id, client_secret, refresh_token
  ** Is quota_project_id optional? I did not get error if I omit it.
• Service account:
  ** API to use is CreateServiceAccountCredentialsFromJsonContents
  ** I need to accept from user: Private_key_id, private_key, client_email, client_id, client_x509_cert_url
  ** Is project_id required?
  ** The credentials don't seem to expire. Isn't it like app key then? Ofcourse user may disable the service account.

It looks like OAuth2 is a better fit to me. Can someone comment?

[coryan]

I'm working on code where I need to read/write data in customer's own Google account.

What kind of data? What service is keeping this data? Google Cloud Storage? Spanner? Bigtable? Google Drive?

I need to get account credentials from user when these routines execute.

Ack.

It looks like OAuth2 is a better fit to me. Can someone comment?

My first impression is that you do, yes. You may need something like #1718, or at least #1557. But the specific answer depends on the service you want to use.

If you let us know what service you are planning to use I can tell you if our current roadmap would help or not.

[shaileshkm]

Hi,
Thanks for being patient with my questions.

I'm working on server side service code which would need to upload/download data in user's storage account.
The service itself may host data.
Access to this service code would be via client tool which would accept specific commands such as 'import data from GCP path'/'export data to GCP path'. Data my be large, maybe in excess of 100Gb.
Typical use case would be data backup/restore, but there are other use cases.

The client tool is kind of dumb to not access client side environment. So we need commands to tell the service all the necessary credential information. When I said command line, the commands issued from the client tool may look like a text command issued from a command line. The client tool authenticates itself using proprietary protocol with our own user details.

Does this clarify?

About gcloud commands,

• On server side, I need to use C++ APIs. Cannot rely on CLI.
• I have experimented with 'gcloud auth application-default login' to get Authorized user credentials. What's the command for service account credentials?
• What would be the equivalent in C++ API for these commands? Maybe that's what I need.

For my prototype, I created a service account, downloaded JSON credentials file and able to use API CreateServiceAccountCredentialsFromJsonFilePath.
However, as I said earlier, number of inputs required (as part of JSON creds) are kind of too many (5) and quite large.

[coryan]

I'm working on server side service code which would need to upload/download data in user's storage account.

I don't think of GCS data as being associated with a user's account. The data is contained in a GCS bucket, the bucket is "owned" by a GCP project. A user account has access to the data in that project. This access can be just read, read-write, or even permissions to delete the bucket.

This distinction is probably unimportant in your case. It can be relevant if there is one project (or a small number of projects) that your service / tool needs access to. If all the data was in a small number of projects then you could create a service account, grant that account the necessary GCS read permissions across the projects, and then run the service under that service account. No need to authenticate each user.

You probably thought of that already, so let's try to fix your actual problem.

The service itself may host data.
Access to this service code would be via client tool which would accept specific commands such as 'import data from GCP path'/'export data to GCP path'. Data my be large, maybe in excess of 100Gb.

For back of the envelope calculations, a single thread on an GCE VM can download 250MiB/s, that is 4 seconds per GiB, or 400 seconds for the full 100GB. The uploads are more like 100MiB/s, so maybe 1,000 seconds. This is to say that 1 hour seems like plenty of time, so maybe access tokens could help. OTOH, I see below why my thinking might be wrong.

nit: I assume you mean GB (gigabytes), and not Gb (gigabits).

Typical use case would be data backup/restore, but there are other use cases.

Ah, maybe you need to do these on a schedule, and therefore an access tokens would expire.

The client tool is kind of dumb to not access client side environment. So we need commands to tell the service all the necessary credential information. When I said command line, the commands issued from the client tool may look like a text command issued from a command line. The client tool authenticates itself using proprietary protocol with our own user details.

Does this clarify?

It does.

About gcloud commands,

• On server side, I need to use C++ APIs. Cannot rely on CLI.
• I have experimented with 'gcloud auth application-default login' to get Authorized user credentials. What's the command for service account credentials?

gcloud auth activate-service-account is what comes to mind.

• What would be the equivalent in C++ API for these commands? Maybe that's what I need.

Let me try to answer that last question. What you seem want is for the user to give your applications the right permissions to access their resources. That is what OAuth2 is for. I think one is supposed to design the application to use the minimal, shortest lived tokens that are strictly needed, that is to protect the user.

• If you need these permissions for a short time (say 1 hour) then you need an access token.
• If you need these permissions for a long time (say days or weeks to perform scheduled backups) then you need a refresh token.

Let's take each in turn.

• For an access token your client-side CLI needs to get an access token and pass it to your service. Currently we do not have a credential type to support this (see Additional credential type: oauth2 access token, non-refreshing #1557). I am actively working on this stuff, but if you cannot wait:

  • Create a class derived from google::cloud::storage::oauth2::Credentials
  • Override the AuthorizationHeader() member function to return Authorization: Bearer <access-token-here>.

• If you need a refresh token then your client-side program has to either

  • Find one (say in the standard location placed by gcloud), or
  • Ask the user for one

To ask the user you will need to implement the OAuth2 workflows in the client-side application. You can do this yourself, or launch gcloud auth login to do it for you.

Once you have the refresh token then send it to your service, and load it with:

google-cloud-cpp/google/cloud/storage/oauth2/google_credentials.h

Lines 72 to 73 in d349a4e

	CreateAuthorizedUserCredentialsFromJsonContents(
	std::string const& contents, ChannelOptions const& options = {});

Or if you prefer, fill up this object:

google-cloud-cpp/google/cloud/storage/oauth2/authorized_user_credentials.h

Lines 36 to 41 in d349a4e

	struct AuthorizedUserCredentialsInfo {
	std::string client_id;
	std::string client_secret;
	std::string refresh_token;
	std::string token_uri;
	};

And create an instance of:

google-cloud-cpp/google/cloud/storage/oauth2/authorized_user_credentials.h

Lines 82 to 83 in d349a4e

	explicit AuthorizedUserCredentials(AuthorizedUserCredentialsInfo const& info,
	ChannelOptions const& channel_options = {})

Then use those credentials to initialize google::cloud::storage::Client.

For my prototype, I created a service account, downloaded JSON credentials file and able to use API CreateServiceAccountCredentialsFromJsonFilePath.
However, as I said earlier, number of inputs required (as part of JSON creds) are kind of too many (5) and quite large.

If you are looking for a simple "username & password" approach then you will not find it. Leaving aside the security implications of your service holding a password for a user: what if the user has enabled two factor authentication? Then the service simply cannot authenticate as the user with just the password, it also needs that 2FA generator, which is typically a physical device. If you want to act of behalf of a user, you need a token that represents the user granting you permissions to act on their behalf. That is either the access token (which are temporary) or a refresh token.

Words to the wise, if you do end up storing these refresh tokens, please use a secure location like a secret manager. I am sure all the cloud providers have one, the one I know of is obviously Google's.

[shaileshkm]

Thanks for an elaborate answer.

To answer some your questions/points:

• Since we need to support multi-tenancy, I can not assume specific service account configuration.
• About data, there is no upper cap on data to upload/download. So 1 hour restriction could still come in play.
• Right now, the requirement is to execute commands interactively. Anyway, we do not plan to cache user credentials.
• We need to support current clients, so changing it to read tokens from gcloud configuration path or OAuth2 workflow support is not feasible.

My takeaway is

• Wait for the release of the new credential type support. I’m not sure what it will accept. Minimally, I think, it should accept access_token and refresh token and internally keep refreshing access token till the validity of refresh_token.
• Rely on OAuth2 using API CreateAuthorizedUserCredentialsFromJsonContents.
  • I will need to obtain details like client_id, client_secret, refresh_token as part of CLI interface. Do you know if 'quota_project_id' is mandatory. In my testing, it was not required.

Thanks for your help. I’m looking forward to improvements in credential management.

[coryan]

My takeaway is

• Wait for the release of the new credential type support. I’m not sure what it will accept. Minimally, I think, it should accept access_token and refresh token and internally keep refreshing access token till the validity of refresh_token.

We are planning to support both, yes.

• Rely on OAuth2 using API CreateAuthorizedUserCredentialsFromJsonContents.

That should work now, and continue to work with the new credentials. There is going to be some overlap / redundancy. Generally all the stuff that works today will continue to work, new features will go into the new credentials though.

• I will need to obtain details like client_id, client_secret, refresh_token as part of CLI interface. Do you know if 'quota_project_id' is mandatory. In my testing, it was not required.

AFAIK, quota_project_id is not required.

Note that client_id and client_secret identify your application in the OAuth2 workflow, you should be able to keep those in the server, and not ask them from the user. You need to register your application with google and have your application run the OAuth2 workflow. This is a bit outside my area of expertise, but this might be a good start:

https://cloud.google.com/docs/authentication/end-user

Note the link to:

https://console.cloud.google.com/apis/credentials

Note that command-line tools can run the OAuth2 workflows, that is what gcloud does. I am not saying that is easy or convenient, of course.