Skip to content

Commit 0c2b716

Browse files
chalmerlowechalmerlowe
authored
feat: updates the user instructions re OAuth (#603)
* updates the user instructions re OAuth * feat: updates the user instructions re OAuth * splits the content for highly constrained environments vs default environments * update toctree references
1 parent 98f1571 commit 0c2b716

File tree

2 files changed

+327
-0
lines changed

2 files changed

+327
-0
lines changed

‎docs/howto/authentication.rst

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,10 @@
11
Authentication
22
==============
33

4+
.. contents:: Table of Contents
5+
:local:
6+
:depth: 1
7+
48
Before you begin, you must create a Google Cloud Platform project. Use the
59
`BigQuery sandbox <https://cloud.google.com/bigquery/docs/sandbox>`__ to try
610
the service for free.
@@ -190,3 +194,25 @@ credentials are not found.
190194
Additional information on the user credentials authentication mechanism
191195
can be found in the `Google Cloud authentication guide
192196
<https://cloud.google.com/docs/authentication/end-user>`__.
197+
198+
Authenticating from Highly Constrained Development Environments
199+
---------------------------------------------------------------
200+
201+
The instructions above may not be adequate for users who are working in
202+
a *highly constrained development environment*:
203+
204+
Highly constrained development environments typically prevent users from using
205+
the `Default Authentication Methods` and are generally characterized by one or
206+
more of the following circumstances:
207+
208+
* There are limitations on what you can install on the development environment
209+
(i.e. you can't install ``gcloud``).
210+
* You don't have access to a graphical user interface (i.e. you are remotely
211+
SSH'ed into a headless server and don't have access to a browser to complete
212+
the authentication process used in the default login workflow) .
213+
* The code is being executed in a typical data science context: using a Jupyter
214+
(or similar) notebook.
215+
216+
If the conditions above apply to you, your needs may be better served
217+
by the content in the `Authentication (Highly Constrained Development Environment)
218+
<authentication_highly_constrained_environments.html>`_ section.

‎docs/howto/authentication_highly_constrained_environments.rst

Lines changed: 301 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,301 @@
1+
:orphan:
2+
3+
Authentication (Highly Constrained Development Environments)
4+
============================================================
5+
6+
Before you begin, you must create a Google Cloud Platform project. Use the
7+
`BigQuery sandbox <https://cloud.google.com/bigquery/docs/sandbox>`__ to try
8+
the service for free.
9+
10+
pandas-gbq `authenticates with the Google BigQuery service
11+
<https://cloud.google.com/bigquery/docs/authentication/>`_ via OAuth 2.0. Use
12+
the ``credentials`` argument to explicitly pass in Google
13+
:class:`~google.auth.credentials.Credentials`.
14+
15+
.. _authentication_hce:
16+
17+
18+
Authenticating from Highly Constrained Development Environments
19+
---------------------------------------------------------------
20+
21+
These instructions are primarily for users who are working in a *highly
22+
constrained development environment*.
23+
24+
Highly constrained development environments typically prevent users from using
25+
the `Default Authentication Methods` and are generally characterized by one or
26+
more of the following circumstances:
27+
28+
* There are limitations on what you can install on the development environment
29+
(i.e. you can't install ``gcloud``).
30+
* You don't have access to a graphical user interface (i.e. you are remotely
31+
SSH'ed into a headless server and don't have access to a browser to complete
32+
the authentication process used in the default login workflow) .
33+
* The code is being executed in a typical data science context: using a Jupyter
34+
(or similar) notebook.
35+
36+
If the conditions above **do not** apply to you, your needs may be better served
37+
by the content in the `Default Authentication Methods <authentication.html>`_ section.
38+
39+
When dealing with highly constrained environments, there are two primary options
40+
that one can choose from: Testing Mode OR an institution-specific authentication
41+
page.
42+
43+
#. Testing Mode: This approach requires that you enable Testing Mode on your
44+
Cloud Project and that you have fewer than 100 users.
45+
#. Institution-specific authentication page: In cases where the Testing Mode
46+
option is not possible and/or there are specific institutional needs,
47+
you/your institution can create and host an institution-specific OAuth
48+
authentication page and associate a redirect URI to that Cloud Project.
49+
50+
OPTION 1 - Testing Mode
51+
^^^^^^^^^^^^^^^^^^^^^^^
52+
53+
This approach is for limited use, such as when testing your product. It is not
54+
intended for production use. If you have fewer than 100 users, it is possible to
55+
configure User Type as External and the Publishing Status of your Project as
56+
Testing Mode to enable OAuth Out-of-Band (OOB) Authentication. NOTE: general
57+
purpose `OOB Authentication was deprecated <https://developers.googleblog.com/2022/02/making-oauth-flows-safer.html>`_ for all use cases except Testing Mode.
58+
59+
.. note:: Projects configured with a Publishing Status of Testing are **limited to
60+
up to 100 test users** who must be individually listed in the OAuth consent
61+
screen. A test user consumes a Project's test user quota once added to the
62+
Project.
63+
64+
Authentications by a test user **will expire seven days from the time of consent.** If your OAuth client requests an offline access type and receives a refresh token, that token will also expire.
65+
66+
To move a project from Testing Mode to In Production requires app verification
67+
and requires your institution to switch to using an alternate authentication
68+
method, such as an institution-specific authentication page.
69+
70+
The test users must be manually and individually added to your Cloud Project (i.e. you can not provide a group email alias for your development team because the system does not support alias expansion).
71+
72+
Google displays a warning message before allowing a specified test user to authenticate scopes requested by your Project's OAuth clients. The warning message confirms the user has test access to your Project and reminds them that they should consider the risks associated with granting access to their data to an unverified app.
73+
74+
For additional limitations and details about Testing Mode, see: `Setting up your OAuth consent screen <https://support.google.com/cloud/answer/10311615?hl=en#zippy=%2Ctesting>`_.
75+
76+
To enable Testing Mode and add users to your Cloud Project, in your Project dashboard:
77+
78+
#. Click on **APIs & Services > OAuth consent screen**.
79+
#. Select **External** to enable Testing Mode.
80+
#. Click **Create**.
81+
#. Fill in the necessary details related to the following:
82+
83+
#. App information
84+
#. App domain, including Authorized domains
85+
#. Developer contact information
86+
87+
#. Click **Save and Continue**.
88+
#. Click **Add or Remove Scopes** to choose appropriate Scopes for your Project. An *Update selected scopes* dialogue will open.
89+
#. Click **Update**.
90+
#. Click **Save and Continue**.
91+
#. Click **Add Users** to add users to your Project. An *Add Users* dialogue will open.
92+
#. Enter the user's name in the text field.
93+
#. Click **Add**.
94+
#. Click **Save and Continue**.
95+
#. A summary screen will display with all the information you entered.
96+
97+
To access BigQuery programmatically, you will need your Client ID and your Client Secret, which can be generated as follows:
98+
99+
#. Click **APIs and Services > Credentials**.
100+
#. Click **+ Create Credentials > OAuth client ID**.
101+
#. Select Desktop app In the **Application Type** field.
102+
#. Fill in the name of your OAuth 2.0 client in the **Name** field.
103+
#. Click **Create**.
104+
105+
Your Client ID and Client Secret are displayed in the pop-up. There is also a reminder that only test users that are listed on the Oauth consent screen can access the application. The client ID and Client Secret can also be found here, if they have already been generated:
106+
107+
#. Click on **APIs and Services > Credentials**.
108+
#. Click on the name of your OAuth 2.0 Client under **OAuth 2.0 Client IDs**.
109+
#. The Client ID and Client Secret will be displayed.
110+
111+
With the Client ID and Client Secret, you are ready to create an OAuth workflow using code similar to the following:
112+
113+
To run this code sample, you will need to have ``python-bigquery-pandas`` installed. The following dependencies will be installed by ``python-bigquery-pandas``:
114+
115+
* pydata-google-auth
116+
* google-auth
117+
* google-auth-oauthlib
118+
* pandas
119+
* google-cloud-bigquery
120+
* tqdm
121+
122+
**Sample code:** ``oauth-read-from-bq-testing-mode.py``
123+
124+
.. code:: python
125+
126+
import pandas_gbq
127+
128+
projectid = "your-project-name here"
129+
130+
CLIENT_ID = "your-client-id here"
131+
132+
# WARNING: for the purposes of this demo code, the Client Secret is
133+
# included here. In your script, take precautions to ensure
134+
# that your Client Secret does not get pushed to a public
135+
# repository or otherwise get compromised
136+
CLIENT_SECRET = "your-client-secret here"
137+
138+
df = pandas_gbq.read_gbq(
139+
"SELECT SESSION_USER() as user_id, CURRENT_TIMESTAMP() as time",
140+
project_id=projectid,
141+
auth_local_webserver=False,
142+
client_id=CLIENT_ID,
143+
client_secret=CLIENT_SECRET,
144+
)
145+
146+
print(df)
147+
148+
OPTION 2 - Institution-specific authentication page
149+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
150+
151+
To access Bigquery programmatically, you will need your Client ID and your Client Secret, an OAuth authorization page, and an assigned redirect URI.
152+
153+
To add a Client ID, Client Secret, and Redirect URI to your Cloud Project, in your Project dashboard:
154+
155+
#. Click on **APIs & Services > OAuth consent screen**.
156+
#. Select **Internal**.
157+
#. Click **Create**.
158+
#. Fill in the necessary details related to the following:
159+
160+
#. App information
161+
#. App domain, including Authorized domains
162+
#. Developer contact information
163+
164+
#. Click **Save and Continue**.
165+
#. Click **Add or Remove Scopes** to choose appropriate Scopes for your Project. An Update selected scopes dialogue will open.
166+
#. Click **Update**.
167+
#. Click **Save and Continue**.
168+
#. Click on **APIs and Services > Credentials**.
169+
#. Click on **+ Create Credentials > OAuth client ID**.
170+
#. Select Web application in the **Application Type** field.
171+
#. Fill in the name of your OAuth 2.0 client in the **Name** field.
172+
#. Click **Add Uri** under the Authorized Redirect URIs section.
173+
#. Add a URI for your application (i.e. the path to where you are hosting a file such as the ``oauth.html`` file shown below).
174+
#. Click **Create**.
175+
176+
Your Client ID and Client Secret will be displayed in the pop-up. The client ID and Client Secret can also be found here:
177+
178+
#. Click on **APIs and Services > Credentials**.
179+
#. Click on the name of your OAuth 2.0 Client under **OAuth 2.0 Client IDs**.
180+
#. The Client ID and Client Secret and the Authorized Redirect URIs will be displayed.
181+
182+
You will need to host a webpage (such as ``oauth.html``) with some associated javascript (such as shown below in ``authcodescripts.js``) to parse the results of the OAuth workflow.
183+
184+
**Code Sample**: ``oauth.html``
185+
186+
.. code:: html
187+
188+
<!DOCTYPE html>
189+
<html>
190+
<head>
191+
<script src="_static/js/authcodescripts.js"></script>
192+
</head>
193+
<body>
194+
<h1>Sign in to BigQuery</h1>
195+
<p>You are seeing this page because you are attempting to access BigQuery via one
196+
of several possible methods, including:</p>
197+
<blockquote>
198+
<div>
199+
<ul>
200+
<li><p>the <code><span>pandas_gbq</span></code> library (<a href="https://github.com/googleapis/python-bigquery-pandas">https://github.com/googleapis/python-bigquery-pandas</a>)</p></li>
201+
</ul>
202+
<p>OR a <code><span>pandas</span></code> library helper function such as:</p>
203+
<ul>
204+
<li><p><code><span>pandas.DataFrame.to_gbq()</span></code></p></li>
205+
<li><p><code><span>pandas.read_gbq()</span></code></p></li>
206+
</ul>
207+
</div>
208+
</blockquote>
209+
<p>from this or another machine. If this is not the case, close this tab.</p>
210+
<p>Enter the following verification code in the CommandLine Interface (CLI) on the
211+
machine you want to log into. This is a credential <strong>similar to your password</strong>
212+
and should not be shared with others.</p>
213+
<script type="text/javascript">
214+
window.addEventListener( "load", onloadoauthcode )
215+
</script>
216+
<div>
217+
<code class="auth-code"></code>
218+
</div>
219+
<br>
220+
<button class="copy" aria-live="assertive">Copy</button>
221+
</body>
222+
</html>
223+
224+
**Code Sample**: ``authcodescripts.js``
225+
226+
.. code:: javascript
227+
228+
function onloadoauthcode() {
229+
const PARAMS = new Proxy(new URLSearchParams(window.location.search), {
230+
get: (searchParams, prop) => searchParams.get(prop),
231+
});
232+
const AUTH_CODE = PARAMS.code;
233+
document.querySelector('.auth-code').textContent = AUTH_CODE;
234+
setupCopyButton(document.querySelector('.copy'), AUTH_CODE);
235+
}
236+
237+
function setupCopyButton(button, text) {
238+
button.addEventListener('click', () => {
239+
navigator.clipboard.writeText(text);
240+
button.textContent = "Verification Code Copied";
241+
setTimeout(() => {
242+
// Remove the aria-live label so that when the
243+
// button text changes back to "Copy", it is
244+
// not read out.
245+
button.removeAttribute("aria-live");
246+
button.textContent = "Copy";
247+
}, 1000);
248+
249+
// Re-Add the aria-live attribute to enable speech for
250+
// when button text changes next time.
251+
setTimeout(() => {
252+
button.setAttribute("aria-live", "assertive");
253+
}, 2000);
254+
});
255+
}
256+
257+
With these items in place:
258+
259+
* Client ID
260+
* Client Secret
261+
* redirect URI
262+
* authentication page
263+
264+
you are ready to create an OAuth workflow using code similar to the following:
265+
266+
To run this code sample, you will need to have ``python-bigquery-pandas`` installed. The following dependencies will be installed by ``python-bigquery-pandas``:
267+
268+
* pydata-google-auth
269+
* google-auth
270+
* google-auth-oauthlib
271+
* pandas
272+
* google-cloud-bigquery
273+
* tqdm
274+
275+
**Sample Code**: ``oauth-read-from-bq-org-specific.py``
276+
277+
.. code:: python
278+
279+
import pandas_gbq
280+
281+
projectid = "your-project-name-here"
282+
283+
REDIRECT_URI = "your-redirect-uri here/oauth.html"
284+
CLIENT_ID = "your-client-id here"
285+
286+
# WARNING: for the purposes of this demo code, the Client Secret is
287+
# included here. In your script, take precautions to ensure
288+
# that your Client Secret does not get pushed to a public
289+
# repository or otherwise compromised
290+
CLIENT_SECRET = "your-client-secret here"
291+
292+
df = pandas_gbq.read_gbq(
293+
"SELECT SESSION_USER() as user_id, CURRENT_TIMESTAMP() as time",
294+
project_id=projectid,
295+
auth_local_webserver=False,
296+
auth_redirect_uri=REDIRECT_URI,
297+
client_id=CLIENT_ID,
298+
client_secret=CLIENT_SECRET,
299+
)
300+
301+
print(df)

0 commit comments

Comments
 (0)