|
1 | | -Python Client for Google BigQuery connector for pandas |
2 | | -====================================================== |
| 1 | +pandas-gbq |
| 2 | +========== |
3 | 3 |
|
4 | 4 | |preview| |pypi| |versions| |
5 | 5 |
|
6 | | -`Google BigQuery connector for pandas`_: |
| 6 | +**pandas-gbq** is a package providing an interface to the Google BigQuery API from pandas. |
7 | 7 |
|
8 | | -- `Client Library Documentation`_ |
9 | | -- `Product Documentation`_ |
| 8 | +- `Library Documentation`_ |
| 9 | +- `Product Documentation`_ |
10 | 10 |
|
11 | 11 | .. |preview| image:: https://img.shields.io/badge/support-preview-orange.svg |
12 | 12 | :target: https://github.com/googleapis/google-cloud-python/blob/main/README.rst#stability-levels |
13 | 13 | .. |pypi| image:: https://img.shields.io/pypi/v/pandas-gbq.svg |
14 | 14 | :target: https://pypi.org/project/pandas-gbq/ |
15 | 15 | .. |versions| image:: https://img.shields.io/pypi/pyversions/pandas-gbq.svg |
16 | 16 | :target: https://pypi.org/project/pandas-gbq/ |
17 | | -.. _Google BigQuery connector for pandas: https://cloud.google.com/bigquery |
18 | | -.. _Client Library Documentation: https://googleapis.dev/python/pandas-gbq/latest/ |
19 | | -.. _Product Documentation: https://cloud.google.com/bigquery |
20 | | - |
21 | | -Quick Start |
22 | | ------------ |
23 | | - |
24 | | -In order to use this library, you first need to go through the following steps: |
25 | | - |
26 | | -1. `Select or create a Cloud Platform project.`_ |
27 | | -2. `Enable billing for your project.`_ |
28 | | -3. `Enable the Google BigQuery connector for pandas.`_ |
29 | | -4. `Set up Authentication.`_ |
30 | | - |
31 | | -.. _Select or create a Cloud Platform project.: https://console.cloud.google.com/project |
32 | | -.. _Enable billing for your project.: https://cloud.google.com/billing/docs/how-to/modify-project#enable_billing_for_a_project |
33 | | -.. _Enable the Google BigQuery connector for pandas.: https://cloud.google.com/bigquery |
34 | | -.. _Set up Authentication.: https://googleapis.dev/python/google-api-core/latest/auth.html |
| 17 | +.. _Library Documentation: https://googleapis.dev/python/pandas-gbq/latest/ |
| 18 | +.. _Product Documentation: https://cloud.google.com/bigquery/docs/reference/v2/ |
35 | 19 |
|
36 | 20 | Installation |
37 | | -~~~~~~~~~~~~ |
38 | | - |
39 | | -Install this library in a virtual environment using `venv`_. `venv`_ is a tool that |
40 | | -creates isolated Python environments. These isolated environments can have separate |
41 | | -versions of Python packages, which allows you to isolate one project's dependencies |
42 | | -from the dependencies of other projects. |
43 | | - |
44 | | -With `venv`_, it's possible to install this library without needing system |
45 | | -install permissions, and without clashing with the installed system |
46 | | -dependencies. |
47 | | - |
48 | | -.. _`venv`: https://docs.python.org/3/library/venv.html |
49 | | - |
50 | | - |
51 | | -Code samples and snippets |
52 | | -~~~~~~~~~~~~~~~~~~~~~~~~~ |
53 | | - |
54 | | -Code samples and snippets live in the `samples/`_ folder. |
55 | | - |
56 | | -.. _samples/: https://github.com/googleapis/google-cloud-python/tree/main/packages/pandas-gbq/samples |
57 | | - |
58 | | - |
59 | | -Supported Python Versions |
60 | | -^^^^^^^^^^^^^^^^^^^^^^^^^ |
61 | | -Our client libraries are compatible with all current `active`_ and `maintenance`_ versions of |
62 | | -Python. |
63 | | - |
64 | | -Python >= 3.7 |
65 | | - |
66 | | -.. _active: https://devguide.python.org/devcycle/#in-development-main-branch |
67 | | -.. _maintenance: https://devguide.python.org/devcycle/#maintenance-branches |
68 | | - |
69 | | -Unsupported Python Versions |
70 | | -^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
71 | | -Python <= 3.6 |
72 | | - |
73 | | -If you are using an `end-of-life`_ |
74 | | -version of Python, we recommend that you update as soon as possible to an actively supported version. |
75 | | - |
76 | | -.. _end-of-life: https://devguide.python.org/devcycle/#end-of-life-branches |
77 | | - |
78 | | -Mac/Linux |
79 | | -^^^^^^^^^ |
80 | | - |
81 | | -.. code-block:: console |
82 | | -
|
83 | | - python3 -m venv <your-env> |
84 | | - source <your-env>/bin/activate |
85 | | - pip install pandas-gbq |
86 | | -
|
| 21 | +------------ |
87 | 22 |
|
88 | | -Windows |
89 | | -^^^^^^^ |
| 23 | +Install latest release version via pip |
| 24 | +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
90 | 25 |
|
91 | | -.. code-block:: console |
| 26 | +.. code-block:: shell |
92 | 27 |
|
93 | | - py -m venv <your-env> |
94 | | - .\<your-env>\Scripts\activate |
95 | | - pip install pandas-gbq |
| 28 | + $ pip install pandas-gbq |
96 | 29 |
|
97 | | -Next Steps |
98 | | -~~~~~~~~~~ |
99 | | - |
100 | | -- Read the `Client Library Documentation`_ for Google BigQuery connector for pandas |
101 | | - to see other available methods on the client. |
102 | | -- Read the `Google BigQuery connector for pandas Product documentation`_ to learn |
103 | | - more about the product and see How-to Guides. |
104 | | -- View this `README`_ to see the full list of Cloud |
105 | | - APIs that we cover. |
106 | | - |
107 | | -.. _Google BigQuery connector for pandas Product documentation: https://cloud.google.com/bigquery |
108 | | -.. _README: https://github.com/googleapis/google-cloud-python/blob/main/README.rst |
109 | | - |
110 | | -Logging |
111 | | -------- |
112 | | - |
113 | | -This library uses the standard Python :code:`logging` functionality to log some RPC events that could be of interest for debugging and monitoring purposes. |
114 | | -Note the following: |
115 | | - |
116 | | -#. Logs may contain sensitive information. Take care to **restrict access to the logs** if they are saved, whether it be on local storage or on Google Cloud Logging. |
117 | | -#. Google may refine the occurrence, level, and content of various log messages in this library without flagging such changes as breaking. **Do not depend on immutability of the logging events**. |
118 | | -#. By default, the logging events from this library are not handled. You must **explicitly configure log handling** using one of the mechanisms below. |
119 | | - |
120 | | -Simple, environment-based configuration |
121 | | -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
122 | | - |
123 | | -To enable logging for this library without any changes in your code, set the :code:`GOOGLE_SDK_PYTHON_LOGGING_SCOPE` environment variable to a valid Google |
124 | | -logging scope. This configures handling of logging events (at level :code:`logging.DEBUG` or higher) from this library in a default manner, emitting the logged |
125 | | -messages in a structured format. It does not currently allow customizing the logging levels captured nor the handlers, formatters, etc. used for any logging |
126 | | -event. |
127 | | - |
128 | | -A logging scope is a period-separated namespace that begins with :code:`google`, identifying the Python module or package to log. |
129 | | - |
130 | | -- Valid logging scopes: :code:`google`, :code:`google.cloud.asset.v1`, :code:`google.api`, :code:`google.auth`, etc. |
131 | | -- Invalid logging scopes: :code:`foo`, :code:`123`, etc. |
132 | | - |
133 | | -**NOTE**: If the logging scope is invalid, the library does not set up any logging handlers. |
134 | | - |
135 | | -Environment-Based Examples |
136 | | -^^^^^^^^^^^^^^^^^^^^^^^^^^ |
137 | | - |
138 | | -- Enabling the default handler for all Google-based loggers |
139 | | - |
140 | | -.. code-block:: console |
141 | | -
|
142 | | - export GOOGLE_SDK_PYTHON_LOGGING_SCOPE=google |
143 | | -
|
144 | | -- Enabling the default handler for a specific Google module (for a client library called :code:`library_v1`): |
145 | | - |
146 | | -.. code-block:: console |
147 | | -
|
148 | | - export GOOGLE_SDK_PYTHON_LOGGING_SCOPE=google.cloud.library_v1 |
| 30 | +Install latest development version |
| 31 | +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
149 | 32 |
|
| 33 | +.. code-block:: shell |
150 | 34 |
|
151 | | -Advanced, code-based configuration |
152 | | -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 35 | + $ pip install "pandas-gbq @ git+https://github.com/googleapis/google-cloud-python#egg=pandas-gbq&subdirectory=packages/pandas-gbq" |
153 | 36 |
|
154 | | -You can also configure a valid logging scope using Python's standard `logging` mechanism. |
155 | 37 |
|
156 | | -Code-Based Examples |
157 | | -^^^^^^^^^^^^^^^^^^^ |
| 38 | +Usage |
| 39 | +----- |
158 | 40 |
|
159 | | -- Configuring a handler for all Google-based loggers |
| 41 | +Perform a query |
| 42 | +~~~~~~~~~~~~~~~ |
160 | 43 |
|
161 | | -.. code-block:: python |
| 44 | +.. code:: python |
162 | 45 |
|
163 | | - import logging |
164 | | - |
165 | | - from google.cloud import library_v1 |
166 | | - |
167 | | - base_logger = logging.getLogger("google") |
168 | | - base_logger.addHandler(logging.StreamHandler()) |
169 | | - base_logger.setLevel(logging.DEBUG) |
| 46 | + import pandas_gbq |
170 | 47 |
|
171 | | -- Configuring a handler for a specific Google module (for a client library called :code:`library_v1`): |
| 48 | + result_dataframe = pandas_gbq.read_gbq("SELECT column FROM dataset.table WHERE value = 'something'") |
172 | 49 |
|
173 | | -.. code-block:: python |
| 50 | +Upload a dataframe |
| 51 | +~~~~~~~~~~~~~~~~~~ |
174 | 52 |
|
175 | | - import logging |
176 | | - |
177 | | - from google.cloud import library_v1 |
178 | | - |
179 | | - base_logger = logging.getLogger("google.cloud.library_v1") |
180 | | - base_logger.addHandler(logging.StreamHandler()) |
181 | | - base_logger.setLevel(logging.DEBUG) |
| 53 | +.. code:: python |
182 | 54 |
|
183 | | -Logging details |
184 | | -~~~~~~~~~~~~~~~ |
| 55 | + import pandas_gbq |
185 | 56 |
|
186 | | -#. Regardless of which of the mechanisms above you use to configure logging for this library, by default logging events are not propagated up to the root |
187 | | - logger from the `google`-level logger. If you need the events to be propagated to the root logger, you must explicitly set |
188 | | - :code:`logging.getLogger("google").propagate = True` in your code. |
189 | | -#. You can mix the different logging configurations above for different Google modules. For example, you may want use a code-based logging configuration for |
190 | | - one library, but decide you need to also set up environment-based logging configuration for another library. |
| 57 | + pandas_gbq.to_gbq(dataframe, "dataset.table") |
191 | 58 |
|
192 | | - #. If you attempt to use both code-based and environment-based configuration for the same module, the environment-based configuration will be ineffectual |
193 | | - if the code -based configuration gets applied first. |
| 59 | +More samples |
| 60 | +~~~~~~~~~~~~ |
194 | 61 |
|
195 | | -#. The Google-specific logging configurations (default handlers for environment-based configuration; not propagating logging events to the root logger) get |
196 | | - executed the first time *any* client library is instantiated in your application, and only if the affected loggers have not been previously configured. |
197 | | - (This is the reason for 2.i. above.) |
| 62 | +See the `pandas-gbq documentation <https://googleapis.dev/python/pandas-gbq/latest/>`_ for more details. |
0 commit comments