|
1 | | -pandas-gbq |
2 | | -========== |
| 1 | +Python Client for Google BigQuery connector for pandas |
| 2 | +====================================================== |
3 | 3 |
|
4 | 4 | |preview| |pypi| |versions| |
5 | 5 |
|
6 | | -**pandas-gbq** is a package providing an interface to the Google BigQuery API from pandas. |
| 6 | +`Google BigQuery connector for pandas`_: |
7 | 7 |
|
8 | | -- `Library Documentation`_ |
9 | | -- `Product Documentation`_ |
| 8 | +- `Client Library Documentation`_ |
| 9 | +- `Product Documentation`_ |
10 | 10 |
|
11 | 11 | .. |preview| image:: https://img.shields.io/badge/support-preview-orange.svg |
12 | | - :target: https://github.com/googleapis/google-cloud-python/blob/main/README.rst#beta-support |
| 12 | + :target: https://github.com/googleapis/google-cloud-python/blob/main/README.rst#stability-levels |
13 | 13 | .. |pypi| image:: https://img.shields.io/pypi/v/pandas-gbq.svg |
14 | 14 | :target: https://pypi.org/project/pandas-gbq/ |
15 | 15 | .. |versions| image:: https://img.shields.io/pypi/pyversions/pandas-gbq.svg |
16 | 16 | :target: https://pypi.org/project/pandas-gbq/ |
17 | | -.. _Library Documentation: https://googleapis.dev/python/pandas-gbq/latest/ |
18 | | -.. _Product Documentation: https://cloud.google.com/bigquery/docs/reference/v2/ |
| 17 | +.. _Google BigQuery connector for pandas: https://cloud.google.com/bigquery |
| 18 | +.. _Client Library Documentation: https://googleapis.dev/python/pandas-gbq/latest/ |
| 19 | +.. _Product Documentation: https://cloud.google.com/bigquery |
| 20 | + |
| 21 | +Quick Start |
| 22 | +----------- |
| 23 | + |
| 24 | +In order to use this library, you first need to go through the following steps: |
| 25 | + |
| 26 | +1. `Select or create a Cloud Platform project.`_ |
| 27 | +2. `Enable billing for your project.`_ |
| 28 | +3. `Enable the Google BigQuery connector for pandas.`_ |
| 29 | +4. `Set up Authentication.`_ |
| 30 | + |
| 31 | +.. _Select or create a Cloud Platform project.: https://console.cloud.google.com/project |
| 32 | +.. _Enable billing for your project.: https://cloud.google.com/billing/docs/how-to/modify-project#enable_billing_for_a_project |
| 33 | +.. _Enable the Google BigQuery connector for pandas.: https://cloud.google.com/bigquery |
| 34 | +.. _Set up Authentication.: https://googleapis.dev/python/google-api-core/latest/auth.html |
19 | 35 |
|
20 | 36 | Installation |
21 | | ------------- |
| 37 | +~~~~~~~~~~~~ |
22 | 38 |
|
23 | | -Install latest release version via pip |
24 | | -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 39 | +Install this library in a virtual environment using `venv`_. `venv`_ is a tool that |
| 40 | +creates isolated Python environments. These isolated environments can have separate |
| 41 | +versions of Python packages, which allows you to isolate one project's dependencies |
| 42 | +from the dependencies of other projects. |
25 | 43 |
|
26 | | -.. code-block:: shell |
| 44 | +With `venv`_, it's possible to install this library without needing system |
| 45 | +install permissions, and without clashing with the installed system |
| 46 | +dependencies. |
27 | 47 |
|
28 | | - $ pip install pandas-gbq |
| 48 | +.. _`venv`: https://docs.python.org/3/library/venv.html |
29 | 49 |
|
30 | | -Install latest development version |
31 | | -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
32 | 50 |
|
33 | | -.. code-block:: shell |
| 51 | +Code samples and snippets |
| 52 | +~~~~~~~~~~~~~~~~~~~~~~~~~ |
34 | 53 |
|
35 | | - $ pip install git+https://github.com/googleapis/python-bigquery-pandas.git |
| 54 | +Code samples and snippets live in the `samples/`_ folder. |
36 | 55 |
|
| 56 | +.. _samples/: https://github.com/googleapis/google-cloud-python/tree/main/packages/pandas-gbq/samples |
37 | 57 |
|
38 | | -Usage |
39 | | ------ |
40 | 58 |
|
41 | | -Perform a query |
42 | | -~~~~~~~~~~~~~~~ |
| 59 | +Supported Python Versions |
| 60 | +^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 61 | +Our client libraries are compatible with all current `active`_ and `maintenance`_ versions of |
| 62 | +Python. |
43 | 63 |
|
44 | | -.. code:: python |
| 64 | +Python >= 3.7 |
45 | 65 |
|
46 | | - import pandas_gbq |
| 66 | +.. _active: https://devguide.python.org/devcycle/#in-development-main-branch |
| 67 | +.. _maintenance: https://devguide.python.org/devcycle/#maintenance-branches |
47 | 68 |
|
48 | | - result_dataframe = pandas_gbq.read_gbq("SELECT column FROM dataset.table WHERE value = 'something'") |
| 69 | +Unsupported Python Versions |
| 70 | +^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 71 | +Python <= 3.6 |
49 | 72 |
|
50 | | -Upload a dataframe |
51 | | -~~~~~~~~~~~~~~~~~~ |
| 73 | +If you are using an `end-of-life`_ |
| 74 | +version of Python, we recommend that you update as soon as possible to an actively supported version. |
52 | 75 |
|
53 | | -.. code:: python |
| 76 | +.. _end-of-life: https://devguide.python.org/devcycle/#end-of-life-branches |
54 | 77 |
|
55 | | - import pandas_gbq |
| 78 | +Mac/Linux |
| 79 | +^^^^^^^^^ |
56 | 80 |
|
57 | | - pandas_gbq.to_gbq(dataframe, "dataset.table") |
| 81 | +.. code-block:: console |
58 | 82 |
|
59 | | -More samples |
60 | | -~~~~~~~~~~~~ |
| 83 | + python3 -m venv <your-env> |
| 84 | + source <your-env>/bin/activate |
| 85 | + pip install pandas-gbq |
| 86 | +
|
| 87 | +
|
| 88 | +Windows |
| 89 | +^^^^^^^ |
| 90 | + |
| 91 | +.. code-block:: console |
| 92 | +
|
| 93 | + py -m venv <your-env> |
| 94 | + .\<your-env>\Scripts\activate |
| 95 | + pip install pandas-gbq |
| 96 | +
|
| 97 | +Next Steps |
| 98 | +~~~~~~~~~~ |
| 99 | + |
| 100 | +- Read the `Client Library Documentation`_ for Google BigQuery connector for pandas |
| 101 | + to see other available methods on the client. |
| 102 | +- Read the `Google BigQuery connector for pandas Product documentation`_ to learn |
| 103 | + more about the product and see How-to Guides. |
| 104 | +- View this `README`_ to see the full list of Cloud |
| 105 | + APIs that we cover. |
| 106 | + |
| 107 | +.. _Google BigQuery connector for pandas Product documentation: https://cloud.google.com/bigquery |
| 108 | +.. _README: https://github.com/googleapis/google-cloud-python/blob/main/README.rst |
| 109 | + |
| 110 | +Logging |
| 111 | +------- |
| 112 | + |
| 113 | +This library uses the standard Python :code:`logging` functionality to log some RPC events that could be of interest for debugging and monitoring purposes. |
| 114 | +Note the following: |
| 115 | + |
| 116 | +#. Logs may contain sensitive information. Take care to **restrict access to the logs** if they are saved, whether it be on local storage or on Google Cloud Logging. |
| 117 | +#. Google may refine the occurrence, level, and content of various log messages in this library without flagging such changes as breaking. **Do not depend on immutability of the logging events**. |
| 118 | +#. By default, the logging events from this library are not handled. You must **explicitly configure log handling** using one of the mechanisms below. |
| 119 | + |
| 120 | +Simple, environment-based configuration |
| 121 | +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 122 | + |
| 123 | +To enable logging for this library without any changes in your code, set the :code:`GOOGLE_SDK_PYTHON_LOGGING_SCOPE` environment variable to a valid Google |
| 124 | +logging scope. This configures handling of logging events (at level :code:`logging.DEBUG` or higher) from this library in a default manner, emitting the logged |
| 125 | +messages in a structured format. It does not currently allow customizing the logging levels captured nor the handlers, formatters, etc. used for any logging |
| 126 | +event. |
| 127 | + |
| 128 | +A logging scope is a period-separated namespace that begins with :code:`google`, identifying the Python module or package to log. |
| 129 | + |
| 130 | +- Valid logging scopes: :code:`google`, :code:`google.cloud.asset.v1`, :code:`google.api`, :code:`google.auth`, etc. |
| 131 | +- Invalid logging scopes: :code:`foo`, :code:`123`, etc. |
| 132 | + |
| 133 | +**NOTE**: If the logging scope is invalid, the library does not set up any logging handlers. |
| 134 | + |
| 135 | +Environment-Based Examples |
| 136 | +^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 137 | + |
| 138 | +- Enabling the default handler for all Google-based loggers |
| 139 | + |
| 140 | +.. code-block:: console |
| 141 | +
|
| 142 | + export GOOGLE_SDK_PYTHON_LOGGING_SCOPE=google |
| 143 | +
|
| 144 | +- Enabling the default handler for a specific Google module (for a client library called :code:`library_v1`): |
| 145 | + |
| 146 | +.. code-block:: console |
| 147 | +
|
| 148 | + export GOOGLE_SDK_PYTHON_LOGGING_SCOPE=google.cloud.library_v1 |
| 149 | +
|
| 150 | +
|
| 151 | +Advanced, code-based configuration |
| 152 | +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 153 | + |
| 154 | +You can also configure a valid logging scope using Python's standard `logging` mechanism. |
| 155 | + |
| 156 | +Code-Based Examples |
| 157 | +^^^^^^^^^^^^^^^^^^^ |
| 158 | + |
| 159 | +- Configuring a handler for all Google-based loggers |
| 160 | + |
| 161 | +.. code-block:: python |
| 162 | +
|
| 163 | + import logging |
| 164 | + |
| 165 | + from google.cloud import library_v1 |
| 166 | + |
| 167 | + base_logger = logging.getLogger("google") |
| 168 | + base_logger.addHandler(logging.StreamHandler()) |
| 169 | + base_logger.setLevel(logging.DEBUG) |
| 170 | +
|
| 171 | +- Configuring a handler for a specific Google module (for a client library called :code:`library_v1`): |
| 172 | + |
| 173 | +.. code-block:: python |
| 174 | +
|
| 175 | + import logging |
| 176 | + |
| 177 | + from google.cloud import library_v1 |
| 178 | + |
| 179 | + base_logger = logging.getLogger("google.cloud.library_v1") |
| 180 | + base_logger.addHandler(logging.StreamHandler()) |
| 181 | + base_logger.setLevel(logging.DEBUG) |
| 182 | +
|
| 183 | +Logging details |
| 184 | +~~~~~~~~~~~~~~~ |
| 185 | + |
| 186 | +#. Regardless of which of the mechanisms above you use to configure logging for this library, by default logging events are not propagated up to the root |
| 187 | + logger from the `google`-level logger. If you need the events to be propagated to the root logger, you must explicitly set |
| 188 | + :code:`logging.getLogger("google").propagate = True` in your code. |
| 189 | +#. You can mix the different logging configurations above for different Google modules. For example, you may want use a code-based logging configuration for |
| 190 | + one library, but decide you need to also set up environment-based logging configuration for another library. |
| 191 | + |
| 192 | + #. If you attempt to use both code-based and environment-based configuration for the same module, the environment-based configuration will be ineffectual |
| 193 | + if the code -based configuration gets applied first. |
61 | 194 |
|
62 | | -See the `pandas-gbq documentation <https://googleapis.dev/python/pandas-gbq/latest/>`_ for more details. |
| 195 | +#. The Google-specific logging configurations (default handlers for environment-based configuration; not propagating logging events to the root logger) get |
| 196 | + executed the first time *any* client library is instantiated in your application, and only if the affected loggers have not been previously configured. |
| 197 | + (This is the reason for 2.i. above.) |
0 commit comments