Merge branch 'master' into mp

Author: dimitri-yatsenko

.coveragerc

@@ -3,3 +3,4 @@ branch = False
 source = datajoint
 
 [report]
+show_missing = True

.dockerignore

@@ -0,0 +1,6 @@
+notebook
+build
+*.egg-info
+dist
+.vscode
+__pycache__

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,37 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: ''
+labels: 'bug, awaiting-triage'
+assignees: ''
+
+---
+
+## Bug Report
+
+### Description
+A clear and concise description of what is the overall operation that is intended to be performed that resulted in an error.
+
+### Reproducibility
+Include:
+- OS (WIN | MACOS | Linux)
+- Python Version OR MATLAB Version
+- MySQL Version
+- MySQL Deployment Strategy (local-native | local-docker | remote)
+- DataJoint Version
+- Minimum number of steps to reliably reproduce the issue
+- Complete error stack as a result of evaluating the above steps
+
+### Expected Behavior
+A clear and concise description of what you expected to happen.
+
+### Screenshots
+If applicable, add screenshots to help explain your problem.
+
+### Additional Research and Context
+Add any additional research or context that was conducted in creating this report.
+
+For example:
+- Related GitHub issues and PR's either within this repository or in other relevant repositories.
+- Specific links to specific lines or a focus within source code.
+- Relevant summary of Maintainers development meetings, milestones, projects, etc.

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,46 @@
+---
+name: Feature request
+about: Suggest an idea for a new feature
+title: ''
+labels: 'enhancement, awaiting-triage'
+assignees: ''
+
+---
+
+## Feature Request
+
+### Problem
+A clear and concise description how this idea has manifested and the context. Elaborate on the need for this feature and/or what could be improved. Ex. I'm always frustrated when [...]
+
+### Requirements
+A clear and concise description of the requirements to satisfy the new feature. Detail what you expect from a successful implementation of the feature. Ex. When using this feature, it should [...]
+
+### Justification
+Provide the key benefits in making this a supported feature. Ex. Adding support for this feature would ensure [...]
+
+### Alternative Considerations
+Do you currently have a work-around for this? Provide any alternative solutions or features you've considered.
+
+### Related Errors
+Add any errors as a direct result of not exposing this feature.
+
+Please include steps to reproduce provided errors as follows:
+- OS (WIN | MACOS | Linux)
+- Python Version OR MATLAB Version
+- MySQL Version
+- MySQL Deployment Strategy (local-native | local-docker | remote)
+- DataJoint Version
+- Minimum number of steps to reliably reproduce the issue
+- Complete error stack as a result of evaluating the above steps
+
+### Screenshots
+If applicable, add screenshots to help explain your feature.
+
+### Additional Research and Context
+Add any additional research or context that was conducted in creating this feature request.
+
+For example:
+- Related GitHub issues and PR's either within this repository or in other relevant repositories.
+- Specific links to specific line or focus within source code.
+- Relevant summary of Maintainers development meetings, milestones, projects, etc.
+- Any additional supplemental web references or links that would further justify this feature request.

.github/workflows/development.yaml

@@ -0,0 +1,51 @@
+name: Development
+on:
+  push:
+    branches:
+      - '**' # every branch
+      - '!stage*' # exclude branches beginning with stage
+  pull_request:
+    branches:
+      - '**' # every branch
+      - '!stage*' # exclude branches beginning with stage
+jobs:
+  test:
+    if: github.event_name == 'push' || github.event_name == 'pull_request'
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        py_ver: ["3.8"]
+        mysql_ver: ["8.0", "5.7"]
+        include:
+          - py_ver: "3.7"
+            mysql_ver: "5.7"
+          - py_ver: "3.6"
+            mysql_ver: "5.7"
+    steps:
+      - uses: actions/checkout@v2
+      - name: Set up Python ${{matrix.py_ver}}
+        uses: actions/setup-python@v2
+        with:
+          python-version: ${{matrix.py_ver}}
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install flake8
+      - name: Run syntax tests
+        run: flake8 datajoint --count --select=E9,F63,F7,F82 --show-source --statistics
+      - name: Run primary tests
+        env:
+          UID: "1001"
+          GID: "121"
+          PY_VER: ${{matrix.py_ver}}
+          MYSQL_VER: ${{matrix.mysql_ver}}
+          ALPINE_VER: "3.10"
+          MINIO_VER: RELEASE.2019-09-26T19-42-35Z
+          COMPOSE_HTTP_TIMEOUT: "120"
+          COVERALLS_SERVICE_NAME: travis-ci
+          COVERALLS_REPO_TOKEN: fd0BoXG46TPReEem0uMy7BJO5j0w1MQiY
+        run: docker-compose -f LNX-docker-compose.yml up --build --exit-code-from app
+      - name: Run style tests
+        run: |
+          flake8 --ignore=E121,E123,E126,E226,E24,E704,W503,W504,E722,F401,W605 datajoint \
+                 --count --max-complexity=62 --max-line-length=127 --statistics

.gitignore

@@ -23,4 +23,6 @@ docker-compose.yml
 notebook
 .vscode
 __main__.py
-jupyter_custom.js
+jupyter_custom.js
+apk_requirements.txt
+.eggs

.travis.yml

@@ -1,5 +1,75 @@
 ## Release notes
 
+### 0.13.3 -- TBD
+* Bugfix - Dependencies not properly loaded on populate. (#902) PR #919
+* Bugfix - Replace use of numpy aliases of built-in types with built-in type. (#938) PR #939
+
+### 0.13.2 -- May 7, 2021
+* Update `setuptools_certificate` dependency to new name `otumat`
+* Bugfix - Explicit calls to `dj.Connection` throw error due to missing `host_input` (#895) PR #907
+* Bugfix - Correct count of deleted items. (#897) PR #912
+
+### 0.13.1 -- Apr 16, 2021
+* Add `None` as an alias for `IS NULL` comparison in `dict` restrictions (#824) PR #893
+* Drop support for MySQL 5.6 since it has reached EOL PR #893
+* Bugfix - `schema.list_tables()` is not topologically sorted (#838) PR #893
+* Bugfix - Diagram part tables do not show proper class name (#882) PR #893
+* Bugfix - Error in complex restrictions (#892) PR #893
+* Bugfix - WHERE and GROUP BY clases are dropped on joins with aggregation (#898, #899) PR #893
+
+### 0.13.0 -- Mar 24, 2021
+* Re-implement query transpilation into SQL, fixing issues (#386, #449, #450, #484, #558). PR #754
+* Re-implement cascading deletes for better performance. PR #839
+* Add support for deferred schema activation to allow for greater modularity. (#834) PR #839
+* Add query caching mechanism for offline development (#550) PR #839
+* Add table method `.update1` to update a row in the table with new values (#867) PR #763, #889
+* Python datatypes are now enabled by default in blobs (#761). PR #859
+* Added permissive join and restriction operators `@` and `^` (#785) PR #754
+* Support DataJoint datatype and connection plugins (#715, #729) PR 730, #735
+* Add `dj.key_hash` alias to `dj.hash.key_hash` (#804) PR #862
+* Default enable_python_native_blobs to True
+* Bugfix - Regression error on joins with same attribute name (#857) PR #878
+* Bugfix - Error when `fetch1('KEY')` when `dj.config['fetch_format']='frame'` set (#876) PR #880, #878
+* Bugfix - Error when cascading deletes in tables with many, complex keys (#883, #886) PR #839
+* Add deprecation warning for `_update`. PR #889
+* Add `purge_query_cache` utility. PR #889
+* Add tests for query caching and permissive join and restriction. PR #889
+* Drop support for Python 3.5 (#829) PR #861
+
+### 0.12.9 -- Mar 12, 2021
+* Fix bug with fetch1 with `dj.config['fetch_format']="frame"`. (#876) PR #880
+
+### 0.12.8 -- Jan 12, 2021
+* table.children, .parents, .descendents, and ancestors can return queryable objects. PR #833
+* Load dependencies before querying dependencies. (#179) PR #833
+* Fix display of part tables in `schema.save`. (#821) PR #833
+* Add `schema.list_tables`. (#838) PR #844
+* Fix minio new version regression.  PR #847
+* Add more S3 logging for debugging. (#831) PR #832
+* Convert testing framework from TravisCI to GitHub Actions (#841) PR #840
+
+### 0.12.7 -- Oct 27, 2020
+* Fix case sensitivity issues to adapt to MySQL 8+.  PR #819
+* Fix pymysql regression bug (#814) PR #816
+* Adapted attribute types now have dtype=object in all recarray results. PR #811
+
+### 0.12.6 -- May 15, 2020
+* Add `order_by` to `dj.kill` (#668, #779) PR #775, #783
+* Add explicit S3 bucket and file storage location existence checks (#748) PR #781
+* Modify `_update` to allow nullable updates for strings/date (#664) PR #760
+* Avoid logging events on auxiliary tables (#737) PR #753
+* Add `kill_quick` and expand display to include host (#740) PR #741
+* Bugfix - pandas insert fails due to additional `index` field (#666) PR #776
+* Bugfix - `delete_external_files=True` does not remove from S3 (#686) PR #781
+* Bugfix - pandas fetch throws error when `fetch_format='frame'` PR #774
+
+### 0.12.5 -- Feb 24, 2020
+* Rename module `dj.schema` into `dj.schemas`. `dj.schema` remains an alias for class `dj.Schema`. (#731) PR #732
+* `dj.create_virtual_module` is now called `dj.VirtualModule` (#731) PR #732
+* Bugfix - SSL `KeyError` on failed connection (#716) PR #725
+* Bugfix - Unable to run unit tests using nosetests (#723) PR #724
+* Bugfix - `suppress_errors` does not suppress loss of connection error (#720) PR #721
+
 ### 0.12.4 -- Jan 14, 2020
 * Support for simple scalar datatypes in blobs (#690) PR #709
 * Add support for the `serial` data type in declarations: alias for `bigint unsigned auto_increment` PR #713
@@ -73,9 +143,9 @@
 * Bugfix in restriction of the form (A & B) * B (#463)
 * Improved error messages (#466)
 
-### 0.10.0 -- Jan 10, 2018 
+### 0.10.0 -- Jan 10, 2018
 * Deletes are more efficient (#424)
-* ERD shows table definition on tooltip hover in Jupyter (#422) 
+* ERD shows table definition on tooltip hover in Jupyter (#422)
 * S3 external storage
 * Garbage collection for external sorage
 * Most operators and methods of tables can be invoked as class methods rather than instance methods (#407)
@@ -89,7 +159,7 @@
 * Implement union operator +
 * Implement file-based external storage
 
-### 0.8.0 -- Jul 26, 2017 
+### 0.8.0 -- Jul 26, 2017
 Documentation and tutorials available at https://docs.datajoint.io and https://tutorials.datajoint.io
 * improved the ERD graphics and features using the graphviz libraries (#207, #333)
 * improved password handling logic (#322, #321)
@@ -108,11 +178,11 @@ Documentation and tutorials available at https://docs.datajoint.io and https://t
 * Added `dj.create_virtual_module`
 
 ### 0.4.10 (#286) -- Feb 6, 2017
-* Removed Vagrant and Readthedocs support 
+* Removed Vagrant and Readthedocs support
 * Explicit saving of configuration (issue #284)
 
 ### 0.4.9 (#285) -- Feb 2, 2017
-* Fixed setup.py for pip install 
+* Fixed setup.py for pip install
 
 ### 0.4.7 (#281) -- Jan 24, 2017
 * Fixed issues related to order of attributes in projection.
@@ -141,10 +211,10 @@ Documentation and tutorials available at https://docs.datajoint.io and https://t
 
 ### 0.3.8  -- Aug 2, 2016
 * added the `_update` method in `base_relation`. It allows updating values in existing tuples.
-* bugfix in reading values of type double.  Previously it was cast as float32. 
+* bugfix in reading values of type double.  Previously it was cast as float32.
 
 ### 0.3.7  -- Jul 31, 2016
-* added parameter `ignore_extra_fields` in `insert` 
+* added parameter `ignore_extra_fields` in `insert`
 * `insert(..., skip_duplicates=True)` now relies on `SELECT IGNORE`.  Previously it explicitly checked if tuple already exists.
 * table previews now include blob attributes displaying the string <BLOB>
 

CHANGELOG.md

@@ -1,5 +1,5 @@
 FROM datajoint/pydev
 
-ADD . /src
-RUN pip install /src && \
-    rm -rf /src
+COPY --chown=dja . /tmp/src
+RUN pip install --user /tmp/src && \
+    rm -rf /tmp/src