🏗️ upgrade all the things 🤖 (#41)

Upgrade robotkernel, jupyterlab, node, conda, etc. Build with constructor 3. Run acceptance tests from shipped code.

Author: bollwyvl

.gitignore

@@ -9,3 +9,4 @@ constructor/construct.yaml
 dist/
 envs/
 **/icons/*.png
+constructor/.*

README.md

@@ -22,13 +22,15 @@ anaconda-project run test:win
 
 ## Motivation
 
-While Robot Framework has no dependencies beyond the Python standard library,
-using it for non-trivial testing or process automation usually requires a fair
-amount of additional Python dependencies, and even some more exotic ones.
+While [Robot Framework][robotframework] has no dependencies beyond the Python
+standard library, using it for non-trivial testing or process automation usually
+requires a fair amount of additional Python dependencies, and even some more
+exotic ones.
 
-As the focus of this workshop, using [robotkernel][] requires the Jupyter stack,
-and to demonstrate some of its more advanced features, some fairly extensive
-extra libraries from both the Robot Framework and the scientific Python ecosystems.
+The focus of this workshop, running Robot tests and tasks interactively with
+[robotkernel][], requires the Jupyter stack, and to demonstrate some of its more
+advanced features, some fairly extensive extra libraries from both the Robot
+Framework and the scientific Python ecosystems.
 
 To make a three-hour workshop reasonable, this repo uses [conda][] and
 [constructor][] to build single-file installers for multiple platforms, reducing
@@ -39,16 +41,12 @@ possible in the event that network problems arise.
 ## conda packaging
 Once it has matured, `robotkernel` will be available from [conda-forge][], the
 community-driven upstream of the Anaconda Distribution.
-However, in the name of efficiency, `conda-forge` builds `noarch: python`
-packages, which are not compatible with `constructor`'s multiplatform magic.
 
-So for these installers, we build (or rebuild) a number of dependencies, such
-as [robotframework][] 3.1 and SeleniumLibrary 3.2.0.
-
-> TODO: automated detection and re-arching of packages from conda-forge feedstocks
+So for these installers, and supporting exmaples, we build a number of
+dependencies.
 
 ### robotlab
-While still pre-`1.0`, JupyterLab's build chain has some negative externalities
+JupyterLab's build chain has some negative externalities
 for end users, namely an install- or run-time dependency on NodeJS and npmjs.org
 when using any labextensions other than the built-in set (e.g. Notebook, Terminal,
 Console, Editor, etc.). Because, for the purposes of the workshop, we want to
@@ -65,7 +63,7 @@ do most of the things `jupyter lab` can do.
 `robotlab` works like `jupyter lab`, while `robotlab-extension` works like
 `jupyter labextension`. This isn't a toy installation: with the bundled `nodejs`,
 an intrepid user can still install any of the [labextensions][] that are
-compatible with the version `robotlab` was built with: as of writing, `0.35.x`.
+compatible with the version `robotlab` was built with: as of writing, `1.1.x`.
 
 ## [constructor][]
 All of the dependencies are captured in [construct.yaml.in][]. In addition to
@@ -75,14 +73,24 @@ In addition to required dependencies a number of extra libraries are included to
 showcase some of the features of using Robot Framework interactively.
 - `JupyterLibrary` for testing Jupyter clients with robotframework
   - `SeleniumLibrary` for controlling browsers
-    - `geckodriver` for interacting with Mozilla Firefox
-    - `chromedriver` for interacting with Chromium and Google Chrome
-    - > it's pretty easy to [get `webdriver`][webdriver] for Microsoft Edge, but
-        can't be redistributed
+    - `geckodriver` for interacting with Mozilla Firefox 57+
 - `opencv` for image-driven testing
 - `robotframework-lint` for helping you write clean robot syntax
 - `RESTInstance` for testing REST APIs, including swagger
 
+## On Browsers and WebDrivers
+RobotLab includes Mozilla Firefox and `geckodriver`, with versions that will be
+supported for _years_, not _months_, by a team that is committed to open source
+and web standards.
+
+### webdriver for Microsoft Internet Explorer
+It's pretty easy to [get `webdriver`][webdriver] for Microsoft Edge, but...
+  - it but can't be redistributed
+  - it has to match the version of Edge/Explorer exactly
+
+### chromedriver for Google Chrome
+It's also pretty easy to get [chromedriver][] for Google Chrome, but...
+  - it has to match the version of Chrome exactly
 
 [anaconda-project]: https://github.com/anaconda-platform/anaconda-project
 [conda-forge]: https://github.com/conda-forge
@@ -94,3 +102,4 @@ showcase some of the features of using Robot Framework interactively.
 [Miniconda]: https://conda.io/miniconda.html
 [robotkernel]: https://github.com/robots-from-jupyter/robotkernel
 [webdriver]: https://developer.microsoft.com/en-us/microsoft-edge/tools/webdriver/#downloads
+[chromedriver]: https://chromedriver.chromium.org/downloads

anaconda-project.yml

@@ -1,154 +1,121 @@
-name: robocon-2019-workshop
+name: robotlab
 
 commands:
-  build:
-    unix: python -m scripts.build
-    env_spec: _build
+  integrity:
+    unix: python -m scripts.integrity
+    env_spec: _qa
 
   lint:
     unix: python -m scripts.lint
-    env_spec: _build
-
-  integrity:
-    unix: python -m scripts.integrity
-    env_spec: _build
+    env_spec: _qa
 
-  build:win:
+  build:
+    unix: python -m scripts.build
     windows: python -m scripts.build
-    env_spec: _build_win
+    env_spec: _robots_from_jupyter
+
+  build:lab:
+    unix: python -m scripts.build lab
+    windows: python -m scripts.build lab
+    env_spec: _qa
 
   test:
     unix: python -m scripts.test
-    env_spec: _test
-
-  test:win:
     windows: python -m scripts.test
-    env_spec: _test_win
+    env_spec: _test
 
   combine:
     unix: python -m scripts.combine
-    env_spec: _build
+    env_spec: _qa
 
   robotlab:setup:
+    unix: cd robotlab && python -m pip install -e . --ignore-installed --no-deps -vv
+    windows: cd robotlab && python -m pip install -e . --ignore-installed --no-deps -vv
     env_spec: robotlab-dev
-    unix: cd robotlab && python -m pip install -e . --ignore-installed --no-deps --no-cache-dir -vv
-    windows: cd robotlab && python -m pip install -e . --ignore-installed --no-deps --no-cache-dir -vv
-
-  robotlab:setup:win:
-    env_spec: robotlab-dev-win
-    windows: cd robotlab && python -m pip install -e . --ignore-installed --no-deps --no-cache-dir -vv
 
-  robotlab:shortcuts:
+  robotlab:
+    unix: robotlab
+    windows: robotlab
     env_spec: robotlab-dev
-    unix: robotlab-shortcuts
 
   robotlab:launch:
-    env_spec: robotlab-dev
-    unix: python ./robotlab/src/robotlab/launch.py
-
-  robotlab:launch:win:
-    env_spec: robotlab-dev-win
+    unix: python -m robotlab.launch
     windows: python -m robotlab.launch
+    env_spec: robotlab-dev
 
   robotlab:build:
-    env_spec: robotlab-dev
     unix: robotlab build
-
-  robotlab:build:win:
-    env_spec: robotlab-dev-win
     windows: robotlab build
-
-  robotlab:
     env_spec: robotlab-dev
-    unix: robotlab
 
-  robotlab:win:
-    env_spec: robotlab-dev-win
-    windows: robotlab
+channels:
+- https://conda.anaconda.org/anaconda
+- https://conda.anaconda.org/conda-forge
 
 env_specs:
-  _build:
-    platforms:
-    - linux-64
-    - osx-64
-    inherit_from:
-    - _robots_from_jupyter
-
-  _build_win:
-    platforms:
-    - win-64
-    inherit_from:
-    - _robots_from_jupyter
-
-  _test:
-    platforms:
-    - linux-64
-    - osx-64
-    inherit_from:
-    - _robots_from_jupyter
+  _robots_from_jupyter:
+    description: the main build/test environment
     channels:
-    - ./_artifacts/conda-bld
-    - https://repo.anaconda.com/pkgs/main
-    - https://repo.anaconda.com/pkgs/free
+    - https://conda.anaconda.org/anaconda
     - https://conda.anaconda.org/conda-forge
     packages:
-    - robotframework-jupyterlibrary
+      - conda >=4.7.12,<4.8
+      - conda-build >=3.18.9,<3.19
+      - conda-verify
+      - constructor >=3,<3.1
+      - firefox >=68.0.2,<69
+      - geckodriver
+      - pillow
+      - python >=3.6,<3.7.0a0
+      - python-libarchive-c
+      - ripgrep
+      - robotframework
+      - robotframework-seleniumlibrary
+
+  _lab:
+    packages:
+      - jupyterlab ==1.1.4
+      - nodejs >=11,<12
+      - python >=3.6,<3.7.0a0
 
-  _test_win:
-    platforms:
-    - win-64
+  _qa:
+    inherit_from:
+      - _lab
+    packages:
+      - black
+      - flake8
+      - robotframework-lint
+
+  _test:
     inherit_from:
     - _robots_from_jupyter
     channels:
     - ./_artifacts/conda-bld
-    - https://repo.anaconda.com/pkgs/main
-    - https://repo.anaconda.com/pkgs/free
-    - https://conda.anaconda.org/conda-forge
     packages:
     - robotframework-jupyterlibrary
 
-
   robotlab-dev:
+    description: basically what goes in the constructor for interactive dev/test
+    inherit_from:
+      - _lab
     channels:
     - ./_artifacts/conda-bld
-    - https://repo.anaconda.com/pkgs/main
-    - https://repo.anaconda.com/pkgs/free
-    - https://conda.anaconda.org/conda-forge
     packages:
-    - jupyterlab >=0.35,<0.36
-    - nodejs >=8,<9
+    - conda
+    - firefox
     - pyshortcuts
-    - python >=3.6,<3.7
     - restinstance
     - robotframework-seleniumscreenshots
     - robotkernel
 
   robotlab-dev-win:
     inherit_from:
     - robotlab-dev
-
+    packages:
+    - robotframework-whitelibrary
 
   robotlab-test:
     inherit_from:
     - robotlab-dev
     packages:
     - robotlab
-
-  _robots_from_jupyter:
-    packages:
-    - black
-    - conda >=4.5.11,<4.6
-    - conda-build >=3.15,<3.16
-    - conda-verify
-    - constructor >=2.3,<2.4
-    - flake8
-    - geckodriver
-    - python >=3.6,<3.7
-    - python-chromedriver-binary ==2.46
-    - robotframework
-    - robotframework-lint
-    - robotframework-seleniumlibrary
-    channels:
-    - https://repo.anaconda.com/pkgs/main
-    - https://repo.anaconda.com/pkgs/free
-    - https://conda.anaconda.org/conda-forge

azure-pipelines.yml

@@ -1,24 +1,51 @@
-jobs:
-- template: ci/job.test.yml
-  parameters:
-    name: Linux
-    vmImage: ubuntu-16.04
-
-- template: ci/job.test.yml
-  parameters:
-    name: MacOSX
-    vmImage: macos-10.13
-
-- template: ci/job.test.yml
-  parameters:
-    name: Windows
-    vmImage: vs2017-win2016
-
-- template: ci/job.combine.yml
-  parameters:
-    name: Linux
-    vmImage: ubuntu-16.04
-
 trigger: [master]
 
 pr: [master]
+
+variables:
+  ROBOTLAB_VERSION: 2019.9.0
+  PYTHONUNBUFFERED: 1
+
+jobs:
+  # group one: simple conda/webpack builds, validation
+  - template: ci/job.noarch.yml
+  - template: ci/job.win.yml
+  - template: ci/job.lab.yml
+
+  # group two: verify conda builds, do acceptance tests
+  - template: ci/job.noarch-test.yml
+  - template: ci/job.win-test.yml
+
+  - template: ci/job.main.yml
+    parameters:
+      name: Linux
+      vmImage: ubuntu-16.04
+
+  - template: ci/job.main.yml
+    parameters:
+      name: MacOSX
+      vmImage: macos-10.13
+
+  - template: ci/job.main.yml
+    parameters:
+      name: Windows
+      vmImage: vs2017-win2016
+      activate: call activate &&
+
+  - template: ci/job.selftest.yml
+    parameters:
+      name: Linux
+      vmImage: ubuntu-16.04
+
+  - template: ci/job.selftest.yml
+    parameters:
+      name: MacOSX
+      vmImage: macos-10.13
+
+  - template: ci/job.selftest.yml
+    parameters:
+      name: Windows
+      vmImage: vs2017-win2016
+
+  # group 3: reporting, etc
+  - template: ci/job.combine.yml

ci/env-combine.yml

@@ -0,0 +1,8 @@
+name: robotlab-combine
+
+channels:
+  - https://conda.anaconda.org/anaconda
+  - https://conda.anaconda.org/conda-forge
+
+dependencies:
+  - robotframework