Skip to content

Commit b458f2a

Browse files
author
David Erb
committed
upload code
1 parent 300bce3 commit b458f2a

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

54 files changed

+2964
-1
lines changed

‎.dae-devops/Makefile‎

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
# ********** Please don't edit this file!
2+
# ********** It has been generated automatically by dae_devops version 0.4.0.
3+
# ********** For repository_name dls-normsql
4+
5+
# ---------------------------------------------------------------------
6+
# These make targets are designed to be called from the command line and from .gitlab-ci.yml.
7+
# ---------------------------------------------------------------------
8+
9+
# I put the package_pip artifacts int this place for now until I can use the corporate internal pipserver.
10+
# Maybe /dls_sw/work/python3/RHEL7-x86_64/distributions would be a better place?
11+
PIP_FIND_LINKS = /dls_sw/apps/bxflow/artifacts
12+
13+
# I got these from https://gitlab.diamond.ac.uk/controls/reports/ci_templates/-/blob/master/defaults.yml.
14+
# Directory where docs are published.
15+
DOCS_PUBLISH_ROOT = /dls/cs-www/reports/gitlab-ci/dls-normsql
16+
17+
# ---------------------------------------------------------------------
18+
# These are called from the default before_script in the gitlab CI.
19+
# That meants it is normally called before any other job.
20+
# It installs things not covered by pyproject.toml.
21+
22+
prepare_git_dependencies:
23+
chmod a+x .dae-devops/prepare_git_dependencies.sh
24+
.dae-devops/prepare_git_dependencies.sh
25+
26+
# ---------------------------------------------------------------------
27+
# The validate stage makes sure code is ready to package and release.
28+
29+
validate: \
30+
validate_pre_commit \
31+
validate_mypy \
32+
validate_pytest
33+
echo "validation complete"
34+
35+
validate_pre_commit:
36+
tox -q -e pre-commit
37+
38+
validate_mypy:
39+
tox -q -e mypy
40+
41+
validate_pytest:
42+
tox -q -e pytest
43+
44+
validate_docs:
45+
tox -q -e docs
46+
47+
# ---------------------------------------------------------------------
48+
# The pip state packages and publishes for pip.
49+
50+
package: \
51+
package_pip
52+
53+
package_pip:
54+
pip install pipx twine
55+
pipx run build
56+
57+
# ---------------------------------------------------------------------
58+
# The publish stage publishes things which have already been built or packaged.
59+
60+
publish: \
61+
publish_pip \
62+
publish_docs
63+
64+
publish_pip:
65+
cp -v -p dist/*.whl $(PIP_FIND_LINKS)
66+
67+
publish_docs:
68+
mkdir -p $(DOCS_PUBLISH_ROOT)
69+
cp -r build/html/* $(DOCS_PUBLISH_ROOT)
70+
71+
72+
# dae_devops_fingerprint 9a5dedf3d519c11e1d36b87095ed4f26

‎.dae-devops/docs/conventions.rst‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
.. # ********** Please don't edit this file!
2+
.. # ********** It has been generated automatically by dae_devops version 0.4.0.
3+
.. # ********** For repository_name dls-normsql
4+
5+
Naming conventions
6+
=======================================================================
7+
8+
Here are the naming conventions used within the source code.
9+
10+
variables
11+
lowercase, underscores
12+
13+
constants
14+
uppercase, underscores
15+
16+
classes
17+
camel case
18+
19+
class methods
20+
lowercase, underscores
21+
22+
imports
23+
require fully qualified package name (import <package> does nothing)
24+
25+
exception: top-level packages __init__.py will have __version__ defined in them
26+
27+
python packages
28+
lowercase, underscores
29+
30+
repository
31+
lowercase, hyphens
32+
33+
34+
.. # dae_devops_fingerprint febbfa9d192af5010a4cd5e6de836097

‎.dae-devops/docs/developing.rst‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
.. # ********** Please don't edit this file!
2+
.. # ********** It has been generated automatically by dae_devops version 0.4.0.
3+
.. # ********** For repository_name dls-normsql
4+
5+
Developing
6+
=======================================================================
7+
8+
If you plan to make change to the code in this repository, you can use the steps below.
9+
10+
Clone the repository::
11+
12+
$ git clone https://gitlab.diamond.ac.uk/kbp43231/dls-normsql.git
13+
14+
It is recommended that you install into a virtual environment so this
15+
installation will not interfere with any existing Python software.
16+
Make sure to have at least python version 3.8 then::
17+
18+
$ python3 -m venv /scratch/$USER/myvenv
19+
$ source /scratch/$USER/myvenv/bin/activate
20+
$ pip install --upgrade pip
21+
22+
Install the package in edit mode which will also install all its dependencies::
23+
24+
$ cd dls-normsql
25+
$ export PIP_FIND_LINKS=/dls_sw/apps/bxflow/artifacts
26+
$ pip install -e .[dev]
27+
28+
Now you may begin modifying the code.
29+
30+
|
31+
32+
If you plan to modify the docs, you will need to::
33+
34+
$ pip install -e .[docs]
35+
36+
37+
38+
39+
.. # dae_devops_fingerprint 07f1e374bc286bfd64edca93fbfbec4b

‎.dae-devops/docs/devops.rst‎

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
.. # ********** Please don't edit this file!
2+
.. # ********** It has been generated automatically by dae_devops version 0.4.0.
3+
.. # ********** For repository_name dls-normsql
4+
5+
Devops
6+
=======================================================================
7+
8+
There exists a a configuration file called ``.dae-devops/project.yaml``.
9+
10+
This file defines the project information needed for CI/CD.
11+
12+
It is parsed by the ``dae_devops.force`` command which creates these files:
13+
14+
- pyproject.toml
15+
- .gitlab-ci.yml
16+
- .dae-devops/Makefile
17+
- .dae-devops/docs/*
18+
19+
Local CI/CD execution
20+
-----------------------------------------------------------------------
21+
22+
All the CI/CD ops which are run by the git server can be run at the command line.
23+
24+
Running these ops before pushing to the git server can make the turnaround quicker to fix things.
25+
26+
Follow the steps in the Developing section. Then you can run the following commands.
27+
28+
Validation of the code::
29+
30+
$ make -f .dae-devops/Makefile validate_pre_commit
31+
$ make -f .dae-devops/Makefile validate_mypy
32+
$ make -f .dae-devops/Makefile validate_pytest
33+
$ make -f .dae-devops/Makefile validate_docs
34+
35+
Packaging::
36+
37+
$ make -f .dae-devops/Makefile package_pip
38+
39+
Publishing::
40+
41+
$ make -f .dae-devops/Makefile publish_pip
42+
$ make -f .dae-devops/Makefile publish_docs
43+
44+
45+
46+
.. # dae_devops_fingerprint 231c4947e546d9e98de707b418797c3b

‎.dae-devops/docs/docs_structure.rst‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
.. # ********** Please don't edit this file!
2+
.. # ********** It has been generated automatically by dae_devops version 0.4.0.
3+
.. # ********** For repository_name dls-normsql
4+
5+
About the documentation
6+
-----------------------
7+
8+
:material-regular:`format_quote;2em`
9+
10+
The Grand Unified Theory of Documentation
11+
12+
-- David Laing
13+
14+
There is a secret that needs to be understood in order to write good software
15+
documentation: there isn't one thing called *documentation*, there are four.
16+
17+
They are: *tutorials*, *how-to guides*, *technical reference* and *explanation*.
18+
They represent four different purposes or functions, and require four different
19+
approaches to their creation. Understanding the implications of this will help
20+
improve most documentation - often immensely.
21+
22+
`More information on this topic. <https://documentation.divio.com>`_
23+
24+
.. # dae_devops_fingerprint d70cbde1ea928b73721419cf1895f056

‎.dae-devops/docs/installing.rst‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
.. # ********** Please don't edit this file!
2+
.. # ********** It has been generated automatically by dae_devops version 0.4.0.
3+
.. # ********** For repository_name dls-normsql
4+
5+
Installing
6+
=======================================================================
7+
8+
9+
You will need python 3.8 or later.
10+
11+
On a Diamond Light Source internal computer, you can achieve Python 3.8 by::
12+
13+
$ module load python/3.8
14+
15+
You can check your version of python by typing into a terminal::
16+
17+
$ python3 --version
18+
19+
It is recommended that you install into a virtual environment so this
20+
installation will not interfere with any existing Python software::
21+
22+
$ python3 -m venv /scratch/$USER/myvenv
23+
$ source /scratch/$USER/myvenv/bin/activate
24+
$ pip install --upgrade pip
25+
26+
27+
You can now use ``pip`` to install the library and its dependencies::
28+
29+
$ export PIP_FIND_LINKS=/dls_sw/apps/bxflow/artifacts
30+
$ python3 -m pip install dls-normsql
31+
32+
If you require a feature that is not currently released you can also install
33+
from git::
34+
35+
$ python3 -m pip install git+https://gitlab.diamond.ac.uk/kbp43231/dls-normsql.git
36+
37+
The library should now be installed and the commandline interface on your path.
38+
You can check the version that has been installed by typing::
39+
40+
$ dls-normsql --version
41+
$ dls-normsql --version-json
42+
43+
.. # dae_devops_fingerprint 99cdabc51a937f9f403b533ea3108c3f

‎.dae-devops/docs/testing.rst‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
.. # ********** Please don't edit this file!
2+
.. # ********** It has been generated automatically by dae_devops version 0.4.0.
3+
.. # ********** For repository_name dls-normsql
4+
5+
Testing
6+
=======================================================================
7+
8+
The package uses pytest for unit testing.
9+
10+
If you want to run the tests, first get a copy of the code per the instructions in the Developing section.
11+
12+
Then you can run all tests by::
13+
14+
$ pytest
15+
16+
Or this, which is the command used by the CI runner.
17+
18+
$ make -f .dae-devops/Makefile validate_pytest
19+
20+
To run a single test you can do::
21+
22+
$ pytest tests/the_test_you_want.py
23+
24+
If you want to see more output of the test while it's running you can do:
25+
26+
$ pytest -sv -ra --tb=line tests/the_test_you_want.py
27+
28+
Each test will write files into its own directory::
29+
30+
/tmp/dls-normsql/tests/....
31+
32+
The tests clear their directory when they start, but not when they finish.
33+
This allows peeking in there to see what's been written by the test.
34+
35+
36+
37+
38+
.. # dae_devops_fingerprint 32ce52a513e208dea899cdb3a378290c

‎.dae-devops/prepare_git_dependencies.sh‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
#!/usr/bin/env bash
2+
3+
# ********** Please don't edit this file!
4+
# ********** It has been generated automatically by dae_devops version 0.4.0.
5+
# ********** For repository_name dls-normsql
6+
7+
me=${BASH_SOURCE}
8+
echo "${me}: installing 0 dependencies for repository_name dls-normsql"
9+
10+
function __install {
11+
echo ${me}: "$@"
12+
"$@"
13+
}
14+
15+
16+
# dae_devops_fingerprint 93e61fd4bac57573417cfd42bb395e85

‎.dae-devops/project.yaml‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
# Primary information needed for devops.
2+
primary:
3+
repository_name: dls-normsql
4+
package_name: dls_normsql
5+
one_line_description: "Normalized API over various sql libraries."
6+
author:
7+
name: David Erb
8+
email: [email protected]
9+
project_urls:
10+
GitLab: https://gitlab.diamond.ac.uk/kbp43231
11+
project_scripts:
12+
dls-normsql: "dls_normsql.__main__:main"
13+
dependencies:
14+
- type: pypi
15+
list:
16+
- aiosqlite
17+
- dls-utilpack

‎.devcontainer/Dockerfile‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
# ********** Please don't edit this file!
2+
# ********** It has been generated automatically by dae_devops version 0.4.0.
3+
# ********** For repository_name dls-normsql
4+
5+
# This file is for use as a devcontainer and a runtime container
6+
#
7+
# The devcontainer should use the build target and run as root with podman
8+
# or docker with user namespaces.
9+
#
10+
FROM python:3.11 as build
11+
12+
ARG PIP_OPTIONS
13+
14+
# Add any system dependencies for the developer/build environment here e.g.
15+
# RUN apt-get update && apt-get upgrade -y && \
16+
# apt-get install -y --no-install-recommends \
17+
# desired-packages \
18+
# && rm -rf /var/lib/apt/lists/*
19+
20+
# set up a virtual environment and put it in PATH
21+
RUN python -m venv /venv
22+
ENV PATH=/venv/bin:$PATH
23+
24+
# Copy any required context for the pip install over
25+
COPY . /context
26+
WORKDIR /context
27+
28+
# install python package into /venv
29+
RUN pip install ${PIP_OPTIONS}
30+
31+
FROM python:3.11-slim as runtime
32+
33+
# Add apt-get system dependecies for runtime here if needed
34+
35+
# copy the virtual environment from the build stage and put it in PATH
36+
COPY --from=build /venv/ /venv/
37+
ENV PATH=/venv/bin:$PATH
38+
39+
# change this entrypoint if it is not the same as the repo
40+
ENTRYPOINT ["dls-normsql"]
41+
CMD ["--version"]
42+
43+
# dae_devops_fingerprint 37bdb2cf713e74e37b952b4e8895c779

0 commit comments

Comments
 (0)