Skip to content

Commit ec98499

Browse files
jaimergpmarcelotrevisani
jaimergp
and
marcelotrevisani
committed
add blog/2020-03-05-grayskull.md
Co-authored-by: marcelotrevisani <[email protected]>
1 parent 1da1414 commit ec98499

File tree

1 file changed

+367
-0
lines changed

1 file changed

+367
-0
lines changed

‎blog/2020-03-05-grayskull.md

Lines changed: 367 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,367 @@
1+
---
2+
authors:
3+
- marcelotrevisani
4+
tags: [conda]
5+
---
6+
7+
# By the power of Grayskull... I have the Conda recipe!
8+
9+
_The main goal of the Skeleto_~~n~~_r is to conquer Grayskull._
10+
11+
## Introduction
12+
13+
All jokes aside, the new project
14+
[grayskull](https://github.com/marcelotrevisani/grayskull) was created
15+
with the **intention** of generating better Conda recipes that would
16+
allow to package properly projects available in different channels such
17+
as PyPI, CRAN, Conan, GitHub register, GitHub repositories and so on. On
18+
top of that, Grayskull is also being developed to help
19+
[conda-forge](https://conda-forge.org/) to update recipes.
20+
21+
<!--truncate-->
22+
23+
## Current status
24+
25+
Currently, Grayskull (version `0.2.1`) is able to generate recipes just
26+
looking for packages on [PyPI](https://pypi.org/), and it is available
27+
on [PyPI](https://pypi.org/project/grayskull/) and
28+
[conda-forge](https://github.com/conda-forge/grayskull-feedstock). The
29+
GitHub repository for this package is:
30+
<https://github.com/marcelotrevisani/grayskull>.
31+
32+
Before Grayskull, we just had `conda-build skeleton` to generate recipes
33+
for Python packages on PyPI. In all other aspects, the difference of
34+
quality of the generated recipes, and also the time spent to generate
35+
them have a big discrepancy when compared to `conda-build skeleton` and
36+
`grayskull`. Grayskull generates recipes taking in consideration the
37+
platform, Python version available, selectors, compilers (Fortran, C and
38+
C++), packages constrains, license type, license file, and so forth. It
39+
uses metadata available from multiple sources to try to create the best
40+
recipe possible.
41+
42+
## Installation
43+
44+
You can install `grayskull` using `pip` or `conda`. `Grayskull` does not
45+
rely on `conda` to run and can generate recipes with minimum
46+
dependencies.
47+
48+
### With conda
49+
50+
Grayskull is available on the conda-forge channel.
51+
52+
```bash
53+
conda install -c conda-forge grayskull
54+
```
55+
56+
### With pip
57+
58+
```bash
59+
pip install grayskull
60+
```
61+
62+
## Grayskull vs conda-build skeleton
63+
64+
There are some differences of recipes generated by `grayskull` and
65+
`conda skeleton`. Taking as example the `pytest` recipe, which has
66+
selectors for platforms, Python version constrains, and has several
67+
package constrains as well.
68+
69+
### Grayskull (0.2.1) - took 4 seconds to generate the recipe
70+
71+
```yaml
72+
{% set name = "pytest" %}
73+
{% set version = "5.3.5" %}
74+
75+
package:
76+
name: {{ name|lower }}
77+
version: {{ version }}
78+
79+
source:
80+
url: https://pypi.io/packages/source/{{ name[0] }}/{{ name }}/{{ name }}-{{ version }}.tar.gz
81+
sha256: 0d5fe9189a148acc3c3eb2ac8e1ac0742cb7618c084f3d228baaec0c254b318d
82+
83+
build:
84+
number: 0
85+
skip: true # [py2k]
86+
entry_points:
87+
- pytest=pytest:main
88+
- py.test=pytest:main
89+
script: {{ PYTHON }} -m pip install . -vv
90+
91+
requirements:
92+
host:
93+
- pip
94+
- python
95+
- setuptools >=40.0
96+
- setuptools_scm
97+
run:
98+
- atomicwrites >=1.0 # [win]
99+
- attrs >=17.4.0
100+
- colorama # [win]
101+
- importlib-metadata >=0.12 # [py<38]
102+
- more-itertools >=4.0.0
103+
- packaging
104+
- pathlib2 >=2.2.0 # [py<36]
105+
- pluggy <1.0,>=0.12
106+
- py >=1.5.0
107+
- python
108+
- wcwidth
109+
110+
test:
111+
imports:
112+
- pytest
113+
commands:
114+
- pip check
115+
- pytest --help
116+
- py.test --help
117+
requires:
118+
- pip
119+
120+
about:
121+
home: https://pypi.org/project/pytest/
122+
summary: 'pytest: simple powerful testing with Python'
123+
dev_url: https://github.com/pytest-dev/pytest
124+
license: MIT
125+
license_file: LICENSE
126+
127+
extra:
128+
recipe-maintainers:
129+
- marcelotrevisani
130+
```
131+
132+
### Skeleton (3.18.11) - took 31 seconds to generate the recipe
133+
134+
```yaml
135+
{% set name = "pytest" %}
136+
{% set version = "5.3.5" %}
137+
138+
package:
139+
name: "{{ name|lower }}"
140+
version: "{{ version }}"
141+
142+
source:
143+
url: "https://pypi.io/packages/source/{{ name[0] }}/{{ name }}/{{ name }}-{{ version }}.tar.gz"
144+
sha256: 0d5fe9189a148acc3c3eb2ac8e1ac0742cb7618c084f3d228baaec0c254b318d
145+
146+
build:
147+
number: 0
148+
script: "{{ PYTHON }} -m pip install . -vv"
149+
150+
requirements:
151+
host:
152+
- atomicwrites >=1.0
153+
- attrs >=17.4.0
154+
- colorama;sys_platform =="win32"
155+
- importlib-metadata >=0.12
156+
- more-itertools >=4.0.0
157+
- packaging
158+
- pathlib2 >=2.2.0
159+
- pip
160+
- pluggy >=0.12,<1.0
161+
- py >=1.5.0
162+
- python
163+
- wcwidth
164+
run:
165+
- atomicwrites >=1.0
166+
- attrs >=17.4.0
167+
- colorama;sys_platform =="win32"
168+
- importlib-metadata >=0.12
169+
- more-itertools >=4.0.0
170+
- packaging
171+
- pathlib2 >=2.2.0
172+
- pluggy >=0.12,<1.0
173+
- py >=1.5.0
174+
- python
175+
- wcwidth
176+
177+
about:
178+
home: The package home page
179+
license: MIT
180+
license_family: MIT
181+
license_file:
182+
summary: "pytest: simple powerful testing with Python"
183+
doc_url:
184+
dev_url:
185+
186+
extra:
187+
recipe-maintainers:
188+
- your-github-id-here
189+
```
190+
191+
### Original recipe on `conda-forge` for `pytest 5.3.5`
192+
193+
```yaml
194+
{% set version = "5.3.5" %}
195+
196+
package:
197+
name: pytest
198+
version: {{ version }}
199+
200+
source:
201+
url: https://pypi.io/packages/source/p/pytest/pytest-{{ version }}.tar.gz
202+
sha256: 0d5fe9189a148acc3c3eb2ac8e1ac0742cb7618c084f3d228baaec0c254b318d
203+
204+
build:
205+
skip: True # [py27]
206+
number: 1
207+
script: "{{ PYTHON }} setup.py install --single-version-externally-managed --record record.txt"
208+
entry_points:
209+
- py.test = py.test:main
210+
- pytest = py.test:main
211+
212+
requirements:
213+
host:
214+
- pip
215+
- python
216+
- setuptools >=40.0
217+
- setuptools_scm
218+
run:
219+
- atomicwrites >=1.0 # [win]
220+
- attrs >=17.4.0
221+
- colorama # [win]
222+
- importlib_metadata >=0.12 # [py<38]
223+
- more-itertools >=4.0
224+
- packaging
225+
- pathlib2 >=2.2.0 # [py<36]
226+
- pluggy >=0.12,<1.0
227+
- py >=1.5.0
228+
- python
229+
- setuptools >=40.0
230+
- wcwidth
231+
run_constrained:
232+
# pytest-faulthandler 2 is a dummy package.
233+
# if an older version of fault-handler is installed, it will conflict with pytest >=5.
234+
- pytest-faulthandler >=2
235+
236+
test:
237+
commands:
238+
- pytest -h
239+
imports:
240+
- pytest
241+
242+
about:
243+
home: https://docs.pytest.org/en/latest/
244+
license: MIT
245+
license_file: LICENSE
246+
summary: 'Simple and powerful testing with Python.'
247+
description: |
248+
The pytest framework makes it easy to write small tests, yet scales to
249+
support complex functional testing for applications and libraries.
250+
doc_url: https://docs.pytest.org/en/latest/
251+
dev_url: https://github.com/pytest-dev/pytest/
252+
253+
extra:
254+
recipe-maintainers:
255+
- flub
256+
- goanpeca
257+
- nicoddemus
258+
- ocefpaf
259+
- mingwandroid
260+
```
261+
262+
### Major differences
263+
264+
| Attribute | Grayskull (0.2.1) | Skeleton (3.18.11) |
265+
| --------- | ----------------- | ------------------ |
266+
**Command** | ✅ grayskull pypi pytest | ✅ conda skeleton pypi pytest |
267+
**Time** | ✅ 4 seconds | ❌ 31 seconds |
268+
**License** | ✅ Added the license file and license type correctly | ❗️ Added just the license type |
269+
**Host Requirements** | ✅ Added correctly all the host requirements | ❌ it didn't add the correctly the host dependencies. It added unnecessary dependencies and it is missing quite a few of them necessary to build the package |
270+
**Run Requirements** | ✅ Missing just setuptools from the host requirements (but this dependency is not defined on pytest package) | ❌ incorrect dependencies added to the project |
271+
**Selectors** | ✅ Skipping correctly Python 2 and added selectors for windows and python versions | ❌ it didn't add any information regarding selectors. Actually conda-build added wrong information which will result in a broken recipe. For example 'sys_platform == win32' was added which is a wrong format for conda recipes |
272+
**Entry points** | ✅ Added all entry points correctly | ❌ No entry points |
273+
**Does it build?** | ✅ YES | ❌ NO |
274+
275+
In the case of `noarch: python`, Grayskull is smart enough to detect
276+
when the recipe supports it, which is not done by Skeleton. It is
277+
important to highlight that Skeleton does not detect compilers as well.
278+
Nevertheless, Grayskull always try to detect it.
279+
280+
## Usage Grayskull (0.2.1)
281+
282+
Project options:
283+
284+
```bash
285+
$ grayskull --help
286+
usage: grayskull [-h] [--version] {pypi} ...
287+
288+
Grayskull - Conda recipe generator
289+
290+
positional arguments:
291+
{pypi} Options to generate PyPI recipes
292+
pypi Generate recipes based on PyPI
293+
294+
optional arguments:
295+
-h, --help show this help message and exit
296+
--version, -v Print Grayskull version and exit
297+
```
298+
299+
```bash
300+
$ grayskull pypi --help
301+
usage: grayskull pypi [-h] [--maintainers MAINTAINERS [MAINTAINERS...]]
302+
[--output OUTPUT]
303+
pypi_packages [pypi_packages ...]
304+
305+
positional arguments:
306+
pypi_packages Specify the PyPI package name.
307+
308+
optional arguments:
309+
-h, --help show this help message and exit
310+
--maintainers MAINTAINERS [MAINTAINERS ...], -m MAINTAINERS [MAINTAINERS...] List of maintainers which will be added to the recipe.
311+
--output OUTPUT, -o OUTPUT Path to where the recipe will be created
312+
```
313+
314+
To generate the recipe you can just call `grayskull` and pass the
315+
channel (as for now we are just supporting PyPI, it should be pypi) and
316+
the package name. You should also specify an output folder using the
317+
option `--output` or `-o` and it will create the package folder, and the
318+
recipe in there. It is important to note that the user can specify a
319+
list of maintainers which will be added to the recipe using the option
320+
`--maintainers`.
321+
322+
Example for pytest:
323+
324+
![Grayskull CLI](https://raw.githubusercontent.com/marcelotrevisani/grayskull/15e4a0317da4e6c2f66a381329682b7e9dc70da0/docs/images/cli_example_grayskull.gif)
325+
326+
If you need to specify the package version you can do it just puting the
327+
equal sign after the package name and the version just right after that.
328+
Example:
329+
330+
```bash
331+
grayskull pypi requests=2.21.0
332+
```
333+
334+
or
335+
336+
```bash
337+
grayskull pypi requests==2.21.0
338+
```
339+
340+
![Grayskull pinned package -requests](https://raw.githubusercontent.com/marcelotrevisani/grayskull/15e4a0317da4e6c2f66a381329682b7e9dc70da0/docs/images/cli_example_grayskull_version.gif)
341+
342+
If you want to generate multiple recipes just pass a list of packages,
343+
such as:
344+
345+
```bash
346+
grayskul pypi pytest requests=2.21.0 colorama
347+
```
348+
349+
## Future plans
350+
351+
> - For the next major version (1.0.0) it is planned to add the
352+
> functionality to be able to load the recipe and update just parts
353+
> of it;
354+
> - Generate Conda recipes using CRAN (R) channel (2.0.0);
355+
> - Generate Conda recipes using Conan (C++) channel (3.0.0);
356+
357+
## Issues
358+
359+
Any problem, question, suggestions please feel free to open an issue on
360+
the repository: <https://github.com/marcelotrevisani/grayskull/issues>.
361+
362+
Contributions are very welcome! :)
363+
364+
------------------------------------------------------------------------
365+
366+
This work was possible thanks to the [NumFOCUS](https://numfocus.org/)
367+
Small Development Grant program.

0 commit comments

Comments
 (0)