Skip to content

Commit e2e1da1

Browse files
mischovmischov
committed
Add NIF precompilation
1 parent 24540b7 commit e2e1da1

File tree

12 files changed

+204
-31
lines changed

12 files changed

+204
-31
lines changed

‎.github/workflows/ci.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,7 @@ jobs:
3636

3737
env:
3838
MIX_ENV: test
39+
MEESEEKS_HTML5EVER_BUILD: "1"
3940

4041
strategy:
4142
fail-fast: false

‎.github/workflows/release.yml‎

Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
name: Release
2+
3+
on:
4+
push:
5+
branches:
6+
# Run for pushes to main
7+
- main
8+
paths:
9+
# If files in "native/" change
10+
- "native/**"
11+
# Or if this file changes
12+
- ".github/workflows/release.yml"
13+
tags:
14+
# Or run for any tag
15+
- "**"
16+
17+
jobs:
18+
precompile_nifs:
19+
name: Precompile NIF ${{ matrix.nif }} - ${{ matrix.job.target }} (${{ matrix.job.os }})
20+
21+
strategy:
22+
fail-fast: false
23+
matrix:
24+
nif: ["2.15", "2.16"]
25+
job:
26+
- { target: arm-unknown-linux-gnueabihf , os: ubuntu-20.04 , use-cross: true }
27+
- { target: aarch64-unknown-linux-gnu , os: ubuntu-20.04 , use-cross: true }
28+
- { target: aarch64-unknown-linux-musl , os: ubuntu-20.04 , use-cross: true }
29+
- { target: aarch64-apple-darwin , os: macos-11 }
30+
- { target: riscv64gc-unknown-linux-gnu , os: ubuntu-20.04 , use-cross: true }
31+
- { target: x86_64-apple-darwin , os: macos-11 }
32+
- { target: x86_64-unknown-linux-gnu , os: ubuntu-20.04 }
33+
- { target: x86_64-unknown-linux-musl , os: ubuntu-20.04 , use-cross: true }
34+
- { target: x86_64-pc-windows-gnu , os: windows-2019 }
35+
- { target: x86_64-pc-windows-msvc , os: windows-2019 }
36+
37+
runs-on: ${{ matrix.job.os }}
38+
39+
steps:
40+
- name: Checkout source code
41+
uses: actions/checkout@v3
42+
43+
- name: Extract project version
44+
shell: bash
45+
run: |
46+
# Get the project version from mix.exs
47+
echo "PROJECT_VERSION=$(sed -n 's/^ @version "\(.*\)"/\1/p' mix.exs | head -n1)" >> $GITHUB_ENV
48+
49+
- name: Install Rust toolchain
50+
uses: dtolnay/rust-toolchain@stable
51+
with:
52+
toolchain: stable
53+
target: ${{ matrix.job.target }}
54+
55+
- name: Build the project
56+
id: build-crate
57+
uses: philss/[email protected]
58+
with:
59+
project-name: meeseeks_html5ever_nif
60+
project-version: ${{ env.PROJECT_VERSION }}
61+
target: ${{ matrix.job.target }}
62+
nif-version: ${{ matrix.nif }}
63+
use-cross: ${{ matrix.job.use-cross }}
64+
project-dir: "native/meeseeks_html5ever_nif"
65+
66+
- name: Artifact upload
67+
uses: actions/upload-artifact@v3
68+
with:
69+
name: ${{ steps.build-crate.outputs.file-name }}
70+
path: ${{ steps.build-crate.outputs.file-path }}
71+
72+
- name: Publish archives and packages
73+
uses: softprops/action-gh-release@v1
74+
with:
75+
files: |
76+
${{ steps.build-crate.outputs.file-path }}
77+
if: startsWith(github.ref, 'refs/tags/')

‎.gitignore‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,3 +22,6 @@ erl_crash.dump
2222
/priv/native
2323

2424
/native/*/target
25+
26+
# The checksum files for precompiled NIFs
27+
checksum-*.exs

‎CHANGELOG.md‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,12 +6,13 @@
66

77
* No longer support Elixir versions under 1.12 or Erlang/OTP versions under 23.0
88
* Support Elixir 1.13 and 1.14 and Erlang/OTP 25.0
9-
* Use Rustler v0.27
109

1110
### Enhancements
1211

1312
* Use Rust 2018 edition
14-
* Update to latest versions of `html5ever`, `xml5ever`, and `rustler`
13+
* Update to Rustler `v0.27`
14+
* Update to latest versions of Html5ever and Xml5ever
15+
* Use `rustler_precompiled` to precompile NIFs
1516

1617
### Fixes
1718

‎README.md‎

Lines changed: 23 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -9,10 +9,6 @@ Originally a fork of Hansihe's [html5ever_elixir](https://github.com/hansihe/htm
99

1010
Meeseeks_Html5ever requires a minimum combination of Elixir 1.12.0 and Erlang/OTP 23.0, and is tested with a maximum combination of Elixir 1.14.0 and Erlang/OTP 25.0.
1111

12-
## Dependencies
13-
14-
Meeseeks_Html5ever depends on the Rust library [html5ever](https://github.com/servo/html5ever), and you will need to have the Rust compiler [installed](https://www.rust-lang.org/en-US/install.html).
15-
1612
## Installation
1713

1814
Ensure Rust is installed, then add Meeseeks_Html5ever to your `mix.exs`:
@@ -27,6 +23,29 @@ end
2723

2824
Finally, run `mix deps.get`.
2925

26+
## Dependencies
27+
28+
Meeseeks_Html5ever depends on the Rust library [html5ever](https://github.com/servo/html5ever), providing a Rustler-based NIF to interface with it.
29+
30+
You do not need to have Rust installed because the library will attempt to download a precompiled NIF file.
31+
32+
To force compilation you can either set the `MEESEEKS_HTML5EVER_BUILD` environment variable to `true` or `1`, or add the following application configuration
33+
34+
```elixir
35+
config :meeseeks_html5ever, MeeseeksHtml5ever, build_from_source: true
36+
```
37+
38+
If you want to force compilation you will need to have the Rust compiler [installed](https://www.rust-lang.org/en-US/install.html), and will need to add Rustler to your dependencies.
39+
40+
```elixir
41+
def deps do
42+
[
43+
{:meeseeks_html5ever, "~> 0.13.1"},
44+
{:rustler, ">= 0.0.0", optional: true}
45+
]
46+
end
47+
```
48+
3049
## Contributing
3150

3251
If you are interested in contributing please read the [contribution guidelines](CONTRIBUTING.md).

‎RELEASE_CHECKLIST.md‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
# Release checklist
2+
3+
In order to release a new version to Hex.pm we first need to:
4+
5+
1. Write the changes in the `CHANGELOG.md` file
6+
1. Create a release branch named `release/<version>`
7+
1. Update the `README.md`, `CHANGELOG.md`, `mix.exs`, and `Cargo.toml` with the new version
8+
1. Commit with message `Release <version>`
9+
1. Merge PR to `main`
10+
1. Tag main with `git tag <version>`
11+
1. Push tag with `git push origin <version>`
12+
1. Wait for the CI to build all release files
13+
1. Run `mix rustler.download MeeseeksHtml5ever.Native --all --print`
14+
1. Copy the output of the mix task and add to the release notes
15+
1. Run `mix hex.publish` and **make sure the checksum file is present**
16+
in the list of files to be published.
17+
18+
It's important to ensure that we publish the checksum file with the
19+
package because otherwise the users won't be able to use the lib
20+
with precompiled files. They will need to always enforce compilation.

‎lib/meeseeks_html5ever.ex‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,20 @@ defmodule MeeseeksHtml5ever do
33
MeeseeksHtml5ever is intended for internal use by
44
[Meeseeks](https://github.com/mischov/meeseeks), and parses HTML or XML into
55
a `Meeseeks.Document`.
6+
7+
By default this lib will try to use a precompiled NIF from the GitHub
8+
releases page. This way you don't need to have the Rust toolchain installed.
9+
In case no precompiled file is found and the Mix env is production then an
10+
error is raised.
11+
12+
You can force the compilation to occur by setting the value of the
13+
`MEESEEKS_HTML5EVER_BUILD` environment variable to "true" or "1".
14+
Alternatively you can also set the application env `:build_from_source` to
15+
`true` in order to force the build:
16+
17+
```
18+
config :meeseeks_html5ever, MeeseeksHtml5ever, build_from_source: true
19+
```
620
"""
721

822
@doc """

‎lib/meeseeks_html5ever/native.ex‎

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,20 @@
11
defmodule MeeseeksHtml5ever.Native do
22
@moduledoc false
33

4-
use Rustler, otp_app: :meeseeks_html5ever, crate: :meeseeks_html5ever_nif, mode: :release
4+
mix_config = Mix.Project.config()
5+
version = mix_config[:version]
6+
github_url = mix_config[:package][:links]["GitHub"]
7+
env_config = Application.compile_env(:meeseeks_html5ever, MeeseeksHtml5ever, [])
8+
9+
use RustlerPrecompiled,
10+
otp_app: :meeseeks_html5ever,
11+
crate: "meeseeks_html5ever_nif",
12+
mode: :release,
13+
base_url: "#{github_url}/releases/download/v#{version}",
14+
force_build:
15+
System.get_env("MEESEEKS_HTML5EVER_BUILD") in ["1", "true"] or
16+
env_config[:build_from_source],
17+
version: version
518

619
def parse_html(_binary), do: err()
720
def parse_xml(_binary), do: err()

‎mix.exs‎

Lines changed: 25 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -1,22 +1,24 @@
11
defmodule MeeseeksHtml5ever.Mixfile do
22
use Mix.Project
33

4+
@description "Meeseeks-specific NIF binding of html5ever using Rustler"
5+
@source_url "https://github.com/mischov/meeseeks_html5ever"
46
@version "0.13.1"
57

68
def project do
79
[
810
app: :meeseeks_html5ever,
9-
name: "MeeseeksHtml5ever",
1011
version: @version,
11-
description: description(),
1212
elixir: "~> 1.12",
1313
deps: deps(),
14+
15+
# Hex
16+
description: @description,
1417
package: package(),
15-
source_url: "https://github.com/mischov/meeseeks_html5ever",
16-
docs: docs(),
17-
build_embedded: Mix.env() == :prod,
18-
start_permanent: Mix.env() == :prod,
19-
compilers: Mix.compilers()
18+
19+
# HexDocs
20+
name: "MeeseeksHtml5ever",
21+
docs: docs()
2022
]
2123
end
2224

@@ -26,17 +28,23 @@ defmodule MeeseeksHtml5ever.Mixfile do
2628

2729
defp deps do
2830
[
29-
{:rustler, "~> 0.27.0"},
31+
{:rustler_precompiled, "~> 0.6.1"},
32+
33+
# Optional
34+
{:rustler, ">= 0.0.0", optional: true},
3035

31-
# docs
32-
{:ex_doc, "~> 0.21.0", only: :docs, runtime: false}
36+
# Docs
37+
{:ex_doc, "~> 0.24.0", only: :docs, runtime: false}
3338
]
3439
end
3540

36-
defp description do
37-
"""
38-
Meeseeks-specific NIF binding of html5ever using Rustler.
39-
"""
41+
defp docs do
42+
[
43+
main: "MeeseeksHtml5ever",
44+
source_url: @source_url,
45+
source_ref: "v#{@version}",
46+
extras: ["CHANGELOG.md"]
47+
]
4048
end
4149

4250
defp package do
@@ -46,17 +54,15 @@ defmodule MeeseeksHtml5ever.Mixfile do
4654
files: [
4755
"lib",
4856
"native",
57+
"checksum-*.exs",
4958
"priv/.gitkeep",
5059
"mix.exs",
5160
"README.md",
61+
"CHANGELOG.md",
5262
"LICENSE-MIT",
5363
"LICENSE-APACHE"
5464
],
55-
links: %{"GitHub" => "https://github.com/mischov/meeseeks_html5ever"}
65+
links: %{"GitHub" => @source_url}
5666
]
5767
end
58-
59-
defp docs do
60-
[main: "MeeseeksHtml5ever"]
61-
end
6268
end

‎mix.lock‎

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,16 @@
11
%{
2+
"castore": {:hex, :castore, "1.0.1", "240b9edb4e9e94f8f56ab39d8d2d0a57f49e46c56aced8f873892df8ff64ff5a", [:mix], [], "hexpm", "b4951de93c224d44fac71614beabd88b71932d0b1dea80d2f80fb9044e01bbb3"},
23
"earmark": {:hex, :earmark, "1.4.15", "2c7f924bf495ec1f65bd144b355d0949a05a254d0ec561740308a54946a67888", [:mix], [{:earmark_parser, ">= 1.4.13", [hex: :earmark_parser, repo: "hexpm", optional: false]}], "hexpm", "3b1209b85bc9f3586f370f7c363f6533788fb4e51db23aa79565875e7f9999ee"},
3-
"earmark_parser": {:hex, :earmark_parser, "1.4.13", "0c98163e7d04a15feb62000e1a891489feb29f3d10cb57d4f845c405852bbef8", [:mix], [], "hexpm", "d602c26af3a0af43d2f2645613f65841657ad6efc9f0e361c3b6c06b578214ba"},
4-
"ex_doc": {:hex, :ex_doc, "0.21.3", "857ec876b35a587c5d9148a2512e952e24c24345552259464b98bfbb883c7b42", [:mix], [{:earmark, "~> 1.4", [hex: :earmark, repo: "hexpm", optional: false]}, {:makeup_elixir, "~> 0.14", [hex: :makeup_elixir, repo: "hexpm", optional: false]}], "hexpm", "0db1ee8d1547ab4877c5b5dffc6604ef9454e189928d5ba8967d4a58a801f161"},
4+
"earmark_parser": {:hex, :earmark_parser, "1.4.31", "a93921cdc6b9b869f519213d5bc79d9e218ba768d7270d46fdcf1c01bacff9e2", [:mix], [], "hexpm", "317d367ee0335ef037a87e46c91a2269fef6306413f731e8ec11fc45a7efd059"},
5+
"ex_doc": {:hex, :ex_doc, "0.24.2", "e4c26603830c1a2286dae45f4412a4d1980e1e89dc779fcd0181ed1d5a05c8d9", [:mix], [{:earmark_parser, "~> 1.4.0", [hex: :earmark_parser, repo: "hexpm", optional: false]}, {:makeup_elixir, "~> 0.14", [hex: :makeup_elixir, repo: "hexpm", optional: false]}, {:makeup_erlang, "~> 0.1", [hex: :makeup_erlang, repo: "hexpm", optional: false]}], "hexpm", "e134e1d9e821b8d9e4244687fb2ace58d479b67b282de5158333b0d57c6fb7da"},
56
"hoedown": {:git, "https://github.com/hoedown/hoedown.git", "980b9c549b4348d50b683ecee6abee470b98acda", []},
67
"jason": {:hex, :jason, "1.4.0", "e855647bc964a44e2f67df589ccf49105ae039d4179db7f6271dfd3843dc27e6", [:mix], [{:decimal, "~> 1.0 or ~> 2.0", [hex: :decimal, repo: "hexpm", optional: true]}], "hexpm", "79a3791085b2a0f743ca04cec0f7be26443738779d09302e01318f97bdb82121"},
7-
"makeup": {:hex, :makeup, "1.0.5", "d5a830bc42c9800ce07dd97fa94669dfb93d3bf5fcf6ea7a0c67b2e0e4a7f26c", [:mix], [{:nimble_parsec, "~> 0.5 or ~> 1.0", [hex: :nimble_parsec, repo: "hexpm", optional: false]}], "hexpm", "cfa158c02d3f5c0c665d0af11512fed3fba0144cf1aadee0f2ce17747fba2ca9"},
8-
"makeup_elixir": {:hex, :makeup_elixir, "0.15.1", "b5888c880d17d1cc3e598f05cdb5b5a91b7b17ac4eaf5f297cb697663a1094dd", [:mix], [{:makeup, "~> 1.0", [hex: :makeup, repo: "hexpm", optional: false]}, {:nimble_parsec, "~> 1.1", [hex: :nimble_parsec, repo: "hexpm", optional: false]}], "hexpm", "db68c173234b07ab2a07f645a5acdc117b9f99d69ebf521821d89690ae6c6ec8"},
8+
"makeup": {:hex, :makeup, "1.1.0", "6b67c8bc2882a6b6a445859952a602afc1a41c2e08379ca057c0f525366fc3ca", [:mix], [{:nimble_parsec, "~> 1.2.2 or ~> 1.3", [hex: :nimble_parsec, repo: "hexpm", optional: false]}], "hexpm", "0a45ed501f4a8897f580eabf99a2e5234ea3e75a4373c8a52824f6e873be57a6"},
9+
"makeup_elixir": {:hex, :makeup_elixir, "0.16.0", "f8c570a0d33f8039513fbccaf7108c5d750f47d8defd44088371191b76492b0b", [:mix], [{:makeup, "~> 1.0", [hex: :makeup, repo: "hexpm", optional: false]}, {:nimble_parsec, "~> 1.2.3", [hex: :nimble_parsec, repo: "hexpm", optional: false]}], "hexpm", "28b2cbdc13960a46ae9a8858c4bebdec3c9a6d7b4b9e7f4ed1502f8159f338e7"},
10+
"makeup_erlang": {:hex, :makeup_erlang, "0.1.1", "3fcb7f09eb9d98dc4d208f49cc955a34218fc41ff6b84df7c75b3e6e533cc65f", [:mix], [{:makeup, "~> 1.0", [hex: :makeup, repo: "hexpm", optional: false]}], "hexpm", "174d0809e98a4ef0b3309256cbf97101c6ec01c4ab0b23e926a9e17df2077cbb"},
911
"markdown": {:git, "https://github.com/devinus/markdown.git", "d065dbcc4e242a85ca2516fdadd0082712871fd8", []},
10-
"nimble_parsec": {:hex, :nimble_parsec, "1.1.0", "3a6fca1550363552e54c216debb6a9e95bd8d32348938e13de5eda962c0d7f89", [:mix], [], "hexpm", "08eb32d66b706e913ff748f11694b17981c0b04a33ef470e33e11b3d3ac8f54b"},
12+
"nimble_parsec": {:hex, :nimble_parsec, "1.2.3", "244836e6e3f1200c7f30cb56733fd808744eca61fd182f731eac4af635cc6d0b", [:mix], [], "hexpm", "c8d789e39b9131acf7b99291e93dae60ab48ef14a7ee9d58c6964f59efb570b0"},
1113
"rustler": {:hex, :rustler, "0.27.0", "53ffe86586fd1a2ea60ad07f1506962914eb669dba26c23010cf672662ec8d64", [:mix], [{:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: false]}, {:toml, "~> 0.6", [hex: :toml, repo: "hexpm", optional: false]}], "hexpm", "d7f5ccaec6e7a96f700330898ff2e9d48818e40789fd2951ba41ecf457986e92"},
14+
"rustler_precompiled": {:hex, :rustler_precompiled, "0.6.1", "160b545bce8bf9a3f1b436b2c10f53574036a0db628e40f393328cbbe593602f", [:mix], [{:castore, "~> 0.1 or ~> 1.0", [hex: :castore, repo: "hexpm", optional: false]}, {:rustler, "~> 0.23", [hex: :rustler, repo: "hexpm", optional: true]}], "hexpm", "0dd269fa261c4e3df290b12031c575fff07a542749f7b0e8b744d72d66c43600"},
1215
"toml": {:hex, :toml, "0.7.0", "fbcd773caa937d0c7a02c301a1feea25612720ac3fa1ccb8bfd9d30d822911de", [:mix], [], "hexpm", "0690246a2478c1defd100b0c9b89b4ea280a22be9a7b313a8a058a2408a2fa70"},
1316
}

0 commit comments

Comments
 (0)