Skip to content

Commit a04d28f

Browse files
ehildenbdwightguth
ehildenb
authored and
dwightguth
committed
Test release builds using Firefly (#407)
* Jenkinsfile: correct directory for arch dockerfile * Jenkinsfile: RELEASE_ID => KEVM_RELEASE_ID variable name * Jenkinsfile, deps/k_release, package/Dockerfile: retrieve K_RELEASE to use from file * Jenkinsfile: set K_RELEASE after checkout * Jenkinsfile: escape literal $ * Jenkinsfile: capture stdout for getting K_RELEASE * package/Dockerfile: echo the K_RELEASE being used, avoid setting default * Makefile: add test-interactive-firefly target * Jenkinsfile, package/Dockerfile: run firefly tests on releases * Jenkinsfile, Dockerfile, package/Dockerfile.ubuntu-bionic: apt => apt-get * package/Dockerfile.ubuntu-bionic: remove npm direct dependency * package/Dockerfile.ubuntu-bionic: add sudo, create user manually * Jenkinsfile: correction to installing kevm * package/Dockerfile.ubuntu-bionic: get node 10 directly * package/Dockerfile.ubuntu-bionic: install nodejs after setting up repos * Makefile: run npm install after npx kevm-ganache-cli * Jenkinsfile, package/Dockerfile: install npx in Dockerfiles * Makefile: correctly run in openzeppelin subdirectory * Makefile: keep background process from confusing shell && * Makefile: change into correct subdir * Jenkinsfile: correct variable name for sh script * Jenkinsfile: correct path to kevm-git package * package/Dockerfile.arch: install base and base-devel * Jenkinsfile: make sure K_RELEASE is set on Deploy phase * Makefile: blow away openzeppelin-solidity directory if it exists * Jenkinsfile: use git_commit variable directly * Jenkinsfile: run upload block in repo subdir * INSTALL: avoid you in install text * Jenkinsfile: tag commit before pushing to avoid race * Jenkinsfile: skip tag creation * Makefile, Jenkinsfile: build release file as make target * Jenkinsfile: use HEAD for commitish * deps/k_release: update release dependency * deps/k: update submodule dependency * Jenkinsfile, package/Dockerfile: install K release after updating containers * Jenkinsfile: release dockerfiles dont need K_RELEASE anymore * Jenkinsfile: rename stage * Jenkinsfile: get k_release from file directly instead of extra env setup * Jenkinsfile: update failure message * Jenkinsfile: add --yes to apt-get instructions * package/Dockerfile.ubuntu-bionic: no need for `jq` * deps/k_release: correct URL * Jenkinsfile: --noconfirm option to pacman * Makefile: peg specific commit of open zeppelin, run all tests * Jenkinsfile, deps/k_release: get release from submodule peg * Jenkinsfile: change to deps/k in subshell * Makefile: only run ERC20 test on firefly * Makefile: formatting * Jenkinsfile: use current HEAD for release commit * Jenkinsfile: fix typo * Jenkinsfile: correct RELEASE_ID => KEVM_RELEASE_ID * .ci: remove old file * Jenkinsfile: build source tarball first * Jenkinsfile: no committish flag
1 parent f5b935c commit a04d28f

File tree

8 files changed

+155
-267
lines changed

8 files changed

+155
-267
lines changed

‎.ci

Lines changed: 0 additions & 209 deletions
This file was deleted.

‎Dockerfile

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -4,12 +4,12 @@ ENV TZ=America/Chicago
44
RUN ln --symbolic --no-dereference --force /usr/share/zoneinfo/$TZ /etc/localtime \
55
&& echo $TZ > /etc/timezone
66

7-
RUN apt update \
8-
&& apt upgrade --yes \
9-
&& apt install --yes \
10-
autoconf bison clang-8 cmake curl flex gcc libboost-test-dev \
7+
RUN apt-get update \
8+
&& apt-get upgrade --yes \
9+
&& apt-get install --yes \
10+
autoconf bison clang-8 cmake curl flex gcc libboost-test-dev \
1111
libcrypto++-dev libffi-dev libjemalloc-dev libmpfr-dev libprocps-dev \
12-
libsecp256k1-dev libssl-dev libtool libyaml-dev lld-8 llvm-8-tools \
12+
libsecp256k1-dev libssl-dev libtool libyaml-dev lld-8 llvm-8-tools \
1313
make maven opam openjdk-8-jdk pandoc pkg-config python3 python-pygments \
1414
python-recommonmark python-sphinx time zlib1g-dev protobuf-compiler \
1515
libprotobuf-dev

‎INSTALL.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,7 @@ cp -r package/debian ./
5252
dpkg-buildpackage --no-sign
5353
```
5454

55-
This will throw an error for any build dependencies you're missing, install them with `sudo apt install ...`.
55+
This will throw an error for any missing build dependencies, install them with `sudo apt install ...`.
5656
The `kevm_X.Y.Z_amd64_ID.deb` package will be placed one directory up from the repository root.
5757

5858
### Arch

‎Jenkinsfile

Lines changed: 106 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ pipeline {
66
stages {
77
stage("Init title") {
88
when {
9-
changeRequest()
9+
changeRequest()
1010
beforeAgent true
1111
}
1212
steps {
@@ -148,7 +148,7 @@ pipeline {
148148
}
149149
}
150150
}
151-
stage('Deploy') {
151+
stage('Release') {
152152
when {
153153
not { changeRequest() }
154154
branch 'master'
@@ -157,89 +157,161 @@ pipeline {
157157
agent { label 'docker' }
158158
options { skipDefaultCheckout() }
159159
environment {
160-
GITHUB_TOKEN = credentials('rv-jenkins')
161-
RELEASE_ID = '1.0.0'
160+
GITHUB_TOKEN = credentials('rv-jenkins')
161+
KEVM_RELEASE_ID = '1.0.0'
162162
}
163163
stages {
164-
stage('Checkout SCM') {
165-
steps { dir("kevm-${env.RELEASE_ID}") { checkout scm } }
164+
stage('Checkout SCM - Download K Release') {
165+
steps {
166+
dir("kevm-${env.KEVM_RELEASE_ID}") {
167+
checkout scm
168+
sh '''
169+
commit_short=$(cd deps/k && git rev-parse --short HEAD)
170+
K_RELEASE="https://github.com/kframework/k/releases/download/nightly-$commit_short"
171+
curl --fail --location "${K_RELEASE}/kframework_5.0.0_amd64_bionic.deb" --output kframework.deb
172+
curl --fail --location "${K_RELEASE}/kframework-5.0.0-1-x86_64.pkg.tar.xz" --output kframework-git.pkg.tar.xz
173+
'''
174+
stash name: 'bionic-kframework', includes: 'kframework.deb'
175+
stash name: 'arch-kframework', includes: 'kframework-git.pkg.tar.xz'
176+
}
177+
}
166178
}
167-
stage('Build Ubuntu Package') {
179+
stage('Build Source Tarball') {
168180
agent {
169181
dockerfile {
170-
dir "kevm-${env.RELEASE_ID}/package"
182+
dir "kevm-${env.KEVM_RELEASE_ID}/package"
183+
filename 'Dockerfile.arch'
184+
additionalBuildArgs '--build-arg USER_ID=$(id -u) --build-arg GROUP_ID=$(id -g)'
185+
reuseNode true
186+
}
187+
}
188+
steps {
189+
dir("kevm-${KEVM_RELEASE_ID}-src") {
190+
checkout scm
191+
sh '''
192+
find . -name .git | xargs rm -r
193+
rm -r deps/k tests/ethereum-tests deps/metropolis
194+
cd ..
195+
tar czvf kevm-${KEVM_RELEASE_ID}-src.tar.gz kevm-${KEVM_RELEASE_ID}-src
196+
'''
197+
}
198+
stash name: 'src-kevm', includes: "kevm-${env.KEVM_RELEASE_ID}-src.tar.gz"
199+
}
200+
}
201+
stage('Build Ubuntu Bionic Package') {
202+
agent {
203+
dockerfile {
204+
dir "kevm-${env.KEVM_RELEASE_ID}/package"
171205
filename 'Dockerfile.ubuntu-bionic'
206+
additionalBuildArgs '--build-arg USER_ID=$(id -u) --build-arg GROUP_ID=$(id -g)'
172207
reuseNode true
173208
}
174209
}
175210
steps {
176-
dir("kevm-${env.RELEASE_ID}") {
211+
dir("kevm-${env.KEVM_RELEASE_ID}") {
212+
unstash 'bionic-kframework'
177213
sh '''
178-
sudo apt update && sudo apt upgrade --yes
214+
sudo apt-get update && sudo apt-get upgrade --yes
215+
sudo apt-get install --yes ./kframework.deb
179216
cp -r package/debian ./
180217
dpkg-buildpackage --no-sign
181218
'''
182219
}
183-
stash name: 'bionic', includes: "kevm_${env.RELEASE_ID}_amd64.deb"
220+
stash name: 'bionic-kevm', includes: "kevm_${env.KEVM_RELEASE_ID}_amd64.deb"
221+
}
222+
}
223+
stage('Test Ubuntu Bionic Package') {
224+
agent {
225+
dockerfile {
226+
dir "kevm-${env.KEVM_RELEASE_ID}/package"
227+
filename 'Dockerfile.ubuntu-bionic'
228+
additionalBuildArgs '--build-arg USER_ID=$(id -u) --build-arg GROUP_ID=$(id -g)'
229+
reuseNode true
230+
}
231+
}
232+
steps {
233+
dir("kevm-${env.KEVM_RELEASE_ID}") {
234+
unstash 'bionic-kevm'
235+
sh '''
236+
sudo apt-get update && sudo apt-get upgrade --yes
237+
sudo apt-get install --yes ./kevm_${KEVM_RELEASE_ID}_amd64.deb
238+
make test-interactive-firefly
239+
'''
240+
}
184241
}
185242
}
186243
stage('Build Arch Package') {
187244
agent {
188245
dockerfile {
189-
dir "kevm-${env.RELEASE_ID}/package"
246+
dir "kevm-${env.KEVM_RELEASE_ID}/package"
190247
filename 'Dockerfile.arch'
191248
additionalBuildArgs '--build-arg USER_ID=$(id -u) --build-arg GROUP_ID=$(id -g)'
192249
reuseNode true
193250
}
194251
}
195252
steps {
196-
dir("kevm-${env.RELEASE_ID}") {
253+
dir("kevm-${env.KEVM_RELEASE_ID}") {
254+
unstash 'arch-kframework'
197255
sh '''
198256
sudo pacman -Syu --noconfirm
257+
sudo pacman --noconfirm -U kframework-git.pkg.tar.xz
199258
cd package
200259
makepkg --noconfirm --syncdeps
201260
'''
202261
}
203-
stash name: 'arch', includes: "kevm-${env.RELEASE_ID}/package/kevm-git-${env.RELEASE_ID}-1-x86_64.pkg.tar.xz"
262+
stash name: 'arch-kevm', includes: "kevm-${env.KEVM_RELEASE_ID}/package/kevm-git-${env.KEVM_RELEASE_ID}-1-x86_64.pkg.tar.xz"
204263
}
205264
}
206-
stage('Upload Packages') {
265+
stage('Test Arch Package') {
207266
agent {
208267
dockerfile {
209-
dir 'package'
268+
dir "kevm-${env.KEVM_RELEASE_ID}/package"
210269
filename 'Dockerfile.arch'
270+
additionalBuildArgs '--build-arg USER_ID=$(id -u) --build-arg GROUP_ID=$(id -g)'
211271
reuseNode true
212272
}
213273
}
214274
steps {
215-
unstash 'bionic'
216-
unstash 'arch'
217-
dir("kevm-${env.RELEASE_ID}") {
218-
checkout scm
275+
dir("kevm-${env.KEVM_RELEASE_ID}") {
276+
unstash 'arch-kevm'
219277
sh '''
220-
find . -name .git | xargs rm -r
221-
rm -r deps/k tests/ethereum-tests deps/metropolis
222-
cd ..
223-
tar czvf kevm-${RELEASE_ID}.tar.gz kevm-${RELEASE_ID}
278+
sudo pacman -Syu --noconfirm
279+
sudo pacman --noconfirm -U package/kevm-git-${KEVM_RELEASE_ID}-1-x86_64.pkg.tar.xz
280+
make test-interactive-firefly
224281
'''
225282
}
283+
}
284+
}
285+
stage('Upload Release') {
286+
agent {
287+
dockerfile {
288+
dir "kevm-${env.KEVM_RELEASE_ID}/package"
289+
filename 'Dockerfile.arch'
290+
additionalBuildArgs '--build-arg USER_ID=$(id -u) --build-arg GROUP_ID=$(id -g)'
291+
reuseNode true
292+
}
293+
}
294+
steps {
295+
unstash 'src-kevm'
296+
unstash 'bionic-kevm'
297+
unstash 'arch-kevm'
226298
sh '''
227-
git_commit=$(cd kevm-$RELEASE_ID && git rev-parse --short HEAD)
228-
hub release create \
229-
--attach "kevm_${RELEASE_ID}_amd64.deb#Ubuntu Bionic (18.04) Package" \
230-
--attach "kevm-${RELEASE_ID}/package/kevm-git-${RELEASE_ID}-1-x86_64.pkg.tar.xz#Arch Package" \
231-
--attach "kevm-${RELEASE_ID}.tar.gz#Source tar.gz" \
232-
--message "$(echo -e "KEVM Release $RELEASE_ID - $git_commit\n\n" ; cat kevm-${RELEASE_ID}/INSTALL.md ;)" \
233-
--commitish $(git rev-parse HEAD) "v$RELEASE_ID-$git_commit"
299+
release_tag="v${KEVM_RELEASE_ID}-$(git rev-parse --short HEAD)"
300+
make release.md KEVM_RELEASE_TAG=${release_tag}
301+
hub release create \
302+
--attach "kevm-${KEVM_RELEASE_ID}-src.tar.gz#Source tar.gz" \
303+
--attach "kevm_${KEVM_RELEASE_ID}_amd64.deb#Ubuntu Bionic (18.04) Package" \
304+
--attach "kevm-${KEVM_RELEASE_ID}/package/kevm-git-${KEVM_RELEASE_ID}-1-x86_64.pkg.tar.xz#Arch Package" \
305+
--file "release.md" "${release_tag}"
234306
'''
235307
}
236308
}
237309
}
238310
post {
239311
failure {
240-
slackSend color: '#cb2431' \
241-
, channel: '#kevm' \
242-
, message: "KEVM Packaging Failed: ${env.BUILD_URL}"
312+
slackSend color: '#cb2431' \
313+
, channel: '#kevm' \
314+
, message: "KEVM Release Failed: ${env.BUILD_URL}"
243315
}
244316
}
245317
}

‎Makefile

Lines changed: 25 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,7 @@ export LUA_PATH
4141
test test-all test-conformance test-slow-conformance test-all-conformance \
4242
test-vm test-slow-vm test-all-vm test-bchain test-slow-bchain test-all-bchain \
4343
test-proof test-klab-prove test-parse test-failure \
44-
test-interactive test-interactive-help test-interactive-run test-interactive-prove test-interactive-search \
44+
test-interactive test-interactive-help test-interactive-run test-interactive-prove test-interactive-search test-interactive-firefly \
4545
media media-pdf sphinx metropolis-theme
4646
.SECONDARY:
4747

@@ -306,11 +306,11 @@ $(DEFN_DIR)/node/$(MAIN_DEFN_FILE)-kompiled/plugin/proto/msg.pb.cc: $(PLUGIN_SUB
306306
$(node_kompiled): $(DEFN_DIR)/node/$(MAIN_DEFN_FILE)-kompiled/interpreter $(libff_out)
307307
mkdir -p $(DEFN_DIR)/vm
308308
$(K_BIN)/llvm-kompile $(DEFN_DIR)/node/$(MAIN_DEFN_FILE)-kompiled/definition.kore $(DEFN_DIR)/node/$(MAIN_DEFN_FILE)-kompiled/dt library $(PLUGIN_SUBMODULE)/vm-c/main.cpp $(PLUGIN_SUBMODULE)/vm-c/vm.cpp \
309-
$(PLUGIN_SUBMODULE)/plugin-c/*.cpp $(DEFN_DIR)/node/$(MAIN_DEFN_FILE)-kompiled/plugin/proto/msg.pb.cc $(PLUGIN_SUBMODULE)/vm-c/kevm/semantics.cpp -o $@ -g -O2 \
310-
-I $(PLUGIN_SUBMODULE)/plugin-c/ -I $(DEFN_DIR)/node/$(MAIN_DEFN_FILE)-kompiled/plugin -I $(PLUGIN_SUBMODULE)/vm-c/ -I $(PLUGIN_SUBMODULE)/vm-c/kevm/ -I node/ \
311-
$(LLVM_KOMPILE_OPTS) \
312-
-L$(LIBRARY_PATH) \
313-
-lff -lprotobuf -lgmp $(LINK_PROCPS) -lcryptopp -lsecp256k1
309+
$(PLUGIN_SUBMODULE)/plugin-c/*.cpp $(DEFN_DIR)/node/$(MAIN_DEFN_FILE)-kompiled/plugin/proto/msg.pb.cc $(PLUGIN_SUBMODULE)/vm-c/kevm/semantics.cpp -o $@ -g -O2 \
310+
-I $(PLUGIN_SUBMODULE)/plugin-c/ -I $(DEFN_DIR)/node/$(MAIN_DEFN_FILE)-kompiled/plugin -I $(PLUGIN_SUBMODULE)/vm-c/ -I $(PLUGIN_SUBMODULE)/vm-c/kevm/ -I node/ \
311+
$(LLVM_KOMPILE_OPTS) \
312+
-L$(LIBRARY_PATH) \
313+
-lff -lprotobuf -lgmp $(LINK_PROCPS) -lcryptopp -lsecp256k1
314314

315315
# LLVM Backend
316316

@@ -329,10 +329,17 @@ $(llvm_kompiled): $(llvm_files) $(libff_out)
329329
# Installing
330330
# ----------
331331

332+
KEVM_RELEASE_TAG?=
333+
332334
install: $(node_kompiled)
333335
mkdir -p $(INSTALL_DIR)
334336
cp $(node_kompiled) $(INSTALL_DIR)/
335337

338+
release.md: INSTALL.md
339+
echo "KEVM Release $(KEVM_RELEASE_TAG)" > $@
340+
echo >> $@
341+
cat INSTALL.md >> $@
342+
336343
# Tests
337344
# -----
338345

@@ -467,6 +474,18 @@ test-interactive-search: $(search_tests:=.search)
467474
test-interactive-help:
468475
$(TEST) help
469476

477+
# Notice that `npm install` comes after `npx kevm-ganache-cli` to allow time for it to start up.
478+
test-interactive-firefly:
479+
mkdir -p $(BUILD_DIR)/firefly
480+
cd $(BUILD_DIR)/firefly \
481+
&& rm -rf openzeppelin-solidity \
482+
&& git clone 'https://github.com/openzeppelin/openzeppelin-solidity' \
483+
&& cd openzeppelin-solidity \
484+
&& git checkout b8c8308 \
485+
&& { npx kevm-ganache-cli & } \
486+
&& npm install \
487+
&& npx truffle test test/token/ERC20/ERC20.test.js
488+
470489
# Media
471490
# -----
472491

‎deps/k

Submodule k updated from 0fdaeb2 to ec2e4e7

‎package/Dockerfile.arch

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,9 @@
11
FROM archlinux/base
22

33
RUN pacman -Syu --noconfirm \
4-
&& pacman -S --noconfirm git hub sudo
4+
&& pacman -S --noconfirm base base-devel curl gcc git hub make nodejs npm python2 sudo
55

6-
ADD https://github.com/kframework/k/releases/download/nightly-89361d7c8/kframework-5.0.0-1-x86_64.pkg.tar.xz kframework_5.0.0-1_x86_64.pkg.tar.xz
7-
RUN pacman --noconfirm -U ./kframework_5.0.0-1_x86_64.pkg.tar.xz
6+
RUN npm install -g npx
87

98
ARG USER_ID=1000
109
ARG GROUP_ID=1000

‎package/Dockerfile.ubuntu-bionic

Lines changed: 15 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -4,11 +4,18 @@ ENV TZ=America/Chicago
44
RUN ln --symbolic --no-dereference --force /usr/share/zoneinfo/$TZ /etc/localtime \
55
&& echo $TZ > /etc/timezone
66

7-
RUN apt update \
8-
&& apt upgrade --yes \
9-
&& apt install --yes clang-8 cmake curl dh-make jq libcrypto++-dev libjemalloc-dev \
10-
libprocps-dev libprotobuf-dev libsecp256k1-dev libssl-dev \
11-
pandoc pkg-config protobuf-compiler
12-
13-
ADD https://github.com/kframework/k/releases/download/nightly-89361d7c8/kframework_5.0.0_amd64_bionic.deb kframework_5.0.0_amd64_bionic.deb
14-
RUN apt install --yes ./kframework_5.0.0_amd64_bionic.deb
7+
RUN apt-get update \
8+
&& apt-get upgrade --yes \
9+
&& apt-get install --yes clang-8 cmake curl dh-make git libcrypto++-dev \
10+
libjemalloc-dev libprocps-dev libprotobuf-dev libsecp256k1-dev \
11+
libssl-dev pandoc pkg-config protobuf-compiler sudo
12+
13+
RUN curl -sL https://deb.nodesource.com/setup_10.x | bash -
14+
RUN apt-get install --yes nodejs
15+
RUN npm install -g npx
16+
17+
ARG USER_ID=1000
18+
ARG GROUP_ID=1000
19+
RUN groupadd -g $GROUP_ID user \
20+
&& useradd --create-home --uid $USER_ID --shell /bin/sh --gid $GROUP_ID user \
21+
&& echo "user ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers

0 commit comments

Comments
 (0)