Skip to content

Commit d18b4da

Browse files
committed
Add infrastructure for pre-built binary packages
1 parent 2848511 commit d18b4da

File tree

11 files changed

+2023
-311
lines changed

11 files changed

+2023
-311
lines changed

.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,9 @@ node_modules
1414
/examples/lobselectout.jpg
1515
/examples/lobbindsout1.txt
1616
/examples/lobbindsout2.txt
17+
/package/SHASUMS256.txt
18+
/package/*.gz
19+
package-lock.json
1720

1821
# Oracle Ignores
1922
sqlnet.log

INSTALL.md

Lines changed: 839 additions & 311 deletions
Large diffs are not rendered by default.

package/Makefile

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
# Copyright (c) 2017, Oracle and/or its affiliates. All rights reserved.
2+
#
3+
# You may not use the identified files except in compliance with the Apache
4+
# License, Version 2.0 (the "License.")
5+
#
6+
# You may obtain a copy of the License at
7+
# http://www.apache.org/licenses/LICENSE-2.0.
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
11+
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
#
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
16+
MAJ=$(shell grep '\#define \+NJS_NODE_ORACLEDB_MAJOR' ../src/njsOracle.h | sed -e 's/[^0-9]*//')
17+
MIN=$(shell grep '\#define \+NJS_NODE_ORACLEDB_MINOR' ../src/njsOracle.h | sed -e 's/[^0-9]*//')
18+
PAT=$(shell grep '\#define \+NJS_NODE_ORACLEDB_PATCH' ../src/njsOracle.h | sed -e 's/[^0-9]*//')
19+
VER=$(MAJ).$(MIN).$(PAT)
20+
21+
# The staging-oracledb-X.Y.Z.tgz package will try to download binaries from
22+
# https://$NODE_ORACLEDB_PACKAGE_HOSTNAME/$NODE_ORACLEDB_PACKAGE_URL_PATH/
23+
ifndef NODE_ORACLEDB_PACKAGE_HOSTNAME
24+
NODE_ORACLEDB_PACKAGE_HOSTNAME='your-staging-server.example.com'
25+
endif
26+
27+
ifndef NODE_ORACLEDB_PACKAGE_URL_PATH
28+
NODE_ORACLEDB_PACKAGE_URL_PATH='/your-url-path-to-directory/'
29+
endif
30+
31+
# Create a package containing the binary and license files.
32+
binarypackage:
33+
(cd .. && rm -f package-lock.json && npm install)
34+
node createpackage.js
35+
@echo "==> Created package for Node.js `node --version`"
36+
37+
npmpackage: oracledb-$(VER).tgz
38+
39+
# Create the npm package with a package.json that invokes oracledbinstall.js at install time
40+
oracledb-$(VER).tgz:
41+
rm -rf ./bundle/
42+
mkdir -m 755 bundle
43+
cp ./package.json ../index.js ../README.md ../LICENSE.md ../CHANGELOG.md bundle/
44+
mkdir -m 755 bundle/lib
45+
cp ../lib/*.js bundle/lib/
46+
mkdir -m 755 bundle/package
47+
cp ./oracledbinstall.js ./extractpackage.js ./util.js bundle/package/
48+
(cd bundle && tar -czvf ../oracledb-$(VER).tgz ./*)
49+
perl -p -i -e "s#'github.com'#'$(NODE_ORACLEDB_PACKAGE_HOSTNAME)'#" bundle/package/oracledbinstall.js
50+
perl -p -i -e "s#'/oracle/node-oracledb/releases/download/'#'$(NODE_ORACLEDB_PACKAGE_URL_PATH)'#" bundle/package/oracledbinstall.js
51+
(cd bundle && tar -czvf ../staging-oracledb-$(VER).tgz ./*)
52+
rm -rf ./bundle/
53+
54+
clean:
55+
rm -rf oracledb-$(VER).tgz staging-oracledb-$(VER).tgz SHASUMS256.txt oracledb-v*-node-*.gz ../package-lock.json ./bundle/

package/README.md

Lines changed: 99 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,99 @@
1+
# Overview
2+
3+
This directory contains scripts for building, extracting and
4+
installing binary packages of node-oracledb. Most users do not need
5+
to use anything in this directory (the exception is when doing a
6+
[manual
7+
install](https://github.com/oracle/node-oracledb/blob/master/INSTALL.md#offline)
8+
instead of using `npm`).
9+
10+
The binary install process requires two kinds of package:
11+
12+
- a gzipped tar file like `oracledb-X.Y.Z.tgz` containing JavaScript
13+
and ancillary files suitable for npm to install.
14+
15+
- a gzipped package like `oracledb-vZ.Y.Z-node-v57-darwin-x64.gz`
16+
containing the binary add-on. The package uses a custom format with
17+
three components: length bytes (giving the length of the license
18+
file), the license file, and then the node-oracledb binary. Each
19+
Node.js version/architecture needs a unique binary package.
20+
21+
A custom package format is used due to business and technical requirements:
22+
23+
- the license text needs to be included with any binary download.
24+
25+
- Node.js doesn't have a native archive module, and the installer
26+
should be lightweight and not have 3rd party dependencies.
27+
28+
When `npm install oracledb` is executed, the JavaScript package is
29+
first installed by npm. An 'install' script in its `package.json`
30+
invokes `oracledbinstall.js`. This downloads the appropriate
31+
node-oracledb binary package, and then extracts and installs the
32+
binary.
33+
34+
If a suitable binary package is not available, users must compile
35+
source code by installing from GitHub.
36+
37+
Installation is described in [INSTALL](../INSTALL.md).
38+
39+
# Maintainers
40+
41+
- The Makefile is used by node-oracledb maintainers to create the
42+
packages to be uploaded to
43+
[GitHub](https://github.com/oracle/node-oracledb) and
44+
[npm](https://www.npmjs.com/package/oracledb).
45+
46+
- `make npmpackage` makes the main node-oracledb package
47+
containing the JavaScript files.
48+
49+
- `make binarypackage` makes a binary package for the current
50+
Node.js/node-oracledb/platform and generates a SHA256 for the
51+
binary.
52+
53+
- The `package.json` in this directory invokes an install script that
54+
downloads a binary package from GitHub. This variant of
55+
`package.json` is the copy bundled for the npm release..
56+
57+
The parent file `../package.json` doesn't have the install target
58+
meaning that node-gyp will be invoked to compile node-oracledb. This
59+
allows installation from source code (via GitHub) when no suitable
60+
pre-built binary is available.
61+
62+
- The `make npmpackage` command creates two variants of the JavaScript bundle:
63+
64+
- `oracledb-X.Y.Z.tgz` which downloads binaries from the
65+
node-oracledb GitHub release page.
66+
67+
- `staging-oracledb-X.Y.Z.tgz` which downloads binaries from a
68+
server of your choice, specified by the environment variables
69+
`NODE_ORACLEDB_PACKAGE_HOSTNAME` (e.g. "your.example.com") and
70+
`NODE_ORACLEDB_PACKAGE_URL_PATH` (e.g. "/yourpath/") which must be set
71+
before running `make`.
72+
73+
You can use `staging-oracledb-X.Y.Z.tgz` to host binaries on your
74+
own network. Copy `staging-oracledb-X.Y.Z.tgz`, the binary packages
75+
for each desired architectures, and a single SHASUMS256.txt file
76+
(with one line per available binary package) to an HTTPS-enabled web
77+
server. Note if the web server has a self-signed certificate, then
78+
before running `npm install
79+
https://your.example.com/yourpath/staging-oracledb-X.Y.X.tgz` you
80+
may need to set:
81+
82+
```
83+
export NODE_TLS_REJECT_UNAUTHORIZED=0
84+
npm config set strict-ssl false
85+
```
86+
87+
Remember to do `npm config delete strict-ssl` when not testing.
88+
89+
- At install time, setting the environment variable
90+
`NODE_ORACLEDB_TRACE_INSTALL` to `TRUE` will cause `npm install` to
91+
display more tracing information.
92+
93+
- The installer scripts assume GitHub tags have the format "vX.Y.Z".
94+
Other assumptions about GitHub paths are also made in the scripts.
95+
96+
- TODO
97+
98+
- oracledbinstall.js should cache SHASUMS256.txt so it doesn't have to be fetched twice.
99+
- Improve oracledbinstall.js `no_proxy` support for domain names and wildcards.

package/createpackage.js

Lines changed: 176 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,176 @@
1+
/* Copyright (c) 2017, Oracle and/or its affiliates. All rights reserved. */
2+
3+
/******************************************************************************
4+
*
5+
* You may not use the identified files except in compliance with the Apache
6+
* License, Version 2.0 (the "License.")
7+
*
8+
* You may obtain a copy of the License at
9+
* http://www.apache.org/licenses/LICENSE-2.0.
10+
*
11+
* Unless required by applicable law or agreed to in writing, software
12+
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
13+
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
*
15+
* See the License for the specific language governing permissions and
16+
* limitations under the License.
17+
*
18+
*****************************************************************************/
19+
20+
'use strict';
21+
22+
const fs = require('fs');
23+
const Readable = require('stream').Readable;
24+
const zlib = require('zlib');
25+
const path = require('path');
26+
const util = require('util');
27+
const packageUtil = require('./util.js');
28+
29+
packageUtil.initDynamicProps();
30+
31+
// writeFileToReadable is used to buffer the contents of files to a readable
32+
// stream that can be written out later.
33+
function writeFileToReadable(path, readable) {
34+
return new Promise((resolve, reject) => {
35+
packageUtil.trace('In writeFileToReadable', path, util.inspect(readable, {depth: 0}));
36+
37+
const rs = fs.createReadStream(path);
38+
39+
rs.on('data', chunk => {
40+
readable.push(chunk);
41+
});
42+
43+
rs.on('error', err => {
44+
reject(err);
45+
});
46+
47+
rs.on('close', () => {
48+
resolve();
49+
});
50+
});
51+
}
52+
53+
// createPackage is used to create a custom file that combines the node-oracledb
54+
// binary with the license. This function is meant to be used at the command line.
55+
function createPackage() {
56+
packageUtil.trace('In createPackage');
57+
58+
let binaryPath = packageUtil.BINARY_PATH_LOCAL;
59+
60+
for (let x = 2; x < process.argv.length; x += 1) {
61+
let argParts = process.argv[x].split('=');
62+
let argName = argParts[0];
63+
let argVal = argParts[1];
64+
65+
if (argName === 'path') {
66+
binaryPath = argVal;
67+
}
68+
}
69+
70+
if (!binaryPath.endsWith(packageUtil.BINARY_FILE_NAME)) {
71+
throw new Error('path should resolve to a file named ' + packageUtil.BINARY_FILE_NAME);
72+
}
73+
74+
let binaryStats = fs.statSync(binaryPath);
75+
76+
if (!binaryStats.isFile()) {
77+
throw new Error('path did not resolve to a file');
78+
}
79+
80+
// Requiring in the binary ensures that it matches the version of Node.js currently
81+
// running. This is important as the process variable is used to file naming.
82+
require(path.normalize(binaryPath + '/../../../'));
83+
84+
let licensePath;
85+
86+
if (binaryPath != packageUtil.BINARY_PATH_LOCAL) {
87+
licensePath = path.normalize(binaryPath + '/../../../' + packageUtil.LICENSE_FILE_NAME);
88+
} else {
89+
licensePath = packageUtil.LICENSE_PATH_LOCAL;
90+
}
91+
92+
class TempReadable extends Readable {
93+
constructor(options) {
94+
super(options);
95+
}
96+
97+
_read(size) {} // Must be implemented but not used
98+
}
99+
100+
const tempReadable = new TempReadable({
101+
highWaterMark: 1048576 // 1 MB
102+
});
103+
104+
packageUtil.getSha(binaryPath)
105+
.then(binarySha => {
106+
const newShaLine = binarySha + ' ' + packageUtil.dynamicProps.BINARY_BUILD_NAME;
107+
let shaFileContents;
108+
109+
try {
110+
shaFileContents = fs.readFileSync(packageUtil.SHA_FILE_NAME, {encoding: 'utf8'});
111+
shaFileContents = shaFileContents.split('\n');
112+
113+
let updatedLine = false;
114+
115+
for (let x = 0; x < shaFileContents.length; x += 1) {
116+
const line = shaFileContents[x];
117+
118+
if (line.split(' ')[1] === packageUtil.dynamicProps.BINARY_BUILD_NAME) {
119+
shaFileContents[x] = newShaLine;
120+
updatedLine = true;
121+
break;
122+
}
123+
}
124+
125+
if (!updatedLine) {
126+
shaFileContents.splice(shaFileContents.length - 1, 0, newShaLine);
127+
}
128+
129+
shaFileContents = shaFileContents.join('\n');
130+
} catch (err) {
131+
shaFileContents = newShaLine + '\n';
132+
}
133+
134+
fs.writeFileSync(packageUtil.SHA_FILE_NAME, shaFileContents);
135+
136+
let stats = fs.statSync(licensePath);
137+
let licenseSize = stats.size.toString();
138+
139+
let zerosToAppend = packageUtil.LICENSE_HEADER_BYTES - licenseSize.length;
140+
let paddedZeros = '';
141+
142+
for (let x = 0; x < zerosToAppend; x += 1) {
143+
paddedZeros += '0';
144+
}
145+
146+
licenseSize = paddedZeros + licenseSize;
147+
148+
// The following line generates an error on Node.js 4.0, but not 4.8.5. Not sure
149+
// when the correct API was added.
150+
const licenseSizeBuf = Buffer.from(licenseSize, 'ascii');
151+
152+
tempReadable.push(licenseSizeBuf);
153+
})
154+
.then(() => {
155+
return writeFileToReadable(licensePath, tempReadable);
156+
})
157+
.then(() => {
158+
return writeFileToReadable(binaryPath, tempReadable);
159+
})
160+
.then(() => {
161+
const ws = fs.createWriteStream(packageUtil.dynamicProps.PACKAGE_FILE_NAME);
162+
const gzip = zlib.createGzip();
163+
const filestream = tempReadable.pipe(gzip).pipe(ws);
164+
165+
tempReadable.push(null); // Signal the end of data in
166+
167+
filestream.on('close', () => {
168+
console.log('Package created: ' + packageUtil.dynamicProps.PACKAGE_FILE_NAME);
169+
});
170+
})
171+
.catch(err => {
172+
console.log('Error creating package', err);
173+
})
174+
}
175+
176+
createPackage();

0 commit comments

Comments
 (0)