Skip to content

Commit 6c05281

Browse files
rsdoielrsdoiel
committed
maintenance: fix test errors for jsonjoin, updated to go1.22, update dependencies
1 parent 37d64be commit 6c05281

File tree

243 files changed

+2523
-2159
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

243 files changed

+2523
-2159
lines changed

‎CITATION.cff‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,6 @@ maintainers:
1515
orcid: "https://orcid.org/0000-0003-0900-6903"
1616

1717
repository-code: "https://github.com/caltechlibrary/datatools"
18-
version: 1.2.5
18+
version: 1.2.6
1919
license-url: "https://data.caltech.edu/license"
20-
keywords: [ "csv", "excel", "sql", "json", "xlsx", "golang", "bash" ]
20+
keywords: [ "csv", "excel", "sql", "json", "yaml", "xlsx", "golang", "bash" ]

‎INSTALL.html‎

Lines changed: 223 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,223 @@
1+
<!DOCTYPE html>
2+
<html>
3+
<head>
4+
<title>Caltech Library's Digital Library Development Sandbox</title>
5+
<link href='https://fonts.googleapis.com/css?family=Open+Sans' rel='stylesheet' type='text/css'>
6+
<link rel="stylesheet" href="/css/site.css">
7+
</head>
8+
<body>
9+
<header>
10+
<a href="http://library.caltech.edu"><img src="/assets/liblogo.gif" alt="Caltech Library logo"></a>
11+
</header>
12+
<nav>
13+
<ul>
14+
<li><a href="/">Home</a></li>
15+
<li><a href="./">README</a></li>
16+
<li><a href="LICENSE">LICENSE</a></li>
17+
<li><a href="INSTALL.html">INSTALL</a></li>
18+
<li><a href="user-manual.html">User Manual</a></li>
19+
<li><a href="how-to/">Tutorials</a></li>
20+
<li><a href="search.html">Search Docs</a></li>
21+
<li><a href="about.html">About</a></li>
22+
<li><a href="https://github.com/caltechlibrary/datatools">GitHub</a></li>
23+
</ul>
24+
</nav>
25+
26+
<section>
27+
<h1 id="installation">Installation</h1>
28+
<p><em>datatools</em> is a collection of command line programs run from
29+
a shell like Bash.</p>
30+
<h2 id="quick-install-using-curl">Quick install using curl</h2>
31+
<p>The following experimental installer should work for macOS and Linux
32+
(e.g. Debian, Ubuntu, Raspberry Pi OS)</p>
33+
<p>Copy and run the following command in your shell (e.g. Terminal)</p>
34+
<pre><code>curl https://caltechlibrary.github.io/datatools/installer.sh | sh</code></pre>
35+
<h2 id="compiled-version">Compiled version</h2>
36+
<p>This is generalized instructions for a release.</p>
37+
<p>Compiled versions are available for Mac OS X (Intel and M1 processor,
38+
macos-x86_64 and macOS-arm64), Linux (Intel process, Linux-x86_64),
39+
Windows (Intel and arm64 processor, windows-x86_64 and Windows-arm64)
40+
and Rapsberry Pi (arm7 processor, RaspberryPiOS-arm7)</p>
41+
<p>VERSION_NUMBER is a <a href="http://semver.org/">symantic version
42+
number</a> (e.g. v0.1.2)</p>
43+
<p>For all the released version go to the project page on Github and
44+
click latest release</p>
45+
<blockquote>
46+
<p>https://github.com/caltechlibrary/datatools/releases/latest</p>
47+
</blockquote>
48+
<table>
49+
<thead>
50+
<tr class="header">
51+
<th>Platform</th>
52+
<th>Zip Filename</th>
53+
</tr>
54+
</thead>
55+
<tbody>
56+
<tr class="odd">
57+
<td>Windows</td>
58+
<td>datatools-VERSION_NUMBER-Windows-x86_64.zip</td>
59+
</tr>
60+
<tr class="even">
61+
<td>Windows</td>
62+
<td>datatools-VERSION_NUMBER-Windows-arm64.zip</td>
63+
</tr>
64+
<tr class="odd">
65+
<td>Mac OS X</td>
66+
<td>datatools-VERSION_NUMBER-macOS-x86_64.zip</td>
67+
</tr>
68+
<tr class="even">
69+
<td>Mac OS X</td>
70+
<td>datatools-VERSION_NUMBER-macOS-arm64.zip</td>
71+
</tr>
72+
<tr class="odd">
73+
<td>Linux/Intel</td>
74+
<td>datatools-VERSION_NUMBER-Linux-x86_64.zip</td>
75+
</tr>
76+
<tr class="even">
77+
<td>Linux/ARM 64</td>
78+
<td>datatools-VERSION_NUMBER-Linux-aarch64.zip</td>
79+
</tr>
80+
<tr class="odd">
81+
<td>Raspbery Pi OS</td>
82+
<td>datatools-VERSION_NUMBER-RaspberryPiOS-arm7.zip</td>
83+
</tr>
84+
</tbody>
85+
</table>
86+
<h2 id="the-basic-recipe">The basic recipe</h2>
87+
<ul>
88+
<li>Find the Zip file listed matching the architecture you’re running
89+
and download it
90+
<ul>
91+
<li>(e.g. if you’re on a Windows 10 laptop/Surface with a Intel style
92+
CPU you’d choose the Zip file with “windows-x86_64” in the name).</li>
93+
</ul></li>
94+
<li>Download the zip file and unzip the file.<br />
95+
</li>
96+
<li>Copy the contents of the folder named “bin” to a folder that is in
97+
your path
98+
<ul>
99+
<li>(e.g. “$HOME/bin” is common).</li>
100+
</ul></li>
101+
<li>Adjust your PATH if needed
102+
<ul>
103+
<li>(e.g. export PATH=“<span
104+
class="math inline"><em>H</em><em>O</em><em>M</em><em>E</em>/<em>b</em><em>i</em><em>n</em>:</span>PATH”)</li>
105+
</ul></li>
106+
<li>Test</li>
107+
</ul>
108+
<h3 id="mac-os">Mac OS</h3>
109+
<ol type="1">
110+
<li>Download the zip file</li>
111+
<li>Unzip the zip file</li>
112+
<li>Copy the executables to $HOME/bin (or a folder in your path)</li>
113+
<li>Make sure the new location in in our path</li>
114+
<li>Test</li>
115+
</ol>
116+
<p>Here’s an example of the commands run in the Terminal App after
117+
downloading the zip file.</p>
118+
<h4 id="intel-x86_64-hardware">Intel (x86_64) Hardware</h4>
119+
<pre class="shell"><code> cd Downloads/
120+
unzip datatools-*-macos-x86_64.zip
121+
mkdir -p $HOME/bin
122+
mv -v bin/* $HOME/bin/
123+
export PATH=$HOME/bin:$PATH
124+
csvfind -version</code></pre>
125+
<h4 id="arm64-arm64-hardware">ARM64 (arm64) Hardware</h4>
126+
<pre class="shell"><code> cd Downloads/
127+
unzip datatools-*-macos-arm64.zip
128+
mkdir -p $HOME/bin
129+
mv -v bin/* $HOME/bin/
130+
export PATH=$HOME/bin:$PATH
131+
csvfind -version</code></pre>
132+
<h3 id="windows">Windows</h3>
133+
<p>(Assumes you’re working from Bash as provided by Linux Subsystem for
134+
Windows)</p>
135+
<ol type="1">
136+
<li>Download the zip file</li>
137+
<li>Unzip the zip file</li>
138+
<li>Copy the executables to $HOME/bin (or a folder in your path)</li>
139+
<li>Test</li>
140+
</ol>
141+
<p>Here’s an example of the commands run in from the Bash shell on
142+
Windows 10 after downloading the zip file.</p>
143+
<h4 id="intel-x86_64-hardware-1">Intel (x86_64) Hardware</h4>
144+
<pre class="shell"><code> cd Downloads/
145+
unzip datatools-*-windows-x86_64.zip
146+
mkdir -p $HOME/bin
147+
mv -v bin/* $HOME/bin/
148+
export PATH=$HOME/bin:$PATH
149+
csvfind -version</code></pre>
150+
<h4 id="arm64-arm64-hardware-1">ARM64 (arm64) Hardware</h4>
151+
<pre class="shell"><code> cd Downloads/
152+
unzip datatools-*-windows-arm64.zip
153+
mkdir -p $HOME/bin
154+
mv -v bin/* $HOME/bin/
155+
export PATH=$HOME/bin:$PATH
156+
csvfind -version</code></pre>
157+
<h3 id="linux">Linux</h3>
158+
<ol type="1">
159+
<li>Download the zip file</li>
160+
<li>Unzip the zip file</li>
161+
<li>Copy the executables to $HOME/bin (or a folder in your path)</li>
162+
<li>Test</li>
163+
</ol>
164+
<p>Here’s an example of the commands run in from the Bash shell after
165+
downloading the zip file.</p>
166+
<pre class="shell"><code> cd Downloads/
167+
unzip datatools-*-linux-x86_64.zip
168+
mkdir -p $HOME/bin
169+
cp -v bin/* $HOME/bin/
170+
export PATH=$HOME/bin:$PATH
171+
csvfind -version</code></pre>
172+
<h3 id="raspberry-pi">Raspberry Pi</h3>
173+
<p>Released version is for a Raspberry Pi 2 or later use (i.e. requires
174+
ARM 7 support).</p>
175+
<ol type="1">
176+
<li>Download the zip file</li>
177+
<li>Unzip the zip file</li>
178+
<li>Copy the executables to $HOME/bin (or a folder in your path)</li>
179+
<li>Test</li>
180+
</ol>
181+
<p>Here’s an example of the commands run in from the Bash shell after
182+
downloading the zip file.</p>
183+
<pre class="shell"><code> cd Downloads/
184+
unzip datatools-*-raspberry_pi_os-arm7.zip
185+
mkdir -p $HOME/bin
186+
cp -v bin/* $HOME/bin/
187+
export PATH=$HOME/bin:$PATH
188+
csvfind -version</code></pre>
189+
<h2 id="compiling-from-source">Compiling from source</h2>
190+
<p><em>datatools</em> is “go gettable” if you have previously gotten
191+
xlsx v1.0.5 package from <a
192+
href="https://github.com/tealeg/xlsx">github.com/tealeg/xlsx</a>. The
193+
datatools package does not support versions v2.x and greater of xlsx.
194+
Below are the steps I use today with “go get” command to download the
195+
dependant packages as well as <em>datatools</em>’s source code.</p>
196+
<p>Setting up the right version of xlsx for datatools</p>
197+
<pre class="shell"><code> cd
198+
go get github.com/tealeg/xlsx
199+
cd src/github.com/tealeg
200+
git checkout v1.0.5
201+
cd</code></pre>
202+
<p>Using <code>go get</code> to install datatools using v1.0.5 of
203+
xlsx.</p>
204+
<pre><code> go get github.com/caltechlibrary/datatools/...</code></pre>
205+
<p>Or clone the repository and then compile</p>
206+
<pre class="shell"><code> cd
207+
git clone https://github.com/caltechlibrary/datatools src/github.com/caltechlibrary/datatools
208+
cd src/github.com/caltechlibrary/datatools
209+
make
210+
make test
211+
make install</code></pre>
212+
</section>
213+
214+
<footer>
215+
<span><h1><A href="http://caltech.edu">Caltech</a></h1></span>
216+
<span>&copy; 2023 <a href="https://www.library.caltech.edu/copyright">Caltech library</a></span>
217+
<address>1200 E California Blvd, Mail Code 1-32, Pasadena, CA 91125-3200</address>
218+
<span>Phone: <a href="tel:+1-626-395-3405">(626)395-3405</a></span>
219+
<span><a href="mailto:[email protected]">Email Us</a></span>
220+
<a class="cl-hide" href="sitemap.xml">Site Map</a>
221+
</footer>
222+
</body>
223+
</html>

‎Makefile‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -67,7 +67,8 @@ installer.sh: .FORCE
6767
$(PROGRAMS): $(PACKAGE)
6868
@mkdir -p bin
6969
go build -o bin/$@$(EXT) cmd/$@/*.go
70-
70+
@./bin/$@ -help >$@.1.md
71+
7172
test: $(PACKAGE)
7273
go test
7374
bash test_cmd.bash
@@ -82,8 +83,8 @@ $(MAN_PAGES): .FORCE
8283
man: $(MAN_PAGES)
8384

8485
website: .FORCE
85-
make -f website.mak
8686
cd how-to && make -f website.mak
87+
make -f website.mak
8788

8889
status:
8990
git status

‎about.html‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@
2525

2626
<section>
2727
<h1 id="about-this-software">About this software</h1>
28-
<h2 id="datatools-1.2.5">datatools 1.2.5</h2>
28+
<h2 id="datatools-1.2.6">datatools 1.2.6</h2>
2929
<h3 id="authors">Authors</h3>
3030
<ul>
3131
<li>R. S. Doiel</li>
@@ -46,7 +46,7 @@ <h3 id="maintainers">Maintainers</h3>
4646
</ul>
4747
<h3 id="programming-languages">Programming languages</h3>
4848
<ul>
49-
<li>Go 1.20</li>
49+
<li>Go 1.22</li>
5050
</ul>
5151
<h3 id="operating-systems">Operating Systems</h3>
5252
<ul>
@@ -56,8 +56,8 @@ <h3 id="operating-systems">Operating Systems</h3>
5656
</ul>
5757
<h3 id="software-requiremets">Software Requiremets</h3>
5858
<ul>
59-
<li>Golang 1.21 or better</li>
60-
<li>Pandoc 2.19 or better</li>
59+
<li>Golang 1.22 or better</li>
60+
<li>Pandoc 3.1 or better</li>
6161
</ul>
6262
</section>
6363

‎about.md‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -11,16 +11,16 @@ authors:
1111
orcid: "https://orcid.org/0000-0003-0900-6903"
1212

1313
repository-code: "https://github.com/caltechlibrary/datatools"
14-
version: 1.2.5
14+
version: 1.2.6
1515
license-url: "https://data.caltech.edu/license"
16-
keywords: [ "csv", "excel", "sql", "json", "xlsx", "golang", "bash" ]
16+
keywords: [ "csv", "excel", "sql", "json", "yaml", "xlsx", "golang", "bash" ]
1717

1818
---
1919

2020
About this software
2121
===================
2222

23-
## datatools 1.2.5
23+
## datatools 1.2.6
2424

2525
### Authors
2626

@@ -41,7 +41,7 @@ and structured text documents.
4141

4242
### Programming languages
4343

44-
- Go 1.20
44+
- Go 1.22
4545

4646
### Operating Systems
4747

@@ -51,5 +51,5 @@ and structured text documents.
5151

5252
### Software Requiremets
5353

54-
- Golang 1.21 or better
55-
- Pandoc 2.19 or better
54+
- Golang 1.22 or better
55+
- Pandoc 3.1 or better

‎cmd/codemeta2cff/codemeta2cff.go‎

Lines changed: 5 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -44,11 +44,9 @@ import (
4444
)
4545

4646
var (
47-
helpText = `---
48-
title: "{app_name}(1) user manual"
49-
author: "R. S. Doiel"
50-
pubDate: 2022-10-28
51-
---
47+
helpText = `%{app_name}(1) user manual | version {version} {release_hash}
48+
% R. S. Doiel
49+
% {release_date}
5250
5351
# NAME
5452
@@ -87,8 +85,8 @@ Specifying the full paths.
8785
`
8886
)
8987

90-
func fmtTxt(src string, appName string, version string) string {
91-
return strings.ReplaceAll(strings.ReplaceAll(helpText, "{app_name}", appName), "{version}", version)
88+
func fmtTxt(src string, appName string, version string) string {
89+
return strings.ReplaceAll(strings.ReplaceAll(helpText, "{app_name}", appName), "{version}", version)
9290
}
9391

9492
func main() {

‎cmd/csv2json/csv2json.go‎

Lines changed: 3 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -34,11 +34,9 @@ import (
3434
)
3535

3636
var (
37-
helpText = `---
38-
title: "csv2json (1) user manual"
39-
author: "R. S. Doiel"
40-
pubDate: 2023-01-06
41-
---
37+
helpText = `%{app_name}(1) user manual | version {version} {release_hash}
38+
% R. S. Doiel
39+
% {release_date}
4240
4341
# NAME
4442

‎cmd/csv2mdtable/csv2mdtable.go‎

Lines changed: 3 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -31,11 +31,9 @@ import (
3131
)
3232

3333
var (
34-
helpText = `---
35-
title: "{app_name} (1) user manual"
36-
author: "R. S. Doiel"
37-
pubDate: 2023-01-06
38-
---
34+
helpText = `%{app_name}(1) user manual | version {version} {release_hash}
35+
% R. S. Doiel
36+
% {release_date}
3937
4038
# NAME
4139

0 commit comments

Comments
 (0)