-
Notifications
You must be signed in to change notification settings - Fork 5
amidict create
petermr edited this page Jul 11, 2020
·
14 revisions
To create an AMI dictionary.
An AMI dictionary is fundamentally a list of terms (words and phrases), with variants (synonyms, abbreviations) and annotations (links
to other resources, especially Wikidata). There are many ways for dictionary authors to select the terms, variants and annotate them ranging from typing on the commandline to searching Wikidata.
see amidict for relation to amidict and general commands.
Also developed in parallel with Java Tests in org.contentmine.ami.dictionary , see
https://github.com/petermr/ami3/blob/master/src/test/java/org/contentmine/ami/tools/AMIDictionaryTest.java
amidict create --help
amidict <input and output> create <options>
The most important are:
This is the name of the dictionary and occurs in its title attribute and filename.
e.g.
--dictionary drug
creates or reads a dictionary file drug.xml which starts:
<?xml version="1.0" encoding="UTF-8"?>
<dictionary title="drug" ...
If the filename is changed then an error will be thrown. Note that names must start with a letter and not contain spaces, or non-alphanumeric characters [a-z0-9_] are allowed.
String cmd = " "
+ " -vvvv" // debug output
+ " --dictionary myterpenes" // name of dictionary
+ " --directory=target/dictionary/create" // where to put dictionary
+ " --inputname miniterpenes" // ??? is this needed?
+ " create" // subcommand
+ " --wikilinks wikidata wikipedia" // create linked to both Wikipedia and Wikidata (IDs)
+ " --terms thymol menthol borneol junkolol " // list of terms to include in dictionary
+ " --informat list" // format of input
+ " --outformats xml" // format of dictionary
;
AbstractAMIDictTool dictionaryTool = AMIDict.execute(DictionaryCreationTool.class, cmd);
}
amidict --dictionary myterpenes --directory=target/dictionary/create --inputname miniterpenes \
create \
--informat list --terms thymol menthol borneol junkolol --wikilinks wikidata wikipedia --outformats xml
org.contentmine.ami.tools.AMIDictionaryTest.testCreateWikipedia()
public void testCreateWikipedia() {
String cmd = " "
+ " -vvvv"
+ " --dictionary myterpenes"
+ " --directory=target/dictionary/create"
+ " --inputname miniterpenes"
+ " create"
+ " --wikilinks wikidata wikipedia"
+ " --terms thymol "
+ " menthol borneol"
+ " junkolol "
+ " --informat list"
+ " --outformats xml"
;
AbstractAMIDictTool dictionaryTool = AMIDict.execute(DictionaryCreationTool.class, cmd);
}
2020-07-10
amidict create --help
Usage: amidict create [-hV] [--query[=query]] [--directory=<directory>]
[-i=FILE] [--informat=input format] [--linkcol=<linkCol>]
[-n=PATH] [--termcol=<termCol>] [--termfile=<termfile>]
[--testString=<testString>] [--wptype=<wptype>]
[--wikilinks[=<wikiLinks>[,<wikiLinks>...]...]]...
[--namecol=<nameCol>...] [-d=<dictionaryList>...]...
[--datacols=datacol[,datacol...]...]...
[--hrefcols=hrefcol[,hrefcol...]...]... [-L=PATH...]...
[--outformats=output format[,output format...]...]...
[--template=<templateNames>...]... [--terms=<terms>[,
<terms>...]...]...
creates dictionaries from text, Wikimedia, etc..
TBD
-d, --dictionary=<dictionaryList>...
input or output dictionary name/s. for 'create'
must be singular; when 'display' or 'translate',
any number. Names should be lowercase, unique.
[a-z][a-z0-9._]. Dots can be used to structure
dictionaries intodirectories. Dictionary names
are relative to 'directory'. If <directory> is
absent then dictionary names are absolute.
--datacols=datacol[,datacol...]...
use these columns (by name) as additional data
fields in dictionary. datacols='foo,bar' creates
foo='fooval1' bar='barval1' if present. No
controlled use or vocabulary and no hyperlinks.
--directory=<directory>
top directory containing dictionary/s.
Subdirectories will use structured names (NYI).
Thus dictionary 'animals' is found in
'<directory>/animals.xml', while 'plants.parts'
is found in <directory>/plants/parts.xml.
Required for relative dictionary names.
-h, --help Show this help message and exit.
--hrefcols=hrefcol[,hrefcol...]...
external hyperlink column from table; might be
Wikidata or remote site(s)
-i, --input=FILE Input filename (no defaults)
--informat=input format
input format (csv, list, mediawikitemplate,
wikisparqlxml, wikicategory, wikipage, wikitable,
wikitemplate)
-L, --inputnamelist=PATH...
List of inputnames; will iterate over them,
essentially compressing multiple commands into
one. Experimental.
--linkcol=<linkCol> column to extract link to internal pages. main use
Wikipedia. Defaults to the 'name' column
-n, --inputname=PATH User's basename for inputfiles (e.g.
foo/bar/<basename>.png) or directories. By
default this is often computed by AMI. However
some files will have variable names (e.g. output
of AMIImage) or from foreign sources or
applications
--namecol=<nameCol>...
column(s) to extract name; use exact case (e.g.
Common name)
--outformats=output format[,output format...]...
output format (xml, html, json)
--query[=query] generate query for cut and paste into EPMC or
similar. value sets size of chunks (too large
crashes EPMC). If missing, no query generated.Not
very useful.
--template=<templateNames>...
names of Wikipedia Templates, e.g.
Viral_systemic_diseases (note underscores not
spaces). Dictionaries will be created with
lowercasenames and all punctuation removed).
--termcol=<termCol> column(s) to extract term; use exact case (e.g.
Term). Could be same as namecol
--termfile=<termfile> list of terms in file, line-separated. <basename>
will become dictionary name, i.e. terpenes.txt
creates basename=terpenes
--terms=<terms>[,<terms>...]...
list of terms (entries), space-separated. Requires
`inputname`
--testString=<testString>
String input for debugging; semantics depend on task
-V, --version Print version information and exit.
--wikilinks[=<wikiLinks>[,<wikiLinks>...]...]
try to add link to Wikidata and/or Wikipedia page
of same name.
--wptype=<wptype> type of input (HTML , mediawiki)