diff --git a/.travis.yml b/.travis.yml index 88b27b34..0adca557 100644 --- a/.travis.yml +++ b/.travis.yml @@ -4,9 +4,31 @@ jdk: - oraclejdk8 - oraclejdk11 +services: + - docker + +env: + - img=existdb/existdb:latest + - img=existdb/existdb:release + +before_install: + - docker pull $img + - docker create --name exist-ci -p 8080:8080 $img + +install: + - mvn clean package + +before_script: + - docker cp ./target/exist-documentation-*.xar exist-ci:exist/autodeploy + - docker start exist-ci + # exist needs time + - sleep 30 + script: - - mvn -B test package - mvn validate +after_success: + - docker ps + notifications: hipchat: ec8fcfa661addc56a361a8ef536320@integrations diff --git a/src/main/xar-resources/data/author-reference/author-reference.xml b/src/main/xar-resources/data/author-reference/author-reference.xml index 8130d1ad..f44999c7 100644 --- a/src/main/xar-resources/data/author-reference/author-reference.xml +++ b/src/main/xar-resources/data/author-reference/author-reference.xml @@ -1,199 +1,456 @@
- - - - Author Reference - 1Q18 - - authoring - exist - - - - - - This article describes the conventions for writing articles for the eXist-db documentation - set. The eXist-db documentation set is (since beginning 2018) based on a subset of the - Docbook 5 content model. If you want to contribute an article (please do) it must conform to - the standards described here. - - - - - General guidelines - - Here are some general writing (and tagging) guidelines: - - - Please start of with a short introduction of your article in one or a few - paragraphs before diving in. These paragraphs must be placed - before the first sect1. - Describe what the article is about and for whom. What is the subject? Is it - for total novices, beginners, advanced programming gurus in Java hyperspace? - Prevent readers from diving in and find out that it's not for them. - - - Presenting longer listings can best be done by storing these in separate - documents and linking them (see ). This makes - maintenance easier (ever edited a large CDATA section with XML inside?). - An additional reason is that somewhere in the future we intend to enable - running code-examples directly from the documentation app. For this these - examples must be in separate documents. So make sure that at least runnable - examples are separated from your text. - - - Of course all the normal technical writing guidelines apply: Keep your target - group in mind (what do they know and, more importantly, what not), write correct - English, use plenty of examples, split the article in sections and sub-sections - that make sense, etc. - - - - - Patterns to avoid - - The following patterns tend to occur again and again in writing documentation. - Please try to avoid them… - The examples exaggerated, but that's for illustration purposes, not to offend - anyone. - - - Avoid the difficult and explain the easy - - It is very tempting to skip quickly over some hard parts of the documentation - and instead describe some of the easy parts in length. For instance: - - - <para>To configure the <emphasis>xyz</emphasis> extension you have to create an - XML configuration file. Here is an example:</para> - <programlisting>(some skimpy example of a complicated configuration file that is supposed to speak for itself + <!-- ================================================================== --> + + <info> + <title>Author Reference + 1Q18 + + authoring + exist + + + + + + This article describes the conventions for writing articles for the eXist-db documentation + set. The eXist-db documentation set is (since beginning 2018) based on a subset of the Docbook 5 + content model. If you want to contribute an article (please do) it must conform to the standards + described here. + + + + + General guidelines + + Here are some general writing (and tagging) guidelines: + + + Please start of with a short introduction of your article in one or a few paragraphs + before diving in. These paragraphs must be placed before the first + sect1. + Describe what the article is about and for whom. What is the subject? Is it for total + novices, beginners, advanced programming gurus in Java hyperspace? Prevent readers from + diving in and find out that it's not for them. + + + Presenting longer listings can best be done by storing these in separate documents and + linking them (see ). This makes maintenance easier (ever edited + a large CDATA section with XML inside?). + An additional reason is that somewhere in the future we intend to enable running + code-examples directly from the documentation app. For this these examples must be in + separate documents. So make sure that at least runnable examples are separated from your + text. + + + Of course all the normal technical writing guidelines apply: Keep your target group in + mind (what do they know and, more importantly, what not), write correct English, use + plenty of examples, split the article in sections and sub-sections that make sense, + etc. + + + + + Patterns to avoid + + The following patterns tend to occur again and again in writing documentation. Please + try to avoid them… + The examples exaggerated, but that's for illustration purposes, not to offend + anyone. + + + Avoid the difficult and explain the easy + + It is very tempting to skip quickly over some hard parts of the documentation and + instead describe some of the easy parts in length. For instance: + + + <para>To configure the <emphasis>xyz</emphasis> extension you have to create an XML + configuration file. Here is an example:</para> + <programlisting>(some skimpy example of a complicated configuration file that is supposed to speak for itself but does not because its too simple)</programlisting> - <para>To work with the user interface open this, click here, enter the password - twice, click ok, bla bla bla, going on for multiple paragraphs…</para> - </example> - - <para>Well, your reader isn't stupid and hasn't seen a GUI for the first time. So - telling him/her that a password needs to be entered twice and that ok must be - clicked is completely superfluous. Just point your reader to the GUI and maybe - to the right menu but that's all. Believe me, they know how to fill in a dialog - box.</para> - <para>But they do not know what all these elements and attributes in the - configuration file mean. Even if the names speak for themselves, some - explanation will be necessary. Best to describe them all, mention defaults, - examples, etc.</para> - <para>And yes, that's hard work, both in thinking and writing. But if you want your - xyz extension to be used and useful (that's what you wrote it for, right?) its - part of the job.</para> - </sect3> - - <sect3 xml:id="polite"> - <title>Being overly polite - - Sometimes writers try to be very polite to both the reader and, strangely - enough, the system. Since the reader wants it clear and you can't offend a - system, don't. - - - Table title - - - - - - Polite version - Straight version - - - - - The logfile should appear. - The logfile appears. - - - You may enter ... - Enter ... - - - If you wish you can also... - It is also possible to ... Also: - ... - - - Please navigate to... - Navigate to ... - - - -
- - - - Using thus instead of therefore - Somehow there seems to be a preference among eXist-db's documentation writers - for the word "thus" when "therefore" is meant. Thus is almost always wrong in - this kind of prose, so please default to therefore. If you want to know more - read on here. - - - - Java isn't always interesting - Some articles that explain stuff about things in the XQuery realm keep - rambling on about stuff in Java or in eXist-db's architecture: Servlets that - execute, methods that are called, etc. - Yes, because you are a Java programmer its probably - interesting to you. But when the intended audience of the article is XQuery - programmers, don't bother them with this. And if you really (really!) think it - is necessary information for some of them, add it to a separate section at the - bottom. - - - - - - - - - - - Technical guidelines - - More information about Docbook itself can be found on the official Docbook site. - - - - - Collection (directory) structure - - - A single article must be stored in a collection on its own, the - article-collection. Related assets or other files will - be stored in sub-collections of this. This setup ensures that everything - belonging to a single article stays together. - - - In the eXist documentation app, the article-collection will become a - sub-collection of the data collection. So an article-collection - called new-cool-feature will become - /db/apps/doc/data/new-cool-feature. - - - The name of the article-collection should be the same as the document-name - of the article itself. So if your article is called - new-cool-feature.xml, store it as - new-cool-feature/new-cool-feature.xml. - - - Any binary assets (images, etc.) referenced by your article must be stored - in a sub-collection assets. So in our example in - new-cool-feature/assets. - - - Any non-binary assets (usually included program/text listings) referenced - by your article must be stored in a sub-collection listings. So - in our example in new-cool-feature/listings. - - + To work with the user interface open this, click here, enter the password twice, + click ok, bla bla bla, going on for multiple paragraphs… + + + Well, your reader isn't stupid and hasn't seen a GUI for the first time. So telling + him/her that a password needs to be entered twice and that ok must be clicked is + completely superfluous. Just point your reader to the GUI and maybe to the right menu but + that's all. Believe me, they know how to fill in a dialog box. + But they do not know what all these elements and attributes in the configuration file + mean. Even if the names speak for themselves, some explanation will be necessary. Best to + describe them all, mention defaults, examples, etc. + And yes, that's hard work, both in thinking and writing. But if you want your xyz + extension to be used and useful (that's what you wrote it for, right?) its part of the + job. + + + + Being overly polite + + Sometimes writers try to be very polite to both the reader and, strangely enough, the + system. Since the reader wants it clear and you can't offend a system, don't. + + + Table title + + + + + + + Polite version + + + Straight version + + + + + + + The logfile should appear. + + + The logfile appears. + + + + + You may enter ... + + + Enter ... + + + + + If you wish you can also... + + + It is also possible to ... + Also: ... + + + + + Please navigate to... + + + Navigate to ... + + + + +
+ + + + Using thus instead of therefore + Somehow there seems to be a preference among eXist-db's documentation writers for the + word "thus" when "therefore" is meant. Thus is almost always wrong in this kind of prose, + so please default to therefore. If you want to know more read on here. + + + + Java isn't always interesting + Some articles that explain stuff about things in the XQuery realm keep rambling on + about stuff in Java or in eXist-db's architecture: Servlets that execute, methods that are + called, etc. + Yes, because you are a Java programmer its probably interesting + to you. But when the intended audience of the article is XQuery programmers, don't bother + them with this. And if you really (really!) think it is necessary information for some of + them, add it to a separate section at the bottom. + + + + + + + + + + + Technical guidelines + + More information about Docbook itself can be found on the official Docbook site. + + + + + Collection (directory) structure + + + A single article must be stored in a collection on its own, the + article-collection. Related assets or other files will be stored + in sub-collections of this. This setup ensures that everything belonging to a single + article stays together. + + + In the eXist documentation app, the article-collection will become a sub-collection + of the data collection. So an article-collection called + new-cool-feature will become + /db/apps/doc/data/new-cool-feature. + + + The name of the article-collection should be the same as the document-name of the + article itself. So if your article is called new-cool-feature.xml, store it + as new-cool-feature/new-cool-feature.xml. + + + Any binary assets (images, etc.) referenced by your article must be stored in a + sub-collection assets. So in our example in + new-cool-feature/assets. + + + Any non-binary assets (usually included program/text listings) referenced by your + article must be stored in a sub-collection listings. So in our example in + new-cool-feature/listings. + + + + All rules regarding collection names are not there for technical reasons but to + promote consistency. If you deviate and make sure all references are correct it will still + work. However, to keep things maintainable it is strongly recommended to follow the rules + stated above (or, if you really think they're stupid, apply for a rule change). + + + + + + + Overall XML structure + Here is an example how an article for this documentation set looks like: + + Both the processing instructions at the top are optional but strongly recommended. + They're present to guide tools like oXygen in validating the document and provide authoring + hints. + + + + + + Article information + Here is an example the article's mandatory information section: + + The following rules and guidelines apply: + + + There must be a title present in a title element. + This is the main title of the document. It will be displayed at the top of the + page. + + + Optional but strongly recommended is a date in the date element. This + will be displayed below the title and provide the reader with some clue on how old this + information is. We all know that sometimes articles get outdated but are not removed... + You don't have to provide an exact date. Just month and year will do, or some indication + like 1Q18. + + + We use keywords to mark an article for certain subject areas. See . + + + You can add more information in info to your heart's desire but the + renderer won't use nor display it. + + + + + Keyword usage + + The info element must contain at least one keyword. This is used to create + an index on subject. Multiple keywords can be specified by repeating the + keyword element. + Keywords are always written lower-case and must not contain + spaces. + The following keywords are in use: + + Keyword usage + + + + + + + Keyword + + + Meaning + + + + + + + + + application-development + + + + Information related to using eXIst-db as an application platform. + + + + + + exist + + + + Anything related to eXist-db itself, like legal statements, how to send in + bugs, etc. + + + + + + getting-started + + + + Basic information on how to get eXist-db up and running. + + + + + + java-development + + + + Information about programming Java for or in eXist-db. + + + + + + indexing + + + + Information related to eXist-db's indexes. + + + + + + installation + + + + Information about installing eXist-db + + + + + + interfaces + + + + Information about specific interfaces eXist-db provides + + + + + + operations + + + + How to keep eXist-db up and running, not from a programmer's but from a + system maintenance point of view. + + + + + + testing + + + + Information related to testing eXist-db or its applications. + + + + + + xquery + + + + Information related to XQuery (or XPath) in general or eXist-db's specific + features of the language. + + + + +
+ + + Tables like the one above tend to get outdated sooner or later (because somebody + adds/deletes a keyword and forgets to update it). To see all the keywords actually in + use, look at the index-by-subject section of the main documentation page. + Watch out: For legibility reasons the keywords there are presented + capitalized. Remember that in the article documents the keywords + must be stated in lower-case! + + + + + + + + + + Supported block elements + + The following Docbook 5 block level elements are supported for the eXist-db + documentation set. Unsupported elements will be flagged when the content is rendered (you'll + see a bold red error message in the middle of your story). + + + + + Paragraphs + + Use para to tag text as a paragraph. + + + + + Section levels + + You can use sect1 up to sect3 as section levels. Only + sect1 and sect2 will appear in the table of contents. Markup + in titles is ignored. + + + + + Lists + + You can use either itemizedlist (with bullets) or + orderedlist (with numbers). + + + + + Variable list + + A variablelist is a bit of a misnomer. It creates lists like this one + (the list of block elements you're looking at now), so it's useful for much more than + variables. + If you want a narrow left column (like this one), add + spacing="compact". + + + + + Notes + All rules regarding collection names are not there for technical reasons but to promote consistency. If you deviate and make sure all references are correct @@ -525,191 +782,318 @@ but does not because its too simple) Inline elements in title elements are not supported. Any markup in titles is ignored. - - - - - Emphasis - - Use emphasis for emphasizing. - Although discouraged, you can request a specific emphasis type: - - - Use role="bold" for bold - emphasis. - - - Use role="underline" for underlined emphasis. - - - Use role="italic" for italic emphasis. This looks the same as no - role attribute but produces an i instead - of an em. - - - - - - - GUI Items - - Use guimenuitem around anything referring to choices in a - GUI, for instance menu choices. The effect is - now simple italic but that might change later. - - - - - Code - - Use code or literal for literals, pieces of - code, commands, etc. The result will be mono-spaced. - The general guideline is to use this for everything that lists pieces - of code, attributes, variable names, etc. For elements use - tag. - - - - - Tags - - A tag element surrounds its content with < and > and - formats it like code. Meant to make life easier: - is the shorter - version of - . - - - - - Figures - - Use the inlinemediaobject to add an asset inline in the - text. Docbook prescribes an elaborate XML construction, here is an - example of a paragraph with an inline figure: - - This produces: - There is an asset - - - - here. - The width attribute is the only size-related attribute - recognized and used. - - - - - Internal links - - You can create a link to any block level element with the - xref element. To do so, give the target an identifier by - using xml:id="..." and mention this identifier in the - linkend attribute of the xref. - You can generate a specific link text by adding an - xreflabel attribute to the target - element, like here: . If this - attribute is not present on the target its title is used, like this: - . If there is no title, the name of - the element is presented. - - - - - External links - - Links to external sources are done with the link element. - Place the target's full URI in the xlink:href attribute. - For instance, - - links to the eXist - home page. - To open the link on a new page, add - condition="_blank". - - - - - Article links - - To create a link to another article in this documentation set, also - use the link element. However, use the target's document name - (without the .xml extension) as the link address. For - instance, - links - to the documentation home - page. - - - - - - - - - - - Generating indexes - - Indexes can be automagicly generated. For examples have a look at the documentation home page. There are three - variations: - - - Alphabetical index on title - - Add a role="indexontitle" attribute to a para - element. An index on title will be generated after it. - - - - Index on keyword, all - - Add a role="indexonkeyword" attribute to a - para element. An index on keywords (as specified in the - info element, see ) will be - generated after it. - - - - Index on keyword, single - - Add a role="indexonkeyword:somekeyword" attribute to a - para element to generate a list of articles with this - specific keyword (as specified in the info element, see ). - - - - - - - - - - - Additional tools - - There are two additional pages in the documentation app to help authors and - editors: - - - Editorial - view - - The editorial view provides an overview of all the articles and their - incoming and outgoing links. - - - - Diagnostics - - Performs limited diagnostics. Checks whether the outgoing links in - articles to other articles, listings and assets are ok. Also checks whether - there are unreferenced (and therefore superfluous) listings and - assets. - - - - - - -
\ No newline at end of file + + + + + Examples + + + An example of an example + The example element produces an example in a layout like this. It + must have a title element, which is displayed at the bottom: + + + + + + Listings + + Anything you want to present as a program listing or other fixed text (e.g. a + sequence of commands) must be enclosed in a programlisting element. There + are two ways to provide the actual contents of the listing: + + + Simply enclose the text in the programlisting element. Use CDATA if + necessary. Any markup will be ignored. + + + Add an xlink:href attribute + (xmlns:xlink="http://www.w3.org/1999/xlink") with a reference to a + file containing the text. This must be a relative reference + (e.g. xlink:href="listings/mylisting.txt"). The contents of the + element will be ignored. + Using the xlink:href attribute is definitely + not the proper way to do this. It should be done by using + an xi:include element inside programlisting. However, + current XInclude limitations in eXist prevent this from working properly (it can + only include XML, not text). So therefore this workaround. + + + Listings have two display modes: + + + If it's a listing in some language (like XML), add a language + attribute with the name of the language. The render will try to add some color + coding (if it recognizes it). Here is a listing with + language="xml": + + + + For a straight listing, don't add a language attribute. Here is + the same listing as above without: + + + + + + + + Figures + + Use figure or informalfigure to add an asset. The effect of + both elements is almost the same, the only difference is that an informal figure has + no title. Docbook prescribes an elaborate XML construction, here is an example: + + This produces: +
+ Figure title here + + + + + +
+ The width attribute is the only size-related attribute used. It sets + the width relative to the available horizontal space. If you don't specify a width a + default of 75% is used. + + + + + Tables + + Use table or informaltable to add a table. The effect of + both elements is almost the same, the only difference is that an informal table has no + title. Table layout is kept simple and straight. Here is how a simple table looks + like: + + And this is the result: + + Table title + + + + + + + Col 1 + + + Col 2 + + + + + + + Bla + + + More bla + + + + + Blegh + + + More blegh + + + + +
+ + + + + + + + + + + Supported inline elements + The following inline elements are supported: + + Inline elements in title elements are not supported. + Any markup in titles is ignored. + + + + + + Emphasis + + Use emphasis for emphasizing. + Although discouraged, you can request a specific emphasis type: + + + Use role="bold" for bold + emphasis. + + + Use role="underline" for underlined + emphasis. + + + Use role="italic" for italic + emphasis. This looks the same as no role attribute but + produces an i instead of an em. + + + + + + + GUI Items + + Use guimenuitem around anything referring to choices in a GUI, for + instance menu choices. The effect is now simple italic but + that might change later. + + + + + Code + + Use code or literal for literals, pieces of code, commands, + etc. The result will be mono-spaced. + The general guideline is to use this for everything that lists pieces of code, + attributes, variable names, etc. For elements use tag. + + + + + Tags + + A tag element surrounds its content with < and > and formats it + like code. Meant to make life easier: xxx]]> is the + shorter version of <xxx>]]>. + + + + + Figures + + Use the inlinemediaobject to add an asset inline in the text. Docbook + prescribes an elaborate XML construction, here is an example of a paragraph with an + inline figure: + + This produces: + There is an asset + + + + here. + The width attribute is the only size-related attribute recognized and + used. + + + + + Internal links + + You can create a link to any block level element with the xref element. + To do so, give the target an identifier by using xml:id="..." and mention + this identifier in the linkend attribute of the xref. + You can generate a specific link text by adding an xreflabel + attribute to the target element, like here: . If this attribute is not present on the target its + title is used, like this: . If there is no title, + the name of the element is presented. + + + + + External links + + Links to external sources are done with the link element. Place the + target's full URI in the xlink:href attribute. For instance, + ]]> + links to the eXist home page. + To open the link on a new page, add condition="_blank". + + + + + Article links + + To create a link to another article in this documentation set, also use the + link element. However, use the target's document name (without the + .xml extension) as the link address. For instance, + ]]> links to the documentation home page. + + + + + + + + + + + Generating indexes + + Indexes can be automagicly generated. For examples have a look at the documentation home page. There are three + variations: + + + Alphabetical index on title + + Add a role="indexontitle" attribute to a para element. An + index on title will be generated after it. + + + + Index on keyword, all + + Add a role="indexonkeyword" attribute to a para element. + An index on keywords (as specified in the info element, see ) will be generated after it. + + + + Index on keyword, single + + Add a role="indexonkeyword:somekeyword" attribute to a + para element to generate a list of articles with this specific keyword + (as specified in the info element, see ). + + + + + + + + + + + Additional tools + + There are two additional pages in the documentation app to help authors and + editors: + + + + Editorial view + + + The editorial view provides an overview of all the articles and their incoming and + outgoing links. + + + + + Diagnostics + + + Performs limited diagnostics. Checks whether the outgoing links in articles + to other articles, listings and assets are ok. Also checks whether there are + unreferenced (and therefore superfluous) listings and assets. + + + + + + + diff --git a/src/main/xar-resources/data/documentation/documentation.xml b/src/main/xar-resources/data/documentation/documentation.xml index eb8c7c58..6899c77e 100644 --- a/src/main/xar-resources/data/documentation/documentation.xml +++ b/src/main/xar-resources/data/documentation/documentation.xml @@ -1,685 +1,829 @@
- - Documentation - 1Q18 - - getting-started - - - - Welcome to eXist-db. This article serves as an index to the eXist-db documentation - articles, which will help you getting to know, install and use eXist-db. - - - - - Getting Started - The following articles and resources will help you getting started using eXist. - - - - Basic Installation - - How to do a basic installation of eXist-db and fire it up for the first - time. - - - - Screencasts - - For the first steps with your freshly installed eXist-db, watch the - screencasts available on the eXist-db homepage - - - - eXist-db - Book - - There is a whole book about eXist-db, written for both the novice - and the more experienced user. - - - - Getting Help - - This article will tell you where to go for help and advice. - - - - Dashboard - - Learn how to use and populate eXist-db's main user interface, the - dashboard. - - - - Uploading files - - Learn how to get files in and populate your database. - - - - WebDAV - - One of the ways to easily view and populate the database is using the - WebDAV protocol. - - - - Using Collections in eXist-db - - Provide information about how to set up a collection structure in - eXist-db. - - - - Learning XQuery - - Provides tips and resources for newcomers to XQuery and eXist-db - - - - How to use oXygen together with eXist-db - - If you're using oXygen, this article describes how - to make the most of it in combination with eXIst-db. - - - - How to report an issue - - If you think you found a bug, this article will tell you how to report - it. - - - - - - - - - - XQuery - eXist-db's main programming language is XQuery. This documentation set does not - contain a full introduction to XQuery. For this read the excellent book about XQuery by - Priscilla Walmsley. The XQuery related articles below discuss specific eXist-db - related details or shed light on some of the lesser known features of the - language. - - - - Learning XQuery - - This provides tips and resources for newcomers to XQuery and - eXist-db - - - - XQuery in eXist-db - - This discusses the XQuery implementation of eXist-db for the somewhat more - experienced user - - - - KWIC (Keywords In Context) - - This article will learn you how to display search results in context - (parts of the document surrounding the search match). - - - - XQuery Update Extensions - - Tells you how to update XML stored in the database using eXist-db's update - extension. - - - - xqDoc - - Describes the built-in XQuery documentation system. - - - - XQsuite - - This is an annotation based framework that allows you to add tests to - XQuery functions and execute them. - - - - - - - - - - Application development - - eXist-db is not only a database but also an excellent application development - platform. The following articles will help you find your way in this: - - - - Getting Started with Web Application - Development - - This will help you build a basic web application using the built-in HTML templating framework. - - - - Using Collections in eXist-db - - Provide information about how to set up your application's collection - structure in eXist-db. - - - - xmldb module - - Provides an overview of the functions to query and manipulate the database - contents in the xmldb module. - - - - URL Rewriting - - How to control the mapping of URL's to code in eXist-db. This is one the - most important mechanisms in eXist-db to base applications on. - - - - XSL Transformations - - Explains how to perform XSL Transformation in XQuery code. - - - - Indexing - - This will provide you with an overview of eXist-db's indexes and how to - configure them. More about indexing in: - - - Full Text indexing - - Will provide with all the information necessary to use - eXist-db's Lucene based full-text indexing. - - - - N-Gram Index - - Provides information on how to configure the - ngram index. - - - - Range Index - - This article describes eXist-db's super fast modularized range - index based on Apache Lucene. - There is also an older - version of the range index, kept for compatibility - reasons. Usage of this range index is discouraged. - - - - - - - XInclude Support - - Explains how to use XInclude in eXist-db. - - - - The beginners guide to - XRX - - This article will learn you how to create a simple application using XRX - (XForms, REST, XQuery). - - - - Package Repository - - How to work with EXPath packages in eXist-db. - - - - Content extraction - - Shows how to extract and index non-XML contents, like PDF or Word - documents. - - - - REST-Style Web API - - Explains how to use eXist-db's REST interface, a useful tool in building - applications. - - - - HTTP Request/Session - - Provides information about the functions available working with HTTP - requests and sessions. - - - - Scheduler Module - - How to regularly schedule jobs. - - - - Security - - When you get serious writing applications, you need to be aware of the - security model of eXist-db. - - - - REST-Style Web API - - Explains how to use eXist-db's REST interface, a useful tool in building - applications. - - - - XML Validation - - Explains how to do XML validation in eXist-db. - - - - HTTP Request/Session - - Provides information about the functions available working with HTTP - requests and sessions. - - - - XForms Introduction - - Introduces the bright shiny world of XForms inside eXist-db to you. - - - - Tuning the Database - - Explains what you can do to optimize your queries and indexes. - - - - Configuring Database Triggers - - Will tell you how to use and configure triggers in your database that - fire when things get created, updated or deleted. - - - - Versioning Extension - - Explains how to use the built-in versioning system of eXist-db. - - - - XQsuite - - This is an annotation based framework that allows you to add tests to - XQuery functions and execute them. - - - - - - - - - - Interfaces - - eXist-db provides many ways of interfacing with the database. - - - - REST-Style Web API - - Explains how to use eXist-db's REST interface. - - - - SOAP Interface Developer's Guide - - Explains how to add a SOAP interface to eXist-db using Java code. - - - - XML-RPC API Developer's Guide - - Explains how to interface with eXist-db using the XML-RPC API. - - - - WebDAV - - Explains how to use eXist-db's WebDAV interface. - - - - - - - - - - Operations - Operations is the art of installing eXist-db and keeping it up-and-running - professionally. This includes things like more advanced installation types, doing - backups and restores, automate data transfers, etc. - - - - Java Admin Client - - This is a utility for performing basic administrative tasks. It has both a - GUI and a command line interface. - - - - Configuration - - How to configure eXist-db using its main configuration file - conf.xml. - - - - Backups - - How to backup and restore an eXist-db database. - - - - Advanced Installation - - How to install eXist-db on a headless (no GUI) system and run it as a - service. - - - - Ant tasks - - How to use the specific eXist-db Ant tasks to automate common - system administration and operation tasks. - - - - Database Deployment - - How to install eXist-db as a stand-alone or embedded server. - - - - Building eXist - - How to build Java .jar or .war files from an - eXist distribution. - - - - Performance FAQ - - Contains a short FAQ about eXist-db's performance. - - - - Production Use - Good - Practice - - When you use eXist-db on a production system, please read this for - advice. - - - - Production use - Proxying eXist-db - behind a Web Server - - You can proxy eXist-db behind a web server like Nginx or Apache. This - article will provide you with some examples. - - - - JMX - - eXist has a JMX interface for access to internal statistics about memory, - caching, etc. - - - - incompatibilities overview - - Consult this article when you upgrade from an older version of - eXist-db. - - - - Scheduler Module - - Scheduling jobs (like backups) is a useful tool in an eXist-db - installation. - - - - Security - - The security model of eXist. Also explains how to connect eXist-db to - other authentication realms like LDAP or OAuth. - - - - Tuning the Database - - Explains what you can do to optimize the database's performance. - - - - Performance FAQ - - Contains a short FAQ about eXist-db's performance. - - - - - - - - - - Upgrade Guide - - Helps you updating to a new version of eXist-db. - - - - incompatibilities overview - - Consult this article when you upgrade from an older version of - eXist-db. - - - - - - - - - - Java development - eXist-db is based on Java. Besides using eXist-db as a stand-alone application - platform, you can also use it from within Java code. The following - articles will help you with this. - - - - Database Deployment - - How to install eXist-db as a stand-alone or embedded server. An embedded - server can be accessed directly from Java code. - - - - Writing Java Applications with the XML:DB - API - - Explains how to work with eXist-db from Java code using the XML:DB API. - This API provides a common interface to native or XML-enabled databases and - supports the development of portable, reusable applications. - - - - Building eXist - - How to build Java .jar or .war files from an - eXist distribution. - - - - Developer's Guide to Modularized - Indexes - - Explains how the internal indexing mechanism works and how to add your own - indexes to it. - - - - Log4j Logging Guide - - This article explains how to add logging to your Java code using - Log4J. - - - - SOAP Interface Developer's Guide - - Explains how to add a SOAP interface to eXist-db using Java code. - - - - XML-RPC API Developer's Guide - - Explains how to interface with eXist-db using the XML-RPC API. - - - - Extension Modules - - Provides an overview of how to create eXist-db extension modules (in Java) - and contains a list of available extension modules. - - - - JMX - - eXist provides access to various management interfaces using JMX. - - - - Developer's Guide to Modularized - Indexes - - Explains how the internal indexing mechanism works and how to add your own - indexes to it. - - - - Log4j Logging Guide - - This article explains how to add logging to your Java code using - Log4J. - - - - SOAP Interface Developer's Guide - - Explains how to add a SOAP interface to eXist-db using Java code. - - - - XML-RPC API Developer's Guide - - Explains how to interface with eXist-db using the XML-RPC API. - - - - Extension Modules - - Provides an overview of how to create eXist-db extension modules (in Java) - and contains a list of available extension modules. - - - - - - - - - - Developing eXist-db - - The following articles provide information on how to work on eXist-db itself, either - by enhancing its code or providing documentation. - - - - eXist-db Developer Manifesto - - This article lays out guidelines for developers that wish to contribute to - eXist-db's code base itself. - - - - Code Review Guide - - Provides instructions how to review somebody else's (or your own of - course) code. - - - - Author Reference - - Explains how to write a documentation article for eXist-db (like the ones - you are looking at now). - - - - Legal Statement - - Provides information about the legal status of eXist as an open source - product. - - - - - - - - - - Alphabetical index - This section lists all available articles in title alphabetical order. - - - - - - - Subject index - This section lists all available articles by subject. - - - -
\ No newline at end of file + + Documentation + 1Q18 + + getting-started + + + + Welcome to eXist-db. This article serves as an index to the eXist-db documentation articles, + which will help you getting to know, install and use eXist-db. + + + + + Getting Started + The following articles and resources will help you getting started using eXist. + + + + + Basic Installation + + + How to do a basic installation of eXist-db and fire it up for the first time. + + + + Screencasts + + For the first steps with your freshly installed eXist-db, watch the screencasts + available on the eXist-db + homepage + + + + + + eXist-db Book + + + There is a whole book about eXist-db, written for both the novice + and the more experienced user. + + + + + Getting Help + + + This article will tell you where to go for help and advice. + + + + + Dashboard + + + Learn how to use and populate eXist-db's main user interface, the dashboard. + + + + + Uploading files + + + Learn how to get files in and populate your database. + + + + + WebDAV + + + One of the ways to easily view and populate the database is using the WebDAV + protocol. + + + + + Using Collections in eXist-db + + + Provide information about how to set up a collection structure in eXist-db. + + + + + Learning XQuery + + + Provides tips and resources for newcomers to XQuery and eXist-db + + + + + How to use oXygen together with eXist-db + + + If you're using oXygen, this article describes how to make the most of it in combination with + eXIst-db. + + + + + How to report an issue + + + If you think you found a bug, this article will tell you how to report it. + + + + + + + + + + XQuery + eXist-db's main programming language is XQuery. This documentation set does not contain a + full introduction to XQuery. For this read the excellent book about XQuery by Priscilla + Walmsley. The XQuery related articles below discuss specific eXist-db related details + or shed light on some of the lesser known features of the language. + + + + + Learning XQuery + + + This provides tips and resources for newcomers to XQuery and eXist-db + + + + + XQuery in eXist-db + + + This discusses the XQuery implementation of eXist-db for the somewhat more + experienced user + + + + + KWIC (Keywords In Context) + + + This article will learn you how to display search results in context (parts of the + document surrounding the search match). + + + + + XQuery Update Extensions + + + Tells you how to update XML stored in the database using eXist-db's update + extension. + + + + + xqDoc + + + Describes the built-in XQuery documentation system. + + + + + XQsuite + + + This is an annotation-based framework that allows you to add tests to XQuery + functions and execute them. + + + + + + + + + + Application development + + eXist-db is not only a database but also an excellent application development platform. + The following articles will help you find your way in this: + + + + + Getting Started with Web Application + Development + + + This will help you build a basic web application using the built-in HTML templating framework. + + + + + Using Collections in eXist-db + + + Provide information about how to set up your application's collection structure in + eXist-db. + + + + + xmldb module + + + Provides an overview of the functions to query and manipulate the database contents + in the xmldb module. + + + + + URL Rewriting + + + How to control the mapping of URL's to code in eXist-db. This is one the most + important mechanisms in eXist-db to base applications on. + + + + + XSL Transformations + + + Explains how to perform XSL Transformation in XQuery code. + + + + + Indexing + + + This will provide you with an overview of eXist-db's indexes and how to configure + them. More about indexing in: + + + + Full Text indexing + + + Will provide with all the information necessary to use eXist-db's Lucene based + full-text indexing. + + + + + N-Gram Index + + + Provides information on how to configure the ngram index. + + + + + Range Index + + + This article describes eXist-db's super fast modularized range index based on + Apache Lucene. + There is also an older version of the range + index, kept for compatibility reasons. Usage of this range index is + discouraged. + + + + + + + + Testing and Validation + + + This will provide you with an overview of testing eXist-db and its applications. More about testing in: + + + + XML Validation + + + Explains how to do XML validation in eXist-db. + + + + + XQsuite + + + This is an annotation-based framework that allows you to add unit tests to XQuery + functions and execute them. + + + + + Integration testing + + + Provides information on how to configure automated test pipelines, and recommends minimal test configurations. + + + + + + + + XInclude Support + + + Explains how to use XInclude in eXist-db. + + + + + The beginners guide to XRX + + + This article will learn you how to create a simple application using XRX (XForms, + REST, XQuery). + + + + + Package Repository + + + How to work with EXPath packages in eXist-db. + + + + + Content extraction + + + Shows how to extract and index non-XML contents, like PDF or Word documents. + + + + + REST-Style Web API + + + Explains how to use eXist-db's REST interface, a useful tool in building + applications. + + + + + HTTP Request/Session + + + Provides information about the functions available working with HTTP requests and + sessions. + + + + + Scheduler Module + + + How to regularly schedule jobs. + + + + + Security + + + When you get serious writing applications, you need to be aware of the security + model of eXist-db. + + + + + XForms Introduction + + + Introduces the bright shiny world of XForms inside eXist-db to you. + + + + + Tuning the Database + + + Explains what you can do to optimize your queries and indexes. + + + + + Configuring Database Triggers + + + Will tell you how to use and configure triggers in your database that fire when + things get created, updated or deleted. + + + + + Versioning Extension + + + Explains how to use the built-in versioning system of eXist-db. + + + + + + + + + + Interfaces + + eXist-db provides many ways of interfacing with the database. + + + + + REST-Style Web API + + + Explains how to use eXist-db's REST interface. + + + + + SOAP Interface Developer's Guide + + + Explains how to add a SOAP interface to eXist-db using Java code. + + + + + XML-RPC API Developer's Guide + + + Explains how to interface with eXist-db using the XML-RPC API. + + + + + WebDAV + + + Explains how to use eXist-db's WebDAV interface. + + + + + + + + + + Operations + Operations is the art of installing eXist-db and keeping it up-and-running professionally. + This includes things like more advanced installation types, doing backups and restores, + automate data transfers, etc. + + + + + Java Admin Client + + + This is a utility for performing basic administrative tasks. It has both a GUI and a + command line interface. + + + + + Configuration + + + How to configure eXist-db using its main configuration file + conf.xml. + + + + + Backups + + + How to backup and restore an eXist-db database. + + + + + Advanced Installation + + + How to install eXist-db on a headless (no GUI) system and run it as a + service. + + + + + Ant tasks + + + How to use the specific eXist-db Ant tasks to automate common system + administration and operation tasks. + + + + + Database Deployment + + + How to install eXist-db as a stand-alone or embedded server. + + + + + Building eXist + + + How to build Java .jar or .war files from an eXist + distribution. + + + + + Performance FAQ + + + Contains a short FAQ about eXist-db's performance. + + + + + Production Use - Good Practice + + + When you use eXist-db on a production system, please read this for advice. + + + + + Production use - Proxying eXist-db behind a Web + Server + + + You can proxy eXist-db behind a web server like Nginx or Apache. This article will + provide you with some examples. + + + + + JMX + + + eXist has a JMX interface for access to internal statistics about memory, caching, + etc. + + + + + incompatibilities overview + + + Consult this article when you upgrade from an older version of eXist-db. + + + + + Scheduler Module + + + Scheduling jobs (like backups) is a useful tool in an eXist-db installation. + + + + + Security + + + The security model of eXist. Also explains how to connect eXist-db to other + authentication realms like LDAP or OAuth. + + + + + Tuning the Database + + + Explains what you can do to optimize the database's performance. + + + + + Performance FAQ + + + Contains a short FAQ about eXist-db's performance. + + + + + + + + + + + Upgrade Guide + + + Helps you updating to a new version of eXist-db. + + + + + incompatibilities overview + + + Consult this article when you upgrade from an older version of eXist-db. + + + + + + + + + + Java development + eXist-db is based on Java. Besides using eXist-db as a stand-alone application platform, + you can also use it from within Java code. The following articles will + help you with this. + + + + + Database Deployment + + + How to install eXist-db as a stand-alone or embedded server. An embedded server can + be accessed directly from Java code. + + + + + Writing Java Applications with the XML:DB API + + + Explains how to work with eXist-db from Java code using the XML:DB API. This API + provides a common interface to native or XML-enabled databases and supports the + development of portable, reusable applications. + + + + + Building eXist + + + How to build Java .jar or .war files from an eXist + distribution. + + + + + Developer's Guide to Modularized Indexes + + + Explains how the internal indexing mechanism works and how to add your own indexes + to it. + + + + + Log4j Logging Guide + + + This article explains how to add logging to your Java code using Log4J. + + + + + SOAP Interface Developer's Guide + + + Explains how to add a SOAP interface to eXist-db using Java code. + + + + + XML-RPC API Developer's Guide + + + Explains how to interface with eXist-db using the XML-RPC API. + + + + + Extension Modules + + + Provides an overview of how to create eXist-db extension modules (in Java) and + contains a list of available extension modules. + + + + + JMX + + + eXist provides access to various management interfaces using JMX. + + + + + Developer's Guide to Modularized Indexes + + + Explains how the internal indexing mechanism works and how to add your own indexes + to it. + + + + + Log4j Logging Guide + + + This article explains how to add logging to your Java code using Log4J. + + + + + SOAP Interface Developer's Guide + + + Explains how to add a SOAP interface to eXist-db using Java code. + + + + + XML-RPC API Developer's Guide + + + Explains how to interface with eXist-db using the XML-RPC API. + + + + + Extension Modules + + + Provides an overview of how to create eXist-db extension modules (in Java) and + contains a list of available extension modules. + + + + + + + + + + Developing eXist-db + + The following articles provide information on how to work on eXist-db itself, either by + enhancing its code or providing documentation. + + + + + eXist-db Developer Manifesto + + + This article lays out guidelines for developers that wish to contribute to + eXist-db's code base itself. + + + + + Code Review Guide + + + Provides instructions how to review somebody else's (or your own of course) + code. + + + + + Author Reference + + + Explains how to write a documentation article for eXist-db (like the ones you are + looking at now). + + + + + Legal Statement + + + Provides information about the legal status of eXist as an open source + product. + + + + + + + + + + Alphabetical index + This section lists all available articles in title alphabetical order. + + + + + + + Subject index + This section lists all available articles by subject. + + + + diff --git a/src/main/xar-resources/data/integration-testing/integration-testing.xml b/src/main/xar-resources/data/integration-testing/integration-testing.xml new file mode 100644 index 00000000..54f1f2a6 --- /dev/null +++ b/src/main/xar-resources/data/integration-testing/integration-testing.xml @@ -0,0 +1,91 @@ +
+ + Integration Testing + 4Q18 + + application-development + testing + + + + This article discusses intergration testing of eXist-db applications. It also covers recommendations for the configuration of automated test environments, and explains the + minimum testing requirements for apps that are published under the eXist-db namespace. + It assumes that you are familiar with the XQSuite framework for unit-testing, and the general strategies for designing tests in eXist-db. + + + Introduction + Creating an automated mininal testsuite is possible with relatively little effort. It pays to take the need for testing into account when you start developing your + application. This enables others to extend your program with new features, by knowning that these don't break existing functions. It also allows test-only contributions, helping you to gradually advance your test coverage. The following section will walk you trough the three main aspects of such a minimal test setup. + + Building on a clean system + Before you start designing tests, you should start to automate your build process. This ensures that things don't only work on your system, and it can catch some common + errors. + The examples in this article will use travis-ci as it is the most popular continual integration (ci) + provider in the eXist-db organization on Github, other popuar choices include appveyor, jenkins, circl-ci, … . + The services typically require a small configuration file, so you can build your code on a clean virtual machine, without the risk of local files interfering. For + travis the required name of such a file is .travis.yml and in its simplest form it would look like this: + + For the correct way to create such a configuration file for other ci providers please consult their documentation. In all cases, since eXist-db is written in Java, your + app should be build on a system that comes with the minimal Java version required by eXist-db. + + Most providers will automatically detect your build tool and run the required command even if you don't specify it. In the above example, our app to be tested uses + ant as a build tool, change ant to suite your needs, e.g. maven clean package, npm install, etc. + + If you have multiple build targets for production and development, you should make sure that each build target is actually run by the ci service. + You can extend this basic template according to your needs, e.g.: apps written in Java might want to run the build step on multiple Java versions (by adding - + openjdk11); or to test building on different operation systems. You should consult your ci services documentation for the list of available configuration options. + + + Add a running eXist-db instance and install your app + The next step takes the result of your automated build process and installs it in a running eXist-db instance (external tools might want to talk to a running eXist-db instance in some other way). We are going to use exist's docker images for this, since it is supported by all ci providers, and it tends to be the quickest way of + getting an instance up and running. Let's extend the file created in the previous section. + + The :release and :latest tags are specifically designed for use in ci environments. You can also specify exact version to use + (e.g. :4.4.0) to ensure backwards compatibility. These two tags will ensure that your code is tested against both the most current stable release, and upcoming changes ahead of time. + + + To actually install the app, we copied it into exist's autodeploy folder, which will make sure that any dependencies that you declared + for your app will also be installed. If you require more complex installation steps, you can find more examples and links in the docker-existdb readme. + + So far, we have simply automated the basic steps of building and installing your app. This already catches some basic and particularly severe errors, but it is not a very realistic test of what users actually experience when they install your application. Before we can refine the way we simulate usage patterns, we first need to add the means of running actual tests within our ci environment. + + + Integrating unit tests into ci + Integration testing and unit testing go hand in hand, as one without the other does not work well. If you are writing an application, you already should have unit tests for the functional components of your code. By running your unit tests inside your ci server, these become immediately visible to potential contributors, and you have the advantage of immediate feedback on every code change. + + Tests that are invisible to other contributers because they are hidden away, and have only ever been run on the original author's system, are of very limited use. + + As with the previous options there are different test runners to do this work for you, such as junit for Java, Mocha for JavaScript, XQSuite for XQuery. To run your tests, we are going to leverage the support for running unit tests of our build system (e.g.: npm test, mvm test, + …): + + + Just as with the building example, many providers will execute this command automatically. But even in a simple case it helps others understand your code, and to find your tests, if you make the test command explicit. + + If you use more then one test runner, you can simply add additional test commands to the script parameter. Since exist with our app is already running in the background, it is also possible to run your XQSuite unit tests. You can see how this is configured, for apps using our the yeoman templates. + How to write good unit tests is beyond the scope of this article. Whenever you are struggling with your integration test you should, however, ask yourself if what you are trying to achieve might not be better served by creating unit tests. Whichever solution works best for you, you should not rely on either unit or integration tests alone, and both should be integrated into your ci pipeline. + + + Testing your app in a controlled context + Unlike unit tests which excel at testing individual functions, and are quick to write and perform, interation tests focus on the complex interaction between your code and that of the larger + environment. Just search for 2 unit tests 0 + integration tests to see what we mean. + For eXist-db applications, integration tests will typically involve a browser, as we are trying to mimick the way a user interacts with our application from their system. + + While our examples focus on browser testing, you can see shell based integration tests at the testsuite for building the docker images we used earlier. + Common tools for browser testing include Cypress, Selenium, and webdriver. As before the choice is up to + you, whichever you choose it should be clearely documented so your contributors know how to adjust test cases for new features and how to maintain your tests. We focus on Cypress, as it does not require any additional steps for configuring a browser first. + + If you need to perform cross-browser testing you can take a look at services such as sauce-labs + + You can simply execute the Cypress test command inside your ci test script after the unit test command we added earlier. + + With Cypress you write your tests in the same fashion as you would with Mocha unit tests; however, you now address the rendered document inside a browser instead of individual js functions. + + The above example opens a page (the dashboard) in the browser, logs in, and closes the window it just opened. You can do many more things, but these examples are meant to provide a good starting point for creating your first integration test. If there are any console errors or problems with rendering content Cypress will create an error message and your tests will fail. To check the syntax of these commands, and to see many more examples please visit the Cypress documentation. + Now that we have built our app in a clean system, executed its unit tests, and opened the start page of our freshly installed app in clean eXist-db instance, we have achieved a basic smoke test. We switched it on and there was no smoke. Proper testing can now commence. Obviously you might want to visit multiple pages, or compare screenshots to avoid visual regressions, or compare images in multiple browsers, or … . + All of which are excellent ideas, and with these basics in place it hopefully no longer seem so daunting a task. Depending on your own specific requirments we would encourage your to browse other eXist-db repositories in addition to the documentation of your ci and testsuite vendors. Chances are someone already has created a solid test, similar to your needs. + + +
diff --git a/src/main/xar-resources/data/integration-testing/listings/cypress.txt b/src/main/xar-resources/data/integration-testing/listings/cypress.txt new file mode 100644 index 00000000..f021e1a2 --- /dev/null +++ b/src/main/xar-resources/data/integration-testing/listings/cypress.txt @@ -0,0 +1,27 @@ +describe('The dashboard', function() { + it('should load', function() { + // Go to Dashboard + cy.visit('/dashboard/index.html') + }) + // Click the login button + describe('login', function() { + before(function() { + cy.get('#user_label').click() + + cy.get('#dijit_form_ValidationTextBox_0').type('admin') + + cy.get('#dijit_form_Button_0_label').click() + }) + + // check if Collection browser is there + it('should see Collection Browser', function() { + // click Collection Browser + cy.contains('Collections').click() + + // more tests go here … + + // close the window + cy.get('#inlineClose') + }) + }) +}) diff --git a/src/main/xar-resources/data/integration-testing/listings/travis-1.txt b/src/main/xar-resources/data/integration-testing/listings/travis-1.txt new file mode 100644 index 00000000..5657c258 --- /dev/null +++ b/src/main/xar-resources/data/integration-testing/listings/travis-1.txt @@ -0,0 +1,10 @@ +# Tell travis that we want java +language: java + +# This should be the minimal java version required by eXist-db +jdk: + - oraclejdk8 + +# This makes the build command explicit. +install: + - ant diff --git a/src/main/xar-resources/data/integration-testing/listings/travis-2.txt b/src/main/xar-resources/data/integration-testing/listings/travis-2.txt new file mode 100644 index 00000000..0e3e0f97 --- /dev/null +++ b/src/main/xar-resources/data/integration-testing/listings/travis-2.txt @@ -0,0 +1,25 @@ +language: java + +jdk: + - oraclejdk8 + - oraclejdk11 + +# Tell travis that we are using docker +services: + - docker + +# Always test against current and upcoming releases +env: + - img=existdb/existdb:latest + - img=existdb/existdb:release + +install: + - ant develop + +# take the .xar created above and install and deploy in a clean exist instance +before_script: + - docker cp ./build/*-dev.xar exist-ci:exist/autodeploy + - docker start exist-ci + # exist needs time + - sleep 30 + - docker ps diff --git a/src/main/xar-resources/data/integration-testing/listings/travis-3.txt b/src/main/xar-resources/data/integration-testing/listings/travis-3.txt new file mode 100644 index 00000000..7c5063f7 --- /dev/null +++ b/src/main/xar-resources/data/integration-testing/listings/travis-3.txt @@ -0,0 +1,26 @@ +language: java + +jdk: + - oraclejdk8 + - oraclejdk11 + +services: + - docker + +env: + - img=existdb/existdb:latest + - img=existdb/existdb:release + +install: + - ant develop + +before_script: + - docker cp ./build/*-dev.xar exist-ci:exist/autodeploy + - docker start exist-ci + # exist needs time + - sleep 30 + - docker ps + +# This makes the test command explicit. +script: + - ant test diff --git a/src/main/xar-resources/data/integration-testing/listings/travis-4.txt b/src/main/xar-resources/data/integration-testing/listings/travis-4.txt new file mode 100644 index 00000000..46fe10aa --- /dev/null +++ b/src/main/xar-resources/data/integration-testing/listings/travis-4.txt @@ -0,0 +1,26 @@ +language: java + +jdk: + - oraclejdk8 + - oraclejdk11 + +services: + - docker + +env: + - img=existdb/existdb:latest + - img=existdb/existdb:release + +install: + - ant develop + +before_script: + - docker cp ./build/*-dev.xar exist-ci:exist/autodeploy + - docker start exist-ci + # exist needs time + - sleep 30 + - docker ps + +script: + - ant test + - npx cypress run diff --git a/src/main/xar-resources/data/testing/assets/browse-form.png b/src/main/xar-resources/data/testing/assets/browse-form.png new file mode 100644 index 00000000..a3f3d61a Binary files /dev/null and b/src/main/xar-resources/data/testing/assets/browse-form.png differ diff --git a/src/main/xar-resources/data/testing/assets/date-form.png b/src/main/xar-resources/data/testing/assets/date-form.png new file mode 100644 index 00000000..eebc4ea5 Binary files /dev/null and b/src/main/xar-resources/data/testing/assets/date-form.png differ diff --git a/src/main/xar-resources/data/testing/listings/data-1.txt b/src/main/xar-resources/data/testing/listings/data-1.txt new file mode 100644 index 00000000..d0573583 --- /dev/null +++ b/src/main/xar-resources/data/testing/listings/data-1.txt @@ -0,0 +1,3 @@ +… +$USER_INPUT +… diff --git a/src/main/xar-resources/data/testing/listings/valid.txt b/src/main/xar-resources/data/testing/listings/valid.txt new file mode 100644 index 00000000..1f4fb216 --- /dev/null +++ b/src/main/xar-resources/data/testing/listings/valid.txt @@ -0,0 +1,3 @@ +declare function local:check-input($input as item()*) as xs:date { + $input cast as xs:date +}; diff --git a/src/main/xar-resources/data/testing/listings/xq-test.txt b/src/main/xar-resources/data/testing/listings/xq-test.txt new file mode 100644 index 00000000..00a1b957 --- /dev/null +++ b/src/main/xar-resources/data/testing/listings/xq-test.txt @@ -0,0 +1,6 @@ +declare + %test:arg("input", '11.11.2010') %test:assertEquals('11.11.2010') + %test:arg("input", '5') %test:assertError() + function local:check-input($input as item()*) as xs:date { + $input cast as xs:date +}; diff --git a/src/main/xar-resources/data/testing/testing.xml b/src/main/xar-resources/data/testing/testing.xml new file mode 100644 index 00000000..17f52a21 --- /dev/null +++ b/src/main/xar-resources/data/testing/testing.xml @@ -0,0 +1,118 @@ +
+ + Testing + 4Q18 + + application-development + testing + + + + In this article we discuss the types of testing available in relation to eXist-db and its applications. It assumes readers have a basic understanding of XML and + XQuery. + + + Overview + Different kinds of tests play an essential role in maintaining a high quality code base both for eXist-db itself, and for applications that interact with eXist-db. + + The test framework of eXist-db and its apps is under continuous development, with frequent changes to the software and services in use. Covering them all is not + feasible in this article. We recommend that you check the official Github repositories and search for + systems that are similar to yours for further insipiration. + + While the terminology varies between different sources dealing with software testing, the following categories are widely acknowledged to apply irrespective of terminological differences. + + + + + Validation: + XML comes with different means of validating and thus testing your data strucutures. When implementing a testsuite, you should first consider how your code can leverage + these native features via strong typing and schema-based validation. + + + + + Unit testing: + These tests typically operate at the level of individual functions. eXist-db has its own unit testing framework for Xquery named XQSuite. It is prominently + used in our bug reports and source-code repo. Most applications will need to run multiple unit test tools to test, e.g. Java, JavaScript, and Xquery code. + + + + + Integration testing: + Here we look at your programm as a whole, similar to how a user would interact with it, by simulating user input and running you program not in isolation but in + concert with different applications. + + + + (WIP) Performance and Stress testing: Performance testing ensures that your code keeps running under heavy load, or with very little + resources. Stress tests go one step further by purposfully crashing your application to ensure that it recovers gracefully. Practically speaking, the difference between + the two is fluid. + + + When running these different tests in concert, we usually speak of end-to-end testing (e2e). There will never be a one-size-fits-all solution to take care of all your testing needs. Writing good testsuites takes time and planning. You can find out more about available + options and their use cases in the articles linked above. The remainder of this article will discuss some general considerations for how to design your testsuite, without + referencing a specifc implementation. The listings use pseudo-code to focus on relevant lines. Pseudo-code is not intended to be executable, so it won't run if you copy-paste it into e.g. eXide. For working code, please consult the articles links throughout. + + + + Picking a Type + Generally speaking, you should follow the sequence provided above. Whatever can be tested as part of your validation tests, is best tested there. When validation testing + isn't a good fit, go for unit tests, followed by integration and lastly performance tests. The reasons for this are twofold: on the one hand, validation tests tend to be + faster than unit tests, which in turn are slower then integration tests and performance tests. On the other hand, each type tends to be easier to configure, with a higher + chance of providing you with the information that you need when a test fails. + + Selection Example + To illustrate this, let's take a theoretical example: imagine your code includes an online form where users submit a date that needs to be stored in the database. For + the purpuse of this example it doesn't matter if this is text-input form, a calendar selector, or other form. Provided that you are using eXist-db, we assume that the date + will be stored in some kind of xml file. + + A basic form example + a simple form: + + + + + that will create something like this: + + + By using a schema for validating this xml file, you can ensure the correct datatype xs:date of the provided input, and prevent users + from storing something in the database that is not a date. + + Using native types + + + To check that your local:check-input function is working as expected, you can define a few unit tests. Typically, such tests + would include corner cases like BCE dates, leap years, or dates in foreign date formats, as well as a basic test date that should simply store in the db. + + Testing Corner Cases + + For in-depth examples of unit testing in xQuery please see the examples in the xQsuite article + + Integration testing is most effective when it can rely on existing unit tests and focus on how a user typically interacts with the UI components. + Does the browser display the input form correctly at different resolutions, can the form be selected with a mouse, or on touch screens, … ? + + browser testing + + + + + + + + + Lastly, performance and stress tests would simply assume all of the above to be in place. They would check for problems that might occur, when + large numbers of users select the same or invalide dates, or access the same form simultaneously. + This quick example should give you an idea on how to think about testing your code. Each type of test has its own reason for being, and requires your attention. A good + performance test cannot substitue unit or integration tests. If you feel that your tests are not working well, because they frequently fail to show problems in your code, + and when they do they don't make identifying the source of the problem easier, chances are that you are trying to test something with one type of test, that would be better + suited for another type. + + + + Test Coverage + Ideally, we would always achieve full test coverage for our code. So that every meaningful unit of code has a corresponding set of tests. While other programming + languages offer automated means of analysing your code to indentify areas that aren't tested, such tools are unfortunately not very common for Xquery. How much testing is necessary for you to ship with confidence, rests on each developer. For apps published under the eXist-db namespace, we have outlined a set of minimum requirementes (more is obviously desirable) which are featured in the section on integration testing. + +
diff --git a/src/main/xar-resources/data/validation/validation.xml b/src/main/xar-resources/data/validation/validation.xml index 49a1dfde..27130249 100644 --- a/src/main/xar-resources/data/validation/validation.xml +++ b/src/main/xar-resources/data/validation/validation.xml @@ -1,20 +1,42 @@ -
- - XML Validation - 2Q18 - - application-development - - - - - - eXist-db supports validation of XML documents. - There are two ways to validate documents: + schematypens="http://purl.oclc.org/dsdl/schematron"?>
+ + XML Validation + 2Q18 + + application-development + testing + + + + + + eXist-db supports validation of XML documents. + There are two ways to validate documents: + + + + is executed automatically when documents are + inserted into the database. + + + + is performed through the use of provided XQuery + extension functions. + + + + + + + + Implicit validation + + Implicit validation is executed automatically when documents are inserted into the + database. + To enable implicit validation, change eXist-db configuration by editing + conf.xml. The following two items must be configured: is executed automatically when @@ -323,177 +345,198 @@ Schematron 1.5 (.sch) - The jing() and jing-report() functions accept the - following parameters: - - - $instance - - The XML instance document as document node, element node, - xs:anyURI, or as Java file object. - - - - $grammar - - The grammar file can be referenced either as document node, element - node, xs:anyURI, binary document, or as Java file - object. - - - - - You can use util:binary-doc() to pass .rnc files as - binary document - - - - - - - - Validation report + + + + - A validation report contains the following for a valid document: + - + + Jing - For an invalid document the following is returned: - - When something goes wrong you might the following: - - - - - - - Grammar management - - The Xerces XML parser compiles all grammar files upon first use. For efficiency - reasons these compiled grammars are cached, resulting in a significant increase in - validation processing performance. However, sometimes it may be desirable to manually - clear this cache. For this purpose two grammar management functions are provided: - - - clear-grammar-cache() - - Removes all cached grammar and returns the number of removed - grammar - - - - pre-parse-grammar(xs:anyURI) - - Parses the referenced grammar and returns the namespace of the parsed - XSD. - - - - show-grammar-cache() - - Returns an XML report about all cached grammars. For instance: - - The BaseSystemId element typically does - not provide useful information. - - - - - + The Jing validation functions are based on James Clark's Jing library. eXist uses the maintained version that is available via Google Code. + The library relies on the com.thaiopensource.validate.ValidationDriver which supports a + wide range of grammar types: + + + XML schema (.xsd) + + + Namespace-based Validation Dispatching Language (.nvdl) + + + RelaxNG (.rng and .rnc) + + + Schematron 1.5 (.sch) + + + The jing() and jing-report() functions accept the following + parameters: + + + + $instance + + + The XML instance document as document node, element node, + xs:anyURI, or as Java file object. + + + + + $grammar + + + The grammar file can be referenced either as document node, element node, xs:anyURI, binary document, or as Java file object. + + + + You can use util:binary-doc() to pass .rnc files as binary document + + + + + + + Validation report + + A validation report contains the following for a valid document: + + + + For an invalid document the following is returned: + + When something goes wrong you might the following: + + + + + + + Grammar management + + The Xerces XML parser compiles all grammar files upon first use. For efficiency reasons + these compiled grammars are cached, resulting in a significant increase in validation + processing performance. However, sometimes it may be desirable to manually clear this cache. + For this purpose two grammar management functions are provided: + + + + clear-grammar-cache() + + + Removes all cached grammar and returns the number of removed grammar + + + + + pre-parse-grammar(xs:anyURI) + + + Parses the referenced grammar and returns the namespace of the parsed XSD. + + + + + show-grammar-cache() + + + Returns an XML report about all cached grammars. For instance: + + The BaseSystemId element typically does not provide + useful information. + + + - + - - Interactive Client + - The interactive shell mode of the Java Admin - Client provides a simple validate command that accepts the - similar explicit validation arguments. - + + Interactive Client - + The interactive shell mode of the Java Admin Client provides a simple + validate command that accepts the similar explicit validation + arguments. + - - Special notes + - - - To avoid potential deadlocking it is considered good practice to store XML - instance documents and grammar documents in separate collections. - - - Tomcat has an long standing bug which makes it impossible to register a custom - protocol handler (object URLStreamHandler) to the JVM. The alternative is to - register the object by setting the system property - java.protocol.handler.pkgs but this fails as well. - As a result, the validation features are only partly useable in Tomcat. There - are two alternatives: (1) switch to a recent version of Jetty, or (2) use - absolute URLs pointing the REST interface, e.g. - http://localhost:8080/exist/rest/db/mycollection/schema.xsd. - - - eXist relies heavily on features provided by the Xerces XML parser. Out of the - box the eXist izPack installer provides all required .jar files. - However, when eXist is installed in for instance Tomcat, the required parser - libraries need to be copied manually from the eXist lib/endorsed - directory into the server endorsed directory. - Required endorsed files: resolver-*.jar xalan-*.jar serializer-*.jar - xercesImpl-*.jar - + + Special notes - - The explicit validation is performed by Xerces (XML schema, DTD) and by oNVDL: oXygen XML NVDL - implementation based on Jing (XSD, RelaxNG, Schematron and Namespace-based - Validation Dispatching Language). - - - + + + To avoid potential deadlocking it is considered good practice to store XML instance + documents and grammar documents in separate collections. + + + Tomcat has an long standing bug which makes it impossible to register a custom + protocol handler (object URLStreamHandler) to the JVM. The alternative is to register + the object by setting the system property java.protocol.handler.pkgs but this + fails as well. + As a result, the validation features are only partly useable in Tomcat. There are two + alternatives: (1) switch to a recent version of Jetty, or (2) use absolute URLs pointing + the REST interface, e.g. + http://localhost:8080/exist/rest/db/mycollection/schema.xsd. + + + eXist relies heavily on features provided by the Xerces XML parser. Out of the box the + eXist izPack installer provides all required .jar files. However, when eXist + is installed in for instance Tomcat, the required parser libraries need to be copied + manually from the eXist lib/endorsed directory into the server endorsed directory. + Required endorsed files: resolver-*.jar xalan-*.jar serializer-*.jar + xercesImpl-*.jar + + + + + The explicit validation is performed by Xerces (XML schema, DTD) and by oNVDL: oXygen + XML NVDL implementation based on Jing (XSD, RelaxNG, Schematron and Namespace-based + Validation Dispatching Language). + + + - + - - References + + References - - - Apache xml-commons - resolver - - - OASIS XML Catalog - Specification V1.1 - - - Xerces caching - grammars. - - - jing-trang Schema validation and conversion based on RELAX NG - - - NVDL (Namespace-based Validation Dispatching Language) - - - Schematron - - - Relax NG - - - + + + Apache xml-commons resolver + + + + OASIS XML Catalog + Specification V1.1 + + + Xerces caching grammars. + + + + jing-trang + Schema validation and conversion based on RELAX NG + + + + NVDL (Namespace-based Validation Dispatching Language) + + + + Schematron + + + + + Relax NG + + + +
diff --git a/src/main/xar-resources/data/xqsuite/xqsuite.xml b/src/main/xar-resources/data/xqsuite/xqsuite.xml index 69b37f9d..26ab3cc3 100644 --- a/src/main/xar-resources/data/xqsuite/xqsuite.xml +++ b/src/main/xar-resources/data/xqsuite/xqsuite.xml @@ -1,16 +1,16 @@ -
- - XQSuite - Annotation-based Test Framework for XQuery - 4Q18 - - application-development - xquery - - +
+ + XQSuite - Annotation-based Test Framework for XQuery + 4Q18 + + application-development + testing + xquery + +