-
Notifications
You must be signed in to change notification settings - Fork 14
Requirements
Neon provides a JavaScript API, which is the simplest way to connect your software to the Neon server.
Alternatively, Neon provides web endpoints, so you can execute queries with any languages capable of making HTTP GET and POST requests.
You need to download and install the following dependencies to build and/or run Neon. They should be added to your PATH environment variable to build from the command line.
Neon is a Java application and therefore requires Java JDK. Java version 7 or 8 is required.
The Neon source code is hosted on GitHub. In order to download the code from the Neon repository, download Git and following the instruction for cloning an existing repository.
- Download Git
Example:
cd ~/workspace
git clone https://github.com/NextCenturyCorporation/neon
Node.js and npm are used to build the Neon JavaScript code.
- Download the Node.js installation, which includes npm
While Neon uses Gradle 2.1 as its build system, you do not need to download and install it yourself. We recommend that you use the Gradle wrapper, ./gradlew when building and working with Neon to avoid version conflicts. Instructions for building Neon using the Gradle wrapper are found in the Building Neon section.
Neon supports Mongo, ElasticSearch, and Spark SQL database systems. We recommend first time users download Mongo and configure Neon to use this database.
Neon also supports Spark database systems via the Hiveserver2 Thrift API. Installation instructions for the Spark SQL database are found here.
- Download Spark
- For testing purposes, you can use this Vagrant Spark SQL VM
- Neon has been tested against stock builds of Spark 1.1 through 1.5. If you have modified the stock build for a particular version of the Hadoop or Hive libraries, you may need to modify the version strings in neon/gradle/versions.gradle or the dependencies in neon/gradle/sharedDependencies.gradle and neon/neon-server/gradle/sparkSQLDependencies.gradle and recompile.
An experimental Elasticsearch (ES) connector is included in version 1.1.5. 1.1.5 will provide a basic capability to execute Neon queries against an Elasticsearch cluster with a few limitations and requirements on the ElasticSearch node. The initial capability is compatible with ES 1.7.x. ES 2.X support will follow in later builds. Installation instructions for ES are found here.
Neon uses a Transport Client to query ES. Transport Clients typically run against ports in the 9300-9400 range with 9300 being the default. The ES node to which Neon communicates must allow UDP and TCP traffic on that port for the connection to work properly.
Neon queries against date fields in ES will use the following date format: E MMM d HH:mm:ss zzz yyyy. For Neon date queries and aggregations to work properly with an ES index, any date fields in that index should allow the 'E MMM d HH:mm:ss zzz yyyy' format in addition to any others.
The Neon JS API uses the terms Database, Table, and Field when connecting to a dataset. When running against ES, these terms would refer to Index, Type, and Field, respectively. Thus, the following NEON calls:
var connection = new neon.query.Connection();
connection.connect(neon.query.Connection.ELASTICSEARCHH, "localhost");
connection.getDatabaseNames(myCallback)''
will return a list ES Indices visible to that ES node.
ES supports caching of aggregation results for ES queries that use search_type=count on a per index basis. Neon queries that involve only aggregations and do not return query hits set search_type=count by default. This allows ES to respond very quickly to repeated aggregation queries. The ES Shard query cache page covers enabling/disabling/clearing index caches in more details. An example of enabling caching on an existing index:
curl -XPUT localhost:9200/my_index/_settings -d'
{ "index.cache.query.enable": true }
'
Note that ES index and type names are passed along as-is. Wildcards are not stripped and can be used in index and type names wherever they are used in the Neon JS API.
For production environments, we recommend running Neon inside a Tomcat web server. Tomcat is not required to build Neon, and for development purposes a Jetty server can be run from the command line using ./gradlew jettyRun. Instructions for running Neon inside of Tomcat are found on the Deploying Neon page.
- Download Tomcat (version 7.0 or later)
Neon has been tested in Firefox and Chrome browsers.
Neon provides two versions of its JavaScript library; one version includes the needed JavaScript dependencies while the second does not include JavaScript dependencies.
To use the neon-nodeps.js no dependencies file, you will need to also include an appropriate version of each JavaScript library in your HTML. Using the neon-nodeps.js file requires the following additional libraries:
- jQuery 1.8.3 or higher
- Lo-Dash 1.3.1 or higher
- log4javascript 1.4.6 or higher
- Postal.js 0.8.11
- node-uuid uuid.js 1.2 or higher