|
| 1 | +##SJS Java Api |
| 2 | + |
| 3 | +The Java API in the Spark Job Server is designed to be a more Java friendly API for users to run Spark Jobs |
| 4 | +inside of SJS. |
| 5 | + |
| 6 | +There are a few differences between this API and the standard one. |
| 7 | + |
| 8 | +- The API is entirely Java based |
| 9 | +- Type safety is done via generics instead of type aliases |
| 10 | +- Requires it's own set of Context factories specific to Java (very important) |
| 11 | + |
| 12 | +###Basic Example |
| 13 | + |
| 14 | +```java |
| 15 | +public class JavaHelloWorldJob implements JSparkJob<String> { |
| 16 | + |
| 17 | + public String run(JavaSparkContext sc, JobEnvironment runtime, Config data) { |
| 18 | + return "Hi!"; |
| 19 | + } |
| 20 | + |
| 21 | + public Config verify(JavaSparkContext sc, JobEnvironment runtime, Config config) { |
| 22 | + return ConfigFactory.empty(); |
| 23 | + } |
| 24 | +} |
| 25 | +``` |
| 26 | +[Full File Here](https://github.com/spark-jobserver/spark-jobserver/blob/master/job-server-tests/src/main/java/spark/jobserver/JavaHelloWorldJob.java) |
| 27 | + |
| 28 | +This example does nothing Spark based, but gives an idea of the difference in structure. In order to run a job |
| 29 | +written like this, the user **MUST** use the `JavaSparkContextFactory`. This context is configured the same as |
| 30 | +any of the Context factories in the Scala API. In addition, there are context types for all Java related jobs. |
| 31 | +In the Java API, there are both context factories and interfaces that must be used for each kind of job. |
| 32 | +For instance the JSparkJob uses the JavaSparkContextFactory. |
| 33 | + |
| 34 | +###Additional Context and Job Types |
| 35 | + |
| 36 | +- `JavaSqlContextFactory` for SQLContext Jobs with `JSqlJob` |
| 37 | +- `JavaStreamingContextFactory` for StreamingContext jobs with `JStreamingJob` |
| 38 | +- `JavaHiveContextFactory` for HiveContext jobs with `JHiveJob` |
| 39 | + |
| 40 | +**NOTE: If you do not use the correct context, the jobs will probably not run correctly, it is imperative you configure |
| 41 | +jobs correctly.** |
0 commit comments