Skip to content

Commit d9f6460

Browse files
committed
remove DI Spark Cluster deployment information
1 parent 97b5c13 commit d9f6460

File tree

1 file changed

+0
-34
lines changed
  • docs/deploy-and-configure/installation

1 file changed

+0
-34
lines changed

docs/deploy-and-configure/installation/index.md

Lines changed: 0 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -35,40 +35,6 @@ Special note on RHEL SELinux Support: there is no limitation for RedHat SELinux.
3535

3636
eccenca is heavily using `docker compose` for all kinds of internal and customer deployments. For more details on how to use `docker compose` based orchestration refer to [Scenario: Local Installation](../installation/scenario-local-installation/index.md) and [Scenario: Single Node Cloud Installation](../installation/scenario-single-node-cloud-installation/index.md).
3737

38-
## DataIntegration
39-
40-
### Running on a Spark Cluster
41-
42-
eccenca Build (DataIntegration) supports the execution of Build (DataIntegration) workflows in a cluster environment with Apache Spark.
43-
44-
#### Prerequisites
45-
46-
For the execution of Build (DataIntegration) in a Spark cluster the following software components from the Hadoop eco-system are recommended:
47-
48-
- Scala 2.11 or 2.10
49-
- Apache Spark 2.1.2 (compiled for Scala 2.11)
50-
- Apache Hadoop 2.7 (HDFS)
51-
- Apache Hive 1.2, with a relational data bases as meta store (e.g. Derby)
52-
53-
Recent versions of the following Hadoop distributions are generally supported as well:
54-
55-
- Hortonworks (HDP 2.5)
56-
- Cloudera (CDH 5.8)
57-
- Oracle Big Data Lite (4.6)
58-
- Microsoft HDInsight (based on HDP)
59-
60-
#### Installation <!-- Different Modes of Installation-->
61-
62-
A Spark application can run in three different modes:
63-
64-
- local mode
65-
- client mode
66-
- cluster mode
67-
68-
The local mode is for running Spark applications on one local machine. In the client mode the Build (DataIntegration) application will run outside of the cluster and create Spark Jobs to be executed in the cluster at run time. The cluster mode requires that the application using Spark runs completely in the cluster and is managed by the software running on the cluster (e.g. Spark, Apache Yarn, Mesos). Build (DataIntegration) supports local mode (for testing), client mode (for production, only with clusters managed by Spark) or cluster mode on Yarn (for production, integrates best with other distributed applications).
69-
70-
When running Build (DataIntegration) in a cluster, the same installation procedure and prerequisites apply as for the local installation. The application can be installed outside the cluster or on any cluster node. A number of configuration options have to be set to be able to connect to and use a Spark cluster. The necessary configuration options are described in [Build (DataIntegration)](./../configuration/dataintegration/index.md).
71-
7238
## Explore
7339

7440
### Scaling

0 commit comments

Comments
 (0)