The Docker swarm cluster with Apache Spark and HDFS filesystem to run SageCal. The setup uses Docker images to build scaleble cluster.
| Name | Version |
|---|---|
| Hadoop | 2.9 |
| Spark | 2.2.0 |
| java | OpenJDK-8 |
| SageCal | - |
Note: The setup was only tested on Linux64 system.
docker pull fdiblen/spark-worker-dirac docker pull fdiblen/spark-master-dirac docker pull fdiblen/hadoop
Intallation instructions INSTALL.md
Instructions to submit a job JOBS.md