building-data-pipeline

INTRODUCTION

This repo contains working code for the talk on Building Scalable Data Pipeline with Apache Kafka and Spark at Nairobi JVM meet-up at iHub on 20-07-2019

INSTALLATIONS

Docker

Install Docker for your OS using this documentation: https://docs.docker.com/install/

Java

Check your Java installation in terminal using $java -version in bash

If Java is not installed, use this link to get a version of Java 8: https://java.com/en/download/help/download_options.xml

Scala 2.11

Check whether you have sbt/scala installed on the terminal using: $sbt sbtVersion in bash

If you don't have it installed, use this link to install: https://www.scala-lang.org/download/

Kafka

Use docker to run kafka off confluent using this documentation: https://docs.confluent.io/current/quickstart/ce-docker-quickstart.html

Scylla

Use docker to run Scylla using this documentation: https://docs.scylladb.com/operating-scylla/manager/1.3/run-in-docker/#running-with-docker