酷玩 Spark: Spark 源代码解析、Spark 类库等

Interactive and Reactive Data Science using Scala and Spark.

Distributed Tensorflow, Keras and PyTorch on Apache Spark/Flink & Ray

Oryx 2: Lambda architecture on Apache Spark, Apache Kafka for real-time large scale machine learning

Apache Spark docker image

Kubernetes operator for managing the lifecycle of Apache Spark applications on Kubernetes.

PySpark + Scikit-learn = Sparkit-learn

(Deprecated) Scikit-learn integration package for Apache Spark

A curated list of awesome Apache Spark packages and resources.

The Internals of Apache Spark

🚚 Agile Data Preparation Workflows made easy with dask, cudf, dask_cudf and pyspark

C# and F# language binding and extensions to Apache Spark

R interface for Apache Spark

Code examples that show to integrate Apache Kafka 0.8+ with Apache Storm 0.9+ and Apache Spark Streaming 1.1+, while using Apache Avro as the data serialization format.

An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.

Distributed Deep Learning, with a focus on distributed training, using Keras and Apache Spark.

Apache Spark enhanced with native Kubernetes scheduler back-end: NOTE this repository is being ARCHIVED as all new development for the kubernetes scheduler back-end is now on https://github.com/apache/spark/

A command-line tool for launching Apache Spark clusters.

REST web service for the true real-time scoring (<1 ms) of R, Scikit-Learn and Apache Spark models

Streaming System 相关的论文读物

Haskell on Apache Spark.

Code for Agile Data Science 2.0, O'Reilly 2017, Second Edition

A list about Apache Kafka

The Internals of Spark Structured Streaming

Wirbelsturm is a Vagrant and Puppet based tool to perform 1-click local and remote deployments, with a focus on big data tech like Kafka.

This is the development repository of SparkMeasure, a tool for performance troubleshooting of Apache Spark workloads. It simplifies the collection and analysis of Spark task metrics data.

Serverless proxy for Spark cluster

A boilerplate for writing PySpark Jobs