In Cassandra Lunch #115, Arpan Patel will discuss how to connect Google Dataproc and DataStax Astra with a demo showing you what configurations you will need to get the connection working! Accompanying Blog: Coming Soon! Sign Up For Our Newsletter: http://eepurl.com/grdMkn Join Cassandra Lunch Weekly at 12 PM EST Every Wednesday: https://www.meetup.com/Cassandra-Data... Cassandra.Link: https://cassandra.link/ Follow Us and Reach Us At: Anant: https://www.anant.us/ Awesome Cassandra: https://github.com/Anant/awesome-cass... Cassandra.Lunch: https://github.com/Anant/Cassandra.Lunch Email: solutions@anant.us LinkedIn: https://www.linkedin.com/company/anant/ Twitter: https://twitter.com/anantcorp Eventbrite: https://www.eventbrite.com/o/anant-10... Facebook: https://www.facebook.com/AnantCorp/ Join The Anant Team: https://www.careers.anant.us #cassandra #dataproc #datastax #apache #apachecassandra #dataengineering

1. Version 1.0
Google Dataproc and DataStax Astra
In Cassandra Lunch #115, Arpan Patel will discuss how to connect Google
Dataproc and DataStax Astra with a demo showing you what configurations
you will need to get the connection working!
Arpan Patel
Engineer @ Anant

2. Google Dataproc
● Fully managed and highly scalable service for running
Apache Spark, Apache Flink, Presto, and 30+ open source
tools and frameworks
○ Lets you take advantage of open source data tools
for batch processing, querying, streaming, and
machine learning
● Dataproc clusters are quick to start, scale, and shutdown,
with each of these operations taking 90 seconds or less,
on average
● Built-in integration with other Google Cloud Platform
services, such as BigQuery, Cloud Storage, Cloud
Bigtable, Cloud Logging, and Cloud Monitoring
● Can easily interact with clusters and Spark or Hadoop
jobs through the Google Cloud console, the Cloud SDK, or
the Dataproc REST API

3. Google Dataproc
● https://cloud.google.com/dataproc/docs/concepts
/versioning/dataproc-version-clusters
○ https://cloud.google.com/dataproc/docs/co
ncepts/versioning/dataproc-release-2.0
○ https://cloud.google.com/dataproc/docs/co
ncepts/versioning/dataproc-release-1.5
● Can run on GCE / GKE
● Dataproc Serverless for Spark

4. Google Dataproc + DataStax Astra
● Cluster Properties
○ dataproc:dataproc.conscrypt.provider.enable=false
● Job Properties
○ spark.jars.packages → com.datastax.spark:spark-cassandra-connector_2.12:3.1.0

5. Demo
● Spin up Dataproc Cluster on GCE
● Place JAR and Secure Connect Bundle into GCP Bucket
● Submit Dataproc Spark Job to read from DataStax Astra
● Check out spark-shell on master node
● Destroy Cluster

6. Strategy: Scalable Fast Data
Architecture: Cassandra, Spark, Kafka
Engineering: Node, Python, JVM,CLR
Operations: Cloud, Container
Rescue: Downtime!! I need help.
www.anant.us | solutions@anant.us | (855) 262-6826
3 Washington Circle, NW | Suite 301 | Washington, DC 20037