Your Journey to Master Structured Streaming Starts Here

You must be tired of hearing this, but data is everywhere! Regardless of your job or industry, the amount you are being surrounded by data is absolutely unfathomable. And with such volumes of data, there has been a global demand for scaling up real-time data processing systems. That is where Apache Spark Streaming comes to the rescue.

We hope this course will help you leverage Spark Streaming to tackle new problems produced by big data efficiently.

In this course, we will learn about Streaming Data, applications and challenges of traditional streaming systems, and how Spark Streaming helps us in dealing with it. We will be covering 

Batch Processing, Distributed Systems, traditional stream processes and its challenges, DStreams, stateless and stateful transformation, and Structured Streaming.

The market for Big Data Analytics is growing tremendously across the world and such a strong growth pattern followed by market demand is a great opportunity for all IT Professionals. Here are a few Professional IT groups, that are continuously enjoying the benefits and perks of moving into the Big Data domain.

  • Developers and Architects

  • BI /ETL/DW Professionals

  • Senior IT Professionals

  • Testing Professionals

  • Mainframe Professionals

  • Freshers

  • Big Data Enthusiasts

  • Software Architects, Engineers, and Developers

  • Data Scientists and Analytics Professionals

Pre-Requisites

  • Good to have knowledge of any programming language like Python
  •  Good understanding of Hadoop
  • Good understand of Apache Spark

Key Takeaways from the Spark Streaming Course

  • Understanding the streaming data applications and challenges
  • Traditional Stream Processing Systems and Challenges
  • Dstreams
  • Stateful and Stateless types of Operation
  • Structured Streaming
Key Takeaways from the Spark Streaming Course

What do I need to start the Spark Streaming Course?

  • A working laptop/desktop

  • A working Internet connection

  • Basic understanding of Python and Knowledge of SQL, Hadoop and Apache Spark

Course curriculum

  • 1
    Course Handouts
    • Course Handouts
  • 2
    Module 1 - Introduction to Streaming
    • Introduction to Streaming
    • What is Data Streaming
    • Quiz : What is Data Streaming
    • What is Batch Processing
    • Quiz : What Is Batch Processing
    • Issues with Traditional Systems
    • Quiz : Issues With Traditional Systems
    • What are Distributed Systems
    • Quiz : What Are Distributed Systems
    • Challenges with Stream Processing
    • Quiz : Challenges With Stream Processing
  • 3
    Module 2 -Introduction to Apache Spark
    • Spark Ecosystem
    • What is Apache Spark
  • 4
    Module 3 - Spark Streaming Execution Model
    • Introduction to Spark Streaming
    • Quiz : Introduction To Spark Streaming
    • Traditional Stream Processing Systems
    • Challenges Of Traditional Stream Processing Systems
    • Quiz : Challenges Of Traditional Stream Processing Systems
    • What Are Dstreams
    • Quiz : What Are Dstreams
    • Architecture Of Spark Streaming- Dstreams
    • Quiz : Architecture Of Spark Streaming- Dstreams
    • Benefits Of Dstream Procesing
    • Quiz : Benefits Of Dstream Procesing
  • 5
    Module 4 - Spark Streaming Basics
    • Streaming Context
    • Quiz : Streaming Context
    • Spark Streaming Data Sources
    • Quiz : Spark Streaming Data Sources
    • Types Of Operations
    • Stateless Transformations
    • Quiz : Stateless Transformations
    • Basics Of Transformations Demo 1
    • Basics Of Transformations Demo 2
    • Transform Operation
    • Transformation Operation
    • Quiz : Transformation Operation
    • Foreachrdd Demo
    • Output Operations On Dstreams
    • Output Operations On Dstreams Demo
    • Quiz : Output Operations On Dstreams Demo
  • 6
    Module 5 - Advanced Concepts
    • Stateful Transformations
    • Quiz : Stateful Transformations
    • Update state by key
    • Update State By Key Nb
    • Windowed Operations
    • Quiz : Updatestatebykey
    • Window Function
    • Quiz : Window Function
    • Window Function Nb
    • Count by window
    • Count By Window Nb
    • Reduce by key and window
    • Reduce By Key And Window Nb
    • Count by value and window
    • Count By Value And Window Nb
    • Fault Tolerance
    • Checkpointing Demo
    • Quiz : Checkpointing Demo
  • 7
    Module 6 - Structured Streaming
    • Structured Streaming
    • Quiz : Structured Streaming
    • Structured Streaming Demo
    • Structured Streaming Example
    • Quiz : Structured Streaming Example
    • Output Modes
    • Input Sources
    • Quiz : Input Sources
    • Basic Operations
    • Operations On Streaming Dataframes
    • Window Operations On Time Event
    • Watermarking
    • Window Operations-Watermarking-Demo
    • Output Sinks
    • Quiz : Output Sinks