{"id":5880,"date":"2026-08-10T09:27:34","date_gmt":"2026-08-10T09:27:34","guid":{"rendered":"https:\/\/nokobox.com\/index.php\/item\/udemy-data-engineering-using-kafka-and-spark-structured-streaming-2022-10\/"},"modified":"2026-08-10T09:27:34","modified_gmt":"2026-08-10T09:27:34","slug":"udemy-data-engineering-using-kafka-and-spark-structured-streaming-2022-10","status":"publish","type":"digital_item","link":"https:\/\/nokobox.com\/index.php\/item\/udemy-data-engineering-using-kafka-and-spark-structured-streaming-2022-10\/","title":{"rendered":"Udemy \u2013 Data Engineering using Kafka and Spark Structured Streaming 2022-10"},"content":{"rendered":"<div class=\"w-post-elm post_content\">\n<h2><span style=\"vertical-align: inherit\">Description<\/span><\/h2>\n<p><span style=\"vertical-align: inherit\">Data Engineering course using Kafka and Spark Structured Streaming. In this course, you will learn how to build a streaming pipeline by integrating Kafka and Spark Structured Streaming. Let\u2019s go through the details of what is covered in the course. First, we need to have a suitable environment for building a streaming pipeline using Kafka and Spark Structured Streaming on Hadoop or any other distributed file system. As part of the course, you will start by setting up a self-supporting lab with all key components such as Hadoop, Hive, Spark and Kafka on a single-node Linux-based system. After setting up the environment, you will review the details of getting started with Kafka. As part of this process, you will create a Kafka topic, generate messages into the topic, and also consume messages from the topic.<\/span><\/p>\n<div data-purpose=\"safely-set-inner-html:description:description\">\n<h3 data-sourcepos=\"19:1-19:91\"><span style=\"vertical-align: inherit\">What you will learn in the Data Engineering using Kafka and Spark Structured Streaming course<\/span><\/h3>\n<ul>\n<li data-sourcepos=\"21:1-26:30\"><span style=\"vertical-align: inherit\">Setting up a self-supporting lab with Hadoop (HDFS and YARN), Hive, Spark and Kafka An overview of Kafka for building a streaming pipeline to receive data to Kafka topics using Kafka Connect using a file source Receiving data to HDFS using Kafka Connect with Using the HDFS 3 Connector Plugin Overview of Spark Structured Streaming to process data as part of a streaming pipeline Incremental data processing using Spark Structured Streaming using file source and file target Integrating Kafka and Spark Structured Streaming \u2013 Reading data from Kafka topics<\/span><\/li>\n<\/ul>\n<h3 data-sourcepos=\"29:1-29:26\"><span style=\"vertical-align: inherit\">This course is suitable for people who<\/span><\/h3>\n<ul>\n<li data-sourcepos=\"31:1-33:101\"><span style=\"vertical-align: inherit\">Experienced ETL developers who want to learn Kafka and Spark to build streaming pipelines Experienced PL\/SQL developers who want to learn Kafka and Spark to build streaming pipelines Beginner or experienced data engineers who want Kafka and Spark to build a streaming pipeline<\/span><\/li>\n<\/ul>\n<\/div>\n<h3><span style=\"vertical-align: inherit\">Data Engineering using Kafka and Spark Structured Streaming course specifications<\/span><\/h3>\n<ul>\n<li><span style=\"vertical-align: inherit\">Publisher:&nbsp; <\/span><a href=\"https:\/\/href.li\/?https:\/\/www.udemy.com\/course\/data-engineering-using-kafka-and-spark-structured-streaming\/?couponCode=ST22MT92324B\" target=\"_blank\" rel=\"noopener\"><span style=\"vertical-align: inherit\">Udemy<\/span><\/a><\/li>\n<li><span style=\"vertical-align: inherit\">Teacher:&nbsp; <\/span><a class=\"ud-btn ud-btn-large ud-btn-link ud-heading-md ud-text-sm ud-instructor-links\" href=\"https:\/\/downloadlynet.ir\/tag\/durga-viswanatha-raju-gadiraju\/\" data-position=\"1\"><span class=\"ud-btn-label\"><span style=\"vertical-align: inherit\">Durga Viswanatha Raju Gadiraju<\/span><\/span><\/a><\/li>\n<li><span style=\"vertical-align: inherit\">Training level: beginner to advanced<\/span><\/li>\n<li><span style=\"vertical-align: inherit\">Training duration: 9 hours and 26 minutes<\/span><\/li>\n<\/ul>\n<h3><span style=\"vertical-align: inherit\">Course topics on 12\/2023<\/span><\/h3>\n<h2>&nbsp;<img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-930481 size-full\" src=\"https:\/\/downloadly.ir\/wp-content\/uploads\/2024\/09\/Data-Engineering-using-Kafka-and-Spark-Structured-Streaming-2.jpg\" alt=\"Data Engineering using Kafka and Spark Structured Streaming\" width=\"788\" height=\"730\"><\/h2>\n<h3><span style=\"vertical-align: inherit\">Course prerequisites<\/span><\/h3>\n<div class=\"ud-block-list-item ud-block-list-item-small ud-block-list-item-tight ud-block-list-item-neutral ud-text-sm\">\n<ul>\n<li class=\"ud-block-list-item-content\" dir=\"ltr\" style=\"text-align: left\">Laptop with decent configuration<\/li>\n<li class=\"ud-block-list-item-content\" dir=\"ltr\" style=\"text-align: left\">Decent internet speed to watch the lessons<\/li>\n<li class=\"ud-block-list-item-content\" dir=\"ltr\" style=\"text-align: left\">Self Support lab (instructions will be provided as part of the course) or ITVersity labs<\/li>\n<li class=\"ud-block-list-item-content\" dir=\"ltr\" style=\"text-align: left\">Knowledge about Functional Programming (preferably Python or Scala)<\/li>\n<li class=\"ud-block-list-item-content\" dir=\"ltr\" style=\"text-align: left\">Knowledge or experience using Spark<\/li>\n<\/ul>\n<\/div>\n<h3><span style=\"vertical-align: inherit\">Course images<\/span><\/h3>\n<h2><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-930480 size-full\" src=\"https:\/\/downloadly.ir\/wp-content\/uploads\/2024\/09\/Data-Engineering-using-Kafka-and-Spark-Structured-Streaming.jpg\" alt=\"Data Engineering using Kafka and Spark Structured Streaming\" width=\"792\" height=\"291\"><\/h2>\n<h3><span style=\"vertical-align: inherit\">Sample video of the course<\/span><\/h3>\n<div style=\"width: 640px;\" class=\"wp-video\"><span class=\"mejs-offscreen\">Video Player<\/span><\/p>\n<div id=\"mep_0\" class=\"mejs-container mejs-container-keyboard-inactive wp-video-shortcode mejs-video\" tabindex=\"0\" role=\"application\" aria-label=\"Video Player\" style=\"width: 640px; height: 360px; min-width: 217px;\">\n<div class=\"mejs-inner\">\n<div class=\"mejs-mediaelement\"><mediaelementwrapper id=\"video-139701-1\"><video class=\"wp-video-shortcode\" id=\"video-139701-1_html5\" width=\"640\" height=\"360\" preload=\"metadata\" src=\"https:\/\/dl.downloadly.ir\/Files\/Elearning\/Sample\/Data_Engineering_using_Kafka_and_Spark_Structured_Streaming_Downloadly.ir.mp4?_=1\" style=\"width: 640px; height: 360px;\"><source type=\"video\/mp4\" src=\"https:\/\/dl.downloadly.ir\/Files\/Elearning\/Sample\/Data_Engineering_using_Kafka_and_Spark_Structured_Streaming_Downloadly.ir.mp4?_=1\"><a href=\"https:\/\/dl.downloadly.ir\/Files\/Elearning\/Sample\/Data_Engineering_using_Kafka_and_Spark_Structured_Streaming_Downloadly.ir.mp4?nocache=1786105512811\">https:\/\/dl.downloadly.ir\/Files\/Elearning\/Sample\/Data_Engineering_using_Kafka_and_Spark_Structured_Streaming_Downloadly.ir.mp4<\/a><\/video><\/mediaelementwrapper><\/div>\n<div class=\"mejs-layers\">\n<div class=\"mejs-poster mejs-layer\" style=\"display: none; width: 100%; height: 100%;\"><\/div>\n<div class=\"mejs-overlay mejs-layer\" style=\"display: none; width: 100%; height: 100%;\">\n<div class=\"mejs-overlay-loading\"><span class=\"mejs-overlay-loading-bg-img\"><\/span><\/div>\n<\/div>\n<div class=\"mejs-overlay mejs-layer\" style=\"display: none; width: 100%; height: 100%;\">\n<div class=\"mejs-overlay-error\"><\/div>\n<\/div>\n<div class=\"mejs-overlay mejs-layer mejs-overlay-play\" style=\"width: 100%; height: 100%;\">\n<div class=\"mejs-overlay-button\" role=\"button\" tabindex=\"0\" aria-label=\"Play\" aria-pressed=\"false\"><\/div>\n<\/div>\n<\/div>\n<div class=\"mejs-controls\">\n<div class=\"mejs-button mejs-playpause-button mejs-play\"><button type=\"button\" aria-controls=\"mep_0\" title=\"Play\" aria-label=\"Play\" tabindex=\"0\"><\/button><\/div>\n<div class=\"mejs-time mejs-currenttime-container\" role=\"timer\" aria-live=\"off\"><span class=\"mejs-currenttime\">00:00<\/span><\/div>\n<div class=\"mejs-time-rail\"><span class=\"mejs-time-total mejs-time-slider\" role=\"slider\" tabindex=\"0\" aria-label=\"Time Slider\" aria-valuemin=\"0\" aria-valuemax=\"0\" aria-valuenow=\"0\" aria-valuetext=\"00:00\"><span class=\"mejs-time-buffering\" style=\"display: none;\"><\/span><span class=\"mejs-time-loaded\"><\/span><span class=\"mejs-time-current\"><\/span><span class=\"mejs-time-hovered no-hover\"><\/span><span class=\"mejs-time-handle\"><span class=\"mejs-time-handle-content\"><\/span><\/span><span class=\"mejs-time-float\"><span class=\"mejs-time-float-current\">00:00<\/span><span class=\"mejs-time-float-corner\"><\/span><\/span><\/span><\/div>\n<div class=\"mejs-time mejs-duration-container\"><span class=\"mejs-duration\">00:00<\/span><\/div>\n<div class=\"mejs-button mejs-volume-button mejs-mute\"><button type=\"button\" aria-controls=\"mep_0\" title=\"Mute\" aria-label=\"Mute\" tabindex=\"0\"><\/button><a href=\"javascript:void(0);\" class=\"mejs-volume-slider\" aria-label=\"Volume Slider\" aria-valuemin=\"0\" aria-valuemax=\"100\" role=\"slider\" aria-orientation=\"vertical\"><span class=\"mejs-offscreen\">Use Up\/Down Arrow keys to increase or decrease volume.<\/span><\/p>\n<div class=\"mejs-volume-total\">\n<div class=\"mejs-volume-current\" style=\"bottom: 0px; height: 100%;\"><\/div>\n<div class=\"mejs-volume-handle\" style=\"bottom: 100%; margin-bottom: -3px;\"><\/div>\n<\/div>\n<p><\/a><\/div>\n<div class=\"mejs-button mejs-fullscreen-button\"><button type=\"button\" aria-controls=\"mep_0\" title=\"Fullscreen\" aria-label=\"Fullscreen\" tabindex=\"0\"><\/button><\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<div>\n<h3><span style=\"vertical-align: inherit\">Installation guide<\/span><\/h3>\n<p><span style=\"vertical-align: inherit\">After Extract, view with your favorite Player.<\/span><\/p>\n<p><span style=\"vertical-align: inherit\">Subtitle: English<\/span><\/p>\n<p><span style=\"vertical-align: inherit\">Quality: 720p<\/span><\/p>\n<\/div>\n<h3><span style=\"vertical-align: inherit\">download link<\/span><\/h3>\n<p><a href=\"https:\/\/dl2.downloadly.ir\/Files\/Elearning\/Udemy_Data_Engineering_using_Kafka_and_Spark_Structured_Streaming_2022-10.part1_Downloadly.ir.rar?nocache=1786105511\"><span style=\"vertical-align: inherit\">Download part 1 \u2013 1 GB<\/span><\/a><\/p>\n<p><a href=\"https:\/\/dl2.downloadly.ir\/Files\/Elearning\/Udemy_Data_Engineering_using_Kafka_and_Spark_Structured_Streaming_2022-10.part2_Downloadly.ir.rar?nocache=1786105511\"><span style=\"vertical-align: inherit\">Download part 2 \u2013 1 GB<\/span><\/a><\/p>\n<p><a href=\"https:\/\/dl2.downloadly.ir\/Files\/Elearning\/Udemy_Data_Engineering_using_Kafka_and_Spark_Structured_Streaming_2022-10.part3_Downloadly.ir.rar?nocache=1786105511\"><span style=\"vertical-align: inherit\">Download part 3 \u2013 1 GB<\/span><\/a><\/p>\n<p><a href=\"https:\/\/dl2.downloadly.ir\/Files\/Elearning\/Udemy_Data_Engineering_using_Kafka_and_Spark_Structured_Streaming_2022-10.part4_Downloadly.ir.rar?nocache=1786105511\"><span style=\"vertical-align: inherit\">Download part 4 \u2013 710 MB<\/span><\/a><\/p>\n<p><span style=\"vertical-align: inherit\">File(s) password: www.downloadly.ir<\/span><\/p>\n<h3><span style=\"vertical-align: inherit\">File size<\/span><\/h3>\n<p><span style=\"vertical-align: inherit\">3.7 GB<\/span><\/p>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>Description Data Engineering course using Kafka and Spark Structured Streaming. In this course, you will learn how to build a streaming pipeline by integrating <\/p>\n","protected":false},"author":1,"template":"","dgi_category":[10458],"dgi_tag":[57260,57261,57262,56492,57263,57264],"class_list":["post-5880","digital_item","type-digital_item","status-publish","has-post-thumbnail","hentry","dgi_category-video-tutorials","dgi_tag-course-data-engineering-using-kafka-and-spark-structured-streaming","dgi_tag-download-course-data-engineering-using-kafka-and-spark-structured-streaming","dgi_tag-download-data-engineering-using-kafka-and-spark-structured-streaming","dgi_tag-durga-viswanatha-raju-gadiraju","dgi_tag-free-data-engineering-using-kafka-and-spark-structured-streaming","dgi_tag-free-download-data-engineering-using-kafka-and-spark-structured-streaming"],"_links":{"self":[{"href":"https:\/\/nokobox.com\/index.php\/wp-json\/wp\/v2\/digital_item\/5880","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/nokobox.com\/index.php\/wp-json\/wp\/v2\/digital_item"}],"about":[{"href":"https:\/\/nokobox.com\/index.php\/wp-json\/wp\/v2\/types\/digital_item"}],"author":[{"embeddable":true,"href":"https:\/\/nokobox.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"version-history":[{"count":0,"href":"https:\/\/nokobox.com\/index.php\/wp-json\/wp\/v2\/digital_item\/5880\/revisions"}],"wp:attachment":[{"href":"https:\/\/nokobox.com\/index.php\/wp-json\/wp\/v2\/media?parent=5880"}],"wp:term":[{"taxonomy":"dgi_category","embeddable":true,"href":"https:\/\/nokobox.com\/index.php\/wp-json\/wp\/v2\/dgi_category?post=5880"},{"taxonomy":"dgi_tag","embeddable":true,"href":"https:\/\/nokobox.com\/index.php\/wp-json\/wp\/v2\/dgi_tag?post=5880"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}