Demo: Spark 3 Dynamic Partition Pruning
Watch this demo on dynamic partition pruning. Spark takes any query and converts the query into logical plan and later physical plan of the query. Dynamic partition pruning reduces the scanned data limiting it to only needed data and avoids scanning or reading of unwanted data. Using this feature will reduce the number of I/O reads and increase the speed and time of queries. This optimization is applied at run time and works when two or more fact tables are joined to refer dimension tables. The partitions are pruned at execution time when the filters are applied on the dimension table. The examples below shows how the tables are created and later joined.
Learn more here - https://blog.cloudera.com/build-on-yo...
Watch video Demo: Spark 3 Dynamic Partition Pruning online, duration hours minute second in high quality that is uploaded to the channel Cloudera, Inc. 11 September 2020. Share the link to the video on social media so that your subscribers and friends will also watch this video. This video clip has been viewed 890 times and liked it 5 visitors.
