Tag “apache-spark”

As for an explanation please refer to the
definition of the tag “apache-spark” on StackOverflow.
Please note that not all tags used on this site have a corresponding tag on StackOverflow, but most technical tags should be defined.

Here is a list of all tags used on this site.

Below you will find pages that utilize the tag “apache-spark”:

Metadata operations in Spark

Derive case classes from data frames: use the schema to generate code

Submit Spark job to Kubernetes

How to deploy a Spark job on Kubernetes (minikube)

Writing type-safe Spark transformations

Type-safe transformations using the Dataset API, Frameless and Quill

Meta-level operations on a Spark data frame

Transform a data frame dynamically, i. e. trim all string columns of a data frame which is not known at compile-time
Projects

AWS datalake

Datalake on AWS using Step Functions and EMR
Projects

Measurement data pipeline

Streaming pipeline using Kafka and Flink on OpenShift
Projects

OsmAnd tracks

Transform OsmAnd tracks for visualization and analysis
Projects

Data ingestion pipeline to update product data

Streaming pipeline using Kafka on AWS
Projects

ETL pipeline for network infrastructure and customer service reporting

ETL pipeline using Spark/Impala/Kudu for MicroStrategy-based reporting
Projects

ETL pipeline into an Azure data lake for Power BI

Development of an ETL pipeline into an Azure data lake for Power BI reporting
Projects

ETL pipeline for big data reporting

Development of an ETL pipeline for big data reporting
Projects

Upload interface and ETL pipeline

Design of an upload interface, and development of an ETL pipeline
Projects

Online shop for IT supplies

Design and development of an online shop for IT supplies
Portfolio

Data Engineering

© Sabine Haag 2021. All rights reserved
Building ETL/ELT pipelines, batch jobs or streaming, optimized for throughput