Introduction
Portfolio
- Focus on
- Cloud Engineering and DevOp is part of the job
- Team building, mentoring
Data Engineering
- As a Big Data Engineer
- Build ETL pipelines: collect, prepare, transform and store data
- To provide the source for further reporting, data analytics, visualization, data science or ML
- Batching and streaming solutions
- Near real-time streaming
- Incremental changes
- Solutions for fast changing data even on readonly filesystems
- Store in DWHs or data lakes
- Emphasis on throughput, distributed processing
Distributed Computing
- As a Distributed Computing expert
- Emphasis on latency, distributed state machines
- Low latency messaging applications
- Parallel, clustered, sharded
- Resilient
- Support for up- and downscaling
- Examples: Price calculation, availability, emergency calls, home automation
Architecture
- As an Architect
- Evaluate, select tools and frameworks
- Design the architecture (tools landscape and in the code)
- Design cache solutions
- Scalable and maintainable SW to reduce costs
- Focus on integration tests to ensure quality
- Specialized on performance optimizations to provide best performance
Cloud Engineering
- As a Cloud Engineer
- Let the batch or streaming jobs run on a cloud platform
DevOp
- As a DevOp
- Automation, scripting
- Provide development environments (docker-compose)
Timeline
➜ Timeline