As our new Data Engineer, you will join a collaborative and skilled organization working with data at scale to enable automation of real business cases through data-driven and Machine Learning-based intelligence.
We are hiring two Data Engineers for two closely collaborating teams with different focus areas:
-
the Data Science team, working directly with forecasting, analytics, and ML-based decision support
-
the Data Streaming Platform (DSP) team, a newly formed team building and operating a central platform for real-time and event-driven data across the organization
We work in a Scrum setup and have in recent years been very successful in forecasting both electricity prices and electricity consumption, while at the same time scaling our real-time data capabilities. Your work will have direct and measurable impact.
**What will you do? **
-
Design, build, and operate batch and real-time data pipelines using Azure, Databricks, and Python
-
Ingest and process external and internal data such as weather data, market prices, grid telemetry, and event streams
-
Work with streaming technologies including Kafka, Spark Structured Streaming, and Flink
-
Ensure high data quality through validation, monitoring, and lineage
-
Develop and improve data workflows, standards, and reliability in production systems
Dependent on your area of expertise and interest, you will work in either the Data Science- or Data Streaming Platform team
Data Science team focus:
-
Enable forecasting, analytics, and ML use cases with high-quality, well-modelled data
-
Collaborate closely with Data Scientists, ML Engineers, and other engineering teams
DSP team focus:
-
Build Kafka based central streaming data platform from scratch to enable traders, analysts and asset managers to publish and consume live and historical data at scale.
-
Operate and continuously enhance the components once the platform is in production.
**Location **
Katowice or remotely in Poland