A Tenth Revolution Group Company

Ref: DCJFIGCP01_1698309290

Senior/Lead Data Engineer - GCP - Salary to £70,000

England, London

Job description

Senior/Lead Data Engineer - GCP - Salary to £70,000


Senior/Lead Data Engineer - Salary to £70,000 - GCP.

We are looking for a GCP Lead Engineer to join our Data Insights and Analytics team. The role will be responsible for building new data pipelines, maintaining existing ETL pipelines, and optimizing data flows using the GCP cloud stack.

The ideal candidate will be an experienced resource with strong experience in building data pipelines, data products and platforms from scratch as well as maintaining and fine tuning existing setups. The data engineer will need to support Business Analysts, Technical team members and other stakeholders.

You must be self-directed and comfortable supporting the data needs of multiple teams, systems and products.

Skills we expect to bring:

* Advanced GCP knowledge, experience working on and migrating data products from on-prem to GCP
* Experience of leading teams to deliver projects
* Experience building and optimizing 'big data' data pipelines, architectures and data sets using GCP services
* Experience building real-time data pipelines including unstructured datasets
* Build processes supporting data transformation, data structures as well ETL, cost, space optimization
* Strong experience in Data Ingestion and Storage including BigQuery, GCS, Dataflow, Datastream, Dataform, Airflow
* Experience with Big data Tools: Hadoop Spark, Kafka
* Experience with Stream processing systems: Spark-streaming, Kafka
* Strong problem solving, quantitative and analytical abilities
* Excellent communication, collaboration and delegation

What Tools and Technologies do we expect you to know?

* SQL using Teradata, DB2, Oracle, etc.
* Cloud native data warehouse like BigQuery
* Python programming including Pandas, Numpy
* Good experience of ETL, Data Modelling, Data Warehousing, etc.
* GCP - BigQuery, GCS, Dataflow, Datastream, Dataform, Airflow