AWS Data Engineer - Apache Hudi

Role - Senior AWS Data Engineer

IR35 - Outside IR35

Contract Length - 6 months (View to extend further)

Location - Fully Remote

Start Date- January



I am working for a large investment company based in London, that provides financial and technology solutions to the global investment industry. My client is looking for an AWS Data Engineer to work on an initial 6 months contract, and fully remote. A fantastic opportunity to be involved in end-to-end data management for cutting-edge Advanced Analytics and Data Science solutions. I am looking for someone to have experience with projects and in handling vast amounts of, and various types of data - working on database design and development, data integration and ingestion, and designing data and ELT architectures using a variety of tools and techniques.



Key Responsibilities

* Ingest from multiple sources and develop data processing pipelines
* Work closely with business analysts, data engineers and product owners to gain a deeper understanding of business processes and data requirements
* Define and manage meta-data standards, data mappings and the data dictionary
* Define data quality rules, validation checks and automated cleansing methods for data pipelines
* Implement and test business logic, transformations and calculations to support data pipelines

Qualifications and Skills

* 5+ years' experience with Python and Spark
* 5+ years' experience working with AWS and serverless technologies
* Proficiency with data lake-related technologies like Apache Hudi, Apache Iceberg or Databricks Delta Lake
* 3+ years' experience with data-related process controls, data mapping, and ETL processes and tools
* 3+ years' experience working with relational databases (eg. PostgreSQL) and NoSQL technologies
* Well-developed analytical skills, quantitative, and problem-solving skills
* Experience in the Fund Administration business is a strong plus
* Financial services technical Experience a plus



Regards

Ollie