Your current job search

25 search results

For Contract in London

AWS Senior Data Engineer - Apache Hudi / Apache Iceberg

England, London, City of London

  • £600 to £700 GBP
  • Engineer Role
  • Skills: AWS Senior Data Engineer - Apache Hudi / Apache Iceberg
  • Seniority: Senior

Job description

Hi,

Thanks for your time a moment ago. As discussed, if you are interested in the role, please can you send me your updated CV and candidate representation



Role - Senior AWS Data Engineer

Rate- £600-£700

Contract Length - 6-12 Months

Location - Remote (Must be able to work to US / Canadian Time Zones)

Start Date - ASAP



I am working for a large investment company based in London, that provides financial and technology solutions to the global investment industry. My client is looking for an AWS Data Engineer to work on an initial 6 months contract, and fully remote. A fantastic opportunity to be involved in end-to-end data management for cutting-edge Advanced Analytics and Data Science solutions. I am looking for someone to have experience with projects and in handling vast amounts of, and various types of data - working on database design and development, data integration and ingestion, and designing data and ELT architectures using a variety of tools and techniques.



Key Responsibilities

* Ingest from multiple sources and develop data processing pipelines
* Automate test and deployment of your infrastructure
* Work closely with business analysts, data engineers and product owners to gain a deeper understanding of business processes and data requirements
* Define and manage meta-data standards, data mappings and the data dictionary
* Define data quality rules, validation checks and automated cleansing methods for data pipelines
* Use data engineering and DevOps techniques on large datasets to gain business insight.
* Implement and test business logic, transformations and calculations to support data pipelines



Qualifications and Skills

* 5+ years experience with Python and Spark
* 5+ years of experience working with AWS and serverless technologies
* Proficiency with data lake-related technologies like Apache Hudi, Apache Iceberg or Databricks Delta Lake
* Experience using DevOps and automation techniques (CICD, infrastructure-as-code, monitoring tools)
* Proficiency in test automation (TDD, unit tests, Deequ)
* 3+ years of experience with data-related process controls, data mapping, and ETL processes and tools
* 3+ years of experience working with relational databases (eg. PostgreSQL) and NoSQL technologies
* Well-developed analytical skills, quantitative, and problem-solving skills
* Experience in the Fund Administration business is a strong plus
* Financial services technical Experience a plus