Ref: 23/08/2021OH_1629727240

AWS Data Engineer

Scotland, Edinburgh

Job description

AWS Data Engineer

23/08/2021OH_1629727240

Good Afternoon,

My client is currently recruiting for the below. This is a very exciting role, with the chance to work remotely and on-site, all Outside IR35. My client has significantly grown over the last 6 months in the data space and wants to continue to increase their headcount for this new project.

Role - Data Engineer

Location - Edinburgh (Remote 3 days, on-site 2 days)

Rate - Competitive (Outside IR35)

Contract Length - 4 Months (with a view to extending)



The Role

My clients Data Team is working hard to collect, transform, store, analyse and present data from their various business teams. The teams mainly work on a backlog of data sources and integrations with third-party vendors and suppliers to create a data warehouse within the AWS ecosystem.

Responsibilities

* You will be helping my client increase the pace of delivery, managing continuous integration and deployment tools.
* Secondly, you will be helping define infrastructure as code, and helping them with estimating system requirements, as well as monitoring smooth operations.
* You will work with external and internal stakeholders as well as team members when capturing and documenting IT requirements.
* You will show attention to detail and show strong problem-solving skills when helping to spec out projects and infrastructure.
* You will help with the development of data pipelines, setting up data lake and data warehousing infrastructure.
* You will help to create new integrations with third-party providers and external data sources.
* You will help with the development of dashboards and other reporting tools.
* You will be comfortable working with Agile principles and development methodologies.



Skills Required

* Excellent knowledge of Python
* You will know common data query languages for relational and document databases.
* Experience setting up data pipelines, data extraction, transformation and load processes.
* Experience working with version control tools, understand the importance of code reviews and coding standards).
* Experience with Messaging queues and event-driven development (eg. Rabbit MQ, SQS).
* Experience developing REST APIs.
* Understanding of common cloud metrics and alerting within AWS.



Desirable

* Experience with Apache Spark and Databricks.
* Experience using the AWS ecosystem, especially VPC, EC2, RDS and Redshift and the AWS Cloud Development Kit, Lambda.
* Experience working with continuous integration, CI/CD processes.
* Experience with orchestration tools (eg. Cloud formation, Kubernetes, etc).
* Experience with workflow management tools (eg. Airflow, Luigi).

Please note this role will require travelling onsite 2 days a week!

If you would like to hear more please drop me your CV and we can have a chat.

Kind Regards
Ollie