Ref: 6/11/2020_1604656753

GCP Big Data Engineer

England, London

  • Negotiable
  • Engineer Role
  • Skills: GCP Migration Projects - BigQuery - GCP
  • Level: Senior

Job Description

GCP Big Data Engineer

6/11/2020_1604656753

Job Description - We are looking for a contractor to help us complete the migration of our applications to use our new big data backend. We have been migrating from a self-hosted Presto cluster to Google's BigQuery, which will provide both us and our users with significant scale and speed benefits. We collect hundreds of millions of internet performance measurements each day in our big data stores, on behalf of our clients - the largest ISPs and governments from around the world.

We are at the point where both big data platforms are live and have data flowing into them in parallel, and we now need to migrate our applications to query BigQuery before we can decommission Presto. The primary focus is our customer-facing analytics API, which underpins our primary web application

Role & Responsibilities

* Migrate applications to query BigQuery
* Help us complete the migration of our applications to use our new big data backend
* Working heavily on BigQuery and making sure the migration is a smooth process

Skills & Qualifications

* You're familiar with developing web-facing APIs using modern PHP
* You have extensive experience with SQL, ideally with a big data platform
* You have a proven track record of designing a plan and executing on it, delivering quality work on schedule
* You are able to work independently and solve ambiguous problems