Ref: 121020_1607626827

Senior Data Engineer

USA, California

  • Negotiable
  • Engineer Role
  • Skills: data, kubernetes, aws, data engineer, airflow, kafka, redshift, developer
  • Level: Senior

Job description

Senior Data Engineer



To be considered for these roles, you must hold either a Green Card or US Citizenship.

The Role

* Developing streaming pipeline using MQTT, Kafka, Spark Structure Streaming
* Hands-on design and develop streaming and IoT data pipelines.
* Design and implement Apache Airflow and other dependency enforcement and scheduling tools.
* Python scripting for automation and application development
* Orchestrate and monitor pipelines using Prometheus and Kubernetes
* Deploy and maintain streaming jobs in CI/CD and relevant tools.
* Hands-on data modeling and data warehousing
* Scripting with Apache Spark and data frame.
* Deploy solution using AWS, S3, Redshift and Docker/Kubernetes
* Develop storage and retrieval system using Presto and Parquet/ORC


* Significant experience in Data Engineering and Business Intelligence.
* Expert in tools such as Apache Spark, Apache Airflow, Presto
* Experienced in containerization, including Docker and Kubernetes
* Proficient in IoT tools such as MQTT, Kafka, Spark
* Proficient with AWS, S3, Redshift
* Proficient with Apache Spark and data frame.
* Experience with Presto and Parquet/ORC
* Expert in design and implement reliable, scalable, and performant distributed systems and data pipelines
* Extensive programming and software engineering experience, especially in Java, Python,
* Experience with Columnar database such as Redshift, Vertica

To be considered for this role, please send me a LinkedIn message or call me at 469-609-3754 or email me at To be considered for these roles, you must hold either a Green Card or US Citizenship.

Jefferson Frank is the global leader for Amazon Web Services recruitment, advertising more AWS roles than any other agency. We deal with both AWS Partners & End Users throughout North America. By specializing solely in placing candidates in the AWS market we have built relationships with most of the key employers in North America and have a complete understanding of where the best opportunities and AWS jobs are.

Jefferson Frank is acting as an Employment Agency in relation to this vacancy.