Ref: 05072021mkd_1620404314

Snr Data Engineer - Data Streaming

USA, California

  • 180000 to 181000 USD
  • Engineer Role
  • Skills: Data Streaming / ETL / Big Data
  • Level: Senior

Job description

Snr Data Engineer - Data Streaming

05072021mkd_1620404314

My client a Cyber Security company with HQ based in Sunnyvale, CA and additional office in Cincinatti, OH are looking for an experienced Data Engineer to join the team. This is a permanent full time position and can be based in either location.

They are looking for a canidate with Data Streaming experience. Please see below full details.



Responsibilities

* Build and enhance an optimal real time data pipeline architecture using technologies such as Spark Streaming, Kafka Streams, Kafka Messaging, Elasticsearch and other Big Data technologies.
* Identify, design, and implement improvements in the data pipelines to achieve ever higher throughput and scalability.
* Work with data scientists and security experts to strive for greater functionality in our core products.
* Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.
* Work within an Agile workflow to organize tasks and collaborate with other team members (Jira).
* Work in a Test-Driven Development environment focused on producing reliable, well-documented production code.

Requirements

* Bachelor's degree or equivalent experience in Computer Science, or another relevant field.
* Expert level experience with programming languages Java/Scala/Kotlin etc.
* Minimum 4 years of experience in building and optimizing 'Big Data' data pipelines, architectures and data sets.
* Experience with message queuing, stream processing, and highly scalable 'big data' data stores.
* Experience with big data tools: Spark, Kafka, Elasticsearch, Hadoop etc.
* Experience with stream-processing systems: Flink, Spark-Streaming, Kafka Streams etc.
* Experience with Cloud services such as AWS EC2, EMR, EKS etc is a plus.
* Experience with working in Docker and Kubernetes is a plus.

If you are interested in this role please APPLY below or send me your resume to m.dawson@jeffersonfrank.com