Senior Data Engineer

 responsibilities: 
Conjure reliable, scalable and cost-effective data lake architectures. 
Summon auto scalable on-demand Spark Clusters, fully managed queues or automation for on-prem solution. 
Helping our customers deliver great products by designing solid architecture and implementing it. 
Early adoption of future Data technologies and help the team absorb that knowledge. 
Creativity and motivation to help our customers’ business growth. 
Guide our customers to the world of Data engineering by writing blogs, doing meetups, teaching at workshops, and more!
Requirements: 
Field proven experience with Big Data technologies ( Hadoop and Spark, including setup, performance tuning and cost optimization) 
Experience working on AWS, GCP or other cloud providers for big data 
Hands on experience with Kafka or RabbitMQ in high scale production environments. 
Experience with Elasticsearch with clusters bigger than 10 data nodes, preferably not as log aggregation solution 
Experience with MongoDB clusters including configuration of sharding and replica sets. 
Experience with relational DBS – MySQL, PostgreSQL 
Experience with streaming technologies – Storm or Flink 
Experience with Git Version Control, working with git workflows 
Writing complex script automation (Bash, Python, Ruby, etc) 
Ability to effectively operate with flexibility in a fast-paced and constantly evolving team 
Customer focused & Problem-solving skills 
ast and efficient autodidact abilities and strong interest in new technologies
Advantages: 
Experience with security – understand GDPR, SOC2, PCI requirements and implications on data, experience with data encryption in transit and at rest 
Machine Learning
AIOps