We are looking for a talented Data Engineer with a passion for data and complex problems.
As a Data Engineer, you will be part of the data organization that design, maintain and develop our data platform, supporting large scale data collection and ingestion operation, using cutting edge data tools and large-scale data on AWS and GCP cloud based data infrastructure.
You will design and create the data pipeline to meet various use-cases and stakeholders including data analysts, data scientists and algorithm developers’ data needs and data related tasks.
- You will create and automate data pipeline including data extraction, validation & anomaly detection, indexing and other data related tasks.
- You will get to learn new methods and innovate new ideas.
- You will follow your project as it is implemented, and fine tune it for various needs.
- Contribute to a wide variety of projects using a range of technologies and tools.
If you are someone that thrives in a fast-paced environment where being self-directed, creative and determined are requirement, we would love for you to join us.
- Bsc in Mathematics/Statistics/Physics/Computer Science or a related field
- Over 3+ years of hands-on experience programming in Python
- Over 2+ years of industry experience working with large data sets
- Experience with working with unstructured data, complex data sets and data modeling
- Knowledge in working with SQL and NoSql databases, performance analysis and optimization
- Experience in ETL and automation, preferably using airflow and spark
- Experience with Kubernetes and kubeflow - Advantage
- Solution orientation and "can do" attitude - with a desire to tackle complex problems
- A team player, excellent collaboration skills