• Design, build, and optimize data pipelines for collecting, transforming, and loading data from various sources into our data warehouse and data lake.
  • Implement and maintain a real-time data streaming infrastructure using Kafka/MSK or DMS.
  • Develop and maintain orchestration and scheduling tools to automate the execution of data pipelines and machine learning workflows.
  • Collaborate with the data team to design and implement scalable and efficient machine learning architectures and workflows.
  • Configure and maintain infrastructure elements using configuration tools such as Ansible and Chef.
  • Implement monitoring for all aspects of the data infrastructure and improve the current monitoring of the core product infrastructure.
  • Ensure continuous documentation of current processes (configuration standards, setups, diagrams, etc.).
  • Participate in on-call rotations.
  • 5+ years of experience in building and maintaining large-scale data infrastructure, with a focus on machine learning, data pipelines, orchestration, and real-time streaming.
  • Strong experience with AWS cloud infrastructure and related services, such as S3, EC2, EMR, and Lambda.
  • Experience in designing and implementing scalable machine learning architectures and workflows.
  • Experience with real-time data streaming technologies, such as Kafka and MSK or DMS.
  • Strong understanding of data orchestration and scheduling tools, such as Prefect and Airflow.
  • Experience with containerization technologies, such as Docker and Kubernetes.
  • Strong knowledge of SQL and NoSQL databases, such as MySQL, Postgres, and MongoDB.
  • Experience in implementing infrastructure as code using tools like Terraform and Cloud Formation.
  • Experience with monitoring and logging tools, such as Datadog and ELK.
  • Experience in configuring and maintaining infrastructure elements using configuration tools such as Ansible and Chef.

Aviada ranked #1 in Best Place to Code

Best Place to Code® is operated by Software Guru. Identifies and awards companies that make efforts to offer the best work conditions to IT Professionals.
Best Place to Code Logo