✨ About The Role
- The role involves developing new features for an event-based processing framework to handle large volumes of data efficiently.
- The candidate will be responsible for creating and maintaining an in-house Data Lake that consolidates various data types for the organization.
- Enhancing system observability through monitoring and alerting tools is a key responsibility.
- The position requires collaboration with multiple teams to gather requirements and design robust architectures.
- The job entails tackling complex challenges related to data processing at the scale of hundreds of petabytes.
âš¡ Requirements
- The ideal candidate will have over 5 years of industry experience in software engineering, demonstrating a strong background in developing complex software solutions.
- Proficiency in Python and Spark is essential, particularly for large-scale data processing tasks.
- Experience with cloud platforms such as AWS and Databricks is crucial for this role.
- Familiarity with Apache Airflow and a working knowledge of C++ will be beneficial for the candidate.
- The successful candidate will be a collaborative team player, able to work effectively with cross-functional teams including software engineers and data engineers.