Key Responsibilities

Data Engineering & Pipeline Development

• Design, develop, and optimize ETL pipelines using Databricks and a configuration-based framework.
• Build and maintain data models, data products, and backend functionalities leveraging Databricks native tools.
• Implement and manage data ingestion processes using AWS DMS (Database Migration Service).
• Work with Kafka for real-time data streaming and integration.

Cloud & Automation

• Utilize AWS cloud services for data storage, transformation, and orchestration.
• Implement CI/CD pipelines for code management and deployment using GitHub or similar tools.
• (Optional) Automate infrastructure and workflows using Terraform and AWS Lambda.

Collaboration & Delivery

• Collaborate with data modelers, business analysts, and cross-functional teams to understand business requirements and translate them into technical solutions.
• Contribute to team ceremonies and development cycles within the Agile/SAFe framework.
• Ensure scalability, performance, and compliance with regulatory standards within data solutions.

Technical Skills / Must-Haves

• 3–5+ years of professional experience in data engineering or software development.
• Strong proficiency in Python programming.
• Hands-on experience with Databricks as an ETL platform.
• Solid understanding of AWS Cloud Services (e.g., S3, Lambda, EC2, Glue, etc.).
• Experience with Kafka for data streaming and integration.
• Familiarity with AWS DMS or similar database replication tools.
• Practical experience with CI/CD pipelines and Git-based version control systems.

Preferred / Advantageous Skills

• Experience with Terraform and AWS Lambda for infrastructure automation.
• Background in pharmaceutical manufacturing or regulated environments.
• Working knowledge of Agile/SAFe development methodologies.