Grade Level (for internal use):

10

Team Overview

Our Data and AI Engineering Team consists of skilled professionals focused on creating innovative and scalable data solutions. We specialise in cloud technologies, especially AWS and Snowflake, and prioritise data security and governance. Collaboration is key, as we work closely with Data Scientists and ML Engineers to advance impactful data projects. We encourage continuous learning and are actively adopting new technologies like Generative AI models and advanced machine learning tools to improve our data platforms and achieve outstanding outcomes.

Role Impact

In this role, you will design and implement scalable data architectures aligned with strategic goals, especially those supporting all stages of the Machine Learning (ML) and Generative AI (GenAI) lifecycle. You will apply your expertise in Snowflake, Python, and cloud technologies to maintain strong data security and integrity. Your work will drive innovation by developing, optimising, and maintaining production-level data pipelines that support both analytical and predictive modelling, enhancing data processing efficiency and enabling data-driven decision-making across the organisation.

Main Contributions

Responsibilities: (Dual Emphasis on Data Engineering & Data Science Enablement)

• ML Data Pipeline Development: Create and uphold scalable, reliable, and secure data infrastructure essential for Machine Learning model training, validation, and inference in production settings.
• Feature Store Collaboration: Partner with Data Scientists to design, build, and maintain a centralized Feature Store, ensuring consistency, reusability, and low-latency access to features for model training and deployment.
• GenAI Data Handling: Develop and execute data ingestion and processing workflows for text and unstructured data, including methods for vectorisation and populating vector databases to support Retrieval-Augmented Generation (RAG) architectures.
• Core Data Engineering: Build and maintain robust ELT/ETL processes within Snowflake and AWS to integrate diverse, large-scale data sources, ensuring excellent data governance, quality, and performance for general business intelligence.
• MLOps Partnership: Collaborate closely with ML Engineers to operationalise data flows to and from MLOps platforms (e.g., SageMaker, MLflow), focusing on automating pipelines and managing data versioning.
• Query and Compute Efficiency: Optimize data storage and computing resources across the platform to speed up data science experiments and model training workloads, while managing costs effectively.
• Data Security and Compliance: Implement advanced security measures to safeguard sensitive data used in AI datasets, maintaining data integrity and confidentiality in line with governance standards.