JOB REQUIREMENTS:

We are seeking an experienced Data Engineer with 4+ years of relevant experience to join our dynamic team. The ideal candidate will be responsible for creating, configuring, and implementing data warehouses using DBT(Data Build Tool). Additionally, the candidate must possess strong skills in writing complex and advanced-level SQL queries, along with proficiency in Python and PySpark. Expertise in AWS Cloud services, particularly in AWSGlue, Lambda functions, and Amazon Redshift, is essential for this role. The successful candidate will be responsible for managing and optimizing Redshift clusters and ensuring efficient data storage and retrieval.

ROLE AND RESPONSIBILITIES:

•  Design, create, and maintain data warehouses using DBT, ensuring optimal performance and scalability.
• Implement DBT Macros to automate and streamline data transformation processes, enhancing efficiency and reliability.
• Utilize DBT Orchestration for scheduling and orchestrating complex data workflows, ensuring timely and accurate data delivery.
• Write and optimize complex SQL queries to extract, transform, and load (ETL) data from various sources into the data warehouse.
• Develop and implement data pipelines using Python and PySpark to process and analyze large datasets.
• Collaborate with cross-functional teams to understand data requirements and translate them into technical solutions.
•  Implement best practices for databases, data modeling, schema design, and data architecture to ensure data integrity and consistency.
• Must have hands on experience of transferring data from data lake to data warehouse with best practices.
• Manage and optimize Amazon Redshift clusters, including provisioning, monitoring, and tuning for performance.
•  Develop and manage stored procedures in Amazon Redshift to encapsulate and optimize complex SQLlogic, improving performance and maintainability.
• Utilize AWS Glue for data cataloging, ETL job orchestration, and data transformation tasks.
•  Develop and maintain Lambda functions for server less data processing and automation tasks.
  

EDUCATION AND EXPERIENCE REQUIRED:

• Bachelor’s degree in computer science, Engineering, or related field.
•  4+ years of experience in a Data Engineering role, with a focus on building and maintaining data warehouses.
• Practical DBT expertise in designing, automating, and orchestrating data warehouses and transformations for optimal performance and scalability.
•  Strong proficiency in SQL, with experience writing complex queries and optimizing query performance.
•  Hands-on experience with Python and PySpark for data processing and analysis.
•  Hand-On experience of Amazon Redshift, AWS Cloud services, particularly AWS Glue, Lambda functions.
• Experience with data modeling, schema design, and ETL processes.