We are seeking a data engineer to join our data engineering & analytics team. The ideal candidate will be responsible for designing, building, and maintaining the infrastructure that enables our organization to store, process, and analyze large volumes of data. The successful candidate will work closely with our data scientists and analysts to ensure that they have access to clean, reliable data for their work. The engineer is required to have hands-on experience with implementing data analytics & data lake solutions on the Azure cloud platform.

Resposibilities

• Design and implement data pipelines that move data from various sources into a central repository or data lake.
• Develop and maintain data warehouses and data marts that enable data analysis and reporting.
• Create and maintain ETL (extract, transform, load) processes that clean and transform data for analysis.
• Build and manage big data platforms such as Hadoop, Spark, or NoSQL databases.
• Design and implement data security and privacy measures to protect sensitive data.
• Collaborate with data scientists and analysts to provide access to the data they need for their work.
• Automate data processing and data quality checks to ensure that data is accurate and reliable.
• Develop and maintain data documentation and data dictionaries.
• Evaluate new data technologies and tools and make recommendations to improve the organization’s data capabilities.
• Troubleshoot data infrastructure issues and optimize performance.
  
  

Qualifications

• Experience with big data technologies such as Hadoop, Spark, or NoSQL databases.
• Hands-on experience Architecting and delivering solutions using the Azure Data Analytics platform including Azure Databricks, Azure Cosmos DB, Azure Data Factory, Azure Logic Apps, Azure Functions, Azure Storage, Azure SQL Data Warehouse/Synapse, Azure Data Lake, Azure Search
• Strong programming skills in Python, Java, or another language.
• Experience with ETL (extract, transform, load) processes and data pipeline design.
• Familiarity with data warehousing and data modeling concepts. Experience with cloud-based data solutions such as AWS, Google Cloud Platform, or Azure.
• Strong problem-solving skills and the ability to work in a fast-paced environment.
• Excellent communication skills and the ability to work effectively in a team.