Data Engineer

Description

• To provide required support within guidance of set processes and procedures to ensure effective movement, collection, integration, storage and provisioning of data to build solutions and meet business objectives through understanding of business unit requirements and collaboration with relevant stakeholders

Duties

• Control expenditure and identify process improvements to contain and reduce costs
• Build working relationships across teams and functional lines to enhance work delivery, collaboration and innovation
• Acquire and collect data via ETL (Extract, Transform and Load) processes from source systems into the Reservoir and Enterprise Data warehouse, adhering to the appropriate data platform standards
• Architect, design, and develop robust data pipelines, ETL processes, and data integration solutions to enable efficient data movement and transformation
• Build scalable and optimized data warehouse solutions for analytics and business intelligence
• Drive performance tuning and optimization of databases, ensuring high availability, scalability, and efficiency
• Leverage cloud-based and on-premises data technologies (e.g., Azure Data Services, AWS, GCP, Hadoop) to meet enterprise data requirements
• Design and enforce data governance, security, and quality best practices across the data platform
• Collaborate with business stakeholders, data analysts, and engineers to align technical solutions with business needs
• Develop and implement data streaming solutions using Kafka, Apache Spark
• Establish unit and integration testing frameworks to ensure data integrity and system reliability
• Own and manage the user acceptance testing (UAT) process, including change control and production deployment readiness
• Provide technical leadership, mentorship, and best practices guidance to junior and mid-level engineers
• Stay up to date with emerging data engineering trends and recommend innovative solutions to improve data platform performance
• Conduct research on emerging technologies. Recommend and implement technologies that improve cost effectiveness and systems flexibility
• Execute remediation actions as agreed with client Install or update required system components
• Learn from the Subject Matter Expert on technical environment or tools for own area of expertise
• Grow online technical knowledge platform, identify best practices and develop guidelines for optimum usage of tools
• Assist and work on projects to roll out and support to team members

Requisites

• Bachelor’s degree in computer science, Information Systems, Data Engineering, or a related field
• 3 to 5 years of hands-on experience in Data Management, Maintenance, and Security, with at least 2+ years at a senior level leading complex data project
• Expertise in SQL Server and PostgreSQL, with a strong understanding of database performance tuning, indexing, and query optimization (advantageous)
• Strong proficiency in SQL-based ETL tools, data modeling, and big data technologies such as Apache Spark, Hadoop, and Kafka
• Experience with Big Data platforms (Vanilla Hadoop, Cloudera, or Hortonworks) and familiarity with data platforms like Teradata, Hadoop, and SQL
• Proficient in SQL, XML, Python, Java, and C# for data processing and automation
• Hands-on experience designing, building, and maintaining scalable and robust data pipelines with real-time stream processing frameworks,
• Experience working with cloud data platforms (AWS, Azure, or GCP)
• Solid knowledge of large-scale data architectures, governance, security, compliance, and best practices
• Strong understanding of GIT version control and agile methodologies
• Experience with DevOps, CI/CD pipelines, and automation in data engineering
• Demonstrated problem-solving, analytical thinking, and leadership abilities

Notes

• Only shortlisted applicants will be contacted