Information Technology & Software
20 Jun
Lead Data Engineer
LOCATION- MUMBAI/ NAVI MUMBAI
We are seeking a highly skilled and delivery-focused Lead Data Engineer to design and build enterprise-grade data platforms and applications under the Durables model. This role will be responsible for leading end-to-end new implementation and development initiatives, building scalable data pipelines, and architecting cloud-native solutions on AWS.
The ideal candidate will act as a key technical leader for new builds, working closely with client stakeholders to translate business requirements into robust data engineering solutions. This is a hands-on leadership role requiring strong design capabilities, cloud expertise, and experience in delivering large-scale data engineering projects in Agile environments.
Key Responsibilities
Solution Design & Implementation
Lead end-to-end design and development of scalable, high-performance data engineering solutions.
Architect and implement new data pipelines, data models, and cloud-native data platforms.
Design modular, reusable, and future-ready data frameworks for enterprise use cases.
Work closely with stakeholders to understand requirements and convert them into technical designs.
Technical Leadership
Provide architectural direction for new data platform development and modernization initiatives.
Define best practices for ETL/ELT design, data modeling, and cloud-native data processing.
Build and optimize data pipelines using AWS services (Glue, Lambda, Redshift, S3, EMR, Athena, etc.).
Leverage PySpark, SQL, Python, and distributed processing frameworks for large-scale data processing.
Drive engineering excellence through code quality, performance tuning, and scalable design patterns.
Agile Delivery & Execution
Lead sprint planning, estimation, and delivery of development milestones in Agile environments.
Ensure timely delivery of high-quality data engineering components and features.
Collaborate with cross-functional teams including product, analytics, and infrastructure teams.
Client Engagement & Solutioning
Act as a technical advisor for clients during solution design and implementation phases.
Present architecture approaches, design decisions, and trade-offs to stakeholders.
Proactively recommend innovative solutions to improve scalability, performance, and business value.
Team Leadership & Mentoring
Lead and mentor a team of data engineers, ensuring strong technical growth and delivery discipline.
Drive task allocation, design reviews, and technical guidance across the team.
Build a strong engineering culture focused on ownership and continuous improvement.
Process, Quality & Governance
Establish CI/CD pipelines, version control practices, and automated testing frameworks.
Ensure adherence to coding standards, documentation practices, and design governance.
Identify risks and dependencies early and implement mitigation strategies.
Required Skills & Qualifications
6+ years of experience in Data Engineering / Data Platform Development / Application Development.
Strong hands-on expertise in AWS cloud services (Glue, Lambda, Redshift, S3, Athena, EMR, CloudWatch).
Proficiency in PySpark, SQL, and Python for large-scale data processing.
Strong experience in designing and building end-to-end data pipelines and data platforms.
Deep understanding of distributed data processing and performance optimization.
Proven experience leading engineering teams (3+ years), ideally in development or implementation projects.
Strong client-facing and communication skills.
Experience with Agile delivery methodologies.
Familiarity with tools like JIRA, Confluence, Git, Jenkins, or CI/CD pipelines.
Good to Have
Experience in greenfield data platform development or AWS migration projects.
Exposure to legacy-to-cloud transformation programs.
Strong understanding of data modeling concepts and Hive/partitioning strategies.
Experience building real-time or near real-time data pipelines.