We are looking for an experienced Databricks Technical Lead to join our Data Engineering team in Bangalore (Whitefield).

🔹Role: Databricks Technical Lead
🔹Experience: 9+ Years
🔹Location: Bangalore – Whitefield
🔹Requirement: Immediate / Short Notice Joiners preferred

Required Skills

• Strong hands-on experience with Azure Databricks – Notebooks, Delta Lake, Jobs & Orchestration
• Deep expertise in Apache Spark, PySpark & Spark SQL
• Strong Python programming skills for data processing & automation
• Strong SQL – complex queries, analytical functions & performance tuning
• Knowledge of Databricks Genie and its application in engineering workflows
• Strong experience with CI/CD & Git-based development
• Expertise in Data Modeling & ETL/ELT pipeline design
• Experience with automation frameworks & scheduling tools
• Strong understanding of Distributed Systems & Big Data concepts

Good to Have

• Experience with Microsoft Fabric.
• Experience with Power BI and analytical workloads.
• Knowledge of Azure Data Lake Storage (ADLS Gen2).
• Experience with streaming technologies such as Kafka / Event Hubs.
• Experience implementing DevOps and Infrastructure-as-Code practices.
• Knowledge of data security, governance and compliance.
• Experience working with large-scale enterprise or manufacturing data platforms.
  
  
  Lead Databricks Development: Lead the design, development, and implementation of scalable data engineering solutions using Databricks, ensuring high-quality and reliable data pipelines for enterprise and semiconductor/manufacturing use cases.
• Technical Leadership: Provide technical leadership to a team of data engineers, guide day-to-day development activities, conduct code/design reviews, resolve technical challenges, and ensure adherence to engineering best practices.
• Strong SQL Expertise: Demonstrate strong hands-on expertise in SQL, including complex joins, CTEs, subqueries, window functions, aggregations, data transformation, performance tuning, and troubleshooting of complex queries.
• Databricks & Data Engineering: Strong working knowledge of Databricks, PySpark, Delta Lake, notebooks, workflows/jobs, data transformation, and ETL/ELT concepts. Should be comfortable remaining hands-on while leading the team.
• Client & Stakeholder Management: Act as a key technical point of contact for client stakeholders. Independently participate in discussions with business and technical teams, understand their requirements, ask the right questions, and provide clear technical perspectives and solutions.
• Requirement Gathering & Translation: Work closely with business stakeholders to understand data requirements and translate business problems into technical requirements, data models, transformation logic, and actionable development tasks for the engineering team.
• Communication & Confidence: Excellent verbal and written communication skills with the confidence to lead discussions with senior client stakeholders. Should be able to explain complex technical concepts in a simple and business-friendly manner and confidently challenge or clarify requirements when required.
• Semiconductor / Manufacturing Data: Experience working with semiconductor, manufacturing, supply chain, equipment, production, quality, or other industrial data is preferred. Exposure to high-volume, complex, and time-series manufacturing datasets will be an advantage.
• Data Quality & Performance: Ensure data quality, reliability, scalability, and performance across pipelines. Troubleshoot data issues, identify root causes, and work with relevant teams to implement sustainable solutions.
• Ownership & Delivery: Take end-to-end ownership of assigned data engineering initiatives, including requirement clarification, technical design, task allocation, development oversight, stakeholder communication, delivery tracking, and production support. Should be capable of managing multiple priorities and coordinating a sizeable engineering team.