Azure Databricks Engineer

📍 Location: Bangalore / Hyderabad / Pune
🕒 Experience: 5+ Years
⚡ Employment Type: Full-Time / Contract

🌟 Role Overview

We are looking for a highly skilled Azure Databricks Engineer with strong expertise in modern data engineering, big data processing, and distributed systems. The ideal candidate should have hands-on experience in building scalable ETL/ELT pipelines, optimizing Spark workloads, and working with enterprise-grade Databricks environments.

🔹 Key Responsibilities

• Design, develop, and maintain scalable data pipelines using Azure Databricks.
• Build and optimize ETL/ELT workflows for large-scale data processing.
• Work extensively with Apache Spark (PySpark & Spark SQL) for distributed data engineering.
• Implement Delta Lake architecture and optimize performance using Liquid Clustering and skew handling techniques.
• Develop reusable frameworks and automation solutions using Python.
• Optimize SQL queries, data models, and analytical workloads.
• Utilize Databricks Genie for engineering productivity and workflow acceleration.
• Collaborate with DevOps teams to implement CI/CD and Git-based deployment workflows.
• Work with orchestration and scheduling tools for automated pipeline execution.
• Ensure data quality, scalability, security, and reliability across platforms.

✅ Required Skills

• 5+ years of experience in Data Engineering or related domains.
• Strong hands-on experience with Azure Databricks.
• Expertise in:
  • Databricks Notebooks
  • Delta Lake
  • Job Orchestration
  • Data Skewing
  • Liquid Clustering
• Strong knowledge of Apache Spark:
  • PySpark
  • Spark SQL
  • Spark Optimization Techniques
• Strong Python programming skills.
• Advanced SQL knowledge including:
  • Complex Queries
  • Performance Tuning
  • Analytical Functions
• Experience with Databricks Genie.
• Strong understanding of CI/CD pipelines and Git workflows.
• Experience in data modeling and ETL/ELT pipeline design.
• Knowledge of automation frameworks and scheduling tools.
• Good understanding of distributed systems and big data architecture.

🌟 Preferred Qualifications

• Experience with cloud platforms like Microsoft Azure.
• Exposure to enterprise-scale data lake architecture.
• Strong problem-solving and analytical skills.
• Excellent communication and collaboration abilities.