Lead Architect - Azure DBX

It's fun to work in a company where people truly BELIEVE in what they are doing!

We're committed to bringing passion and customer focus to the business.

Senior Azure Databricks Data Engineer

Fractal is a strategic AI partner to Fortune 500 companies with a vision to power every human decision in the enterprise. Fractal is building a world where individual choices, freedom, and diversity are the greatest assets; an ecosystem where human imagination is at the heart of every decision. Where no possibility is written off, only challenged to get better. We believe that a true Fractalite is the one who empowers imagination with intelligence. Fractal has been featured as a Great Place to Work by The Economic Times in partnership with the Great Place to Work® Institute and recognized as a ‘Cool Vendor’ and a ‘Vendor to Watch’ by Gartner.

Please visit Fractal | Intelligence for Imagination for more information about Fractal.

Role: Senior Azure Databricks Data Engineer

Role Summary

We are looking for a highly experienced Senior Azure Databricks Data Engineer with 10+ years of IT experience to lead the design, development, and optimization of enterprise-scale data platforms on Microsoft Azure. The ideal candidate will have deep expertise in Azure Databricks, PySpark, Delta Lake, cloud-native data architectures, and large-scale data modernization initiatives.

Key Responsibilities

• Design and implement scalable, high-performance data platforms using Azure Databricks and Azure cloud services.
• Develop robust ETL/ELT pipelines using PySpark, Spark SQL, and Databricks workflows.
• Architect and implement Lakehouse solutions leveraging Delta Lake and Medallion architecture.
• Lead complex data migration and modernization projects from on-premises and legacy platforms to Azure.
• Build and maintain batch and real-time data processing solutions.
• Integrate data pipelines with Azure Data Factory (ADF), ADLS Gen2, Azure Synapse Analytics, Azure SQL Database, and other Azure services.
• Optimize Spark jobs, cluster utilization, partitioning strategies, and query performance.
• Establish data quality, monitoring, governance, and security frameworks.
• Implement CI/CD pipelines using Azure DevOps and Git-based source control.
• Mentor junior engineers and provide technical leadership across project teams.
• Collaborate with architects, stakeholders, and business teams to define data strategies and technical roadmaps.
• Ensure adherence to enterprise security, compliance, and governance standards.

Required Technical Skills

• 10+ years of experience in Data Engineering, BI, or Data Warehousing.
• 5+ years of hands-on experience with Azure Databricks.
• Strong expertise in PySpark, Spark SQL, Python, and advanced SQL.
• Experience designing enterprise-scale Data Lakes and Lakehouse architectures.
• Hands-on experience with Delta Lake, Delta Live Tables (DLT), and Unity Catalog.
• Extensive experience with Azure Data Factory (ADF).
• Strong knowledge of Azure Data Lake Storage (ADLS Gen2).
• Experience with Azure Synapse Analytics and Azure SQL Database.
• Expertise in performance tuning, troubleshooting, and Spark optimization.
• Strong understanding of data modeling, data warehousing concepts, and dimensional modeling.

Cloud & DevOps

• Azure DevOps CI/CD implementation experience.
• Git version control and branching strategies.
• Knowledge of Infrastructure as Code (Terraform, ARM, or Bicep).
• Experience with environment migration and deployment automation.

Governance & Security

• Data governance and metadata management.
• RBAC, Unity Catalog, credential passthrough, and security best practices.
• Knowledge of compliance frameworks and enterprise data security standards.

Preferred Skills

• Databricks Workflows and Delta Live Tables.
• Streaming frameworks using Structured Streaming/Event Hub/Kafka.
• Experience with Power BI integration.
• Knowledge of Data Mesh and modern data architecture principles.
• Exposure to Machine Learning workflows within Databricks.
• Experience working in Agile/Scrum environments.

Education

• Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or related discipline.

Certifications (Preferred)

• Databricks Certified Data Engineer Professional
• Databricks Certified Data Engineer Associate
• Microsoft Certified: Azure Data Engineer Associate (DP-203)
• Microsoft Azure Solutions Architect Expert

Key Competencies

• Technical leadership and solution architecture capabilities.
• Strong stakeholder management and communication skills.
• Problem-solving and decision-making abilities.
• Ability to drive technical discussions and mentor teams.
• Ownership mindset and commitment to delivery excellence.
• Level Expectation (Grade 5): Senior Engineer / Lead Data Engineer capable of owning end-to-end Azure Databricks solutions, guiding teams, and driving enterprise data transformation initiatives.

Pay:

The wage range for this role takes into account the wide range of factors that are considered in making compensation decisions including but not limited to skill sets; experience and training; licensure and certifications; and other business and organizational needs.  The disclosed range estimate has not been adjusted for the applicable geographic differential associated with the location at which the position may be filled.  At Fractal, it is not typical for an individual to be hired at or near the top of the range for their role and compensation decisions are dependent on the facts and circumstances of each case.  A reasonable estimate of the current range is: $180,000 - $190,000. In addition, you may be eligible for a discretionary bonus for the current performance period.

Benefits:

As a full-time employee of the company or as an hourly employee working more than 30 hours per week, you will be eligible to participate in the health, dental, vision, life insurance, and disability plans in accordance with the plan documents, which may be amended from time to time.  You will be eligible for benefits on the first day of employment with the Company.  In addition, you are eligible to participate in the Company 401(k) Plan after 30 days of employment, in accordance with the applicable plan terms.   The Company provides for 11 paid holidays and 12 weeks of Parental Leave. We also follow a “free time” PTO policy, allowing you the flexibility to take time needed for either sick time or vacation.

Fractal provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state or local laws.

If you like wild growth and working with happy, enthusiastic over-achievers, you'll enjoy your career with us!

Hiring Related Queries

India: HiringsupportIndia@fractal.ai

Outside India: HiringsupportROW@fractal.ai

This inbox does not process resume submissions. All applications must be made through posted job openings

Not the right fit?  Let us know you're interested in a future opportunity by clicking Introduce Yourself in the top-right corner of the page or create an account to set up email alerts as new job postings become available that meet your interest!