{"id":80255,"date":"2026-03-08T21:24:38","date_gmt":"2026-03-08T17:24:38","guid":{"rendered":"https:\/\/herecareers.com\/job\/lead-data-architect-cloud-lakehouse-azure-databricks-spark\/"},"modified":"2026-03-08T21:24:39","modified_gmt":"2026-03-08T17:24:39","slug":"lead-data-architect-cloud-lakehouse-azure-databricks-spark","status":"publish","type":"job_listing","link":"https:\/\/herecareers.com\/ar\/job\/lead-data-architect-cloud-lakehouse-azure-databricks-spark\/","title":{"rendered":"Lead Data Architect \u2013 Cloud Lakehouse (Azure | Databricks | Spark)"},"content":{"rendered":"<p>411_3290861<\/p>\n<p><p><b>Urgent requirement for <\/b> <b>Lead Data Architect \u2013 Cloud Lakehouse (Azure | Databricks | Spark) <\/b> <b>is required for our client in Abu Dhabi, UAE <\/b> <\/p>\n<h3>Must-Have Experience <\/h3>\n<ul>\n<li>Strong experience on end-to-end architecture of a production data platform (lakehouse \/ warehouse \/ analytics) <\/li>\n<li>Strong experience on Advanced PySpark optimization (joins, shuffles, skew handling, caching, AQE) <\/li>\n<li>Strong experience on Databricks on Azure <\/li>\n<li>Strong experience on Implementing lineage, metadata, and observability <\/li>\n<li>Strong experience on CI\/CD pipelines for data using Jenkins or GitLab CI\/CD <\/li>\n<\/ul>\n<h3>Core Responsibilities <\/h3>\n<ul>\n<li>Own the end-to-end data architecture for cloud-native analytical platforms, from ingestion to consumption, with zero tolerance for brittle or over-engineered designs <\/li>\n<li>Design and evolve enterprise-grade data lake and warehouse architectures on Azure that scale to billions of records and multiple consumption patterns (BI, ML, analytics) <\/li>\n<li>Make irreversible architectural decisions around storage formats, partitioning strategies, schema evolution, and data modeling \u2014 and stand behind them <\/li>\n<li>Define and enforce non-negotiable architectural standards for performance, cost efficiency, reliability, and security <\/li>\n<\/ul>\n<h3>Advanced Data Engineering Leadership <\/h3>\n<ul>\n<li>Architect and optimize high-throughput, low-latency data pipelines using Databricks, PySpark, and Azure-native services <\/li>\n<li>Set the technical bar for ETL\/ELT frameworks, orchestration, dependency management, and failure recovery patterns <\/li>\n<li>Personally review and challenge pipeline designs, Spark jobs, and SQL logic \u2014 no rubber\u2011stamp approvals <\/li>\n<li>Lead the transition from ad-hoc pipelines to fully productionized, observable, and automated data workflows <\/li>\n<\/ul>\n<h3>Data Quality, Governance &#038; Observability <\/h3>\n<ul>\n<li>Design and implement enterprise-grade data quality frameworks (validation, anomaly detection, reconciliation) <\/li>\n<li>Establish data lineage, metadata management, and monitoring as first-class architectural components <\/li>\n<li>Ensure datasets are audit-ready, reproducible, and trustworthy for executive, regulatory, and ML use cases <\/li>\n<\/ul>\n<h3>CI\/CD &#038; Engineering Excellence <\/h3>\n<ul>\n<li>Architect CI\/CD pipelines for data using Git-based workflows and tools such as Jenkins or GitLab CI\/CD <\/li>\n<li>Enforce automated testing strategies for data (unit, integration, data quality checks) <\/li>\n<li>Eliminate manual deployments and fragile handoffs across environments <\/li>\n<\/ul>\n<h3>Cross-Functional &#038; Strategic Influence <\/h3>\n<ul>\n<li>Translate ambiguous business requirements into clear, scalable data architectures <\/li>\n<li>Partner deeply with ML engineers, analysts, product leaders, and executives to design data assets that directly enable business outcomes <\/li>\n<li>Act as the final technical authority in data architecture discussions, tradeoffs, and escalations <\/li>\n<\/ul>\n<h3>Team Enablement (Without Micromanagement) <\/h3>\n<ul>\n<li>Mentor senior data engineers and technical leads, pushing them toward architectural thinking and ownership <\/li>\n<li>Set expectations for engineering rigor, documentation, and decision-making clarity <\/li>\n<li>Raise the technical maturity of the organization, not just deliver projects <\/li>\n<\/ul>\n<h3>Hard Requirements (Non-Negotiable) <\/h3>\n<ul>\n<li>7+ years in Data Engineering \/ Data Architecture, with proven ownership of large-scale production data platforms <\/li>\n<li>3+ years making architectural decisions, not just implementing someone\u2019s design <\/li>\n<li>Deep, hands-on expertise with Databricks + PySpark in real-world, high-volume environments <\/li>\n<li>Strong command of Microsoft Azure data services and cloud-native architecture patterns <\/li>\n<li>Expert-level Python and strong Spark optimization skills (partitioning, joins, caching, tuning) <\/li>\n<li>Proven experience designing fault-tolerant, highly available, cost-efficient data systems <\/li>\n<li>Strong Git-based development practices and experience enforcing engineering standards <\/li>\n<li>Demonstrated success implementing CI\/CD for data pipelines <\/li>\n<li>Ability to explain complex architectural tradeoffs clearly to both engineers and senior stakeholders <\/li>\n<\/ul>\n<p>Skills: architecture, cloud lakehouse, data, spark <\/p>\n<\/p>\n<p> #J-18808-Ljbffr<\/p>\n<p>2026-02-27 08:36:29<\/p>","protected":false},"author":100,"featured_media":0,"comment_status":"open","ping_status":"open","template":"","job_listing_type":[68],"job_listing_category":[1348],"job_listing_location":[720],"job_listing_tag":[],"class_list":["post-80255","job_listing","type-job_listing","status-publish","hentry","job_listing_type-full-time","job_listing_category-it-technology","job_listing_location-united-arab-emirates"],"metas":{"_job_featured_image":"","_job_featured":"","_job_filled":"","_job_urgent":"","_job_category":{"1348":"IT &amp; Technology"},"_job_type":{"68":"Full Time"},"_job_tag":[],"_job_expiry_date":"","_job_gender":"","_job_apply_type":"external","_job_phone":"","_job_apply_url":"https:\/\/en-ae.whatjobs.com\/coopob__cpl___411_3290861__3583?utm_source=3583&utm_medium=feed&keyword=Lead-Data-Architect&location=Abu-Dhabi&geoID=8","_job_apply_email":"","_job_salary_type":"","_job_salary":"","_job_experience":"","_job_career_level":"","_job_qualification":"","_job_video_url":"","_job_photos":"","_job_application_deadline_date":"","_job_address":"","_job_location":{"720":"United Arab Emirates"},"_job_map_location":{"address":"","latitude":"","longitude":""},"_job_logo":"https:\/\/herecareers.com\/wp-content\/uploads\/wp-job-board-pro-uploads\/_employer_featured_image\/2025\/11\/ICON-Here-Careers-Logo-150-150x150.png","_job_employer_name":"HR","_job_employer_url":"https:\/\/herecareers.com\/ar\/employer\/hr\/"},"cmb2":{"_job_careerjet_job_fields":{"_job_careerjet_detail_url":"","_job_careerjet_company_name":""}},"_links":{"self":[{"href":"https:\/\/herecareers.com\/ar\/wp-json\/wp\/v2\/job_listing\/80255","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/herecareers.com\/ar\/wp-json\/wp\/v2\/job_listing"}],"about":[{"href":"https:\/\/herecareers.com\/ar\/wp-json\/wp\/v2\/types\/job_listing"}],"author":[{"embeddable":true,"href":"https:\/\/herecareers.com\/ar\/wp-json\/wp\/v2\/users\/100"}],"replies":[{"embeddable":true,"href":"https:\/\/herecareers.com\/ar\/wp-json\/wp\/v2\/comments?post=80255"}],"wp:attachment":[{"href":"https:\/\/herecareers.com\/ar\/wp-json\/wp\/v2\/media?parent=80255"}],"wp:term":[{"taxonomy":"job_listing_type","embeddable":true,"href":"https:\/\/herecareers.com\/ar\/wp-json\/wp\/v2\/job_listing_type?post=80255"},{"taxonomy":"job_listing_category","embeddable":true,"href":"https:\/\/herecareers.com\/ar\/wp-json\/wp\/v2\/job_listing_category?post=80255"},{"taxonomy":"job_listing_location","embeddable":true,"href":"https:\/\/herecareers.com\/ar\/wp-json\/wp\/v2\/job_listing_location?post=80255"},{"taxonomy":"job_listing_tag","embeddable":true,"href":"https:\/\/herecareers.com\/ar\/wp-json\/wp\/v2\/job_listing_tag?post=80255"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}