Job description

Sutherland is seeking an experienced Lead Databricks Architect with deep domain expertise in the banking and financial services industry to drive the strategy, design, and implementation of our enterprise lakehouse platform. In this role, you will bridge the gap between complex banking business requirements and cutting-edge data architecture. You will be responsible for building a secure, scalable, and highly performant Databricks lakehouse environment that powers critical banking applications, including regulatory reporting, risk management, fraud detection, customer analytics, and real-time transaction processing. Key Responsibilities Architecture & Platform Strategy • Design and implement an enterprise-grade Databricks Lakehouse architecture leveraging Delta Lake, Unity Catalog, and Databricks SQL tailored for financial data models. • Establish architectural patterns for high-throughput batch and real-time streaming data pipelines (e.g., Structured Streaming, Auto Loader) integrating core banking systems, payment gateways, and third-party data providers. • Drive the migration strategy from legacy data warehouses (e.g., Teradata, Oracle, Snowflake, mainframe environments) to Databricks. Security, Compliance & Governance • Implement strict data security, enterprise access controls, and fine-grained permissions using Databricks Unity Catalog to comply with financial regulations (e.g., GDPR, SOX, PCI-DSS). • Architect data masking, tokenization, and encryption strategies for Personally Identifiable Information (PII) and Sensitive Personal Data (SPD). • Ensure full data lineage, auditing, and metadata management across multi-cloud or hybrid-cloud deployment environments (Azure). Data Engineering & Performance Optimization • Define best practices for PySpark, Delta Live Tables (DLT), and SQL optimization to maximize compute efficiency and reduce total cost of ownership (TCO). • Establish CI/CD, DevOps, and MLOps automation frameworks for code deployment, data quality validation (e.g., Great Expectations), and model lifecycle management. • Partner with Data Science, Quantitative Analytics, and BI teams to support advanced financial modeling, predictive risk analysis, and customer 360 initiatives. Stakeholder & Domain Leadership • Collaborate with business leaders, Chief Risk Officers (CROs), and compliance teams to align data platform capabilities with core banking business objectives. Provide technical leadership, mentorship, and architecture governance to data engineering and analytics teams. Required Technical Skills • Databricks Expertise: 5+ years of hands-on experience designing and running large-scale Databricks environments in production. • Cloud Architecture: Deep knowledge of AWS, Azure, or GCP data services (e.g., Azure Data Lake Storage/Data Factory, AWS S3/Glue, or GCP GCS/BigQuery). • Data Processing & Storage: Mastery of Delta Lake, PySpark, Databricks SQL, Structured Streaming, and Delta Live Tables. • Governance & Security: Strong hands-on experience configuring Databricks Unity Catalog, RBAC, ABAC, and data governance frameworks. • DevOps & Infrastructure as Code: Proficiency with Terraform, CI/CD pipelines (GitHub Actions, Azure DevOps, Jenkins), and Databricks Asset Bundles (DABs). Required Banking Domain Knowledge • Domain Experience: 3+ years working within a retail bank, investment bank, or financial services institution. • Financial Data Models: Familiarity with core banking operations, payments, lending, credit risk, anti-money laundering (AML), or fraud data structures. • Regulatory Landscape: Strong understanding of banking data compliance requirements, auditability, and data sovereignty standards. Education & Certifications • Education: Bachelor’s or Master’s degree in Computer Science, Data Engineering, Information Systems, or a related quantitative field (or equivalent practical experience). • Certifications (Preferred): Databricks Certified Data Engineer Professional Cloud Solutions Architect Certification (AWS Certified Solutions Architect / Azure Solutions Architect Expert) All your information will be kept confidential according to EEO guidelines. EEOC and Veteran Documentation During employment, employees are treated without regard to race, color, religion, sex, national origin, age, marital or veteran status, medical condition or handicap, or any other legally protected status. At times, government agencies require periodic reports from employers on the sex, ethnicity, handicap, veteran and other protected status of employees. The purpose of this Administrative EEO Record is for statistical analysis only and is used to comply with government record keeping, reporting, and other legal requirements. Periodic reports are made to the government on the following information. The completion of the Administrative EEO record is optional. If you choose to volunteer the requested information, please note that all Administrative EEO Records are kept in a Confidential File and are not part of your Application for Employment or Personnel file. Please note: YOUR COOPERATION IS VOLUNTARY. INCLUSION OR EXCLUSION OF ANY DATA WILL NOT AFFECT ANY EMPLOYMENT DECISION.

Requirements

Must-haves

Architecture & Platform Strategy
Design and implement an enterprise-grade Databricks Lakehouse architecture leveraging Delta Lake, Unity Catalog, and Databricks SQL tailored for financial data models.
Drive the migration strategy from legacy data warehouses (e.g., Teradata, Oracle, Snowflake, mainframe environments) to Databricks.
Security, Compliance & Governance
Implement strict data security, enterprise access controls, and fine-grained permissions using Databricks Unity Catalog to comply with financial regulations (e.g., GDPR, SOX, PCI-DSS).
Architect data masking, tokenization, and encryption strategies for Personally Identifiable Information (PII) and Sensitive Personal Data (SPD).
Ensure full data lineage, auditing, and metadata management across multi-cloud or hybrid-cloud deployment environments (Azure).
Data Engineering & Performance Optimization
Define best practices for PySpark, Delta Live Tables (DLT), and SQL optimization to maximize compute efficiency and reduce total cost of ownership (TCO).
Establish CI/CD, DevOps, and MLOps automation frameworks for code deployment, data quality validation (e.g., Great Expectations), and model lifecycle management.
Partner with Data Science, Quantitative Analytics, and BI teams to support advanced financial modeling, predictive risk analysis, and customer 360 initiatives.
Stakeholder & Domain Leadership
Collaborate with business leaders, Chief Risk Officers (CROs), and compliance teams to align data platform capabilities with core banking business objectives.
Provide technical leadership, mentorship, and architecture governance to data engineering and analytics teams.
Required Technical Skills
Databricks Expertise: 5+ years of hands-on experience designing and running large-scale Databricks environments in production.
Cloud Architecture: Deep knowledge of AWS, Azure, or GCP data services (e.g., Azure Data Lake Storage/Data Factory, AWS S3/Glue, or GCP GCS/BigQuery).
Data Processing & Storage: Mastery of Delta Lake, PySpark, Databricks SQL, Structured Streaming, and Delta Live Tables.
Governance & Security: Strong hands-on experience configuring Databricks Unity Catalog, RBAC, ABAC, and data governance frameworks.
DevOps & Infrastructure as Code: Proficiency with Terraform, CI/CD pipelines (GitHub Actions, Azure DevOps, Jenkins), and Databricks Asset Bundles (DABs).
Required Banking Domain Knowledge
Domain Experience: 3+ years working within a retail bank, investment bank, or financial services institution.
Financial Data Models: Familiarity with core banking operations, payments, lending, credit risk, anti-money laundering (AML), or fraud data structures.
Regulatory Landscape: Strong understanding of banking data compliance requirements, auditability, and data sovereignty standards.

Nice-to-haves

None flagged.

Application prep

Use the role’s language and application details while tailoring your materials.

Keywords to mirror

sqlazureci/cdawsgcp

Platform: smartrecruiters
Effort: medium