Job description

Position: Senior Azure Databricks Data Engineer Location: Fort Worth, Texas Duration: Contract Job ID: 180344 Job Overview: This Sr. Engineer 2 role is a hands-on, senior-level engineering position focused on designing and implementing scalable data and application solutions using Python, Spark, Databricks, and Azure Cloud (including Azure ML). The position is hybrid, requiring onsite work Tuesday through Thursday and virtual work on Mondays and Fridays. The engineer will work across multiple agile pods, driving best practices in data engineering, DevOps, monitoring, and governance, while collaborating closely with data scientists, analysts, and business stakeholders to deliver robust, production-grade solutions. Responsibilities: • Participate in and contribute to end-to-end architecture, design, and implementation of scalable data and application solutions using Python and Azure ML. • Design and develop complex ETL/ELT pipelines to process, transform, and integrate large-scale datasets from diverse sources using Spark and Python. • Optimize Python and Spark jobs for performance and cost-efficiency through techniques such as partitioning, caching, cluster configuration tuning, and bottleneck analysis. • Collaborate with data scientists, analysts, and business stakeholders to translate complex business problems into robust, maintainable technical solutions. • Implement and enforce data governance and security strategies, including access controls, encryption, data lineage, and frameworks such as Unity Catalog. • Champion code quality, engineering best practices, and standards across pods, including code reviews, testing, and documentation. • Design, build, and maintain DevOps CI/CD pipelines (e.g., Azure DevOps, GitHub Actions, Jenkins) to automate build, test, deployment, and management of data assets and infrastructure. • Develop and implement comprehensive monitoring, logging, and alerting systems for production-grade pipelines to proactively detect, troubleshoot, and resolve issues. • Support multiple agile pods by guiding project planning, sprint execution, and tracking deliverables across various initiatives. • Promote and help enforce agile methodologies across pods, ensuring consistent ceremonies, backlog management, and delivery tracking. • Leverage AI tools and automation to improve productivity, code quality, and operational efficiency. • Participate in technical interviews, including coding assessments, and contribute to the evaluation of candidates when needed. • Work effectively in a hybrid environment, attending mandatory onsite meetings and interviews as required. Qualifications: • 7–10 years of overall experience in software engineering and data engineering roles. • At least 7 years of hands-on experience with Python, ETL development, data handling, and SQL. • Advanced proficiency in Python, Scala, and/or SQL for data processing and application development. • Minimum 5 years of experience with Spark and Databricks programming in production environments. • Strong experience designing and implementing complex ETL/ELT pipelines for large-scale datasets. • Solid background in Azure Cloud services, including Azure ML and other relevant Azure data and application services. • Experience implementing data governance, security, and compliance frameworks, including access control, encryption, and data lineage (e.g., Unity Catalog). • Proven experience designing and developing CI/CD pipelines that deploy code across multiple development and production environments, preferably on Azure using GitHub Actions or similar tools. • At least 5 years of experience with logging, monitoring, and alerting systems for production workloads. • Strong understanding of the full software development life cycle (SDLC), including requirements, design, development, testing, deployment, and support. • Experience using AI tools to enhance productivity, code quality, or data workflows. • Nice to have: Experience with Java and Spring for application development. • Nice to have: Broad knowledge of different Azure services and how they integrate within data and application architectures. • Excellent communication skills, with the ability to explain complex technical concepts to both technical and non-technical stakeholders. • Demonstrated ability to work in and support agile, pod-based team structures, including cross-team coordination and planning. • Ability to work in a hybrid schedule, with mandatory onsite presence Tuesday through Thursday and flexibility to work virtually on Mondays and Fridays. • Comfort participating in a multi-round interview process, including a virtual interview, mandatory onsite interview, and coding assignment. About PTR Global: PTR Global is a leading provider of information technology and workforce solutions. PTR Global has become one of the largest providers in its industry, with over 5000 professionals providing services across the U.S. and Canada. For more information visit www.ptrglobal.com At PTR Global, we understand the importance of your privacy and security. We NEVER ASK job applicants to: • Pay any fee to be considered for, submitted to, or selected for any opportunity. • Purchase any product, service, or gift cards from us or for us as part of an application, interview, or selection process. • Provide sensitive financial information such as credit card numbers or banking information. Successfully placed or hired candidates would only be asked for banking details after accepting an offer from us during our official onboarding processes as part of payroll setup. Pay Range: $60 - $65/Hr The specific compensation for this position will be determined by several factors, including the scope, complexity, and location of the role, as well as the cost of labor in the market; the skills, education, training, credentials, and experience of the candidate; and other conditions of employment. Our full-time consultants have access to benefits, including medical, dental, vision, and 401K contributions, as well as PTO, sick leave, and other benefits mandated by applicable state or localities where you reside or work. If you receive a suspicious message, email, or phone call claiming to be from PTR Global do not respond or click on any links. Instead, contact us directly at +1 214-740-2424. To report any concerns, please email us at legal@pinnacle1.com During the hiring process, we may use artificial intelligence (AI) tools to assist in evaluating information related to your application. These tools may help analyze job-related qualifications or assist recruiters in reviewing applications and interview responses. AI-generated information is one factor considered in our hiring process. Final employment decisions are made by qualified hiring personnel and are not based solely on AI-generated recommendations. #LI-SA4

Requirements

Must-haves

Position: Senior Azure Databricks Data Engineer
Location: Fort Worth, Texas
Duration: Contract
Job ID: 180344
Participate in and contribute to end-to-end architecture, design, and implementation of scalable data and application solutions using Python and Azure ML.
Design and develop complex ETL/ELT pipelines to process, transform, and integrate large-scale datasets from diverse sources using Spark and Python.
Optimize Python and Spark jobs for performance and cost-efficiency through techniques such as partitioning, caching, cluster configuration tuning, and bottleneck analysis.
Collaborate with data scientists, analysts, and business stakeholders to translate complex business problems into robust, maintainable technical solutions.
Implement and enforce data governance and security strategies, including access controls, encryption, data lineage, and frameworks such as Unity Catalog.
Champion code quality, engineering best practices, and standards across pods, including code reviews, testing, and documentation.
Design, build, and maintain DevOps CI/CD pipelines (e.g., Azure DevOps, GitHub Actions, Jenkins) to automate build, test, deployment, and management of data assets and infrastructure.
Develop and implement comprehensive monitoring, logging, and alerting systems for production-grade pipelines to proactively detect, troubleshoot, and resolve issues.
Support multiple agile pods by guiding project planning, sprint execution, and tracking deliverables across various initiatives.
Promote and help enforce agile methodologies across pods, ensuring consistent ceremonies, backlog management, and delivery tracking.
Leverage AI tools and automation to improve productivity, code quality, and operational efficiency.
Participate in technical interviews, including coding assessments, and contribute to the evaluation of candidates when needed.
Work effectively in a hybrid environment, attending mandatory onsite meetings and interviews as required.
7–10 years of overall experience in software engineering and data engineering roles.
At least 7 years of hands-on experience with Python, ETL development, data handling, and SQL.
Advanced proficiency in Python, Scala, and/or SQL for data processing and application development.
Minimum 5 years of experience with Spark and Databricks programming in production environments.
Strong experience designing and implementing complex ETL/ELT pipelines for large-scale datasets.

Nice-to-haves

Nice to have: Experience with Java and Spring for application development.
Nice to have: Broad knowledge of different Azure services and how they integrate within data and application architectures.

Application prep

Use the role’s language and application details while tailoring your materials.

Keywords to mirror

azurepythonmlagileci/cdaisqljava

Platform: bullhorn
Effort: medium

Benefits

Benefits detected in the listing.

Dental

Vision

401k

Pto