Job description

We are seeking a Senior Data Engineer Job Functions & Responsibilities • Cloud Data Engineering & Integration: • Design and implement data pipelines across AWS, Azure, and Google Cloud. • Develop SAP BTP integrations with cloud and on-premise systems. • Ensure seamless data movement and storage between cloud platforms. • ETL & Data Pipeline Development: • Develop and optimize ETL workflows using Pentaho and Microsoft ADF /or equivalent ETL tools. • Design scalable and efficient data transformation, movement, and ingestion processes. • Monitor and troubleshoot ETL jobs to ensure high availability and performance. • API Development & Data Integration: • Develop and integrate RESTful APIs to support data exchange between SAP and other platforms. • Work with API gateways and authentication methods like OAuth, JWT, API keys. • Implement API-based data extractions and real-time event-driven architectures. • Data Analysis & SQL Development: • Write and optimize SQL queries, stored procedures, and scripts for data analysis, reporting, and integration. • Perform data profiling, validation, and reconciliation to ensure data accuracy and consistency. • Support data transformation logic and business rules for ERP reporting needs. • Data Governance & Quality (Ataccama, Collibra): • Work with Ataccama and Collibra to define and enforce data quality and governance policies. • Implement data lineage, metadata management, and compliance tracking across systems. • Ensure compliance with enterprise data security and governance standards. • Cloud & DevOps (AWS, Azure, GCP): • Utilize Azure DevOps and GitHub for version control, CI/CD, and deployment automation. • Deploy and manage data pipelines on AWS, Azure, and Google Cloud. • Work with serverless computing (Lambda, Azure Functions, Google Cloud Functions) to automate data workflows. • Collaboration & Documentation: • Collaborate with SAP functional teams, business analysts, and data architects to understand integration requirements. • Document ETL workflows, API specifications, data models, and governance policies. • Provide technical support and troubleshooting for data pipelines and integrations. Required Skills & Experience: • 7+ years of experience in Data Engineering, ETL, and SQL development. • Hands-on experience with SAP BTP Integration Suite for SAP and non-SAP integrations. • Strong expertise in Pentaho (PDI), Microsoft ADF, and API development. • Proficiency in SQL (stored procedures, query optimization, performance tuning). • Experience working with Azure DevOps, GitHub, and CI/CD for data pipelines. • Good understanding of data governance tools (Ataccama, Collibra) and data quality management. • Experience working with AWS, Azure, and Google Cloud (GCP) for data integration and cloud-based workflows. • Strong problem-solving skills and ability to work independently in a fast-paced environment. Preferred Qualifications: • Experience working on SAP S/4HANA and cloud-based ERP implementations. • Familiarity with Python, PySpark for data processing and automation. • Experience working on Pentaho, Microsoft ADF/or equivalent ETL tools • Knowledge of event-driven architectures • Familiarity with CI/CD for data pipelines (Azure DevOps, GitHub Actions, etc.). Education & Qualifications • Bachelor's or Master's degree in a relevant field like Computer science , Data Engineering or related technical field. • Nice to have below certifications: • Azure Data Engineer associate • SAP Certified Associate - Integration Developer Work hours and location: • Monday through Friday 7.5 hour work day with .5 hour unpaid lunch Paid for time worked only; there are no paid holidays, vacation, or sick days. Overtime is not permitted unless authorized in advanced. • Contingent worker is required to badge in and out of the turnstiles located in the lobby for all daily entrances and exits. • Candidate REQUIRED to be LOCAL to Tri-State area Spruce Technology, Inc. is a mid-size, award-winning (Inc 5000, SmartCEO, Entrepreneur of the Year) technology services firm with a steadily growing portfolio of commercial and government clients. Spruce provides innovative technology solutions, specialized IT staff, and IT strategy consulting nationwide. Spruce maintains partnerships with major technology vendors and continually develops leading-edge offerings in service areas such as digital experience, data services, application development, infrastructure, cyber security, and IT staffing. Spruce Technology, Inc. is an affirmative action and equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, disability, age, sexual orientation, gender identity, national origin, veteran status, or genetic information. Consistent with the Americans with Disabilities Act, it is the policy of Spruce Technology, Inc. to provide reasonable accommodation when requested by a qualified applicant or employee with a disability, unless such accommodation would cause an undue hardship. The policy regarding requests for reasonable accommodation applies to all aspects of employment, including the application process.

Requirements

Must-haves

We are seeking a Senior Data Engineer
Job Functions & Responsibilities
Design and implement data pipelines across AWS, Azure, and Google Cloud.
Develop SAP BTP integrations with cloud and on-premise systems.
Ensure seamless data movement and storage between cloud platforms.
Develop and optimize ETL workflows using Pentaho and Microsoft ADF /or equivalent ETL tools.
Design scalable and efficient data transformation, movement, and ingestion processes.
Monitor and troubleshoot ETL jobs to ensure high availability and performance.
Develop and integrate RESTful APIs to support data exchange between SAP and other platforms.
Work with API gateways and authentication methods like OAuth, JWT, API keys.
Implement API-based data extractions and real-time event-driven architectures.
Write and optimize SQL queries, stored procedures, and scripts for data analysis, reporting, and integration.
Perform data profiling, validation, and reconciliation to ensure data accuracy and consistency.
Support data transformation logic and business rules for ERP reporting needs.
Work with Ataccama and Collibra to define and enforce data quality and governance policies.
Implement data lineage, metadata management, and compliance tracking across systems.
Ensure compliance with enterprise data security and governance standards.

Nice-to-haves

Experience working on SAP S/4HANA and cloud-based ERP implementations.
Familiarity with Python, PySpark for data processing and automation.
Experience working on Pentaho, Microsoft ADF/or equivalent ETL tools
Knowledge of event-driven architectures
Familiarity with CI/CD for data pipelines (Azure DevOps, GitHub Actions, etc.).
Education & Qualifications
Bachelor's or Master's degree in a relevant field like Computer science , Data Engineering or related technical field.

Application prep

Use the role’s language and application details while tailoring your materials.

Keywords to mirror

awsazureapisqlgcpci/cdpython

Platform: jobdiva
Effort: medium