Site Reliability Engineer at Sand Tech Holdings Limited

Job description

Site Reliability Engineer About Sand Sand Technologies is a global Physical AI company using data and AI to make critical industries work better. We partner with governments, cities and enterprises to improve how essential systems operate across healthcare, water, energy, telecommunications and infrastructure. Our work delivers proven real-world impact. We have built AI systems that help manage London’s water supply, supported telecom network planning across hundreds of cities, and developed digital healthcare platforms serving tens of millions of people across Africa. From intelligent command centers to AI-powered infrastructure platforms, we help organizations sense, analyze and act in complex environments. Our people are ambitious, curious and relentlessly practical. Our teams work alongside clients in the field, solving hard problems and deploying solutions that last. With colleagues across Africa, Europe, the UK and the US, we operate across the full stack - from research and engineering to deployment and capability building. Our mission is simple: to harness AI to solve humanity’s most pressing challenges. Our platform Sand builds the intelligent platforms behind critical infrastructure: water, healthcare, telecommunications, energy, for enterprises and governments worldwide. Our AI and data intelligence systems help manage London's water supply and power digital health platforms serving tens of millions of people. We have delivered successfully into over fifty client environments globally and are rapidly growing our footprint in the US. The successful applicant for this role will have an opportunity to grow with these expansion efforts, taking on significant technical and operational leadership responsibilities as we scale. We are investing in a platform, Symmetri, which underpins and accelerates our client impact. It integrates heterogeneous operational data, models the entities and relationships of an entire domain, and lets operators of critical infrastructure understand the state of their world, reason about it, and act. It runs in multi-tenant cloud, in single-tenant customer environments, on premises, and is already deployed and in use in multiple fully air-gapped installations at ministry and utility level. Our US customers are city governments, water utilities and public agencies who will run their operations on these systems and audit how we deploy and manage them. About the role Sand operates in partnership with senior executives at our customers. Your primary client within these relationships will be the customer's enterprise IT organization. You will spend as much time in front of a municipal CIO's security team, an architecture review board and a cloud governance committee as you will in a terminal. You will be asked where the data sits, who has access to it, how it is encrypted, and what happens during an incident, and you will need to answer those questions accurately and calmly without escalating every one of them. Symmetri is designed to be highly modular and extensible, supporting rapid development and the mechanisms to support reusable intelligence capabilities across clients without sharing their data. As it continues to become the standard way we deliver, the line between deploying the platform and building it converges. You will be expected to see that coming and drive the strategy and execution that helps us get there. Your role will be to lead and deliver the deployment and the operation of cloud infrastructure, including Symmetri environments, for US customers, end to end. What you’ll do Provisioning: Standing up infrastructure for a new environment, where the shape of the build depends on the environment type and variant, from managed cloud through single-tenant to on premises based on established patterns and practices. Configuration: Tuning environments to a customer's needs as we land and expand on the use cases they are consuming: performance, scale, resource ceilings, network boundaries, identity, applying their security model rather than ours. Lifecycle: Running the upgrade and maintenance cycles, and coordinating releases across live customer production estates without breaking the operations that depend on them. Operation: Monitoring, alerting, and being first responder for critical production outages at any time of the day or night - which we see as an exceptional occurrence to be solved for, not the norm. Collaboration & Growth: You will be the first US member of our Delivery Infra & Enablement function, but you will not need to work in isolation. You will collaborate directly with an established and experienced remote enablement team and regional operations counterparts in other geographies. This will assist you to establish a full US-based operations team and set the standard of how US deployment and operations actually works in practice. Your first 6 months You have taken a US customer environment from provisioning to production and you are the named owner of it. Runbooks for that environment exist, are accurate, and someone other than you (or their agent) could follow them safely. Monitoring and alerting are in place and tuned, so that a page means something and silence means something too. You have been through at least one customer security review as our technical voice in the room. You can say, with evidence, which parts of our global deployment approach need to work differently in the United States. Who you are 5 to 8 years in platform engineering, cloud infrastructure, DevOps or site reliability, operating production systems that mattered to someone. Cloud infrastructure, hands on. Azure and AWS. You have provisioned and run production data infrastructure rather than only managing it, with a strong understanding of networking, IAM and secure data handling, and how to design effectively within a client’s existing landscape and governance constraints. Networking you can reason about under pressure. VPCs and VNets, subnets, security groups, DNS, load balancing, private connectivity, and the patterns for getting services to talk to each other across a customer's internal and external boundaries. Kubernetes and containers. Pods, deployments, services, namespaces. Comfortable in kubectl or any kube-api interface for inspection and troubleshooting. You do not need to have written an operator. Infrastructure as code. You are comfortable with using IaC by default, from the start, with more than one toolchain. Demonstrable experience across one or more of Pulumi, Terragrunt, Terraform, OpenTofu, CDK or equivalents is required. Pulumi is used for Symmetri and experience with it is a strong plus. Python for automation. As an organisation with strong data science roots we are strongly biased to python for many coding use cases, so familiarity is important and experience is a plus. Change management and branching discipline. Standard git workflows, and the willingness to follow and improve a team's existing strategy rather than your own. Resource planning. Sizing compute, memory and storage against what a customer actually needs and what their environment can actually give you, including when that environment is on premises and finite. Incident response. You have been on call. You know the difference between fixing an outage and fixing the cause of one, you are able to run effective root cause analysis processes that result in long term improvements. Client-facing competence with enterprise IT. You can hold a technical conversation with a customer's infrastructure and security people, be trusted by them, and represent a commitment without over-promising. You will not be equally strong across all of these aspects. Strong in most, familiar with the rest, unafraid of learning fast while being aware of and drawing in support for your current limitations. Advantageous experience Public sector, utility or other regulated operational environments, and the security review processes that come with them Air-gapped, sovereign or disconnected deployments of data-intensive systems Observability and monitoring stacks, and building the alerting rather than only responding to it Serving ML models and LLM systems in production, including offline Location We are looking to hire on the East Coast in the United States, as this is where the majority of our clients are based. How we work This role has an on-call component and accountability. Real production systems, real operational consequences. It also has less scaffolding than you might expect. We optimize first to fall in love with our clients' challenges and to deliver impactful solutions. This means you will not always receive neatly scoped work, and part of the job is creating that clarity yourself and sharing it. We are not looking for someone to invent everything from scratch or replicate exactly what they have done in the past either. We believe in strong opinions, loosely held . We have established patterns and approaches that work, built by teams that have run these environments for years, and the person who succeeds here is one who learns them properly first and then improves them while participating in raising our global standard. Due to the highly collaborative and internationally distributed nature of our work, successful candidates must be comfortable operating in small teams while contributing to larger, globally coordinated efforts. A strong sense of ownership, self-motivation and discipline in maintaining clear and consistent communication through virtual collaboration tools and video conferencing is essential.

Requirements

Must-haves

Site Reliability Engineer
Your role will be to lead and deliver the deployment and the operation of cloud infrastructure, including Symmetri environments, for US customers, end to end.
What you’ll do
Lifecycle: Running the upgrade and maintenance cycles, and coordinating releases across live customer production estates without breaking the operations that depend on them.
Operation: Monitoring, alerting, and being first responder for critical production outages at any time of the day or night - which we see as an exceptional occurrence to be solved for, not the norm.
Your first 6 months
You have taken a US customer environment from provisioning to production and you are the named owner of it.
Runbooks for that environment exist, are accurate, and someone other than you (or their agent) could follow them safely.
Monitoring and alerting are in place and tuned, so that a page means something and silence means something too.
You have been through at least one customer security review as our technical voice in the room.
You can say, with evidence, which parts of our global deployment approach need to work differently in the United States.
5 to 8 years in platform engineering, cloud infrastructure, DevOps or site reliability, operating production systems that mattered to someone.
Kubernetes and containers. Pods, deployments, services, namespaces. Comfortable in kubectl or any kube-api interface for inspection and troubleshooting. You do not need to have written an operator.
Change management and branching discipline. Standard git workflows, and the willingness to follow and improve a team's existing strategy rather than your own.
Resource planning. Sizing compute, memory and storage against what a customer actually needs and what their environment can actually give you, including when that environment is on premises and finite.
Incident response. You have been on call. You know the difference between fixing an outage and fixing the cause of one, you are able to run effective root cause analysis processes that result in long term improvements.
Client-facing competence with enterprise IT. You can hold a technical conversation with a customer's infrastructure and security people, be trusted by them, and represent a commitment without over-promising.
You will not be equally strong across all of these aspects. Strong in most, familiar with the rest, unafraid of learning fast while being aware of and drawing in support for your current limitations.
Advantageous experience
Public sector, utility or other regulated operational environments, and the security review processes that come with them
Air-gapped, sovereign or disconnected deployments of data-intensive systems
Observability and monitoring stacks, and building the alerting rather than only responding to it
Serving ML models and LLM systems in production, including offline

Nice-to-haves

Python for automation. As an organisation with strong data science roots we are strongly biased to python for many coding use cases, so familiarity is important and experience is a plus.

Application prep

Use the role’s language and application details while tailoring your materials.

Keywords to mirror

aiazureawskubernetesapipythonrestmlllm

Platform: greenhouse
Effort: quick

Benefits

Benefits detected in the listing.

Vision