Job description

About the role Our client is looking for experienced healthcare operations professionals to evaluate how frontier AI models handle the business side of healthcare: coding an encounter from the documentation, working a denial, auditing a claim, making a prior authorization determination, writing a physician query, or reconciling charges. You bring the judgment you have built defending a code set to a payer auditor, catching the unbundled charge, and writing the query a physician will actually answer. We bring the model output that judgment is needed to grade. In this role, you will design challenging, realistic tasks drawn from your own practice, such as a coded encounter with rationale and guideline citations, a denial appeal letter, a clinical documentation query, a prior authorization determination, a claims audit report, or a charge capture reconciliation, run them through frontier AI agents, and evaluate what comes back against a professional standard. You will work with realistic professional files, the kind a practitioner in your field actually handles, which you assemble yourself. Some tasks are compact, built around a handful of files; others are larger scenarios that take several days to build. In every case the goal is the same: a task a competent professional in your field would complete correctly and a frontier model currently gets wrong. This is not a traditional healthcare operations role. You will be helping build better AI by putting your knowledge to work in a structured, flexible, fully remote environment. The work is long form and self directed, and clear written reasoning matters as much as technical depth. Responsibilities • Design challenging, realistic healthcare operations tasks drawn from your own day to day work: the scenario, a prompt phrased the way you would brief a trusted colleague, and the supporting files a professional would need (de identified or constructed encounter documentation, claim forms, remittance advice, payer policies, correspondence), which you author yourself. • Run those tasks through frontier AI models and evaluate the deliverable they produce (the code set, appeal, query, determination or audit) against the standard you would hold a colleague to. • Compare two model outputs on identical prompts and files, decide which performed better, and document where each fell short. • Write detailed grading rubrics that specify what a correct deliverable must contain, such as the right codes at the right specificity, the right guideline cited, the right payer rule applied, and explain in writing why a response passes or fails each one. • Flag concrete failures with evidence: codes the documentation does not support, wrong sequencing, missed or fabricated guideline references, ignored payer policy, and off brief interpretation of the ask. • Contribute across coding, CDI, revenue cycle, utilization review and payer operations, and review and refine tasks built by other experts. Domain qualifications • 2+ years of hands on experience preferred in medical coding and billing, revenue cycle, health information management, clinical documentation integrity, utilization review and case management, or payer operations. • Depth in at least one of: inpatient or outpatient coding; professional fee coding; coding audit and compliance; CDI; denials and appeals; prior authorization and utilization review; claims examination; provider credentialing and network operations; practice or revenue cycle management. • Working understanding of several of the others, enough to know what those workflows involve and how they are run, so you can assess work in an adjacent area and point out what was done correctly or incorrectly. • Fluent in the code sets and rules of your area (ICD-10-CM and PCS, CPT, HCPCS, DRG or APC logic, payer policy, NCCI edits) and able to cite the guideline behind a decision. • Coding or HIM credential (CPC, CCS, CCA, RHIA, RHIT, CDIP, CCDS) preferred but not required. Practical production experience outweighs credentials. Credential holders may be asked to verify the credential during the assessment. General requirements • In progress Bachelor’s degree or higher, or an equivalent coding or HIM credential (CPC, CCS, CCA, RHIA, RHIT, CDIP). Credentials preferred but not required; production experience outweighs them. • 2+ years of hands on experience in your field preferred (see Domain qualifications above). Candidates with less experience are considered where the practical work is real. • Able to draw on your own real world experience and day to day workflows to craft scenarios that test whether an AI system can actually do the work. • Hands on practitioner: you currently do (or recently did) the work yourself at an individual contributor level, not solely in a managerial capacity. • Full professional or native level written and spoken English, with strong written communication. You can explain complex professional reasoning clearly and concisely, and articulate why a result is wrong, not only that it is. • Comfort with ambiguity and attention to detail. You can orient in a new set of files and build an accurate, deep working picture of it quickly, especially when the subject sits partly outside your own specialization. You verify what a document claims against the underlying numbers, sources or facts. • Capable of interpreting feedback, judging which parts of it are actually correct, and applying it without hand holding. When stuck, you look for the answer rather than waiting for one. • Ability to ramp quickly on unfamiliar work from written material and instructions alone, including where that material is incomplete (for example, writing grading rubrics for the first time). • General familiarity with AI and LLM tools. You have used models like Claude or ChatGPT in professional work and have the judgment to tell a well reasoned answer from a plausible sounding but incorrect one. • Baseline tech literacy: comfortable with cloud file tools (e.g., Google Workspace), managing browser profiles, downloading and installing desktop apps (e.g., Claude), and everyday file handling (e.g., converting between Excel and Google Sheets, zipping files for sharing). • Available at least 10 hours per week, with no weekly maximum. Consistent availability is valued and full time hours are available. • Based in the United States, Canada, or the UK.

Requirements

Must-haves

Run those tasks through frontier AI models and evaluate the deliverable they produce (the code set, appeal, query, determination or audit) against the standard you would hold a colleague to.
Compare two model outputs on identical prompts and files, decide which performed better, and document where each fell short.
Flag concrete failures with evidence: codes the documentation does not support, wrong sequencing, missed or fabricated guideline references, ignored payer policy, and off brief interpretation of the ask.
Contribute across coding, CDI, revenue cycle, utilization review and payer operations, and review and refine tasks built by other experts.
Domain qualifications
Working understanding of several of the others, enough to know what those workflows involve and how they are run, so you can assess work in an adjacent area and point out what was done correctly or incorrectly.
Fluent in the code sets and rules of your area (ICD-10-CM and PCS, CPT, HCPCS, DRG or APC logic, payer policy, NCCI edits) and able to cite the guideline behind a decision.
General requirements
Able to draw on your own real world experience and day to day workflows to craft scenarios that test whether an AI system can actually do the work.
Hands on practitioner: you currently do (or recently did) the work yourself at an individual contributor level, not solely in a managerial capacity.
Capable of interpreting feedback, judging which parts of it are actually correct, and applying it without hand holding. When stuck, you look for the answer rather than waiting for one.
Ability to ramp quickly on unfamiliar work from written material and instructions alone, including where that material is incomplete (for example, writing grading rubrics for the first time).
General familiarity with AI and LLM tools. You have used models like Claude or ChatGPT in professional work and have the judgment to tell a well reasoned answer from a plausible sounding but incorrect one.
Available at least 10 hours per week, with no weekly maximum. Consistent availability is valued and full time hours are available.
Based in the United States, Canada, or the UK.

Nice-to-haves

2+ years of hands on experience preferred in medical coding and billing, revenue cycle, health information management, clinical documentation integrity, utilization review and case management, or payer operations.
In progress Bachelor’s degree or higher, or an equivalent coding or HIM credential (CPC, CCS, CCA, RHIA, RHIT, CDIP). Credentials preferred but not required; production experience outweighs them.
2+ years of hands on experience in your field preferred (see Domain qualifications above). Candidates with less experience are considered where the practical work is real.

Application prep

Use the role’s language and application details while tailoring your materials.

Keywords to mirror

aillm

Platform: wp-job-manager
Effort: medium