Intelligence Analyst (Strategic Defense & Assessment) — 100% Remote AI Benchmark Specialist

As a seasoned practitioner in Analytic rigor, Source handling, Over-refusal detection, All-Source Intelligence Analysis, your specialized judgment represents a rare and indispensable asset. Here at RemoteJob USA Staffing, we are proud to advocate for credentialed specialists like you. We are recruiting an experienced Intelligence Analyst (Strategic Defense & Assessment) — 100% Remote AI Benchmark Specialist earning $30 - $80/hour to collaborate on domain-specific LLM post-training. You will serve as the final authority on technical accuracy, clinical rigor, and ethical soundness in complex model outputs.

Clinical & Technical Focus Areas

• Assess and grade AI-generated responses based on criteria such as reasoning, source use, recognition of limitations, and practical utility.
• Identify and flag instances where legitimate analytic exploration is over-refused by the model, providing clear documentation and feedback.
• Document edge cases and analytic disagreements, supporting continuous improvement of evaluation methodologies.
• Collaborate with project leads to refine procedures for source evaluation and confidence-level assignment.
• Contribute insights to help calibrate analytic standards and enhance the overall quality of AI system benchmarks.
• Develop and map analytic cases to defined taxonomies across multiple evaluation areas, ensuring comprehensive coverage of relevant intelligence scenarios.
• Author complex test cases designed to evaluate model performance on source handling, analytic rigor, and nuanced uncertainty expression.

Domain Qualifications & Background

• Exceptional written and verbal communication skills, with a focus on clarity and precision.
• Ability to identify and articulate nuanced model limitations and over-refusal scenarios.
• Familiarity with structured analytic techniques and intelligence tradecraft standards.
• Previous participation in AI evaluation or model assessment initiatives is a plus.
• Demonstrated all-source analytic experience or deep expertise in a specific analytic discipline, ideally with a civilian-IC background.
• Strong proficiency in source evaluation, confidence-level assessment, and established analytic standards.
• Experience in teaching, mentoring, or leading analytic reviews, with a record of fostering analytic rigor.

Contract Terms & Professional Remuneration

• Compensation Rate: $30 - $80/hour (verified contractor remuneration)
• Work Arrangement: 100% Remote / Work From Home
• Eligible Remote Countries: United States, United Kingdom, Canada, Australia, New Zealand, Ireland, and South Africa
• Core Competencies: Analytic rigor, Source handling, Over-refusal detection, All-Source Intelligence Analysis
• Hiring Lead: Dr. Rachel Adams (Academic & Clinical Sourcing Lead) at RemoteJob USA Staffing

Direct Application Process: Click the Apply button on this page to submit your profile directly for immediate evaluation.