# AI Evaluation Engineer > Raydar · United States (Remote) · Full-time · Posted 2026-10-15 **Salary:** USD 200,000–300,000 **Workplace:** remote ## Description About the company Our client is a software company. The role Raydar is recruiting for this role on behalf of our client. Lead the creation of a measurement approach for an AI system that carries out technical design work. Draw on both hands-on engineering judgment and evaluation know-how, and collaborate with domain experts and other teams to convert findings into clear improvement priorities. What you'll do \- Establish a new assessment framework that defines quality for design work where several answers can be valid. \- Collaborate with domain experts, external users and internal teams to see how seasoned engineers operate and where the system struggles. \- Reproduce realistic design tasks and typical failure cases in repeatable test setups that developers can run again. \- Create and tune automated graders alongside human review processes, and confirm that the graders agree with expert opinion. \- Convert assessment results into a ranked list of improvements to the system's reasoning. ## Requirements What we're looking for \- About five or more years of professional experience in hardware engineering, AI, or a blend of both, with depth in one and practical familiarity with the other. \- Substantial hands-on background assessing AI systems, including authoring test tasks, building automated graders and benchmarking them against human review. \- Solid grasp of how autonomous, agent-style AI systems behave. \- Grounding in a hardware engineering field, enough to tell sound design from weak design. \- Comfort working directly with external users, specialists, partners and technical colleagues across functions. \- Self-assurance to guide work among many stakeholders in a role with high ownership. \- Based in the US or Canada, and able to travel to an office for one week every 6 weeks. Bonus points \- Prior time at a robotics or physical AI company. \- Experience building things from zero in an early-stage startup. ## Benefits Compensation and benefits \- Base salary: USD 200,000 to 300,000 per year \- Equity Location and work model \- Remote (US or Canada) \- Full-time ## Apply [Apply at Raydar](https://apply.workable.com/raydar/j/E04B39F384/apply) --- Powered by [Workable](https://www.workable.com)