# Machine Learning Engineer > Raydar · New York, United States · Full-time · Posted 2026-10-01 **Salary:** USD 220,000–300,000 **Workplace:** on_site ## Description About the company Our client is an AI technology company. The role Raydar is recruiting for this role on behalf of our client. Own the measurement of model quality from start to finish, defining metrics, building the tooling to compute them and turning the results into concrete improvements. The work centers on evaluating complex multi-component machine learning systems where no ground truth exists. You will carry projects from the initial question through to a written conclusion without handoffs. What you'll do \- Define evaluation metrics and what improvement means for outputs that evolve over time, keeping those definitions current as methods change. \- Build data pipelines and test harnesses covering data ingestion, labeling, versioning, reruns and automated judging. \- Analyze run traces and results to find real failure modes rather than only what is easy to measure, and propose fixes. \- Develop simulation tooling that lets many scenarios be modeled and measured in parallel to speed up iteration. \- Own each evaluation loop end to end, from the question to the pipeline, rerun and final writeup. ## Requirements What we're looking for \- Experience designing, building and running evaluations for large language models from start to finish. \- Prior work in areas such as memory, personalization, agent evaluation or agent optimization. \- Published research as a first or co-author at a top machine learning venue, or open-source work of comparable quality. \- A university-level background in a quantitative discipline such as computer science or statistics. \- Production-quality Python, including testing, maintenance and regular releases. \- Working knowledge of PyTorch and common machine learning tooling and experiment tracking. Bonus points \- Experience with data pipeline tooling such as Kafka and SQL. ## Benefits Compensation and benefits \- Base salary: USD 220,000 to 300,000 per year \- Equity \- Health, dental and vision coverage \- Retirement plan \- Paid time off \- Relocation support Location and work model \- New York, NY, United States \- On-site, 5 days per week in office \- Full-time ## Apply [Apply at Raydar](https://apply.workable.com/raydar/j/F43A82D550/apply) --- Powered by [Workable](https://www.workable.com)