# QA Automation Engineer (Senior)

> Data Wow Co., Ltd. · Pathum Wan, Thailand (Hybrid) · Full-time · Posted 2026-10-01

**Workplace:** hybrid

## Description

-   Build evaluation suites for LLM and ML features: curated test sets, scoring against reference answers, LLM-as-judge where it is justified, and human review where it is not.
-   Define what "good enough" means with the delivery team before launch, in numbers a client can accept, and hold releases to it.
-   Test retrieval pipelines end to end, covering chunking and indexing, retrieval quality, grounding of the final answer, and behaviour when the right document is simply not there.
-   Write regression tests that tolerate non-deterministic output, using semantic similarity, structured-output validation, tolerance bands, and repeated sampling rather than exact string matching.
-   Cover failure modes that matter to clients: hallucination, prompt injection through user or document content, leakage of data across tenants or sessions, refusal and over-refusal, and degradation on Thai-language input.
-   Track cost and latency alongside accuracy, so that a quality improvement that triples token spend is visible before it ships.
-   Detect drift after release by comparing production samples against the baseline, and raise it early rather than after the client does.
-   Design and maintain automated test suites for APIs, services, data pipelines, and web applications built for client projects.
-   Run tests in CI, keep them fast and stable, and treat a flaky suite as a defect in its own right.
-   Set the test strategy at the start of an engagement: what gets automated, what stays manual, what the exit criteria are, and what evidence the client will receive.

## Requirements

-   **4-5 years in test automation**, QA engineering, or software engineering with substantial testing ownership.
-   Strong programming ability in **Python**, or in TypeScript or Java with the willingness to work primarily in Python.
-   Hands-on experience **testing an LLM or ML-based product** in production or close to it.
-   Practical command of at least one automation framework such as Pytest, Playwright, Cypress, or RestAssured, and experience running suites in CI.
-   Working knowledge of API and data testing, including SQL and validating data through a pipeline.
-   The judgement to decide what deserves automation and what does not, and to say so when a suite is producing noise rather than signal.
-   Clear written and spoken communication in Thai and English.

## Benefits

-   **Hybrid Working** options with **Flexible Working Hours**.
-   **Your Choice of Gear:** Mac or Windows? You pick your preferred work equipment.
-   Comprehensive Group Health Insurance.
-   Monthly **Grab Transportation** credits.
-   Job-related training courses and certificate fee coverage.
-   **10+ days of annual leave**.
-   Free snacks and drinks available at the office.

## Apply

[Apply at Data Wow Co., Ltd.](https://apply.workable.com/datawow/j/1524BEECE9/apply)

---
Powered by [Workable](https://www.workable.com)