Senior AI Test Automation Engineer

We are looking for the very Top Talent and we would be delighted if you were to join our team More in detail UST is a multinational company based in North America certified as a Top Employer and Great Place to Work with over 35000 employees worldwide and presence in more than 35 countries.

We are leaders in digital technology services delivering large-scale technological solutions to top-tier companies.

What we look for We are looking for a Senior AI Test Automation Engineer to join the ML AI ART owning the end-to-end test automation strategy for an advanced AI/MLOps platform.

This platform powers business-critical AI capabilities such as LLM-based assistants RAG systems agentic workflows and ML pipelines.

Main tasks and accountabilities will be - Define and evolve the test automation architecture aligned with the Banks Test Strategy - Design scalable reusable automation frameworks and patterns across teams - Analyse requirements and ensure testability during Agile ceremonies (PI Planning Backlog Refinement etc.) - Design and automate test cases (functional non-functional) based on risk and coverage - Implement automation using Python frameworks (Playwright pytest Behave/BDD) - Develop AI/ML-specific testing approaches (LLM evaluation non-deterministic behaviour RAG workflows) - Build and maintain API contract tests and microservices validation - Integrate and run automated tests in CI/CD pipelines (GitLab) - Use Docker/Kubernetes for reproducible test environments - Manage test execution across environments (DEV INT UAT pre-PROD) - Monitor test health handle flaky tests and ensure reliable reporting - Report defects in Jira with proper traceability and evidence - Produce audit-ready test artefacts and ensure compliance with governance standards - Collaborate with cross-functional teams (POs Engineers MLOps Data Engineers) - Act as Test Automation Ambassador within the ART - Ensure end-to-end traceability across requirements tests and results - Maintain and evolve test suites as systems (APIs models prompts) change What do we expect from you - Strong experience in Python-based test automation (Playwright pytest BDD frameworks) - Proven ability to design and own test frameworks not just test scripts - Experience integrating automation into CI/CD pipelines (GitLab preferred) - Knowledge of parallel execution reporting tools and flaky test management - Solid understanding of Git clean code and code review practices - Experience with Docker and basic knowledge of Kubernetes - Exposure to AI/ML testing (LLMs RAG non-deterministic systems) or strong motivation to learn - Understanding of APIs microservices event-driven architectures - Familiarity with SAFe Agile and DevOps principles - Experience with Jira and test management tools (Xray Agile Hive etc.) - Strong analytical mindset with end-to-end system understanding - Awareness of data privacy security and compliance in regulated environments Work Location Hybrid position in Madrid (3 days at clients office Sol area next to the underground).

Work Schedule Business hours (40 hours weekly with 1 hour for lunch).

What can we offer 23 days of Annual Leave plus the 24th and 31st of December as discretionary days Numerous benefits (Health Care Plan teleworking compensation Life and Accident Insurances).

Retribución Flexible Program (Meals Kinder Garden Transport online English lessons Health Care Plan) Free access to several training platforms Professional stability and career plans UST also compensates referrals from which you could benefit when you refer professionals.

The option to pick between 12 or 14 payments along the year.

Real Work Life Balance measures (flexibility WFH or remote work policy compacted hours during summertime) UST Club Platform discounts and gym Access discounts If you would like to know more dont hesitate to apply and well get in touch to fill you in detail.

We are waiting for you