Senior Artificial Intelligence Evaluation Engineer

Ciklum is looking for a Senior AI Evaluation Engineer to join our team full-time in Canada.

We are a custom product engineering company that supports both multinational organizations and scaling startups to solve their most complex business challenges. With a global team of over 4,000 highly skilled developers, consultants, analysts and product owners, we engineer technology that redefines industries and shapes the way people live.

About the role:

As a Senior AI Evaluation Engineer, become a part of a cross-functional development team, engineering experiences of tomorrow.

Responsibilities:

Build the central eval harness and templates every pod uses; hub-and-spoke: central standards, pod-written tests

Create golden datasets with business and clinical reviewers, including safety and medical-accuracy suites

Set and defend pass thresholds; audit pod evals; report eval pass rates and incidents monthly

Stand up production monitoring for drift and regression with the pipeline engineer

Requirements:

4+ years in ML/LLM evaluation, QA engineering for AI systems, or applied research engineering

Hands-on with eval frameworks, LLM-as-judge patterns and their failure modes, statistical rigor on small samples

Independent spine: the team shipping a thing does not set its own pass bar

Desirable:

Healthcare or safety-critical evaluation experience; red-teaming background

What’s in it for you?

Strong community: Work alongside top professionals in a friendly, open-door environment

Growth focus: Take on large-scale projects with a global impact and expand your expertise

Tailored learning: Boost your skills with internal events (meetups, conferences, workshops), Udemy access, language courses, and company-paid certifications

Endless opportunities: Explore diverse domains through internal mobility, finding the best fit to gain hands-on experience with cutting-edge technologies

Care: Healthcare, Basic Life Insurance, Short and Long-term disability insurance according to the Company’s Benefit Plans

About us:

At Ciklum, we are always exploring innovations, empowering each other to achieve more, and engineering solutions that matter. With us, you’ll work with cutting-edge technologies, contribute to impactful projects, and be part of a One Team culture that values collaboration and progress. Now expanding across Canada, we’re looking for talented professionals to strengthen our North American footprint. Join us to innovate at scale and deliver world-class solutions to global clients.

Explore, empower, engineer with Ciklum!

Interested already? We would love to get to know you! Submit your application. We can’t wait to see you at Ciklum.

#LI-VH1

Required Skills

AI Platform

Job Info

Job Identification 4470

Job Category Data Science and Machine Learning

Posting Date 09/04/2026, 02:11 AM

Locations CA-Remote (Remote)