Senior Research Engineer – Evaluations

Job Title: Senior Research Engineer – Evaluations

Company: Canva

Location: San Francisco, CA (Hybrid)

Salary Range: $220,000 – $280,000/year

Type: Full-time

About the Role

Canva is seeking a Senior Research Engineer to develop advanced evaluation systems that assess the quality and human alignment of generative AI models. This role is ideal for someone who combines strong machine learning expertise with practical engineering skills, especially in LLM-based evaluation, prompt engineering, and model benchmarking.

What You'll Do

• Build AI evaluation agents using Multimodal Large Language Models (MLLMs) to assess generative design outputs.
• Design and run inference-time alignment techniques like prompt engineering, in-context learning (ICL), and Retrieval-Augmented Generation (RAG).
• Create a robust benchmarking system to evaluate and compare model quality.
• Analyze results, detect model weaknesses, and suggest improvements.
• Collaborate with research scientists and ML engineers to integrate evaluation tools into the development lifecycle.
• Convert the latest LLM and agentic AI research into usable, production-ready systems.

Key Responsibilities

• Develop and maintain infrastructure for automated evaluation systems using “MLLM-as-a-Judge”.
• Improve generative model output quality through non-training-based methods.
• Design benchmarks and evaluation metrics for design-focused tasks.
• Turn evaluation data into actionable feedback for research teams.
• Work across teams to incorporate evaluation tools into production.

You’re a Strong Match If You Have

• Deep understanding of generative AI models (e.g., diffusion models, GANs, transformers).
• Experience in evaluating and analyzing generative models.
• Expertise in large-scale model training and distributed computing.
• Strong programming skills, especially with Python, PyTorch, and cloud infrastructure (AWS).
• Familiarity with clean code practices, pull requests, and collaborative development.
• Strong analytical mindset and ability to translate findings into clear insights.

Bonus Points For

• Experience with evaluation frameworks or agentic AI systems.
• Knowledge of data visualization tools.
• Background in human-computer interaction or design principles.

Benefits

• Equity packages – share in Canva’s success.
• Health plans and 401(k) contributions.
• Inclusive parental leave for all types of families.
• Vibe & Thrive allowance – support for well-being, social connection, or home office setup.
• Flexible leave options to recharge or contribute to causes.