Senior AI/ML Engineer – Multimodal Digital Human & Memory Systems
Project: Recallem
Level: Senior / strong Mid-Senior candidates considered
Type: Full-time / Part-time / Contract
Location: Toronto / Hybrid
Embedded Files
About Recallem
Recallem, developed by Genotive AI, is an AI project focused on creating interactive digital representations of real people based on information preserved across their lives.
The source material may include photographs, home videos, interviews, voice recordings, documents, correspondence, stories, biographical records, and other personal or archival material. Some projects may involve well-documented historical or public figures, while others may involve private individuals and family collections containing sensitive personal information.
Our goal is to transform this fragmented multimodal information into a structured, secure, and efficiently retrievable representation of a person—their memories, experiences, relationships, knowledge, speech patterns, voice, appearance, expressions, and behavioural characteristics—so that it can ultimately support an interactive digital human experience.
About the Role
We are looking for a hands-on Senior AI/ML Engineer to help design and build the intelligence and data pipeline at the core of Recallem.
This is not primarily a prompt-engineering or API-integration role.
You will investigate and implement the best ways to ingest heterogeneous information about a person, identify and extract meaningful information from it, connect information across sources and time, represent it appropriately, and retrieve the right information quickly and accurately during interaction.
You will work across multimodal AI, machine learning, LLMs, computer vision, speech processing, information extraction, RAG, knowledge graphs and retrieval systems.
What You'll Build
Multimodal ingestion and extraction
· Build pipelines for text, documents, photographs, audio and video.
· Identify and isolate a target person across multi-person photographs, recordings and videos.
· Perform speaker diarization, speaker verification, speech recognition and audio segmentation.
· Extract facial expressions, head movement, gestures, visual characteristics and other useful behavioural signals.
· Transcribe and temporally align speech, speakers, video and other extracted features.
· Handle imperfect real-world material including old photographs, home videos, noisy audio and archival recordings.
Personal knowledge and memory
· Extract people, places, events, dates, relationships, experiences, preferences and other meaningful information from unstructured sources.
· Resolve the same people, events and concepts across multiple sources.
· Preserve provenance so retrieved information can be traced back to its original source.
· Represent temporal and relational information such as who knew whom, what happened, where it occurred and when.
· Distinguish documented facts from uncertain, inferred or conflicting information.
Storage and retrieval architecture
· Determine the appropriate representation for different information rather than forcing everything into a single database.
· Design combinations of structured databases, object/media storage, vector stores and knowledge graphs.
· Build semantic, lexical, hybrid and graph-based retrieval.
· Develop RAG/GraphRAG pipelines capable of retrieving relevant memories and contextual information with low latency.
· Implement embeddings, metadata filtering, reranking and contextual retrieval.
· Design persistent-memory architectures capable of growing as additional information about a person is added.
Digital-human intelligence
· Extract characteristics useful for reproducing an individual's conversational style, vocabulary, speaking patterns and behavioural tendencies.
· Prepare structured voice, facial, expression and behavioural data for downstream avatar, speech and digital-human systems.
· Work with the broader team on connecting retrieved memories and personal context to real-time interaction.
· Evaluate emerging approaches for voice, facial animation, digital humans and multimodal models rather than assuming a fixed technology stack.
Privacy, Security & Data Governance
Because Recallem may work with private family collections as well as public archival material, privacy must be part of the architecture—not an afterthought.
The engineer will help:
· Keep each person's information logically isolated.
· Design access controls and secure storage for private media and personal information.
· Track the source and provenance of extracted information.
· Support deletion, correction and updating of stored information.
· Evaluate when processing should occur locally/on-premises versus through approved cloud services.
· Evaluate model, dataset and software licensing before incorporating technologies into the product.
Required Qualifications
· Strong Python and production ML/AI engineering experience.
· Experience building end-to-end AI or ML pipelines rather than only calling model APIs.
· Strong understanding of LLMs, embeddings, RAG and modern information retrieval.
· Experience with multimodal AI involving text, images, audio and/or video.
· Practical computer-vision and/or speech-processing experience.
· Experience with vector databases and semantic search.
· Understanding of knowledge graphs or graph-based information representation.
· Experience with PyTorch and modern ML/model ecosystems such as Hugging Face.
· Ability to evaluate competing models and architectures experimentally.
· Strong software-engineering fundamentals, including Git, APIs, Docker and Linux.
· Ability to work with incomplete requirements, prototype rapidly, measure results and turn successful experiments into maintainable systems.
Particularly Valuable Experience
· Speaker diarization, speaker verification and ASR.
· Multilingual or code-switched speech.
· Face detection, tracking, recognition, landmarks or 3D facial representations.
· Gesture/body-pose or expression analysis.
· Vision-language models and multimodal LLMs.
· Knowledge graphs, GraphRAG and entity resolution.
· Hybrid search and reranking.
· Long-term/persistent AI memory.
· Audio/video restoration and archival media.
· Voice synthesis or voice cloning.
· Digital humans, avatars or facial animation.
· Fine-tuning, LoRA or model adaptation.
· Privacy-preserving/local AI inference.
· GPU inference, optimization and model deployment.
Technologies You May Work With
Examples include PyTorch, Hugging Face, Whisper-family/WhisperX, pyannote, OpenCV, VLMs, Qdrant, Neo4j/GraphRAG, PostgreSQL, object storage, Docker and modern LLM/embedding APIs.
These are examples, not a fixed stack. The successful engineer should be capable of evaluating new tools and replacing existing choices when a better approach is demonstrated.
What We Are Really Looking For
We are looking for someone who can answer questions such as:
How do we take decades of scattered information about one human being—photos, conversations, videos, documents, relationships, stories and memories—and transform it into a secure computational representation that an AI system can retrieve and use naturally during an interaction?
The right person should be able to move through the complete pipeline:
Raw Personal Data → Multimodal Extraction → Identity & Temporal Alignment → Structured Knowledge & Memory → Vector/Graph Representation → Retrieval → Context → Interactive Digital Human
The candidate does not need to already know the answer to every part of this problem. We want an engineer capable of researching alternatives, designing experiments, measuring quality and helping Recallem discover the best technical approach.
Interested in this position?
Send your résumé to careers@genotive.com
Page updated
Report abuse