Explainability¶
Why an item was recommended, made visible to the visitor and auditable by the team.
A recommendation that arrives with no reason is hard to trust, and in a Holocaust-related context trust and accountability matter deeply. The engine should not only deliver relevant content but make clear why each item is suggested.
The recommender is a glass box. The user model is structured (tag affinity over the
expert taxonomy, aversions, sequence, an engagement summary), and every recommendation
carries a per-scorer breakdown on each ScoredCandidate. That breakdown is the raw
material for per-item cues; this page covers the layer on top of it: turning the model into
an interpretable persona and explainable visitor segments, grounded in
museum-visitor theory.
Module: recsys/explain/ (pure, no IO). Endpoints: /api/usermodel/explain, /api/clusters.
Per-item cues¶
Embeddings are latent and hard to read directly, so the system explains itself through signals a person can understand:
- Similarity cues: "because you viewed X" from the nearest-neighbour relationship.
- Thematic cues: alignment with a recognisable theme or topic cluster.
- Entity cues: a shared place or event, "recommended due to shared location and event."
flowchart LR
rec["recommendation"] --> kg["entity / shared place"]
rec --> theme["topic / tag match"]
rec --> sim["similar to viewed item"]
kg & theme & sim --> cue["human-readable cue"]These derive directly from the per-scorer breakdown; see Ranking
and the Recommendation model.
Theory grounding¶
The persona layer maps the structured user model onto established museum-visitor frameworks, so a segment reads as a recognisable visit identity rather than an opaque vector.
| Framework | Used for | Signal it maps to |
|---|---|---|
| Falk (2009), Identity and the Museum Visitor Experience | visitor_type |
breadth of interests (distinct themes) by engagement depth (dwell/completion) by pace |
| Pekarik, Doering & Karns (1999), "Satisfying Experiences in Museums" | experience_preference |
which taxonomy facet family dominates affinity (medium then object, theme then cognitive, personal-story/person then introspective) |
| Tintarev & Masthoff (explanation aims) | the whole design | scrutability: every claim carries its evidence so a visitor can inspect and correct it |
| Csikszentmihalyi (flow) | engagement_style |
dwell ratio vs reading-time estimate (the engagement strength already computed) |
Falk visitor types¶
Five visit identities, picked heuristically and transparently from the model:
| Type | Reads as | Heuristic driver |
|---|---|---|
| Hobbyist | narrow + deep + repeat | (1-breadth)·depth·cognitive·views |
| Explorer | broad + engaged | breadth·depth |
| Experience-Seeker | broad + light / skims | breadth·(1-min(dwell,completion)) |
| Recharger | few, slow, introspective | depth·(1-breadth)·few_views·introspective |
| Facilitator | social / accompanied visit | from the personal_connection demographic |
breadth is the number of distinct content themes engaged (not affinity sub-labels,
which would make one theme look broad). The pick is argmax; confidence is the margin to
the runner-up; the rationale names the drivers. The heuristics are intentionally simple and
auditable, to be calibrated against real visitor data later.
What the persona contains (PersonaExplanation)¶
interests/aversions: top taxonomy tags withevidence(the content ids that drove each), so every claim is inspectable.engagement_style:deep_reader|completionist|skimmer|sampler|contemplative(from the engagementbehaviorsummary).experience_preference: Pekarikobject|cognitive|introspective|social.visitor_type: the Falk type plusconfidence,rationale, and per-typescores.trajectory: recent thematic arc (the dominant theme per recent view, most-recent first).summary: optional prose fromexplain/verbalize.py, a deterministic template, orverbalize_llm, which words the same structured facts so nothing is invented.
A browser who never "liked" anything is not treated as cold. Their skim behaviour plus thematic trajectory still produce a persona (typically Experience-Seeker / skimmer).
Scrutability
Every interest, aversion, and visitor-type claim ships with its evidence (content ids, per-type scores, named drivers). A visitor can see exactly which of their actions produced each statement, satisfying the Tintarev & Masthoff scrutability aim and aligning with the WP7 ethical framework for respectful, accountable digital engagement.
The verbalizer¶
explain/verbalize.py turns the structured PersonaExplanation into a readable prose
summary. Two modes share one rule: only the structured facts may be spoken.
- Template (
verbalize): deterministic phrasing, no model, no invention. - LLM (
verbalize_llm): an LLM words the same facts more naturally; it is handed the structured fields and may not add themes or claims that are not in them.
Explainable clusters (explain/clusters.py)¶
K-means or fuzzy-c-means over the tag-affinity vectors. The cluster is the explanation: each centroid is a tag-weight profile in the taxonomy, so a segment reads as "Forced Labour + Resistance, narrow (Hobbyist-like)" rather than an opaque embedding. Pure-Python, no sklearn.
| Method | Membership | Fits |
|---|---|---|
k-means (cluster_users / assign) |
hard buckets | the simplest segmentation |
fuzzy-c-means (cluster_users_fuzzy / assign_fuzzy) |
soft (rows sum to 1) | Falk's overlapping identities |
Fuzzy membership fits visitors who sit between identities: a visitor deep in two themes reads
as c0:0.45 c1:0.55 instead of being forced into one bucket. m is the fuzziness (2.0
default; as it approaches 1 the method approaches k-means). A single-theme visitor is crisp
(about 0.98) under both methods; only fuzzy reveals that a blended visitor is split. Compare
them on archetypes with python explain/compare_clustering.py.
The method is chosen at train time; the API assigns with the matching function automatically
(it reads method from the saved model).
Clusters are offline-trained from the live user models:
REDIS_URL=redis://localhost:6379 python explain/cluster_train.py --k 4 --method fcm --out ./data/clusters.json
CLUSTER_MODEL_PATH=./data/clusters.json uvicorn ai_engine.recsys.api:app # GET /api/clusters
When CLUSTER_MODEL_PATH is set, /api/usermodel/explain also returns the visitor's
cluster (the soft membership under fuzzy-c-means).
Endpoints¶
| Endpoint | Returns | Guard |
|---|---|---|
GET /api/usermodel/explain?user_id=&verbalize=true |
persona (+ cluster if a model is loaded) | INGEST_API_KEY (exposes demographics) |
GET /api/clusters |
the segment profiles | INGEST_API_KEY |
Cohort content statistics¶
Beyond a single visitor, the engine aggregates content engagement across all visitors:
content seen, liked, disliked, like-rate, popular themes, and content preferences per cluster.
See Cohort statistics and the GET /api/content/stats endpoint.
Scope / next¶
- The Falk/Pekarik mappings are heuristic. Replace the thresholds with data-calibrated boundaries once labelled visitor data exists; the structure (signal then type) stays.
- Cluster-aware recommendation (e.g. a per-segment bandit θ) is the bridge to the contextual bandit's per-segment policy option, see Bandit / online learning.
- A memorial-specific empathy/identification axis (dark-heritage / prosthetic-memory literature) can be added as a third lens beside Falk and Pekarik.