Pipeline Walkthrough

A step-by-step look at what happens for one visitor, start to finish: opening the app, answering the two surveys, tapping a spot in the AR app, getting their first recommendations, and how each recommendation after that gets better.

This is a how it actually runs document. Each step links to the concrete code that does the work in the Python code reference.

Domain

An in-memorial WW2 museum app. "Visitors" are the subjects; content is mainly short stories, plus images, exhibitions, and POIs. The AR app overlays stories on physical camp locations (e.g. Barrack 56).

---

The two doors into the engine

Everything the visitor does reaches the engine through exactly two HTTP endpoints, wired in ai_engine.recsys.api (make_router, api.py:328):

Door	Endpoint	Purpose
Write	POST /api/ingest	every event in (surveys, views) → rebuilds the user model
Read	GET /api/recommend	every recommendation out (incl. the AR location tap)

The model is rebuilt on the write path (at ingest time), not lazily on read. So by the time the AR app calls /api/recommend, the survey-derived model already exists. That is the spine of the whole flow.

sequenceDiagram
    participant App as UI / AR app
    participant W as POST /api/ingest
    participant R as GET /api/recommend
    App->>W: IDENTIFY (open app)
    App->>W: SURVEY_SUBMITTED (pre-survey)
    App->>W: SURVEY_SUBMITTED (personalization)
    Note over W: normalize → buffer+log → refresh → save UserSignals (cold: survey tags)
    App->>R: recommend(filter=Barrack 56)
    R-->>App: first recommendations (survey-led)
    App->>W: CONTENT_VIEW_ENDED (read a story, echoes request_id)
    Note over W: refresh → +engagement → warm
    App->>R: recommend(...)
    R-->>App: better recommendations

---

00 Identify the visitor

A stable id ties every event to one profile.

flowchart LR
  A[App opens] -->|generates id| ID["user_id<br>(e.g. 7QF2)"]
  ID -->|sent on every event| RS[RudderStack]

Every visitor needs a stable id so all their events accumulate into one user model. There is no login: the app generates the id itself, a short Crockford Base32 string (currently 4 characters, for example 7QF2), and attaches it to the visitor.

When the UI loads, the RudderStack SDK emits an IDENTIFY event carrying that id (before the visitor is identified, events instead carry an anonymousId). Every later event, surveys and views, travels through RudderStack under the same id.

When the first event reaches POST /api/ingest, normalize_events() (per-event normalize_event, rudderstack.py:51) reads userId, falling back to anonymousId (rudderstack.py:58), and stores it as user_id on the canonical InteractionEvent (models.py:44). From here on that user_id keys the visitor: every survey answer and view sent with the same id folds into the same UserSignals.

At this point there is no model yet. Any /api/recommend now takes the cold-start path (Step 4).

---

01 Pre-survey

Demographics seed the first, lightly-weighted persona tags.

flowchart LR
  D["age, gender,<br>nationality, province"] --> SA[survey_affinity]
  SA --> T["person_who tags<br>(weight 0.3 to 0.5)"]

The visitor answers the demographic pre-survey: age, gender, nationality, NL province, WW2 personal connection. The app sends these as survey_answers on a SURVEY_SUBMITTED (or RudderStack IDENTIFY traits) event to POST /api/ingest.

The ingest handler (api.py:359):

1. normalize_events → InteractionEvent(event="SURVEY_SUBMITTED", survey_answers={…}).
2. Appends to the durable Parquet log and the hot buffer.
3. Per touched user_id, calls UserModelUpdater.refresh (updater.py:46), which rebuilds the model.

refresh re-folds the visitor's whole recent buffer through build_user_signals() (signal_builder.py:91), a rebuild-from-buffer that stays the single source of truth. Survey answers become taxonomy tags via two pure functions in ai_engine.recsys.survey:

Function	Demographic → tag key	Seed weight
extract_demographics()	stores raw {age, gender, nationality, province, personal_connection}	n/a
survey_affinity()	age → person_who.age_group:{bucket}	0.5
	gender → person_who.gender_and_age:{…}	0.3
	nationality → person_who.city_village_country:From: {Country} (+ :International rollup for non-core)	0.4 (0.3)
	province → person_who.province_netherlands:{Province}	0.5

The seed weight is the starting strength of each tag in the persona: a number from 0 to 1 that says how strongly to pull the visitor towards content carrying that tag. It is the value score_tag uses when it scores a candidate (it multiplies the visitor's weight for a tag by the content's weight for the same tag, so a higher seed weight means a bigger push). The weights are relative, a 0.5 tag pulls harder than a 0.3 one. Demographics get modest weights (0.3 to 0.5) because they are only weak hints of interest; the personalization answers in Step 2 get the maximum 1.0, because the visitor stated those outright.

The visitor's tag interests come from two sources: what they told us in the survey, and what they later browse. The model keeps these in two separate buckets rather than one merged list. The tags built here go into the survey bucket; the browsing bucket stays empty for now. Keeping them apart is what lets the survey tags stay in charge on cold start instead of being drowned out once browsing begins (Step 3 shows how the two are blended). The rebuilt UserSignals is then saved. The visitor is still cold (is_cold is not positives), no content engaged yet.

---

02 Personalization

The chosen theme, interest and area become the strongest tags.

flowchart LR
  V["answer = taxonomy label<br>e.g. Forced Labor"] --> A["+ facet from<br>the question"]
  A --> K["theme_what:Forced Labor<br>(weight 1.0)"]

Next the visitor picks a theme, an interest type, and a camp area.

One piece of background first. Every piece of content in the collection is labelled with tags written as facet:label, where:

• facet is the category, for example theme_what (a historical theme) or place_where.camp_areas (a location).
• label is the value within that category, for example Forced Labor.

A visitor is matched to content when the two share the same facet:label tag. See the Tag system (taxonomy) page for the full list of facets, categories, and the theme hierarchy.

For these three personalization questions, the options the visitor can pick are written to be those content labels. So the answer the visitor selects already is the tag label, and nothing has to be translated: the code only needs to add the facet, meaning which category the answer belongs to, and that is fixed per question (_PERSONALIZATION, survey.py:41):

Question the visitor answers	Category (facet) added	Answer becomes the tag key	Weight
theme (q:personalization_theme)	theme_what	"Forced Labor" → theme_what:Forced Labor	1.0
interest type (q:personalization_interest)	theme_how.type_of_stores	"Personal stories" → theme_how.type_of_stores:Personal stories	1.0
camp area (q:personalization_area)	place_where.camp_areas	"Barracks" → place_where.camp_areas:Barracks	1.0

(This is different from the demographic questions in Step 1, whose answers are codes like 55_64 that need a lookup table to become a label. Here the answer value passes straight through.) The answer is only lightly cleaned (_canonical_label, survey.py:63) so spelling or formatting differences, such as British "forced labour" versus the content label "Forced Labor", still line up.

These are the strongest seeds (weight 1.0), because they are the visitor's explicitly stated interests. The same ingest path (Step 1) rebuilds the model.

---

03 Cold-start model

Survey tags lead while cold; browsing fades in as it grows.

flowchart LR
  S["survey bucket"] --> M((blend))
  E["browsing bucket"] -->|× eng_trust| M
  M --> TA["tag_affinity<br>(the interest list)"]

The critical bit for good first recommendations: survey-stated tags dominate while the visitor is cold, and browsing only takes over as it accumulates. Inside build_user_signals():

1. The two "buckets" from Step 1 are just two dictionaries, each mapping a tag to a weight ({"facet:label": weight}):
   • survey_affinity holds the tags from the survey and demographics (for example {"theme_what:forced labor": 1.0, "person_who.age_group:age 55-64": 0.5}).
   • eng_affinity holds tags from content the visitor engages with while browsing (built in Step 6). It is empty until they read something.
2. Each dictionary is cleaned up (keys lower-cased and merged) and scaled to [0,1] on its own, by dividing by its own largest weight. Scaling them separately is what stops a few survey tags from being washed out later by browsing, whose raw weights can grow much larger.

3. The two buckets are merged into the final interest list, tag_affinity, but the browsing bucket is faded in gradually so it does not take over too early. Two quantities do this:

   eng_trust  = n_positive / (n_positive + tag_engagement_trust_k)   # k = 5.0 (config)
   tag_affinity[t] = survey_norm[t]  +  eng_trust · eng_norm[t]      # then rescaled to [0,1]
   

   • n_positive is how many items the visitor has engaged with positively so far.
   • eng_trust is a dial between 0 and 1 that says how much to trust browsing. It starts at 0 (no browsing yet) and creeps towards 1 as positives pile up. k (default 5) sets the pace: eng_trust reaches 0.5 at k positives.
   • The final weight for each tag t is its survey weight (always counted in full) plus its browsing weight scaled down by eng_trust. So while cold, browsing contributes almost nothing and the survey wins; as the visitor reads more, browsing counts for more.

   positives so far	eng_trust	effect
   0 (just finished surveys)	0.00	survey tags fully drive recommendations
   1	0.17	survey still dominant
   5	0.50	engagement now equal-ish
   20	0.80	browsing leads, survey still contributes

How this drives recommendations. tag_affinity is the visitor's interest list: each tag with a weight from 0 to 1. When ranking a candidate story, the scorer score_tag adds up the visitor's weight times the story's weight for every tag they share, so a story matching the visitor's strong interests scores high and rises to the top of the list (full scoring in Step 5). Because the blend above keeps survey weights high while cold, the very first recommendations follow what the visitor said in the survey.

The result is still one tag_affinity dict, consumed by that one scorer. The pace knob is RecConfig.tag_engagement_trust_k: a larger k means browsing takes over more slowly.

---

04 First recommendations

A location tap returns survey-led stories from that spot.

flowchart TB
  F["filter = Barrack 56"] --> P["stories tagged there"]
  P --> SC["score_tag vs persona"]
  SC --> D["fuse + diversify"]
  D --> L["recommendation list"]

The visitor walks up to Barrack 56 and taps it. The app issues:

GET /api/recommend?user_id=<id>&filter=AiARLocationBarrack56

For now the AR location tap sends only the filter. The recommend handler (api.py:377) maps the query to Recommender.recommend (recommender.py:51):

Query param	Meaning	Plumbing
user_id	the visitor	loads saved UserSignals via model_store.get_signals
filter	the tapped location as a tag value	restricts candidates to that tag
limit, include_content	response shaping	truncate / compact

So an AR location tap is a filter=<location tag> request. The AiAR machine tag is decoded to the curator label by memorise_taxonomy.normalize_filter_value (AiARLocationBarrack56 → barrack 56) so it matches hand-tagged content.

Geo params exist but are not used yet

/api/recommend also accepts near_lat / near_lon (device GPS → score_geo proximity) and geo_radius_m (a hard radius), and combines them with filter by intersection. The AR app does not send these today; the tapped-location filter alone drives geo relevance.

Inside Recommender.recommend_for_signals (recommender.py:58) the request is handled in four moves. The visitor is still cold (no browsing yet), so their persona is just the survey tags from Steps 1 to 3.

1. Narrow to the location. Only content tagged place_where.camp_areas:barrack 56 is considered. (If every such story has already been seen and filter_reshow_when_exhausted is on, the location's already-seen stories are re-shown rather than returning nothing.)
2. Score each candidate by how well it matches the persona. score_tag (scorers.py:76) gives each story a 0 to 1 match score by comparing the visitor's interest weights against the story's tags on the shared facet:label key (the arithmetic is in the worked example below). It divides by only the visitor's few strongest interests; tag_match_topk is that count (6 by default), which stops a long tail of weak tags from shrinking the score. The other scorers (semantic similarity, recency) barely contribute while the visitor is cold, because they need browsing history. Those scorers are then merged into a single ranking number by weighted_fuse (fusion.py:15): it multiplies each scorer's score by a fixed weight and adds them up, so the signal with the largest weight has the most say. Tag matching has the largest weight (Step 5 lists them all), so it leads, and a per-signal breakdown is kept so the explanation view can show why an item ranked where it did.
3. Diversify and add a little novelty. Ranking purely by score can bunch near-identical stories at the top. mmr_rerank fixes that with MMR (Maximal Marginal Relevance): going down the list it prefers items that are both high-scoring and not too similar to the ones already picked. mmr_lambda sets the balance (0.7 means 70% weight on relevance, 30% on being different). Then, on every request (distractor_probability = 1.0), one deliberately off-profile "distractor" is slipped into slot 3 or 4 to show the visitor something new.
4. Return and record. The response is a Recommendation (items, strategy, diagnostics). recommend also mints a request_id, logs what was shown, and stores each item's feature vector under that id, so that when the visitor later views one of these stories the reward can be tied back to it (Steps 6 to 7).

Worked example

Suppose the surveys left this cold persona (survey bucket only):

Visitor interest (tag)	weight
theme_what:forced labor	1.0
theme_how.type_of_stores:personal stories	0.7
person_who.age_group:age 55-64	0.5

Tapping Barrack 56 narrows the pool to stories tagged there. Say three qualify. score_tag adds up the visitor's weight for each tag the story also has, then divides by the visitor's strongest interests, whose weights sum to 1.0 + 0.7 + 0.5 = 2.2 here (all three fit within topk = 6):

Story	Its other tags	Matched weight	Tag score
A	forced labor, personal stories	1.0 + 0.7 = 1.7	1.7 / 2.2 = 0.77
B	forced labor	1.0	1.0 / 2.2 = 0.45
C	daily life	0	0.00

Result order: A, then B, then C. Story A wins because it matches the visitor's two strongest stated interests; C only matches the location. So the very first list is stories about forced labor and personal accounts at Barrack 56, exactly what the visitor said they cared about, produced purely from the survey.

---

05 Scoring

Several signals combine into one ranking, with tags leading.

flowchart LR
  t["tag 0.45"] --> W((weighted sum))
  s["semantic 0.20"] --> W
  r["recency 0.05"] --> W
  av["aversion −0.30"] --> W
  g["geo 0.20"] --> W
  W --> F["final score"]

fused = Σ weight[s] · score[s]: deliberately simple so every recommendation is explainable. Weights live in FusionWeights, overridable by RECSYS_W_* env vars and per-tenant overrides. Basis says what a signal compares: tag (shared facet:label tags), vector (embedding meaning), or geo (physical distance).

Scorer	Weight	Basis	What it measures	When applied
tag	0.45	tag	overlap between the visitor's tag interests and the content's tags (survey + engagement), the dominant signal	always
semantic	0.20	vector	cosine to the whole-history taste centroid	always (0 for a cold visitor with no taste vector)
recency	0.05	vector	cosine to the most-recent view	always (0 until the visitor has viewed something)
aversion	−0.30	tag	overlap with disliked themes (a penalty)	always
geo	0.20	geo	exp(−distance/geo_scale_m) proximity	only when near_lat / near_lon are passed to /api/recommend

Which signals apply. The four model-based signals (tag, semantic, recency, aversion) come from the stored UserSignals, so they are always in play, though a signal contributes 0 when its input is missing (a cold visitor has no taste or recency vector). geo is the one per-request signal: it is scored only when the API call carries a location. (The tag filter is likewise applied only when passed, but it restricts which candidates are considered rather than scoring them.)

Tag matching (0.45) outweighs the vector signals combined (0.25), so recommendations track stated and browsed interests legibly; the vector signals add coverage for cold/unseen content, they don't lead.

Why several signals? Each one catches good matches the others miss. tag is precise and explainable, but only sees concepts a curator actually tagged. semantic works on meaning rather than tags, finding stories close to the visitor's overall taste even when the words differ. recency follows the thread of what they just read; aversion pushes away themes they disliked; and geo keeps results near where the visitor physically is. Blending them means a thin or missing signal (no tags, no history yet, no location) is covered by the others, while the weights keep tag in charge so the result stays legible.

Optional learned ranking: a LinUCB contextual bandit (LinearBandit) is available but off by default (ranking_mode="static"). Its prior is exactly FusionWeights, so at day zero it reproduces the static ranking, then learns away. Feature vectors are logged in both modes. See Bandit / online learning.

---

06 Learn from a view

Reading a story updates the model and warms the visitor.

flowchart LR
  V["CONTENT_VIEW_ENDED"] --> S["engagement strength"]
  S -->|positive| P["eng_affinity +<br>taste & recency vectors"]
  S -->|negative| N["tag_aversion"]

The visitor opens a recommended story. The app emits CONTENT_VIEW_STARTED and, on leaving, CONTENT_VIEW_ENDED carrying dwell_seconds, end_reason, content_id, the echoed request_id, and the impressions (what else was on screen) to the same write door, POST /api/ingest.

Per ingest:

1. Normalize + append to log & buffer.
2. UserModelUpdater.refresh rebuilds the model.
3. Each viewed item becomes a continuous engagement strength in [−1, 1] via engagement_strength() (engagement.py:53): 0.4·dwell_ratio + 0.3·completion + 0.2·revisit + 0.1·survey, classified by classify_outcome() (positive ≥ 0.30).
4. Positive items feed eng_affinity, the weighted taste vector, and the recency vector; negatives and un-engaged impressions (soft negatives, 0.30) feed tag_aversion.
5. The survey ↔ engagement blend (Step 3) now has eng_trust > 0, so browsing begins to shape the model alongside the still-present survey seeds. The visitor flips warm after the first positive.

The taste vector and recency vector capture meaning, not just tags. Every story has an embedding: a list of numbers produced by a language model that places stories with similar meaning near each other. The taste vector is the average of the embeddings of the stories the visitor liked (weighted by how strongly), a single point standing for their overall taste; score_semantic scores a candidate by how close its embedding sits to it. The recency vector is simply the embedding of the story they viewed most recently, which score_recency uses for the "more like what you just read" nudge. Together these let the engine surface stories that feel related even when they carry different tags.

If the bandit is running online (bandit_online=True, off by default), _online_bandit_update (api.py:277) also fires: it looks up the served feature vector for the (request_id, content_id) reward, computes the same engagement_strength as the reward, and nudges θ via LinearBandit.update, persisted to BANDIT_STATE_PATH and made idempotent (one reward per impression).

---

07 Warm loop

Every view sharpens the next recommendations.

flowchart LR
  R["recommend"] --> V["view"]
  V --> U["update model"]
  U --> R

Same GET /api/recommend, but the model is now warm, so recommend_for_signals takes the warm path:

• Candidate generation widens from the filter set to the union of semantic recall (taste vector) + tag recall (top affinity tags), minus everything already seen (seen = positives ∪ negatives ∪ viewed), still intersected with any filter/near constraint.
• Scoring is unchanged (Step 5), but now semantic and recency have real signal, and the tag scorer blends survey + engagement per the current eng_trust. Aversion actively demotes disliked themes.
• MMR + distractor apply as before.

Every subsequent view loops back through Step 6, so the model tracks the visitor in near-real-time: survey-led at first, engagement-led as the visit goes on, while tag matching stays the dominant lever throughout.

flowchart LR
    A[recommend] --> B[view CONTENT_VIEW_ENDED]
    B --> C[ingest]
    C --> D[refresh model]
    D --> A
    B -. request_id joins impression → reward .-> A

---

Wiring & operations

• Composition root: ai_engine.recsys.composition builds the Components bundle and reads env overrides. Weights: RECSYS_W_TAG, RECSYS_W_SEMANTIC, … ; policy: RECSYS_RANKING_MODE, RECSYS_BANDIT_*, BANDIT_STATE_PATH; infra: REDIS_URL, QDRANT_API_URL. Tenancy via the X-Tenant-Id header.
• App factory / serving: create_app() (api.py:1056) assembles the FastAPI app; uvicorn ai_engine.recsys.api:app is the entry. Health at GET /health, inspector UI at GET /dashboard. See AI Engine API and Online serving model.
• Auth: POST /api/ingest and the usermodel/ops/admin routers require INGEST_API_KEY; GET /api/recommend is public by default (SERVING_REQUIRES_KEY=1 to lock it per-tenant).

---

Symbol index

Each stage → the module in the Python code reference.

Stage	Primary symbols
Ingest & normalize	api, normalize_events
Surveys → tags	survey_affinity, extract_demographics
Model build	build_user_signals, UserModelUpdater
Engagement	engagement_strength, classify_outcome
Recommend	Recommender, score_features
Scoring	scorers, fusion
Learned ranking	LinearBandit
Config	RecConfig, FusionWeights
Wiring	composition, create_app