Prototyping an Online Product Recommender in Python

Overview

Traditional recommendation approaches such as Collaborative Filtering remain widely adopted, yet they come with notable constraints. They are particularly vulnerable to the cold-start problem, where new users lack sufficient interaction history, and they depend heavily on long-term behavioral data. As a result, they frequently overlook real-time contextual signals, including time of day, device type, location, or session intent. This can prevent them from capturing situational preferences, such as someone preferring coffee in the morning but pizza in the evening.

Contextual Multi-Armed Bandits (CMAB) address these gaps through online learning.

As a practical form of reinforcement learning, CMAB balances two goals in real time:

1. Exploitation: Recommending what is known to work.
2. Exploration: Trying less-tested options to discover new favorites.

By conditioning decisions on live context, CMAB adapts instantly to changing user behavior.

Why CMAB?

• Beyond A/B Testing: Instead of finding a single global winner, CMAB enables 1:1 personalization, selecting the best option for this user in this context.
• Real-Time Adaptation: Unlike batch-trained models that quickly become stale, CMAB updates incrementally, making it ideal for news/products recommendation, dynamic pricing, or inventory-aware ranking.

Several CMAB implementations exist, including Vowpal Wabbit and River ML. In this post, we use Mab2Rec for offline policy evaluation and MABWiser to build the product recommender prototype.

Data Streaming Opportunity

CMAB performs well in data streaming environments. Integrated with platforms like Kafka and Flink, it learns directly from event streams, creating a feedback loop that responds to trends and shifts in user intent in sub-seconds.

In this series, Part 1 (this post) builds a complete Python prototype to validate the algorithm and simulate user behavior. Part 2 will scale this to a distributed, event-driven architecture.

Tech Stack

We are building this prototype using Python 3.11.

Engineering Note: We explicitly chose Python 3.11 because parts of our stack (specifically mabwiser dependencies) rely on older versions of pandas (< 2.0). On Python 3.12+, installing these dependencies often triggers long compilation times or failures due to missing binary wheels.

For the same reason, requirements.txt floors flair at 0.12. TextWiser only asks for flair>=0.9, and with pandas/numpy held below 2.0 the resolver otherwise backtracks to flair 0.11.3, which hard-pins sentencepiece==0.1.95. That release predates Python 3.11 and has no matching wheel, so the install dies trying to build it from source.

We use uv for Python environment management. The core libraries include:

• MABWiser: The engine. It implements the core Contextual Bandit algorithms.
• Mab2Rec: The vehicle. A high-level wrapper that streamlines Recommender System pipelines.
• TextWiser: For converting raw text features into numerical embeddings.
• scikit-learn: For feature scaling and encoding.
• Faker & Pandas: For synthetic data generation and simulation.

The development environment can be constructed as follows:

 1$ git clone https://github.com/jaehyeon-kim/benchtop.git
 2$ cd benchtop/product-recommender
 3$ uv python install 3.11
 4$ uv venv --python 3.11 venv
 5$ source venv/bin/activate
 6(venv) $ uv pip install -r requirements.txt
 7(venv) $ uv pip list | grep -E "mab|wiser|flair|panda|numpy|scikit|faker"
 8# Using Python 3.11.15 environment at: venv
 9# faker                              40.36.0
10# flair                              0.15.1
11# mab2rec                            1.3.1
12# mabwiser                           2.7.4
13# numpy                              1.26.4
14# pandas                             1.5.3
15# scikit-learn                       1.9.0
16# textwiser                          2.0.3

📂 Source Code for the Post

The source code for this post is available in the product-recommender folder of the benchtop GitHub repository. It is one of the Benchtop projects, which run locally from a fresh clone.

Data Generation

We first need product and user data to generate the required features.

Products

We utilize a set of 200 raw products, each containing a product ID, name, text description, price, and high-level category.

Here is a list of sample products:

Users

We generate 1,000 Synthetic Users using Faker. Each user is assigned static attributes like Age, Gender, Location, and Traffic Source. These attributes will serve as the “Context” for our Bandit.

Here is a sample of our user base:

Note that street address, postal code, city, state, and country are omitted, as only latitude and longitude are used for feature generation.

Feature Engineering

Bandit algorithms operate on numerical vectors, not raw text. In other words, they cannot interpret "Burger" unless it is converted into numbers. To address this, we developed a transformation pipeline to properly prepare our data:

1. Product Features: We used TextWiser to convert raw product descriptions into vector embeddings. This allows the model to understand that “Burger” and “Sandwich” are semantically closer than “Burger” and “Headphones”. We also applied One-Hot Encoding to categories (Product Category) and MinMax scaling to the price. Finally, we added a binary feature, is_coffee, which is set to 1 for coffee products (e.g., espresso, cappuccino) and 0 otherwise.
2. User Features: Similar to the product features, we applied One-Hot Encoding to categories (Gender and Traffic Source) and MinMax scaling to numerical fields (Age, Latitude, and Longitude).
3. Pipeline Artifacts: We save these transformers as preprocessing_artifacts.pkl. This allows our system to instantly transform any new user/product record into a compatible feature vector during inference.

Sample Processed Product Features:

Notice how the description is now represented by txt_0…txt_9 embeddings.

Sample Processed User Features:

Notice that Age, Latitude and Longitude are normalized between 0 and 1, and categorical fields are binary.

Bandit History Simulation

To evaluate whether our model can truly learn user behavior, we need a controlled Ground Truth, which is an Oracle that determines the likelihood of a simulated user clicking on a recommendation.

Crucially, this Oracle is hidden from the model. The model’s task is to infer these patterns purely from trial and error.

We also inject Dynamic Context features like Time of Day and Day of Week into the user profile at the moment of interaction. These temporal signals create realistic, fluctuating patterns that the model must adapt to.

Simulation Logic

The simulation is implemented as two functions, click_probability and will_click, and we define specific rules that govern user behaviour:

• Start from a low base logit (−2.5) to model generally low click probability.
• Rule 1: Morning coffee preference: if the user is browsing in the morning and the item is a coffee product, add a strong positive boost to the score.
• Rule 2: Weekend comfort food: if the session is on a weekend and the item is Pizza or Burgers & Sandwiches, add a moderate positive boost.
• Rule 3: Budget sensitivity: if the user is young (normalized age < 0.25) and the item is expensive (normalized price > 0.8), apply a strong negative penalty.
• Rule 4: Traffic source bias: if the user arrived via Search, add a small intent-based boost.
• Convert the final logit score into a click probability using a sigmoid function, then sample a Bernoulli trial to simulate whether a click occurs.

 1# product-recommender/recommender/engine/simulation.py
 2def click_probability(user_ctx: dict, item_ctx: dict) -> float:
 3    """
 4    Returns the chance that a user clicks an item: the hidden formula the bandit learns.
 5
 6    Args:
 7        user_ctx (dict): The user and time features.
 8        item_ctx (dict): The product features.
 9
10    Returns:
11        float: The click probability, from 0 to 1.
12    """
13    score = -2.5  # the base logit, a low probability
14
15    # Rule 1: Morning Coffee
16    if user_ctx.get("is_morning") == 1 and item_ctx.get("is_coffee") == 1:
17        score += 2.5
18
19    # Rule 2: Weekend Comfort Food
20    if user_ctx.get("is_weekend") == 1 and (
21        item_ctx.get("cat_Pizzas") == 1 or item_ctx.get("cat_Burgers & Sandwiches") == 1
22    ):
23        score += 1.8
24
25    # Rule 3: Budget Constraint (age and price are scaled from 0 to 1)
26    if user_ctx.get("age", 0.5) < 0.25 and item_ctx.get("price", 0.5) > 0.8:
27        score -= 3.0
28
29    # Rule 4: Traffic Bias
30    if user_ctx.get("traffic_source_Search") == 1:
31        score += 0.5
32
33    return 1 / (1 + np.exp(-score))
34
35
36def will_click(user_ctx: dict, item_ctx: dict, fake: Faker) -> int:
37    """
38    Decides whether a user clicks an item, with the formula's probability.
39
40    Args:
41        user_ctx (dict): The user and time features.
42        item_ctx (dict): The product features.
43        fake (Faker): The seeded Faker, whose random generator decides.
44
45    Returns:
46        int: 1 for a click, 0 for none.
47    """
48    return 1 if fake.random.random() < click_probability(user_ctx, item_ctx) else 0

Data Preparation

We generate 10,000 historical events to serve as our “Offline Training” dataset. This process involves picking a random user and a random product, then asking the Oracle “Did they click?”.

Because the user and product are matched randomly (not by a recommender), the Average Click Rate (CTR) is naturally low. In this example, it is around 13.65%, and this serves as our baseline.

💡 There are three main scripts for this post: recommender.run.prepare for feature engineering and bandit history simulation, recommender.run.evaluate for offline policy evaluation, and recommender.run.local for running product recommendation locally. Each script accepts a --seed argument, which defaults to 1237. As long as the seed remains the same, running the scripts will produce identical outputs.

 1(venv) $ python -m recommender.run.prepare
 2[2026-01-26 19:16:09] INFO    : Generating 1000 synthetic users...
 3[2026-01-26 19:16:09] INFO    : Saved raw users to: .../users.csv
 4[2026-01-26 19:16:09] INFO    : Starting Feature Engineering...
 5[2026-01-26 19:16:09] INFO    : Saved User Features: (1000, 11)
 6[2026-01-26 19:16:10] INFO    : Saved Product Features: (200, 21)
 7[2026-01-26 19:16:10] INFO    : Saved Pipeline Artifacts to: .../preprocessing_artifacts.pkl
 8[2026-01-26 19:16:10] INFO    : Loaded 1000 users and 200 products.
 9[2026-01-26 19:16:10] INFO    : Generating 10000 events...
10[2026-01-26 19:16:10] INFO    : Done. Saved Training Log to .../training_log.csv
11[2026-01-26 19:16:10] INFO    : Avg Click Rate: 13.65%
12[2026-01-26 19:16:10] INFO    : Data Preparation Complete.

The main dataset (training_log.csv) combines user features, dynamic context (e.g., is_morning), product ID, and the interaction result (response):

The user’s 10 features and the 5 time flags, 15 numbers in all, are the context vector $x$ that the bandit sees for a visit. The product features are not part of it: the bandit keeps a separate model for each product instead, and only the Oracle reads the product features.

Offline Policy Evaluation

We benchmarked several policies using Mab2Rec on the 10,000 historical events.

Candidate Policies

• Random: The baseline. Recommends items blindly.
• Popularity: Recommends items drawn at random, each weighted by its global click rate.
  • Result: Mediocre (AUC ~0.59). While better than random, it still fails to capture specific rules, such as “Morning Coffee” vs. “Weekend Pizza.”
• LinGreedy: Disjoint Linear Regression with $\epsilon$-greedy exploration.
• LinUCB (The Winner): a separate (disjoint) linear regression for each product, with an Upper Confidence Bound.
• LinTS (Thompson Sampling): Bayesian regression that samples from a probability distribution.

Winner: LinUCB

LinGreedy achieved the highest ranking accuracy (AUC ~0.88). Its click rate (CTR ~11%) rests on only about 50 test visits, so it is close to LinUCB rather than behind it.

LinUCB is the practical winner. It ranks almost as well (AUC ~0.86), and its picks are clicked about twice as often as random picks (CTR ~20.5% against 10.2%).

This algorithm excels because it balances two competing goals:

1. Exploitation: It uses the predicted reward, $x^T \theta$, to find good items. $\theta = A^{-1} b$ holds the weights the model has learned, where $A$ records the contexts a product was shown in and $b$ the contexts it was clicked in.
2. Exploration: It adds an upper confidence bound, $\alpha \sqrt{x^T A^{-1} x}$, to the score. If the model is uncertain about a specific context (e.g., “I haven’t seen a user drink Coffee at 8 PM before”), the bound grows, boosting the score and forcing the model to test that hypothesis. $\alpha$ is a hyperparameter that sets how much exploration counts; it is 1.0 here.

So LinUCB tries the products it knows least about, while LinGreedy explores by picking at random on one visit in ten.

 1(venv) $ python -m recommender.run.evaluate
 2Running Benchmark... (This trains and scores all models automatically)
 3--------------------------------------------------------------------------------
 4Available Metrics: ['AUC(score)@5', 'CTR(score)@5', 'Precision@5', 'Recall@5']
 5            AUC(score)@5  CTR(score)@5  Precision@5  Recall@5
 6Random          0.550000      0.102041     0.003876  0.019380
 7Popularity      0.592857      0.192308     0.007752  0.038760
 8LinGreedy       0.885185      0.117647     0.004651  0.023256
 9LinUCB          0.860317      0.204545     0.006977  0.034884
10LinTS           0.640798      0.211538     0.008527  0.042636
11ClustersTS      0.550505      0.153846     0.004651  0.023256
12--------------------------------------------------------------------------------

How the Click Rate Is Measured

The history records only the one product each visit was shown. If a policy would have shown a different product, what the user would have done is unknown. So the benchmark uses the replay method from Li et al. (2011). For each test visit it asks the policy for its five recommendations. If the logged product is among them, the visit is kept and its click counted. Otherwise the visit is skipped. Jurity calls this matching, and keeping the visits that pass a test while discarding the rest is a form of rejection sampling.

The estimate is fair only because the history picked each product uniformly at random. Every product then had the same chance of being logged, so the kept visits are a fair sample of the visits on which the policy’s choice was shown. The chance is 5 in 200, or 1 in 40, whatever the policy.

Four of LinUCB’s test visits:

On weekend evenings LinUCB’s five are mostly pizzas, and on weekday mornings they include coffees, as the Oracle rewards. Visit 8003 was clicked, but it does not count, because LinUCB would not have shown Dim Sims. Across the 2,000 test visits, 44 are kept and 9 of those were clicked, so LinUCB’s CTR(score)@5 is 9 / 44 = 20.5%.

So the 20.5% is the click rate of LinUCB’s own choices, measured on the visits where the random history happened to show one of them. The other policies’ click rates rest on 39 to 52 kept visits each. With so few, a difference of a few points can come from chance, and a longer history would make the comparison more certain.

Simulation of the Selected Product Recommender Locally

With the model selected, we built a script to simulate the product recommender locally. This script acts as the Server, the User, and the Trainer simultaneously in a continuous loop.

Step 1: Pre-training (Offline Replay)

We don’t want to start with a “dumb” model. We load the 10,000 historical events (training_log.csv) and run model.fit(). This gives the bandit a baseline knowledge of the world before the live loop begins.

Step 2: The Online Loop

We simulate a sequence of user visits:

1. User Arrival: Pick a random user from the pool.
2. Contextualize: Inject a simulated timestamp (e.g., varying between Mon 08:00 AM and Sat 09:00 PM). This is the key “Context” the model must react to.
3. Recommend: LinUCB calculates scores for all 200 products and returns the Top 5.
4. Reaction: The will_click Oracle decides if the user clicks.
5. Online Update: We call model.partial_fit() with the reward, 1 or 0, for one product: the clicked one, or the first one shown when nothing was clicked. This updates that product’s matrix $A$ and vector $b$ instantly. The very next recommendation will reflect this new learning.

Here is a sample of 30 recommendation records from the local simulation.

 1(venv) $ python -m recommender.run.local
 2[2026-02-05 15:47:43] INFO    : Loaded 1000 users
 3[2026-02-05 15:47:43] INFO    : Loading artifacts...
 4[2026-02-05 15:47:48] INFO    : Loaded 200 products.
 5[2026-02-05 15:47:48] INFO    : Pre-training model from history...
 6[2026-02-05 15:47:48] INFO    : Model pre-trained on 10000 events.
 7
 8--- STARTING LIVE LOOP (30 visits) ---
 9
10User 0153 (56 yo) @ Fri 00:58 -> Recs: [200, 124, 015, 058, 011] -> Clicked: None (❌), Reward: 0 on 200
11User 0909 (21 yo) @ Sat 15:53 -> Recs: [038, 040, 017, 020, 046] -> Clicked: None (❌), Reward: 0 on 038
12User 0406 (30 yo) @ Sat 05:24 -> Recs: [020, 041, 008, 055, 040] -> Clicked: 020  (✅), Reward: 1 on 020
13User 0317 (31 yo) @ Sat 05:38 -> Recs: [008, 055, 057, 059, 139] -> Clicked: 055  (✅), Reward: 1 on 055
14User 0246 (44 yo) @ Mon 02:04 -> Recs: [015, 058, 057, 011, 124] -> Clicked: None (❌), Reward: 0 on 015
15User 0974 (61 yo) @ Fri 01:16 -> Recs: [058, 073, 124, 074, 051] -> Clicked: None (❌), Reward: 0 on 058
16User 0234 (26 yo) @ Thu 12:16 -> Recs: [036, 103, 002, 186, 070] -> Clicked: None (❌), Reward: 0 on 036
17User 0360 (35 yo) @ Sat 20:23 -> Recs: [058, 051, 008, 042, 018] -> Clicked: 042  (✅), Reward: 1 on 042
18User 0513 (51 yo) @ Sun 05:37 -> Recs: [051, 059, 043, 014, 020] -> Clicked: 051  (✅), Reward: 1 on 051
19User 0640 (33 yo) @ Mon 00:49 -> Recs: [073, 124, 023, 147, 074] -> Clicked: None (❌), Reward: 0 on 073
20User 0363 (31 yo) @ Fri 23:35 -> Recs: [200, 126, 085, 058, 018] -> Clicked: 018  (✅), Reward: 1 on 018
21User 0718 (58 yo) @ Sat 23:05 -> Recs: [018, 036, 040, 042, 020] -> Clicked: 018  (✅), Reward: 1 on 018
22User 0390 (49 yo) @ Tue 00:56 -> Recs: [147, 165, 020, 089, 047] -> Clicked: None (❌), Reward: 0 on 147
23User 0425 (39 yo) @ Thu 03:59 -> Recs: [147, 165, 062, 028, 055] -> Clicked: None (❌), Reward: 0 on 147
24User 0792 (21 yo) @ Sun 23:28 -> Recs: [042, 056, 018, 043, 046] -> Clicked: 042  (✅), Reward: 1 on 042
25User 0190 (41 yo) @ Sat 10:54 -> Recs: [192, 139, 189, 008, 055] -> Clicked: 192  (✅), Reward: 1 on 192
26User 0544 (41 yo) @ Tue 20:42 -> Recs: [018, 058, 090, 043, 147] -> Clicked: None (❌), Reward: 0 on 018
27User 0192 (17 yo) @ Sat 18:38 -> Recs: [042, 056, 018, 046, 043] -> Clicked: 042  (✅), Reward: 1 on 042
28User 0757 (55 yo) @ Thu 16:08 -> Recs: [015, 171, 165, 037, 126] -> Clicked: 126  (✅), Reward: 1 on 126
29User 0904 (60 yo) @ Sat 02:07 -> Recs: [103, 041, 017, 042, 057] -> Clicked: 042  (✅), Reward: 1 on 042
30User 0552 (39 yo) @ Tue 11:33 -> Recs: [192, 190, 189, 194, 193] -> Clicked: 192  (✅), Reward: 1 on 192
31User 0540 (36 yo) @ Sat 08:56 -> Recs: [043, 041, 192, 014, 073] -> Clicked: 043  (✅), Reward: 1 on 043
32User 0326 (26 yo) @ Wed 13:04 -> Recs: [015, 171, 165, 023, 126] -> Clicked: None (❌), Reward: 0 on 015
33User 0834 (29 yo) @ Sat 22:58 -> Recs: [051, 002, 042, 058, 036] -> Clicked: 051  (✅), Reward: 1 on 051
34User 0290 (21 yo) @ Mon 19:09 -> Recs: [058, 200, 004, 018, 085] -> Clicked: 018  (✅), Reward: 1 on 018
35User 0275 (18 yo) @ Wed 11:10 -> Recs: [189, 002, 160, 078, 103] -> Clicked: None (❌), Reward: 0 on 189
36User 0327 (23 yo) @ Wed 19:54 -> Recs: [200, 126, 018, 085, 058] -> Clicked: None (❌), Reward: 0 on 200
37User 0144 (67 yo) @ Thu 12:31 -> Recs: [087, 126, 047, 103, 034] -> Clicked: None (❌), Reward: 0 on 087
38User 0497 (60 yo) @ Sun 08:26 -> Recs: [192, 139, 008, 189, 059] -> Clicked: 192  (✅), Reward: 1 on 192
39User 0508 (64 yo) @ Tue 12:41 -> Recs: [165, 087, 026, 171, 037] -> Clicked: None (❌), Reward: 0 on 165
40
41--- END LOOP ---

Evaluation of Simulation

The system is behaving exactly as a Contextual Bandit should. It is aggressively exploiting known high-probability zones while struggling (realistically) in neutral zones.

“Weekend Pizza” Strategy Dominates

The model has learned that Weekends (Sat/Sun) are for Pizzas (Category 40s).

• User 0360 (Sat 20:23): Recommended [..., 042 (Aussie Pizza), ...] $\to$ Clicked ✅.
• User 0513 (Sun 05:37): Recommended [051 (Buffalo Pizza), 059 (Lamb Pizza)...] $\to$ Clicked ✅.
• User 0834 (Sat 22:58): Recommended [051 (Buffalo Pizza)... 042 (Aussie Pizza)] $\to$ Clicked ✅.
• Insight: The model pushes Pizzas hard on weekends regardless of the specific hour, resulting in a very high conversion rate for these users.

“Morning Coffee” Precision

The model correctly switches strategies based on the hour, even distinguishing “Weekend Morning” from “Weekend Night”.

• User 0552 (Tue 11:33): It is a Weekday Morning. The model recommended 5 Coffees [192, 190, 189, 194, 193]. The user clicked 192 (Long Black). ✅
• User 0497 (Sun 08:26): It is a Weekend, but it is Morning. The model prioritized Coffee (192, 189) over Pizza. The user clicked 192. ✅
• User 0508 (Tue 12:41): This is a great edge case. It is 41 minutes past the “Morning” cutoff (12:00). The model stopped recommending Coffee and switched to Lunch items (Steamed Veggies, Burritos). The user did not click ❌, but the behavior change proves the features are working perfectly.

Perfect Recommendation, No Interaction (Realism)

• User 0275 (Wed 11:10): The model recommended 189 (Flat White). However, the user ignored it, and the result was ❌ (No Reward).
• Why? This mimics real life. Even if the recommendation is perfect, users don’t always convert. In click_probability, the probability caps at ~50% (sigmoid of 0). This “Bad Luck” outcome confirms your evaluation pipeline is honest.

Conclusion

Across 30 simulated visits, CTR reached 53%.

This elevated CTR indicates that the model has internalized dominant temporal and demographic patterns encoded in the simulation. It aggressively exploits strong signals such as weekend pizza and morning coffee while still exploring less-certain regions.

Importantly, the simulation includes realistic noise. Even strong recommendations do not always convert, reflecting probabilistic user behavior.

Related posts

• Building a Real-Time Industrial Digital Twin with Apache Flink and Online Machine Learning - another online learning system, where a model learns on Flink as machinery wears and detects concept drift

What’s Next?

We have successfully prototyped a Contextual Bandit that learns time-based preferences. However, this Python script has major limitations for a production environment:

1. Scalability: Disjoint LinUCB maintains a matrix for every product. With 10 million products, a single server will run out of memory.
2. Latency: The training (partial_fit) blocks the inference (recommend). In a real system, you cannot make a user wait for the model to update.
3. Fault Tolerance: If the script crashes, the learned state is lost.
4. Concurrency: A single Python process cannot handle thousands of concurrent requests.

In Part 2, we will transform this prototype into an Event-Driven Architecture:

• Kafka will transport click events asynchronously.
• Flink will handle distributed, stateful model training.
• Redis will serve the model matrices for sub-millisecond inference.

Stay tuned!