studio-lab

Taste Profile play counts

2025

•
rhizae
•
Last updated

The play-count data behind the recommender comparison: what a row of it actually records, and why the extract that was modelled is the dense middle of a distribution rather than the whole of one.

  • datasets
  • recommender-systems
  • provenance

Two files, joined on a song id. One holds 2,000,000 rows of play counts; the other holds 1,000,000 song descriptions. Between them they support exactly one kind of statement — this user played this song this many times — and the value of writing that down is that it is narrower than what a recommender trained on it appears to claim.

What one row is

A user id, a song id, and a count. Nothing else. The song file adds a title, release, artist and year, but those describe the song rather than the listening, so joining them does not add information about the person.

The count is not a rating. Nobody in this data was asked what they thought of anything. A play happened, and something caused it — a search, an autoplay queue, a recommendation the platform made — and the data does not distinguish between those causes. That is the sense in which provenance is part of the analysis: the same number means different things depending on which of them it was.

What the data cannot show

Anything about who. There are no listener attributes at all, so a finding about a group cannot be produced from this data, and one that appears to be is coming from somewhere else.

Anything about when. There are no timestamps.

Anything about the tail, once it has been filtered. The extract the models were fit to keeps users with at least 90 songs, songs with at least 120 users, and play counts of 5 or fewer, which leaves 117,876 rows across 3,155 users and 563 songs. That is a dense, well-observed core, and it is the part of a catalogue a popularity baseline is already good at. The popularity baseline was harder to beat than expected is that result, and it is difficult to separate from the shape of the filtering.

How it was read

Reading a recommender notebook in the open loads a course-supplied copy of these files from a mounted drive rather than downloading them from the distribution above. The commit is pinned, so the code that read them can be inspected, but the copy itself is not addressable — which is the one provenance gap here that could actually be closed.

Sources