The model

Riffle does not wait for a repository to accumulate history before it can rank. One global model serves every repository from the first pull request, and each repository earns its own adjustment as outcomes arrive.

Two layers

The global model is gradient-boosted. The repository layer is deliberately light, so it can be refitted often and cheaply.

How the repository layer earns its weight

%% caption: Illustrative shape only. How layer_weight grows is still open.
xychart-beta
  x-axis "mature labels" [none, ·, ·, ·, ·, ·, ·, ·, many]
  y-axis "layer_weight" 0 --> 1
  line [0, 0.02, 0.07, 0.2, 0.45, 0.72, 0.88, 0.96, 0.99]

• layer_weight starts at 0.0 and grows with the number of mature labels. There is no cutover and no threshold to argue about.
• Each layer is checked against its own holdout. If it ranks worse than the global model alone, its weight drops to 0.0 automatically.
• Path history is computed from git on install, so repository-specific signal exists before the first label matures.

Version pinning

A score records exactly what produced it. model_version covers the global model, the repository layer, the feature extractor and the compiled rules hash. A request never mixes a new model with an old feature extractor.

Training corpus

The first global model trains on 50 public repositories across five domains: 1,899,996 pull requests, open and closed, measured on 48 of them from each repository's Pull Requests tab between July and September 2026. rust-lang/rust and envoyproxy/envoy are not measured yet.

Forty repositories train the model; ten are held out entirely, to measure how it ranks repositories it has never seen. The target there is a ROC-AUC of 0.65 or better, following Kamei et al., 2016↗. Closed counts mix merged, rejected, auto-closed and bot pull requests; bot, backport and sync PRs are filtered before training.

The 50 repositories

Every repository in the first training corpus, by domain, with its measured pull request counts.

Systems, languages and developer tools

Web and application frameworks

Databases and data systems

Cloud, infrastructure and observability

ML, science, media and end-user software

The next model: 100 repositories

Pull requests are measured on 98 of the 100 repositories; rust-lang/rust and envoyproxy/envoy are still pending. Most commit counts are estimates, replaced after cloning. The plan also replaces the 40 / 10 split with 70 training, 15 validation and 15 final-test repositories. Every repository, its split, and the plan's estimates are on Dataset plan: 100 repositories.