The Data Collection Handbook · Part V. The Other Side of the Table
Chapter 14. How to Buy Data Well
In progress: this is the chapter plan. The full chapter, its charts, and its runnable experiment publish on this page.
What this chapter answers
I sell data for a living, so consider this chapter written against my own interest: it teaches buyers to evaluate vendors the way an engineer would, without writing code. A structured sample audit, the questions that expose weak operations, contract terms that map to measurable things, and the discipline to walk away.
What you will learn
- The code-free sample audit: fill rates per column, duplicate rates, freshness spread, plausibility spot checks, and hand-checking rows against the live source.
- How to test a coverage claim instead of trusting it.
- The seven questions that separate serious operators from resellers, starting with: how do you know when your collection breaks?
- SLAs and remedies that map to measurable things: delivery manifests, verification on both sides, recredits for bad data, exit portability.
- Usage rights as their own question: internal use, resale, model training, retention, and what happens to derived data when the contract ends.
- When the right decision is not to buy, and not to collect either.
In this chapter
- The buyer's disadvantage. The vendor sees the pipeline; you see a file.
- Auditing a sample delivery. The one-page audit a spreadsheet can run.
- Testing the coverage claim. Two samples, or the source's own public counts.
- The seven questions. And how to read the answers.
- SLAs and remedies. Terms that are measurable, or theater.
- Usage rights. The contract clauses that matter now.
- Walking away. Hostile source, murky rights, negative economics.
The experiment
The question: Can a buyer-side audit, using only the delivered files, reliably tell an honest delivery from a padded one?
Two simulated vendor deliveries of the same catalog ship with the chapter: one honest, with its gaps openly present, and one padded with disguised duplicates, re-dated stale records, and filler rows. The audit runs fill rates, near-duplicate detection, freshness distributions, and a coverage cross-check, mirroring a printable checklist a spreadsheet user can follow by hand. The data and code ship with the chapter, sized to run offline on a laptop with one command.
My hypothesis is that the padded vendor fails at least three of the five audit checks on internal consistency alone, with no ground truth needed. If a simulated cheat can be caught this easily, a buyer has no reason to skip the audit.