Open data is plentiful; good, well-documented open data takes a little more finding.
Where to Look
- Government open-data portals: national and local statistics, transport, environment, health.
- International organisations: the World Bank, OECD, UN agencies.
- Scientific agencies: NASA, national weather and geological services.
- Research repositories: the UCI Machine Learning Repository, Zenodo, institutional repositories.
- Machine learning hubs: Hugging Face datasets, Kaggle datasets.
- Curated catalogues: such as the glitchdata hub, which documents and links datasets to their publishers.
Judging Suitability
- Relevance: does it contain the variables and time period you need?
- Documentation: is there a clear description of fields, units and collection method?
- Licence: does it allow your intended use?
- Quality: missing values, duplicates, obvious errors.
- Representativeness: does it cover the population you care about?
- Maintenance: is it updated, and are versions tracked?
Beware Common Traps
- Datasets copied and re-uploaded without their original documentation or licence.
- "Toy" datasets that are much cleaner than real-world data.
- Datasets with known ethical problems or retracted sources.
Go to the Source
Prefer the original publisher's copy, which is most likely to be current and correctly licensed, and cite it properly.