Skip to content

Working With CSV Files

Delimiters, headers, quoting, encodings and types: how to read and write CSV files without the usual surprises.

Editorial team 2 min read

CSV (comma-separated values) is the most common format for sharing tabular data — and one of the most error-prone.

Delimiters

Not every "CSV" uses commas. Semicolons are common in regions that use commas as decimal separators (the UCI wine quality data uses semicolons), and tab-separated files are common too. Check the first few lines before loading.

import pandas as pd
df = pd.read_csv("data.csv", sep=";")

Headers

Some files have no header row; others have title lines above the header. Use header=None with names=[...], or skiprows to skip preamble lines.

Quoting

Values containing the delimiter or line breaks should be enclosed in quotes. Badly quoted files cause misaligned columns — count columns per row to spot problems.

Encoding

Text can be UTF-8, Latin-1 or Windows-1252, among others. Garbled accented characters signal the wrong encoding; specify encoding= when reading and write UTF-8 when producing files.

Types

CSV stores everything as text. Check that numbers, dates and IDs are interpreted correctly — leading zeros in postcodes and account numbers disappear if read as numbers, so read such columns as strings.

Missing Values

Specify the markers used (na_values=["?", "NA", ""]).

Large Files

Read in chunks, select only needed columns, or convert to a columnar format such as Parquet for faster repeated use.

More in Data for AI

All Data for AI guides →
Data for AI Guide · 1 min

How to Read a Dataset Card

The questions a dataset card should answer — what one row is, where the data came from, its licence and its quirks — before you use it.

Data for AI 1 min read 5 Apr 2026

Data for AI Guide · 2 min

Data Quality Dimensions

Accuracy, completeness, consistency, timeliness, validity and uniqueness: a framework for checking whether data is fit for purpose.

Data for AI 2 min read 4 Apr 2026

Data for AI Guide · 2 min

Handling Missing Data

Why data goes missing, how to find out, and the options — dropping, imputing, flagging — with their trade-offs.

Data for AI 2 min read 3 Apr 2026

Data for AI Guide · 2 min

Detecting and Handling Outliers

How to spot unusual values, decide whether they are errors or genuine extremes, and treat them appropriately.

Data for AI 2 min read 2 Apr 2026