Raw data is read-only

The single most important rule for data: never edit your raw data. Not by hand, not "just this once", not to fix one typo.

Why

If you change the raw file, you lose the original forever. Nobody, including you, can check what you changed or why. A typo fixed by hand today is an unexplained difference tomorrow.

Do this instead

1. Save the raw data exactly as you received it in data/raw/.
2. Make it read-only (in your file settings or with a command).
3. Write a cleaning script that reads the raw file and writes a cleaned copy to data/processed/.
4. Every fix, even one typo, goes in the script, with a comment.

The spreadsheet trap

Spreadsheets "helpfully" change data. In 2016, Ziemann and colleagues found that about one in five genomics papers with gene lists in Excel supplementary files contained errors where gene names had been auto-converted into dates: the gene SEPT2 became "2-Sep". The problem was so persistent that in 2020 the official human gene naming committee renamed some genes (SEPT1 became SEPTIN1, MARCH1 became MARCHF1) to stop it.

Keep a record of where data came from

In data/raw/, add a short text file: where the data came from, when you downloaded it, the link, the licence, and a checksum if possible.

RememberRaw data is read-only. Every change happens in a script, so it can be seen, checked and undone.