File formats and organization

File formats

Before starting a project, it is important to think about file formats as this may have implications for the life of the data. Follow these best practices to reduce the chances of data loss from software or data obsolescence.
Best practices

File formats when collecting data

  • Open and non-propriety data are preferable
  • If data must be in a proprietary format, ensure that it can easily be converted to open, non-proprietary format
  • Select formats commonly used by research community
  • More best practices...

File formats when sharing / preserving data

  • Format should be open, non-proprietary, machine-readable
  • Share multiple formats if format used by research community is typically proprietary (eg. MonaLisa_v1.psd AND MonaLisa_v1.tiff)
  • For proprietary files, indicate (using a readme file) software/hardware needed to open files
  • If compression is necessary, use lossless format
  • More best practices...

 

Recommended formats for sharing, reuse and preservation

Sources: UK Data Service, Oregon State University

Type of data Recommended formats Acceptable formats
Tabular data (with extensive metadata)
variable labels, code labels, defined missing values
.por (SPSS portable format) .sav, .dta, .mdb,.accdb
Tabular data (with minimal metadata)
column headings, variable names
.csv, .tab .txt, .xls, .xlsx, .mdb, .accdb, .dbf, .ods
Geospatial data
vector & raster data
.shp, .shx, .dbf, .prj, .sbx, .sbn, .tif, .tfw, .dwg, .gml .mdb, .mif, .kml, .dxf, .svg
Textual data .rtf, .txt, .xml .html, .doc, .docx
Image data .tif (TIFF 6.0) .jpeg, .jpg, .jp2, .gif, .tif, .tiff, .raw, .psd, .bmp, .png, .pdf
Audio data .flac .mp3, .aif, .wav
Video data .mp4, ogv, .ogg, .mj2 .avchd
Documentation and scripts .rtf, .pdf, .xhtml, .htm, .odt ..txt, .doc, .docx, .xls, .xlsx, .xml
Chemistry data
spectroscopy
.jdx (JCAMP)  

Consult the annually updated Library of Congress Recommended Formats Statement for more information on recommended file formats.

 
Updated: Thursday 18 February 2021
Back to top Back to top
https://library.concordia.ca/research/data/data-files.php?guid=file-formats
library.concordia.ca
crawl=CC-MAIN-2021-39
46187
application/http; msgtype=response
sha1:TTGN3MFIJNRCO6DJJHW7UFH23OXTXCSE
<urn:uuid:e4964a81-d28a-465a-8a07-40cfdec31622>
2021-09-19T05:45:27Z
132.205.204.100
text/html
sha1:3SZKXN347YTQ2QDPDUGYY5YJLEJQKN4T
<urn:uuid:ccd4ce1c-3a94-4ee4-b112-c56c08f219a7>
https://library.concordia.ca/research/data/data-files.php?guid=file-formats
response
<urn:uuid:b22bd404-9ff4-476f-8db0-56c1ecf19c8c>
isPartOf: CC-MAIN-2021-39
publisher: Common Crawl
description: Wide crawl of the web for September 2021
operator: Common Crawl Admin (info@commoncrawl.org)
hostname: ip-10-67-67-150
software: Apache Nutch 1.18 (modified, https://github.com/commoncrawl/nutch/)
robots: checked via crawler-commons 1.2-SNAPSHOT (https://github.com/crawler-commons/crawler-commons)
format: WARC File Format 1.1
conformsTo: https://iipc.github.io/warc-specifications/specifications/warc-format/warc-1.1/