Common Scientific Data Formats
Scientific Data Formats
Descriptions of Common Scientific Data Formats. Note that these are generally self-describing (so metadata is contained within the file), with some caveats. Additionally, there are typically tools developed to work with and explore data in these formats developed for specific communities (climate, weather, oceanography, remote sensing), which are not explored here but may be worth an extra look if you are going to make regular use of data in these formats.
- Overview of Common Scientific Data Formats used for Earth Sciences
- Wikipedia's Take on Data Formats
- Climate Data Analysis Tools and Methods provides a summary overview of tools and formats used for climate analysis supported by NSF and NCAR
Links to Specific Data Formats
- NetCDF home page: http://www.unidata.ucar.edu/software/netcdf/
- HDF home page: http://hdf.ncsa.uiuc.edu/
- GRIB (GRIdded Binary or General Regularly-distributed Information in Binary form): Wikipedia overview
Tutorials for Working with Common Scientific Formats in Python
- General guidance for reading multiple data file format types with Python
- Flat ASCII files
- NetCDF files
- HDF5 files (HDF and HDF5 files are not the same thing)