File I/O

The read function works for both EMD 1.0 and older EMD 0.1 files. It reads whole files, single nodes, or selected subsets of data. The save function writes new files or appends to existing ones. Appending includes diffmerge-like behavior with options to skip or overwrite conflicting data nodes. More information follows.

Read

emdfile.read(filepath, emdpath: str | None = None, tree: bool | str | None = True, **legacy_options)

A file reader for EMD (Berkeley) files.

Both the current EMD 1.0 file format and the older EMD 0.1 format are supported. The specifications can be found at https://emdatasets.com/format/ and https://emdatasets.com/emd-0-1/, respectively.

To read a subset of a file, specify a target node with the emdpath argument and the recursion behavior with the tree argument. For files containing a single EMD tree, read the entire file by passing no arguments beyond the filepath. For files containing multiple EMD trees, trees must be read one and a time - specify a tree of interest by passing its root to emdpath.

Parameters:

  • filepath (str or Path) – the file path

  • emdpath (str or None) – to read a subset of the file, set this argument to the HDF5 goup path of a target node in the file. If None and the file contains a single root, that root is set as the target node. If None and the file contains multiple roots, a list of rootnames is returned and a warning raised; target the tree associated with a single root by passing its name to this argument. To target any other node, specify its path using '/' delimiters.

  • tree (True or False or None) – indicates what data should be loaded. If True reads the entire tree starting at and downstream of the target node. If False, read the target node only. If None, return the entire tree downstream of but excluding the target node. Note that if emdpath points to a root node, setting tree to None or True are equivalent - both return the whole data tree.

Return type:

Root or Node or Metadata

Save

emdfile.save(filepath, data, mode='w', tree=True, emdpath=None)

Saves data to an .h5 file at filepath.

Parameters:

  • filepath (str) – The file path

  • data (see below) – The data to save. May be an emdfile Node, a numpy array, a Python dictionary, or a list of objects of those types. Numpy arrays will be saved as a root containing a single Array type node and Python dictionaries will be stored as root Metadata. Lists will be stored as a flat tree, i.e. all objects stored under a single root. Lists of N roots will be stored as a set of N EMD trees. For Node-like inputs, beavior depends on the mode, tree, and emdpath arguments.

  • mode (str) – Valid modes are write (‘w’,’write’), overwrite (‘o’,’overwrite’), append (‘a’,’+’,’append’), and append-over (‘ao’,’oa’,’o+’,’+o’,’appendover’). Write mode writes a new file, and raises an exception if a file of this name already exists. Overwrite mode deletes any file of this name that already exists and writes a new file. Append and appendover mode write a new file if no file of this name exists, or if a file of this name does exist, adds new data to the file, attempting to detemine the difference between the data passed and that already present in the HDF5 file. Data not already in the file is added, and existing data is either skipped (append mode) or overwritten (appendover mode).

  • tree (True or False or None) – If data is a node, tree determines how the tree of nodes downstream of data should be treated. If True (default), the full downstream tree is saved. If False, only the node itself is saved. If None, the downstream tree is saved, but the data node itself is excluded.

  • emdpath (str or None) – In append or append-over mode, indicates where in the existing HDF5 file to graft the data. Must be a '/' delimited string pointing to an existing node. Passing the emdpath argument automatically changes the mode to append if it was set to write or overwrite. If append mode is specified and emdpath is not passed, the name of data’s root is compared to the root names in the H5 file. If the passed root name is not already in the file, a new EMD tree is added. If the name is already in the file, a diffmerge-like append is performed, comparing the trees and adding any new nodes and skipping or overwriting existing nodes according to the mode argument.