Data Classes

All emdfile data classes inherit from the Node class, which adds core tree-building and metadata storage functionality. Each additionally has its own data and built-in metadata interface.

Array

class emdfile.Array(data: ndarray, name: str | None = 'array', units: str | None = '', dims: list | None = None, dim_names: list | None = None, dim_units: list | None = None, slicelabels=None)

Array instances store N-dimensional array-like data.

Instantiation

For some numpy array x

>>> ar = Array( x )

creates an Array instance. Calibrations may be passed on instantion

>>> ar = Array(
>>>     np.ones((20,20,256,256)),
>>>     name = '4ddatacube',
>>>     units = 'intensity',
>>>     dims = [
>>>         [0,5],
>>>         [0,5],
>>>         [0,0.01],
>>>         [0,0.01]
>>>     ],
>>>     dim_units = [
>>>         'nm',
>>>         'nm',
>>>         'A^-1',
>>>         'A^-1'
>>>     ],
>>>     dim_names = [
>>>         'rx',
>>>         'ry',
>>>         'qx',
>>>         'qy'
>>>     ],
>>> )

“Stack-arrays” are constructed by passing the slicelables argument

>>> ar = Array(
>>>     np.ones((4,50,50)),
>>>     name = 'test_array_stack',
>>>     units = 'intensity',
>>>     dims = [
>>>         [0,2],
>>>         [0,2]
>>>     ],
>>>     dim_units = [
>>>         'nm',
>>>         'nm'
>>>     ],
>>>     dim_names = [
>>>         'rx',
>>>         'ry'
>>>     ],
>>>     slicelabels = [
>>>         'a',
>>>         'b',
>>>         'c',
>>>         'd'
>>>     ]
>>> )

representing a set of M arrays, each of shape N-1 all calibrated by one set of object calibrations, given Array shape N and initial dimension extent M. Above, M is 4 and the arrays are called ‘a’, ‘b’, ‘c’, ‘d’.

The dims argument calibrates the array axes. Constant pixel sizes may be specified as length 2 lists, which will extrapolate linearly. For non- linear pixel extents an array of the dim length should be specified, e.g.

>>> x = np.logspace(0,1,100)
>>> y = np.sin(x)
>>> ar = Array(
>>>     y,
>>>     dims = [
>>>         x
>>>     ]
>>> )

Data Access

For an Array instance ar, data can be accessed as

>>> ar.data

or can be accessed using numpy-like slicing into the object itself, e.g.

>>> ar[:]

For stack-arrays, a slice called ‘a’ can be retrieved with

>>> ar['a']

Dimension Vectors

The set of all dim vectors can be retrieved with

>>> ar.dims

and the n’th dim vector with

>>> ar.get_dim(n)

Dim vectors should be modified using

>>> ar.set_dim

to write a new dimension vector or

>>> ar.set_dim_units
>>> ar.set_dim_name

to modify dim vector metadata only.

Shape Information

Shape information is accessible as normal through

>>> ar.data.shape

Additionally, the following shape properties account for stack arrays

>>> ar.depth    # 0 for non stacks; # arrays for stacks
>>> ar.rank     # N for non stacks; N-1 for stacks
>>> ar.shape    # length N for non stacks; length N-1 for stacks

__init__(data: ndarray, name: str | None = 'array', units: str | None = '', dims: list | None = None, dim_names: list | None = None, dim_units: list | None = None, slicelabels=None)

Parameters:

• data (np.ndarray)

• name (str)

• units (str) – units for the pixel values

• dims (variable) – specify calibration vectors for each axis of the data array. Valid values for each element of the list are None, a number, a 2-element list/array, or an M-element list/array where M is the extent of the corresponding array dimension. If None is passed, the dim will be populated with integer values starting at 0 and its units will be set to pixels. If a number is passed, the dim is populated with a vector beginning at zero and increasing linearly by this step size. If a 2-element list/array is passed, the dim is populated with a linear vector with these two numbers as the first two elements. If a list/array of length M is passed, this is used as the dim vector. If dims recieves a list of fewer than N arguments for an N-dimensional data array, the extra dimensions are populated as if None were passed, using integer pixel values. If the dims parameter is not passed, all dim vectors are populated this way.

• dim_units (list) – the units for the calibration dim vectors. If nothing is passed, dims vectors which have been populated automatically with integers corresponding to pixel numbers will be assigned units of ‘pixels’, and any other dim vectors will be assigned units of ‘unknown’. If a list with length < the array dimensions, the passed values are assumed to apply to the first N dimensions, and the remaining values are populated with ‘pixels’ or ‘unknown’ as above.

• dim_names (list) – labels for each axis of the data array. Values which are not passed will be autopopulated with the name “dim#” where # is the axis number.

• slicelabels (None or True or list) – if not None, array will be promoted to a stack array - see object docstring for details. If a list is passed it should specify the sub-array names.

Return type:

Array

dim(n)

Return the n’th dim vector

get_dim(n)

Return the n’th dim vector

get_dim_name(n)

Get the n’th dim vector name

get_dim_units(n)

Return the n’th dim vector units

set_dim(n: int, dim: list | ndarray, units: str | None = None, name: str | None = None)

Sets the n’th dim vector, using dim as described in the Array documentation. If units and/or name are passed, sets these values for the n’th dim vector.

Parameters:

• n (int) – specifies which dim vector

• dim (list or array) – length must be either 2, or match the length of the n’th axis

• units (str)

• name (str)

set_dim_name(n: int, name: str)

Sets the n’th dim vector name to name.

Parameters:

• n (int) – which dim vector

• name (str) – new name

set_dim_units(n: int, units: str)

Sets the n’th dim vector units to units.

Parameters:

• n (int) – which dim vector

• units (str) – new units

to_h5(group)

Calls Node.to_h5 to greate the group’s node and write its metadata. Then writes Array data, calibration vectors, units, and any stack/label info.

Parameters:

group (h5py Group)

Return type:

(h5py Group) the new array’s Group

PointList

class emdfile.PointList(data: ndarray, name: str | None = 'pointlist')

PointList instances represent sets of points in some M dimensional space. Each dimension is given by a named field and has its own dtype. See also the documentation for numpy structured arrays.

Instantiation

For some numpy structured array like

>>> x = np.ones(
>>>     10,
>>>     dtype = [('x',float),('y',int)]
>>> )

then calling

>>> pl = PointList(
>>>     x,
>>>     name = 'my_pointlist',
>>> )

will create a pointlist of length 10 with fields ‘x’ and ‘y’.

Data Access

The data can be accessed by

>>> pl.data

or by numpy-like slicing into the object directly

>>> pl[:]

and he individual fields can be accessed by slicing like

>>> pl['x']

Properties & Methods

The following are valid pointlist properties or constructions

>>> pl.dtype
>>> pl.fields
>>> pl.length == len(pl)

The following are valid methods

>>> pl.copy()                 # return a copy
>>> pl.add(data)              # concatenates additional data
>>> pl + data                 # concatenates additional data
>>> pl.remove(mask)           # removes indicated data points
>>> pl.sort('x','ascending')  # sort by the selected field
>>> pl.add_fields(new_fields) # return a new pointlist with added fields

__init__(data: ndarray, name: str | None = 'pointlist')

Parameters:

• data (structured numpy ndarray) – the data

• name (str) – name for the PointList

Return type:

(PointList)

add(data)

Appends a numpy structured array. Its dtypes must agree with the existing data.

add_data_by_field(data, fields=None)

Add a list of data arrays to the PointList, in the fields given by fields. If fields is not specified, assumes the data arrays are in the same order as self.fields

Parameters:

data (list) – arrays of data to add to each field

add_fields(new_fields, name='')

Creates a copy of the PointList, but with additional fields given by new_fields.

Parameters:

• new_fields (list of 2-tuples, ('name', dtype))

• name (string)

copy(name=None)

Returns a copy of the PointList. If name=None, sets to {name}_copy

remove(mask)

Removes points wherever mask==True

sort(field, order='ascending')

Sorts the point list according to field, which must be a field in self.dtype. order should be ‘descending’ or ‘ascending’.

to_h5(group)

Calls Node.to_h5 to greate the group’s node and write its metadata. Then writes PointList data including the structured data array and field names and dtypes.

Parameters:

group (h5py Group)

Returns:

h5py Group

Return type:

the new pointlist’s group

PointListArray

class emdfile.PointListArray(dtype, shape, name: str | None = 'pointlistarray')

A PointListArray instance comprises a 2D grid of PointLists, each sharing a single dtype and set of fields, and each having any variable length. It therefore represents a “ragged array” in 2+1 dimensions, i.e. with two dimensions of a fixed shape and one of variable length, embedded in an M dimensional space for PointLists with M fields.

Instantiation

Calling

>>> pla = PointListArray(
>>>     [('a',float),('b',float)],
>>>     (5,5)
>>> )

will create a 5x5 PointListArray instance with fields ‘a’ and ‘b’, and

>>> dt = np.dtype([('a',float,('b',float)])
>>> for x in range(5):
>>>     for y in range(5):
>>>         pla[x,y] += np.zeros(x+y,dt)

will populate it with some data. Element assignment currently may only be made to PointLists, so using direct assignment the code above must be

>>> dt = np.dtype([('a',float,('b',float)])
>>> for x in range(5):
>>>     for y in range(5):
>>>         pla[x,y] = PointList(np.zeros(x+y,dt))

Attributes & Methods

PointListArrays include attributes

>>> pla.shape
>>> pla.dtype
>>> pla.fields

and methods

>>> pla.copy        # returns a copy
>>> pla.add_fields  # returns a copy with additional fields

__init__(dtype, shape, name: str | None = 'pointlistarray')

Creates an empty PointListArray.

Parameters:

• dtype (dtype) – the dtype of the data comprising each PointList

• shape (2-tuple of ints) – the shape of the array of PointLists

• name (str)

Return type:

(PointListArray)

add_fields(new_fields, name='')

Creates a copy of the PointListArray, but with additional fields given by new_fields.

Parameters:

• new_fields (list of 2-tuples, ('name', dtype))

• name (string)

copy(name='')

Returns a copy of itself.

get_pointlist(i, j, name=None)

Returns the pointlist at i,j

to_h5(group)

Calls Node.to_h5 to greate the group’s node and write its metadata. Then writes PointListArray data including the data itself, array shape and the dtype.

Parameters:

group (h5py Group)

Return type:

(h5py Group) the new pointlistarray’s group