The Measurement class in the center

The measurement (meas) is the central object in the pluggable structure of ixdat, and the main interface for user interaction. A measurement is an object of the generalized class Measurement, defined in the measurements module, or an inheriting *TechniqueMeasurement* class defined in a module of the techniques folder (see Techniques: ixdat’s measurement subclasses).

The general pluggable structure is defined by Measurement, connecting every measurement to a reader for importing from text, a backend for saving and loading in ixdat, a plotter for visualization, and an exporter for saving outside of ixdat. Each TechniqueMeasurement class will likely have its own default reader, plotter, and exporter, while an ixdat session will typically work with one backend handled by the db model.

Subclasses for measurement techniques

The TechniqueMeasurement subclasses are structured such that they all inherit basic properties and methods from the Measurement base class. Additionally, classes for related techniques, in particular for combined techniques, inerit from more general other technique classes for shared functionality. The figure below shows this with the ECMSMeasurement subclass inheriting both EC specific methods (e.g.: cycle selection) and MS specific methods (e.g.: gas calibration), as well as providing additional methods relevant for the combined technique (e.g.: EC calibration).

A full list of TechniqueMeasurement classes can be found in Techniques: ixdat’s measurement subclasses.

What’s in a measurement

A measurement is basically a wrapper around a collection of data_series (see The data series structure).

There are several ways of interracting with a measurement’s data_series:

• Most TechniqueMeasurements provide attribute-style access to essential DataSeries and data. For example, ECMeasurement has properties for potential and current series, as well as t, v, and j for data.

• meas.grab() is the canonical way of getting numerical data out of a measurement. Given the name of a ValueSeries, it returns two numpy arrays, t and v where t is the time (wrt meas.tstamp) and v is the value as a function of that time vector. grab takes a series name as its first argument and can also take a tspan argument in which case it cuts the vectors to return data for the specific timespan of the measurement.

• Indexing a measurement with the name of a data series returns that data series, with any time values tstamp’d at meas.tstamp

• The names of the series are available in meas.series_names.

• The raw series are available in meas.series_list.

The measurements module

Here is the full in-line documentation of the measurements module containing the Measurement class.

This module defines the Measurement class, the central data structure of ixdat

An ixdat Measurement is a collection of references to DataSeries and the metadata needed to combine them, i.e. “build” the combined dataset. It has a number of general methods to visualize and analyze the combined dataset. Measurement is also the base class for a number of technique-specific Measurement-derived classes.

A Measurement will typically be accompanied by one or more Calibration. This module also defines the base class for Calibration, while technique-specific Calibration classes will be defined in the corresponding module in ./techniques/

class ixdat.measurements.Calibration(*, name=None, technique=None, tstamp=None, measurement=None)

Base class for calibrations.

calibrate_series(key, measurement=None)

This should be overwritten in real calibration classes.

FIXME: Add more documentation about how to write this in inheriting classes.

export(path_to_file=None)

Export an ECMSCalibration as a json-formatted text file

classmethod from_dict(obj_as_dict)

Return an object of the Calibration class of the right technique

Parameters

obj_as_dict (dict) – The full serializaiton (rows from table and aux tables) of the measurement. obj_as_dict[“technique”] specifies the technique class to use, from TECHNIQUE_CLASSES

classmethod read(path_to_file)

Read a Calibration from a json-formatted text file

class ixdat.measurements.Measurement(name, technique=None, metadata=None, s_ids=None, series_list=None, c_ids=None, calibration_list=None, m_ids=None, component_measurements=None, aliases=None, reader=None, plotter=None, exporter=None, sample=None, lablog=None, tstamp=None)

The Measurement class

property aliases

series_names} pointing to where desired raw data is

TODO: get the possible aliases based on calibrations, etc, in here.

Type

Dictionary of {key

property c_ids

List of the id’s of the measurement’s Calibrations FIXME: c.id can be (backend, id) if it’s not on the active backend.

This is as of now necessary to find it if you’re only given self.as_dict()

see https://github.com/ixdat/ixdat/pull/11#discussion_r746632897

calibrate(*args, **kwargs)

Add a calibration of the Measurement’s default calibration type

The calibration class is determined by the measurement’s technique. *args and **kwargs are passed to the calibration class’s __init__.

Raises

TechniqueError if no calibration class for the measurement's technique –

property calibration_list

List of calibrations (with placeholders filled)

property calibrations

List of calibrations with any needed manipulation done.

Type

For overriding

clear_cache()

Clear the cache so derived series are constructed again with updated info

property component_measurements

List of the component measurements of which this measurement is a combination

For a pure measurement (not a measurement set), this is itself in a list.

control_series_name = None

Name (or alias) for main time variable or main time-dependent value variable, typically of the control technique

control_technique_name = None

Name of the control technique primarily used to control the experiment

copy()

Make a copy of the Measurement via its dictionary representation

correct_data(value_name, new_data)

Replace the old data for ´value_name´ (str) with ´new_data` (np array)

cut(tspan, t_zero=None)

Return a new measurement with the data in the given time interval

Parameters

• tspan (iter of float) – The time interval to use, relative to self.tstamp tspan[0] is the start time of the interval, and tspan[-1] is the end time of the interval. Using tspan[-1] means you can directly use a long time vector that you have at hand to describe the time interval you’re looking for.

• t_zero (float or str) – The time in the measurement to set to t=0. If a float, it is interpreted as wrt the original tstamp. String options include “start”, which puts t=0 at the start of the cut interval.

property data_cols

Return a set of the names of all of the measurement’s VSeries and TSeries

default_exporter

alias of CSVExporter

default_plotter

alias of ValuePlotter

essential_series_names = None

Series which should always be present

classmethod from_component_measurements(component_measurements, keep_originals=True, sorted=True, **kwargs)

Return a measurement with the data contained in the component measurements

TODO: This function “builds” the resulting measurement, i.e. it appends series

of the same name rather than keeping all the original copies. This should be made more explicit, and a build() method should take over some of the work.

Parameters

• component_measurements (list of Measurement) –

• keep_originals – Whether to keep a list of component_measurements referenced. This may result in redundant numpy arrays in RAM.

• sorted (bool) – Whether to sort the series according to time

• kwargs – key-word arguments are added to the dictionary for cls.from_dict()

Returns cls: the combined measurement.

classmethod from_dict(obj_as_dict)

Return an object of the measurement class of the right technique

Parameters

obj_as_dict (dict) – The full serializaiton (rows from table and aux tables) of the measurement. obj_as_dict[“technique”] specifies the technique class to use, from TECHNIQUE_CLASSES

get_original_m_ids_of_series(series)

Return a list of id’s of component measurements to which series belongs.

get_series(key)

Find or build the data series corresponding to key without direct cache’ing

See more detailed documentation under __getitem__, for which this is a helper method. This method (A) looks for a method for key in the measurement’s series_constructors; (B) requests its calibration for key; and if those fail appends the data series that either (Ci) are returned by looking up the key’s aliases or (Cii) have key as their name; and finally (D) check if the user was using a key with a suffix.

Parameters

key (str) – The key to look up

Returns DataSeries: the data series corresponding to key Raises SeriesNotFoundError if no series found for key

get_series_names(key)

Return list: series names for key found by (recursive) lookup in aliases

grab(item, tspan=None, include_endpoints=False, tspan_bg=None)

Return a value vector with the corresponding time vector

Grab is the canonical way to retrieve numerical time-dependent data from a measurement in ixdat. The first argument is always the name of the value to get time-resolved data for (the name of a ValueSeries). The second, optional, argument is a timespan to select the data for. Two vectors are returned: first time (t), then value (v). They are of the same length so that v can be plotted against t, integrated over t, interpolated via t, etc. t and v are returned in the units of their DataSeries. TODO: option to specifiy desired units

Typical usage::

t, v = measurement.grab(“potential”, tspan=[0, 100])

Parameters

• item (str) –

  The name of the DataSeries to grab data for TODO: Should this be called “name” or “key” instead? And/or, should

  the argument to __getitem__ be called “item” instead of “key”?

• tspan (iter of float) – Defines the timespan with its first and last values. Optional. By default the entire time of the measurement is included.

• include_endpoints (bool) – Whether to add a points at t = tspan[0] and t = tspan[-1] to the data returned. This makes trapezoidal integration less dependent on the time resolution. Default is False.

• tspan_bg (iterable) – Optional. A timespan defining when item is at its baseline level. The average value of item in this interval will be subtracted from the values returned.

grab_for_t(item, t, tspan_bg=None)

Return a numpy array with the value of item interpolated to time t

Parameters

• item (str) – The name of the value to grab

• t (np array) – The time vector to grab the value for

• tspan_bg (iterable) – Optional. A timespan defining when item is at its baseline level. The average value of item in this interval will be subtracted from what is returned.

integrate(item, tspan=None, ax=None)

Return the time integral of item in the specified timespan

join(other, join_on=None)

Join two measurements based on a shared data series

This involves projecting all timeseries from other’s data series so that the variable named by join_on is shared between all data series. This is analogous to an explicit inner join.

Parameters

• other (Measurement) – a second measurement to join to self

• join_on (str or tuple) – Either a string, if the value to join on is called the same thing in both measurements, or a tuple of two strings where the first is the name of the variable in self and the second in other. The variable described by join_on must be monotonically increasing in both measurements.

property m_ids

List of the id’s of a combined measurement’s component measurements FIXME: m.id can be (backend, id) if it’s not on the active backend.

This is as of now necessary to find it if you’re only given self.as_dict() see https://github.com/ixdat/ixdat/pull/11#discussion_r746632897

property metadata_json_string

Measurement metadata as a JSON-formatted string

multicut(tspans)

Return a selection of the measurement including each of the given tspans

classmethod read(path_to_file, reader=None, **kwargs)

Return a Measurement object from parsing a file with the specified reader

Parameters

• path_to_file (Path or str) – The path to the file to read

• reader (str or Reader class) – The (name of the) reader to read the file with. If not specified, ixdat will try to determine the reader from the file suffix.

• kwargs – key-word arguments are passed on to the reader’s read() method.

classmethod read_set(path_to_file_start=True, part=None, suffix=None, file_list=None, reader=None, **kwargs)

Read and append a set of files.

Parameters

• path_to_file_start (Path or str) – The path to the files to read including the shared start of the file name: Path(path_to_file).parent is interpreted as the folder where the file are. Path(path_to_file).name is interpreted as the shared start of the files to be appended. Alternatively, path_to_file_start can be a folder, in which case all files in that folder (with the specified suffix) are included.

• part (Path or str) – A path where the folder is the folder containing data and the name is a part of the name of each of the files to be read and combined.

• suffix (str) – If a suffix is given, only files with the specified ending are added to the file list

• file_list (list of Path) – As an alternative to path_to_file_start, the exact files to append can be specified in a list

• reader (str or Reader class) – The (name of the) reader to read the files with

• kwargs – Key-word arguments are passed via cls.read() to the reader’s read() method, AND to cls.from_component_measurements()

classmethod read_url(url, reader=None, **kwargs)

Read a url (via a temporary file) using the specified reader

rebuild_selector(selector_string=None, columns=None, extra_columns=None)

Build a new selector series for the measurement and cache it.

This can be useful if a user wants to change how their measurement counts sections (for example, only count sections when technique or file number changes)

Parameters

• selector_string (str) – The name to use for the selector series

• columns (list) – The list of demarcation series. The demarcation series have to have the same tseries, which should be the one pointed to by the meausrement’s control_series_name.

• extra_columns (list) – Extra demarcation series to include if needed.

replace_series(series_name, new_series=None)

Remove an existing series, add a series to the measurement, or both.

FIXME: This will not appear to change the series for the user if the

measurement’s calibration returns something for ´series_name´, since __getitem__ asks the calibration before looking in series_list.

Parameters

• series_name (str) – The name of a series. If the measurement has (raw) data series with this name, cached series with this name, and/or aliases for this name, they will be removed.

• new_series (DataSeries) – Optional new series to append to the measurement’s series_list. To sanity check, it must have ´series_name´ as its ´name´.

property reverse_aliases

standard_names} indicating how raw data can be accessed

Type

{series_name

property s_ids

List of the id’s of the measurement’s DataSeries FIXME: m.id can be (backend, id) if it’s not on the active backend.

This is as of now necessary to find it if you’re only given self.as_dict() see https://github.com/ixdat/ixdat/pull/11#discussion_r746632897

property sample_name

Name of the sample on which the measurement was conducted

select(*args, tspan=None, **kwargs)

cut (with tspan) and select_values (with *args and/or **kwargs).

These all work for measurements that have a default selector and/or the indicated columns: - meas.select(1, 2) - meas.select(tspan=[200, 300]) - meas.select(range(10)) - meas.select(cycle=4) - meas.select(**{“cycle number”: [20, 21]}) - `meas.select(loop_number=1, tspan=[1000, 2000]) - `meas.select(1, range(5, 20), file_number=1, tspan=[1000, 2000])

select_value(*args, **kwargs)

Return a selection of the measurement where a criterion is matched.

Specifically, this method returns a new Measurement where the time(s) returned are those where the values match the provided criteria, i.e. the part of the measurement where self[series_name] == value

Can only take one arg or kwarg! The series_name is self.selector_name if given an argument without keyword. If given a keyword argument, the kyword is the name of the series to select on. Either way the argument is the value to be selected for.

The method finds all time intervals for which self[series_name] == value It then cuts the measurement according to each time interval and adds these segments together. TODO: This can maybe be done better, i.e. without chopping series. TODO: Some way of less than and greater than kwargs.

Ideally you should be able to say e.g., select(cycle=1, 0.5<potential<1) But this is hard, see: https://github.com/ixdat/ixdat/pull/11#discussion_r677272239

select_values(*args, selector_name=None, **kwargs)

Return a selection of the measurement based on one or several criteria

Specifically, this method returns a new Measurement where the time(s) returned are those where the values match the provided criteria, i.e. the part of the measurement where self[series_name] == value

Any series can be selected for using the series name as a key-word. Arguments can be single acceptable values or lists of acceptable values. You can select for one or more series without valid python variable names by providing the kwargs using ** notation (see last example below).

Arguments without key-word are considered valid values of the default selector, which is normally self.selector_name but can also be specified here using the key-word argument selector_name. Multiple criteria are applied sequentially, i.e. you get the intersection of satisfying parts.

Examples of valid calls given a measurement meas: ``` # to select where the default selector is 3, use: meas.select_values(3) # to select for where the default selector is 4 or 5: meas.select_values(4, 5) # to select for where “cycle” (i.e. the value of meas[“cycle”].data) is 4: meas.select_values(cycle=4) # to select for where “loop_number” is 1 AND “cycle” is 3, 4, or 5: meas.select_values(loop_number=1, cycle=[3, 4, 5]) # to select for where “cycle number” (notice the space) is 2 or 3: meas.select_values([2, 3], selector_name=”cycle number”) # which is equivalent to: meas.select_values(**{“cycle number”: [2, 3]})

Parameters

• args (tuple) – Argument(s) given without keyword are understood as acceptable value(s) for the selector (that named by selector_name or self.selector_name).

• selector_name – The name of the selector to which the args specify

• kwargs (dict) – Each key-word arguments is understood as the name of a series and its acceptable value(s).

selection_series_names = ('file_number',)

Name of the default things to use to construct the selector

selector_name = 'selector'

Name of the default selector

series_constructors = {'file_number': '_build_file_number_series', 'scan_rate': '_build_scan_rate', 'selector': '_build_selector_series'}

Series which should be constructed from other series by the specified method and cached the first time they are looked up

property series_list

List of the DataSeries containing the measurement’s data

property series_names

Set of the names of the series in the measurement

property time_names

Set of the names of the VSeries in the measurement’s DataSeries

property time_series

List of the TSeries in the measurement’s DataSeries. NOT timeshifted!

property tspan

The minimum timespan (with respect to self.tstamp) containing all the data

property tstamp

The unix epoch time used by the measurement as t=0

Type

Float

property value_names

Set of the names of the VSeries in the measurement’s DataSeries

property value_series

List of the VSeries in the measurement’s DataSeries

ixdat.measurements.get_combined_technique(technique_1, technique_2)

Return the name of the technique resulting from adding two techniques