stytra.collectors package

Submodules

stytra.collectors.accumulators module

class stytra.collectors.accumulators.Accumulator(experiment, name='', max_history_if_not_running=1000)[source]

Bases: PyQt5.QtCore.QObject

class stytra.collectors.accumulators.DataFrameAccumulator(*args, fps_calc_points=10, monitored_headers=None, **kwargs)[source]

Bases: stytra.collectors.accumulators.Accumulator

Abstract class for accumulating streams of data.

It is use to save or plot in real time data from stimulus logs or behavior tracking. Data is stored in a list in the stored_data attribute.

Specific methods for updating the stored_data list (e.g., by acquiring data from a Queue or a DynamicStimulus attribute) are defined in subclasses of the Accumulator.

Data that end up in the stored_data list must be NamedTuples where the first element is a timestamp. Therefore, stored_data of an Accumulator that is fed 2 values will be something like [(t_0, x_0, y_0), (t_0, x_0, y_0), …]

Data can be retrieved from the Accumulator as a pandas DataFrame with the get_dataframe() method.

Parameters

fps_calc_points (int) – number of data points used to calculate the sampling rate of the data.

sig_acc_reset
sig_acc_init
t
values_at_abs_time(time)[source]

Finds the values in the accumulator closest to the datetime time

Parameters

time (datetime) – time to search for

Returns

Return type

namedtuple of values

columns
header_dict

for each header name gives the column

reset(monitored_headers=None)[source]

Reset accumulator and assign a new headers list.

Parameters

monitored_headers (list of str) – List with the headers displayed by default Default value = None)

trim_data()[source]
get_fps()[source]
get_last_n(n=None)[source]

Return the last n data points.

Parameters

n (int) – number of data points to be returned

Returns

NxJ Array containing the last n data points, where J is the number of values collected at each timepoint + 1 (the timestamp)

Return type

np.array

get_last_t(t)[source]
Parameters

t (float) – Time window in seconds from which data should be returned

Returns

NxJ Array containing the last n data points, where J is the number of values collected at each timepoint + 1 (the timestamp) and N is t*fps

Return type

np.array

get_dataframe()[source]

Returns pandas DataFrame with data and headers.

save(path, format='csv')[source]

Saves the content of the accumulator in a tabular format. Choose CSV for widest compatibility, HDF if using Python only, or feather for efficient storage compatible with Python and Julia data frames

Parameters
  • path (str) – output path, without extension name

  • format (str) – output format, csv, feather, hdf5, json

is_empty()[source]
class stytra.collectors.accumulators.QueueDataAccumulator(data_queue, **kwargs)[source]

Bases: stytra.collectors.accumulators.DataFrameAccumulator

General class for retrieving data from a Queue.

The QueueDataAccumulator takes as input a multiprocessing.Queue object and retrieves data from it whenever its update_list() method is called. All the data are then put in the stored_data list. It is usually connected with a QTimer() timeout to make sure that data from the Queue are constantly retrieved.

Parameters
  • data_queue ((multiprocessing.Queue object)) – queue from witch to retrieve data.

  • header_list (list of str) – headers for the data to stored.

update_list()[source]

Upon calling put all available data into a list.

class stytra.collectors.accumulators.FramerateAccumulator(*args, goal_framerate=None, **kwargs)[source]

Bases: stytra.collectors.accumulators.Accumulator

trim_data()[source]
reset()[source]
update_list(fps)[source]
class stytra.collectors.accumulators.FramerateQueueAccumulator(*args, queue, **kwargs)[source]

Bases: stytra.collectors.accumulators.FramerateAccumulator

A simple accumulator, just for framerates

update_list()[source]
class stytra.collectors.accumulators.DynamicLog(stimuli, **kwargs)[source]

Bases: stytra.collectors.accumulators.DataFrameAccumulator

Accumulator to save feature of a stimulus, e.g. velocity of gratings in a closed-loop experiment.

Parameters

stimuli (list) – list of the stimuli to be logged

columns
update_list(time, data)[source]
Parameters

data

update_stimuli(stimuli)[source]
class stytra.collectors.accumulators.EstimatorLog(*args, **kwargs)[source]

Bases: stytra.collectors.accumulators.DataFrameAccumulator

update_list(t, data)[source]
Parameters

data

stytra.collectors.data_collector module

class stytra.collectors.data_collector.DataCollector(*data_tuples_list, extra_settings=None, instance_number=-1, folder_path='C:/')[source]

Bases: lightparam.ParameterTree

Class for saving all data and data_log produced during an experiment.

There are two kind of data that are collected:

  • Metadata/parameters: values that should restored from previous sessions. These values don’t have to be explicitely added. they are automatically read from all the objects in the stytra Experiment process which are instances of HasPyQtGraphParams.

  • Static data: (tail tracking, stimulus log…), that should not be restored. Those have to be added one by one via the add_data_source() method.

Inputs from both types of sources are eventually saved in the .json file containing all the information from the experiment. In this file data are divided into fixed categories:

  • general: info about the experiment (date, setup, session…)

  • animal: info about the animal (line, age, etc.)

  • stimulus: info about the stimulation (stimuli log, screen dimensions, etc.)

  • imaging: info about the connected microscope, if present

  • behavior: info about fish behavior (tail log…) and parameters for tracking

  • camera: parameters of the camera for behavior, if one is present

See documentation of the clean_data_dict() method for a description of conventions for dividing the entries among the categories. In the future this function may structure its output in other standard formats for scientific data (e.g., NWB).

In addition to the .json file, parameters from Parametrized objects are stored in a config.h5 file (located in the experiment directory) which is used for restoring the last configuration of the GUI and of the experiment parameters.

Parameters
  • data_tuples_list (tuple) – (optional) tuple of data to be added

  • folder_path (str) – destination where the final json file will be saved

restore_from_saved()[source]

If a config.h5 file is available, use the data there to restore the state of the HasPyQtGraph._params tree to last session values. Before, we make sure that the dictionary that we try to restore differs from our parameter structure only in the values. Without this control, changing any of the parameters in the code could result in bugs and headaches due to the change of the values from a config.h5 file from the previous program version.

add_static_data(entry, name='unspecified_entry')[source]

Add new data to the dictionary.

Parameters
  • entry – data that will be stored;

  • name (str) – name in the dictionary. It should take the form “category/name”, where “category” should be one of the possible keys of the dictionary produced in get_clean_dict() (animal, stimulus, etc.). Nested categories are supported, eg. animal/screening/image (Default value = ‘unspecified_entry’)

get_clean_dict(**kwargs)[source]

Collect data from all sources and put them together in the final hierarchical dictionary that will be saved in the .json file. The first level in the dictionary is fixed and defined by the keys of the clean_data_dict that will be returned. data from all sources are divided in these categories according to the key preceding the underscore in their name (e.g., value of general_db_idx will be put in [‘general’][‘db_idx’]).

Parameters
  • eliminate_df (bool) – see prepare_json docs; (Default value = False)

  • convert_datetime (bool) – see prepare_json docs; (Default value = False)

Returns

dictionary with the sorted data.

Return type

dict

get_last_value(class_param_key)[source]

Get the last saved value for a specific class_param_key.

Parameters

class_param_key (str) – name of the parameter whose value is required.

Returns

value of the parameter in the config.h5 file.

Return type

save_config_file()[source]

Save the config.json file with the current state of the params data_log.

save_json_log(output_path)[source]

Save the .json file with all the data from both static sources and the updated params.

Parameters

timestamp – (Default value = None)

save(output_path='')[source]

Save both the data_log.json log and the config.h5 file

Parameters

output_path – (Default value = “”) Path where to save the metadata

stytra.collectors.namedtuplequeue module

class stytra.collectors.namedtuplequeue.NamedTupleQueue(*args, **kwargs)[source]

Bases: object

put(t, obj, block=True, timeout=None)[source]
get(block=True, timeout=-1)[source]

Module contents