Designing and running experiments

You don’t need to get acquainted with the full feature set of stytra to start running experiments. Here and in the stytra/examples directory, we provide a number of example protocols that you can use to get inspiration for your own. In this section, we will illustrate general concepts of designing and running experiments with examples.

All examples in this section can be run in two ways: copy and paste the code in a python script and run it from your favorite IDE, or simply type on the command prompt:

python -m stytra.examples.name_of_the_example

Create a new protocol

To run a stytra experiment, we simply need to create a script were we define a protocol, and we assign it to a Stytra object. Running this script will create the Stytra GUI with controls for editing and running the protocol.

The essential ingredient of protocols is the list of stimuli that will be displayed. To create it, we need to define the Protocol.get_stim_sequence() method. This method returns a list of Stimulus objects which will be presented in succession.

In stytra.examples.most_basic_exp.py we define a very simple experiment:

../../../stytra/examples/most_basic_exp.py
from stytra import Stytra, Protocol
from stytra.stimulation.stimuli.visual import Pause, FullFieldVisualStimulus

# 1. Define a protocol subclass
class FlashProtocol(Protocol):
    name = "flash_protocol"  # every protocol must have a name.

    def get_stim_sequence(self):
        # This is the method we need to write to create a new stimulus list.
        # In this case, the protocol is simply a 1 second flash on the entire screen
        # after a pause of 4 seconds:
        stimuli = [
            Pause(duration=4.),
            FullFieldVisualStimulus(duration=1., color=(255, 255, 255)),
        ]
        return stimuli


if __name__ == "__main__":
    # This is the line that actually opens stytra with the new protocol.
    st = Stytra(protocol=FlashProtocol())

It is important to note that stimuli should be instances, not classes!

Try to run this code or type in the command prompt:

python -m stytra.examples.most_basic_exp

This will open two windows: one is the main control GUI to run the experiments, the second is the screen used to display the visual stimuli. In a real experiment, you want to make sure this second window is presented to the animal. For details on positioning and calibration, please refer to Calibration

For an introduction to the functionality of the user interface, see Stytra user interface. To start the experiment, just press the play button: a flash will appear on the screen after 4 seconds.

Parametrise the protocol

Sometimes, we want to control a protocol parameters from the interface. To do this, we can define protocol class attributes as Param. All attributes defined as ``Param``s will be modifiable from the user interface.

For a complete description of Params inside stytra see Parameters in stytra.

../../../stytra/examples/flash_exp.py
from stytra import Stytra, Protocol
from stytra.stimulation.stimuli.visual import Pause, FullFieldVisualStimulus
from lightparam import Param


class FlashProtocol(Protocol):
    name = "flash_protocol"  # every protocol must have a name.

    def __init__(self):
        super().__init__()
        # Here we define these attributes as Param s. This will automatically
        #  build a control for them and make them modifiable live from the
        # interface.
        self.period_sec = Param(10., limits=(0.2, None))
        self.flash_duration = Param(1., limits=(0., None))

    def get_stim_sequence(self):
        # This is the
        stimuli = [
            Pause(duration=self.period_sec - self.flash_duration),
            FullFieldVisualStimulus(
                duration=self.flash_duration, color=(255, 255, 255)
            ),
        ]
        return stimuli


if __name__ == "__main__":
    st = Stytra(protocol=FlashProtocol())

Note that Parameters in Protocol param are the ones that can be changed from the GUI, but all stimulus attributes will be saved in the final log, both parameterized and unparameterized ones. No aspect of the stimulus configuration will be unsaved.

Define dynamic stimuli

Many stimuli may have quantities, such as velocity for gratings or angular velocity for windmills, that change over time. To define these kind of stimuli Stytra use a convenient syntax: a param_df pandas DataFrame with the specification of the desired parameter value at specific timepoints. The value at all the other timepoints will be linearly interpolated from the DataFrame. The dataframe has to contain a t column with the time, and one column for each quantity that has to change over time (x, theta, etc.). This stimulus behaviour is handled by the Stimulus class.

In this example, we use a dataframe for changing the diameter of a circle stimulus, making it a looming object:

../../../stytra/examples/looming_exp.py
import numpy as np
import pandas as pd

from stytra import Stytra
from stytra.stimulation import Protocol
from stytra.stimulation.stimuli import InterpolatedStimulus, CircleStimulus
from lightparam import Param


# A looming stimulus is an expanding circle. Stimuli which contain
# some kind of parameter change inherit from InterpolatedStimulus
# which allows for specifying the values of parameters of the
# stimulus at certain time points, with the intermediate
# values interpolated

# Use the 3-argument version of the Python type function to
# make a temporary class combining two classes


class LoomingStimulus(InterpolatedStimulus, CircleStimulus):
    name = "looming_stimulus"


# Let's define a simple protocol consisting of looms at random locations,
# of random durations and maximal sizes

# First, we inherit from the Protocol class
class LoomingProtocol(Protocol):

    # We specify the name for the dropdown in the GUI
    name = "looming_protocol"

    def __init__(self):
        super().__init__()

        # It is convenient for a protocol to be parametrized, so
        # we name the parameters we might want to change,
        # along with specifying the the default values.
        # This automatically creates a GUI to change them
        # (more elaborate ways of adding parameters are supported,
        # see the documentation of lightparam)

        # if you are not interested in parametrizing your
        # protocol the the whole __init__ definition
        # can be skipped

        self.n_looms = Param(10, limits=(0, 1000))
        self.max_loom_size = Param(60, limits=(0, 100))
        self.max_loom_duration = Param(5, limits=(0, 100))
        self.x_pos_pix = Param(10, limits=(0, 2000))
        self.y_pos_pix = Param(10, limits=(0, 2000))

    # This is the only function we need to define for a custom protocol
    def get_stim_sequence(self):
        stimuli = []

        for i in range(self.n_looms):
            # The radius is only specified at the beginning and at the
            # end of expansion. More elaborate functional relationships
            # than linear can be implemented by specifying a more
            # detailed interpolation table

            radius_df = pd.DataFrame(
                dict(
                    t=[0, np.random.rand() * self.max_loom_duration],
                    radius=[0, np.random.rand() * self.max_loom_size],
                )
            )

            # We construct looming stimuli with the radius change specification
            # and a random point of origin within the projection area
            # (specified in fractions from 0 to 1 for each dimension)
            stimuli.append(
                LoomingStimulus(
                    df_param=radius_df, origin=(self.x_pos_pix, self.y_pos_pix)
                )
            )

        return stimuli


if __name__ == "__main__":
    # We make a new instance of Stytra with this protocol as the only option:
    s = Stytra(protocol=LoomingProtocol())

Use velocities instead of quantities

For every quantity we can specify the velocity at which it changes instead of the value itself. This can be done prefixing vel_ to the quantity name in the DataFrame. In the next example, we use this syntax to create moving gratings. What is dynamically updated is the position x of the gratings, but with the dictionary we specify its velocity with vel_x.

../../../stytra/examples/gratings_exp.py
import numpy as np
import pandas as pd

from stytra import Stytra
from stytra.stimulation import Protocol
from stytra.stimulation.stimuli import MovingGratingStimulus
from lightparam import Param
from pathlib import Path


class GratingsProtocol(Protocol):
    name = "gratings_protocol"

    def __init__(self):
        super().__init__()

        self.t_pre = Param(5.)  # time of still gratings before they move
        self.t_move = Param(5.)  # time of gratings movement
        self.grating_vel = Param(-10.)  # gratings velocity
        self.grating_period = Param(10)  # grating spatial period
        self.grating_angle_deg = Param(90.)  # grating orientation

    def get_stim_sequence(self):
        # Use six points to specify the velocity step to be interpolated:
        t = [
            0,
            self.t_pre,
            self.t_pre,
            self.t_pre + self.t_move,
            self.t_pre + self.t_move,
            2 * self.t_pre + self.t_move,
        ]

        vel = [0, 0, self.grating_vel, self.grating_vel, 0, 0]

        df = pd.DataFrame(dict(t=t, vel_x=vel))

        return [
            MovingGratingStimulus(
                df_param=df,
                grating_angle=self.grating_angle_deg * np.pi / 180,
                grating_period=self.grating_period,
            )
        ]


if __name__ == "__main__":
    # We make a new instance of Stytra with this protocol as the only option
    s = Stytra(protocol=GratingsProtocol())

You can look in the code of the windmill_exp.py example to see how to use the dataframe to specify a more complex motion - in this case, a rotation with sinusoidal velocity.

Note

If aspects of your stimulus change abruptly, you can put twice the same timepoint in the param_df, for example: param_df = pd.DataFrame(dict(t = [0, 10, 10, 20], vel_x = [0, 0, 10, 10])

Visualise with stim_plot parameter

If you want to monitor in real time the changes in your experiment parameters, you can pass the stim_plot argument to the call to stytra to add to the interface an online plot:

../../../stytra/examples/plot_dynamic_exp.py
from stytra import Stytra


if __name__ == "__main__":
    from stytra.examples.gratings_exp import GratingsProtocol

    # We make a new instance of Stytra with this protocol as the only option:
    s = Stytra(protocol=GratingsProtocol(), stim_plot=True)

Stimulation and tracking

Add a camera to a protocol

We often need to have frames streamed from a file or a camera. In the following example we comment on how to achieve this when defining a protocol:

../../../stytra/examples/display_camera_exp.py
from stytra import Stytra
from stytra.stimulation.stimuli import Pause
from pathlib import Path
from stytra.stimulation import Protocol


class Nostim(Protocol):
    name = "empty_protocol"

    # In the stytra_config class attribute we specify a dictionary of
    # parameters that control camera, tracking, monitor, etc.
    # In this particular case, we add a stream of frames from one example
    # movie saved in stytra assets.
    stytra_config = dict(
        camera=dict(type="spinnaker"))

    #  For a streaming from real cameras connected to the computer, specify camera type, e.g.:
    # stytra_config = dict(camera=dict(type="ximea"))

    def get_stim_sequence(self):
        return [Pause(duration=10)]  # protocol does not do anything


if __name__ == "__main__":
    s = Stytra(protocol=Nostim())

Note however that usually the camera settings are always the same on the computer that controls a setup, therefore the camera settings are defined in the user config file and generally not required at the protocol level. See Configuring a computer for Stytra experiments for more info.

Add tracking to a defined protocol

To add tail or eye tracking to a protocol, it is enough to change the stytra_config attribute to contain a tracking argument as well. See the experiment documentation for a description of the available tracking methods.

In this example, we redefine the previously defined windmill protocol (which displays a rotating windmill) to add tracking of the eyes as well:

../../../stytra/examples/tail_tracking_exp.py
from pathlib import Path
from stytra import Stytra
from stytra.examples.gratings_exp import GratingsProtocol


class TrackingGratingsProtocol(GratingsProtocol):
    name = "gratings_tail_tracking"

    # To add tracking to a protocol, we simply need to add a tracking
    # argument to the stytra_config:
    stytra_config = dict(
        tracking=dict(embedded=True, method="tail"),
        camera=dict(
            video_file=str(Path(__file__).parent / "assets" / "fish_compressed.h5")
        ),
    )


if __name__ == "__main__":
    s = Stytra(protocol=TrackingGratingsProtocol())

Now a window with the fish image an a ROI to control tail position will appear, and the tail will be tracked! See Embedded fish for instructions on how to adjust tracking parameters.

Closed-loop experiments

Stytra allows to simple definition of closed-loop experiments where quantities tracked from the camera are dynamically used to update some stimulus variable. In the example below we create a full-screen stimulus that turns red when the fish is swimming above a certain threshold (estimated with the vigour method).

../../../stytra/examples/custom_visual_exp.py
from stytra import Stytra, Protocol
from stytra.stimulation.stimuli import VisualStimulus
from PyQt5.QtCore import QRect
from PyQt5.QtGui import QBrush, QColor
from pathlib import Path


class NewStimulus(VisualStimulus):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.color = (255, 255, 255)

    def paint(self, p, w, h):
        p.setBrush(QBrush(QColor(*self.color)))  # Use chosen color
        p.drawRect(QRect(0, 0, w, h))  # draw full field rectangle

    def update(self):
        fish_vel = self._experiment.estimator.get_velocity()
        # change color if speed of the fish is higher than threshold:
        if fish_vel < -5:
            self.color = (255, 0, 0)
        else:
            self.color = (255, 255, 255)


class CustomProtocol(Protocol):
    name = "custom protocol"  # protocol name

    stytra_config = dict(
        tracking=dict(method="tail", estimator="vigor"),
        camera=dict(
            video_file=str(Path(__file__).parent / "assets" / "fish_compressed.h5")
        ),
    )

    def get_stim_sequence(self):
        return [NewStimulus(duration=10)]


if __name__ == "__main__":
    Stytra(protocol=CustomProtocol())

Freely-swimming experiments

For freely swimming experiments, it is important to calibrate the camera view to the displayed image. This is explained in Calibration. Then, we can easily create stimuli that track or change depending on the location of the fish. The following example shows the implementation of a simple phototaxis protocol, where the bright field is always displayed on the right side of the fish, and a centering stimulus is activated if the fish swims out of the field of view. Configuring tracking for freely-swimming experiments is explained here Freely-swimming fish

../../../stytra/examples/phototaxis.py
from stytra import Stytra
from stytra.stimulation.stimuli import (
    FishTrackingStimulus,
    HalfFieldStimulus,
    RadialSineStimulus,
    FullFieldVisualStimulus,
)
from stytra.stimulation.stimuli.conditional import CenteringWrapper

from stytra.stimulation import Protocol
from lightparam import Param
from pathlib import Path


class PhototaxisProtocol(Protocol):
    name = "phototaxis"
    stytra_config = dict(
        display=dict(min_framerate=50),
        tracking=dict(method="fish", embedded=False, estimator="position"),
        camera=dict(
            video_file=str(Path(__file__).parent / "assets" / "fish_free_compressed.h5"),
            min_framerate=100,
        ),
    )

    def __init__(self):
        super().__init__()
        self.n_trials = Param(120, (0, 2400))
        self.stim_on_duration = Param(10, (0, 30))
        self.stim_off_duration = Param(10, (0, 30))
        self.center_offset = Param(0, (-100, 100))
        self.brightness = Param(255, (0, 255))

    def get_stim_sequence(self):
        stimuli = []
        # The phototaxis stimulus for zebrafish is bright on one side of the
        # fish and dark on the other. The type function combines two classes:
        stim = type("phototaxis", (FishTrackingStimulus, HalfFieldStimulus), {})

        # The stimuli are a sequence of a phototactic stimulus and full-field
        # illumination
        for i in range(self.n_trials):

            # The stimulus of interest is wrapped in a CenteringStimulus,
            # so if the fish moves out of the field of view, a stimulus is
            # displayed which brings it back
            stimuli.append(
                CenteringWrapper(
                    stimulus=stim(
                        duration=self.stim_on_duration,
                        color=(self.brightness,) * 3,
                        center_dist=self.center_offset,
                    ),
                )
            )

            stimuli.append(
                FullFieldVisualStimulus(
                    color=(self.brightness,) * 3,
                    duration=self.stim_off_duration
                )
            )

        return stimuli


if __name__ == "__main__":
    s = Stytra(protocol=PhototaxisProtocol())

Defining custom Experiment classes

New Experiment objects with custom requirements might be needed; for example, if one wants to implement more events or controls when the experiment start and finishes, or if custom UIs with new plots are desired. In this case, we will have to sublcass the stytra Experiment class. This class already has the minimal structure for running an experimental protocol and collect metadata. Using it as a template, we can define a new custom class.

Start an Experiment bypassing the Stytra constructor

First, to use a custom Experiment we need to see how we can start it bypassing the Stytra constructor class, which by design deals only with standard Experiment classes. This is very simple, and it is described in the example below:

../../../stytra/examples/no_stytra_exp.py
from stytra.experiments import Experiment
from stytra.stimulation import Protocol
import qdarkstyle
from PyQt5.QtWidgets import QApplication
from stytra.stimulation.stimuli import Pause, Stimulus


# Here ve define an empty protocol:
class FlashProtocol(Protocol):
    name = "empty_protocol"  # every protocol must have a name.

    def get_stim_sequence(self):
        return [Stimulus(duration=5.),]


if __name__ == "__main__":
    # Here we do not use the Stytra constructor but we instantiate an experiment
    # and we start it in the script. Even though this is an internal Experiment
    # subtype, a user can define a new Experiment subclass and start it
    # this way.
    app = QApplication([])
    app.setStyleSheet(qdarkstyle.load_stylesheet_pyqt5())
    protocol = FlashProtocol()
    exp = Experiment(protocol=protocol,
                     app=app)
    exp.start_experiment()
    app.exec_()

Customise an Experiment

To customize an experiment, we need to subclass Experiment, or the existing subclasses VisualExperiment and TrackingExperiment, which deal with experiments with a projector or with tracking from a camera, respectively. In the example below, we see how to make a very simple subclass, with an additional event (a dialog waiting for an OK from the user) implemented at protocol onset. For a description of how the Experiment class fits in the structure of stytra, please refer to the corresponding section.

../../../stytra/examples/custom_exp.py
from stytra.experiments import Experiment
from stytra.stimulation import Protocol
import qdarkstyle
from PyQt5.QtWidgets import QApplication
from stytra.stimulation.stimuli import Stimulus
from PyQt5.QtWidgets import QMessageBox
from PyQt5.QtWidgets import QPushButton


# Here ve define an empty protocol:
class FlashProtocol(Protocol):
    name = "empty_protocol"  # every protocol must have a name.

    def get_stim_sequence(self):
        return [Stimulus(duration=5.),]


class CustomExperiment(Experiment):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)

        self.start = False

    def start_protocol(self):
        self.start = False

        msgBox = QMessageBox()
        msgBox.setText('Start the protocol when ready')
        msgBox.setStandardButtons(QMessageBox.Ok)
        ret = msgBox.exec_()
        super().start_protocol()



if __name__ == "__main__":
    # Here we do not use the Stytra constructor but we instantiate an experiment
    # and we start it in the script. Even though this is an internal Experiment
    # subtype, a user can define a new Experiment subclass and start it
    # this way.
    app = QApplication([])
    app.setStyleSheet(qdarkstyle.load_stylesheet_pyqt5())
    protocol = FlashProtocol()
    exp = CustomExperiment(protocol=protocol,
                     app=app)
    exp.start_experiment()
    app.exec_()