Designing and running experiments¶
You don’t need to get acquainted with the full feature set of stytra to start running experiments. Here and in the stytra/examples directory, we provide a number of example protocols that you can use to get inspiration for your own. In this section, we will illustrate general concepts of designing and running experiments with examples.
All examples in this section can be run in two ways: copy and paste the code in a python script and run it from your favorite IDE, or simply type on the command prompt:
python -m stytra.examples.name_of_the_example
Create a new protocol¶
To run a stytra experiment, we simply need to create a script were we define a protocol,
and we assign it to a Stytra object. Running this script will create
the Stytra GUI with controls for editing and running the protocol.
The essential ingredient of protocols is the list of stimuli that will be displayed.
To create it, we need to define the
Protocol.get_stim_sequence() method.
This method returns a list of Stimulus objects
which will be presented in succession.
In stytra.examples.most_basic_exp.py we define a very simple experiment:
from stytra import Stytra, Protocol
from stytra.stimulation.stimuli.visual import Pause, FullFieldVisualStimulus
# 1. Define a protocol subclass
class FlashProtocol(Protocol):
name = "flash_protocol" # every protocol must have a name.
def get_stim_sequence(self):
# This is the method we need to write to create a new stimulus list.
# In this case, the protocol is simply a 1 second flash on the entire screen
# after a pause of 4 seconds:
stimuli = [
Pause(duration=4.0),
FullFieldVisualStimulus(duration=1.0, color=(255, 255, 255)),
]
return stimuli
if __name__ == "__main__":
# This is the line that actually opens stytra with the new protocol.
st = Stytra(protocol=FlashProtocol())
It is important to note that stimuli should be instances, not classes!
Try to run this code or type in the command prompt:
python -m stytra.examples.most_basic_exp
This will open two windows: one is the main control GUI to run the experiments, the second is the screen used to display the visual stimuli. In a real experiment, you want to make sure this second window is presented to the animal. For details on positioning and calibration, please refer to Calibration
For an introduction to the functionality of the user interface, see Stytra user interface. To start the experiment, just press the play button: a flash will appear on the screen after 4 seconds.
Parametrise the protocol¶
Sometimes, we want to control a protocol parameters from the interface. To do this, we can define protocol class attributes as Param. All attributes defined as ``Param``s will be modifiable from the user interface.
For a complete description of Params inside stytra see Parameters in stytra.
from stytra import Stytra, Protocol
from stytra.stimulation.stimuli.visual import Pause, FullFieldVisualStimulus
from lightparam import Param
class FlashProtocol(Protocol):
name = "flash_protocol" # every protocol must have a name.
def __init__(self):
super().__init__()
# Here we define these attributes as Param s. This will automatically
# build a control for them and make them modifiable live from the
# interface.
self.period_sec = Param(10.0, limits=(0.2, None))
self.flash_duration = Param(1.0, limits=(0.0, None))
def get_stim_sequence(self):
# This is the
stimuli = [
Pause(duration=self.period_sec - self.flash_duration),
FullFieldVisualStimulus(
duration=self.flash_duration, color=(255, 255, 255)
),
]
return stimuli
if __name__ == "__main__":
st = Stytra(protocol=FlashProtocol())
Note that Parameters in Protocol param are the ones that can be changed from the GUI, but all stimulus attributes will be saved in the final log, both parameterized and unparameterized ones. No aspect of the stimulus configuration will be unsaved.
Define dynamic stimuli¶
Many stimuli may have quantities, such as velocity for gratings or
angular velocity for windmills, that change over time. To define these
kind of stimuli Stytra use a convenient syntax: a param_df pandas DataFrame
with the specification of the desired parameter value at specific timepoints.
The value at all the other timepoints will be linearly interpolated from the
DataFrame. The dataframe has to contain a t column with the time, and one
column for each quantity that has to change over time (x, theta, etc.).
This stimulus behaviour is handled by the Stimulus
class.
In this example, we use a dataframe for changing the diameter of a circle stimulus, making it a looming object:
import numpy as np
import pandas as pd
from stytra import Stytra
from stytra.stimulation import Protocol
from stytra.stimulation.stimuli import InterpolatedStimulus, CircleStimulus
from lightparam import Param
# A looming stimulus is an expanding circle. Stimuli which contain
# some kind of parameter change inherit from InterpolatedStimulus
# which allows for specifying the values of parameters of the
# stimulus at certain time points, with the intermediate
# values interpolated
# Use the 3-argument version of the Python type function to
# make a temporary class combining two classes
class LoomingStimulus(InterpolatedStimulus, CircleStimulus):
name = "looming_stimulus"
# Let's define a simple protocol consisting of looms at random locations,
# of random durations and maximal sizes
# First, we inherit from the Protocol class
class LoomingProtocol(Protocol):
# We specify the name for the dropdown in the GUI
name = "looming_protocol"
def __init__(self):
super().__init__()
# It is convenient for a protocol to be parametrized, so
# we name the parameters we might want to change,
# along with specifying the the default values.
# This automatically creates a GUI to change them
# (more elaborate ways of adding parameters are supported,
# see the documentation of lightparam)
# if you are not interested in parametrizing your
# protocol the the whole __init__ definition
# can be skipped
self.n_looms = Param(10, limits=(0, 1000))
self.max_loom_size = Param(60, limits=(0, 100))
self.max_loom_duration = Param(5, limits=(0, 100))
self.x_pos_pix = Param(10, limits=(0, 2000))
self.y_pos_pix = Param(10, limits=(0, 2000))
# This is the only function we need to define for a custom protocol
def get_stim_sequence(self):
stimuli = []
for i in range(self.n_looms):
# The radius is only specified at the beginning and at the
# end of expansion. More elaborate functional relationships
# than linear can be implemented by specifying a more
# detailed interpolation table
radius_df = pd.DataFrame(
dict(
t=[0, np.random.rand() * self.max_loom_duration],
radius=[0, np.random.rand() * self.max_loom_size],
)
)
# We construct looming stimuli with the radius change specification
# and a random point of origin within the projection area
# (specified in fractions from 0 to 1 for each dimension)
stimuli.append(
LoomingStimulus(
df_param=radius_df, origin=(self.x_pos_pix, self.y_pos_pix)
)
)
return stimuli
if __name__ == "__main__":
# We make a new instance of Stytra with this protocol as the only option:
s = Stytra(protocol=LoomingProtocol())
Use velocities instead of quantities¶
For every quantity we can specify the velocity at which it changes instead of the value itself. This can be done prefixing vel_ to the quantity name in the DataFrame. In the next example, we use this syntax to create moving gratings. What is dynamically updated is the position x of the gratings, but with the dictionary we specify its velocity with vel_x.
import numpy as np
import pandas as pd
from stytra import Stytra
from stytra.stimulation import Protocol
from stytra.stimulation.stimuli import MovingGratingStimulus
from lightparam import Param
from pathlib import Path
class GratingsProtocol(Protocol):
name = "gratings_protocol"
def __init__(self):
super().__init__()
self.t_pre = Param(5.0) # time of still gratings before they move
self.t_move = Param(5.0) # time of gratings movement
self.grating_vel = Param(-10.0) # gratings velocity
self.grating_period = Param(10) # grating spatial period
self.grating_angle_deg = Param(90.0) # grating orientation
def get_stim_sequence(self):
# Use six points to specify the velocity step to be interpolated:
t = [
0,
self.t_pre,
self.t_pre,
self.t_pre + self.t_move,
self.t_pre + self.t_move,
2 * self.t_pre + self.t_move,
]
vel = [0, 0, self.grating_vel, self.grating_vel, 0, 0]
df = pd.DataFrame(dict(t=t, vel_x=vel))
return [
MovingGratingStimulus(
df_param=df,
grating_angle=self.grating_angle_deg * np.pi / 180,
grating_period=self.grating_period,
)
]
if __name__ == "__main__":
# We make a new instance of Stytra with this protocol as the only option
s = Stytra(protocol=GratingsProtocol())
You can look in the code of the windmill_exp.py example to see how to use the dataframe to specify a more complex motion - in this case, a rotation with sinusoidal velocity.
Note
If aspects of your stimulus change abruptly, you can put twice the same timepoint in the param_df, for example: param_df = pd.DataFrame(dict(t = [0, 10, 10, 20], vel_x = [0, 0, 10, 10])
Visualise with stim_plot parameter¶
If you want to monitor in real time the changes in your experiment parameters, you can pass the stim_plot argument to the call to stytra to add to the interface an online plot:
from stytra import Stytra
if __name__ == "__main__":
from stytra.examples.gratings_exp import GratingsProtocol
# We make a new instance of Stytra with this protocol as the only option:
s = Stytra(protocol=GratingsProtocol(), stim_plot=True)
Stimulation and tracking¶
Add a camera to a protocol¶
We often need to have frames streamed from a file or a camera. In the following example we comment on how to achieve this when defining a protocol:
from stytra import Stytra
from stytra.stimulation.stimuli import Pause
from pathlib import Path
from stytra.stimulation import Protocol
class Nostim(Protocol):
name = "empty_protocol"
# In the stytra_config class attribute we specify a dictionary of
# parameters that control camera, tracking, monitor, etc.
# In this particular case, we add a stream of frames from one example
# movie saved in stytra assets.
stytra_config = dict(camera=dict(type="spinnaker"))
# For a streaming from real cameras connected to the computer, specify camera type, e.g.:
# stytra_config = dict(camera=dict(type="ximea"))
def get_stim_sequence(self):
return [Pause(duration=10)] # protocol does not do anything
if __name__ == "__main__":
s = Stytra(protocol=Nostim())
Note however that usually the camera settings are always the same on the computer that controls a setup, therefore the camera settings are defined in the user config file and generally not required at the protocol level. See Configuring a computer for Stytra experiments for more info.
Add tracking to a defined protocol¶
To add tail or eye tracking to a protocol, it is enough to change the stytra_config attribute to contain a tracking argument as well. See the experiment documentation for a description of the available tracking methods.
In this example, we redefine the previously defined windmill protocol (which displays a rotating windmill) to add tracking of the eyes as well:
from pathlib import Path
from stytra import Stytra
from stytra.examples.gratings_exp import GratingsProtocol
class TrackingGratingsProtocol(GratingsProtocol):
name = "gratings_tail_tracking"
# To add tracking to a protocol, we simply need to add a tracking
# argument to the stytra_config:
stytra_config = dict(
tracking=dict(embedded=True, method="tail"),
camera=dict(
video_file=str(Path(__file__).parent / "assets" / "fish_compressed.h5")
),
)
if __name__ == "__main__":
s = Stytra(protocol=TrackingGratingsProtocol())
Now a window with the fish image an a ROI to control tail position will appear, and the tail will be tracked! See Embedded fish for instructions on how to adjust tracking parameters.
Closed-loop experiments¶
Stytra allows to simple definition of closed-loop experiments where quantities tracked from the camera are dynamically used to update some stimulus variable. In the example below we create a full-screen stimulus that turns red when the fish is swimming above a certain threshold (estimated with the vigour method).
from stytra import Stytra, Protocol
from stytra.stimulation.stimuli import VisualStimulus
from PyQt5.QtCore import QRect
from PyQt5.QtGui import QBrush, QColor
from pathlib import Path
class NewStimulus(VisualStimulus):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.color = (255, 255, 255)
def paint(self, p, w, h):
p.setBrush(QBrush(QColor(*self.color))) # Use chosen color
p.drawRect(QRect(0, 0, w, h)) # draw full field rectangle
def update(self):
fish_vel = self._experiment.estimator.get_velocity()
# change color if speed of the fish is higher than threshold:
if fish_vel < -5:
self.color = (255, 0, 0)
else:
self.color = (255, 255, 255)
class CustomProtocol(Protocol):
name = "custom protocol" # protocol name
stytra_config = dict(
tracking=dict(method="tail", estimator="vigor"),
camera=dict(
video_file=str(Path(__file__).parent / "assets" / "fish_compressed.h5")
),
)
def get_stim_sequence(self):
return [NewStimulus(duration=10)]
if __name__ == "__main__":
Stytra(protocol=CustomProtocol())
Freely-swimming experiments¶
For freely swimming experiments, it is important to calibrate the camera view to the displayed image. This is explained in Calibration. Then, we can easily create stimuli that track or change depending on the location of the fish. The following example shows the implementation of a simple phototaxis protocol, where the bright field is always displayed on the right side of the fish, and a centering stimulus is activated if the fish swims out of the field of view. Configuring tracking for freely-swimming experiments is explained here Freely-swimming fish
from stytra import Stytra
from stytra.stimulation.stimuli import (
FishTrackingStimulus,
HalfFieldStimulus,
RadialSineStimulus,
FullFieldVisualStimulus,
)
from stytra.stimulation.stimuli.conditional import CenteringWrapper
from stytra.stimulation import Protocol
from lightparam import Param
from pathlib import Path
class PhototaxisProtocol(Protocol):
name = "phototaxis"
stytra_config = dict(
display=dict(min_framerate=50),
tracking=dict(method="fish", embedded=False, estimator="position"),
camera=dict(
video_file=str(
Path(__file__).parent / "assets" / "fish_free_compressed.h5"
),
min_framerate=100,
),
)
def __init__(self):
super().__init__()
self.n_trials = Param(120, (0, 2400))
self.stim_on_duration = Param(10, (0, 30))
self.stim_off_duration = Param(10, (0, 30))
self.center_offset = Param(0, (-100, 100))
self.brightness = Param(255, (0, 255))
def get_stim_sequence(self):
stimuli = []
# The phototaxis stimulus for zebrafish is bright on one side of the
# fish and dark on the other. The type function combines two classes:
stim = type("phototaxis", (FishTrackingStimulus, HalfFieldStimulus), {})
# The stimuli are a sequence of a phototactic stimulus and full-field
# illumination
for i in range(self.n_trials):
# The stimulus of interest is wrapped in a CenteringStimulus,
# so if the fish moves out of the field of view, a stimulus is
# displayed which brings it back
stimuli.append(
CenteringWrapper(
stimulus=stim(
duration=self.stim_on_duration,
color=(self.brightness,) * 3,
center_dist=self.center_offset,
)
)
)
stimuli.append(
FullFieldVisualStimulus(
color=(self.brightness,) * 3, duration=self.stim_off_duration
)
)
return stimuli
if __name__ == "__main__":
s = Stytra(protocol=PhototaxisProtocol())
Defining custom Experiment classes¶
New Experiment objects with custom requirements might be needed; for example, if one wants
to implement more events or controls when the experiment start and finishes, or if custom
UIs with new plots are desired. In this case, we will have to sublcass the stytra Experiment
class. This class already has the minimal structure for running an experimental protocol and
collect metadata. Using it as a template, we can define a new custom class.
Start an Experiment bypassing the Stytra constructor¶
First, to use a custom Experiment we need to see how we can start it bypassing the Stytra
constructor class, which by design deals only with standard Experiment classes. This is very
simple, and it is described in the example below:
from stytra.experiments import Experiment
from stytra.stimulation import Protocol
import qdarkstyle
from PyQt5.QtWidgets import QApplication
from stytra.stimulation.stimuli import Pause, Stimulus
# Here ve define an empty protocol:
class FlashProtocol(Protocol):
name = "empty_protocol" # every protocol must have a name.
def get_stim_sequence(self):
return [Stimulus(duration=5.0)]
if __name__ == "__main__":
# Here we do not use the Stytra constructor but we instantiate an experiment
# and we start it in the script. Even though this is an internal Experiment
# subtype, a user can define a new Experiment subclass and start it
# this way.
app = QApplication([])
app.setStyleSheet(qdarkstyle.load_stylesheet_pyqt5())
protocol = FlashProtocol()
exp = Experiment(protocol=protocol, app=app)
exp.start_experiment()
app.exec_()
Customise an Experiment¶
To customize an experiment, we need to subclass Experiment, or the existing subclasses
VisualExperiment and
TrackingExperiment,
which deal with experiments with a projector or with tracking
from a camera, respectively.
In the example below, we see how to make a very simple subclass, with an additional event
(a dialog waiting for an OK from the user) implemented at protocol onset. For a description
of how the Experiment class fits in the structure of stytra, please refer to the corresponding section.
from stytra.experiments import Experiment
from stytra.stimulation import Protocol
import qdarkstyle
from PyQt5.QtWidgets import QApplication
from stytra.stimulation.stimuli import Stimulus
from PyQt5.QtWidgets import QMessageBox
# Here ve define an empty protocol:
class FlashProtocol(Protocol):
name = "empty_protocol" # every protocol must have a name.
def get_stim_sequence(self):
return [Stimulus(duration=5.0)]
# Little demonstration on how to use a custom experiment to bypass standard
# launching through Stytra class. This little experiment simply add an additional
# message box warning the user to confirm before running the protocol.
class CustomExperiment(Experiment):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.start = False
def start_protocol(self):
self.start = False
# Show message box with PyQt:
msgBox = QMessageBox()
msgBox.setText("Start the protocol when ready!")
msgBox.setStandardButtons(QMessageBox.Ok)
_ = msgBox.exec_()
super().start_protocol() # call the super() start_protocol method
if __name__ == "__main__":
# Here we do not use the Stytra constructor but we instantiate an experiment
# and we start it in the script. Even though this is an internal Experiment
# subtype, a user can define a new Experiment subclass and start it
# this way.
app = QApplication([])
app.setStyleSheet(qdarkstyle.load_stylesheet_pyqt5())
protocol = FlashProtocol()
exp = CustomExperiment(protocol=protocol, app=app)
exp.start_experiment()
app.exec_()