ArtiSynth User Interface Guide

7 The Timeline

The timeline is a panel that provides “play” controls for starting and stopping the simulation, and allows the user to graphically arrange temporal sequences of probes and waypoints to control and monitor the simulation. If not initially visible, its visibility can be toggled by hitting the ‘t’ key from within the viewer (Section 3.9).

7.1 Probes and waypoints

ArtiSynth allows models to connect to special components, known as probes, which can either supply input values or monitor output values over time as the simulation proceeds. Probes attached to simulation inputs are known as input probes (class InputProbe), while those attached to outputs are known as output probes (class OutputProbe). Each probe has a start time and a stop time, and implements an apply(t0,t1) method that supplies (or collects) data for the simulation step between time t0 and t1. Probes are described in more detail, along with specifics about how to code them into applications, in the “Simulation Control” chapter of the ArtiSynth Modeling Guide.

The most commonly used probe subclasses are NumericInputProbe and NumericOutputProbe, each of which is associated with a vector of floating point values that are interpolated over time. This data is usually connected to various model component properties, and used to either set (for input probes) or collect (for output probes) the values of those properties. The size of the vector is known as the probe’s vector size and matches the properties that the probe is connected to. For example, a probe controlling a single muscle excitation value will have a vector size of one, whereas a probe collecting the 3D velocity of a point will have a vector size of three.

Within a numeric probe, data is defined by a temporal sequence of knots points which give the vector values at prescribed times, with values in between determined by interpolation (Section 7.5.4). For input probes, the knot density is often sparse, whereas for output probes it matches the sample rate at which data is collected, which is usually the simulation step size.

Input and output probes are arranged and displayed graphically in the timeline, within a set of tracks (Section 7.4). Each probe is displayed as a bar within one of these tracks. The display for numeric probes can also be expanded to show a graph of the numeric data .

ArtiSynth allows models to set temporal waypoints, which are designated times at which the model state is saved during simulation. This allows the simulation to be later reset to that time without having to recompute the simulation from the beginning. A special type of waypoint is known as a breakpoint, which causes the simulation to pause when it is reached. The timeline displays the waypoints, and allows them to be created and edited (Section 7.3).

7.2 Basic timeline structure

The basic structure of the timeline is shown in Figure 18.

Figure 18: The timeline panel, containing: (A) zoom controls, (B) play controls, (C) save button, (D) time cursor, (E) toolbar, (F) waypoint track, (G) input track, (H) output track.

The toolbar at the top contains the following widgets:

Zoom controls

Shrinks or expands the timescale.

Play controls

Starts, pauses, or resets the simulation.

Save button

Saves the data for all output probes with attached files.

Time box

Shows the current simulation time.

Time step box

Shows the length of time associated with a “single step” command.

7.2.1 Play controls

The play controls are in turn comprised of the following buttons:

Reset: Resets the simulation to the beginning at time 0.
Skip-back: Moves the simulation time backward to the previous valid waypoint (see Section 7.3).
Play: Starts the simulation.
Pause: Takes the place of the play button and pauses the simulation.
Single-step: Advances the simulation by a single step, specified in the time step box.
Skip-forward: Moves the simulation time forward to the next valid waypoint (see Section 7.3).

7.2.2 Tracks

The timeline proper is divided into a number of tracks. At the top is the waypoint track, which is used to arrange waypoints and breakpoints. Below that are a variable number of input and output tracks, which are used respectively for arranging the input and output probes.

7.3 Viewing and setting waypoints

7.3.1 Waypoints

Waypoints are arranged along the waypoint track at the top of the timeline and are indicated by a small rectangular blue box (Figure 19). A solid box indicates a waypoint that contains a valid state and thus can be advanced to using the fast forward/backward buttons.

Figure 19: Waypoints (blue) and breakpoints (red) arranged in the waypoint track.

To add a waypoint, right-click on the waypoint track. A popup menu will show a number of options, including Add waypoint here, which adds a waypoint at the current location of the time cursor, and Add waypoint(s), which will bring up a dialog prompting for a specific time to add a Waypoint. The "Add Waypoint" dialog also contains a repeat field, which will cause additional waypoints to be added with a spacing indicated by the time value, and an option to create breakpoints instead of waypoints.

Once created, waypoints can be moved by dragging them. They can also be deleted by right-clicking on them and selecting the Delete waypoint option.

To delete all the waypoints, select Delete waypoints, either from the main File menu, or after right-clicking on the waypoint track.

7.3.2 Breakpoints

Breakpoints are waypoints that also cause the simulation to stop. They are displayed in red instead of blue.

Breakpoints can be added in the same way as waypoints, i.e., by right clicking on the waypoint track and selecting Add breakpoint here or Add waypoint(s).

Waypoints can be converted into breakpoints (and vice versa) through the context menu.

7.3.3 Saving and loading

It is possible to save and load waypoints to and from an external file. The following menu options may be selected to do this, either from the main File menu, or after right-clicking on the waypoint track:

Save waypoints as ...

Brings up a file chooser that allows the user to specify a file for saving all waypoints and their state data. Clicking Save As completes the operation.

Save waypoints

If a waypoint file has already been specified using either Save waypoints as ... or Load waypoints ..., then all waypoints and their state data are saved to that file.

Load waypoints ...

Brings up a file chooser that allows the user to specify a file from which waypoints and their state data will be loaded. Clicking Load completes the operation. The new waypoints are added to any existing ones, but previously waypoints are not deleted. Checks are made to help ensure that the waypoint data is consistent with the model’s current state.

Reload waypoints

Identical to Load waypoints ..., except that it uses a file that has already been specified using either Save waypoints as ... or Load waypoints ....

Delete waypoints

Deletes all waypoints except the one at time zero.

Waypoint files are currently stored as binary files. The reason for this is that the required storage is about 1/2 of that required for text files, while the writing and parsing times are as much as 10\times faster.

7.4 Tracks and probes

Probes are arranged on tracks located beneath the waypoint track. Input probes must be placed on input tracks and output probes must be placed on output tracks. Furthermore, probes on the same track are not allowed to overlap.

Note:
In the future, additional restrictions may be placed on what type of probe can be placed on what track.

Probes can be moved horizontally to different times as well as vertically onto different tracks. They can also be stretched by dragging the edges of the probe and cropped by holding the control key while stretching.

On the left side of the timeline is the track panel, which contains a number of track control widgets (Figure 20).

Figure 20: Close up of the track panel, showing: (A) track panel, (B) expand all button, (C) mute all button, (D) expand button, (E) mute button.

7.4.1 Creating, moving, and deleting tracks

New tracks may be added by right-clicking in the waypoint track and selecting Add input track or Add output track.

The vertical location of a track can be moved by left clicking on it in the left panel and dragging it up or down to a new location.

A track can be deleted by right-clicking on the track in the track panel and selecting Delete track.

7.4.2 Muting tracks

A track can be muted or unmuted by clicking on its gray mute button in the track panel (Figure 20). All probes on a muted track are ignored during simulation.

All tracks can be muted, thus disabling all probes, by clicking on the mute all button in the gray panel above the tracks.

7.4.3 Expanding tracks

A track can be expanded or collapsed by clicking on its green expand button in the track panel (Figure 20). Expanding a track creates an additional area in which the data associated with the track’s probes may be displayed (see Figure 21). The way in which this data is displayed is probe-specific. Probes containing numeric data usually show their data graphically, as described in Section 7.5.

7.4.4 Grouping tracks

Multiple contiguous tracks can be selected by clicking on them while holding the control key. Furthermore, they can be grouped together or ungrouped by selecting the appropriate option from the context menu. Grouped tracks can be collapsed, moved simultaneously and muted together.

7.5 Numeric probe displays

Data associated with numeric probes is displayed as a graph within the display (Figure 21), with each entry in a probe’s data vector drawn as a separate trace. If the probe’s vector size is greater than one, each trace is drawn using a different color (up to a limit, after which colors are recycled).

Figure 21: Expanded input and output tracks showing numeric data.

7.5.1 Setting the range and display properties

The range and other properties of the display can be set by right clicking in the display and selecting Edit range and properties ..., which creates a dialog that allows these to be adjusted. If the autoRanging property is set to true, then the display range automatically expands as needed to accommodate new data. Display ranges can also be adjusted to fit the current data by right-clicking in the display and selecting Fit ranges to data.

Large data displays (Section 7.5.5) contain additional GUI-based features to adjust the display range.

7.5.2 Visibility control

As mentioned above, each entry in a numeric probe’s data vector is rendered in a different color (up to a limit). If the vector size is large (say more than three or four), or if there is much overlapping of values, then the result can be difficult to visualize.

To manage this problem, numeric displays provide a legend tool that allows the user to control the color, drawing order, and visibility for each vector entry (Figure 22).

Figure 22: Legend tool for controlling display visibility.

To create the legend tool, right-click in the display panel and select Show legend. Each row in the legend tool is associated with an entry in the data vector. The dimensions of the vectors are listed, followed by the color the entry is drawn in and a toggle controlling its visibility. Entries are rendered in bottom-to-top order, so those at the top will be most visible.

  • To change an entry’s color, click the Set button, which will bring up a color menu.

  • To make an entry visible or invisible, use the Visible toggle box.

  • To change the order in which an entry is drawn, click and drag the entry vertically within the panel.

7.5.3 Editing and scaling data

As mentioned in Section 7.1, numeric probe data is described by a temporal sequence of knots, between which data is interpolated as described in Section 7.5.4.

Knot points can be made visible or invisible by setting the display’s knotsVisible property. This can also be done by right-clicking in the display and selecting Show knot points or Hide knot points. The rendered size of the knot, in pixels, is controlled by the knotSize property.

Since output probes typically have a very high knot density, their knots are set to be invisible by default.

For input probes, knot points (when visible) can be edited by moving, adding, or deleting them:

  • To move a knot point, left click on the knot and drag it.

  • To add a knot point, place the mouse at the desired location and double left-click.

  • To delete a knot point, right-click on the knot and select Delete knot point.

  • To edit a knot point data value, right-click on the knot and select Edit knot point.

The data for all numeric probes (input or output) can be scaled by right-clicking in the display and selecting Scale data .... This allows the user to enter a scaling factor that is applied uniformly to all the knot points.

7.5.4 Interpolation control

The data between knot points in a numeric probe is interpolated, using one of the interpolation orders described below, with the default interpolation order being Linear. Linear interpolation is almost always sufficient for output probes, which typically have a close spacing between knots that equals the simulation step size. Input probes, on the other hand, often have a much sparser knot spacing and so different interpolation orders can be useful. The interpolation order of input probes can be set by right-clicking in the display and selecting the Interpolation menu item. This is illustrated in Figure 23, which shows two input probes, one using Cubic interpolation and the other using Linear interpolation.

Figure 23: Two expanding tracks with probes showing cubic interpolation (upper) and linear interpolation (lower).

The interpolation options for a numeric probe include:

Step

Values are set to the values of the closest previous knots points.

Linear

Values are set by linear interpolation of the closest surrounding knot points.

Cubic

Values are set by cubic Catmull interpolation between the surrounding knot points.

CubicStep

Values are set by cubic Hermite interpolation between the surrounding knot points, with the slopes at knot points set to zero.

SphericalLinear

When interpolating quaternions or 4\times 4 rigid transformation matrices, 3D rotation values are set by piecewise spherical linear interpolation (i.e., "slerp", as described by Shoemake’s 1985 SIGGRAPH paper). Otherwise, interpolation is linear. Quaternions are assumed if the vector size of the numeric probe is 4, and 4\times 4 rigid transformation matrices are assumed if the vector size is 16.

SphericalCubic

When interpolating quaternions or 4\times 4 rigid transformation matrices, 3D rotation values are set by spherical cubic interpolation (i.e., "slerp", as described by Shoemake’s 1985 SIGGRAPH paper). Otherwise, interpolation is cubic. Quaternions are assumed if the vector size of the numeric probe is 4, and 4\times 4 rigid transformation matrices are assumed if the vector size is 16.

All of the above interpolation orders are instances of the enumeration type Interpolation.Order.

7.5.5 Large displays

A large display for a numeric probe can be created by right-clicking on the probe icon and selecting Large Display. This will create a large numeric display in a separate panel, allowing more precise inspection of data (Figure 24).

Figure 24: A large display for a numeric probe.

In addition to the functionality of the smaller displays, large displays have additional buttons, located across the top, for controlling the display range and other items. The first four of these are mode buttons:

Select: Places the display into selection mode, allowing knots (when visible) to be edited as described in Section 7.5.3.
Zoom in: Places the display into zoom in mode, in which the user can zoom in by either drag-selecting a region, or by left clicking on a point of interest. The mouse wheel can also be used to zoom in or out.
Zoom out:M Places the display into zoom out mode, in which the user can zoom out by left clicking on a point of interest. The mouse wheel can be also used to zoom in or out.
Translate: Places the display into translate mode, in which the user can translate a (zoomed-in) display using the left mouse button. The mouse wheel can also be used to zoom in or out.

There are also several additional buttons:

Increase y: Increases the vertical y range.
Decrease y: Decreases the vertical y range.
Decrease x: Decreases the horizontal x range.
Increase x: Increases the horizontal x range.
Fit Range: Fits the vertical and horizontal ranges to the current data.
Grid: Enables or disables visibility of a grid that aligns with the x and y axis tick marks.
Auto range: Enables or disables auto-ranging, in which the y axis is automatically adjusted to fit new data.

7.5.6 Cloning displays and exporting plots

Large data displays can be cloned by right-clicking in the display and selecting Clone display. This creates a duplicate display (Figure 25) containing a copy of all the data currently in the probe. However, the duplicate display is not attached to the probe, and so the data does not change when the probe is reset or additional data is added to the probe. This is useful for comparing outputs between different simulations.

Figure 25: A large display of an output probe and its clone.

In addition, it is possible to export a large display’s plot to an image file. Right-click in the display, choose Export image as ..., and enter the desired file name and file type in the chooser. The file’s type is indicated by its extension. A range of image file types are supported, including JPEG (.jpg, .jpeg), PNG (.png), scalable vector graphics (.svg), and encapsulated PostScript (.eps).