Construct a Tracks from a data.frame or file(s)

Usage

read_tracks(
  x,
  mapping = list(id = NULL, time = NULL, x = NULL, y = NULL, angle = NULL, weight = NULL),
  angle_unit = c("radians", "degrees", "auto"),
  time_type = c("auto", "posix", "seconds", "frames"),
  tz = "UTC",
  fps = NULL,
  normalize_xy = FALSE,
  origin = NULL,
  radius = NULL,
  dialect = NULL,
  dialect_args = list(),
  read_opts = list(delim = NULL, decimal = NULL, sheet = NULL),
  mutate = NULL,
  keep = NULL,
  drop = NULL,
  id_from_filename = TRUE,
  validate = TRUE,
  format = NULL,
  id_collision = c("error", "namespace"),
  on_invalid = c("error", "drop")
)

Arguments

x: data.frame, file path, or character vector of file paths
mapping: named list for column mapping: id, time, x, y, angle, weight. Any missing will be guessed when possible.
angle_unit: "radians","degrees", or "auto" to guess from values
time_type: one of "auto","posix","seconds","frames". Numeric time is taken to be a frame index unless `time_type = "seconds"` (or the column is a duration): seconds are recorded as such in the Tracks metadata (`time_unit`), so [elapsed_seconds()] and the kinematics use them directly, with no frame rate.
tz: Time zone of timestamps that carry none (default "UTC"): in the time column and the kept columns (see `keep`) of a file, in an Excel date-time cell, and in a text time column of a data frame. A dialect is not given `tz`: it applies to the dates and text date-times of the table the dialect returns. A bare date is the first instant of that day in `tz` (midnight, or the end of a DST change that skips midnight; so is an Excel date-time cell at such a midnight). A wall-clock time that does not exist in `tz` (the hour skipped when clocks go forward, or a day skipped altogether) is an error. A timestamp that ends in a zone designator (`Z`, `+02:00`, `-0130`, `+02`) is that instant whatever `tz` is, and a date-time column of a data frame keeps its instants.
fps: Frames-per-second. Required when time_type = "frames" (the time column is kept as a raw frame index). When a single finite positive value is supplied and the time column is frame-indexed, it is stored as the Tracks capture frame rate (see [frame_rate()]); [elapsed_seconds()] then performs the frame -> seconds conversion once, on demand, so kinematics and the app adopt it automatically without converting twice.
normalize_xy: FALSE (default) keeps (x,y) as supplied. TRUE independently centres each trajectory on its own bounding-box midpoint and scales it to the unit circle – shape-preserving only, NOT calibration (it does not preserve bearing from a fixed arena origin). See [tracks()] for the full explanation.
origin, radius: Optional fixed unit-circle calibration forwarded to [tracks()]: `origin = c(x, y)` centre and positive scalar `radius` in raw coordinate units. Supplied together, they map every trajectory uniformly via `(xy - origin) / radius`, preserving bearings from the origin. Mutually exclusive with `normalize_xy = TRUE`.
dialect: optional registered dialect name. The dialect reads `x` itself (`read_tracks()` does not parse the file first) and is called once per file when `x` holds several paths. See [register_loader_dialect()].
dialect_args: named list of extra arguments forwarded to the dialect function (e.g. list(bodypart = c("head","thorax")))
read_opts: list of file-reading overrides: `delim` (field separator) and `decimal` (decimal mark) for a text file, `sheet` (Excel worksheet name or number) for an Excel file, and `ext` (file type, for a path without its extension). Any `NULL` element is auto-detected. Defaults to all-auto. With a `dialect`, the non-`NULL` entries are handed to the dialect: every built-in takes `decimal`; `trackmate`, `trex`, `sleap` and `idtrackerai_wide` also take `delim` and `ext` for a text file and `sheet` for an Excel file (`decimal` and `delim` are refused for Excel, `sheet` for a text file); `ethovision` and `anymaze` take `delim` (for EthoVision it must be the export's own separator). An entry a dialect cannot use stops with a `radiatR_invalid_argument` error, as does any entry for a registered dialect whose function has no `read_opts` argument. Without a dialect, an entry that does not fit the file's type (`delim` or `decimal` for an Excel or Arrow file, `sheet` for a text or Arrow file) stops the same way. A data frame `x` takes only `decimal`, the mark of any numeric column it holds as text.
mutate: list of transformations applied after reading (named functions or formulas)
keep: extra columns to retain besides the mapped ones and those a dialect produces (NULL = none). All other columns are dropped; a name that is not in the data is ignored without a message. A kept column of a file is read as date-times by content, under every reader: when its reader typed it as dates or date-times, or every value that is not missing is a date-time in one layout, it is parsed in `tz` as the time column is (a value that is no instant there stops the load); otherwise it stays as read.
drop: drop these columns after mapping
id_from_filename: if TRUE and id missing, derive id from file stem when reading multiple files
validate: Deprecated and ignored: `Tracks` invariants are always checked. Any value but `TRUE` warns.
format: Optional loader format name or list spec registered via [register_loader_format()]
id_collision: how to handle a trajectory id that appears under more than one source file during a multi-file import: `"error"` (default) stops with a message naming the colliding id(s) and files; `"namespace"` rewrites every id as `<file>::<id>`, where `<file>` is the file name less extension, or the full file name where two files share that, or the trailing part of the path that tells them apart (`a/n1.csv`). Has no effect on single-file or data.frame input.
on_invalid: How to handle rows whose x/y coordinates are missing or non-finite: "error" (default) aborts with a radiatR_invalid_rows condition listing the affected rows and ids; "drop" removes them and signals a radiatR_dropped_rows warning with the same payload. It covers nothing else: a missing time, id or weight is an error either way, and so is a malformed value (see "Malformed values").

Value

Tracks

Time units from a dialect

A dialect that knows the unit of its time column says so, and under the default `time_type = "auto"` that decides how the column is read: TrackMate `POSITION_T` (when the version 7 units row gives seconds), ToxTrac `Time (sec)`, idtracker.ai `seconds` and TRex `time` are seconds, and need no `fps`; `FRAME`/`frame` columns, the DeepLabCut frame index and SLEAP `frame_idx` are frames. `ethovision` (`Trial time`, or `Recording time` through `time_col`) and `anymaze` (`Time`, a duration) declare seconds; an `ethovision` data frame's time is taken as seconds (EthoVision writes no other unit), and a file's units row is checked. A contrary explicit `time_type` stops with a `radiatR_time_type_mismatch` error, as do files of one import that disagree.

Masked points

The `deeplabcut`, `sleap`, `ethovision` and `anymaze` dialects take each position from one or more points (bodyparts, nodes, body points; ANY-maze's centre point alone). EthoVision writes `-` and ANY-maze an empty cell for a sample with no position; neither has scores, thresholds, or for `anymaze` a `partial` argument. A point is masked in a frame when its coordinates are missing or not finite, when its score (likelihood) is 0, or when a threshold is given (`dialect_args = list(likelihood_min = )` for DeepLabCut, `score_min` for SLEAP) and its score is missing or below it. A frame with a masked point has no position, unless `dialect_args = list(partial = "mean")` asks for the score-weighted mean of the points that remain. The dialect never removes a row: frames without a position meet `on_invalid` like any other missing coordinate.

What was masked is reported in one message (class `radiatR_attrition`) and kept in the `attrition` element of the Tracks metadata: a data frame with one row per source file and point, giving the rows in which the point was masked (`masked`), the rows read from that file (`frames`), the rows left without a position (`frames_no_position`), and the threshold and `partial` setting used. `c()` row-binds these records; a subset keeps the whole record of its load, so recombining subsets of one load repeats it.

Ids

An id is a label, not a measurement. In a delimited file the id column (the one named in `mapping`, or any column named `id`, `track`, `trajectory`, `animal` or `subject`) is kept as the text in the file wherever reading it as numbers would merge ids that are written differently: `1.1` and `1.10`, `1` and `1.0`, `01` and `1`, or `2024-01-01 00:00` and `2024-01-01 00:00:00`. Ids that read as distinct numbers (or dates) stay numbers. Where the delimited files of one import do not agree on the type of the id column, or two of them hold one number (written `1.1` in one and `1.10` in the other, or alike), the ids of each file are the text in that file (for an Excel file, the text of its cells); two files that write one id alike collide (see `id_collision`). Where the types still disagree in tables that cannot be read again for their text (a dialect's tables, Parquet or Feather files), the load stops with a `radiatR_id_type_mismatch` error. A data frame and an Excel or Arrow file read alone keep the types they come with. Ids that differ only by surrounding whitespace (`" 1"` and `"1"`) stop the load with a `radiatR_ambiguous_ids` error, as readers disagree on whether they are one id. An `anymaze` file is read as text, so the column its `id_col` names keeps its text. For `ethovision`, `id_col` applies to a data frame only (a file is one track, id `"1"`). Not covered: an id column that another dialect is pointed to by an argument of its own (its ids may still be merged).

Reading delimited files

Whichever reader is installed (readr, then data.table, then base R), a delimited file is read as one table: `"` is the only quote character, nothing is a comment (`#` and `'` are text), blank lines are skipped, an empty cell is missing, a cell keeps its surrounding whitespace and column names lose theirs. The fields of each line are counted first: a row with more or fewer fields than the header (a last line cut off, an unclosed quote, a table written with row names or with a separator at the end of every row) stops the load with a `radiatR_malformed_file` error naming the lines (`problem = "ragged"`), as does a reader returning fewer rows than the file holds (`"rows"`) and a column name used twice (`"duplicate_columns"`; a dialect handles its format's repeated columns itself), and a delimited file that reads as a single column (`"one_column"`).

Malformed values

A cell in an x, y, angle, weight or time column that is present but does not parse (`"oops"`, `"1,5"`, a stray header or units row) stops the load with a `radiatR_malformed_values` error, a subclass of `radiatR_invalid_rows`. Its `columns`, `rows`, `values`, `source` and `dialect` fields name the column, the data rows (counted from the first row after the header, within its own file in a multi-file import), the offending strings, the file and the dialect, `decimal` (the mark the file was read with) and `suggested_decimal` (the other mark when the values look written with it, else `NULL`). Such a value is never dropped or turned into `NA`. Blank cells and `NA` are missing values; `nan` and `inf` are read as such. Files are read with no digit-grouping mark, so `1,5` is malformed unless `read_opts = list(decimal = ",")` says the comma is the decimal mark. Where it is, a value holding a `.` is malformed in turn, since `1.234` could as well be 1234. A semicolon-separated file is taken to use decimal commas unless its first lines show decimal points and no decimal comma (`1.5` or `0.125`, which cannot be digit grouping, or the fraction of a duration such as `0:00:00.320`); `read_opts = list(decimal = ".")` says so outright. A point found that way must be shown by a column read for a role (x, y, time, angle, weight): where such a column holds only values that could be digit grouping (`523.117`, `1.234`) and the clear decimal points are all in other columns (a label `1.5`), the load stops with a `radiatR_ambiguous_decimal` error (fields `columns`, `values`, `evidence`, `decimal`, `source`, `dialect`) rather than read `1.234` for what may be 1234. So does a file where no data row of the first 20 lines is whole (a quoted field runs past them), as the point cannot then be checked. Give the mark with `read_opts`. The same holds for the built-in dialects that sniff the mark (ANY-maze, the others read as tables, and EthoVision when not comma-separated). Other tab- and comma-separated files are read with a point.

Every built-in dialect checks its source columns the same way, before any reshaping, so the error names the header as the program wrote it and the row of the file's table. A missing or non-finite position found afterwards (see `on_invalid`) is still counted in the rows of the table the dialect returns, which for a wide format is the reshaped one. ANY-maze writes time as a duration (`0:01:02.500`), read as seconds; a cell that is not one is malformed.

A time column given as text must hold either numbers throughout or timestamps in one layout throughout: `YYYY-MM-DD HH:MM:SS`, `YYYY-MM-DD HH:MM` or `YYYY-MM-DD` (with `-` or `/`), or ISO 8601 `YYYY-MM-DDTHH:MM:SS` with an optional `Z` for UTC. A bare date beside date-times is midnight.

Examples

df <- data.frame(
  id   = rep(c("A", "B"), each = 4),
  time = rep(0:3, times = 2),
  x    = c(1, 1.2, 1.4, 1.5, -0.2, -0.1, 0, 0.15),
  y    = c(0, 0.1, 0.3, 0.4, 1.0, 1.1, 1.3, 1.5)
)
read_tracks(df, mapping = list(id = "id", time = "time", x = "x", y = "y"))
#> Tracks: 2 trajectories, 8 observations
#> Columns: id='id', time='time', angle='angle' (radians), x='x', y='y'
#>   id time    x   y      angle
#> 1  A    0  1.0 0.0 0.00000000
#> 2  A    1  1.2 0.1 0.08314123
#> 3  A    2  1.4 0.3 0.21109333
#> 4  A    3  1.5 0.4 0.26060239
#> 5  B    0 -0.2 1.0 1.76819189
#> 6  B    1 -0.1 1.1 1.66145621

# A DeepLabCut predictions file, recorded at 40 frames per second
dlc <- system.file("extdata", "dlc_single_wasp_predictions.csv",
                   package = "radiatR")
read_tracks(dlc, dialect = "deeplabcut", fps = 40)
#> Tracks: 1 trajectories, 300 observations
#> Columns: id='id', time='time', angle='angle' (radians), x='x', y='y'
#>   id time        x        y   head_x   head_y stinger_x stinger_y     angle
#> 1  1    0 1084.859 421.3867 1086.238 421.5013  1083.866  421.3042 0.3704886
#> 2  1    1 1084.742 421.5025 1086.020 421.5443  1083.370  421.4577 0.3706179
#> 3  1    2 1084.573 421.7333 1085.545 421.8074  1082.950  421.6096 0.3708553
#> 4  1    3 1083.631 423.8910 1085.084 422.5898  1081.628  425.6838 0.3728769
#> 5  1    4 1082.897 424.1358 1085.193 422.4883  1078.615  427.2081 0.3733029
#> 6  1    5 1082.439 424.0955 1084.933 422.5358  1077.953  426.9004 0.3734143