Add a timeline type for relative time

[emilk]

We currently have two time types:

• Sequence
• Temporal

The problem is that the temporal type is used both for absolute times (2024-01-10 09:44:01Z) and relative times (+32m 12s).
Because of this problem, relative times will show up as 1970-01-01 00:32:12Z in arrow formatting.

Relative times are useful for "time since robot started" or whatever.

I suggest we split Temporal into absolute and relative. The arrow datatype would then become:

• Sequence - > DataType::Int64
• Duration - > DataType::Duration64(TimeUnit::Nanosecond)
• Timestamp - > DataType::Timestamp(TimeUnit::Nanosecond, None)

Should do this first

• Uniquely define an index/timeline only by name #9084

Related

• Should Temporal vs. Sequence really be a property of the timeline itself? #8471
• Unify RowId and time columns #9082

Future work

• Adding even more types of indices, e.g. UUID

TODO

• Introduce new SDK APIs (see below)
  • Python
    • Add rr.set_index to replace rr.set_time_* #9166
    • Add rr.IndexColumn #9179
    • Update docstrings for TimeInt and TimeRangeBoundary #9191
    • Python API: Rename sequence time parameter to seq #9192
    • Change signature of set_time_ctrl to match set_time #9342
  • New C++ API for timestamp/duration indices #9200
  • Rust
    • Introduce IndexCell #9226
    • Add RecordingStream::set_index #9236
• Split TimeType::Time; changing dataloader API #9292
• Remove struct Time
• Fix all TODO(#8635):s
• Unify naming
  • Use the same names (sequence, duration, timestamp) in all language APIs
  • Use consistent time unit names for our API #9343
  • Use consistent time unit names for our API #9343
  • Revert index related names
    • Rename IndexCell -> TimeCell
    • Rename set_index -> set_time
    • Rename nonstatic -> temporal

Future work

• Rename "timeline" to "index" everywhere
  • Rename disable_timeline and reset_time

[abey79]

Here is a fleshed out version of the proposed Python API that annotates the single-non-None arg restriction:

from __future__ import annotations

from typing import overload
import datetime

import numpy as np


@overload
def set_index(timeline: str, *, sequence: int) -> None: ...


@overload
def set_index(timeline: str, *, timedelta: int | float | datetime.timedelta | np.timedelta64) -> None: ...


@overload
def set_index(timeline: str, *, datetime: datetime.datetime | np.datetime64) -> None: ...


def set_index(
    timeline: str,
    *,
    sequence: int | None = None,
    timedelta: int | float | datetime.timedelta | np.timedelta64 | None = None,
    datetime: datetime.datetime | np.datetime64 | None = None,
) -> None:
    """Set exactly ONE of the optional arguments"""

    if sum(x is not None for x in (sequence, timedelta, datetime)) != 1:
        raise ValueError("Exactly one of `sequence`, `timedelta`, and `datetime` must be set")

    # this is needed if we want to use the `datetime` module, as it is shadowed by the argument name
    import datetime as dt

    ...


nanos = 1000
set_index("frame_nr", sequence=42)

set_index("time_since_start", timedelta=np.timedelta64(nanos, "ns"))
set_index("time_since_start", timedelta=15)
set_index("time_since_start", timedelta=15.0)
set_index("time_since_start", timedelta=datetime.timedelta(weeks=3))

set_index("capture_time", datetime=np.datetime64(nanos, "ns"))
set_index("capture_time", datetime=datetime.datetime(2012, month=3, day=14, hour=15, minute=9, second=26))

# NO!
set_index("invalid", sequence=42, timedelta=15)

Here is what mypy has to say about the above:

❯ mypy set_index.py
set_index.py:39: error: No overload variant of "set_index" matches argument types "str", "int", "int"  [call-overload]
set_index.py:39: note: Possible overload variants:
set_index.py:39: note:     def set_index(timeline: str, *, sequence: int) -> None
set_index.py:39: note:     def set_index(timeline: str, *, timedelta: int | timedelta64[timedelta | int | None]) -> None
set_index.py:39: note:     def set_index(timeline: str, *, datetime: int | datetime64[date | int | None]) -> None
Found 1 error in 1 file (checked 1 source file)

[abey79]

One thing to clarify for duration is the meaning of int and float value. It might be tempting to use int as nano, as this is the underlying stored value, but:

• it would be weird for float to NOT be seconds
• it would be extremely error prone for int and float to not have the same unit (the former is very often silently cast to the latter in Python)

So my suggestion would be for both int and float to be interpreted as seconds