Skip to content

SpikeGadgetsRawIO: detect timestamp gaps and expose acquisition timestamps (impl of #1773) #1856

Open

Description

@h-mayorquin

I have been working with SpikeGadgets and I realized that this format also has timestamps in each packet's header (4-byte uint32, see the packet diagram in spikegadgetsrawio.py), but SpikeGadgetsRawIO does not currently read it. We should use it to run an integrity check on parse (like the one added to IntanRawIO in #1484) and to expose the per-packet hardware timestamps via a get_spikegadgets_timestamps() method (parallel to #1652 for Intan). This is the SpikeGadgets-specific implementation of the general gap-handling API in #1773.

Activity

  1. DanceoftheAurora commented on Nov 6, 2026

    @DanceoftheAurora

    AI-assisted reproduction of #1856, checked locally on d50b9b9 (Neo 0.15.0.dev0, Python 3.12.14, NumPy 2.5.4, quantities 0.16.4).

    I generated five 340-byte .rec files with one channel at 30 kHz; no external recordings were needed. All five parse without a warning into one segment, and SpikeGadgetsIO produces the same AnalogSignal.times: [0, 33.333, 66.667, 100] µs relative to the first sample.

    Stored ticks Synthetic case Timing result
    [100,101,102,103] Continuous Correct relative times
    [100,101,103,104] One missing tick Last two samples placed 33.333 µs early
    [2**32-2,2**32-1,0,1] Normal uint32 wrap Correct relative times
    [2**32-2,0,1,2] Missing tick at wrap Last three samples placed 33.333 µs early
    [100,101,0,1] Counter reset Silently treated as continuous; elapsed time is unknown

    For either missing-tick example, t_stop is 133.333 µs instead of the known synthetic 166.667 µs. The sample values themselves are intact. Minimal synthetic reproducer and measured results.

    I can help with a focused implementation and synthetic tests. Two scope questions before proposing a change:

    1. Would you prefer (A) native tick access plus default continuity errors and an explicit integrity-check bypass first, or (B) full gap_tolerance_ms segmentation in the same PR? A would change today's read behavior: files with gaps would need an explicit opt-out.
    2. Should the accessor be private, as suggested in the recent General API for handling sample gaps on rawio #1773 discussion, or use the public get_spikegadgets_timestamps name originally proposed here? Either way, I suggest preserving the original uint32 ticks, with wrap/gap/reset tests, rather than silently unwrapping or interpolating them.
  2. h-mayorquin commented on Nov 6, 2026

    @h-mayorquin
    ContributorAuthor

    Can I ask you what is your use case? do you have real files that have this problem?

  3. DanceoftheAurora commented on Nov 7, 2026

    @DanceoftheAurora

    Thanks for asking. The use case I am exploring is timing QC before aligning invasive neural recordings with behavioral labels for decoding: being able to inspect the acquisition ticks, including a nonzero initial tick or a discontinuity, before relying on a nominally regular time axis. This is prospective validation work; I do not currently have a verified real recording showing the missing-tick/reset problem. The examples in my earlier comment are synthetic packet fixtures.

    I have now checked the public Neo/Gin fixtures as well. Three complete, structurally valid excerpts contain 8,000, 2 and 6,228 packets. All 14,227 adjacent transitions are +1, and the proposed getter agrees with an independent struct.unpack_from("<I") decode of every stored tick. I did not find an acquisition gap, reset or rollover in these excerpts. Together they contain less than half a second of data, so this is limited negative evidence. Two other downloaded stubs did not pass complete-packet/framing checks and were excluded from the acquisition-timing conclusion.

    Script, results, source links and file hashes are available for reproduction. No recording bytes were modified or fault-injected in that check.

    That is why #1949 is kept as a draft accessor-only step: it exposes the stored ticks without changing segmentation or nominal times, and does not claim to solve real-world gap handling. I would be glad to validate against an affected example if one is available before proposing continuity-check behavior.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions