Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Plexon2RawIO's implementation of _get_analogsignal_chunks can be improved #1569

Open
nikhilchandra opened this issue Sep 25, 2024 · 6 comments

Comments

@nikhilchandra
Copy link

nikhilchandra commented Sep 25, 2024

Is your feature request related to a problem? Please describe.
The latest version of the PL2FileReader API includes a function capable of retrieving subsets of data from an analog channel. Switching _get_analogsignal_chunks to this function should (hopefully) remove the need to load entire channels into self._analogsignal_cache (this takes up a LOT of memory) and might also speed things up.

Describe the solution you'd like
I want to do the following:

  • Add content to the currently empty pypl2lib function "pl2_get_analog_data_subset"
  • Replace the underlying DLL to the latest version
  • In plexon2rawio.py, add code to retrieve fragment (i.e., segment) information for each stream to the _parse_header function
  • Rewrite _get_analogsignal_chunks to use the subset function in conjunction with fragment information in lieu of the present cached solution
  • Run tests to check whether the new method is faster

Describe alternatives you've considered
There are none that I can think of.

Related Question
What is the proper way to handle requests that exceed the limits of a recording's segments? For example, suppose I have a multi-trial recording in which 2 recording segments run from [0, 10] and [50, 60] (units in seconds). Suppose the user requests data from [5, 15] seconds -- obviously we have data from [5, 10] from the first segment. But for the interval [10, 15] -- do we just return zeros?

@zm711
Copy link
Contributor

zm711 commented Sep 25, 2024

Sounds cool to me. I'm sure @h-mayorquin is also interested.

@h-mayorquin
Copy link
Contributor

That sounds great. Concerning the question of what to do when asking for out of bound data. That is not clear to me, is this something that you have made a made a decision about @zm711 ?

@samuelgarcia
Copy link
Contributor

Hi. On spikeinterface side we decided to raise an error.
On neo side this have never been clear.
I think the most often case is to clip not to zero pad.

@zm711
Copy link
Contributor

zm711 commented Sep 27, 2024

I'm looking right now and a bunch of the memmap ones just do slicing [start:stop, :] so that would just follow whatever the numpy memmap convention allows, no. Where is the clipping machinery?

@nikhilchandra
Copy link
Author

@samuelgarcia Please, could you clarify what you mean by "clipping"?

@samuelgarcia
Copy link
Contributor

Hi sorry just reading this.
I am a bit fuzzy.
cliping was for : if stop in [start:stop, :] si greater than the shape[0] then stop is is "clip" to shape[0].
In short very often neo behave like numpy for upper boudary.
but spikeinterface do not, it is more strick.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants