Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Emit an event to state approximate record size, to check for reaching 400KB record sizes #3

Open
a-h opened this issue Nov 24, 2021 · 0 comments

Comments

@a-h
Copy link
Owner

a-h commented Nov 24, 2021

Consider whether it's possible to implement a pointer system.

For example, given the input data:

{
  "a": "large_blob",
  "b": "small datum", 
}

The library could be designed to break it up into pointers to the large data, i.e. would store two records:

The state where a key is replaced with a pointer.

{
  "a": { "stream_pointer": "hashOfBlob" },
  "b": "small datum"
}

And a pointer record:

{
   "_pk": "hashOfBlob",
  "data": "large_blob"
}

When the data is requested, the pointers would be followed to collect the data from the table.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant