Bad things happen when you exceed the size of the scratch arena #146
Loading…
Reference in New Issue
No description provided.
Delete Branch "%!s(<nil>)"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
I attempted to allocate more memory than the scratch pool could contain, and got bad accesses. This makes me sad.
It seems like it should either behave like a ring buffer, or just assert that we haven't overflowed our backing memory?
it should allocate a new chunk (calling
oc_mem_grow
if needed) and continue allocating from this chunk...I'm not sure how to easily send you a repro, but this is the structure of what failed:
The issue, presumably, was reading off the end of the allocated memory. I would have expected a crash or error log or something earlier while pushing chunks onto the arena.
Oh, ok. You just can't do that. Allocations are not guaranteed to be contiguous, because the arenas implementation must linked chunks within the current contraints of wasm memory.
So (assuming you were writing something to those allocations), you were at some point overwriting the header of the next chunk, meaning the next allocations would bound check on a garbage chunk cap. I was able to repro this, here's a log of what happens when your buffer catches up with the next chunk header:
You can see that after iteration 1085, the end of your buffer (which lags behind the actual memory allocated by the arena) overlaps the beginning of the current arena chunk. The next iteration the chunk has garbage cap, so way later you can allocate past wasm memory.
So... I don't know what we should/could do about this. At the very least we should emphasize this in the doc. We could also have canaries in the chunk headers in debug mode to detect that kind of things, but ultimately, there's not much we can do (at least in C) to catch that kind of error before it happens (for instance we can't check that people correctly use memory returned by malloc either).
As to what you should do to achieve what you want, if you need your data in a continuous array and can't compute the size beforehand at all, you should chain your allocations in a list, and do a collect pass once you know the total size.
Sounds good, thanks!