Skip to content

Fix undefined behavior in the OASIS real and integer encoding - #2509

Closed
nikosavola wants to merge 2 commits into
KLayout:masterfrom
nikosavola:pr-oasis-ub
Closed

nikosavola wants to merge 2 commits into
KLayout:masterfrom
nikosavola:pr-oasis-ub

Conversation

@nikosavola

Copy link
Copy Markdown
Contributor

The OASIS reader and writer read the bits of a float or double through a union. That is undefined behavior in C++, so they now use memcpy. The writer also had two copies of the real encoding, one for float and one for double. They are now one template.

The writer negated a negative int64 before writing it. For the minimum value $-2^{63}$ this is undefined, and in practice it wrote $-0$. OASIS cannot store this value, so the writer now reports an error for it.

The reader reserved space for the point count given in the file. A broken file could ask for a huge amount of memory this way. The reservation is now capped at $2^{16}$ points, and the list grows as usual after that.

The new tests write integer and real values at the edges of their ranges and read them back. The test for the minimum int64 fails on master.

- the minimum int64 is written as -0 on master, so that test fails until the fix
- reals are copied with `memcpy` instead of read through a union, one `write_real` template serves float and double
- the minimum int64 is now an error, negating it was undefined and wrote -0
- the point list reservation is capped, since the point count comes from the file
@nikosavola nikosavola closed this Oct 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant