Skip to content

Fix General scientific notation carry (10E+14 -> 1E+15) - #106

Merged
borgar merged 1 commit into
borgar:masterfrom
arancool3000:fix/105-general-exp-carry
Nov 2, 2026
Merged

borgar merged 1 commit into
borgar:masterfrom
arancool3000:fix/105-general-exp-carry

Conversation

@arancool3000

Copy link
Copy Markdown
Contributor

What was wrong

In General, getExp() rounds the mantissa to 5 decimals but never checks for a carry, so values like 999999999999997 came out as 10E+14 instead of Excel's 1E+15 (also -999999999999997, 99999999999999.7, 9.9999999999997E-11).

Fix

In src/general.ts getExp(), if the rounded mantissa reaches 10, reset it to 1 and bump the exponent before building the output.

How tested

  • Added a regression test in test/ssf-general.spec.ts with the five values from the issue (fails before the fix with 10E+14, passes after).
  • npm test and the full table suite (vitest run) pass: 53080 passed; eslint and tsc --noEmit clean.

This addresses point 1 of #105 only. Points 2 (five significant digits from exponent ±99) and 3 (subnormals shown as 0) are left for a separate change.

Part of #105

AI-assisted (Claude); reviewed before submission.

@borgar
borgar merged commit 4bae55e into borgar:master Nov 2, 2026
3 checks passed
borgar pushed a commit that referenced this pull request Nov 2, 2026
**What:** For a three-digit exponent, Excel's General format writes five
significant digits in the mantissa, not six. For example,
`1.23456789E+99` gives `1.2346E+99` in Excel and `1.23457E+99` in
numfmt. This is point 2 of #105.

**Fix (`src/general.ts`, `getExp`):** the mantissa is rounded to 4
decimals when `|exp| >= 99`, and to 5 otherwise. A mantissa that rounds
to 10 is carried into the exponent (`9.999994E+200` → `1E+201`). The
carry is the same hunk as #106. If #106 merges first, this PR is left
with only the digit change.

**Tests:** added a case to `test/ssf-general.spec.ts` with the issue's
table: ±98 still gives six digits, and +99, +307, −200, +200 (with
carry) and −99 (with carry) give five. It fails before the fix and
passes after. The full `npm test` suite, eslint and tsc are clean.

**Not included:** point 3, a subnormal number written as `0`. The
existing test for #64 expects `General` of `2.33e-321` to give
`2.33E-321`, so that change needs the maintainer to decide first.

Part of #105 (point 2; point 1 is #106).

Co-authored-by: Aran Keyhan-Rad <arankeyhan@Arans-MacBook-Air.local>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants