Skip to content

Latest commit

 

History

History
138 lines (113 loc) · 7.03 KB

NATIVE_PRIMITIVES.md

File metadata and controls

138 lines (113 loc) · 7.03 KB

Native checksums, cryptography, and SIMD search

MiniLang exposes reusable public modules in std and keeps native hot paths behind compiler/runtime intrinsics. Argument validation stays in the public wrapper. Programs that do not use a checksum intrinsic do not include its helper or lookup table. Search accelerates existing first-class builtins, whose runtime entry points remain available under the established builtin model.

Checksums

std.checksum.crc32c and std.checksum.crc32 each provide:

  • compute(buffer)
  • computeRange(buffer, offset, length)
  • update(previous, buffer, offset, length)
  • verify(buffer, expected)
  • verifyRange(buffer, offset, length, expected)

CRC-32C uses the reflected Castagnoli polynomial 0x82F63B78 (0x1EDC6F41 in normal form). CRC-32/IEEE uses the reflected polynomial 0xEDB88320 (0x04C11DB7 in normal form). Both use initial value 0xFFFFFFFF, reflected input/output, and final XOR 0xFFFFFFFF.

CRC-32C dispatches to x64 SSE4.2 CRC32 instructions, processing eight bytes per main-loop iteration. The bit-compatible table implementation is selected when SSE4.2 is unavailable or disabled. CRC-32/IEEE uses runtime-gated PCLMULQDQ folding for ranges of at least 64 bytes. Short inputs, disabled/unsupported PCLMUL, and the last 0–15 bytes use its separate table. The x86 CRC32 instruction itself does not implement IEEE CRC-32. Incremental seeds and zero-length updates have identical semantics in both paths. No AVX, SSSE3 or SSE4.1 capability is needed by the IEEE folding path. Range and tag checks occur once before the allocation-free raw update loop.

The IEEE folding/reduction implementation is derived from Intel ISA-L. Distribute its notice with generated programs that include that helper. See the measurements.

Cryptography

std.crypto provides SHA-256, SHA-384, HMAC-SHA-256, HMAC-SHA-384, HKDF-SHA-256, HKDF-SHA-384, PBKDF2-HMAC-SHA-256, PBKDF2-HMAC-SHA-384, X25519, secure random bytes, constantTimeEquals, and best-effort secureZero. std.crypto.aes_gcm provides AES-256-GCM seal/open and encrypt/decrypt. std.crypto.ecdsa_p256.verify(publicKey, message, signature) verifies ECDSA P-256/SHA-256 signatures. Public keys use the 64-byte big-endian X || Y representation and signatures use the fixed-width 64-byte IEEE-P1363 r || s representation.

Cryptographic operations are delegated to Windows CNG (bcrypt.dll) or Linux OpenSSL 3 (libcrypto.so.3) rather than application-level MiniLang loops. The public API, validation and result layout are identical on both targets. Windows one-shot SHA-256/384 and HMAC use CNG algorithm pseudo-handles, so repeated calls do not reopen a provider; PBKDF2 and AES-GCM still create their own required native state. Concurrent hash/HMAC calls are regression tested on both targets. AES-GCM accepts exactly 32-byte keys, nonces from 12 through 16 bytes, and tags from 12 through 16 bytes. Decryption returns an error and wipes its temporary output when native authentication fails, so unauthenticated plaintext is never returned. Ciphertext, tag, nonce, AAD, or key changes therefore fail authentication.

ECDSA verification hashes the message with SHA-256 in the public wrapper and delegates curve operations to CNG or OpenSSL. It returns false for malformed arguments, invalid public points, backend failures, and signature mismatch. Signing and key generation intentionally remain build-time concerns rather than executable runtime APIs.

X25519 uses CNG's curve25519 provider on Windows and EVP raw X25519 keys on Linux. The Windows bridge converts CNG's raw-secret byte order to RFC 7748 order; both backends clamp a temporary private-key copy and reject an all-zero shared secret. Random bytes use CNG's system-preferred RNG or OpenSSL RAND_bytes.

pbkdf2Sha256(password, salt, iterations, length) and pbkdf2Sha384(password, salt, iterations, length) validate positive iteration and output counts, then call CNG BCryptDeriveKeyPBKDF2 or OpenSSL PKCS5_PBKDF2_HMAC. Password and salt are byte arrays; callers remain responsible for choosing a protocol-specific iteration count and wiping caller-owned secret buffers.

Security boundaries:

  • constantTimeEquals has no content-dependent early exit for equal-length byte arrays; a public length mismatch is rejected immediately.
  • secureZero overwrites the supplied mutable byte buffer, but it cannot erase earlier language/runtime copies. Avoid unnecessary secret copies and wipe caller-owned key buffers as soon as possible.
  • Errors never include keys, plaintext, derived secrets, or random material.
  • Linux crypto images require an x64 OpenSSL 3 installation providing libcrypto.so.3; Windows uses the system-provided CNG library.
  • Protocol-level nonce uniqueness, key rotation, transcript construction, and trust decisions remain the caller's responsibility.

Search dispatch

Byte and byte-indexed string indexOf/lastIndexOf use AVX2 candidate scans in 32-byte blocks, SSE2 scans in 16-byte blocks, or a scalar fallback. A candidate is confirmed by the existing exact memory comparison, so multi-byte patterns and overlapping reverse matches preserve previous semantics. startsWith, endsWith, and equality continue using the SIMD memory comparison path. Empty patterns, clamped start positions, embedded zero bytes, and invalid argument behavior remain unchanged.

CPU features and testing

std.cpu publishes these flags: SSE2, SSE42, AVX, AVX2, AES_NI, PCLMULQDQ, SHA, ERMS (128), and FSRM (256). The existing flag values are unchanged. Detection runs once during process startup. AVX and AVX2 are active only when CPUID and XGETBV confirm OS support for XMM/YMM state.

features() returns detected capabilities and activeFeatures() returns the current dispatch mask. setDispatchMaskForTesting(mask) can only remove detected features; passing -1 restores them. It exists for deterministic fallback tests and benchmarks, not for application feature spoofing.

Copy helpers use bounded scalar/SSE2 loads for small ranges, unrolled AVX copies where available, and ERMS REP MOVSB starting at 64 KiB. FSRM does not unconditionally select short REP: it was slower for unaligned short copies on the evaluation CPU. Fill helpers use SIMD through 256 bytes and retain the established REP STOSB path above that, bypassing redundant vector setup. FSRM is a MOVSB capability, not a guarantee about STOSB. Baseline x64 remains supported; no new minimum CPU requirement is introduced. Copy ranges must remain non-overlapping, as already required by copyBytes and copyArray.

The focused suites are tests/isa_runtime.ml, tests/checksum_runtime.ml, tests/crypto_cng.ml, tests/ecdsa_p256.ml, and tests/simd_search.ml. Reproducible diagnostic benchmarks and comparison guidance are in benchmarks/.