diff --git a/CHANGELOG.md b/CHANGELOG.md index 77e38e8fc..2f7cd1e55 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,6 +11,22 @@ - **Explicit SIMD128 bodies for every sidecar matmul kernel**: Q4_K, Q5_K, Q6_K, Q5_0 and Q5_1 join Q8_0 and Q4_0 with hand-written `__wasm_simd128__` paths (the K-quant int8 dots on `i32x4.dot_i16x8_s`, the Q5 fifth-bit plane via a byte swizzle); the dense FP32 GEMM gets a 4×8 register-blocked, k-blocked micro-kernel; BF16 and FP16 widen eight weights per step. BF16 and FP16 also gain NEON bodies (none existed on any tier). Results stay bit-identical in accumulation order to the portable bodies. - **Threaded dense kernels**: `skainet_fp32_matmul`, `skainet_bf16_matmul` and `skainet_fp16_matmul` now split the output over the shared row-thread pool like the packed kernels do (the GEMM over the longer side of `C`, the 16-bit kernels over columns) — the mt sidecar's pthreads in a browser, the native pool on the JVM. - **`WasmCpuOps.matmul`**: a dense 2-D `A · B` on heap FP32 operands runs the sidecar GEMM; the common `[k, n]` loop no longer serves that shape on wasm (it was the only dense path that never reached the sidecar). +- **Ternary kernels on the wasm tier**: the sidecar now exports `skainet_ternary_f32_gemv`, `skainet_ternary_lmhead_stage1` and `skainet_bitnet_gemv_tq2_0`, and `WasmTernaryKernelPack` (installed by `WasmSimd.install()`) registers them under the same `KernelDispatch` keys the JVM FFM and Android JNI ternary packs serve — exact FP32 × `BitNet-b1.58`, the fused multi-plane `BitNet-planes` lm_head and int8 × `TQ2_0`. A BitNet weight in a browser previously went to the decoding reference kernel. The portable branch of `skainet_ternary_f32.c` (MSVC and Emscripten) now holds the vendored kernel's 256-entry decode table as a compile-time constant with a SIMD128 body and threads over output rows through the shared row pool; `bitnet_gemv` gains a SIMD128 block body and the same row threading on every tier. +- **Direct `x · Wᵀ` on the JVM**: `Fp32MatmulWtKernel` / `KernelProvider.matmulFp32Wt()` (and the JVM-only `Fp32MemSegMatmulWtKernel` / `MemSegKernelProvider.matmulFp32WtMemSeg()`) expose `skainet_fp32_matmul_wt`, which the wasm sidecar has served since 0.61.0 and the JVM never bound. `NativeKernelProvider` carries it; `DefaultCpuOpsJvm.matmulWeightTransposed` multiplies a dense `[out, in]` FP32 weight in place through it — a segment-backed weight where it lies, a heap weight after one resident off-heap copy — instead of transposing the weight once into `[k, n]` and caching the copy. The wasm provider exposes the same accessor (`WasmFp32MatmulWtKernel`), and `KernelProvider.supports("matmul_wt", …)` reports it. +- **Kernel support matrix, fourth table** *Kernel packs (view dispatch)*: the ternary formats and the direct `x · Wᵀ` per platform, which the `matmul` accessors the first table introspects do not cover. +- **`engine-fp32-linear`** in `jvm-cpu-publish` and `wasm-bench`: a `Linear` layer's `x · Wᵀ` (16 tokens, 768 → 3072) through `ctx.ops.matmulWeightTransposed`; on the JVM `--provider panama` measures the cached-transpose path and `--provider ffm` the direct kernel. +- **Resident activations on the wasm tier**: `PlatformStorage.offHeapAllocator` (Kotlin/Wasm) binds `MemoryDomain.HOST_OFFHEAP` to a backend's memory; `WasmSimd.install()` binds it to the SIMD128 sidecar's linear memory (`WasmLinearStorage : Storage.OffHeap`, `WasmLinearTensorData : FloatArrayTensorData`), so `StorageCapabilities.current()` reports off-heap support in a browser and a `ForwardScope` frees sidecar activations at `reset()`. `WasmCpuOps` now keeps the outputs of `matmul`/`matmulWeightTransposed` resident and runs `add`/`subtract`/`multiply`/`divide`, `relu`/`sigmoid`/`silu`/`gelu`/`erf`/`tanh` and `sum`/`mean` in the sidecar on resident operands (transcendentals also on heap ones), reading a tensor back to the Kotlin heap only when something outside the sidecar needs it. Ambient outputs are freed when their tensor is collected (`FinalizationRegistry`); `WasmSimd.residentBudgetBytes` bounds what stays resident and `WasmCpuOps.toResident()` uploads an input once. New shared C kernels `skainet_f32_binary` / `skainet_f32_unary` / `skainet_f32_reduce_all` / `skainet_f32_reduce_axis` (SIMD128 and NEON bodies, row-pool threading) are in every native build as well. +- **`wasm-bench`**: a `laya.chain` case (`x · Wᵀ`, GELU, multiply, bias add, sum, end to end) and `operands=resident` variants of `engine-elementwise-add` and `engine-reductions-sum`. +- **Fused softmax, normalisations and attention on both tiers**: new shared C kernels `skainet_f32_softmax_rows`, `skainet_f32_layer_norm`, `skainet_f32_rms_norm` and `skainet_f32_sdpa` (`f32_fused.c`, SIMD128 and NEON bodies, row-pool threading) — one pass each where the ops layer ran several allocating scalar passes. `skainet_f32_sdpa` carries the full `scaledDotProductAttention` contract: grouped-query heads, `causal`, the sliding window with `rightContext`, per-batch `keyLengths` and the dense additive mask with its trailing-aligned broadcast. New SPI faces `Fp32SoftmaxKernel` / `Fp32NormKernel` / `Fp32AttentionKernel` behind `KernelProvider.softmaxFp32()` / `normFp32()` / `attentionFp32()` (reported by `supports("softmax" | "layer_norm" | "rms_norm" | "sdpa", ["Float32"])`), served by `NativeKernelProvider` (FFM) and `WasmKernelProvider`. On the JVM, `DefaultCpuOpsJvm` now routes `softmax`/`logSoftmax` over the last dimension (from 1024 elements) and `scaledDotProductAttention` to the native kernels when the FFM provider is loaded, replacing the scalar per-head loop; on wasmJs `WasmCpuOps` runs them in the sidecar with the outputs resident. +- **`FusedNormOps`** (`skainet-lang-core`): an optional interface an op set may implement beside `TensorOps` — `layerNorm` / `rmsNorm` over the trailing dimensions — which `LayerNormalization` consults for FP32 input before its decomposed chain. `DefaultCpuOpsJvm` (native kernel loaded) and `WasmCpuOps` implement it; recording and differentiating op sets never see it, so recorded graphs and gradients are unchanged. +- **`engine-softmax`, `engine-layer-norm`, `engine-sdpa`** in `jvm-cpu-publish` and `wasm-bench` (`--provider ffm` measures the fused kernels on the JVM, `panama`/`scalar` the common paths); the kernel support matrix's *Kernel packs* table gains rows for the three fused ops. +- **Native library no longer references libmvec**: GCC with `-ffast-math` vectorised the `expf`/`tanhf`/`erf` loops of `f32_ops.c` (and the new softmax) into glibc libmvec calls the shared library is not linked against — the symbol resolves lazily, so the first JVM call into such a kernel terminated the process (exit 127, no crash log). The transcendentals in element loops now go through opaque scalar wrappers in `skainet_simd.h`; `libskainet_kernels.so` has no `_ZGV*` reference left and stays loadable on glibc builds without libmvec. +- **Browser IO parity** (`skainet-io-core`, Kotlin/Wasm): `JsBlobRandomAccessSource` addresses a `Blob` with `Long` positions (a shard over 2 GB opens and reads), moves bytes into Kotlin in 4 MB bulk stages through linear memory instead of one interop call per byte, and serves blocking reads beyond the preloaded window from the Blob itself inside a Web Worker through `FileReaderSync` (`supportsSyncReads`, `openSync`, `hasFileReaderSync()`); on the main thread such a read throws with the explanation instead of blocking. The Kotlin/JS variant gets the same `Long` addressing and a copy-free `ByteArray` view. +- **Sharded SafeTensors without a file system**: `ShardedSafeTensorsParametersLoader(indexJson, shardSource)` and `StreamingShardedSafeTensorsReader.openFromIndexContent` / `openFromParsedIndex` take the index as text and a (suspending) provider that opens each shard by file name — a `Blob` per shard in a browser, `ByteArrayRandomAccessSource` over fetched bytes — with `withPolicy` for the new form. Every shard is now opened exactly once on both forms, and a failed open closes what was already opened. +- **`PlannerProfile.BROWSER`** (and `Budget.RESERVE_BROWSER`, `PlannerProfile.BROWSER_AVAILABLE_BYTES`): the tab profile — 256 MB reserved, dense weights quantized to `Q8_0` at load, KV auto-quantized past 80 %, nothing mapped. `PlannerProfile.forDevice()` returns it on Kotlin/JS and Kotlin/Wasm; `skainet-plan --profile browser`. +- **`MemoryProbe` in a browser**: on Kotlin/Wasm `rssBytes()` is the JavaScript heap in use (Chromium's `performance.memory`), the module's linear memory and the bytes the installed `OffHeapAllocator` holds (`OffHeapAllocator.heldBytes()`, which the SIMD128 sidecar reports as its resident weights and activations), with `linearMemoryBytes()` / `jsHeapUsedBytes()` / `jsHeapLimitBytes()` / `offHeapBytes()` beside it; Kotlin/JS reports the JavaScript heap. Fault counters stay `null`, as before. +- **IO docs**: `how-to/io-readers.adoc` gains the browser section (Blob sources, worker-side loads, sharded checkpoints from Blobs) and a per-target table naming every remaining difference with its reason (no file system, no `mmap`, no Kotlin threads, 2 GB arrays). +- **Convolutions on the wasm tier**: `conv1d` / `conv2d` / `conv3d` on dense FP32 run as im2col + GEMM in the SIMD128 sidecar (`WasmCpuOps`), with stride, padding, dilation and groups (depthwise included), the weight resident after its first call and the output resident — the JVM's im2col + Vector API path had no counterpart in a browser, where a convolution was the common per-element loop. The new shared C kernel `skainet_f32_conv` (`f32_conv.c`, in every native build) gathers the receptive fields patch-major, threaded over patches through the row pool, and multiplies them against the weight rows with the existing `skainet_fp32_matmul_wt` SIMD128 / NEON dot kernel. `engine-conv2d` joins `jvm-cpu-publish` and `wasm-bench`; the kernel support matrix's *Kernel packs* table gains the row. - **`wasm-bench` runs the `engine-*` scenarios** of the JVM publish harness (`engine-fp32-gemm`, `engine-kernel-matmul`, `engine-bf16-matmul`, `engine-q8-matmul`, `engine-q4-gemm`, `engine-ternary-f32-gemv`, `engine-elementwise-add`, `engine-reductions-sum/mean`) with identical shapes and metrics on every tier it measures, so the JVM and wasm numbers sit in one table. ## [0.61.0] - 2026-10-16 diff --git a/build-logic/convention/src/main/kotlin/GenerateKernelMatrixTask.kt b/build-logic/convention/src/main/kotlin/GenerateKernelMatrixTask.kt index 6964d6e02..4e8a836d3 100644 --- a/build-logic/convention/src/main/kotlin/GenerateKernelMatrixTask.kt +++ b/build-logic/convention/src/main/kotlin/GenerateKernelMatrixTask.kt @@ -128,6 +128,39 @@ abstract class GenerateKernelMatrixTask : DefaultTask() { appendLine("|===") appendLine("") } + if (module.packs.isNotEmpty()) { + appendLine("== Kernel packs (view dispatch)") + appendLine("") + appendLine( + "Kernels that are not reached through the `matmul` accessors above: the ternary formats " + + "are served by `ViewKernelPack`s keyed in `KernelDispatch` (`BitNet-b1.58` with exact " + + "FP32 activations, the fused multi-plane `BitNet-planes` lm_head, `TQ2_0` with int8 " + + "activations), and the dense `x · Wᵀ` of a `Linear` layer by the `matmulFp32Wt` " + + "accessor, which multiplies a row-major `[out, in]` weight in place instead of " + + "transposing and caching a copy. The fused row kernels — softmax / log-softmax, layer " + + "norm / RMS norm and scaled dot-product attention with its structural masks and " + + "grouped-query heads (`softmaxFp32`, `normFp32`, `attentionFp32`) — run those ops in one " + + "pass where the ops layer otherwise runs its scalar loops or the normalisation's " + + "decomposition. The convolutions run as im2col + GEMM: on the JVM inside the ops layer " + + "itself (`JvmVectorKernels`, declared as the `panama-vector` tier), on wasm as the sidecar's " + + "`skainet_f32_conv`. The scalar reference kernels serve every cell that is empty here (`—`), " + + "at reference speed; the common row-dot serves `x · Wᵀ` there.", + ) + appendLine("") + appendLine("[cols=\"$colSpec\", options=\"header\"]") + appendLine("|===") + append("| Kernel ") + module.platforms.forEach { append("| $it ") } + appendLine("") + appendLine("") + module.packs.forEach { fmt -> + append("| `${fmt.name}` ") + module.platforms.forEach { p -> append("| ${fmt.byPlatform[p] ?: "—"} ") } + appendLine("") + } + appendLine("|===") + appendLine("") + } appendLine( "See also the eager backends & kernels mindmap " + "(xref:explanation/eager-execution.adoc[]) for the narrative overview and gaps.", diff --git a/build-logic/convention/src/main/kotlin/models/KernelSupportModels.kt b/build-logic/convention/src/main/kotlin/models/KernelSupportModels.kt index 90da53887..fe8a43ebd 100644 --- a/build-logic/convention/src/main/kotlin/models/KernelSupportModels.kt +++ b/build-logic/convention/src/main/kotlin/models/KernelSupportModels.kt @@ -23,6 +23,12 @@ data class KernelSupportModule( */ val mapped: List = emptyList(), val resident: List = emptyList(), + /** + * Kernel packs (view dispatch): the ternary formats and the direct dense `x · Wᵀ`, served by + * `ViewKernelPack`s / the `matmulFp32Wt` accessor rather than the `matmul` accessors. Absent in + * older JSON. + */ + val packs: List = emptyList(), ) @Serializable diff --git a/docs/modules/ROOT/pages/contributing/benchmarks.adoc b/docs/modules/ROOT/pages/contributing/benchmarks.adoc index 69a2ed12d..9a1dc806d 100644 --- a/docs/modules/ROOT/pages/contributing/benchmarks.adoc +++ b/docs/modules/ROOT/pages/contributing/benchmarks.adoc @@ -33,6 +33,7 @@ microbench tests under `skainet-backends/skainet-backend-native-cpu`: |=== | Scenario | What it exercises | Unit | Direction | engine-fp32-gemm | `ctx.ops.matmul` end-to-end FP32 square matmul | GFLOPS | higher is better +| engine-fp32-linear | `ctx.ops.matmulWeightTransposed` — a `Linear` layer's `x · Wᵀ` with the weight stored `[out, in]` (panama: cached transpose + GEMM; ffm: native direct kernel, no transposed copy) | GFLOPS | higher is better | engine-q4-gemm | `PanamaVectorQ4KMatmulKernel` F32 × Q4_K matmul at LLM shapes | GOP/s | higher is better | engine-kernel-matmul | Direct `Fp32MatmulKernel` (scalar vs panama-vector) | GFLOPS | higher is better | engine-bf16-matmul | `Bf16MatmulKernel` F32 × BF16-packed matmul (scalar vs panama-vector) | GFLOPS | higher is better @@ -40,6 +41,10 @@ microbench tests under `skainet-backends/skainet-backend-native-cpu`: | engine-elementwise-add | `ctx.ops.add` on 1M FP32 elements | M elements/s | higher is better | engine-reductions-sum | `ctx.ops.sum` on 1M FP32 elements | M elements/s | higher is better | engine-reductions-mean | `ctx.ops.mean` on 1M FP32 elements | M elements/s | higher is better +| engine-softmax | `ctx.ops.softmax` over the last dimension of `[1024, 1024]` FP32 (ffm: fused native rows; panama/scalar: the common loop) | M elements/s | higher is better +| engine-layer-norm | a `LayerNormalization` layer with affine over `[1024, 768]` FP32 (ffm: one fused native pass; panama/scalar: the decomposed elementwise chain) | M elements/s | higher is better +| engine-sdpa | `ctx.ops.scaledDotProductAttention`, causal, 12 heads × 150 × 150, head dim 64 (ffm: fused native kernel; panama/scalar: the common per-head kernel under the context's schedule) | GFLOPS (`4·h·s²·d`) | higher is better +| engine-conv2d | `ctx.ops.conv2d`, a 3 × 3 same-padding layer, 64 → 64 channels over a 56 × 56 map, with bias (JVM: the im2col + Vector API path whichever provider is registered; wasm: im2col + GEMM in the sidecar) | GFLOPS (`2·c_out·h·w·c_in·k²`) | higher is better |=== Each scenario is wired up as a Phoronix test profile under diff --git a/docs/modules/ROOT/pages/explanation/eager-execution.adoc b/docs/modules/ROOT/pages/explanation/eager-execution.adoc index f251bfbe6..d206f92f7 100644 --- a/docs/modules/ROOT/pages/explanation/eager-execution.adoc +++ b/docs/modules/ROOT/pages/explanation/eager-execution.adoc @@ -29,7 +29,8 @@ mindmap Native FFM JVM only — C kernels via CMake Q4_K MemSeg zero-copy - Q6_K missing + direct x·Wᵀ, no transposed copy + ternary LUT gemv and planes lm_head Native JNI Android only — same C kernels via JNI two .so tiers, cpuinfo-gated armv8-a vs armv8.2+dotprod @@ -47,7 +48,7 @@ mindmap Android — scalar, Native-JNI Native linux — scalar, Native-cinterop Native apple — scalar, Accelerate, Native-cinterop - JS and WASM — scalar; packed weights relaid once into kernel feed order (SKEEP-007) + JS and WASM — scalar; wasmJs plus the SIMD128 sidecar pack: every matmul format, ternary, direct x·Wᵀ, resident activations, fused softmax / norms / attention (SKEEP-007) ---- == Kernel × provider (matmul, FP32 activations) @@ -56,6 +57,7 @@ mindmap |=== | Weight format | Scalar (all targets) | Panama (JVM) | Native FFM (JVM) | Native JNI (Android) | Native cinterop (K/N) | Wasm SIMD128 (wasmJs) | FP32 | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ +| FP32 `x · Wᵀ` direct (weight `[out, in]`, no transposed copy) | row-dot | ❌ (cached transpose) | ✅ | ❌ | ❌ | ✅ | BF16 | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ | FP16 | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ | Q8_0 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ @@ -65,9 +67,9 @@ mindmap | Q5_K | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | Q5_1 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | Q5_0 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ -| TQ2_0 / BitNet b1.58 (int8 activations) | ✅ | — | — | ✅ NEON | — | — -| BitNet b1.58 (exact FP32 activations, vendored NeoGPU kernel) | ✅ | ✅ FFM | — | ✅ JNI + fused lm_head | ✅ K/N cinterop | ❌ (reference) -| BITNET_PLANES (multi-plane ternary packing) | ✅ | via kernel pack | — | via kernel pack | via kernel pack | ❌ (reference) +| TQ2_0 / BitNet b1.58 (int8 activations) | ✅ | — | — | ✅ NEON | — | ✅ SIMD128 +| BitNet b1.58 (exact FP32 activations, LUT gemv) | ✅ | — | ✅ FFM (vendored NeoGPU) | ✅ JNI | ✅ K/N cinterop | ✅ SIMD128 LUT +| BITNET_PLANES (multi-plane ternary packing, fused lm_head) | ✅ | — | ✅ FFM (vendored NeoGPU) | ✅ JNI | ✅ K/N cinterop | ✅ SIMD128 LUT | Q2_K / Q3_K / Q8_K / IQ4 | ❌ (dequant to FP32 only) | ❌ | ❌ | ❌ | ❌ | ❌ |=== @@ -78,7 +80,24 @@ columns are mutually exclusive per platform rather than stacked. The Wasm SIMD12 sidecar pack (`skainet-backend-wasm-simd`), present only after `WasmSimd.install()`; every matmul kernel in it has an explicit SIMD128 body (the K-quant int8 dots on `i32x4.dot_i16x8_s`, a 4×8 register-blocked dense GEMM, eight-wide BF16/FP16 widening), and the dense kernels thread over the -sidecar's pthread pool like the packed ones when the page is cross-origin isolated. +sidecar's pthread pool like the packed ones when the page is cross-origin isolated. The ternary +kernels in the sidecar are the portable mirror of the vendored NeoGPU kernel — the same 256-entry +decode table as a compile-time constant, read four codes at a time in SIMD128 — registered under the +same dispatch keys as the FFM and JNI ternary packs (`WasmTernaryKernelPack`). + +The dense `x · Wᵀ` row is the product a `Linear` layer computes. The JVM and the wasm tier multiply +the `[out, in]` weight in place through `skainet_fp32_matmul_wt` (`KernelProvider.matmulFp32Wt`; +the JVM makes a heap weight resident off-heap once, the sidecar keeps it in its linear memory) and +never materialise the transposed copy the Panama path caches per weight. + +The fused row kernels — softmax / log-softmax over the last dimension, layer norm and RMS norm +(what `LayerNormalization` calls through the optional `FusedNormOps` hook), and scaled dot-product +attention with its structural masks and grouped-query heads — are one C pass each (`f32_fused.c`), +served by the JVM FFM provider (`softmaxFp32`, `normFp32`, `attentionFp32`) and the wasm sidecar +alike. Where neither is present the ops layer runs its common implementation: attention scalar per +head and parallel over heads through the schedule, the normalisation as its decomposed elementwise +chain. The decomposition is also what a recording or differentiating op set always sees, so the +hook changes no recorded graph and no gradient. Packed weights reach a kernel through the registry by declaring their xref:explanation/packed-weight-layout.adoc[block order]; a canonical weight is relayouted once, @@ -93,7 +112,7 @@ not per call. | *Android* | Scalar + Native-JNI | Panama (`jdk.incubator.vector`) and FFM (`java.lang.foreign`) are JDK-only — ART has neither, so Android's native tier is JNI, not a degraded JVM. Includes the NEON `bitnet_gemv` pack. | *Kotlin/Native — linux x64/arm64* | Scalar + Native-cinterop | Static archive embedded in the klib; manual `installNativeKernels()`, no ServiceLoader on K/N. | *Kotlin/Native — macOS/iOS* | Scalar + Apple Accelerate + Native-cinterop | Accelerate covers dense FP32 and reductions; cinterop covers packed quant with runtime `FEAT_DotProd` dispatch, so one archive serves A12 through M-series. -| *JS / WASM (Js, Wasi)* | Scalar; wasmJs + `skainet-backend-wasm-simd` | No SIMD in Kotlin code: packed weights are relaid once per weight into kernel feed order and served by the scalar packed kernels; dense `[out, in]` weights are multiplied in place without a transpose copy (SKEEP-007 Tier 0). On wasmJs, `WasmSimd.install()` adds the SIMD128 sidecar pack — the engine's C kernels compiled by Emscripten, with pthreads on cross-origin-isolated pages (SKEEP-007 Tier 1). +| *JS / WASM (Js, Wasi)* | Scalar; wasmJs + `skainet-backend-wasm-simd` | No SIMD in Kotlin code: packed weights are relaid once per weight into kernel feed order and served by the scalar packed kernels; dense `[out, in]` weights are multiplied in place without a transpose copy (SKEEP-007 Tier 0). On wasmJs, `WasmSimd.install()` adds the SIMD128 sidecar pack — the engine's C kernels compiled by Emscripten, with pthreads on cross-origin-isolated pages (SKEEP-007 Tier 1) — and binds `HOST_OFFHEAP` to the sidecar's linear memory, so matmul and convolution outputs, elementwise arithmetic, activations, reductions, softmax, the normalisations and attention stay resident there between kernels and cross back to the Kotlin heap only when read. |=== Packed-quant matmul works on every target: those formats have a `commonMain` scalar kernel and @@ -101,11 +120,11 @@ Packed-quant matmul works on every target: those formats have a `commonMain` sca == Gaps -* *Wasm ternary / BitNet* — the ternary gemv and planes lm-head kernels are not in the sidecar -build yet, so BitNet models in a browser run the decoding reference kernel. -* *Wasm beyond matmul* — elementwise, activations, reductions, normalisation and attention are the -common scalar Kotlin paths on wasmJs; Kotlin/Wasm has no SIMD types and no shared-memory threads, so -these follow the matmuls into the sidecar (SKEEP-007 rollout step 3). +* *Wasm beyond the kernel set* — what stays common Kotlin on wasmJs is memory movement (`reshape`, +`permute`, `concat`, `gather`, `unfold`, pooling) and the integer and narrow-float element paths; +every FP32 op a transformer or CNN forward pass spends its time in — the matmuls, convolutions +(im2col + GEMM), elementwise arithmetic, activations, reductions, softmax, the normalisations and +attention — runs in the sidecar with the activations resident in its linear memory. * *Native JNI dense FP32/BF16* — the Android JNI provider has no GEMM shim, so dense ops fall through to scalar there whichever `.so` tier loaded (SKaiNET#920). * *Dense FP32/BF16 SIMD on Kotlin/Native linux* — still the scalar floor; Apple has Accelerate diff --git a/docs/modules/ROOT/pages/explanation/kernel-selection.adoc b/docs/modules/ROOT/pages/explanation/kernel-selection.adoc index b522b0177..a7a582539 100644 --- a/docs/modules/ROOT/pages/explanation/kernel-selection.adoc +++ b/docs/modules/ROOT/pages/explanation/kernel-selection.adoc @@ -91,7 +91,7 @@ platform support is absent (a missing native library, no vector unit). | Kotlin/Native, wasm, JS | manual — e.g. `installNativeKernels()`; on wasmJs `WasmSimd.install()` (SKEEP-007) -| manual — call the pack's `install()` yourself; `WasmSimd.install()` registers the browser pack and its provider +| manual — call the pack's `install()` yourself; `WasmSimd.install()` registers the browser pack, its ternary pack and its provider |=== Without a row-major pack, a canonical packed weight is relaid **once per weight** into kernel feed diff --git a/docs/modules/ROOT/pages/explanation/memory-model.adoc b/docs/modules/ROOT/pages/explanation/memory-model.adoc index c520a878e..8aa0e8d73 100644 --- a/docs/modules/ROOT/pages/explanation/memory-model.adoc +++ b/docs/modules/ROOT/pages/explanation/memory-model.adoc @@ -188,7 +188,10 @@ breakdown, adapter cost, and *effective memory bandwidth* — bytes a decode ste divided by how long it took. Rates are `null` rather than infinite when a span is too short for the platform clock. * `MemoryProbe` reads RSS and page faults from the OS, so "the resident set is flat" and "nothing -paged in during decode" are measurements rather than inferences. +paged in during decode" are measurements rather than inferences. A browser has no process +counters: there the probe sums the JavaScript heap in use (Chromium reports it), the module's +linear memory and the bytes the kernel sidecar holds for resident weights and activations, and the +fault counters stay `null`. `SKAINET_MEMORY_DEBUG=1` adds allocation-site tags, use-after-close reporting with the closing stack, and a leak check at every forward-scope reset. diff --git a/docs/modules/ROOT/pages/how-to/io-readers.adoc b/docs/modules/ROOT/pages/how-to/io-readers.adoc index d0310b58c..ddb868718 100644 --- a/docs/modules/ROOT/pages/how-to/io-readers.adoc +++ b/docs/modules/ROOT/pages/how-to/io-readers.adoc @@ -9,7 +9,9 @@ SKaiNET provides two main I/O modules for reading AI model formats: * *skainet-io-gguf*: For reading GGUF (GPT-Generated Unified Format) files * *skainet-io-onnx*: For reading ONNX (Open Neural Network Exchange) files -Both modules are built on Kotlin Multiplatform and support JVM, Android, iOS, JS, WASM, and Native platforms. +Both modules are built on Kotlin Multiplatform and support JVM, Android, iOS, JS, WASM, and Native platforms; +`skainet-io-safetensors` reads single-file and sharded SafeTensors checkpoints the same way. What a +target cannot offer — a file system in a browser, memory mapping — is listed in <<_what_each_target_can_do>>. === Dependencies @@ -414,6 +416,79 @@ fun readFromNativePath(pathString: String): Source { } ---- +==== Browser (Kotlin/JS, Kotlin/Wasm) + +There is no file system, so a model arrives as a `Blob` — a `File` from an ``, a fetched body, +a Cache API entry — and `JsBlobRandomAccessSource` (`skainet-io-core`) is the `RandomAccessSource` +over it. Positions and sizes are `Long` (a shard over 2 GB is fine), bytes cross into Kotlin in bulk +through linear memory, and the source serves two kinds of read: + +* the suspending `read()` works on any thread for any range; +* the blocking `readAt()` — what the streaming readers call — is served from the preloaded window + (the first `preloadSize` bytes, 50 MB by default, enough for any header) and, **inside a Web + Worker**, from the Blob itself through `FileReaderSync`, the browser's only synchronous file read. + On the main thread a blocking read beyond the window throws with that explanation. + +So a whole model loads through the same blocking reader code the JVM uses when the load runs in a +worker (`openSync` opens the Blob without preloading anything there); on the main thread, parse the +header from the preload and fetch tensor payloads with the suspending read, or move the load to a +worker. + +[source,kotlin] +---- +// In a Web Worker: the same StreamingSafeTensorsReader the JVM runs, over a Blob +val source = JsBlobRandomAccessSource.openSync(file) // FileReaderSync is defined here +StreamingSafeTensorsReader.open(source).use { reader -> + val bytes = reader.loadTensorData("model.embed_tokens.weight") +} + +// On the main thread: header from the preload, payloads through the suspending read +val source = JsBlobRandomAccessSource.open(file) // preloads 50 MB +val header = source.readAt(0, 8) // blocking, inside the window +val payload = source.read(dataOffset, tensorBytes) // suspending, anywhere in the Blob +---- + +A sharded checkpoint (`model.safetensors.index.json` + `model-NNNNN-of-NNNNN.safetensors`) has no +path to resolve shards beside, so `ShardedSafeTensorsParametersLoader` takes the index as text and a +provider that opens each shard by file name — a `Blob` per shard here, `ByteArrayRandomAccessSource` +over fetched bytes elsewhere. The provider may suspend, is asked once per shard, and answers `null` +for a shard that is not there: + +[source,kotlin] +---- +val files: Map = pickedFiles.associateBy { it.name } // from an +val indexJson = files.getValue("model.safetensors.index.json").text().await() + +val loader = ShardedSafeTensorsParametersLoader(indexJson, { shardName -> + files[shardName]?.let { JsBlobRandomAccessSource.open(it) } +}) +loader.load(ctx, FP32::class) { name, tensor -> model.bind(name, tensor) } +---- + +The same two forms exist on `StreamingShardedSafeTensorsReader` (`openFromIndexContent`, +`openFromParsedIndex`). Mapped loading (`loadTensorStorageMapped`) needs file-backed shards and +throws for Blobs. + +=== What each target can do + +The IO layer is one code base; where a platform lacks the facility an entry point needs, the entry +point says so rather than failing later, and a counterpart exists where one can. The remaining +differences and their reasons: + +[cols="2,1,1,1,3",options="header"] +|=== +| Capability | JVM / Android | Kotlin/Native | JS / wasmJs | Notes +| `openRandomAccessSource(path)`, path-based loaders (`StreamingGgufParametersLoader(path)`, `SafeTensorsParametersLoader { JvmRandomAccessSource.open(path) }`, `ShardedSafeTensorsParametersLoader(indexPath)`) | yes | yes | returns `null` / unavailable | No file system in a browser. Use `JsBlobRandomAccessSource` (a `Blob`) or `ByteArrayRandomAccessSource` (fetched bytes) with the source-based constructors; sharded checkpoints through the `(indexJson, shardSource)` form. +| Blocking reads from a `RandomAccessSource` | yes | yes | preloaded window; whole Blob inside a Web Worker | `FileReaderSync` exists on worker threads only. The suspending `read()` works everywhere. +| Sources and reads over 2 GB | yes | yes | yes | `Long` positions throughout; one read is bounded by the 2 GB `ByteArray`, the planner splits above that (`ByteArrayRandomAccessSource` takes chunks). +| Memory-mapped weights (`WeightResidency.MAPPED`, `PlatformStorage.mapFile`, `loadTensorStorageMapped`) | yes | yes | no | No `mmap` in a browser. The counterpart is a weight resident in the kernel sidecar's linear memory (`WasmSimd.install()`), which the kernel support matrix lists as *Resident serving*. +| Off-heap storage (`MemoryDomain.HOST_OFFHEAP`) | FFM / direct buffers | native heap | the sidecar's linear memory once `WasmSimd.install()` ran, else the heap | `StorageCapabilities.current()` reports which; the planner records the fallback. +| Writing files by path (GGUF writer, JSON export, IRPA `FileBacked`) | yes | yes | no | No file system; the `Sink`- and bytes-returning variants of the writers exist on every target — hand the bytes to a `Blob` and a download link. +| `MemoryProbe` | RSS and page faults from `/proc` (Linux) | RSS | JS heap in use (Chromium), the module's linear memory and the sidecar's held bytes; no page faults | A tab has no process counters; the probe sums what the engine reports. +| `PlannerProfile.forDevice()` | `MOBILE_2GB` / `DESKTOP` by RAM | `NATIVE` by hand | `BROWSER` | 256 MB reserved, dense weights quantized to `Q8_0` at load, nothing mapped; plans against `BROWSER_AVAILABLE_BYTES` (2 GB) unless told otherwise. +| Parallel loading and compute | all cores (`CoroutineSchedule.hardware()`) | all cores | `Schedule.Sequential` in Kotlin; the sidecar's pthreads inside kernels on cross-origin-isolated pages | Kotlin/Wasm has no shared-memory threads; parallelism lives in the C kernels. +|=== + === Performance Tips ==== Memory Management diff --git a/docs/modules/ROOT/pages/how-to/plan-model-memory.adoc b/docs/modules/ROOT/pages/how-to/plan-model-memory.adoc index 82b5d2c1a..ef44d05f6 100644 --- a/docs/modules/ROOT/pages/how-to/plan-model-memory.adoc +++ b/docs/modules/ROOT/pages/how-to/plan-model-memory.adoc @@ -30,7 +30,7 @@ Useful flags: | `--ctx N` | Plan for this context length (default: the model's trained length, or 2048) | `--budget 1.3G` | The memory available; without it, the JVM's own maximum is used | `--kv bf16\|turboquant` | KV cache format -| `--profile mobile\|desktop\|native` | Plan under a device profile — see below +| `--profile mobile\|desktop\|native\|browser` | Plan under a device profile — see below | `--list 'model.layers[3].*'` | List matching tensors with their `TensorId`, format and size | `--no-budget` | Print the plan without a fit check |=== @@ -56,7 +56,8 @@ profiled.requireFits() // throws, naming the pool that ran out | `DESKTOP` | The same reserve, no automatic KV quantization, heap staging | `NATIVE` | The smaller 300 MB Kotlin/Native reserve | `EDGE` | An embedded device with limited memory and compute: **zero reserve** — the number you pass is the *usable* RAM, already net of what the OS holds — weights mapped, KV auto-quantized past 80 % -| `PlannerProfile.forDevice(device)` | Picks `MOBILE_2GB` at or below 3 GB of RAM +| `BROWSER` | A tab (Kotlin/Wasm or Kotlin/JS): 256 MB reserved for the page, the JavaScript heap and the Kotlin runtime; dense checkpoints quantized to `Q8_0` at load (`quantizeDenseTo`) so a Hugging Face FP32 checkpoint is held in a quarter of its bytes and served by the SIMD128 kernels; KV auto-quantized past 80 %; nothing mapped — there is no file. Off-heap allocations go to the kernel sidecar's linear memory once `WasmSimd.install()` has bound it. Plan against `PlannerProfile.BROWSER_AVAILABLE_BYTES` (2 GB) unless the host tells you more. +| `PlannerProfile.forDevice(device)` | `BROWSER` on a web host; otherwise `MOBILE_2GB` at or below 3 GB of RAM, `DESKTOP` above |=== A `ProfiledPlan` records what the profile decided — the KV switch appears as a note, not a silent diff --git a/docs/modules/ROOT/pages/reference/kernel-support-matrix.adoc b/docs/modules/ROOT/pages/reference/kernel-support-matrix.adoc index a3317ed37..82c380897 100644 --- a/docs/modules/ROOT/pages/reference/kernel-support-matrix.adoc +++ b/docs/modules/ROOT/pages/reference/kernel-support-matrix.adoc @@ -58,4 +58,22 @@ The browser has no files to map, so the wasm tier keeps weights *resident* in th | `Q5_0` | — | — | — | — | wasm-resident |=== +== Kernel packs (view dispatch) + +Kernels that are not reached through the `matmul` accessors above: the ternary formats are served by `ViewKernelPack`s keyed in `KernelDispatch` (`BitNet-b1.58` with exact FP32 activations, the fused multi-plane `BitNet-planes` lm_head, `TQ2_0` with int8 activations), and the dense `x · Wᵀ` of a `Linear` layer by the `matmulFp32Wt` accessor, which multiplies a row-major `[out, in]` weight in place instead of transposing and caching a copy. The fused row kernels — softmax / log-softmax, layer norm / RMS norm and scaled dot-product attention with its structural masks and grouped-query heads (`softmaxFp32`, `normFp32`, `attentionFp32`) — run those ops in one pass where the ops layer otherwise runs its scalar loops or the normalisation's decomposition. The convolutions run as im2col + GEMM: on the JVM inside the ops layer itself (`JvmVectorKernels`, declared as the `panama-vector` tier), on wasm as the sidecar's `skainet_f32_conv`. The scalar reference kernels serve every cell that is empty here (`—`), at reference speed; the common row-dot serves `x · Wᵀ` there. + +[cols="1,1,1,1,1,1", options="header"] +|=== +| Kernel | JVM | Android | Native·linux | Native·apple | JS/WASM + +| `BitNet-b1.58 (FP32 activations)` | ffm-ternary | native-jni | native-cinterop | native-cinterop | wasm-simd128 +| `BitNet-planes` | ffm-ternary | native-jni | native-cinterop | native-cinterop | wasm-simd128 +| `TQ2_0 (int8 activations)` | — | native-jni | — | — | wasm-simd128 +| `Float32 x·Wᵀ direct` | native-ffm | — | — | — | wasm-simd128 +| `Float32 softmax (fused rows)` | native-ffm | — | — | — | wasm-simd128 +| `Float32 layer_norm / rms_norm (fused)` | native-ffm | — | — | — | wasm-simd128 +| `Float32 SDPA (fused, banded, GQA)` | native-ffm | — | — | — | wasm-simd128 +| `Float32 conv1d / conv2d / conv3d (im2col + GEMM)` | panama-vector | — | — | — | wasm-simd128 +|=== + See also the eager backends & kernels mindmap (xref:explanation/eager-execution.adoc[]) for the narrative overview and gaps. diff --git a/docs/modules/skeep/pages/007-browser-lane.adoc b/docs/modules/skeep/pages/007-browser-lane.adoc index 087cc1de5..eda85f498 100644 --- a/docs/modules/skeep/pages/007-browser-lane.adoc +++ b/docs/modules/skeep/pages/007-browser-lane.adoc @@ -1,7 +1,7 @@ = SKEEP-007: The browser lane — packed SIMD kernels and quantize-at-load :description: SKaiNET proposal for making inference in a browser usable and quick: stop the scalar wasm lane from falling to the decoding reference kernel, serve dense weights without transpose copies, keep token tables narrow, add a SIMD128 kernel pack with pthreads as a sidecar wasm module, and let the SafeTensors loader quantize at load so Hugging Face checkpoints are served packed on every target. -Status: Implemented (Tiers 0, 1 and 2 in engine 0.61.0); the resident-activation step, the SKaiNET-transformers consumer and the WebGPU proposal remain open + +Status: Implemented (Tiers 0, 1 and 2 in engine 0.61.0; resident activations with sidecar elementwise, activation and reduction kernels, then fused softmax / normalisation / attention kernels on the wasm and JVM tiers, then the browser IO gaps — `Long` Blob sources with bulk copies and worker-side blocking reads, sharded SafeTensors from Blobs, `PlannerProfile.BROWSER`, the browser `MemoryProbe` — and convolutions as im2col + GEMM in the sidecar, in the parity series that followed); the SKaiNET-transformers consumer and the WebGPU proposal remain open + Implementation: Tier 0 https://github.com/SKaiNET-developers/SKaiNET/pull/1352[#1352], Tier 2 quantize-at-load https://github.com/SKaiNET-developers/SKaiNET/pull/1353[#1353], Tier 1 `skainet-backend-wasm-simd` https://github.com/SKaiNET-developers/SKaiNET/pull/1354[#1354] — engine 0.61.0; the consumer work in SKaiNET-transformers follows, see <<_rollout_plan>>. + Audience: SKaiNET maintainers and contributors; SKaiNET-transformers maintainers (Laya and BERT in the browser); SKaiNET-examples maintainers (every web sample runs through this lane) + Created: 2026-10-16 + @@ -270,8 +270,10 @@ copies and the thread hand-off, which is what resident activations address next. * Whether the dense FP32 fast path in `matmul` (the `[k, n]` loop) should also route through `KernelDispatch` first, so a registered dense kernel wins without an ops override. Deferred to PR 2, where the first such kernel appears. -* Whether `TensorOps` should grow a fused `layerNorm`; the cost on the encoder is allocation rather - than arithmetic, and `ForwardScope` may already answer it. +* Whether `TensorOps` should grow a fused `layerNorm` — answered without touching `TensorOps`: the + optional `FusedNormOps` interface beside it, which `LayerNormalization` consults for FP32 input + and only the sidecar and the JVM FFM op sets implement, so recording and differentiating op sets + keep the decomposition. == References diff --git a/skainet-apps/skainet-plan/src/main/kotlin/sk/ainet/apps/plan/SkainetPlan.kt b/skainet-apps/skainet-plan/src/main/kotlin/sk/ainet/apps/plan/SkainetPlan.kt index f6c9cdf0a..007762167 100644 --- a/skainet-apps/skainet-plan/src/main/kotlin/sk/ainet/apps/plan/SkainetPlan.kt +++ b/skainet-apps/skainet-plan/src/main/kotlin/sk/ainet/apps/plan/SkainetPlan.kt @@ -30,7 +30,8 @@ import kotlin.system.exitProcess * `--profile` (M2-F6, #1039) plans under a device profile instead of raw defaults: `mobile` is the * 2 GB phone (700 MB reserved, weights mapped, KV quantized automatically once the plan passes 80 % * of the budget), `desktop` keeps today's behaviour, `native` reserves the smaller Kotlin/Native - * amount. The profile and every decision it made are printed above the table, so a plan can be read + * amount, `browser` is a tab (256 MB reserved, dense weights quantized to Q8_0 at load, nothing + * mapped). The profile and every decision it made are printed above the table, so a plan can be read * back later without guessing which rules produced it. */ public fun main(args: Array) { @@ -104,6 +105,7 @@ internal fun profileFor(name: String): PlannerProfile? = when (name) { "mobile" -> PlannerProfile.MOBILE_2GB "desktop" -> PlannerProfile.DESKTOP "native" -> PlannerProfile.NATIVE + "browser" -> PlannerProfile.BROWSER else -> null } diff --git a/skainet-apps/skainet-plan/src/test/kotlin/sk/ainet/apps/plan/SkainetPlanTest.kt b/skainet-apps/skainet-plan/src/test/kotlin/sk/ainet/apps/plan/SkainetPlanTest.kt index 34771690e..52f8d34c9 100644 --- a/skainet-apps/skainet-plan/src/test/kotlin/sk/ainet/apps/plan/SkainetPlanTest.kt +++ b/skainet-apps/skainet-plan/src/test/kotlin/sk/ainet/apps/plan/SkainetPlanTest.kt @@ -55,6 +55,7 @@ class SkainetPlanTest { assertEquals(PlannerProfile.MOBILE_2GB, profileFor("mobile")) assertEquals(PlannerProfile.DESKTOP, profileFor("desktop")) assertEquals(PlannerProfile.NATIVE, profileFor("native")) + assertEquals(PlannerProfile.BROWSER, profileFor("browser")) assertEquals(null, profileFor("none"), "no profile means the plan's own defaults, as before") } diff --git a/skainet-backends/benchmarks/jvm-cpu-publish/src/main/kotlin/sk/ainet/bench/publish/scenarios/Fp32LinearScenario.kt b/skainet-backends/benchmarks/jvm-cpu-publish/src/main/kotlin/sk/ainet/bench/publish/scenarios/Fp32LinearScenario.kt new file mode 100644 index 000000000..2600d365e --- /dev/null +++ b/skainet-backends/benchmarks/jvm-cpu-publish/src/main/kotlin/sk/ainet/bench/publish/scenarios/Fp32LinearScenario.kt @@ -0,0 +1,79 @@ +package sk.ainet.bench.publish.scenarios + +import sk.ainet.backend.api.kernel.KernelRegistry +import sk.ainet.bench.publish.runner.Scenario +import sk.ainet.context.DirectCpuExecutionContext +import sk.ainet.exec.kernel.NativeKernelProvider +import sk.ainet.exec.kernel.PanamaVectorKernelProvider +import sk.ainet.exec.kernel.ScalarKernelProvider +import sk.ainet.lang.tensor.Shape +import sk.ainet.lang.tensor.Tensor +import sk.ainet.lang.types.FP32 + +/** + * A `Linear` layer's product, `x · Wᵀ` with the weight stored `[out, in]`, through the public + * `ctx.ops.matmulWeightTransposed` path — what every projection of a transformer pays, as opposed to + * the `[k, n]` GEMM of `engine-fp32-gemm`. Shape: a 16-token prefill against a 768 → 3072 FFN + * projection. Primary metric: GFLOP/s over `2·m·n·k`. + * + * The provider selects the path: `panama` (the default) transposes the weight once into `[k, n]` + * and runs the Vector API GEMM; `ffm` runs the native direct kernel (`skainet_fp32_matmul_wt`) over a + * weight made resident off-heap, no transposed copy; `scalar` is the common row-dot. Only the chosen + * provider is registered, so the ops layer cannot fall through to another. + */ +internal class Fp32LinearScenario( + smoke: Boolean, + private val providerName: String, +) : Scenario { + override val id: String = "engine-fp32-linear" + override val suite: String = "skainet-engine" + override val primaryMetric: String = "gflops" + override val unit: String = "gflops" + override val higherIsBetter: Boolean = true + override val kernelProvider: String = providerName + + private val m: Int = 16 + private val k: Int = 768 + private val n: Int = if (smoke) 768 else 3072 + override val parameters: Map = mapOf( + "m" to m.toString(), + "k" to k.toString(), + "n" to n.toString(), + "kernel" to providerName, + ) + + private lateinit var ctx: DirectCpuExecutionContext + private lateinit var x: Tensor + private lateinit var w: Tensor + + override fun setup() { + KernelRegistry.clearForTesting() + when (providerName.lowercase()) { + "panama", "panama-vector" -> KernelRegistry.register(PanamaVectorKernelProvider) + "ffm", "native", "native-ffm" -> { + check(NativeKernelProvider.isAvailable()) { "bundled libskainet_kernels missing — the ffm provider cannot run" } + KernelRegistry.register(NativeKernelProvider) + } + "scalar" -> KernelRegistry.register(ScalarKernelProvider) + else -> error("unknown kernel provider: $providerName (use 'panama', 'ffm' or 'scalar')") + } + ctx = DirectCpuExecutionContext() + x = ctx.fromFloatArray(Shape(m, k), FP32::class, FloatArray(m * k) { ((it % 251) - 125).toFloat() / 127f }) + w = ctx.fromFloatArray(Shape(n, k), FP32::class, FloatArray(n * k) { ((it * 13 % 257) - 128).toFloat() / 127f }) + // First call: the transpose cache or the resident copy is built here, not in a measured run. + ctx.ops.matmulWeightTransposed(x, w) + } + + override fun runOnce(): Double { + val start = System.nanoTime() + val result = ctx.ops.matmulWeightTransposed(x, w) + val elapsedNs = System.nanoTime() - start + @Suppress("UNUSED_VARIABLE") val sink = result.hashCode() + val flops = 2.0 * m.toDouble() * n.toDouble() * k.toDouble() + return (flops / (elapsedNs / 1_000_000_000.0)) / 1e9 + } + + override fun teardown() { + KernelRegistry.clearForTesting() + } +} diff --git a/skainet-backends/benchmarks/jvm-cpu-publish/src/main/kotlin/sk/ainet/bench/publish/scenarios/FusedOpsScenarios.kt b/skainet-backends/benchmarks/jvm-cpu-publish/src/main/kotlin/sk/ainet/bench/publish/scenarios/FusedOpsScenarios.kt new file mode 100644 index 000000000..77972ae01 --- /dev/null +++ b/skainet-backends/benchmarks/jvm-cpu-publish/src/main/kotlin/sk/ainet/bench/publish/scenarios/FusedOpsScenarios.kt @@ -0,0 +1,241 @@ +package sk.ainet.bench.publish.scenarios + +import sk.ainet.backend.api.kernel.KernelRegistry +import sk.ainet.bench.publish.runner.Scenario +import sk.ainet.context.DirectCpuExecutionContext +import sk.ainet.exec.kernel.NativeKernelProvider +import sk.ainet.exec.kernel.PanamaVectorKernelProvider +import sk.ainet.exec.kernel.ScalarKernelProvider +import sk.ainet.lang.nn.normalization.LayerNormalization +import sk.ainet.lang.tensor.Shape +import sk.ainet.lang.tensor.Tensor +import sk.ainet.lang.types.FP32 + +/* + * The fused row ops through the public ops layer (SKEEP-007 fused ops): softmax, a `LayerNormalization` + * layer and scaled dot-product attention. The provider selects the path, as in [Fp32LinearScenario]: + * `ffm` runs the native fused kernels (`f32_fused.c` through FFM), `panama` and `scalar` the common + * Kotlin implementation (attention parallel over heads through the context's schedule, the + * normalisation as its decomposed elementwise chain). Only the chosen provider is registered. + */ + +private fun registerOnly(providerName: String) { + KernelRegistry.clearForTesting() + when (providerName.lowercase()) { + "panama", "panama-vector" -> KernelRegistry.register(PanamaVectorKernelProvider) + "ffm", "native", "native-ffm" -> { + check(NativeKernelProvider.isAvailable()) { "bundled libskainet_kernels missing — the ffm provider cannot run" } + KernelRegistry.register(NativeKernelProvider) + } + "scalar" -> KernelRegistry.register(ScalarKernelProvider) + else -> error("unknown kernel provider: $providerName (use 'panama', 'ffm' or 'scalar')") + } +} + +private fun pseudo(n: Int, seed: Int, scale: Float = 1f): FloatArray { + var s = seed + return FloatArray(n) { + s = s * 1103515245 + 12345 + ((s ushr 16) % 2001 - 1000) / 1000f * scale + } +} + +/** `ctx.ops.softmax` over the last dimension of `[rows, 1024]` FP32. Primary metric: million elements per second. */ +internal class SoftmaxScenario( + smoke: Boolean, + private val providerName: String, +) : Scenario { + override val id: String = "engine-softmax" + override val suite: String = "skainet-engine" + override val primaryMetric: String = "melems_per_sec" + override val unit: String = "melems_per_sec" + override val higherIsBetter: Boolean = true + override val kernelProvider: String = providerName + + private val rows: Int = if (smoke) 64 else 1024 + private val cols: Int = 1024 + override val parameters: Map = mapOf( + "rows" to rows.toString(), + "cols" to cols.toString(), + "kernel" to providerName, + ) + + private lateinit var ctx: DirectCpuExecutionContext + private lateinit var x: Tensor + + override fun setup() { + registerOnly(providerName) + ctx = DirectCpuExecutionContext() + x = ctx.fromFloatArray(Shape(rows, cols), FP32::class, pseudo(rows * cols, 3, 4f)) + ctx.ops.softmax(x, -1) + } + + override fun runOnce(): Double { + val start = System.nanoTime() + val result = ctx.ops.softmax(x, -1) + val elapsedNs = System.nanoTime() - start + @Suppress("UNUSED_VARIABLE") val sink = result.hashCode() + return (rows.toDouble() * cols / (elapsedNs / 1_000_000_000.0)) / 1e6 + } + + override fun teardown() = KernelRegistry.clearForTesting() +} + +/** A `LayerNormalization` layer with affine over `[rows, 768]` FP32. Primary metric: million elements per second. */ +internal class LayerNormScenario( + smoke: Boolean, + private val providerName: String, +) : Scenario { + override val id: String = "engine-layer-norm" + override val suite: String = "skainet-engine" + override val primaryMetric: String = "melems_per_sec" + override val unit: String = "melems_per_sec" + override val higherIsBetter: Boolean = true + override val kernelProvider: String = providerName + + private val rows: Int = if (smoke) 64 else 1024 + private val cols: Int = 768 + override val parameters: Map = mapOf( + "rows" to rows.toString(), + "cols" to cols.toString(), + "affine" to "true", + "kernel" to providerName, + ) + + private lateinit var ctx: DirectCpuExecutionContext + private lateinit var x: Tensor + private lateinit var layer: LayerNormalization + + override fun setup() { + registerOnly(providerName) + ctx = DirectCpuExecutionContext() + x = ctx.fromFloatArray(Shape(rows, cols), FP32::class, pseudo(rows * cols, 5, 2f)) + val gamma = ctx.fromFloatArray(Shape(cols), FP32::class, FloatArray(cols) { 0.5f + (it % 7) * 0.1f }) + val beta = ctx.fromFloatArray(Shape(cols), FP32::class, FloatArray(cols) { ((it % 5) - 2) * 0.05f }) + layer = LayerNormalization(intArrayOf(cols), eps = 1e-5, elementwiseAffine = true, name = "ln", initGamma = gamma, initBeta = beta) + layer.forward(x, ctx) + } + + override fun runOnce(): Double { + val start = System.nanoTime() + val result = layer.forward(x, ctx) + val elapsedNs = System.nanoTime() - start + @Suppress("UNUSED_VARIABLE") val sink = result.hashCode() + return (rows.toDouble() * cols / (elapsedNs / 1_000_000_000.0)) / 1e6 + } + + override fun teardown() = KernelRegistry.clearForTesting() +} + +/** + * `ctx.ops.scaledDotProductAttention`, causal, at an encoder's shape: 12 heads, 150 queries over 150 + * keys, head dim 64. Primary metric: GFLOP/s over `4 · heads · seqQ · seqKV · headDim` (the two + * products; the causal half that is never computed is counted, as the dense reference counts it). + */ +internal class SdpaScenario( + smoke: Boolean, + private val providerName: String, +) : Scenario { + override val id: String = "engine-sdpa" + override val suite: String = "skainet-engine" + override val primaryMetric: String = "gflops" + override val unit: String = "gflops" + override val higherIsBetter: Boolean = true + override val kernelProvider: String = providerName + + private val heads: Int = 12 + private val seq: Int = if (smoke) 32 else 150 + private val headDim: Int = 64 + override val parameters: Map = mapOf( + "batch" to "1", + "heads" to heads.toString(), + "seq" to seq.toString(), + "head_dim" to headDim.toString(), + "causal" to "true", + "kernel" to providerName, + ) + + private lateinit var ctx: DirectCpuExecutionContext + private lateinit var q: Tensor + private lateinit var k: Tensor + private lateinit var v: Tensor + + override fun setup() { + registerOnly(providerName) + ctx = DirectCpuExecutionContext() + val n = heads * seq * headDim + q = ctx.fromFloatArray(Shape(1, heads, seq, headDim), FP32::class, pseudo(n, 7)) + k = ctx.fromFloatArray(Shape(1, heads, seq, headDim), FP32::class, pseudo(n, 11)) + v = ctx.fromFloatArray(Shape(1, heads, seq, headDim), FP32::class, pseudo(n, 13)) + ctx.ops.scaledDotProductAttention(q, k, v, causal = true) + } + + override fun runOnce(): Double { + val start = System.nanoTime() + val result = ctx.ops.scaledDotProductAttention(q, k, v, causal = true) + val elapsedNs = System.nanoTime() - start + @Suppress("UNUSED_VARIABLE") val sink = result.hashCode() + val flops = 4.0 * heads * seq * seq * headDim + return (flops / (elapsedNs / 1_000_000_000.0)) / 1e9 + } + + override fun teardown() = KernelRegistry.clearForTesting() +} + +/** + * `ctx.ops.conv2d`: a CNN's 3 × 3 same-padding layer, 64 → 64 channels over a 56 × 56 map, with bias. + * Primary metric: GFLOP/s over `2 · c_out · out_h · out_w · c_in · k_h · k_w`. On the JVM every + * provider measures the same path — `DefaultCpuOpsJvm`'s im2col + Vector API kernels, which no + * provider replaces; the `kernel` parameter records which one was registered. The wasm tier runs the + * same scenario as im2col + GEMM in the sidecar. + */ +internal class Conv2dScenario( + smoke: Boolean, + private val providerName: String, +) : Scenario { + override val id: String = "engine-conv2d" + override val suite: String = "skainet-engine" + override val primaryMetric: String = "gflops" + override val unit: String = "gflops" + override val higherIsBetter: Boolean = true + override val kernelProvider: String = providerName + + private val cIn: Int = if (smoke) 16 else 64 + private val cOut: Int = if (smoke) 16 else 64 + private val hw: Int = if (smoke) 28 else 56 + private val k: Int = 3 + override val parameters: Map = mapOf( + "batch" to "1", + "c_in" to cIn.toString(), + "c_out" to cOut.toString(), + "hw" to hw.toString(), + "kernel_size" to k.toString(), + "bias" to "true", + "kernel" to providerName, + ) + + private lateinit var ctx: DirectCpuExecutionContext + private lateinit var x: Tensor + private lateinit var w: Tensor + private lateinit var b: Tensor + + override fun setup() { + registerOnly(providerName) + ctx = DirectCpuExecutionContext() + x = ctx.fromFloatArray(Shape(1, cIn, hw, hw), FP32::class, pseudo(cIn * hw * hw, 9)) + w = ctx.fromFloatArray(Shape(cOut, cIn, k, k), FP32::class, pseudo(cOut * cIn * k * k, 17, 0.2f)) + b = ctx.fromFloatArray(Shape(cOut), FP32::class, pseudo(cOut, 19)) + ctx.ops.conv2d(x, w, b, 1 to 1, 1 to 1, 1 to 1, 1) + } + + override fun runOnce(): Double { + val start = System.nanoTime() + val result = ctx.ops.conv2d(x, w, b, 1 to 1, 1 to 1, 1 to 1, 1) + val elapsedNs = System.nanoTime() - start + @Suppress("UNUSED_VARIABLE") val sink = result.hashCode() + val flops = 2.0 * cOut * hw * hw * cIn * k * k + return (flops / (elapsedNs / 1_000_000_000.0)) / 1e9 + } + + override fun teardown() = KernelRegistry.clearForTesting() +} diff --git a/skainet-backends/benchmarks/jvm-cpu-publish/src/main/kotlin/sk/ainet/bench/publish/scenarios/ScenarioRegistry.kt b/skainet-backends/benchmarks/jvm-cpu-publish/src/main/kotlin/sk/ainet/bench/publish/scenarios/ScenarioRegistry.kt index 74a595fea..b3d644ff1 100644 --- a/skainet-backends/benchmarks/jvm-cpu-publish/src/main/kotlin/sk/ainet/bench/publish/scenarios/ScenarioRegistry.kt +++ b/skainet-backends/benchmarks/jvm-cpu-publish/src/main/kotlin/sk/ainet/bench/publish/scenarios/ScenarioRegistry.kt @@ -6,6 +6,7 @@ public object ScenarioRegistry { public fun ids(): List = listOf( "engine-fp32-gemm", + "engine-fp32-linear", "engine-q4-gemm", "engine-kernel-matmul", "engine-bf16-matmul", @@ -14,10 +15,15 @@ public object ScenarioRegistry { "engine-elementwise-add", "engine-reductions-sum", "engine-reductions-mean", + "engine-softmax", + "engine-layer-norm", + "engine-sdpa", + "engine-conv2d", ) public fun byId(id: String, provider: String, smoke: Boolean): Scenario? = when (id) { "engine-fp32-gemm" -> Fp32GemmScenario(smoke = smoke, providerName = provider) + "engine-fp32-linear" -> Fp32LinearScenario(smoke = smoke, providerName = provider) "engine-q4-gemm" -> Q4GemmScenario(smoke = smoke, providerName = provider) "engine-kernel-matmul" -> KernelMatmulScenario(smoke = smoke, providerName = provider) "engine-bf16-matmul" -> Bf16MatmulScenario(smoke = smoke, providerName = provider) @@ -26,6 +32,10 @@ public object ScenarioRegistry { "engine-elementwise-add" -> ElementwiseAddScenario(smoke = smoke, providerName = provider) "engine-reductions-sum" -> ReductionsScenario(op = ReductionOp.SUM, smoke = smoke, providerName = provider) "engine-reductions-mean" -> ReductionsScenario(op = ReductionOp.MEAN, smoke = smoke, providerName = provider) + "engine-softmax" -> SoftmaxScenario(smoke = smoke, providerName = provider) + "engine-layer-norm" -> LayerNormScenario(smoke = smoke, providerName = provider) + "engine-sdpa" -> SdpaScenario(smoke = smoke, providerName = provider) + "engine-conv2d" -> Conv2dScenario(smoke = smoke, providerName = provider) else -> null } } diff --git a/skainet-backends/benchmarks/wasm-bench/src/wasmJsMain/kotlin/sk/ainet/bench/wasm/Main.kt b/skainet-backends/benchmarks/wasm-bench/src/wasmJsMain/kotlin/sk/ainet/bench/wasm/Main.kt index 72c130075..b1108a97b 100644 --- a/skainet-backends/benchmarks/wasm-bench/src/wasmJsMain/kotlin/sk/ainet/bench/wasm/Main.kt +++ b/skainet-backends/benchmarks/wasm-bench/src/wasmJsMain/kotlin/sk/ainet/bench/wasm/Main.kt @@ -1,6 +1,7 @@ package sk.ainet.bench.wasm import sk.ainet.context.DirectCpuExecutionContext +import sk.ainet.exec.wasm.WasmCpuOps import sk.ainet.exec.wasm.WasmKernels import sk.ainet.exec.wasm.WasmSimd import sk.ainet.lang.tensor.Shape @@ -90,8 +91,15 @@ private fun report(tier: String, case: Case, form: String, ms: Double) { /** * Three passes: the scalar lane, then the SIMD128 sidecar single-threaded, then with pthreads when * the runtime allows them (Node, or a cross-origin-isolated page). Each line carries `tier=`. + * + * `SKAINET_WASM_RESIDENT=0` in the environment (Node) runs the sidecar passes with + * `WasmSimd.residentBudgetBytes = 0`, i.e. every op output read back to the heap — the A/B for the + * resident-activation step. */ suspend fun main() { + val resident = envVar("SKAINET_WASM_RESIDENT") != "0" + if (!resident) WasmSimd.residentBudgetBytes = 0L + println("[SKAINET_PERF_WASM] residentActivations=$resident") runPass("scalar") WasmSimd.install(threads = false) runPass(WasmKernels.mode) @@ -122,4 +130,39 @@ private fun runPass(tier: String) { report(tier, case, "dense", timeMedianMs(reps) { ctx.ops.matmulWeightTransposed(x, dense) }) report(tier, case, "q8_0", timeMedianMs(reps) { ctx.ops.matmulWeightTransposed(x, packed) }) } + runChain(tier, ctx) +} + +/** + * The resident-activation case: one Laya MLP block's shape of ops between two matmuls — + * `x · Wᵢᵀ`, GELU, a gated multiply, a bias add, then a reduction that reads the result back — + * timed end to end. On the scalar lane every op allocates and walks a heap array; on the sidecar + * the intermediates stay in linear memory and only the final scalar crosses the boundary. + * + * `[SKAINET_PERF_WASM] tier= laya.chain m=150 n=2304 k=768 ops=matmulWt+gelu+mul+add+sum residentBudget= ms=` + */ +private fun runChain(tier: String, ctx: DirectCpuExecutionContext) { + val m = 150; val n = 2304; val k = 768 + val x = ctx.fromFloatArray(Shape(m, k), FP32::class, pseudo(m * k, 3)) + val w = ctx.fromFloatArray(Shape(n, k), FP32::class, pseudo(n * k, 7)) + val bias = ctx.fromFloatArray(Shape(n), FP32::class, pseudo(n, 11)) + fun chain(): Float { + val h = ctx.ops.matmulWeightTransposed(x, w) + val g = ctx.ops.gelu(h) + val s = ctx.ops.multiply(g, h) + val o = ctx.ops.add(s, bias) + return ctx.ops.sum(o).data.copyToFloatArray()[0] + } + chain() + val ms = timeMedianMs(5) { chain() } + val msText = ((ms * 100).roundToInt() / 100.0).toString() + val budget = if (ctx.ops is WasmCpuOps) WasmSimd.residentBudgetBytes else 0L + println("[SKAINET_PERF_WASM] tier=$tier laya.chain m=$m n=$n k=$k ops=matmulWt+gelu+mul+add+sum residentBudget=$budget ms=$msText") } + +@OptIn(kotlin.js.ExperimentalWasmJsInterop::class) +private fun envVar(name: String): String? = jsEnvVar(name)?.toString() + +@OptIn(kotlin.js.ExperimentalWasmJsInterop::class) +private fun jsEnvVar(name: String): JsString? = + js("(typeof process !== 'undefined' && process.env && process.env[name] != null) ? process.env[name] : null") diff --git a/skainet-backends/benchmarks/wasm-bench/src/wasmJsMain/kotlin/sk/ainet/bench/wasm/Scenarios.kt b/skainet-backends/benchmarks/wasm-bench/src/wasmJsMain/kotlin/sk/ainet/bench/wasm/Scenarios.kt index 271fc27be..52d927da7 100644 --- a/skainet-backends/benchmarks/wasm-bench/src/wasmJsMain/kotlin/sk/ainet/bench/wasm/Scenarios.kt +++ b/skainet-backends/benchmarks/wasm-bench/src/wasmJsMain/kotlin/sk/ainet/bench/wasm/Scenarios.kt @@ -4,9 +4,11 @@ import sk.ainet.backend.api.kernel.KernelDispatch import sk.ainet.backend.api.kernel.KernelKey import sk.ainet.backend.api.kernel.KernelRegistry import sk.ainet.context.DirectCpuExecutionContext +import sk.ainet.exec.wasm.WasmCpuOps import sk.ainet.lang.memory.Scope import sk.ainet.lang.memory.Storage import sk.ainet.lang.memory.TensorView +import sk.ainet.lang.nn.normalization.LayerNormalization import sk.ainet.lang.tensor.Shape import sk.ainet.lang.tensor.data.BitNetB158TensorData import sk.ainet.lang.types.FP32 @@ -48,6 +50,15 @@ internal fun engineScenarios(): List = listOf( val flops = 2.0 * n * n * n; { flops / seconds { ctx.ops.matmul(a, b) } / 1e9 } }, + WasmScenario("engine-fp32-linear", "gflops", mapOf("m" to "16", "k" to "768", "n" to "3072"), reps = 10) { + val m = 16; val k = 768; val n = 3072 + val ctx = DirectCpuExecutionContext() + val x = ctx.fromFloatArray(Shape(m, k), FP32::class, FloatArray(m * k) { ((it % 251) - 125).toFloat() / 127f }) + val w = ctx.fromFloatArray(Shape(n, k), FP32::class, FloatArray(n * k) { ((it * 13 % 257) - 128).toFloat() / 127f }) + ctx.ops.matmulWeightTransposed(x, w) // the resident upload happens here, not in a measured run + val flops = 2.0 * m * n * k; + { flops / seconds { ctx.ops.matmulWeightTransposed(x, w) } / 1e9 } + }, WasmScenario("engine-kernel-matmul", "gflops", mapOf("size" to "1024"), reps = 3) { val n = 1024 DirectCpuExecutionContext() @@ -123,6 +134,23 @@ internal fun engineScenarios(): List = listOf( val b = ctx.fromFloatArray(Shape(n), FP32::class, FloatArray(n) { 1f }); { n / seconds { ctx.ops.add(a, b) } / 1e6 } }, + // The same three ops on operands already resident in the sidecar (the shape a chain between two + // matmuls has); heap operands above stay on the common lane by design. + WasmScenario("engine-elementwise-add", "melems_per_sec", mapOf("elements" to "1000000", "operands" to "resident"), reps = 10) { + val n = 1_000_000 + val ctx = DirectCpuExecutionContext() + val ops = ctx.ops as? WasmCpuOps ?: error("resident operands need the sidecar") + val a = ops.toResident(ctx.fromFloatArray(Shape(n), FP32::class, FloatArray(n) { it.toFloat() * 0.5f })) + val b = ops.toResident(ctx.fromFloatArray(Shape(n), FP32::class, FloatArray(n) { 1f })); + { n / seconds { ctx.ops.add(a, b) } / 1e6 } + }, + WasmScenario("engine-reductions-sum", "melems_per_sec", mapOf("elements" to "1000000", "op" to "sum", "operands" to "resident"), reps = 10) { + val n = 1_000_000 + val ctx = DirectCpuExecutionContext() + val ops = ctx.ops as? WasmCpuOps ?: error("resident operands need the sidecar") + val a = ops.toResident(ctx.fromFloatArray(Shape(n), FP32::class, FloatArray(n) { (it % 1024).toFloat() * 0.25f })); + { n / seconds { ctx.ops.sum(a).data.copyToFloatArray() } / 1e6 } + }, WasmScenario("engine-reductions-sum", "melems_per_sec", mapOf("elements" to "1000000", "op" to "sum"), reps = 10) { val n = 1_000_000 val ctx = DirectCpuExecutionContext() @@ -135,6 +163,51 @@ internal fun engineScenarios(): List = listOf( val a = ctx.fromFloatArray(Shape(n), FP32::class, FloatArray(n) { (it % 1024).toFloat() * 0.25f }); { n / seconds { ctx.ops.mean(a) } / 1e6 } }, + // The fused row ops (softmax, a LayerNormalization layer, causal attention), the shapes of the JVM harness. + WasmScenario("engine-softmax", "melems_per_sec", mapOf("rows" to "1024", "cols" to "1024"), reps = 10) { + val rows = 1024; val cols = 1024 + val ctx = DirectCpuExecutionContext() + val rnd = lcg(3) + val x = ctx.fromFloatArray(Shape(rows, cols), FP32::class, FloatArray(rows * cols) { (rnd() - 0.5f) * 8f }) + ctx.ops.softmax(x, -1); + { rows.toDouble() * cols / seconds { ctx.ops.softmax(x, -1) } / 1e6 } + }, + WasmScenario("engine-layer-norm", "melems_per_sec", mapOf("rows" to "1024", "cols" to "768", "affine" to "true"), reps = 10) { + val rows = 1024; val cols = 768 + val ctx = DirectCpuExecutionContext() + val rnd = lcg(5) + val x = ctx.fromFloatArray(Shape(rows, cols), FP32::class, FloatArray(rows * cols) { (rnd() - 0.5f) * 4f }) + val gamma = ctx.fromFloatArray(Shape(cols), FP32::class, FloatArray(cols) { 0.5f + (it % 7) * 0.1f }) + val beta = ctx.fromFloatArray(Shape(cols), FP32::class, FloatArray(cols) { ((it % 5) - 2) * 0.05f }) + val layer = LayerNormalization(intArrayOf(cols), eps = 1e-5, elementwiseAffine = true, name = "ln", initGamma = gamma, initBeta = beta) + layer.forward(x, ctx); + { rows.toDouble() * cols / seconds { layer.forward(x, ctx) } / 1e6 } + }, + WasmScenario("engine-sdpa", "gflops", mapOf("batch" to "1", "heads" to "12", "seq" to "150", "head_dim" to "64", "causal" to "true"), reps = 5) { + val heads = 12; val seq = 150; val headDim = 64 + val ctx = DirectCpuExecutionContext() + val rnd = lcg(7) + val n = heads * seq * headDim + val q = ctx.fromFloatArray(Shape(1, heads, seq, headDim), FP32::class, FloatArray(n) { rnd() - 0.5f }) + val k = ctx.fromFloatArray(Shape(1, heads, seq, headDim), FP32::class, FloatArray(n) { rnd() - 0.5f }) + val v = ctx.fromFloatArray(Shape(1, heads, seq, headDim), FP32::class, FloatArray(n) { rnd() - 0.5f }) + ctx.ops.scaledDotProductAttention(q, k, v, causal = true) + val flops = 4.0 * heads * seq * seq * headDim; + { flops / seconds { ctx.ops.scaledDotProductAttention(q, k, v, causal = true) } / 1e9 } + }, + // A CNN's 3×3 same-padding layer, 64 → 64 channels over a 56 × 56 map, with bias: im2col + GEMM + // in the sidecar (the common scalar loop before WasmSimd.install()). + WasmScenario("engine-conv2d", "gflops", mapOf("batch" to "1", "c_in" to "64", "c_out" to "64", "hw" to "56", "kernel_size" to "3", "bias" to "true"), reps = 5) { + val cIn = 64; val cOut = 64; val hw = 56; val k = 3 + val ctx = DirectCpuExecutionContext() + val rnd = lcg(9) + val x = ctx.fromFloatArray(Shape(1, cIn, hw, hw), FP32::class, FloatArray(cIn * hw * hw) { rnd() - 0.5f }) + val w = ctx.fromFloatArray(Shape(cOut, cIn, k, k), FP32::class, FloatArray(cOut * cIn * k * k) { (rnd() - 0.5f) * 0.2f }) + val b = ctx.fromFloatArray(Shape(cOut), FP32::class, FloatArray(cOut) { rnd() - 0.5f }) + ctx.ops.conv2d(x, w, b, 1 to 1, 1 to 1, 1 to 1, 1) // the weight's resident upload happens here + val flops = 2.0 * cOut * hw * hw * cIn * k * k; + { flops / seconds { ctx.ops.conv2d(x, w, b, 1 to 1, 1 to 1, 1 to 1, 1) } / 1e9 } + }, ) private inline fun seconds(block: () -> Unit): Double { diff --git a/skainet-backends/skainet-backend-api/src/commonMain/kotlin/sk/ainet/backend/api/kernel/Fp32FusedKernels.kt b/skainet-backends/skainet-backend-api/src/commonMain/kotlin/sk/ainet/backend/api/kernel/Fp32FusedKernels.kt new file mode 100644 index 000000000..7081399a8 --- /dev/null +++ b/skainet-backends/skainet-backend-api/src/commonMain/kotlin/sk/ainet/backend/api/kernel/Fp32FusedKernels.kt @@ -0,0 +1,84 @@ +package sk.ainet.backend.api.kernel + +/** + * Row-wise softmax over dense FP32: `out[r, :] = softmax(input[r, :])` for `rows` contiguous rows of + * `cols`, or the log-softmax (`input - (log Σ exp(input - max) + max)`) when [log] is `true`. The + * SPI face of `skainet_f32_softmax_rows` (SKEEP-007 fused ops), which the wasm sidecar and the JVM + * FFM provider both serve; a provider without it returns `null` from [KernelProvider.softmaxFp32] + * and the ops layer keeps its common loop. + * + * Operands are contiguous from their offsets: `rows × cols` floats each. Implementations must not + * mutate [input] and must fully overwrite the output block; `rows == 0` or `cols == 0` is a no-op. + * `exp` and `log` are the platform's libm, so results agree with the common Kotlin loop to the + * last-ulp differences between the two. + */ +public interface Fp32SoftmaxKernel { + public fun softmaxRows( + input: FloatArray, inputOffset: Int, + out: FloatArray, outOffset: Int, + rows: Int, cols: Int, + log: Boolean, + ) +} + +/** + * Fused normalisations over the last dimension of dense FP32, one pass per row — the SPI face of + * `skainet_f32_layer_norm` and `skainet_f32_rms_norm` (SKEEP-007 fused ops): + * + * - [layerNorm]: `out[r, c] = (x[r, c] − mean_r) / sqrt(var_r + eps) · gamma[c] + beta[c]` with the + * mean and the biased variance over the row; + * - [rmsNorm]: `out[r, c] = x[r, c] / sqrt(mean(x[r, :]²) + eps) · gamma[c]`. + * + * [gamma] and [beta] are `cols` floats or `null` (the affine step is skipped). Operands are + * contiguous from their offsets; implementations must not mutate the inputs and must fully overwrite + * the `rows × cols` output block. The row sums accumulate in lanes where the kernel is vectorised, + * so results differ from the decomposed `mean`/`subtract`/`sqrt`/`divide` chain in the last bits. + */ +public interface Fp32NormKernel { + public fun layerNorm( + input: FloatArray, inputOffset: Int, + gamma: FloatArray?, beta: FloatArray?, + out: FloatArray, outOffset: Int, + rows: Int, cols: Int, + eps: Float, + ) + + public fun rmsNorm( + input: FloatArray, inputOffset: Int, + gamma: FloatArray?, + out: FloatArray, outOffset: Int, + rows: Int, cols: Int, + eps: Float, + ) +} + +/** + * Fused scaled dot-product attention over dense FP32 — the SPI face of `skainet_f32_sdpa` (SKEEP-007 + * fused ops), with the full contract of `TensorOps.scaledDotProductAttention`: + * + * `out = softmax(query · keyᵀ · scale + rules + mask) · value` + * + * [query] is `[batch, heads, seqQ, headDim]`, [key] and [value] are `[batch, kvHeads, seqKV, headDim]` + * with query head `h` reading K/V head `h / (heads / kvHeads)` (grouped-query attention), [out] is + * `[batch, heads, seqQ, headDim]`, all contiguous whole arrays. [scale] is applied as given — the + * caller resolves the `0 → 1/sqrt(headDim)` default. The structural rules are those of + * `AttentionMask`: [causal], a sliding [window] with [rightContext], and per-batch [keyLengths] + * (`[batch]` or `null`); a key outside a row's attended set scores `-1e30` and its product is + * never computed, so a fully masked row is uniform and finite. [mask] is a dense additive mask + * (or `null`) broadcast trailing-aligned over [maskDims] `= [batch | 1, heads | kvHeads | 1, + * seqQ | 1, seqKV | 1]`, added to every score. + * + * The `q · k` dot products accumulate in lanes where the kernel is vectorised; the + * probability-weighted sum of `value` accumulates per output element in ascending key order. + * Implementations must not mutate the inputs and must fully overwrite [out]. + */ +public interface Fp32AttentionKernel { + public fun sdpa( + query: FloatArray, key: FloatArray, value: FloatArray, out: FloatArray, + batch: Int, heads: Int, kvHeads: Int, + seqQ: Int, seqKV: Int, headDim: Int, + scale: Float, causal: Boolean, window: Int, rightContext: Int, + keyLengths: IntArray?, + mask: FloatArray?, maskDims: IntArray?, + ) +} diff --git a/skainet-backends/skainet-backend-api/src/commonMain/kotlin/sk/ainet/backend/api/kernel/Fp32MatmulWtKernel.kt b/skainet-backends/skainet-backend-api/src/commonMain/kotlin/sk/ainet/backend/api/kernel/Fp32MatmulWtKernel.kt new file mode 100644 index 000000000..d85d77a18 --- /dev/null +++ b/skainet-backends/skainet-backend-api/src/commonMain/kotlin/sk/ainet/backend/api/kernel/Fp32MatmulWtKernel.kt @@ -0,0 +1,28 @@ +package sk.ainet.backend.api.kernel + +/** + * Dense `x · Wᵀ` for a row-major `[out, in]` FP32 weight — the shape a `Linear` layer stores its + * weight in, multiplied in place: + * + * `out[i, o] = Σ_p x[i, p] · w[o, p]` + * + * Every output is one contiguous dot product over the weight's own row, so no transposed copy of + * the weight is ever made. This is the SPI face of `skainet_fp32_matmul_wt` (SKEEP-007), which the + * wasm sidecar has served since 0.61.0 and the JVM FFM provider binds as well: a tier whose + * [Fp32MatmulKernel] wants the weight input-major (`[k, n]`) otherwise pays a scattered transpose + * copy per weight and caches it — about 0.5 GB for a 22-layer encoder. + * + * Operands are contiguous: `x` is `m` rows of `k`, `w` is `n` rows of `k`, `out` is `m` rows of + * `n`, each from its offset. Implementations must not mutate `x` or `w`, must fully overwrite the + * `m × n` block of `out`, and may assume the arrays do not alias. `k == 0` zeroes the block; + * `m == 0` or `n == 0` is a no-op. Each output accumulates its products in ascending `p`, so a + * conforming kernel is bit-identical to transpose-then-GEMM with the same accumulation order. + */ +public interface Fp32MatmulWtKernel { + public fun matmulWt( + x: FloatArray, xOffset: Int, + w: FloatArray, wOffset: Int, + out: FloatArray, outOffset: Int, + m: Int, n: Int, k: Int, + ) +} diff --git a/skainet-backends/skainet-backend-api/src/commonMain/kotlin/sk/ainet/backend/api/kernel/KernelProvider.kt b/skainet-backends/skainet-backend-api/src/commonMain/kotlin/sk/ainet/backend/api/kernel/KernelProvider.kt index d1126a82f..2936d374d 100644 --- a/skainet-backends/skainet-backend-api/src/commonMain/kotlin/sk/ainet/backend/api/kernel/KernelProvider.kt +++ b/skainet-backends/skainet-backend-api/src/commonMain/kotlin/sk/ainet/backend/api/kernel/KernelProvider.kt @@ -103,6 +103,34 @@ public interface KernelProvider { */ public fun matmulQ5_0(): Q5_0MatmulKernel? = null + /** + * Dense `x · Wᵀ` over a row-major `[out, in]` FP32 weight ([Fp32MatmulWtKernel]), or `null` + * when this provider has no direct kernel for it and the caller should transpose once and use + * [matmulFp32]. Same fall-through pattern as [matmulQ4K]. + */ + public fun matmulFp32Wt(): Fp32MatmulWtKernel? = null + + /** + * Row-wise softmax / log-softmax over dense FP32 ([Fp32SoftmaxKernel]), or `null` when this + * provider has no fused kernel for it and the ops layer keeps its common loop. Same + * fall-through pattern as [matmulQ4K]; reported by [supports] as `"softmax"` with `["Float32"]`. + */ + public fun softmaxFp32(): Fp32SoftmaxKernel? = null + + /** + * Fused layer norm and RMS norm over the last dimension of dense FP32 ([Fp32NormKernel]), or + * `null`. Reported by [supports] as `"layer_norm"` / `"rms_norm"` with `["Float32"]`. + */ + public fun normFp32(): Fp32NormKernel? = null + + /** + * Fused scaled dot-product attention over dense FP32 ([Fp32AttentionKernel]) with the ops + * layer's structural masks (causal, sliding window, key lengths) and grouped-query heads, or + * `null` when the ops layer runs its common kernel. Reported by [supports] as `"sdpa"` with + * `["Float32"]`. + */ + public fun attentionFp32(): Fp32AttentionKernel? = null + /** * Capability query: does this provider carry a kernel for * [opName] with the given [dtypeKeys]? @@ -122,15 +150,29 @@ public interface KernelProvider { * - For ops that aren't matmul (future: SDPA, gather, RMSNorm…), * providers can override this method to declare those kernels. * - * The default body covers the four matmul accessors that exist - * on this interface today. Providers that ship additional - * kernels override and chain through `super.supports(...)` for - * the matmul base cases. + * The default body covers the matmul accessors on this interface, + * plus `"matmul_wt"` with `["Float32", "Float32"]` for the direct + * `x · Wᵀ` kernel ([matmulFp32Wt]) and the fused row kernels with a + * single `["Float32"]` key — `"softmax"` ([softmaxFp32]), + * `"layer_norm"` / `"rms_norm"` ([normFp32]) and `"sdpa"` + * ([attentionFp32]). Providers that ship additional kernels override + * and chain through `super.supports(...)` for the base cases. */ public fun supports(opName: String, dtypeKeys: List): Boolean { - if (opName != "matmul" || dtypeKeys.size != 2) return false + if (dtypeKeys.size == 1) { + if (dtypeKeys[0] != "Float32") return false + return when (opName) { + "softmax" -> softmaxFp32() != null + "layer_norm", "rms_norm" -> normFp32() != null + "sdpa" -> attentionFp32() != null + else -> false + } + } + if (dtypeKeys.size != 2) return false val (input, weight) = dtypeKeys if (input != "Float32") return false + if (opName == "matmul_wt") return weight == "Float32" && matmulFp32Wt() != null + if (opName != "matmul") return false return when (weight) { "Float32" -> matmulFp32() != null "BFloat16" -> matmulBf16() != null diff --git a/skainet-backends/skainet-backend-api/src/jvmMain/kotlin/sk/ainet/backend/api/kernel/Fp32MemSegMatmulWtKernel.kt b/skainet-backends/skainet-backend-api/src/jvmMain/kotlin/sk/ainet/backend/api/kernel/Fp32MemSegMatmulWtKernel.kt new file mode 100644 index 000000000..6b95f2711 --- /dev/null +++ b/skainet-backends/skainet-backend-api/src/jvmMain/kotlin/sk/ainet/backend/api/kernel/Fp32MemSegMatmulWtKernel.kt @@ -0,0 +1,24 @@ +package sk.ainet.backend.api.kernel + +import java.lang.foreign.MemorySegment + +/** + * [Fp32MatmulWtKernel] with the `[out, in]` FP32 weight supplied as an off-heap + * `java.lang.foreign.MemorySegment` instead of a heap `FloatArray`. JVM-only sibling, for the same + * reason [Q4KMemSegMatmulKernel] exists: a downcall cannot read heap memory on JDK 21, so the heap + * face copies the whole weight on every call — and the weight is the large operand. A weight that + * already lives in a segment (loaded off-heap, or made resident once by the ops layer) is passed + * through with no copy at all; only the `m × k` activation and the `m × n` output are staged. + * + * Same contract as [Fp32MatmulWtKernel]: contiguous operands, `out` fully overwritten, products + * accumulated in ascending `p`. The caller owns the segment's arena and keeps it alive for the + * duration of the call; the kernel retains nothing past return. + */ +public interface Fp32MemSegMatmulWtKernel { + public fun matmulWt( + x: FloatArray, xOffset: Int, + weight: MemorySegment, weightByteOffset: Long, + out: FloatArray, outOffset: Int, + m: Int, n: Int, k: Int, + ) +} diff --git a/skainet-backends/skainet-backend-api/src/jvmMain/kotlin/sk/ainet/backend/api/kernel/MemSegKernelProvider.kt b/skainet-backends/skainet-backend-api/src/jvmMain/kotlin/sk/ainet/backend/api/kernel/MemSegKernelProvider.kt index 527401a5b..c567aa11d 100644 --- a/skainet-backends/skainet-backend-api/src/jvmMain/kotlin/sk/ainet/backend/api/kernel/MemSegKernelProvider.kt +++ b/skainet-backends/skainet-backend-api/src/jvmMain/kotlin/sk/ainet/backend/api/kernel/MemSegKernelProvider.kt @@ -34,4 +34,10 @@ public interface MemSegKernelProvider { * keep compiling. */ public fun matmulQ4KMemSeg(): Q4KMemSegMatmulKernel? = null + + /** + * Dense `x · Wᵀ` with the `[out, in]` FP32 weight supplied as a `MemorySegment` + * ([Fp32MemSegMatmulWtKernel]), or `null`. Same `null` default as [matmulQ4KMemSeg]. + */ + public fun matmulFp32WtMemSeg(): Fp32MemSegMatmulWtKernel? = null } diff --git a/skainet-backends/skainet-backend-cpu/api/jvm/skainet-backend-cpu.api b/skainet-backends/skainet-backend-cpu/api/jvm/skainet-backend-cpu.api index e0b809808..1d671e936 100644 --- a/skainet-backends/skainet-backend-cpu/api/jvm/skainet-backend-cpu.api +++ b/skainet-backends/skainet-backend-cpu/api/jvm/skainet-backend-cpu.api @@ -61,12 +61,14 @@ public final class sk/ainet/exec/kernel/PanamaVectorFp16MatmulKernel : sk/ainet/ public final class sk/ainet/exec/kernel/PanamaVectorKernelProvider : sk/ainet/backend/api/kernel/KernelProvider { public static final field INSTANCE Lsk/ainet/exec/kernel/PanamaVectorKernelProvider; + public fun attentionFp32 ()Lsk/ainet/backend/api/kernel/Fp32AttentionKernel; public fun getName ()Ljava/lang/String; public fun getPriority ()I public fun isAvailable ()Z public fun matmulBf16 ()Lsk/ainet/backend/api/kernel/Bf16MatmulKernel; public fun matmulFp16 ()Lsk/ainet/backend/api/kernel/Fp16MatmulKernel; public fun matmulFp32 ()Lsk/ainet/backend/api/kernel/Fp32MatmulKernel; + public fun matmulFp32Wt ()Lsk/ainet/backend/api/kernel/Fp32MatmulWtKernel; public fun matmulQ4K ()Lsk/ainet/backend/api/kernel/Q4KMatmulKernel; public fun matmulQ4_0 ()Lsk/ainet/backend/api/kernel/Q4_0MatmulKernel; public fun matmulQ5K ()Lsk/ainet/backend/api/kernel/Q5KMatmulKernel; @@ -74,17 +76,21 @@ public final class sk/ainet/exec/kernel/PanamaVectorKernelProvider : sk/ainet/ba public fun matmulQ5_1 ()Lsk/ainet/backend/api/kernel/Q5_1MatmulKernel; public fun matmulQ6K ()Lsk/ainet/backend/api/kernel/Q6KMatmulKernel; public fun matmulQ8_0 ()Lsk/ainet/backend/api/kernel/Q8_0MatmulKernel; + public fun normFp32 ()Lsk/ainet/backend/api/kernel/Fp32NormKernel; + public fun softmaxFp32 ()Lsk/ainet/backend/api/kernel/Fp32SoftmaxKernel; public fun supports (Ljava/lang/String;Ljava/util/List;)Z } public final class sk/ainet/exec/kernel/PanamaVectorKernelProviderFactory : sk/ainet/backend/api/kernel/KernelProvider { public fun ()V + public fun attentionFp32 ()Lsk/ainet/backend/api/kernel/Fp32AttentionKernel; public fun getName ()Ljava/lang/String; public fun getPriority ()I public fun isAvailable ()Z public fun matmulBf16 ()Lsk/ainet/backend/api/kernel/Bf16MatmulKernel; public fun matmulFp16 ()Lsk/ainet/backend/api/kernel/Fp16MatmulKernel; public fun matmulFp32 ()Lsk/ainet/backend/api/kernel/Fp32MatmulKernel; + public fun matmulFp32Wt ()Lsk/ainet/backend/api/kernel/Fp32MatmulWtKernel; public fun matmulQ4K ()Lsk/ainet/backend/api/kernel/Q4KMatmulKernel; public fun matmulQ4_0 ()Lsk/ainet/backend/api/kernel/Q4_0MatmulKernel; public fun matmulQ5K ()Lsk/ainet/backend/api/kernel/Q5KMatmulKernel; @@ -92,6 +98,8 @@ public final class sk/ainet/exec/kernel/PanamaVectorKernelProviderFactory : sk/a public fun matmulQ5_1 ()Lsk/ainet/backend/api/kernel/Q5_1MatmulKernel; public fun matmulQ6K ()Lsk/ainet/backend/api/kernel/Q6KMatmulKernel; public fun matmulQ8_0 ()Lsk/ainet/backend/api/kernel/Q8_0MatmulKernel; + public fun normFp32 ()Lsk/ainet/backend/api/kernel/Fp32NormKernel; + public fun softmaxFp32 ()Lsk/ainet/backend/api/kernel/Fp32SoftmaxKernel; public fun supports (Ljava/lang/String;Ljava/util/List;)Z } @@ -162,12 +170,14 @@ public final class sk/ainet/exec/kernel/ScalarFp16MatmulKernel : sk/ainet/backen public final class sk/ainet/exec/kernel/ScalarKernelProvider : sk/ainet/backend/api/kernel/KernelProvider { public static final field INSTANCE Lsk/ainet/exec/kernel/ScalarKernelProvider; + public fun attentionFp32 ()Lsk/ainet/backend/api/kernel/Fp32AttentionKernel; public fun getName ()Ljava/lang/String; public fun getPriority ()I public fun isAvailable ()Z public fun matmulBf16 ()Lsk/ainet/backend/api/kernel/Bf16MatmulKernel; public fun matmulFp16 ()Lsk/ainet/backend/api/kernel/Fp16MatmulKernel; public fun matmulFp32 ()Lsk/ainet/backend/api/kernel/Fp32MatmulKernel; + public fun matmulFp32Wt ()Lsk/ainet/backend/api/kernel/Fp32MatmulWtKernel; public fun matmulQ4K ()Lsk/ainet/backend/api/kernel/Q4KMatmulKernel; public fun matmulQ4_0 ()Lsk/ainet/backend/api/kernel/Q4_0MatmulKernel; public fun matmulQ5K ()Lsk/ainet/backend/api/kernel/Q5KMatmulKernel; @@ -175,17 +185,21 @@ public final class sk/ainet/exec/kernel/ScalarKernelProvider : sk/ainet/backend/ public fun matmulQ5_1 ()Lsk/ainet/backend/api/kernel/Q5_1MatmulKernel; public fun matmulQ6K ()Lsk/ainet/backend/api/kernel/Q6KMatmulKernel; public fun matmulQ8_0 ()Lsk/ainet/backend/api/kernel/Q8_0MatmulKernel; + public fun normFp32 ()Lsk/ainet/backend/api/kernel/Fp32NormKernel; + public fun softmaxFp32 ()Lsk/ainet/backend/api/kernel/Fp32SoftmaxKernel; public fun supports (Ljava/lang/String;Ljava/util/List;)Z } public final class sk/ainet/exec/kernel/ScalarKernelProviderFactory : sk/ainet/backend/api/kernel/KernelProvider { public fun ()V + public fun attentionFp32 ()Lsk/ainet/backend/api/kernel/Fp32AttentionKernel; public fun getName ()Ljava/lang/String; public fun getPriority ()I public fun isAvailable ()Z public fun matmulBf16 ()Lsk/ainet/backend/api/kernel/Bf16MatmulKernel; public fun matmulFp16 ()Lsk/ainet/backend/api/kernel/Fp16MatmulKernel; public fun matmulFp32 ()Lsk/ainet/backend/api/kernel/Fp32MatmulKernel; + public fun matmulFp32Wt ()Lsk/ainet/backend/api/kernel/Fp32MatmulWtKernel; public fun matmulQ4K ()Lsk/ainet/backend/api/kernel/Q4KMatmulKernel; public fun matmulQ4_0 ()Lsk/ainet/backend/api/kernel/Q4_0MatmulKernel; public fun matmulQ5K ()Lsk/ainet/backend/api/kernel/Q5KMatmulKernel; @@ -193,6 +207,8 @@ public final class sk/ainet/exec/kernel/ScalarKernelProviderFactory : sk/ainet/b public fun matmulQ5_1 ()Lsk/ainet/backend/api/kernel/Q5_1MatmulKernel; public fun matmulQ6K ()Lsk/ainet/backend/api/kernel/Q6KMatmulKernel; public fun matmulQ8_0 ()Lsk/ainet/backend/api/kernel/Q8_0MatmulKernel; + public fun normFp32 ()Lsk/ainet/backend/api/kernel/Fp32NormKernel; + public fun softmaxFp32 ()Lsk/ainet/backend/api/kernel/Fp32SoftmaxKernel; public fun supports (Ljava/lang/String;Ljava/util/List;)Z } @@ -337,10 +353,13 @@ public class sk/ainet/exec/tensor/ops/DefaultCpuOpsBase : sk/ainet/context/sched public fun relayoutPackedWeightForKernels (Lsk/ainet/lang/tensor/Tensor;)Lsk/ainet/lang/tensor/Tensor; public fun relu (Lsk/ainet/lang/tensor/Tensor;)Lsk/ainet/lang/tensor/Tensor; protected final fun requireSameDType (Lsk/ainet/lang/tensor/Tensor;Lsk/ainet/lang/tensor/Tensor;)V + protected final fun requireSdpaOperands (Lsk/ainet/lang/tensor/Tensor;Lsk/ainet/lang/tensor/Tensor;Lsk/ainet/lang/tensor/Tensor;II)V public fun reshape (Lsk/ainet/lang/tensor/Tensor;Lsk/ainet/lang/tensor/Shape;)Lsk/ainet/lang/tensor/Tensor; protected final fun rewrapFeedOrderWeight (Lsk/ainet/lang/tensor/Tensor;)Lsk/ainet/lang/tensor/Tensor; public fun rsubScalar (Ljava/lang/Number;Lsk/ainet/lang/tensor/Tensor;)Lsk/ainet/lang/tensor/Tensor; public fun scaledDotProductAttention (Lsk/ainet/lang/tensor/Tensor;Lsk/ainet/lang/tensor/Tensor;Lsk/ainet/lang/tensor/Tensor;Lsk/ainet/lang/tensor/Tensor;FZLsk/ainet/lang/tensor/Tensor;II)Lsk/ainet/lang/tensor/Tensor; + protected final fun sdpaKeyLengths (Lsk/ainet/lang/tensor/Tensor;II)[I + protected final fun sdpaMaskDims (Lsk/ainet/lang/tensor/Tensor;IIIII)[I public fun sigmoid (Lsk/ainet/lang/tensor/Tensor;)Lsk/ainet/lang/tensor/Tensor; public fun sign (Lsk/ainet/lang/tensor/Tensor;)Lsk/ainet/lang/tensor/Tensor; public fun silu (Lsk/ainet/lang/tensor/Tensor;)Lsk/ainet/lang/tensor/Tensor; diff --git a/skainet-backends/skainet-backend-cpu/src/commonMain/kotlin/sk/ainet/exec/tensor/ops/DefaultCpuOps.kt b/skainet-backends/skainet-backend-cpu/src/commonMain/kotlin/sk/ainet/exec/tensor/ops/DefaultCpuOps.kt index f340df8e1..7e648d0e5 100644 --- a/skainet-backends/skainet-backend-cpu/src/commonMain/kotlin/sk/ainet/exec/tensor/ops/DefaultCpuOps.kt +++ b/skainet-backends/skainet-backend-cpu/src/commonMain/kotlin/sk/ainet/exec/tensor/ops/DefaultCpuOps.kt @@ -3825,30 +3825,7 @@ public open class DefaultCpuOpsBase( window: Int, rightContext: Int ): Tensor { - // Expected shapes: [batch, heads, seqQ, headDim] for Q, [batch, heads, seqKV, headDim] for K/V - require(query.rank == 4) { "SDPA: expected rank-4 query, got rank ${query.rank}" } - require(key.rank == 4) { "SDPA: expected rank-4 key, got rank ${key.rank}" } - require(value.rank == 4) { "SDPA: expected rank-4 value, got rank ${value.rank}" } - require(query.shape[0] == key.shape[0] && query.shape[0] == value.shape[0]) { - "SDPA: batch mismatch — Q=${query.shape[0]} K=${key.shape[0]} V=${value.shape[0]}" - } - require(key.shape[1] == value.shape[1]) { - "SDPA: K/V head count mismatch — K=${key.shape[1]} V=${value.shape[1]}" - } - require(key.shape[1] > 0 && query.shape[1] % key.shape[1] == 0) { - "SDPA: Q heads (${query.shape[1]}) must be a multiple of K/V heads (${key.shape[1]}) — " + - "grouped-query attention maps query head h onto K/V head h / (nHeads / nKVHeads)." - } - require(query.shape[3] == key.shape[3]) { - "SDPA: Q head_dim (${query.shape[3]}) does not match K head_dim (${key.shape[3]})" - } - require(query.shape[3] == value.shape[3]) { - "SDPA: Q head_dim (${query.shape[3]}) does not match V head_dim (${value.shape[3]})" - } - require(key.shape[2] == value.shape[2]) { - "SDPA: K seqKV (${key.shape[2]}) does not match V seqKV (${value.shape[2]})" - } - sk.ainet.lang.tensor.ops.AttentionMask.validate(window, rightContext) + requireSdpaOperands(query, key, value, window, rightContext) val batch = query.shape[0] val heads = query.shape[1] @@ -3862,15 +3839,7 @@ public open class DefaultCpuOpsBase( // attended keys (the masked exp underflows to exactly 0 either way) and keeps a fully // masked row finite. `causal` folds into the range on this path. Without lengths or a // window the kernel below runs unchanged. - val lengths: IntArray? = keyLengths?.let { kl -> - require(kl.rank == 1 && kl.shape[0] == batch) { "SDPA: keyLengths must be [batch=$batch], got ${kl.shape}" } - val raw = kl.data.copyToFloatArray() - IntArray(batch) { b -> - val len = raw[b].toInt() - require(len in 0..seqKV) { "SDPA: keyLengths[$b] = $len is outside 0..$seqKV" } - len - } - } + val lengths: IntArray? = sdpaKeyLengths(keyLengths, batch, seqKV) val structured = lengths != null || window > 0 // Grouped-query attention (SKEEP-005 phase 2): K/V are read in place through the // head-group index instead of being tiled to Q's head count upstream. Same loop @@ -3891,16 +3860,7 @@ public open class DefaultCpuOpsBase( // Dense mask broadcast, trailing-aligned like the StableHLO lowering: [b | 1, H | nKV | 1, Sq | 1, Sk | 1]. // (The former `i % size` read batch row 0 for every batch, so a [batch, 1, Sq, Sk] mask with batch > 1 // was silently wrong; shapes that worked — [1, 1, Sq, Sk], [1, 1, 1, Sk], [Sq, Sk] — index identically here.) - val maskDims = mask?.let { m -> - require(m.rank in 1..4) { "SDPA: mask must have rank 1..4, got ${m.rank}" } - val d = IntArray(4) { 1 } - for (i in 0 until m.rank) d[4 - m.rank + i] = m.shape[i] - require(d[0] == 1 || d[0] == batch) { "SDPA: mask batch (${d[0]}) must be 1 or $batch" } - require(d[1] == 1 || d[1] == heads || d[1] == key.shape[1]) { "SDPA: mask heads (${d[1]}) must be 1, ${key.shape[1]} or $heads" } - require(d[2] == 1 || d[2] == seqQ) { "SDPA: mask seqQ (${d[2]}) must be 1 or $seqQ" } - require(d[3] == 1 || d[3] == seqKV) { "SDPA: mask seqKV (${d[3]}) must be 1 or $seqKV" } - d - } + val maskDims = sdpaMaskDims(mask, batch, heads, key.shape[1], seqQ, seqKV) val outBuf = FloatArray(batch * heads * seqQ * headDim) @@ -4027,6 +3987,66 @@ public open class DefaultCpuOpsBase( return newTensor(data, query.dtype, query, key, value) } + // ── scaledDotProductAttention contract, shared with the fused-kernel overrides ───────────────── + + /** The operand and argument contract of [scaledDotProductAttention]; a platform override that hands the op to a fused kernel checks the same. */ + protected fun requireSdpaOperands(query: Tensor<*, *>, key: Tensor<*, *>, value: Tensor<*, *>, window: Int, rightContext: Int) { + // Expected shapes: [batch, heads, seqQ, headDim] for Q, [batch, heads, seqKV, headDim] for K/V + require(query.rank == 4) { "SDPA: expected rank-4 query, got rank ${query.rank}" } + require(key.rank == 4) { "SDPA: expected rank-4 key, got rank ${key.rank}" } + require(value.rank == 4) { "SDPA: expected rank-4 value, got rank ${value.rank}" } + require(query.shape[0] == key.shape[0] && query.shape[0] == value.shape[0]) { + "SDPA: batch mismatch — Q=${query.shape[0]} K=${key.shape[0]} V=${value.shape[0]}" + } + require(key.shape[1] == value.shape[1]) { + "SDPA: K/V head count mismatch — K=${key.shape[1]} V=${value.shape[1]}" + } + require(key.shape[1] > 0 && query.shape[1] % key.shape[1] == 0) { + "SDPA: Q heads (${query.shape[1]}) must be a multiple of K/V heads (${key.shape[1]}) — " + + "grouped-query attention maps query head h onto K/V head h / (nHeads / nKVHeads)." + } + require(query.shape[3] == key.shape[3]) { + "SDPA: Q head_dim (${query.shape[3]}) does not match K head_dim (${key.shape[3]})" + } + require(query.shape[3] == value.shape[3]) { + "SDPA: Q head_dim (${query.shape[3]}) does not match V head_dim (${value.shape[3]})" + } + require(key.shape[2] == value.shape[2]) { + "SDPA: K seqKV (${key.shape[2]}) does not match V seqKV (${value.shape[2]})" + } + sk.ainet.lang.tensor.ops.AttentionMask.validate(window, rightContext) + } + + /** The per-batch key-padding lengths of [scaledDotProductAttention] as ints, validated against [batch] and [seqKV]; `null` for none. */ + protected fun sdpaKeyLengths(keyLengths: Tensor?, batch: Int, seqKV: Int): IntArray? { + return keyLengths?.let { kl -> + require(kl.rank == 1 && kl.shape[0] == batch) { "SDPA: keyLengths must be [batch=$batch], got ${kl.shape}" } + val raw = kl.data.copyToFloatArray() + IntArray(batch) { b -> + val len = raw[b].toInt() + require(len in 0..seqKV) { "SDPA: keyLengths[$b] = $len is outside 0..$seqKV" } + len + } + } + } + + /** + * The dense additive mask's broadcast dims `[batch | 1, heads | kvHeads | 1, seqQ | 1, seqKV | 1]`, + * trailing-aligned like the StableHLO lowering and validated against the operand shapes; `null` for no mask. + */ + protected fun sdpaMaskDims(mask: Tensor<*, *>?, batch: Int, heads: Int, kvHeads: Int, seqQ: Int, seqKV: Int): IntArray? { + return mask?.let { m -> + require(m.rank in 1..4) { "SDPA: mask must have rank 1..4, got ${m.rank}" } + val d = IntArray(4) { 1 } + for (i in 0 until m.rank) d[4 - m.rank + i] = m.shape[i] + require(d[0] == 1 || d[0] == batch) { "SDPA: mask batch (${d[0]}) must be 1 or $batch" } + require(d[1] == 1 || d[1] == heads || d[1] == kvHeads) { "SDPA: mask heads (${d[1]}) must be 1, $kvHeads or $heads" } + require(d[2] == 1 || d[2] == seqQ) { "SDPA: mask seqQ (${d[2]}) must be 1 or $seqQ" } + require(d[3] == 1 || d[3] == seqKV) { "SDPA: mask seqKV (${d[3]}) must be 1 or $seqKV" } + d + } + } + } public class DefaultCpuOps( diff --git a/skainet-backends/skainet-backend-cpu/src/commonTest/kotlin/sk/ainet/exec/harness/M2AcceptanceTest.kt b/skainet-backends/skainet-backend-cpu/src/commonTest/kotlin/sk/ainet/exec/harness/M2AcceptanceTest.kt index de4cbc2d6..a735f2288 100644 --- a/skainet-backends/skainet-backend-cpu/src/commonTest/kotlin/sk/ainet/exec/harness/M2AcceptanceTest.kt +++ b/skainet-backends/skainet-backend-cpu/src/commonTest/kotlin/sk/ainet/exec/harness/M2AcceptanceTest.kt @@ -102,8 +102,9 @@ class M2AcceptanceTest { println("[m2] faults: 4 steps → $short, ${steps * 4} steps → $long (after: $afterLong)") if (short == null || long == null) { - // a browser or Wasm host cannot answer; the structural assertions above still hold - assertTrue(afterLong.rssBytes == null, "a platform that knows its RSS should know its faults too") + // A browser or Wasm host has no fault counters: it reports what the engine holds for the + // tab (JS heap, linear memory, sidecar bytes) but nothing about paging, so RSS may be + // known while the faults are not. The structural assertions above still hold. return } assertTrue(short >= 0 && long >= 0, "fault counters only move forward: $short, $long") diff --git a/skainet-backends/skainet-backend-cpu/src/jvmMain/kotlin/sk/ainet/exec/tensor/ops/DefaultCpuOpsJvm.kt b/skainet-backends/skainet-backend-cpu/src/jvmMain/kotlin/sk/ainet/exec/tensor/ops/DefaultCpuOpsJvm.kt index 8fe1cf911..4e592bba3 100644 --- a/skainet-backends/skainet-backend-cpu/src/jvmMain/kotlin/sk/ainet/exec/tensor/ops/DefaultCpuOpsJvm.kt +++ b/skainet-backends/skainet-backend-cpu/src/jvmMain/kotlin/sk/ainet/exec/tensor/ops/DefaultCpuOpsJvm.kt @@ -6,7 +6,13 @@ import jdk.incubator.vector.VectorOperators import sk.ainet.backend.api.kernel.Bf16MatmulKernel import sk.ainet.backend.api.kernel.Fp16MatmulKernel import sk.ainet.backend.api.kernel.Fp32MatmulKernel +import sk.ainet.backend.api.kernel.Fp32MemSegMatmulWtKernel +import sk.ainet.backend.api.kernel.Fp32AttentionKernel +import sk.ainet.backend.api.kernel.Fp32NormKernel +import sk.ainet.backend.api.kernel.Fp32SoftmaxKernel +import sk.ainet.backend.api.kernel.KernelProvider import sk.ainet.backend.api.kernel.KernelRegistry +import sk.ainet.backend.api.kernel.MemSegKernelProvider import sk.ainet.backend.api.kernel.KernelServiceLoader import sk.ainet.backend.api.kernel.KernelStrictness import sk.ainet.backend.api.kernel.Q4KMatmulKernel @@ -45,7 +51,7 @@ import kotlin.math.max internal class DefaultCpuOpsJvm( dataFactory: TensorDataFactory, schedule: sk.ainet.context.schedule.Schedule = sk.ainet.context.schedule.Schedule.Sequential, -) : DefaultCpuOpsBase(dataFactory, schedule) { +) : DefaultCpuOpsBase(dataFactory, schedule), sk.ainet.lang.tensor.ops.FusedNormOps { override fun withSchedule(schedule: sk.ainet.context.schedule.Schedule): sk.ainet.lang.tensor.ops.TensorOps = if (schedule === this.schedule) this else DefaultCpuOpsJvm(dataFactory, schedule) @@ -196,12 +202,19 @@ internal class DefaultCpuOpsJvm( * tensor nothing could decode correctly (#1124). Here the kernels do exist, so the relayout * pays: once per weight, cached by the identity of its bytes. */ - /** The JVM keeps the cached transpose: its vectorised `[k, n]` kernels are the fast path (SKEEP-007 §T0.2). */ + /** + * The common row-dot is not this tier's direct path: a dense weight goes to the native `x · Wᵀ` + * kernel ([matmulDenseWeightTransposedNative]) when the FFM provider is loaded, and to the cached + * transpose plus the vectorised `[k, n]` kernels otherwise (SKEEP-007 §T0.2). + */ override fun preferDirectWeightTransposed(): Boolean = false @Suppress("UNCHECKED_CAST") override fun matmulWeightTransposed(x: Tensor, weight: Tensor): Tensor { if (weight.shape.rank != 2 || !isHeapPackedWeightForJvm(weight.data)) { + // Dense FP32 `[out, in]` weight with a native direct kernel: multiplied in place, no + // transposed copy made or cached (the weight is made resident off-heap once instead). + matmulDenseWeightTransposedNative(x, weight)?.let { return it } // Packed weights OUTSIDE the JVM relayout list (the ternary formats, #1136) go to the // common views→KernelDispatch path — whose kernels and reference are commonMain and // cannot read a MemorySegment-backed activation (SKEEP-004 forbids element access over @@ -265,6 +278,225 @@ internal class DefaultCpuOpsJvm( private val prepackedWeightsJvm: MutableList>> = mutableListOf() private val PREPACK_CACHE_LIMIT_JVM: Int = 64 + /** + * The direct `x · Wᵀ` kernel over an off-heap weight (`skainet_fp32_matmul_wt` through FFM), from + * the highest-priority available provider that carries one; `null` on a JVM without the native + * library, where the cached-transpose path below still serves. + */ + private val fp32WtKernel: Fp32MemSegMatmulWtKernel? by lazy { + ensureKernelProviders() + KernelRegistry.providers().asSequence() + .filter { it.isAvailable() } + .mapNotNull { (it as? MemSegKernelProvider)?.matmulFp32WtMemSeg() } + .firstOrNull() + } + + /** + * Dense weights made resident off-heap for [fp32WtKernel], keyed by the identity of their + * `TensorData` — the counterpart of the transposed-weight cache this path replaces, holding the + * same bytes untransposed where a downcall can read them without a per-call copy. Bounded like + * that cache; an evicted weight is simply copied again on its next use. Segments come from + * `Arena.ofAuto()`, so a dropped entry is reclaimed with the tensor that owned it. + */ + private val residentDenseWeights: MutableList> = mutableListOf() + private val RESIDENT_DENSE_CACHE_LIMIT: Int = 256 + + /** + * `x · Wᵀ` for a dense FP32 `[out, in]` weight through the native direct kernel — the JVM-side + * half of SKEEP-007 §T0.2: no transposed copy of the weight is made, the weight is read where it + * lies (a segment-backed weight as is, a heap one after a single off-heap copy), and the C kernel + * threads over output rows. `null` when the kernel is not available or the operands are not + * dense FP32 the kernel can take, in which case the caller continues to the cached transpose. + */ + @Suppress("UNCHECKED_CAST") + private fun matmulDenseWeightTransposedNative(x: Tensor, weight: Tensor): Tensor? { + if (weight.shape.rank != 2 || x.rank < 1 || x.dtype != FP32::class || weight.dtype != FP32::class) return null + val kernel = fp32WtKernel ?: return null + val n = weight.shape[0] + val k = weight.shape[1] + if (x.shape.dimensions.last() != k) return null + val resident = residentDenseWeight(weight, n.toLong() * k) ?: return null + val activation = heapActivation(x) ?: return null + val m = if (k == 0) 0 else x.shape.volume / k + val out = FloatArray(m * n) + if (m > 0 && n > 0) kernel.matmulWt(activation.first, activation.second, resident.first, resident.second, out, 0, m, n, k) + val outShape = if (x.rank == 1) Shape(n) else Shape(*(x.shape.dimensions.copyOf(x.rank - 1) + n)) + val outData = dataFactory.adoptFloatArray(outShape, x.dtype, out) as TensorData + return newTensor(outData, x.dtype, x, weight) + } + + /** + * The activation's floats and their offset on the heap; a dense segment-backed activation is + * copied (it is `m × k`, small). `MemorySegmentTensorData` specifically, not the marker + * interface: the Q4/Q8 segment types implement that too and hold packed bytes, not floats. + */ + private fun heapActivation(x: Tensor<*, *>): Pair? = when (val d = x.data) { + is FloatArrayTensorData<*> -> d.buffer to 0 + is MemorySegmentTensorData<*> -> { + val count = x.shape.volume + val arr = FloatArray(count) + java.lang.foreign.MemorySegment.copy(d.segment, java.lang.foreign.ValueLayout.JAVA_FLOAT, d.segmentByteOffset, arr, 0, count) + arr to 0 + } + else -> null + } + + /** The weight's `floats` floats as an off-heap segment and byte offset: its own segment, or the resident copy of its heap array. */ + private fun residentDenseWeight(weight: Tensor<*, *>, floats: Long): Pair? { + val bytes = floats * java.lang.Float.BYTES + when (val d = weight.data) { + is MemorySegmentTensorData<*> -> { + if (d.segmentByteOffset + bytes > d.segment.byteSize()) return null + return d.segment to d.segmentByteOffset + } + is FloatArrayTensorData<*> -> { + if (d.buffer.size.toLong() < floats) return null + synchronized(residentDenseWeights) { + residentDenseWeights.firstOrNull { it.first === d }?.let { return it.second to 0L } + val seg = Arena.ofAuto().allocate(maxOf(bytes, 4L), 64L) + if (floats > 0) { + java.lang.foreign.MemorySegment.copy(d.buffer, 0, seg, java.lang.foreign.ValueLayout.JAVA_FLOAT, 0L, floats.toInt()) + } + if (residentDenseWeights.size >= RESIDENT_DENSE_CACHE_LIMIT) residentDenseWeights.removeAt(0) + residentDenseWeights.add(Pair(d, seg)) + return seg to 0L + } + } + else -> return null + } + } + + // ── fused row kernels (SKEEP-007 fused ops): softmax, the normalisations, attention ────────── + + /** The highest-priority available provider's kernel from [pick], or `null` (the common path serves). */ + private fun fusedKernel(pick: (KernelProvider) -> K?): K? { + ensureKernelProviders() + return KernelRegistry.providers().asSequence().filter { it.isAvailable() }.mapNotNull(pick).firstOrNull() + } + + private val softmaxKernel: Fp32SoftmaxKernel? by lazy { fusedKernel { it.softmaxFp32() } } + private val normKernel: Fp32NormKernel? by lazy { fusedKernel { it.normFp32() } } + private val attentionKernel: Fp32AttentionKernel? by lazy { fusedKernel { it.attentionFp32() } } + + /** + * Below this many elements a row kernel is not worth the downcall and its two staging copies: + * the common loop over a few hundred floats finishes before the segments are allocated. + */ + private val FUSED_MIN_ELEMENTS: Int = 1024 + + /** A dense FP32 tensor's floats and the offset of element 0: the array itself, a slab window, or a copy of a segment. */ + private fun denseWindow(t: Tensor<*, *>): Pair? = when (val d = t.data) { + is FloatArrayTensorData<*> -> d.buffer to 0 + is sk.ainet.lang.tensor.data.StorageFloatTensorData<*> -> { + val s = d.storage + s.checkAlive() + s.floats!! to s.arrayOffset + } + is MemorySegmentTensorData<*> -> heapActivation(t) + else -> null + } + + /** A dense FP32 tensor as one whole array from index 0 — its own buffer when it is one, a copy otherwise. */ + private fun wholeFloats(t: Tensor<*, *>): FloatArray = when (val d = t.data) { + is FloatArrayTensorData<*> -> if (d.buffer.size == t.shape.volume) d.buffer else d.buffer.copyOf(t.shape.volume) + else -> t.data.copyToFloatArray() + } + + private fun softmaxNative(tensor: Tensor, dim: Int, log: Boolean): Tensor? { + val kernel = softmaxKernel ?: return null + if (tensor.dtype != FP32::class) return null + val rank = tensor.rank + if (rank == 0) return null + val nd = if (dim < 0) dim + rank else dim + if (nd != rank - 1) return null + val cols = tensor.shape[nd] + val n = tensor.shape.volume + if (cols <= 0 || n < FUSED_MIN_ELEMENTS) return null + val src = denseWindow(tensor) ?: return null + val out = FloatArray(n) + kernel.softmaxRows(src.first, src.second, out, 0, n / cols, cols, log) + return floatResult(tensor.shape, tensor.dtype, out, tensor) + } + + override fun softmax(tensor: Tensor, dim: Int): Tensor = + softmaxNative(tensor, dim, log = false) ?: super.softmax(tensor, dim) + + override fun logSoftmax(tensor: Tensor, dim: Int): Tensor = + softmaxNative(tensor, dim, log = true) ?: super.logSoftmax(tensor, dim) + + /** `cols` values of a norm parameter as one array, or `null` when its size does not match. */ + private fun normParam(t: Tensor<*, *>, cols: Int): FloatArray? { + if (t.shape.volume != cols) return null + return wholeFloats(t) + } + + private fun normNative( + input: Tensor, gamma: Tensor?, beta: Tensor?, eps: Float, normalizedRank: Int, rms: Boolean, + ): Tensor? { + val kernel = normKernel ?: return null + if (input.dtype != FP32::class) return null + val rank = input.rank + if (normalizedRank !in 1..rank) return null + var cols = 1 + for (i in rank - normalizedRank until rank) cols *= input.shape[i] + val n = input.shape.volume + if (cols == 0 || n == 0) return null + val src = denseWindow(input) ?: return null + val g = gamma?.let { normParam(it, cols) ?: return null } + val b = beta?.let { normParam(it, cols) ?: return null } + val out = FloatArray(n) + if (rms) kernel.rmsNorm(src.first, src.second, g, out, 0, n / cols, cols, eps) + else kernel.layerNorm(src.first, src.second, g, b, out, 0, n / cols, cols, eps) + return floatResult(input.shape, input.dtype, out, input) + } + + override fun layerNorm(input: Tensor, gamma: Tensor?, beta: Tensor?, eps: Float, normalizedRank: Int): Tensor? = + normNative(input, gamma, beta, eps, normalizedRank, rms = false) + + override fun rmsNorm(input: Tensor, gamma: Tensor?, eps: Float, normalizedRank: Int): Tensor? = + normNative(input, gamma, null, eps, normalizedRank, rms = true) + + /** + * Attention through the fused native kernel when the FFM provider carries it: the same operand + * contract and mask rules as the common implementation (which validates them here), the + * `q · k` dot products vectorised, rows threaded in the native row pool rather than through the + * schedule. Without the kernel, or for non-FP32 operands, the common implementation serves. + */ + override fun scaledDotProductAttention( + query: Tensor, + key: Tensor, + value: Tensor, + mask: Tensor?, + scale: Float, + causal: Boolean, + keyLengths: Tensor?, + window: Int, + rightContext: Int, + ): Tensor { + val kernel = attentionKernel + if (kernel == null || query.dtype != FP32::class || key.dtype != FP32::class || value.dtype != FP32::class) { + return super.scaledDotProductAttention(query, key, value, mask, scale, causal, keyLengths, window, rightContext) + } + requireSdpaOperands(query, key, value, window, rightContext) + val batch = query.shape[0] + val heads = query.shape[1] + val seqQ = query.shape[2] + val headDim = query.shape[3] + val kvHeads = key.shape[1] + val seqKV = key.shape[2] + val lengths = sdpaKeyLengths(keyLengths, batch, seqKV) + val maskDims = sdpaMaskDims(mask, batch, heads, kvHeads, seqQ, seqKV) + val effectiveScale = if (scale == 0f) (1.0 / kotlin.math.sqrt(headDim.toDouble())).toFloat() else scale + val out = FloatArray(batch * heads * seqQ * headDim) + kernel.sdpa( + wholeFloats(query), wholeFloats(key), wholeFloats(value), out, + batch, heads, kvHeads, seqQ, seqKV, headDim, + effectiveScale, causal, window, rightContext, + lengths, mask?.let { wholeFloats(it) }, maskDims, + ) + return floatResult(Shape(batch, heads, seqQ, headDim), query.dtype, out, query, key, value) + } + /** * Below this many multiply-accumulates a dense FP32 matmul is faster done directly than handed * to the tile-blocked SPI kernel, whose per-call setup dominates at these sizes. Sized to cover diff --git a/skainet-backends/skainet-backend-native-cpu/native/CMakeLists.txt b/skainet-backends/skainet-backend-native-cpu/native/CMakeLists.txt index b85ed75a3..80daab3c4 100644 --- a/skainet-backends/skainet-backend-native-cpu/native/CMakeLists.txt +++ b/skainet-backends/skainet-backend-native-cpu/native/CMakeLists.txt @@ -26,6 +26,9 @@ set(SKAINET_KERNEL_SOURCES src/q5_0_matmul.c src/q5_1_matmul.c src/skainet_ternary_f32.c + src/f32_ops.c + src/f32_fused.c + src/f32_conv.c ) # Vendored NeoGPU ternary LUT kernel (MIT, verbatim — src/vendor/neogpu/). diff --git a/skainet-backends/skainet-backend-native-cpu/native/include/skainet_kernels.h b/skainet-backends/skainet-backend-native-cpu/native/include/skainet_kernels.h index 60663bfc9..cbfbbf808 100644 --- a/skainet-backends/skainet-backend-native-cpu/native/include/skainet_kernels.h +++ b/skainet-backends/skainet-backend-native-cpu/native/include/skainet_kernels.h @@ -426,6 +426,10 @@ SKAINET_API void skainet_q5_1_matmul( * unpacking, after which `sdot` accumulates sixteen products per instruction * where the core has ARMv8.2 dotprod. * + * The Emscripten sidecar runs the same block in SIMD128 (`i32x4.dot_i16x8_s` on + * widened halves). Output rows are split over the shared row pool once there + * are enough of them (#1195 pattern); bit-identical to a single thread. + * * @param activation int8 codes of one token, `input_dim` of them * @param activation_scale the token's absmax scale (`absmax / 127`) * @param weight canonical TQ2_0 blocks, row-major per output row @@ -454,9 +458,10 @@ SKAINET_API void skainet_bitnet_gemv_tq2_0( * input_dim must be a multiple of 4. Unlike the int8 `bitnet_gemv` path * there is no activation quantization: results are exact. Backed by the * vendored NeoGPU LUT kernel (baseline NEON, no dotprod — the fast path for - * Cortex-A72/Pi-4 class cores); it threads internally with pthreads once - * output_dim >= 512. A portable scalar build stands in where the vendored - * file cannot compile (MSVC). + * Cortex-A72/Pi-4 class cores), driven from the shared row pool. Where the + * vendored file cannot compile (MSVC, the Emscripten sidecar) a portable + * mirror with the same compile-time LUT stands in — scalar on MSVC, SIMD128 + * in the sidecar — threaded over output rows through the same pool. */ SKAINET_API void skainet_ternary_f32_gemv( const float* input, int32_t input_offset, @@ -477,7 +482,9 @@ SKAINET_API void skainet_ternary_f32_gemv( * plane_stride_bytes == output_dim * input_dim / 4. row_scale holds raw * little-endian IEEE binary16 bit patterns, one per output row (sign * ignored — encoders store max|row| >= 0). input_dim must be a multiple - * of 4. Threads internally with pthreads at any output_dim. + * of 4. The vendored kernel threads internally with pthreads at any + * output_dim; the portable mirror (MSVC, Emscripten) threads over output + * rows through the shared row pool. */ SKAINET_API void skainet_ternary_lmhead_stage1( const float* input, int32_t input_offset, @@ -503,6 +510,94 @@ SKAINET_API void skainet_fp32_matmul_wt( int32_t m, int32_t n, int32_t k ); +/* + * Dense FP32 elementwise, activation and reduction kernels (SKEEP-007 resident + * activations; see src/f32_ops.c for the contracts). Operands are contiguous. + * + * skainet_f32_binary: out[i] = a[ia] op b[ib], op ∈ {0 add, 1 sub, 2 mul, 3 div}; + * a_mode / b_mode ∈ {0 full (ia = i), 1 scalar (ia = 0), 2 bias (ia = i % inner)}. + * skainet_f32_unary: out[i] = f(in[i]), f ∈ {0 relu, 1 sigmoid, 2 silu, + * 3 gelu (tanh form), 4 gelu (erf form), 5 erf, 6 tanh}. + * skainet_f32_reduce_all: *out = Σ in[0..n) (/ n when mean != 0). + * skainet_f32_reduce_axis: out[o*inner + x] = Σ_k in[(o*red + k)*inner + x] + * (/ red when mean != 0). + * + * Threaded over chunks of the output through the shared row pool; per-element + * results do not depend on the partition. + */ +SKAINET_API void skainet_f32_binary( + int32_t op, + const float* a, int32_t a_mode, + const float* b, int32_t b_mode, + float* out, int32_t n, int32_t inner +); + +SKAINET_API void skainet_f32_unary(int32_t op, const float* in, float* out, int32_t n); + +SKAINET_API void skainet_f32_reduce_all(const float* in, int32_t n, int32_t mean, float* out); + +SKAINET_API void skainet_f32_reduce_axis( + const float* in, int32_t outer, int32_t red, int32_t inner, int32_t mean, float* out +); + +/* + * Fused dense FP32 row kernels (SKEEP-007 resident activations, step 3; see + * src/f32_fused.c for the contracts). Operands are contiguous, row-major. + * + * skainet_f32_softmax_rows: out[r, :] = softmax(in[r, :]) per row of `cols`, + * or the log-softmax when log_form != 0. + * skainet_f32_layer_norm: out[r, c] = (in[r, c] - mean_r) / sqrt(var_r + eps) + * · gamma[c] + beta[c]; gamma / beta may be NULL. + * skainet_f32_rms_norm: out[r, c] = in[r, c] / sqrt(mean(in[r, :]²) + eps) + * · gamma[c]; gamma may be NULL. + * skainet_f32_sdpa: softmax(q · kᵀ · scale + rules + mask) · v over + * q [batch, heads, seq_q, head_dim], k / v [batch, kv_heads, seq_kv, head_dim] + * (grouped-query heads), with the structural rules of the ops layer's + * AttentionMask (causal, sliding window with right_context, per-batch key + * lengths — NULL for none) and an optional dense additive mask broadcast + * over the mask_* dims (NULL for none). `scale` is applied as given. + * + * Threaded over rows (query rows for attention) through the shared row pool. + */ +SKAINET_API void skainet_f32_softmax_rows(const float* in, float* out, int32_t rows, int32_t cols, int32_t log_form); + +SKAINET_API void skainet_f32_layer_norm( + const float* in, const float* gamma, const float* beta, float* out, + int32_t rows, int32_t cols, float eps +); + +SKAINET_API void skainet_f32_rms_norm( + const float* in, const float* gamma, float* out, int32_t rows, int32_t cols, float eps +); + +SKAINET_API void skainet_f32_sdpa( + const float* q, const float* k, const float* v, float* out, + int32_t batch, int32_t heads, int32_t kv_heads, + int32_t seq_q, int32_t seq_kv, int32_t head_dim, + float scale, int32_t causal, int32_t window, int32_t right_context, + const int32_t* lengths, + const float* mask, int32_t mask_b, int32_t mask_h, int32_t mask_q, int32_t mask_k +); + +/* + * Dense FP32 convolution as im2col + GEMM (see src/f32_conv.c for the contract). + * One kernel for conv1d/2d/3d over three spatial dims (depth, height, width; a + * lower rank sets the leading ones to the identity geometry). Layouts are the + * ops layer's: in [n, c_in, d, h, w], w [c_out, c_in / groups, k_d, k_h, k_w], + * bias [c_out] or NULL, out [n, c_out, out_d, out_h, out_w]. `geo` holds + * SKAINET_CONV_GEO_INTS int32 values: + * n, c_in, c_out, groups, in_d, in_h, in_w, k_d, k_h, k_w, s_d, s_h, s_w, + * p_d, p_h, p_w, d_d, d_h, d_w, out_d, out_h, out_w. + * `col` is caller scratch of (out_d · out_h · out_w) × ((c_in / groups) · k_d · k_h · k_w) + * floats. Gather and GEMM thread over patches through the shared row pool; each + * output accumulates its products in ascending k. + */ +#define SKAINET_CONV_GEO_INTS 22 + +SKAINET_API void skainet_f32_conv( + const float* in, const float* w, const float* bias, float* out, float* col, const int32_t* geo +); + #ifdef __cplusplus } #endif diff --git a/skainet-backends/skainet-backend-native-cpu/native/include/skainet_simd.h b/skainet-backends/skainet-backend-native-cpu/native/include/skainet_simd.h index b90d3c1d7..be4e86e52 100644 --- a/skainet-backends/skainet-backend-native-cpu/native/include/skainet_simd.h +++ b/skainet-backends/skainet-backend-native-cpu/native/include/skainet_simd.h @@ -166,4 +166,24 @@ static inline void skainet_wasm_q5_fifth_bits(uint32_t qh32, v128_t* fifth_lo, v } #endif +/* + * Transcendentals called from element loops go through these opaque wrappers. GCC with + * -ffast-math and glibc's math-vector.h otherwise vectorises such a loop into libmvec's + * _ZGVbN4v_expf / _ZGVbN4v_tanhf / _ZGVbN2v_erf, which the shared library is not linked + * against (and which older glibc and aarch64 builds lack): the symbol fails to resolve at + * the first call and the loader terminates the process. Under -ffast-math GCC also narrows + * (float) erf((double) x) to erff. A non-inlined scalar call per element costs nothing next + * to the function itself; clang and MSVC never emit the vector variants, so there the + * wrappers inline away. + */ +#include +#if defined(__GNUC__) && !defined(__clang__) +# define SKAINET_LIBM_FN static __attribute__((noinline, unused)) +#else +# define SKAINET_LIBM_FN static inline +#endif +SKAINET_LIBM_FN float skainet_expf(float x) { return expf(x); } +SKAINET_LIBM_FN float skainet_tanhf(float x) { return tanhf(x); } +SKAINET_LIBM_FN double skainet_erf(double x) { return erf(x); } + #endif /* SKAINET_SIMD_H */ diff --git a/skainet-backends/skainet-backend-native-cpu/native/src/bitnet_gemv.c b/skainet-backends/skainet-backend-native-cpu/native/src/bitnet_gemv.c index e91442875..2f1878179 100644 --- a/skainet-backends/skainet-backend-native-cpu/native/src/bitnet_gemv.c +++ b/skainet-backends/skainet-backend-native-cpu/native/src/bitnet_gemv.c @@ -1,5 +1,6 @@ #include "skainet_kernels.h" #include "skainet_simd.h" +#include "skainet_row_threads.h" #include #include @@ -31,6 +32,17 @@ * than it saves, and a checkpoint that is going to be run through this kernel * ships as TQ2_0. TQ1_0's base-3 packing is likewise scalar — it is a storage * format, not a runtime one. + * + * The Emscripten sidecar has the same block body in SIMD128: the unpacking is + * identical (shift, mask) and the sixteen int8 products go through + * `i32x4.dot_i16x8_s` on widened halves, the stand-in for `sdot` every other + * int8 kernel in the sidecar uses. + * + * Output rows are independent, so they are split over the shared row pool + * (skainet_row_threads.h) when there are enough of them — the mt sidecar's + * pthreads in a browser, the native pool elsewhere. Per-row arithmetic never + * changes with the partition, so the result is bit-identical to a single + * thread's. */ #define SKAINET_TQ2_0_BLOCK_SIZE 256 @@ -160,7 +172,76 @@ static inline int32_t skainet_tq2_0_block_neon( return vaddvq_s32(acc); } -#endif /* SKAINET_HAVE_NEON */ +#define skainet_tq2_0_block_simd skainet_tq2_0_block_neon +#elif defined(SKAINET_HAVE_WASM_SIMD128) +/* + * The NEON block in SIMD128: the same two chunks × four bit pairs, codes kept in + * {0,1,2} with the Σ a bias removed by the caller. `wasm_u8x16_shr` takes a + * runtime shift, so the strips can loop over the bit pair. + */ +static inline int32_t skainet_tq2_0_block_wasm( + const uint8_t* SKAINET_RESTRICT qs, + const int8_t* SKAINET_RESTRICT act) { + const v128_t mask = wasm_u8x16_splat(3); + v128_t acc = wasm_i32x4_splat(0); + for (int32_t c = 0; c < 2; ++c) { + const v128_t lo = wasm_v128_load(qs + c * 32); + const v128_t hi = wasm_v128_load(qs + c * 32 + 16); + for (int32_t l = 0; l < 4; ++l) { + const v128_t wlo = wasm_v128_and(wasm_u8x16_shr(lo, (uint32_t) (2 * l)), mask); + const v128_t whi = wasm_v128_and(wasm_u8x16_shr(hi, (uint32_t) (2 * l)), mask); + const int8_t* a = act + c * 128 + l * 32; + acc = wasm_i32x4_add(acc, skainet_wasm_dot_u8x16_i8x16(wlo, wasm_v128_load(a))); + acc = wasm_i32x4_add(acc, skainet_wasm_dot_u8x16_i8x16(whi, wasm_v128_load(a + 16))); + } + } + return skainet_wasm_hadd_i32(acc); +} +#define skainet_tq2_0_block_simd skainet_tq2_0_block_wasm +#endif /* SKAINET_HAVE_NEON / SKAINET_HAVE_WASM_SIMD128 */ + +typedef struct { + const int8_t* act; + const uint8_t* w; + const int32_t* activation_sums; + float* out; + float activation_scale; + int32_t input_dim; + int32_t blocks_per_row; +} skainet_bitnet_gemv_job; + +/* Output rows [o_start, o_end): every block of the row, scaled and summed. */ +static void skainet_bitnet_gemv_rows(void* ctx, int32_t o_start, int32_t o_end) { + const skainet_bitnet_gemv_job* j = (const skainet_bitnet_gemv_job*) ctx; + const int8_t* act = j->act; + const int32_t input_dim = j->input_dim; + const int32_t blocks_per_row = j->blocks_per_row; + for (int32_t o = o_start; o < o_end; ++o) { + float sum = 0.0f; + for (int32_t b = 0; b < blocks_per_row; ++b) { + const uint8_t* block = j->w + (size_t)(o * blocks_per_row + b) * SKAINET_TQ2_0_BYTES_PER_BLOCK; + const int32_t first = b * SKAINET_TQ2_0_BLOCK_SIZE; + const int32_t count = (input_dim - first) < SKAINET_TQ2_0_BLOCK_SIZE + ? (input_dim - first) + : SKAINET_TQ2_0_BLOCK_SIZE; + uint16_t scale_bits; + memcpy(&scale_bits, block + 64, sizeof(scale_bits)); + const float d = skainet_bitnet_fp16(scale_bits); + int32_t partial; +#ifdef skainet_tq2_0_block_simd + if (count == SKAINET_TQ2_0_BLOCK_SIZE) { + partial = skainet_tq2_0_block_simd(block, act + first) - j->activation_sums[b]; + } else { + partial = skainet_tq2_0_block_scalar(block, act + first, count); + } +#else + partial = skainet_tq2_0_block_scalar(block, act + first, count); +#endif + sum += (float)partial * d; + } + j->out[o] = sum * j->activation_scale; + } +} SKAINET_API void skainet_bitnet_gemv_tq2_0( const int8_t* SKAINET_RESTRICT activation, int32_t activation_offset, @@ -203,30 +284,14 @@ SKAINET_API void skainet_bitnet_gemv_tq2_0( activation_sums[b] = s; } - for (int32_t o = 0; o < output_dim; ++o) { - float sum = 0.0f; - for (int32_t b = 0; b < blocks_per_row; ++b) { - const uint8_t* block = w + (size_t)(o * blocks_per_row + b) * SKAINET_TQ2_0_BYTES_PER_BLOCK; - const int32_t first = b * SKAINET_TQ2_0_BLOCK_SIZE; - const int32_t count = (input_dim - first) < SKAINET_TQ2_0_BLOCK_SIZE - ? (input_dim - first) - : SKAINET_TQ2_0_BLOCK_SIZE; - uint16_t scale_bits; - memcpy(&scale_bits, block + 64, sizeof(scale_bits)); - const float d = skainet_bitnet_fp16(scale_bits); - int32_t partial; -#if defined(SKAINET_HAVE_NEON) - if (count == SKAINET_TQ2_0_BLOCK_SIZE) { - partial = skainet_tq2_0_block_neon(block, act + first) - activation_sums[b]; - } else { - partial = skainet_tq2_0_block_scalar(block, act + first, count); - } -#else - partial = skainet_tq2_0_block_scalar(block, act + first, count); -#endif - sum += (float)partial * d; - } - output[output_offset + o] = sum * activation_scale; - } + skainet_bitnet_gemv_job job; + job.act = act; + job.w = w; + job.activation_sums = activation_sums; + job.out = output + output_offset; + job.activation_scale = activation_scale; + job.input_dim = input_dim; + job.blocks_per_row = blocks_per_row; + skainet_run_rows(skainet_bitnet_gemv_rows, &job, output_dim); free(heap_sums); } diff --git a/skainet-backends/skainet-backend-native-cpu/native/src/f32_conv.c b/skainet-backends/skainet-backend-native-cpu/native/src/f32_conv.c new file mode 100644 index 000000000..8fa7327cc --- /dev/null +++ b/skainet-backends/skainet-backend-native-cpu/native/src/f32_conv.c @@ -0,0 +1,158 @@ +#include "skainet_kernels.h" +#include "skainet_simd.h" +#include "skainet_row_threads.h" + +#include +#include +#include + +/* + * Dense FP32 convolution as im2col + GEMM (SKEEP-007 resident activations, conv step). + * + * One kernel serves conv1d, conv2d and conv3d: the geometry is three spatial dims (depth, height, + * width) and a lower-rank convolution sets the leading ones to 1 (kernel 1, stride 1, padding 0, + * dilation 1, output 1). Layouts are the ops layer's: input `[n, c_in, d, h, w]`, weight + * `[c_out, c_in / groups, k_d, k_h, k_w]`, bias `[c_out]` or NULL, output `[n, c_out, out_d, out_h, out_w]`. + * + * skainet_f32_conv(in, w, bias, out, col, geo) + * geo is SKAINET_CONV_GEO_INTS int32 values in the order + * n, c_in, c_out, groups, + * in_d, in_h, in_w, k_d, k_h, k_w, s_d, s_h, s_w, p_d, p_h, p_w, d_d, d_h, d_w, + * out_d, out_h, out_w + * with out_x = (in_x + 2 p_x - d_x (k_x - 1) - 1) / s_x + 1 as the caller computed it. + * `col` is caller-provided scratch of patch_count × col_size floats, where + * patch_count = out_d · out_h · out_w and col_size = (c_in / groups) · k_d · k_h · k_w; it is + * reused for every (batch, group) pair, so the caller allocates it once per call. + * + * For each batch and group the receptive fields are gathered patch-major into `col` + * (`col[p, :]` is patch p's c_in_g × k_d × k_h × k_w window, zero where it falls in the padding — + * the layout the JVM's `JvmVectorKernels` im2col produces), and the output block + * `out[oc, p] = bias[oc] + Σ_k w[oc, k] · col[p, k]` is one `skainet_fp32_matmul_wt` call with the + * weight rows as `in` and the patches as `w`: every output is a contiguous dot product over two + * rows, which is the SIMD128 / NEON body that kernel already has, accumulated in ascending k. + * + * Threading: the gather runs over patches and the GEMM over patches through the shared row pool, + * so a 28 × 28 map (784 patches) threads on the mt sidecar and the native pool, while a tiny map + * stays on the caller. Workers write disjoint rows of `col` and disjoint columns of `out`. + */ + +enum { + G_N = 0, G_C_IN, G_C_OUT, G_GROUPS, + G_IN_D, G_IN_H, G_IN_W, + G_K_D, G_K_H, G_K_W, + G_S_D, G_S_H, G_S_W, + G_P_D, G_P_H, G_P_W, + G_D_D, G_D_H, G_D_W, + G_OUT_D, G_OUT_H, G_OUT_W +}; + +typedef struct { + const float* in; /* this batch, this group's first channel */ + float* col; /* patch_count rows of col_size */ + int32_t c_in_g; + int32_t in_d, in_h, in_w; + int32_t k_d, k_h, k_w; + int32_t s_d, s_h, s_w; + int32_t p_d, p_h, p_w; + int32_t d_d, d_h, d_w; + int32_t out_h, out_w; + int32_t col_size; +} skainet_im2col_ctx; + +/* Patches [p0, p1): one row of `col` each. */ +static void skainet_im2col_rows(void* vctx, int32_t p0, int32_t p1) { + const skainet_im2col_ctx* c = (const skainet_im2col_ctx*) vctx; + const int32_t out_hw = c->out_h * c->out_w; + const size_t in_hw = (size_t) c->in_h * c->in_w; + const size_t in_dhw = in_hw * (size_t) c->in_d; + const int32_t k_w = c->k_w; + for (int32_t p = p0; p < p1; ++p) { + const int32_t od = p / out_hw; + const int32_t rem = p - od * out_hw; + const int32_t oh = rem / c->out_w; + const int32_t ow = rem - oh * c->out_w; + const int32_t d0 = od * c->s_d - c->p_d; + const int32_t h0 = oh * c->s_h - c->p_h; + const int32_t w0 = ow * c->s_w - c->p_w; + float* dst = c->col + (size_t) p * c->col_size; + for (int32_t ci = 0; ci < c->c_in_g; ++ci) { + const float* chan = c->in + (size_t) ci * in_dhw; + for (int32_t kd = 0; kd < c->k_d; ++kd) { + const int32_t id = d0 + kd * c->d_d; + const int d_ok = id >= 0 && id < c->in_d; + for (int32_t kh = 0; kh < c->k_h; ++kh) { + const int32_t ih = h0 + kh * c->d_h; + if (!d_ok || ih < 0 || ih >= c->in_h) { + memset(dst, 0, (size_t) k_w * sizeof(float)); + dst += k_w; + continue; + } + const float* row = chan + (size_t) id * in_hw + (size_t) ih * c->in_w; + if (c->d_w == 1) { + /* kw with w0 + kw inside [0, in_w): one contiguous copy, zeros either side. */ + int32_t kw0 = w0 < 0 ? -w0 : 0; + int32_t kw1 = c->in_w - w0; + if (kw1 > k_w) kw1 = k_w; + if (kw1 < kw0) kw1 = kw0; + if (kw0 > 0) memset(dst, 0, (size_t) kw0 * sizeof(float)); + if (kw1 > kw0) memcpy(dst + kw0, row + w0 + kw0, (size_t) (kw1 - kw0) * sizeof(float)); + if (kw1 < k_w) memset(dst + kw1, 0, (size_t) (k_w - kw1) * sizeof(float)); + dst += k_w; + } else { + for (int32_t kw = 0; kw < k_w; ++kw) { + const int32_t iw = w0 + kw * c->d_w; + *dst++ = (iw >= 0 && iw < c->in_w) ? row[iw] : 0.0f; + } + } + } + } + } + } +} + +SKAINET_API void skainet_f32_conv( + const float* in, const float* w, const float* bias, float* out, float* col, const int32_t* geo +) { + const int32_t n = geo[G_N], c_in = geo[G_C_IN], c_out = geo[G_C_OUT], groups = geo[G_GROUPS]; + if (n <= 0 || c_out <= 0 || groups <= 0) return; + const int32_t out_d = geo[G_OUT_D], out_h = geo[G_OUT_H], out_w = geo[G_OUT_W]; + const int32_t patch_count = out_d * out_h * out_w; + if (patch_count <= 0) return; + const int32_t c_in_g = c_in / groups; + const int32_t c_out_g = c_out / groups; + const int32_t col_size = c_in_g * geo[G_K_D] * geo[G_K_H] * geo[G_K_W]; + const size_t in_dhw = (size_t) geo[G_IN_D] * geo[G_IN_H] * geo[G_IN_W]; + const size_t in_batch = (size_t) c_in * in_dhw; + const size_t out_batch = (size_t) c_out * patch_count; + + skainet_im2col_ctx ctx; + ctx.col = col; + ctx.c_in_g = c_in_g; + ctx.in_d = geo[G_IN_D]; ctx.in_h = geo[G_IN_H]; ctx.in_w = geo[G_IN_W]; + ctx.k_d = geo[G_K_D]; ctx.k_h = geo[G_K_H]; ctx.k_w = geo[G_K_W]; + ctx.s_d = geo[G_S_D]; ctx.s_h = geo[G_S_H]; ctx.s_w = geo[G_S_W]; + ctx.p_d = geo[G_P_D]; ctx.p_h = geo[G_P_H]; ctx.p_w = geo[G_P_W]; + ctx.d_d = geo[G_D_D]; ctx.d_h = geo[G_D_H]; ctx.d_w = geo[G_D_W]; + ctx.out_h = out_h; ctx.out_w = out_w; + ctx.col_size = col_size; + + for (int32_t b = 0; b < n; ++b) { + for (int32_t g = 0; g < groups; ++g) { + const float* w_g = w + (size_t) g * c_out_g * col_size; + float* out_bg = out + (size_t) b * out_batch + (size_t) g * c_out_g * patch_count; + if (col_size > 0 && c_out_g > 0) { + ctx.in = in + (size_t) b * in_batch + (size_t) g * c_in_g * in_dhw; + skainet_run_rows(skainet_im2col_rows, &ctx, patch_count); + } + /* k <= 0 zeroes the block inside the kernel (an empty receptive field). */ + skainet_fp32_matmul_wt(w_g, 0, col, 0, out_bg, 0, c_out_g, patch_count, col_size); + if (bias) { + for (int32_t oc = 0; oc < c_out_g; ++oc) { + const float bv = bias[(size_t) g * c_out_g + oc]; + float* row = out_bg + (size_t) oc * patch_count; + for (int32_t p = 0; p < patch_count; ++p) row[p] += bv; + } + } + } + } +} diff --git a/skainet-backends/skainet-backend-native-cpu/native/src/f32_fused.c b/skainet-backends/skainet-backend-native-cpu/native/src/f32_fused.c new file mode 100644 index 000000000..72df08d74 --- /dev/null +++ b/skainet-backends/skainet-backend-native-cpu/native/src/f32_fused.c @@ -0,0 +1,396 @@ +#include "skainet_kernels.h" +#include "skainet_simd.h" +#include "skainet_row_threads.h" + +#include +#include +#include +#include + +/* + * Fused dense FP32 row kernels: softmax, layer norm, RMS norm and scaled dot-product attention + * (SKEEP-007 resident activations, step 3). Each is one pass over its rows with the intermediates + * in registers or a per-worker scratch, where the ops-layer decomposition runs 3 to 12 allocating + * passes (a layer norm is two reductions, two broadcasts, a square, a sqrt and a divide; attention + * is a `[seqQ, seqKV]` score matrix, a mask add, a softmax and a second product). They exist for + * the wasm tier first — Kotlin/Wasm has no SIMD and no shared-memory threads — and are compiled into + * every native build, so the JVM binds the same symbols through FFM and both tiers run one code + * base for these ops. + * + * Contracts (all operands contiguous, row-major): + * + * skainet_f32_softmax_rows(in, out, rows, cols, log_form) + * out[r, :] = softmax(in[r, :]) — max-subtracted, exp, divided by the row sum; with log_form != 0 + * out[r, :] = in[r, :] - (log Σ exp(in[r, :] - max) + max). + * + * skainet_f32_layer_norm(in, gamma, beta, out, rows, cols, eps) + * out[r, c] = (in[r, c] - mean_r) / sqrt(var_r + eps) · gamma[c] + beta[c], with mean and the + * biased variance over the row; gamma and/or beta may be NULL (the affine is skipped). + * + * skainet_f32_rms_norm(in, gamma, out, rows, cols, eps) + * out[r, c] = in[r, c] / sqrt(mean(in[r, :]²) + eps) · gamma[c]; gamma may be NULL. + * + * skainet_f32_sdpa(q, k, v, out, batch, heads, kv_heads, seq_q, seq_kv, head_dim, scale, + * causal, window, right_context, lengths, mask, mask_b, mask_h, mask_q, mask_k) + * out = softmax(q · kᵀ · scale + rules + mask) · v, the ops-layer `scaledDotProductAttention`: + * q is [batch, heads, seq_q, head_dim], k and v are [batch, kv_heads, seq_kv, head_dim] with + * query head h reading K/V head h / (heads / kv_heads) (grouped-query attention), out is + * [batch, heads, seq_q, head_dim]. `scale` is applied as given (the caller resolves the + * 1/sqrt(head_dim) default). The structural rules are the ops layer's AttentionMask: a query at + * absolute position absQ = seq_kv - seq_q + qi attends to keys ki with + * ki <= absQ when causal != 0, + * absQ - window + 1 <= ki <= absQ + right_context when window > 0, + * ki < lengths[b] when lengths != NULL ([batch] int32), + * intersected; a key outside the set scores -1e30 (finite, so a fully masked row is uniform) + * and its score is never computed. `mask` (or NULL) is a dense additive mask broadcast + * trailing-aligned over [batch | 1, heads | kv_heads | 1, seq_q | 1, seq_kv | 1] as the + * mask_* dims say, added to every score. Scores and probabilities are float; the dot products + * accumulate in four lanes where a SIMD body exists, the probability-weighted sum of v + * accumulates per output element in ascending key order (the reference's order). + * + * The exp calls go through skainet_expf (skainet_simd.h): a plain scalar libm call per element, + * never a libmvec vector variant the shared library would fail to resolve. + * + * Threading: rows (a query row of one head for attention) go to the shared row pool + * (skainet_row_threads.h), so a 12-head × 150-token attention threads on the mt sidecar and the JVM + * pool, and a decode step with a handful of rows stays on the caller. Attention needs a + * `seq_kv`-float scratch per worker; it is heap-allocated once per worker range. + */ + +#if defined(SKAINET_HAVE_WASM_SIMD128) +#define SKAINET_FZ_VEC v128_t +#define SKAINET_FZ_LOAD(p) wasm_v128_load(p) +#define SKAINET_FZ_STORE(p, v) wasm_v128_store((p), (v)) +#define SKAINET_FZ_SPLAT(x) wasm_f32x4_splat(x) +#define SKAINET_FZ_ADD wasm_f32x4_add +#define SKAINET_FZ_SUB wasm_f32x4_sub +#define SKAINET_FZ_MUL wasm_f32x4_mul +#define SKAINET_FZ_MAX wasm_f32x4_max +#define SKAINET_FZ_HADD skainet_wasm_hadd_f32 +#define SKAINET_FZ_HAVE_VEC 1 +static inline float skainet_fz_hmax(v128_t v) { + const float a = wasm_f32x4_extract_lane(v, 0), b = wasm_f32x4_extract_lane(v, 1); + const float c = wasm_f32x4_extract_lane(v, 2), d = wasm_f32x4_extract_lane(v, 3); + const float ab = a > b ? a : b, cd = c > d ? c : d; + return ab > cd ? ab : cd; +} +#elif defined(SKAINET_HAVE_NEON) +#define SKAINET_FZ_VEC float32x4_t +#define SKAINET_FZ_LOAD(p) vld1q_f32(p) +#define SKAINET_FZ_STORE(p, v) vst1q_f32((p), (v)) +#define SKAINET_FZ_SPLAT(x) vdupq_n_f32(x) +#define SKAINET_FZ_ADD vaddq_f32 +#define SKAINET_FZ_SUB vsubq_f32 +#define SKAINET_FZ_MUL vmulq_f32 +#define SKAINET_FZ_MAX vmaxq_f32 +#define SKAINET_FZ_HADD skainet_neon_hadd_f32 +#define SKAINET_FZ_HAVE_VEC 1 +static inline float skainet_fz_hmax(float32x4_t v) { return vmaxvq_f32(v); } +#endif + +/* The masked score of the ops layer's AttentionMask.MASKED: finite, so exp underflows to 0. */ +#define SKAINET_SDPA_MASKED (-1.0e30f) + +/* ------------------------------------------------------------ row helpers */ + +/* Σ x[i] over [0, n), four lanes then the tail. */ +static inline float skainet_fz_sum(const float* SKAINET_RESTRICT x, int32_t n) { + float s = 0.0f; + int32_t i = 0; +#ifdef SKAINET_FZ_HAVE_VEC + SKAINET_FZ_VEC acc = SKAINET_FZ_SPLAT(0.0f); + for (; i + 4 <= n; i += 4) acc = SKAINET_FZ_ADD(acc, SKAINET_FZ_LOAD(x + i)); + s = SKAINET_FZ_HADD(acc); +#endif + for (; i < n; ++i) s += x[i]; + return s; +} + +/* Σ (x[i] - c)² over [0, n). */ +static inline float skainet_fz_sum_sq_centered(const float* SKAINET_RESTRICT x, int32_t n, float c) { + float s = 0.0f; + int32_t i = 0; +#ifdef SKAINET_FZ_HAVE_VEC + const SKAINET_FZ_VEC vc = SKAINET_FZ_SPLAT(c); + SKAINET_FZ_VEC acc = SKAINET_FZ_SPLAT(0.0f); + for (; i + 4 <= n; i += 4) { + const SKAINET_FZ_VEC d = SKAINET_FZ_SUB(SKAINET_FZ_LOAD(x + i), vc); + acc = SKAINET_FZ_ADD(acc, SKAINET_FZ_MUL(d, d)); + } + s = SKAINET_FZ_HADD(acc); +#endif + for (; i < n; ++i) { const float d = x[i] - c; s += d * d; } + return s; +} + +/* max x[i] over [0, n), n >= 1. */ +static inline float skainet_fz_max(const float* SKAINET_RESTRICT x, int32_t n) { + float m = x[0]; + int32_t i = 1; +#ifdef SKAINET_FZ_HAVE_VEC + if (n >= 4) { + SKAINET_FZ_VEC vm = SKAINET_FZ_LOAD(x); + for (i = 4; i + 4 <= n; i += 4) vm = SKAINET_FZ_MAX(vm, SKAINET_FZ_LOAD(x + i)); + m = skainet_fz_hmax(vm); + } +#endif + for (; i < n; ++i) if (x[i] > m) m = x[i]; + return m; +} + +/* Σ a[i] · b[i] over [0, n), four accumulators then the tail. */ +static inline float skainet_fz_dot(const float* SKAINET_RESTRICT a, const float* SKAINET_RESTRICT b, int32_t n) { + float s = 0.0f; + int32_t i = 0; +#ifdef SKAINET_FZ_HAVE_VEC + SKAINET_FZ_VEC acc0 = SKAINET_FZ_SPLAT(0.0f), acc1 = acc0, acc2 = acc0, acc3 = acc0; + for (; i + 16 <= n; i += 16) { + acc0 = SKAINET_FZ_ADD(acc0, SKAINET_FZ_MUL(SKAINET_FZ_LOAD(a + i), SKAINET_FZ_LOAD(b + i))); + acc1 = SKAINET_FZ_ADD(acc1, SKAINET_FZ_MUL(SKAINET_FZ_LOAD(a + i + 4), SKAINET_FZ_LOAD(b + i + 4))); + acc2 = SKAINET_FZ_ADD(acc2, SKAINET_FZ_MUL(SKAINET_FZ_LOAD(a + i + 8), SKAINET_FZ_LOAD(b + i + 8))); + acc3 = SKAINET_FZ_ADD(acc3, SKAINET_FZ_MUL(SKAINET_FZ_LOAD(a + i + 12), SKAINET_FZ_LOAD(b + i + 12))); + } + SKAINET_FZ_VEC acc = SKAINET_FZ_ADD(SKAINET_FZ_ADD(acc0, acc1), SKAINET_FZ_ADD(acc2, acc3)); + for (; i + 4 <= n; i += 4) acc = SKAINET_FZ_ADD(acc, SKAINET_FZ_MUL(SKAINET_FZ_LOAD(a + i), SKAINET_FZ_LOAD(b + i))); + s = SKAINET_FZ_HADD(acc); +#endif + for (; i < n; ++i) s += a[i] * b[i]; + return s; +} + +/* out[i] = (x[i] - shift) · scale over [0, n), then · gamma[i] + beta[i] when given. */ +static inline void skainet_fz_affine_row(const float* SKAINET_RESTRICT x, float shift, float scale, + const float* SKAINET_RESTRICT gamma, const float* SKAINET_RESTRICT beta, + float* SKAINET_RESTRICT out, int32_t n) { + int32_t i = 0; +#ifdef SKAINET_FZ_HAVE_VEC + const SKAINET_FZ_VEC vshift = SKAINET_FZ_SPLAT(shift), vscale = SKAINET_FZ_SPLAT(scale); + for (; i + 4 <= n; i += 4) { + SKAINET_FZ_VEC y = SKAINET_FZ_MUL(SKAINET_FZ_SUB(SKAINET_FZ_LOAD(x + i), vshift), vscale); + if (gamma) y = SKAINET_FZ_MUL(y, SKAINET_FZ_LOAD(gamma + i)); + if (beta) y = SKAINET_FZ_ADD(y, SKAINET_FZ_LOAD(beta + i)); + SKAINET_FZ_STORE(out + i, y); + } +#endif + for (; i < n; ++i) { + float y = (x[i] - shift) * scale; + if (gamma) y *= gamma[i]; + if (beta) y += beta[i]; + out[i] = y; + } +} + +/* out[i] += p · v[i] over [0, n). */ +static inline void skainet_fz_axpy(float p, const float* SKAINET_RESTRICT v, float* SKAINET_RESTRICT out, int32_t n) { + int32_t i = 0; +#ifdef SKAINET_FZ_HAVE_VEC + const SKAINET_FZ_VEC vp = SKAINET_FZ_SPLAT(p); + for (; i + 4 <= n; i += 4) SKAINET_FZ_STORE(out + i, SKAINET_FZ_ADD(SKAINET_FZ_LOAD(out + i), SKAINET_FZ_MUL(vp, SKAINET_FZ_LOAD(v + i)))); +#endif + for (; i < n; ++i) out[i] += p * v[i]; +} + +/* Softmax of one row over [0, n) (n >= 1): exp(x - max) / Σ, or the log form. `out` may be `x` (attention runs it in place). */ +static inline void skainet_fz_softmax_row(const float* x, float* out, int32_t n, int32_t log_form) { + const float m = skainet_fz_max(x, n); + float sum = 0.0f; + if (log_form) { + for (int32_t i = 0; i < n; ++i) sum += skainet_expf(x[i] - m); + const float lse = logf(sum) + m; + int32_t i = 0; +#ifdef SKAINET_FZ_HAVE_VEC + const SKAINET_FZ_VEC vl = SKAINET_FZ_SPLAT(lse); + for (; i + 4 <= n; i += 4) SKAINET_FZ_STORE(out + i, SKAINET_FZ_SUB(SKAINET_FZ_LOAD(x + i), vl)); +#endif + for (; i < n; ++i) out[i] = x[i] - lse; + return; + } + for (int32_t i = 0; i < n; ++i) { const float e = skainet_expf(x[i] - m); out[i] = e; sum += e; } + if (sum > 0.0f) { + const float inv = 1.0f / sum; + int32_t i = 0; +#ifdef SKAINET_FZ_HAVE_VEC + const SKAINET_FZ_VEC vi = SKAINET_FZ_SPLAT(inv); + for (; i + 4 <= n; i += 4) SKAINET_FZ_STORE(out + i, SKAINET_FZ_MUL(SKAINET_FZ_LOAD(out + i), vi)); +#endif + for (; i < n; ++i) out[i] *= inv; + } +} + +/* ---------------------------------------------------------------- softmax */ + +typedef struct { + const float* in; + float* out; + int32_t cols; + int32_t log_form; +} skainet_f32_softmax_ctx; + +static void skainet_f32_softmax_rows_fn(void* vctx, int32_t r0, int32_t r1) { + const skainet_f32_softmax_ctx* c = (const skainet_f32_softmax_ctx*) vctx; + for (int32_t r = r0; r < r1; ++r) { + skainet_fz_softmax_row(c->in + (size_t) r * c->cols, c->out + (size_t) r * c->cols, c->cols, c->log_form); + } +} + +SKAINET_API void skainet_f32_softmax_rows(const float* in, float* out, int32_t rows, int32_t cols, int32_t log_form) { + if (rows <= 0 || cols <= 0) return; + skainet_f32_softmax_ctx ctx; + ctx.in = in; ctx.out = out; ctx.cols = cols; ctx.log_form = log_form; + skainet_run_rows(skainet_f32_softmax_rows_fn, &ctx, rows); +} + +/* -------------------------------------------------------------- layer norm */ + +typedef struct { + const float* in; + const float* gamma; + const float* beta; + float* out; + int32_t cols; + float eps; + int32_t rms; +} skainet_f32_norm_ctx; + +static void skainet_f32_norm_rows_fn(void* vctx, int32_t r0, int32_t r1) { + const skainet_f32_norm_ctx* c = (const skainet_f32_norm_ctx*) vctx; + const int32_t n = c->cols; + const float inv_n = 1.0f / (float) n; + for (int32_t r = r0; r < r1; ++r) { + const float* x = c->in + (size_t) r * n; + float* out = c->out + (size_t) r * n; + if (c->rms) { + const float ms = skainet_fz_sum_sq_centered(x, n, 0.0f) * inv_n; + skainet_fz_affine_row(x, 0.0f, 1.0f / sqrtf(ms + c->eps), c->gamma, NULL, out, n); + } else { + const float mean = skainet_fz_sum(x, n) * inv_n; + const float var = skainet_fz_sum_sq_centered(x, n, mean) * inv_n; + skainet_fz_affine_row(x, mean, 1.0f / sqrtf(var + c->eps), c->gamma, c->beta, out, n); + } + } +} + +SKAINET_API void skainet_f32_layer_norm(const float* in, const float* gamma, const float* beta, float* out, + int32_t rows, int32_t cols, float eps) { + if (rows <= 0 || cols <= 0) return; + skainet_f32_norm_ctx ctx; + ctx.in = in; ctx.gamma = gamma; ctx.beta = beta; ctx.out = out; ctx.cols = cols; ctx.eps = eps; ctx.rms = 0; + skainet_run_rows(skainet_f32_norm_rows_fn, &ctx, rows); +} + +SKAINET_API void skainet_f32_rms_norm(const float* in, const float* gamma, float* out, int32_t rows, int32_t cols, float eps) { + if (rows <= 0 || cols <= 0) return; + skainet_f32_norm_ctx ctx; + ctx.in = in; ctx.gamma = gamma; ctx.beta = NULL; ctx.out = out; ctx.cols = cols; ctx.eps = eps; ctx.rms = 1; + skainet_run_rows(skainet_f32_norm_rows_fn, &ctx, rows); +} + +/* --------------------------------------------------------------- attention */ + +typedef struct { + const float* q; + const float* k; + const float* v; + float* out; + int32_t heads, kv_heads, seq_q, seq_kv, head_dim; + float scale; + int32_t causal, window, right_context; + const int32_t* lengths; + const float* mask; + int32_t mask_b, mask_h, mask_q, mask_k; +} skainet_f32_sdpa_ctx; + +/* One query row (b, h, qi): scores into `scores` (seq_kv floats), softmax, then Σ p · v. */ +static void skainet_f32_sdpa_row(const skainet_f32_sdpa_ctx* c, int32_t b, int32_t h, int32_t qi, float* SKAINET_RESTRICT scores) { + const int32_t seq_kv = c->seq_kv, d = c->head_dim; + const int32_t kvh = h / (c->heads / c->kv_heads); + const float* q = c->q + (((size_t) b * c->heads + h) * c->seq_q + qi) * d; + const float* kbase = c->k + ((size_t) b * c->kv_heads + kvh) * seq_kv * d; + const float* vbase = c->v + ((size_t) b * c->kv_heads + kvh) * seq_kv * d; + float* out = c->out + (((size_t) b * c->heads + h) * c->seq_q + qi) * d; + + /* The attended key range [first, end) of this row under the structural rules. */ + const int32_t abs_q = seq_kv - c->seq_q + qi; + int32_t first = 0, end = seq_kv; + if (c->window > 0) { + const int32_t f = abs_q - c->window + 1; + if (f > first) first = f; + const int32_t e = abs_q + c->right_context + 1; + if (e < end) end = e; + } + if (c->causal && abs_q + 1 < end) end = abs_q + 1; + if (c->lengths && c->lengths[b] < end) end = c->lengths[b]; + if (end < 0) end = 0; + if (first > end) first = end; + + /* Scores: q · k · scale inside the range, MASKED outside; never computed outside. */ + if (first > 0 || end < seq_kv) { + for (int32_t ki = 0; ki < first; ++ki) scores[ki] = SKAINET_SDPA_MASKED; + for (int32_t ki = end; ki < seq_kv; ++ki) scores[ki] = SKAINET_SDPA_MASKED; + } + for (int32_t ki = first; ki < end; ++ki) scores[ki] = skainet_fz_dot(q, kbase + (size_t) ki * d, d) * c->scale; + + /* The dense additive mask, broadcast trailing-aligned. */ + int32_t pv_first = first, pv_end = end; + if (c->mask) { + const int32_t bi = c->mask_b == 1 ? 0 : b; + const int32_t hi = c->mask_h == 1 ? 0 : (c->mask_h == c->heads ? h : kvh); + const size_t row = ((size_t) bi * c->mask_h + hi) * c->mask_q + (c->mask_q == 1 ? 0 : qi); + const float* m = c->mask + row * c->mask_k; + if (c->mask_k == 1) { + const float mv = m[0]; + for (int32_t ki = 0; ki < seq_kv; ++ki) scores[ki] += mv; + } else { + for (int32_t ki = 0; ki < seq_kv; ++ki) scores[ki] += m[ki]; + } + /* A mask can raise a key the rules excluded: weigh every key below. */ + pv_first = 0; pv_end = seq_kv; + } + if (first >= end) { + /* Every key masked: a uniform row, as the dense -1e30 mask yields. */ + pv_first = 0; pv_end = seq_kv; + } + + skainet_fz_softmax_row(scores, scores, seq_kv, 0); + + /* out = Σ_ki p[ki] · v[ki, :], ascending ki per output element. Keys the rules excluded have + * p == 0 exactly (exp underflow against a finite max), so skipping them changes nothing. */ + for (int32_t x = 0; x < d; ++x) out[x] = 0.0f; + for (int32_t ki = pv_first; ki < pv_end; ++ki) skainet_fz_axpy(scores[ki], vbase + (size_t) ki * d, out, d); +} + +static void skainet_f32_sdpa_rows_fn(void* vctx, int32_t r0, int32_t r1) { + const skainet_f32_sdpa_ctx* c = (const skainet_f32_sdpa_ctx*) vctx; + float* scores = (float*) malloc((size_t) c->seq_kv * sizeof(float)); + if (!scores) return; + const int32_t per_batch = c->heads * c->seq_q; + for (int32_t r = r0; r < r1; ++r) { + const int32_t b = r / per_batch; + const int32_t rem = r - b * per_batch; + skainet_f32_sdpa_row(c, b, rem / c->seq_q, rem % c->seq_q, scores); + } + free(scores); +} + +SKAINET_API void skainet_f32_sdpa( + const float* q, const float* k, const float* v, float* out, + int32_t batch, int32_t heads, int32_t kv_heads, + int32_t seq_q, int32_t seq_kv, int32_t head_dim, + float scale, int32_t causal, int32_t window, int32_t right_context, + const int32_t* lengths, + const float* mask, int32_t mask_b, int32_t mask_h, int32_t mask_q, int32_t mask_k +) { + if (batch <= 0 || heads <= 0 || kv_heads <= 0 || seq_q <= 0 || head_dim <= 0) return; + const int32_t rows = batch * heads * seq_q; + if (seq_kv <= 0) { + for (int32_t i = 0; i < rows * head_dim; ++i) out[i] = 0.0f; + return; + } + skainet_f32_sdpa_ctx ctx; + ctx.q = q; ctx.k = k; ctx.v = v; ctx.out = out; + ctx.heads = heads; ctx.kv_heads = kv_heads; ctx.seq_q = seq_q; ctx.seq_kv = seq_kv; ctx.head_dim = head_dim; + ctx.scale = scale; ctx.causal = causal; ctx.window = window; ctx.right_context = right_context; + ctx.lengths = lengths; + ctx.mask = mask; ctx.mask_b = mask_b; ctx.mask_h = mask_h; ctx.mask_q = mask_q; ctx.mask_k = mask_k; + skainet_run_rows(skainet_f32_sdpa_rows_fn, &ctx, rows); +} diff --git a/skainet-backends/skainet-backend-native-cpu/native/src/f32_ops.c b/skainet-backends/skainet-backend-native-cpu/native/src/f32_ops.c new file mode 100644 index 000000000..1670bcba6 --- /dev/null +++ b/skainet-backends/skainet-backend-native-cpu/native/src/f32_ops.c @@ -0,0 +1,345 @@ +#include "skainet_kernels.h" +#include "skainet_simd.h" +#include "skainet_row_threads.h" + +#include +#include +#include + +/* + * Dense FP32 elementwise, activation and reduction kernels (SKEEP-007 resident activations). + * + * These exist for the wasm tier first: Kotlin/Wasm has no SIMD and no shared-memory threads, so + * everything between two matmuls of a browser forward pass — bias add, GELU, residual add, the + * reductions of a norm — ran as scalar Kotlin on one thread while the matmuls ran in the SIMD128 + * sidecar. With these exports the activations can stay in the sidecar's linear memory from one + * op to the next (`Storage.WasmLinear`) and only cross the boundary when something outside the + * sidecar needs them. They are compiled into every native build as well, so a JVM/Android/K-N + * binding is a few lines when wanted; the JVM has Vector API bodies for all of them today. + * + * Contracts (all operands contiguous): + * + * skainet_f32_binary(op, a, a_mode, b, b_mode, out, n, inner) + * out[i] = a[ia] (op) b[ib] for i in [0, n), where an operand's index follows its mode: + * SKAINET_F32_FULL ia = i (same shape as the output) + * SKAINET_F32_SCALAR ia = 0 (a single value broadcast over the output) + * SKAINET_F32_BIAS ia = i % inner (a vector over the last dimension, `inner` long) + * — the three broadcast shapes the common ops layer takes on its fast path (same shape, + * scalar, last-dim "bias"); anything else is the caller's generic path. + * + * skainet_f32_unary(op, in, out, n) + * out[i] = f(in[i]) with f one of relu, sigmoid, silu, gelu (tanh form), gelu (erf form), + * erf, tanh. The formulas are those of the Kotlin reference ops, so results agree to the + * last-ulp differences between libm and the Kotlin math functions. The transcendental ones are + * scalar libm calls through the opaque wrappers of skainet_simd.h (no libmvec dependency); + * relu has SIMD bodies. + * + * skainet_f32_reduce_all(in, n, mean, out) *out = Σ in[i], divided by n when mean != 0 + * skainet_f32_reduce_axis(in, outer, red, inner, mean, out) + * out[o * inner + x] = Σ_k in[(o * red + k) * inner + x] (/ red when mean != 0) + * — the (outer, reduced, inner) decomposition of a reduction over one axis. + * + * Threading: the work is split into fixed-size chunks of output elements (rows for the axis + * reduction) and handed to the shared row pool when there are enough of them + * (skainet_row_threads.h), so a 1M-element op uses the mt sidecar's pthreads and a 768-wide bias + * add stays on the caller. Per-element results do not depend on the partition. The full + * reduction accumulates in four lanes, so its rounding differs from a sequential sum in the last + * bits, as the JVM Vector API path's does. + */ + +#define SKAINET_F32_FULL 0 +#define SKAINET_F32_SCALAR 1 +#define SKAINET_F32_BIAS 2 + +#define SKAINET_F32_ADD 0 +#define SKAINET_F32_SUB 1 +#define SKAINET_F32_MUL 2 +#define SKAINET_F32_DIV 3 + +#define SKAINET_F32_RELU 0 +#define SKAINET_F32_SIGMOID 1 +#define SKAINET_F32_SILU 2 +#define SKAINET_F32_GELU_TANH 3 +#define SKAINET_F32_GELU_EXACT 4 +#define SKAINET_F32_ERF 5 +#define SKAINET_F32_TANH 6 + +/* Output elements per pool "row": the pool threads at 512 rows, so 1024 → 512K elements. */ +#define SKAINET_F32_BINARY_CHUNK 1024 +/* The transcendental unaries cost ~20 ns a lane in scalar libm; thread earlier. */ +#define SKAINET_F32_UNARY_CHUNK 256 + +/* ---------------------------------------------------------------- binary */ + +typedef struct { + const float* a; + const float* b; + float* out; + int32_t n, inner; + int32_t op, a_mode, b_mode; +} skainet_f32_binary_ctx; + +/* The two SIMD shapes: lane-wise a[i] (op) b[i], and lane-wise a[i] (op) splat. */ +#if defined(SKAINET_HAVE_WASM_SIMD128) +#define SKAINET_F32_VEC v128_t +#define SKAINET_F32_LOAD(p) wasm_v128_load(p) +#define SKAINET_F32_STORE(p, v) wasm_v128_store((p), (v)) +#define SKAINET_F32_SPLAT(x) wasm_f32x4_splat(x) +#define SKAINET_F32_VADD wasm_f32x4_add +#define SKAINET_F32_VSUB wasm_f32x4_sub +#define SKAINET_F32_VMUL wasm_f32x4_mul +#define SKAINET_F32_VDIV wasm_f32x4_div +#define SKAINET_F32_VMAX wasm_f32x4_max +#define SKAINET_F32_HAVE_VEC 1 +#elif defined(SKAINET_HAVE_NEON) +#define SKAINET_F32_VEC float32x4_t +#define SKAINET_F32_LOAD(p) vld1q_f32(p) +#define SKAINET_F32_STORE(p, v) vst1q_f32((p), (v)) +#define SKAINET_F32_SPLAT(x) vdupq_n_f32(x) +#define SKAINET_F32_VADD vaddq_f32 +#define SKAINET_F32_VSUB vsubq_f32 +#define SKAINET_F32_VMUL vmulq_f32 +#define SKAINET_F32_VDIV vdivq_f32 +#define SKAINET_F32_VMAX vmaxq_f32 +#define SKAINET_F32_HAVE_VEC 1 +#endif + +static inline float skainet_f32_apply(int32_t op, float x, float y) { + switch (op) { + case SKAINET_F32_ADD: return x + y; + case SKAINET_F32_SUB: return x - y; + case SKAINET_F32_MUL: return x * y; + default: return x / y; + } +} + +#ifdef SKAINET_F32_HAVE_VEC +static inline SKAINET_F32_VEC skainet_f32_vapply(int32_t op, SKAINET_F32_VEC x, SKAINET_F32_VEC y) { + switch (op) { + case SKAINET_F32_ADD: return SKAINET_F32_VADD(x, y); + case SKAINET_F32_SUB: return SKAINET_F32_VSUB(x, y); + case SKAINET_F32_MUL: return SKAINET_F32_VMUL(x, y); + default: return SKAINET_F32_VDIV(x, y); + } +} +#endif + +/* out[i0..i1) = a[i0..i1) (op) b[i0..i1): both full. */ +static void skainet_f32_binary_full(int32_t op, const float* SKAINET_RESTRICT a, const float* SKAINET_RESTRICT b, + float* SKAINET_RESTRICT out, int32_t i0, int32_t i1) { + int32_t i = i0; +#ifdef SKAINET_F32_HAVE_VEC + for (; i + 4 <= i1; i += 4) { + SKAINET_F32_STORE(out + i, skainet_f32_vapply(op, SKAINET_F32_LOAD(a + i), SKAINET_F32_LOAD(b + i))); + } +#endif + for (; i < i1; ++i) out[i] = skainet_f32_apply(op, a[i], b[i]); +} + +/* out[i0..i1) = a[i0..i1) (op) s, or s (op) a[i0..i1) when `swap`. */ +static void skainet_f32_binary_scalar(int32_t op, const float* SKAINET_RESTRICT a, float s, int swap, + float* SKAINET_RESTRICT out, int32_t i0, int32_t i1) { + int32_t i = i0; +#ifdef SKAINET_F32_HAVE_VEC + const SKAINET_F32_VEC vs = SKAINET_F32_SPLAT(s); + for (; i + 4 <= i1; i += 4) { + const SKAINET_F32_VEC va = SKAINET_F32_LOAD(a + i); + SKAINET_F32_STORE(out + i, swap ? skainet_f32_vapply(op, vs, va) : skainet_f32_vapply(op, va, vs)); + } +#endif + for (; i < i1; ++i) out[i] = swap ? skainet_f32_apply(op, s, a[i]) : skainet_f32_apply(op, a[i], s); +} + +static void skainet_f32_binary_chunks(void* vctx, int32_t c0, int32_t c1) { + const skainet_f32_binary_ctx* c = (const skainet_f32_binary_ctx*) vctx; + const int32_t i0 = c0 * SKAINET_F32_BINARY_CHUNK; + const int32_t i1 = (c1 * SKAINET_F32_BINARY_CHUNK < c->n) ? c1 * SKAINET_F32_BINARY_CHUNK : c->n; + if (c->a_mode == SKAINET_F32_FULL && c->b_mode == SKAINET_F32_FULL) { + skainet_f32_binary_full(c->op, c->a, c->b, c->out, i0, i1); + return; + } + if (c->a_mode == SKAINET_F32_FULL && c->b_mode == SKAINET_F32_SCALAR) { + skainet_f32_binary_scalar(c->op, c->a, c->b[0], 0, c->out, i0, i1); + return; + } + if (c->a_mode == SKAINET_F32_SCALAR && c->b_mode == SKAINET_F32_FULL) { + skainet_f32_binary_scalar(c->op, c->b, c->a[0], 1, c->out, i0, i1); + return; + } + if (c->a_mode == SKAINET_F32_FULL && c->b_mode == SKAINET_F32_BIAS) { + /* Walk the chunk row by row of the last dimension so the bias is read contiguously. */ + int32_t i = i0; + while (i < i1) { + const int32_t x = i % c->inner; + const int32_t run = (c->inner - x < i1 - i) ? c->inner - x : i1 - i; + skainet_f32_binary_full(c->op, c->a + i, c->b + x, c->out + i, 0, run); + i += run; + } + return; + } + if (c->a_mode == SKAINET_F32_BIAS && c->b_mode == SKAINET_F32_FULL) { + int32_t i = i0; + while (i < i1) { + const int32_t x = i % c->inner; + const int32_t run = (c->inner - x < i1 - i) ? c->inner - x : i1 - i; + skainet_f32_binary_full(c->op, c->a + x, c->b + i, c->out + i, 0, run); + i += run; + } + return; + } + /* Remaining mode pairs (scalar×scalar, bias×bias, scalar×bias): index each side by its mode. */ + for (int32_t i = i0; i < i1; ++i) { + const int32_t ia = c->a_mode == SKAINET_F32_FULL ? i : (c->a_mode == SKAINET_F32_SCALAR ? 0 : i % c->inner); + const int32_t ib = c->b_mode == SKAINET_F32_FULL ? i : (c->b_mode == SKAINET_F32_SCALAR ? 0 : i % c->inner); + c->out[i] = skainet_f32_apply(c->op, c->a[ia], c->b[ib]); + } +} + +SKAINET_API void skainet_f32_binary( + int32_t op, + const float* a, int32_t a_mode, + const float* b, int32_t b_mode, + float* out, int32_t n, int32_t inner +) { + if (n <= 0) return; + skainet_f32_binary_ctx ctx; + ctx.a = a; ctx.b = b; ctx.out = out; + ctx.n = n; + ctx.inner = inner > 0 ? inner : 1; + ctx.op = op; ctx.a_mode = a_mode; ctx.b_mode = b_mode; + skainet_run_rows(skainet_f32_binary_chunks, &ctx, (n + SKAINET_F32_BINARY_CHUNK - 1) / SKAINET_F32_BINARY_CHUNK); +} + +/* ----------------------------------------------------------------- unary */ + +typedef struct { + const float* in; + float* out; + int32_t n; + int32_t op; +} skainet_f32_unary_ctx; + +static inline float skainet_f32_sigmoid(float x) { return 1.0f / (1.0f + skainet_expf(-x)); } + +static void skainet_f32_unary_chunks(void* vctx, int32_t c0, int32_t c1) { + const skainet_f32_unary_ctx* c = (const skainet_f32_unary_ctx*) vctx; + const float* SKAINET_RESTRICT in = c->in; + float* SKAINET_RESTRICT out = c->out; + const int32_t i0 = c0 * SKAINET_F32_UNARY_CHUNK; + const int32_t i1 = (c1 * SKAINET_F32_UNARY_CHUNK < c->n) ? c1 * SKAINET_F32_UNARY_CHUNK : c->n; + switch (c->op) { + case SKAINET_F32_RELU: { + int32_t i = i0; +#ifdef SKAINET_F32_HAVE_VEC + const SKAINET_F32_VEC zero = SKAINET_F32_SPLAT(0.0f); + for (; i + 4 <= i1; i += 4) SKAINET_F32_STORE(out + i, SKAINET_F32_VMAX(SKAINET_F32_LOAD(in + i), zero)); +#endif + for (; i < i1; ++i) out[i] = in[i] < 0.0f ? 0.0f : in[i]; + break; + } + case SKAINET_F32_SIGMOID: + for (int32_t i = i0; i < i1; ++i) out[i] = skainet_f32_sigmoid(in[i]); + break; + case SKAINET_F32_SILU: + for (int32_t i = i0; i < i1; ++i) out[i] = in[i] * skainet_f32_sigmoid(in[i]); + break; + case SKAINET_F32_GELU_TANH: + for (int32_t i = i0; i < i1; ++i) { + const float x = in[i]; + const float inner = x + 0.044715f * (x * x * x); + out[i] = 0.5f * x * (1.0f + skainet_tanhf(0.7978845608f * inner)); + } + break; + case SKAINET_F32_GELU_EXACT: + for (int32_t i = i0; i < i1; ++i) { + const double x = (double) in[i]; + out[i] = (float) (0.5 * x * (1.0 + skainet_erf(x * 0.70710678118654752440))); + } + break; + case SKAINET_F32_ERF: + for (int32_t i = i0; i < i1; ++i) out[i] = (float) skainet_erf((double) in[i]); + break; + default: /* SKAINET_F32_TANH */ + for (int32_t i = i0; i < i1; ++i) out[i] = skainet_tanhf(in[i]); + break; + } +} + +SKAINET_API void skainet_f32_unary(int32_t op, const float* in, float* out, int32_t n) { + if (n <= 0) return; + skainet_f32_unary_ctx ctx; + ctx.in = in; ctx.out = out; ctx.n = n; ctx.op = op; + skainet_run_rows(skainet_f32_unary_chunks, &ctx, (n + SKAINET_F32_UNARY_CHUNK - 1) / SKAINET_F32_UNARY_CHUNK); +} + +/* ------------------------------------------------------------- reductions */ + +SKAINET_API void skainet_f32_reduce_all(const float* in, int32_t n, int32_t mean, float* out) { + float s = 0.0f; + if (n > 0) { + int32_t i = 0; +#ifdef SKAINET_F32_HAVE_VEC + SKAINET_F32_VEC acc0 = SKAINET_F32_SPLAT(0.0f), acc1 = acc0, acc2 = acc0, acc3 = acc0; + for (; i + 16 <= n; i += 16) { + acc0 = SKAINET_F32_VADD(acc0, SKAINET_F32_LOAD(in + i)); + acc1 = SKAINET_F32_VADD(acc1, SKAINET_F32_LOAD(in + i + 4)); + acc2 = SKAINET_F32_VADD(acc2, SKAINET_F32_LOAD(in + i + 8)); + acc3 = SKAINET_F32_VADD(acc3, SKAINET_F32_LOAD(in + i + 12)); + } + const SKAINET_F32_VEC acc = SKAINET_F32_VADD(SKAINET_F32_VADD(acc0, acc1), SKAINET_F32_VADD(acc2, acc3)); +#if defined(SKAINET_HAVE_WASM_SIMD128) + s = skainet_wasm_hadd_f32(acc); +#else + s = skainet_neon_hadd_f32(acc); +#endif +#endif + for (; i < n; ++i) s += in[i]; + } + *out = (mean && n > 0) ? s / (float) n : s; +} + +typedef struct { + const float* in; + float* out; + int32_t red, inner; + int32_t mean; +} skainet_f32_reduce_axis_ctx; + +static void skainet_f32_reduce_axis_rows(void* vctx, int32_t o0, int32_t o1) { + const skainet_f32_reduce_axis_ctx* c = (const skainet_f32_reduce_axis_ctx*) vctx; + const int32_t inner = c->inner; + const int32_t red = c->red; + for (int32_t o = o0; o < o1; ++o) { + float* SKAINET_RESTRICT out = c->out + (size_t) o * inner; + const float* SKAINET_RESTRICT base = c->in + (size_t) o * red * inner; + for (int32_t x = 0; x < inner; ++x) out[x] = 0.0f; + for (int32_t k = 0; k < red; ++k) { + const float* SKAINET_RESTRICT row = base + (size_t) k * inner; + int32_t x = 0; +#ifdef SKAINET_F32_HAVE_VEC + for (; x + 4 <= inner; x += 4) SKAINET_F32_STORE(out + x, SKAINET_F32_VADD(SKAINET_F32_LOAD(out + x), SKAINET_F32_LOAD(row + x))); +#endif + for (; x < inner; ++x) out[x] += row[x]; + } + if (c->mean && red > 0) { + const float inv = 1.0f / (float) red; + int32_t x = 0; +#ifdef SKAINET_F32_HAVE_VEC + const SKAINET_F32_VEC vinv = SKAINET_F32_SPLAT(inv); + for (; x + 4 <= inner; x += 4) SKAINET_F32_STORE(out + x, SKAINET_F32_VMUL(SKAINET_F32_LOAD(out + x), vinv)); +#endif + for (; x < inner; ++x) out[x] *= inv; + } + } +} + +SKAINET_API void skainet_f32_reduce_axis(const float* in, int32_t outer, int32_t red, int32_t inner, int32_t mean, float* out) { + if (outer <= 0 || inner <= 0) return; + if (red <= 0) { + for (int32_t i = 0; i < outer * inner; ++i) out[i] = 0.0f; + return; + } + skainet_f32_reduce_axis_ctx ctx; + ctx.in = in; ctx.out = out; ctx.red = red; ctx.inner = inner; ctx.mean = mean; + skainet_run_rows(skainet_f32_reduce_axis_rows, &ctx, outer); +} diff --git a/skainet-backends/skainet-backend-native-cpu/native/src/skainet_ternary_f32.c b/skainet-backends/skainet-backend-native-cpu/native/src/skainet_ternary_f32.c index b9cf2824f..cdf4015a5 100644 --- a/skainet-backends/skainet-backend-native-cpu/native/src/skainet_ternary_f32.c +++ b/skainet-backends/skainet-backend-native-cpu/native/src/skainet_ternary_f32.c @@ -6,10 +6,13 @@ * pthread_once warm-up, so first use is race-free under any threading. * * The vendored file needs pthreads and GNU/Clang builtins, so it is only in - * the build where SKAINET_HAVE_NEOGPU_TERNARY is defined (non-MSVC). The - * fallback branch below is a portable scalar mirror with identical semantics, - * including the byte-code-3 → +2.0 decode (loaders reject code 3 at import; - * the kernel contract is "garbage in, garbage out", never a crash). + * the build where SKAINET_HAVE_NEOGPU_TERNARY is defined (non-MSVC native + * builds). The other branch below is a portable mirror with identical + * semantics, including the byte-code-3 → +2.0 decode (loaders reject code 3 + * at import; the kernel contract is "garbage in, garbage out", never a + * crash). It is what MSVC and the Emscripten sidecar compile: the same + * 256-entry LUT as a compile-time constant (no init guard to race on), a + * SIMD128 body in the sidecar, and the shared row pool for threading. */ #include "skainet_kernels.h" @@ -108,12 +111,92 @@ void skainet_ternary_lmhead_stage1( (uint32_t)input_dim); } -#else /* !SKAINET_HAVE_NEOGPU_TERNARY — portable scalar mirror (MSVC etc.) */ +#else /* !SKAINET_HAVE_NEOGPU_TERNARY — portable mirror (MSVC, Emscripten) */ -/* decode(byte, lane) = ((byte >> (lane*2)) & 3) - 1, exactly the vendored LUT - * (so byte code 3 yields +2, matching TernaryCodec.decodeBitNet). */ -static float skainet_ternary_decode(uint8_t b, int lane) { - return (float)((int)((b >> (lane * 2)) & 3) - 1); +#include "skainet_simd.h" +#include "skainet_row_threads.h" + +/* + * decode(byte, lane) = ((byte >> (lane*2)) & 3) - 1 — exactly the vendored + * LUT (so byte code 3 yields +2, matching TernaryCodec.decodeBitNet), as a + * compile-time constant: one 16-byte row per weight byte, the four decoded + * codes of its four elements. The vendored file builds the same table on + * first use behind a non-atomic flag; a constant needs no guard, which is + * what makes the row pool below safe without pthread_once. + */ +#define SKAINET_TERNARY_D(b, lane) ((float) ((int) (((b) >> ((lane) * 2)) & 3) - 1)) +#define SKAINET_TERNARY_E(b) \ + { SKAINET_TERNARY_D(b, 0), SKAINET_TERNARY_D(b, 1), SKAINET_TERNARY_D(b, 2), SKAINET_TERNARY_D(b, 3) } +#define SKAINET_TERNARY_E16(b) \ + SKAINET_TERNARY_E(b + 0), SKAINET_TERNARY_E(b + 1), SKAINET_TERNARY_E(b + 2), SKAINET_TERNARY_E(b + 3), \ + SKAINET_TERNARY_E(b + 4), SKAINET_TERNARY_E(b + 5), SKAINET_TERNARY_E(b + 6), SKAINET_TERNARY_E(b + 7), \ + SKAINET_TERNARY_E(b + 8), SKAINET_TERNARY_E(b + 9), SKAINET_TERNARY_E(b + 10), SKAINET_TERNARY_E(b + 11), \ + SKAINET_TERNARY_E(b + 12), SKAINET_TERNARY_E(b + 13), SKAINET_TERNARY_E(b + 14), SKAINET_TERNARY_E(b + 15) + +static const float skainet_ternary_lut[256][4] = { + SKAINET_TERNARY_E16(0), SKAINET_TERNARY_E16(16), SKAINET_TERNARY_E16(32), SKAINET_TERNARY_E16(48), + SKAINET_TERNARY_E16(64), SKAINET_TERNARY_E16(80), SKAINET_TERNARY_E16(96), SKAINET_TERNARY_E16(112), + SKAINET_TERNARY_E16(128), SKAINET_TERNARY_E16(144), SKAINET_TERNARY_E16(160), SKAINET_TERNARY_E16(176), + SKAINET_TERNARY_E16(192), SKAINET_TERNARY_E16(208), SKAINET_TERNARY_E16(224), SKAINET_TERNARY_E16(240), +}; + +#undef SKAINET_TERNARY_E16 +#undef SKAINET_TERNARY_E +#undef SKAINET_TERNARY_D + +/* FP16 row scale decode, same bit arithmetic as the vendored file (sign + * ignored — encoders store max|row| >= 0). */ +static inline float skainet_ternary_row_scale(uint16_t h16) { + const uint32_t eu = (h16 >> 10) & 0x1Fu; + const uint32_t mu = h16 & 0x03FFu; + const uint32_t fu = ((eu + 112u) << 23) | (mu << 13); + float rsc; + memcpy(&rsc, &fu, 4); + return rsc; +} + +typedef struct { + float* out; + const float* in; + const uint8_t* w; + int32_t row_bytes; +} skainet_ternary_gemv_job; + +/* out[o] = Σ_bi lut[w[o][bi]] · in[bi*4 .. bi*4+3] for rows [o_start, o_end). */ +static void skainet_ternary_gemv_rows(void* ctx, int32_t o_start, int32_t o_end) { + const skainet_ternary_gemv_job* j = (const skainet_ternary_gemv_job*) ctx; + const float* in = j->in; + const int32_t row_bytes = j->row_bytes; + for (int32_t o = o_start; o < o_end; ++o) { + const uint8_t* wrow = j->w + (size_t) o * row_bytes; +#ifdef SKAINET_HAVE_WASM_SIMD128 + /* One LUT row is one f32x4 of decoded codes against four activations: + * a load, a multiply and an add per weight byte. Four accumulators + * keep the adds independent; the vendored NEON body does the same. */ + v128_t acc0 = wasm_f32x4_splat(0.0f), acc1 = acc0, acc2 = acc0, acc3 = acc0; + int32_t bi = 0; + for (; bi + 4 <= row_bytes; bi += 4) { + acc0 = wasm_f32x4_add(acc0, wasm_f32x4_mul(wasm_v128_load(skainet_ternary_lut[wrow[bi]]), wasm_v128_load(in + (size_t) bi * 4))); + acc1 = wasm_f32x4_add(acc1, wasm_f32x4_mul(wasm_v128_load(skainet_ternary_lut[wrow[bi + 1]]), wasm_v128_load(in + (size_t) bi * 4 + 4))); + acc2 = wasm_f32x4_add(acc2, wasm_f32x4_mul(wasm_v128_load(skainet_ternary_lut[wrow[bi + 2]]), wasm_v128_load(in + (size_t) bi * 4 + 8))); + acc3 = wasm_f32x4_add(acc3, wasm_f32x4_mul(wasm_v128_load(skainet_ternary_lut[wrow[bi + 3]]), wasm_v128_load(in + (size_t) bi * 4 + 12))); + } + for (; bi < row_bytes; ++bi) { + acc0 = wasm_f32x4_add(acc0, wasm_f32x4_mul(wasm_v128_load(skainet_ternary_lut[wrow[bi]]), wasm_v128_load(in + (size_t) bi * 4))); + } + j->out[o] = skainet_wasm_hadd_f32(wasm_f32x4_add(wasm_f32x4_add(acc0, acc1), wasm_f32x4_add(acc2, acc3))); +#else + float acc = 0.0f; + for (int32_t bi = 0; bi < row_bytes; ++bi) { + const float* d = skainet_ternary_lut[wrow[bi]]; + acc += d[0] * in[bi * 4 ]; + acc += d[1] * in[bi * 4 + 1]; + acc += d[2] * in[bi * 4 + 2]; + acc += d[3] * in[bi * 4 + 3]; + } + j->out[o] = acc; +#endif + } } void skainet_ternary_f32_gemv( @@ -123,20 +206,76 @@ void skainet_ternary_f32_gemv( float* output, int32_t output_offset) { if (output_dim <= 0) return; - const float* in = input + input_offset; - const uint8_t* w = weight + weight_byte_offset; - const int32_t row_bytes = input_dim / 4; - for (int32_t n = 0; n < output_dim; n++) { - const uint8_t* wrow = w + (size_t)n * row_bytes; - float acc = 0.0f; - for (int32_t bi = 0; bi < row_bytes; bi++) { - const uint8_t b = wrow[bi]; - acc += skainet_ternary_decode(b, 0) * in[bi * 4 ]; - acc += skainet_ternary_decode(b, 1) * in[bi * 4 + 1]; - acc += skainet_ternary_decode(b, 2) * in[bi * 4 + 2]; - acc += skainet_ternary_decode(b, 3) * in[bi * 4 + 3]; + skainet_ternary_gemv_job job = { + output + output_offset, + input + input_offset, + weight + weight_byte_offset, + input_dim / 4, + }; + /* input_dim == 0: row_bytes 0, every row is written as 0.0f. */ + skainet_run_rows(skainet_ternary_gemv_rows, &job, output_dim); +} + +typedef struct { + float* out; + const float* in; + const uint8_t* p0; + const uint8_t* p1; + const uint8_t* p2; + const uint8_t* p3; + const uint16_t* row_scale; + int32_t row_bytes; +} skainet_ternary_lmhead_job; + +/* out[o] = rs[o] · Σ_bi (lut[p0] + lut[p1]/3 + lut[p2]/9 + lut[p3]/27) · in[bi*4 ..]. */ +static void skainet_ternary_lmhead_rows(void* ctx, int32_t o_start, int32_t o_end) { + const skainet_ternary_lmhead_job* j = (const skainet_ternary_lmhead_job*) ctx; + const float* in = j->in; + const int32_t row_bytes = j->row_bytes; + const float w1 = 1.0f / 3.0f, w2 = 1.0f / 9.0f, w3 = 1.0f / 27.0f; + for (int32_t o = o_start; o < o_end; ++o) { + const uint8_t* r0 = j->p0 + (size_t) o * row_bytes; + const uint8_t* r1 = j->p1 + (size_t) o * row_bytes; + const uint8_t* r2 = j->p2 + (size_t) o * row_bytes; + const uint8_t* r3 = j->p3 + (size_t) o * row_bytes; +#ifdef SKAINET_HAVE_WASM_SIMD128 + const v128_t vw1 = wasm_f32x4_splat(w1), vw2 = wasm_f32x4_splat(w2), vw3 = wasm_f32x4_splat(w3); + v128_t acc0 = wasm_f32x4_splat(0.0f), acc1 = acc0; + int32_t bi = 0; + for (; bi + 2 <= row_bytes; bi += 2) { + v128_t u0 = wasm_v128_load(skainet_ternary_lut[r0[bi]]); + u0 = wasm_f32x4_add(u0, wasm_f32x4_mul(wasm_v128_load(skainet_ternary_lut[r1[bi]]), vw1)); + u0 = wasm_f32x4_add(u0, wasm_f32x4_mul(wasm_v128_load(skainet_ternary_lut[r2[bi]]), vw2)); + u0 = wasm_f32x4_add(u0, wasm_f32x4_mul(wasm_v128_load(skainet_ternary_lut[r3[bi]]), vw3)); + acc0 = wasm_f32x4_add(acc0, wasm_f32x4_mul(u0, wasm_v128_load(in + (size_t) bi * 4))); + v128_t u1 = wasm_v128_load(skainet_ternary_lut[r0[bi + 1]]); + u1 = wasm_f32x4_add(u1, wasm_f32x4_mul(wasm_v128_load(skainet_ternary_lut[r1[bi + 1]]), vw1)); + u1 = wasm_f32x4_add(u1, wasm_f32x4_mul(wasm_v128_load(skainet_ternary_lut[r2[bi + 1]]), vw2)); + u1 = wasm_f32x4_add(u1, wasm_f32x4_mul(wasm_v128_load(skainet_ternary_lut[r3[bi + 1]]), vw3)); + acc1 = wasm_f32x4_add(acc1, wasm_f32x4_mul(u1, wasm_v128_load(in + (size_t) bi * 4 + 4))); } - output[output_offset + n] = acc; + for (; bi < row_bytes; ++bi) { + v128_t u = wasm_v128_load(skainet_ternary_lut[r0[bi]]); + u = wasm_f32x4_add(u, wasm_f32x4_mul(wasm_v128_load(skainet_ternary_lut[r1[bi]]), vw1)); + u = wasm_f32x4_add(u, wasm_f32x4_mul(wasm_v128_load(skainet_ternary_lut[r2[bi]]), vw2)); + u = wasm_f32x4_add(u, wasm_f32x4_mul(wasm_v128_load(skainet_ternary_lut[r3[bi]]), vw3)); + acc0 = wasm_f32x4_add(acc0, wasm_f32x4_mul(u, wasm_v128_load(in + (size_t) bi * 4))); + } + const float s = skainet_wasm_hadd_f32(wasm_f32x4_add(acc0, acc1)); +#else + float s = 0.0f; + for (int32_t bi = 0; bi < row_bytes; ++bi) { + const float* d0 = skainet_ternary_lut[r0[bi]]; + const float* d1 = skainet_ternary_lut[r1[bi]]; + const float* d2 = skainet_ternary_lut[r2[bi]]; + const float* d3 = skainet_ternary_lut[r3[bi]]; + for (int lane = 0; lane < 4; ++lane) { + const float u = d0[lane] + d1[lane] * w1 + d2[lane] * w2 + d3[lane] * w3; + s += u * in[bi * 4 + lane]; + } + } +#endif + j->out[o] = s * skainet_ternary_row_scale(j->row_scale[o]); } } @@ -148,34 +287,18 @@ void skainet_ternary_lmhead_stage1( float* output, int32_t output_offset) { if (output_dim <= 0) return; - const float* in = input + input_offset; const uint8_t* base = planes + planes_byte_offset; - const uint16_t* rs = row_scale + row_scale_offset; - const int32_t row_bytes = input_dim / 4; - const float pw[4] = { 1.0f, 1.0f / 3.0f, 1.0f / 9.0f, 1.0f / 27.0f }; - for (int32_t n = 0; n < output_dim; n++) { - float acc = 0.0f; - for (int32_t bi = 0; bi < row_bytes; bi++) { - for (int lane = 0; lane < 4; lane++) { - float u = 0.0f; - for (int p = 0; p < 4; p++) { - const uint8_t b = - base[(size_t)plane_stride_bytes * p + (size_t)n * row_bytes + bi]; - u += skainet_ternary_decode(b, lane) * pw[p]; - } - acc += u * in[bi * 4 + lane]; - } - } - /* FP16 row scale decode, same bit arithmetic as the vendored file - * (sign ignored — encoders store max|row| >= 0). */ - const uint32_t h16 = rs[n]; - const uint32_t eu = (h16 >> 10) & 0x1Fu; - const uint32_t mu = h16 & 0x03FFu; - const uint32_t fu = ((eu + 112u) << 23) | (mu << 13); - float rsc; - memcpy(&rsc, &fu, 4); - output[output_offset + n] = acc * rsc; - } + skainet_ternary_lmhead_job job = { + output + output_offset, + input + input_offset, + base, + base + (size_t) plane_stride_bytes, + base + (size_t) plane_stride_bytes * 2, + base + (size_t) plane_stride_bytes * 3, + row_scale + row_scale_offset, + input_dim / 4, + }; + skainet_run_rows(skainet_ternary_lmhead_rows, &job, output_dim); } #endif /* SKAINET_HAVE_NEOGPU_TERNARY */ diff --git a/skainet-backends/skainet-backend-native-cpu/src/jvmMain/kotlin/sk/ainet/exec/kernel/NativeFp32MatmulWtKernel.kt b/skainet-backends/skainet-backend-native-cpu/src/jvmMain/kotlin/sk/ainet/exec/kernel/NativeFp32MatmulWtKernel.kt new file mode 100644 index 000000000..1ec68d773 --- /dev/null +++ b/skainet-backends/skainet-backend-native-cpu/src/jvmMain/kotlin/sk/ainet/exec/kernel/NativeFp32MatmulWtKernel.kt @@ -0,0 +1,116 @@ +package sk.ainet.exec.kernel + +import java.lang.foreign.Arena +import java.lang.foreign.FunctionDescriptor +import java.lang.foreign.Linker +import java.lang.foreign.MemorySegment +import java.lang.foreign.ValueLayout +import java.lang.invoke.MethodHandle +import sk.ainet.backend.api.kernel.Fp32MatmulWtKernel +import sk.ainet.backend.api.kernel.Fp32MemSegMatmulWtKernel + +/** + * Native (FFM) `x · Wᵀ` over a row-major `[out, in]` FP32 weight — the JVM binding of + * + * void skainet_fp32_matmul_wt( + * const float* in, int32_t in_offset, + * const float* w, int32_t w_offset, + * float* out, int32_t out_offset, + * int32_t m, int32_t n, int32_t k); + * + * which the wasm sidecar has served since 0.61.0 and the JVM never bound: `DefaultCpuOpsJvm` + * transposed every dense weight once into `[k, n]` for the GEMM kernels and cached the copy. The C + * kernel computes every output as one contiguous dot product over the weight's own row (NEON or + * auto-vectorised portable body, four rows per tile), threads over output rows through the shared + * row pool, and accumulates in ascending `p` — bit-identical to transpose-then-GEMM. + * + * Two faces. The heap face ([Fp32MatmulWtKernel]) stages both operands off-heap per call, like + * [NativeFp32MatmulKernel]; for a decode step that copy is the weight, so it is the slow one. The + * segment face ([Fp32MemSegMatmulWtKernel]) takes the weight where it lies and copies only the + * `m × k` activation and the `m × n` output, which is what the ops layer uses once it has made a + * weight resident off-heap (the JVM counterpart of the sidecar's resident dense weights). + */ +internal object NativeFp32MatmulWtKernel : Fp32MatmulWtKernel, Fp32MemSegMatmulWtKernel { + + fun isAvailable(): Boolean = handle != null + + override fun matmulWt( + x: FloatArray, xOffset: Int, + w: FloatArray, wOffset: Int, + out: FloatArray, outOffset: Int, + m: Int, n: Int, k: Int, + ) { + require(m >= 0 && n >= 0 && k >= 0) { "NativeFp32MatmulWtKernel: m, n, k must be non-negative; got m=$m n=$n k=$k" } + if (m == 0 || n == 0) return + val mh = handle ?: error("NativeFp32MatmulWtKernel invoked while native library unavailable") + val wFloats = n.toLong() * k + require(wOffset >= 0 && wOffset + wFloats <= w.size) { + "NativeFp32MatmulWtKernel: weight needs $wFloats floats from $wOffset, array has ${w.size}" + } + Arena.ofConfined().use { arena -> + val align = ValueLayout.JAVA_FLOAT.byteAlignment() + val wSeg: MemorySegment = if (wFloats > 0) arena.allocate(wFloats * java.lang.Float.BYTES, align) else MemorySegment.NULL + if (wFloats > 0) MemorySegment.copy(w, wOffset, wSeg, ValueLayout.JAVA_FLOAT, 0L, wFloats.toInt()) + invoke(mh, arena, x, xOffset, wSeg, out, outOffset, m, n, k) + } + } + + override fun matmulWt( + x: FloatArray, xOffset: Int, + weight: MemorySegment, weightByteOffset: Long, + out: FloatArray, outOffset: Int, + m: Int, n: Int, k: Int, + ) { + require(m >= 0 && n >= 0 && k >= 0) { "NativeFp32MatmulWtKernel: m, n, k must be non-negative; got m=$m n=$n k=$k" } + if (m == 0 || n == 0) return + val mh = handle ?: error("NativeFp32MatmulWtKernel invoked while native library unavailable") + val wBytes = n.toLong() * k * java.lang.Float.BYTES + require(weightByteOffset >= 0 && weightByteOffset + wBytes <= weight.byteSize()) { + "NativeFp32MatmulWtKernel: weight segment too small — needs $wBytes bytes from offset " + + "$weightByteOffset, segment is ${weight.byteSize()} bytes" + } + Arena.ofConfined().use { arena -> + invoke(mh, arena, x, xOffset, if (wBytes > 0) weight.asSlice(weightByteOffset) else MemorySegment.NULL, out, outOffset, m, n, k) + } + } + + /** Stage the activation and the output, run, copy the output back. The weight is already a segment. */ + private fun invoke( + mh: MethodHandle, arena: Arena, + x: FloatArray, xOffset: Int, wSeg: MemorySegment, + out: FloatArray, outOffset: Int, + m: Int, n: Int, k: Int, + ) { + val xFloats = m.toLong() * k + val outFloats = m.toLong() * n + require(xOffset >= 0 && xOffset + xFloats <= x.size) { + "NativeFp32MatmulWtKernel: activation needs $xFloats floats from $xOffset, array has ${x.size}" + } + require(outOffset >= 0 && outOffset + outFloats <= out.size) { + "NativeFp32MatmulWtKernel: output needs $outFloats floats from $outOffset, array has ${out.size}" + } + val align = ValueLayout.JAVA_FLOAT.byteAlignment() + val xSeg: MemorySegment = if (xFloats > 0) arena.allocate(xFloats * java.lang.Float.BYTES, align) else MemorySegment.NULL + val outSeg: MemorySegment = arena.allocate(outFloats * java.lang.Float.BYTES, align) + if (xFloats > 0) MemorySegment.copy(x, xOffset, xSeg, ValueLayout.JAVA_FLOAT, 0L, xFloats.toInt()) + mh.invoke(xSeg, 0, wSeg, 0, outSeg, 0, m, n, k) + MemorySegment.copy(outSeg, ValueLayout.JAVA_FLOAT, 0L, out, outOffset, outFloats.toInt()) + } + + private val handle: MethodHandle? by lazy { + val lookup = NativeLibraryLoader.lookup() ?: return@lazy null + val symbol = lookup.find("skainet_fp32_matmul_wt").orElse(null) ?: return@lazy null + val descriptor = FunctionDescriptor.ofVoid( + ValueLayout.ADDRESS, // in + ValueLayout.JAVA_INT, // in_offset + ValueLayout.ADDRESS, // w + ValueLayout.JAVA_INT, // w_offset + ValueLayout.ADDRESS, // out + ValueLayout.JAVA_INT, // out_offset + ValueLayout.JAVA_INT, // m + ValueLayout.JAVA_INT, // n + ValueLayout.JAVA_INT, // k + ) + runCatching { Linker.nativeLinker().downcallHandle(symbol, descriptor) }.getOrNull() + } +} diff --git a/skainet-backends/skainet-backend-native-cpu/src/jvmMain/kotlin/sk/ainet/exec/kernel/NativeFusedKernels.kt b/skainet-backends/skainet-backend-native-cpu/src/jvmMain/kotlin/sk/ainet/exec/kernel/NativeFusedKernels.kt new file mode 100644 index 000000000..27d2aa39a --- /dev/null +++ b/skainet-backends/skainet-backend-native-cpu/src/jvmMain/kotlin/sk/ainet/exec/kernel/NativeFusedKernels.kt @@ -0,0 +1,231 @@ +package sk.ainet.exec.kernel + +import java.lang.foreign.Arena +import java.lang.foreign.FunctionDescriptor +import java.lang.foreign.Linker +import java.lang.foreign.MemorySegment +import java.lang.foreign.ValueLayout +import java.lang.invoke.MethodHandle +import sk.ainet.backend.api.kernel.Fp32AttentionKernel +import sk.ainet.backend.api.kernel.Fp32NormKernel +import sk.ainet.backend.api.kernel.Fp32SoftmaxKernel + +/* + * Native (FFM) bindings of the fused dense FP32 row kernels (`f32_fused.c`, SKEEP-007 fused ops): + * softmax rows, layer norm / RMS norm and banded scaled dot-product attention. The same symbols the + * wasm sidecar exports; on the JVM they replace the scalar Kotlin loops of `DefaultCpuOpsBase` + * for these ops (attention there is scalar per head, parallel over heads through the schedule; + * here the C kernel vectorises the dot products and threads over query rows in the row pool). + * Every face stages its operands off-heap per call and copies the output back, like + * [NativeFp32MatmulKernel]; the operands of these ops are activations, so there is nothing to keep + * resident. + */ + +private fun bind(symbol: String, descriptor: FunctionDescriptor): MethodHandle? { + val lookup = NativeLibraryLoader.lookup() ?: return null + val sym = lookup.find(symbol).orElse(null) ?: return null + return runCatching { Linker.nativeLinker().downcallHandle(sym, descriptor) }.getOrNull() +} + +/** [count] floats of [src] from [offset] as a fresh confined segment; `MemorySegment.NULL` for an empty count. */ +private fun stageFloats(arena: Arena, src: FloatArray, offset: Int, count: Long, what: String): MemorySegment { + require(offset >= 0 && offset + count <= src.size) { "$what needs $count floats from $offset, array has ${src.size}" } + if (count == 0L) return MemorySegment.NULL + val seg = arena.allocate(count * java.lang.Float.BYTES, ValueLayout.JAVA_FLOAT.byteAlignment()) + MemorySegment.copy(src, offset, seg, ValueLayout.JAVA_FLOAT, 0L, count.toInt()) + return seg +} + +private fun outSegment(arena: Arena, count: Long): MemorySegment = + arena.allocate(maxOf(count, 1L) * java.lang.Float.BYTES, ValueLayout.JAVA_FLOAT.byteAlignment()) + +private fun readFloats(seg: MemorySegment, dst: FloatArray, offset: Int, count: Long, what: String) { + require(offset >= 0 && offset + count <= dst.size) { "$what needs $count floats from $offset, array has ${dst.size}" } + if (count > 0) MemorySegment.copy(seg, ValueLayout.JAVA_FLOAT, 0L, dst, offset, count.toInt()) +} + +/** `skainet_f32_softmax_rows` through FFM. */ +internal object NativeFp32SoftmaxKernel : Fp32SoftmaxKernel { + + fun isAvailable(): Boolean = handle != null + + override fun softmaxRows(input: FloatArray, inputOffset: Int, out: FloatArray, outOffset: Int, rows: Int, cols: Int, log: Boolean) { + require(rows >= 0 && cols >= 0) { "NativeFp32SoftmaxKernel: rows and cols must be non-negative; got rows=$rows cols=$cols" } + if (rows == 0 || cols == 0) return + val mh = handle ?: error("NativeFp32SoftmaxKernel invoked while native library unavailable") + val n = rows.toLong() * cols + Arena.ofConfined().use { arena -> + val inSeg = stageFloats(arena, input, inputOffset, n, "NativeFp32SoftmaxKernel: input") + val outSeg = outSegment(arena, n) + mh.invoke(inSeg, outSeg, rows, cols, if (log) 1 else 0) + readFloats(outSeg, out, outOffset, n, "NativeFp32SoftmaxKernel: output") + } + } + + private val handle: MethodHandle? by lazy { + bind( + "skainet_f32_softmax_rows", + FunctionDescriptor.ofVoid( + ValueLayout.ADDRESS, // in + ValueLayout.ADDRESS, // out + ValueLayout.JAVA_INT, // rows + ValueLayout.JAVA_INT, // cols + ValueLayout.JAVA_INT, // log_form + ), + ) + } +} + +/** `skainet_f32_layer_norm` and `skainet_f32_rms_norm` through FFM. */ +internal object NativeFp32NormKernel : Fp32NormKernel { + + fun isAvailable(): Boolean = layerNormHandle != null && rmsNormHandle != null + + override fun layerNorm( + input: FloatArray, inputOffset: Int, + gamma: FloatArray?, beta: FloatArray?, + out: FloatArray, outOffset: Int, + rows: Int, cols: Int, eps: Float, + ) { + require(rows >= 0 && cols >= 0) { "NativeFp32NormKernel: rows and cols must be non-negative; got rows=$rows cols=$cols" } + if (rows == 0 || cols == 0) return + val mh = layerNormHandle ?: error("NativeFp32NormKernel invoked while native library unavailable") + val n = rows.toLong() * cols + Arena.ofConfined().use { arena -> + val inSeg = stageFloats(arena, input, inputOffset, n, "NativeFp32NormKernel: input") + val gSeg = gamma?.let { stageFloats(arena, it, 0, cols.toLong(), "NativeFp32NormKernel: gamma") } ?: MemorySegment.NULL + val bSeg = beta?.let { stageFloats(arena, it, 0, cols.toLong(), "NativeFp32NormKernel: beta") } ?: MemorySegment.NULL + val outSeg = outSegment(arena, n) + mh.invoke(inSeg, gSeg, bSeg, outSeg, rows, cols, eps) + readFloats(outSeg, out, outOffset, n, "NativeFp32NormKernel: output") + } + } + + override fun rmsNorm( + input: FloatArray, inputOffset: Int, + gamma: FloatArray?, + out: FloatArray, outOffset: Int, + rows: Int, cols: Int, eps: Float, + ) { + require(rows >= 0 && cols >= 0) { "NativeFp32NormKernel: rows and cols must be non-negative; got rows=$rows cols=$cols" } + if (rows == 0 || cols == 0) return + val mh = rmsNormHandle ?: error("NativeFp32NormKernel invoked while native library unavailable") + val n = rows.toLong() * cols + Arena.ofConfined().use { arena -> + val inSeg = stageFloats(arena, input, inputOffset, n, "NativeFp32NormKernel: input") + val gSeg = gamma?.let { stageFloats(arena, it, 0, cols.toLong(), "NativeFp32NormKernel: gamma") } ?: MemorySegment.NULL + val outSeg = outSegment(arena, n) + mh.invoke(inSeg, gSeg, outSeg, rows, cols, eps) + readFloats(outSeg, out, outOffset, n, "NativeFp32NormKernel: output") + } + } + + private val layerNormHandle: MethodHandle? by lazy { + bind( + "skainet_f32_layer_norm", + FunctionDescriptor.ofVoid( + ValueLayout.ADDRESS, // in + ValueLayout.ADDRESS, // gamma (nullable) + ValueLayout.ADDRESS, // beta (nullable) + ValueLayout.ADDRESS, // out + ValueLayout.JAVA_INT, // rows + ValueLayout.JAVA_INT, // cols + ValueLayout.JAVA_FLOAT, // eps + ), + ) + } + + private val rmsNormHandle: MethodHandle? by lazy { + bind( + "skainet_f32_rms_norm", + FunctionDescriptor.ofVoid( + ValueLayout.ADDRESS, // in + ValueLayout.ADDRESS, // gamma (nullable) + ValueLayout.ADDRESS, // out + ValueLayout.JAVA_INT, // rows + ValueLayout.JAVA_INT, // cols + ValueLayout.JAVA_FLOAT, // eps + ), + ) + } +} + +/** `skainet_f32_sdpa` through FFM. */ +internal object NativeFp32AttentionKernel : Fp32AttentionKernel { + + fun isAvailable(): Boolean = handle != null + + override fun sdpa( + query: FloatArray, key: FloatArray, value: FloatArray, out: FloatArray, + batch: Int, heads: Int, kvHeads: Int, + seqQ: Int, seqKV: Int, headDim: Int, + scale: Float, causal: Boolean, window: Int, rightContext: Int, + keyLengths: IntArray?, + mask: FloatArray?, maskDims: IntArray?, + ) { + require(batch >= 0 && heads >= 0 && kvHeads >= 0 && seqQ >= 0 && seqKV >= 0 && headDim >= 0) { + "NativeFp32AttentionKernel: dimensions must be non-negative" + } + require(kvHeads > 0 && heads % kvHeads == 0) { "NativeFp32AttentionKernel: heads ($heads) must be a multiple of kvHeads ($kvHeads)" } + require(keyLengths == null || keyLengths.size == batch) { "NativeFp32AttentionKernel: keyLengths has ${keyLengths?.size} rows, batch is $batch" } + require((mask == null) == (maskDims == null) && (maskDims == null || maskDims.size == 4)) { + "NativeFp32AttentionKernel: mask and maskDims (4 dims) go together" + } + val outCount = batch.toLong() * heads * seqQ * headDim + require(outCount <= out.size) { "NativeFp32AttentionKernel: output needs $outCount floats, array has ${out.size}" } + if (outCount == 0L) return + val mh = handle ?: error("NativeFp32AttentionKernel invoked while native library unavailable") + val qCount = batch.toLong() * heads * seqQ * headDim + val kvCount = batch.toLong() * kvHeads * seqKV * headDim + Arena.ofConfined().use { arena -> + val qSeg = stageFloats(arena, query, 0, qCount, "NativeFp32AttentionKernel: query") + val kSeg = stageFloats(arena, key, 0, kvCount, "NativeFp32AttentionKernel: key") + val vSeg = stageFloats(arena, value, 0, kvCount, "NativeFp32AttentionKernel: value") + val outSeg = outSegment(arena, outCount) + val lSeg = if (keyLengths != null && batch > 0) { + val seg = arena.allocate(batch.toLong() * Integer.BYTES, ValueLayout.JAVA_INT.byteAlignment()) + MemorySegment.copy(keyLengths, 0, seg, ValueLayout.JAVA_INT, 0L, batch) + seg + } else MemorySegment.NULL + val md = maskDims ?: intArrayOf(1, 1, 1, 1) + val mSeg = if (mask != null) { + stageFloats(arena, mask, 0, md[0].toLong() * md[1] * md[2] * md[3], "NativeFp32AttentionKernel: mask") + } else MemorySegment.NULL + mh.invoke( + qSeg, kSeg, vSeg, outSeg, + batch, heads, kvHeads, seqQ, seqKV, headDim, + scale, if (causal) 1 else 0, window, rightContext, + lSeg, mSeg, md[0], md[1], md[2], md[3], + ) + readFloats(outSeg, out, 0, outCount, "NativeFp32AttentionKernel: output") + } + } + + private val handle: MethodHandle? by lazy { + bind( + "skainet_f32_sdpa", + FunctionDescriptor.ofVoid( + ValueLayout.ADDRESS, // q + ValueLayout.ADDRESS, // k + ValueLayout.ADDRESS, // v + ValueLayout.ADDRESS, // out + ValueLayout.JAVA_INT, // batch + ValueLayout.JAVA_INT, // heads + ValueLayout.JAVA_INT, // kv_heads + ValueLayout.JAVA_INT, // seq_q + ValueLayout.JAVA_INT, // seq_kv + ValueLayout.JAVA_INT, // head_dim + ValueLayout.JAVA_FLOAT, // scale + ValueLayout.JAVA_INT, // causal + ValueLayout.JAVA_INT, // window + ValueLayout.JAVA_INT, // right_context + ValueLayout.ADDRESS, // lengths (nullable) + ValueLayout.ADDRESS, // mask (nullable) + ValueLayout.JAVA_INT, // mask_b + ValueLayout.JAVA_INT, // mask_h + ValueLayout.JAVA_INT, // mask_q + ValueLayout.JAVA_INT, // mask_k + ), + ) + } +} diff --git a/skainet-backends/skainet-backend-native-cpu/src/jvmMain/kotlin/sk/ainet/exec/kernel/NativeKernelProvider.kt b/skainet-backends/skainet-backend-native-cpu/src/jvmMain/kotlin/sk/ainet/exec/kernel/NativeKernelProvider.kt index cff68465f..14c929edd 100644 --- a/skainet-backends/skainet-backend-native-cpu/src/jvmMain/kotlin/sk/ainet/exec/kernel/NativeKernelProvider.kt +++ b/skainet-backends/skainet-backend-native-cpu/src/jvmMain/kotlin/sk/ainet/exec/kernel/NativeKernelProvider.kt @@ -3,6 +3,11 @@ package sk.ainet.exec.kernel import sk.ainet.backend.api.kernel.Bf16MatmulKernel import sk.ainet.backend.api.kernel.Fp16MatmulKernel import sk.ainet.backend.api.kernel.Fp32MatmulKernel +import sk.ainet.backend.api.kernel.Fp32MatmulWtKernel +import sk.ainet.backend.api.kernel.Fp32AttentionKernel +import sk.ainet.backend.api.kernel.Fp32MemSegMatmulWtKernel +import sk.ainet.backend.api.kernel.Fp32NormKernel +import sk.ainet.backend.api.kernel.Fp32SoftmaxKernel import sk.ainet.backend.api.kernel.KernelProvider import sk.ainet.backend.api.kernel.MemSegKernelProvider import sk.ainet.backend.api.kernel.Q4KMatmulKernel @@ -82,6 +87,13 @@ import sk.ainet.backend.api.kernel.Q8_0MatmulKernel * Every narrow-float accessor the SPI declares is wired here; a format * served natively on one side and by the JVM fallback on the other looks * like a slow kernel rather than a missing one. + * - Now: the direct `x · Wᵀ` kernel (`skainet_fp32_matmul_wt`) in + * [matmulFp32Wt] and its segment face [matmulFp32WtMemSeg], so a dense + * `[out, in]` weight is multiplied in place instead of transposed and cached. + * - Now: the fused row kernels (`f32_fused.c`) in [softmaxFp32], [normFp32] and + * [attentionFp32] — softmax, layer norm / RMS norm and banded scaled dot-product + * attention in one C pass each, the same symbols the wasm sidecar serves, so the + * ops that were scalar Kotlin on the JVM (attention, the normalisations) run native. */ public object NativeKernelProvider : KernelProvider, MemSegKernelProvider { override val name: String = "native-ffm" @@ -92,6 +104,12 @@ public object NativeKernelProvider : KernelProvider, MemSegKernelProvider { override fun matmulFp32(): Fp32MatmulKernel? = if (NativeFp32MatmulKernel.isAvailable()) NativeFp32MatmulKernel else null + override fun matmulFp32Wt(): Fp32MatmulWtKernel? = + if (NativeFp32MatmulWtKernel.isAvailable()) NativeFp32MatmulWtKernel else null + + override fun matmulFp32WtMemSeg(): Fp32MemSegMatmulWtKernel? = + if (NativeFp32MatmulWtKernel.isAvailable()) NativeFp32MatmulWtKernel else null + override fun matmulQ4K(): Q4KMatmulKernel? = if (NativeQ4KMatmulKernel.isAvailable()) NativeQ4KMatmulKernel else null @@ -121,4 +139,13 @@ public object NativeKernelProvider : KernelProvider, MemSegKernelProvider { override fun matmulQ5_1(): Q5_1MatmulKernel? = if (NativeQ5_1MatmulKernel.isAvailable()) NativeQ5_1MatmulKernel else null + + override fun softmaxFp32(): Fp32SoftmaxKernel? = + if (NativeFp32SoftmaxKernel.isAvailable()) NativeFp32SoftmaxKernel else null + + override fun normFp32(): Fp32NormKernel? = + if (NativeFp32NormKernel.isAvailable()) NativeFp32NormKernel else null + + override fun attentionFp32(): Fp32AttentionKernel? = + if (NativeFp32AttentionKernel.isAvailable()) NativeFp32AttentionKernel else null } diff --git a/skainet-backends/skainet-backend-native-cpu/src/jvmTest/kotlin/sk/ainet/exec/kernel/JvmDirectWeightTransposedTest.kt b/skainet-backends/skainet-backend-native-cpu/src/jvmTest/kotlin/sk/ainet/exec/kernel/JvmDirectWeightTransposedTest.kt new file mode 100644 index 000000000..806622903 --- /dev/null +++ b/skainet-backends/skainet-backend-native-cpu/src/jvmTest/kotlin/sk/ainet/exec/kernel/JvmDirectWeightTransposedTest.kt @@ -0,0 +1,79 @@ +package sk.ainet.exec.kernel + +import kotlin.math.abs +import kotlin.random.Random +import kotlin.test.AfterTest +import kotlin.test.BeforeTest +import kotlin.test.Test +import kotlin.test.assertEquals +import kotlin.test.assertTrue +import sk.ainet.backend.api.kernel.KernelRegistry +import sk.ainet.backend.api.kernel.KernelServiceLoader +import sk.ainet.context.DirectCpuExecutionContext +import sk.ainet.lang.tensor.Shape +import sk.ainet.lang.tensor.Tensor +import sk.ainet.lang.tensor.data.MemorySegmentBackedData +import sk.ainet.lang.tensor.data.MemorySegmentTensorDataFactory +import sk.ainet.lang.types.FP32 + +/** + * `x · Wᵀ` on the JVM ops layer with a dense `[out, in]` weight goes through the native direct + * kernel (`skainet_fp32_matmul_wt`) when the FFM provider is loaded — heap weights after one + * resident off-heap copy, segment-backed weights where they lie — and agrees with the + * transpose-then-matmul result it replaces, for every activation rank a `Linear` layer sees. + */ +class JvmDirectWeightTransposedTest { + + @BeforeTest + fun setUp() { + KernelRegistry.clearForTesting() + KernelServiceLoader.installAll() + assertTrue(NativeFp32MatmulWtKernel.isAvailable(), "bundled libskainet_kernels must resolve") + } + + @AfterTest + fun tearDown() = KernelRegistry.clearForTesting() + + private fun assertClose(expected: FloatArray, actual: FloatArray, what: String) { + assertEquals(expected.size, actual.size, "$what size") + var maxAbs = 0f + for (v in expected) maxAbs = maxOf(maxAbs, abs(v)) + val tol = maxAbs * 1e-5f + 1e-6f + for (i in expected.indices) { + assertTrue(abs(expected[i] - actual[i]) <= tol, "$what[$i]: ${actual[i]} vs ${expected[i]} (tol $tol)") + } + } + + private fun check(ctx: DirectCpuExecutionContext, xShape: Shape, n: Int, seed: Int, what: String) { + val rng = Random(seed) + val k = xShape.dimensions.last() + val x: Tensor = ctx.fromFloatArray(xShape, FP32::class, FloatArray(xShape.volume) { rng.nextFloat() - 0.5f }) + val w: Tensor = ctx.fromFloatArray(Shape(n, k), FP32::class, FloatArray(n * k) { rng.nextFloat() - 0.5f }) + val got = ctx.ops.matmulWeightTransposed(x, w) + val expectedShape = if (xShape.rank == 1) Shape(n) else Shape(*(xShape.dimensions.copyOf(xShape.rank - 1) + n)) + assertEquals(expectedShape, got.shape, "$what shape") + val expected = ctx.ops.matmul(x, ctx.ops.transpose(w)).data.copyToFloatArray() + assertClose(expected, got.data.copyToFloatArray(), what) + // A second call reuses the resident weight: same answer. + assertClose(expected, ctx.ops.matmulWeightTransposed(x, w).data.copyToFloatArray(), "$what again") + } + + @Test + fun heap_weights_match_transpose_then_matmul_for_every_activation_rank() { + val ctx = DirectCpuExecutionContext() + assertEquals("native-ffm", KernelRegistry.bestAvailable()?.name, "the FFM provider must win on this runner") + check(ctx, Shape(40), 70, 1, "rank-1") + check(ctx, Shape(3, 40), 70, 2, "rank-2") + check(ctx, Shape(2, 3, 40), 600, 3, "rank-3, threaded rows") + check(ctx, Shape(1, 1), 1, 4, "1×1") + } + + @Test + fun segment_backed_weights_are_read_where_they_lie() { + val ctx = DirectCpuExecutionContext(tensorDataFactory = MemorySegmentTensorDataFactory()) + val w: Tensor = ctx.fromFloatArray(Shape(5, 8), FP32::class, FloatArray(40) { it.toFloat() }) + assertTrue(w.data is MemorySegmentBackedData, "this factory stages tensors off-heap: ${w.data::class.simpleName}") + check(ctx, Shape(4, 32), 48, 5, "segment-backed") + check(ctx, Shape(32), 520, 6, "segment-backed rank-1, threaded rows") + } +} diff --git a/skainet-backends/skainet-backend-native-cpu/src/jvmTest/kotlin/sk/ainet/exec/kernel/KernelSupportMatrixTest.kt b/skainet-backends/skainet-backend-native-cpu/src/jvmTest/kotlin/sk/ainet/exec/kernel/KernelSupportMatrixTest.kt index 84bf97d50..90ba5dfd1 100644 --- a/skainet-backends/skainet-backend-native-cpu/src/jvmTest/kotlin/sk/ainet/exec/kernel/KernelSupportMatrixTest.kt +++ b/skainet-backends/skainet-backend-native-cpu/src/jvmTest/kotlin/sk/ainet/exec/kernel/KernelSupportMatrixTest.kt @@ -4,6 +4,7 @@ import java.io.File import kotlin.test.AfterTest import kotlin.test.Test import kotlin.test.assertEquals +import kotlin.test.assertTrue import sk.ainet.backend.api.kernel.KernelDispatch import sk.ainet.backend.api.kernel.KernelProvider import sk.ainet.lang.memory.plan.StorageCapabilities @@ -106,6 +107,47 @@ class KernelSupportMatrixTest { Tier("ffm-rowmajor", 100, setOf("JVM"), KernelDispatch.mappedServableEncodings().map { it.name }.toSet()), ) + /** + * Kernel packs (view dispatch): the ternary formats and the direct dense `x · Wᵀ` are not served + * through the `matmul` accessors the tables above introspect — the ternary kernels are + * `ViewKernelPack`s keyed in `KernelDispatch` (`FfmTernaryKernelPackFactory`, the JNI ternary pack, + * `NativeKnTernaryF32Gemv` / `NativeKnTernaryLmhead`, `WasmTernaryKernelPack`), and `x · Wᵀ` is the + * `matmulFp32Wt` accessor the ops layer consults before transposing a weight, and the fused row + * kernels (softmax, layer norm / RMS norm, banded SDPA — `softmaxFp32`, `normFp32`, + * `attentionFp32`) are what the ops layer calls instead of its scalar loops. Hand-declared per + * platform like the Android rows; the JVM cells are gated below against what the FFM provider and + * pack actually expose on this runner. Row names are the encodings as `TensorEncoding.name` spells + * them, plus the dense direct product and the fused ops. + */ + private val packRows = listOf( + TensorEncoding.BITNET_B1_58.name + " (FP32 activations)", + TensorEncoding.BITNET_PLANES.name, + TensorEncoding.TQ2_0.name + " (int8 activations)", + "Float32 x·Wᵀ direct", + "Float32 softmax (fused rows)", + "Float32 layer_norm / rms_norm (fused)", + "Float32 SDPA (fused, banded, GQA)", + "Float32 conv1d / conv2d / conv3d (im2col + GEMM)", + ) + + /** + * The convolutions are the one row with a JVM cell that is not a provider: `DefaultCpuOpsJvm` + * runs im2col + the Vector API itself (`JvmVectorKernels`), so the cell is declared as the + * `panama-vector` tier by hand; on wasm `WasmCpuOps` runs `skainet_f32_conv` in the sidecar. + */ + private fun packTiers(): List { + val b158 = packRows[0]; val planes = packRows[1]; val tq2 = packRows[2]; val wt = packRows[3] + val softmax = packRows[4]; val norm = packRows[5]; val sdpa = packRows[6]; val conv = packRows[7] + return listOf( + Tier("ffm-ternary", 100, setOf("JVM"), setOf(b158, planes)), + Tier("native-ffm", 100, setOf("JVM"), setOf(wt, softmax, norm, sdpa)), + Tier("panama-vector", 50, setOf("JVM"), setOf(conv)), + Tier("native-jni", 100, setOf("Android"), setOf(b158, planes, tq2)), + Tier("native-cinterop", 100, setOf("Native·linux", "Native·apple"), setOf(b158, planes)), + Tier("wasm-simd128", 50, setOf("JS/WASM"), setOf(b158, planes, tq2, wt, softmax, norm, sdpa, conv)), + ) + } + private fun best(fmt: String, platform: String, tiers: List): String? = tiers.filter { platform in it.platforms && fmt in it.formats }.maxByOrNull { it.priority }?.name @@ -145,6 +187,15 @@ class KernelSupportMatrixTest { .append(cells.joinToString(", ")).append("}}") sb.append(if (i == residentFormats.lastIndex) "\n" else ",\n") } + sb.append(" ],\n") + val packs = packTiers() + sb.append(" \"packs\": [\n") + packRows.forEachIndexed { i, fmt -> + val cells = platforms.mapNotNull { p -> best(fmt, p, packs)?.let { "\"$p\": \"$it\"" } } + sb.append(" {\"name\": \"").append(fmt).append("\", \"byPlatform\": {") + .append(cells.joinToString(", ")).append("}}") + sb.append(if (i == packRows.lastIndex) "\n" else ",\n") + } sb.append(" ]\n}\n") return sb.toString() } @@ -189,6 +240,19 @@ class KernelSupportMatrixTest { "native-jni-direct row", ) + // Drift gate on the JVM pack rows, where the environment allows it: the FFM provider must + // carry the direct `x · Wᵀ` kernel and the ternary pack both of its kernels. + if (NativeKernelProvider.isAvailable()) { + assertTrue(NativeKernelProvider.matmulFp32Wt() != null, "native-ffm lost its direct x·Wᵀ kernel — update packTiers()") + assertTrue( + NativeKernelProvider.supports("softmax", listOf("Float32")) && NativeKernelProvider.supports("layer_norm", listOf("Float32")) && + NativeKernelProvider.supports("rms_norm", listOf("Float32")) && NativeKernelProvider.supports("sdpa", listOf("Float32")), + "native-ffm lost a fused row kernel — update packTiers()", + ) + assertTrue(NativeTernaryF32GemvKernel.isAvailable() && NativeTernaryLmheadKernel.isAvailable(), + "ffm-ternary lost a kernel — update packTiers()") + } + val jsonText = renderJson(tiers) val outDir = File("build/generated/kernel-support").apply { mkdirs() } File(outDir, "kernel-support.json").writeText(jsonText) diff --git a/skainet-backends/skainet-backend-native-cpu/src/jvmTest/kotlin/sk/ainet/exec/kernel/NativeFp32MatmulWtKernelTest.kt b/skainet-backends/skainet-backend-native-cpu/src/jvmTest/kotlin/sk/ainet/exec/kernel/NativeFp32MatmulWtKernelTest.kt new file mode 100644 index 000000000..78b56d8f4 --- /dev/null +++ b/skainet-backends/skainet-backend-native-cpu/src/jvmTest/kotlin/sk/ainet/exec/kernel/NativeFp32MatmulWtKernelTest.kt @@ -0,0 +1,111 @@ +package sk.ainet.exec.kernel + +import java.lang.foreign.Arena +import java.lang.foreign.MemorySegment +import java.lang.foreign.ValueLayout +import kotlin.math.abs +import kotlin.random.Random +import kotlin.test.BeforeTest +import kotlin.test.Test +import kotlin.test.assertContentEquals +import kotlin.test.assertEquals +import kotlin.test.assertNotNull +import kotlin.test.assertTrue + +/** + * [NativeFp32MatmulWtKernel] — the JVM binding of `skainet_fp32_matmul_wt` — against a plain + * row-dot reference, on both of its faces. The row-pool threshold (512 output rows) is crossed by + * one shape so the threaded partition is covered; per-row arithmetic does not depend on it. + */ +class NativeFp32MatmulWtKernelTest { + + @BeforeTest + fun checkAvailable() { + assertTrue(NativeFp32MatmulWtKernel.isAvailable(), "bundled libskainet_kernels must resolve skainet_fp32_matmul_wt") + } + + private fun reference(x: FloatArray, xOff: Int, w: FloatArray, wOff: Int, m: Int, n: Int, k: Int): FloatArray { + val out = FloatArray(m * n) + for (i in 0 until m) for (o in 0 until n) { + var acc = 0.0 + for (p in 0 until k) acc += x[xOff + i * k + p].toDouble() * w[wOff + o * k + p] + out[i * n + o] = acc.toFloat() + } + return out + } + + private fun assertClose(expected: FloatArray, actual: FloatArray, what: String) { + assertEquals(expected.size, actual.size, "$what size") + var maxAbs = 0f + for (v in expected) maxAbs = maxOf(maxAbs, abs(v)) + val tol = maxAbs * 1e-5f + 1e-6f + for (i in expected.indices) { + assertTrue(abs(expected[i] - actual[i]) <= tol, "$what[$i]: ${actual[i]} vs ${expected[i]} (tol $tol)") + } + } + + @Test + fun heap_face_matches_the_row_dot_reference_on_every_shape() { + val rng = Random(3) + // (m, n, k): a single token; a prefill against a threaded row count with a 4-lane tail; one row of weight. + for ((m, n, k) in listOf(Triple(1, 7, 5), Triple(3, 600, 37), Triple(5, 1, 64))) { + val x = FloatArray(m * k) { rng.nextFloat() - 0.5f } + val w = FloatArray(n * k) { rng.nextFloat() - 0.5f } + val out = FloatArray(m * n) + NativeFp32MatmulWtKernel.matmulWt(x, 0, w, 0, out, 0, m, n, k) + assertClose(reference(x, 0, w, 0, m, n, k), out, "wt $m×$k · ($n×$k)ᵀ") + } + } + + @Test + fun offsets_are_honoured_and_the_rest_of_the_arrays_is_left_alone() { + val rng = Random(5) + val m = 2; val n = 9; val k = 12 + val x = FloatArray(3 + m * k) { rng.nextFloat() } + val w = FloatArray(2 + n * k) { rng.nextFloat() } + val out = FloatArray(1 + m * n + 4) { 7f } + NativeFp32MatmulWtKernel.matmulWt(x, 3, w, 2, out, 1, m, n, k) + assertClose(reference(x, 3, w, 2, m, n, k), out.copyOfRange(1, 1 + m * n), "offset wt") + assertEquals(7f, out[0]) + for (i in 1 + m * n until out.size) assertEquals(7f, out[i], "tail[$i] untouched") + } + + @Test + fun k_zero_zeroes_the_block_and_empty_shapes_are_no_ops() { + val out = FloatArray(6) { 1f } + NativeFp32MatmulWtKernel.matmulWt(FloatArray(0), 0, FloatArray(0), 0, out, 0, 2, 3, 0) + assertContentEquals(FloatArray(6), out) + val untouched = FloatArray(4) { 2f } + NativeFp32MatmulWtKernel.matmulWt(FloatArray(0), 0, FloatArray(8), 0, untouched, 0, 0, 4, 2) + NativeFp32MatmulWtKernel.matmulWt(FloatArray(4), 0, FloatArray(0), 0, untouched, 0, 2, 0, 2) + assertContentEquals(FloatArray(4) { 2f }, untouched) + } + + @Test + fun segment_face_is_bit_identical_to_the_heap_face() { + val rng = Random(11) + val m = 4; val n = 530; val k = 96 + val x = FloatArray(m * k) { rng.nextFloat() - 0.5f } + val w = FloatArray(n * k) { rng.nextFloat() - 0.5f } + val viaHeap = FloatArray(m * n) + NativeFp32MatmulWtKernel.matmulWt(x, 0, w, 0, viaHeap, 0, m, n, k) + Arena.ofConfined().use { arena -> + // The weight sits 16 bytes into a larger segment: the offset is honoured and no copy is made. + val seg = arena.allocate(16L + n.toLong() * k * java.lang.Float.BYTES, 64L) + MemorySegment.copy(w, 0, seg, ValueLayout.JAVA_FLOAT, 16L, n * k) + val viaSegment = FloatArray(m * n) + NativeFp32MatmulWtKernel.matmulWt(x, 0, seg, 16L, viaSegment, 0, m, n, k) + assertContentEquals(viaHeap, viaSegment) + } + assertClose(reference(x, 0, w, 0, m, n, k), viaHeap, "threaded wt") + } + + @Test + fun the_provider_exposes_both_faces() { + assertTrue(NativeKernelProvider.isAvailable()) + assertNotNull(NativeKernelProvider.matmulFp32Wt()) + assertNotNull(NativeKernelProvider.matmulFp32WtMemSeg()) + assertTrue(NativeKernelProvider.supports("matmul_wt", listOf("Float32", "Float32"))) + assertTrue(!ScalarKernelProvider.supports("matmul_wt", listOf("Float32", "Float32")), "the scalar floor has no direct kernel") + } +} diff --git a/skainet-backends/skainet-backend-native-cpu/src/jvmTest/kotlin/sk/ainet/exec/kernel/NativeFusedKernelsTest.kt b/skainet-backends/skainet-backend-native-cpu/src/jvmTest/kotlin/sk/ainet/exec/kernel/NativeFusedKernelsTest.kt new file mode 100644 index 000000000..614c81e86 --- /dev/null +++ b/skainet-backends/skainet-backend-native-cpu/src/jvmTest/kotlin/sk/ainet/exec/kernel/NativeFusedKernelsTest.kt @@ -0,0 +1,219 @@ +package sk.ainet.exec.kernel + +import kotlin.math.abs +import kotlin.math.sqrt +import kotlin.test.AfterTest +import kotlin.test.BeforeTest +import kotlin.test.Test +import kotlin.test.assertContentEquals +import kotlin.test.assertEquals +import kotlin.test.assertTrue +import sk.ainet.backend.api.kernel.KernelRegistry +import sk.ainet.backend.api.kernel.KernelServiceLoader +import sk.ainet.context.DirectCpuExecutionContext +import sk.ainet.context.schedule.Schedule +import sk.ainet.exec.tensor.ops.DefaultCpuOps +import sk.ainet.lang.nn.normalization.LayerNormalization +import sk.ainet.lang.tensor.Shape +import sk.ainet.lang.tensor.Tensor +import sk.ainet.lang.tensor.data.DenseTensorDataFactory +import sk.ainet.lang.tensor.ops.AttentionMask +import sk.ainet.lang.tensor.ops.FusedNormOps +import sk.ainet.lang.types.DType +import sk.ainet.lang.types.FP32 +import sk.ainet.lang.types.Int32 + +/** + * The fused row kernels on the JVM ops layer (`f32_fused.c` through FFM): softmax / log-softmax, + * the normalisations behind `LayerNormalization` and the `FusedNormOps` hook, and scaled + * dot-product attention with every mask rule — all routed to the native kernels when the FFM + * provider is loaded, and agreeing with the common scalar implementation within the rounding of a + * vectorised accumulation. Routing is proved by bit-identity with the kernel called directly. + */ +class NativeFusedKernelsTest { + + @BeforeTest + fun setUp() { + KernelRegistry.clearForTesting() + KernelServiceLoader.installAll() + assertTrue(NativeFp32SoftmaxKernel.isAvailable() && NativeFp32NormKernel.isAvailable() && NativeFp32AttentionKernel.isAvailable(), + "bundled libskainet_kernels must resolve the fused kernels") + } + + @AfterTest + fun tearDown() = KernelRegistry.clearForTesting() + + /** The common scalar implementation, the reference every tier is checked against. */ + private val reference = DefaultCpuOps(DenseTensorDataFactory(), Schedule.Sequential) + + private fun fill(size: Int, seed: Int, scale: Float = 1f) = FloatArray(size) { i -> (((i * 31 + seed * 17) % 23) - 11) / 11f * scale } + + private fun floats(t: Tensor<*, *>) = t.data.copyToFloatArray() + + private fun assertClose(expected: FloatArray, actual: FloatArray, what: String, rel: Float = 1e-5f, abs: Float = 1e-6f) { + assertEquals(expected.size, actual.size, "$what size") + var maxAbs = 0f + for (v in expected) maxAbs = maxOf(maxAbs, abs(v)) + val tol = maxAbs * rel + abs + for (i in expected.indices) { + assertTrue(abs(expected[i] - actual[i]) <= tol, "$what[$i]: ${actual[i]} vs ${expected[i]} (tol $tol)") + } + } + + @Test + fun softmax_and_log_softmax_run_native_and_match_the_common_loop() { + val ctx = DirectCpuExecutionContext() + val rows = 64; val cols = 256 + val arr = fill(rows * cols, 1, 4f) + val x = ctx.fromFloatArray(Shape(rows, cols), FP32::class, arr) + val sm = floats(ctx.ops.softmax(x, -1)) + val lsm = floats(ctx.ops.logSoftmax(x, -1)) + assertClose(floats(reference.softmax(x, -1)), sm, "softmax") + assertClose(floats(reference.logSoftmax(x, -1)), lsm, "logSoftmax", abs = 1e-5f) + // Routed to the kernel: bit-identical to calling it directly. + val direct = FloatArray(rows * cols) + NativeFp32SoftmaxKernel.softmaxRows(arr, 0, direct, 0, rows, cols, log = false) + assertContentEquals(direct, sm, "softmax went through the native kernel") + NativeFp32SoftmaxKernel.softmaxRows(arr, 0, direct, 0, rows, cols, log = true) + assertContentEquals(direct, lsm, "logSoftmax went through the native kernel") + // Every row sums to one. + for (r in 0 until rows) { + var s = 0.0 + for (c in 0 until cols) s += sm[r * cols + c] + assertTrue(abs(s - 1.0) < 1e-4, "row $r sums to $s") + } + // A small tensor and a non-last dim keep the common loop; the answer is the same. + val small = ctx.fromFloatArray(Shape(4, 8), FP32::class, fill(32, 2)) + assertClose(floats(reference.softmax(small, -1)), floats(ctx.ops.softmax(small, -1)), "small softmax") + assertClose(floats(reference.softmax(x, 0)), floats(ctx.ops.softmax(x, 0)), "softmax over dim 0") + } + + @Test + fun layer_norm_runs_fused_through_the_module_and_matches_the_decomposition() { + val ctx = DirectCpuExecutionContext() + assertTrue(ctx.ops is FusedNormOps, "the JVM ops carry the fused-norm hook") + val rows = 6; val cols = 768 + val xArr = fill(rows * cols, 3, 2f) + val gArr = FloatArray(cols) { 0.5f + (it % 7) * 0.1f } + val bArr = FloatArray(cols) { ((it % 5) - 2) * 0.05f } + val x = ctx.fromFloatArray(Shape(rows, cols), FP32::class, xArr) + val gamma = ctx.fromFloatArray(Shape(cols), FP32::class, gArr) + val beta = ctx.fromFloatArray(Shape(cols), FP32::class, bArr) + val ln = LayerNormalization(intArrayOf(cols), eps = 1e-5, elementwiseAffine = true, name = "ln", initGamma = gamma, initBeta = beta) + val got = floats(ln.forward(x, ctx)) + // The decomposition's arithmetic, in double. + val ref = FloatArray(rows * cols) + for (r in 0 until rows) { + var mean = 0.0 + for (c in 0 until cols) mean += xArr[r * cols + c] + mean /= cols + var v = 0.0 + for (c in 0 until cols) { val d = xArr[r * cols + c] - mean; v += d * d } + v /= cols + val inv = 1.0 / sqrt(v + 1e-5) + for (c in 0 until cols) ref[r * cols + c] = ((xArr[r * cols + c] - mean) * inv * gArr[c] + bArr[c]).toFloat() + } + assertClose(ref, got, "layerNorm", rel = 1e-4f, abs = 1e-4f) + // The hook answered, not the decomposition: bit-identical to the kernel called directly. + val direct = FloatArray(rows * cols) + NativeFp32NormKernel.layerNorm(xArr, 0, gArr, bArr, direct, 0, rows, cols, 1e-5f) + assertContentEquals(direct, got, "layerNorm went through the native kernel") + + // RMS norm through the hook. + val rms = floats(checkNotNull((ctx.ops as FusedNormOps).rmsNorm(x, gamma, 1e-6f, 1)) { "rmsNorm must be served" }) + val rmsRef = FloatArray(rows * cols) + for (r in 0 until rows) { + var ms = 0.0 + for (c in 0 until cols) { val d = xArr[r * cols + c].toDouble(); ms += d * d } + ms /= cols + val inv = 1.0 / sqrt(ms + 1e-6) + for (c in 0 until cols) rmsRef[r * cols + c] = (xArr[r * cols + c] * inv * gArr[c]).toFloat() + } + assertClose(rmsRef, rms, "rmsNorm", rel = 1e-4f, abs = 1e-4f) + + // Two trailing dims and no affine: one row of 32 per [2, 3] prefix. + val x3Arr = fill(2 * 3 * 4 * 8, 5) + val x3 = ctx.fromFloatArray(Shape(2, 3, 4, 8), FP32::class, x3Arr) + val ln2 = LayerNormalization(intArrayOf(4, 8), eps = 1e-5, elementwiseAffine = false, name = "ln2") + val got3 = floats(ln2.forward(x3, ctx)) + val ref3 = FloatArray(x3Arr.size) + for (r in 0 until 6) { + var mean = 0.0 + for (c in 0 until 32) mean += x3Arr[r * 32 + c] + mean /= 32 + var v = 0.0 + for (c in 0 until 32) { val d = x3Arr[r * 32 + c] - mean; v += d * d } + v /= 32 + val inv = 1.0 / sqrt(v + 1e-5) + for (c in 0 until 32) ref3[r * 32 + c] = ((x3Arr[r * 32 + c] - mean) * inv).toFloat() + } + assertClose(ref3, got3, "layerNorm over two dims", rel = 1e-4f, abs = 1e-4f) + } + + private class Inputs(val q: Tensor, val k: Tensor, val v: Tensor) + + private fun inputs(ctx: DirectCpuExecutionContext, batch: Int, heads: Int, kvHeads: Int, seqQ: Int, seqKV: Int, headDim: Int, seed: Int = 0) = Inputs( + ctx.fromFloatArray(Shape(batch, heads, seqQ, headDim), FP32::class, fill(batch * heads * seqQ * headDim, 1 + seed)), + ctx.fromFloatArray(Shape(batch, kvHeads, seqKV, headDim), FP32::class, fill(batch * kvHeads * seqKV * headDim, 2 + seed)), + ctx.fromFloatArray(Shape(batch, kvHeads, seqKV, headDim), FP32::class, fill(batch * kvHeads * seqKV * headDim, 3 + seed)), + ) + + @Suppress("UNCHECKED_CAST") + private fun lengthsTensor(ctx: DirectCpuExecutionContext, lengths: IntArray): Tensor = + ctx.fromIntArray(Shape(lengths.size), Int32::class, lengths) as Tensor + + @Test + fun attention_runs_native_with_every_mask_rule_and_matches_the_common_kernel() { + val ctx = DirectCpuExecutionContext() + val batch = 3 + for ((heads, kvHeads) in listOf(4 to 4, 4 to 2)) for ((seqQ, seqKV) in listOf(6 to 6, 3 to 10, 1 to 7)) for (causal in listOf(false, true)) { + val x = inputs(ctx, batch, heads, kvHeads, seqQ, seqKV, 8) + for (lengths in listOf(null, intArrayOf(seqKV, (seqKV + 1) / 2, 1))) for ((window, rightContext) in listOf(0 to 0, 3 to 0, 4 to 2)) { + val what = "heads=$heads kv=$kvHeads seqQ=$seqQ seqKV=$seqKV causal=$causal lengths=${lengths?.toList()} window=$window rc=$rightContext" + val kl = lengths?.let { lengthsTensor(ctx, it) } + val got = ctx.ops.scaledDotProductAttention(x.q, x.k, x.v, mask = null, scale = 0f, causal = causal, keyLengths = kl, window = window, rightContext = rightContext) + assertEquals(Shape(batch, heads, seqQ, 8), got.shape, what) + val want = reference.scaledDotProductAttention(x.q, x.k, x.v, mask = null, scale = 0f, causal = causal, keyLengths = kl, window = window, rightContext = rightContext) + assertClose(floats(want), floats(got), what, rel = 1e-4f, abs = 1e-5f) + // The structured rules equal the dense -1e30 mask on the fused kernel too. + val dense = ctx.fromFloatArray(Shape(batch, 1, seqQ, seqKV), FP32::class, AttentionMask.additive(batch, seqQ, seqKV, causal, window, rightContext, lengths)) + val masked = ctx.ops.scaledDotProductAttention(x.q, x.k, x.v, mask = dense, scale = 0f, causal = false) + assertClose(floats(got), floats(masked), "$what vs dense mask", rel = 1e-5f, abs = 1e-6f) + } + } + // Mask broadcast shapes: a per-key bias and a [seqQ, seqKV] matrix. + val x = inputs(ctx, 2, 4, 2, 5, 9, 16, seed = 4) + for (mask in listOf( + ctx.fromFloatArray(Shape(1, 1, 1, 9), FP32::class, fill(9, 6)), + ctx.fromFloatArray(Shape(5, 9), FP32::class, fill(45, 7)), + ctx.fromFloatArray(Shape(2, 4, 5, 9), FP32::class, fill(2 * 4 * 5 * 9, 8)), + )) { + val got = ctx.ops.scaledDotProductAttention(x.q, x.k, x.v, mask = mask, scale = 0.3f, causal = true) + val want = reference.scaledDotProductAttention(x.q, x.k, x.v, mask = mask, scale = 0.3f, causal = true) + assertClose(floats(want), floats(got), "mask ${mask.shape}", rel = 1e-4f, abs = 1e-5f) + } + } + + @Test + fun fully_padded_row_stays_finite_and_a_large_call_threads_in_the_native_pool() { + val ctx = DirectCpuExecutionContext() + val p = inputs(ctx, 2, 2, 2, 3, 5, 4) + val out = floats(ctx.ops.scaledDotProductAttention(p.q, p.k, p.v, keyLengths = lengthsTensor(ctx, intArrayOf(0, 5)))) + assertTrue(out.all { it.isFinite() }, "fully padded row produced a non-finite value") + val dense = ctx.fromFloatArray(Shape(2, 1, 3, 5), FP32::class, AttentionMask.additive(2, 3, 5, false, 0, 0, intArrayOf(0, 5))) + assertClose(floats(reference.scaledDotProductAttention(p.q, p.k, p.v, mask = dense)), out, "fully padded", rel = 1e-5f, abs = 1e-6f) + + // Laya's attention shape: 12 heads × 150 queries = 1800 rows, above the row pool's threading threshold. + val x = inputs(ctx, 1, 12, 12, 150, 150, 64, seed = 9) + val got = ctx.ops.scaledDotProductAttention(x.q, x.k, x.v, causal = true) + val want = reference.scaledDotProductAttention(x.q, x.k, x.v, causal = true) + assertClose(floats(want), floats(got), "laya shape", rel = 1e-4f, abs = 1e-5f) + // Routed to the kernel: bit-identical to calling it directly. + val direct = FloatArray(12 * 150 * 64) + NativeFp32AttentionKernel.sdpa( + floats(x.q), floats(x.k), floats(x.v), direct, + 1, 12, 12, 150, 150, 64, (1.0 / sqrt(64.0)).toFloat(), true, 0, 0, null, null, null, + ) + assertContentEquals(direct, floats(got), "attention went through the native kernel") + } +} diff --git a/skainet-backends/skainet-backend-wasm-simd/native/build-wasm.sh b/skainet-backends/skainet-backend-wasm-simd/native/build-wasm.sh index 0c9009de8..54943c8b6 100644 --- a/skainet-backends/skainet-backend-wasm-simd/native/build-wasm.sh +++ b/skainet-backends/skainet-backend-wasm-simd/native/build-wasm.sh @@ -11,8 +11,9 @@ # # The sources are skainet-backend-native-cpu/native/src — the same files the JVM FFM and Android JNI # packs are built from — compiled with Emscripten. Only the portable (#else) kernel bodies and the -# explicit __wasm_simd128__ bodies are used; NEON, Apple dispatch and the vendored ternary kernels -# are not part of this build. +# explicit __wasm_simd128__ bodies are used; NEON and Apple dispatch are not part of this build. +# The ternary kernels come from the portable branch of skainet_ternary_f32.c (the compile-time LUT +# with its SIMD128 body), not from the vendored NeoGPU file, which needs native pthreads and NEON. # # Toolchain: emsdk, pinned below. `EMSDK_PYTHON` must point at a Python >= 3.10 when the default # python3 is older (pyenv). Usage: @@ -44,11 +45,20 @@ SOURCES=( "$NATIVE/src/q4k_matmul.c" "$NATIVE/src/q5k_matmul.c" "$NATIVE/src/q6k_matmul.c" + "$NATIVE/src/skainet_ternary_f32.c" + "$NATIVE/src/bitnet_gemv.c" + "$NATIVE/src/f32_ops.c" + "$NATIVE/src/f32_fused.c" + "$NATIVE/src/f32_conv.c" "$NATIVE/src/skainet_row_threads.c" "$NATIVE/src/skainet_cpu_features.c" ) EXPORTS='_malloc,_free,_skainet_fp32_matmul,_skainet_fp32_matmul_wt,_skainet_bf16_matmul,_skainet_fp16_matmul' +EXPORTS="$EXPORTS,_skainet_ternary_f32_gemv,_skainet_ternary_lmhead_stage1,_skainet_bitnet_gemv_tq2_0" +EXPORTS="$EXPORTS,_skainet_f32_binary,_skainet_f32_unary,_skainet_f32_reduce_all,_skainet_f32_reduce_axis" +EXPORTS="$EXPORTS,_skainet_f32_softmax_rows,_skainet_f32_layer_norm,_skainet_f32_rms_norm,_skainet_f32_sdpa" +EXPORTS="$EXPORTS,_skainet_f32_conv" for f in q4_0 q5_0 q5_1 q8_0 q4k q5k q6k; do EXPORTS="$EXPORTS,_skainet_${f}_matmul,_skainet_${f}_matmul_rm" done diff --git a/skainet-backends/skainet-backend-wasm-simd/src/wasmJsMain/kotlin/sk/ainet/exec/wasm/SidecarArtifacts.kt b/skainet-backends/skainet-backend-wasm-simd/src/wasmJsMain/kotlin/sk/ainet/exec/wasm/SidecarArtifacts.kt index b33356e50..0f90b70d2 100644 --- a/skainet-backends/skainet-backend-wasm-simd/src/wasmJsMain/kotlin/sk/ainet/exec/wasm/SidecarArtifacts.kt +++ b/skainet-backends/skainet-backend-wasm-simd/src/wasmJsMain/kotlin/sk/ainet/exec/wasm/SidecarArtifacts.kt @@ -1,13 +1,13 @@ // GENERATED by native/build-wasm.sh (emsdk 4.0.14) -- do not edit. // SKEEP-007 Tier 1: the SIMD128 kernel sidecar, embedded so every consumer of this module carries it. -// sha256 st: c1a6213df16738714532551430ae508eb29154459336c5069bf27be3205ba3f3 -// sha256 mt: 590e0bfe8a35c5fa259305688d3c964c5bf3013fab611b5557dcce65d3b8b097 +// sha256 st: 7cb490e5151ae1b7e4b3f70a62c5759d737d1127dc110b4fba1127dac6a85d41 +// sha256 mt: c1e713fada16ef008aaa500f4411d1e0d6c90556ece201f30fdbf3d9a9f68777 package sk.ainet.exec.wasm internal object SidecarArtifacts { const val EMSDK: String = "4.0.14" - const val ST_MJS: String = "YXN5bmMgZnVuY3Rpb24gY3JlYXRlU2thaW5ldEtlcm5lbHMobW9kdWxlQXJnPXt9KXt2YXIgbW9kdWxlUnRuO3ZhciBNb2R1bGU9bW9kdWxlQXJnO3ZhciBFTlZJUk9OTUVOVF9JU19XRUI9dHlwZW9mIHdpbmRvdz09Im9iamVjdCI7dmFyIEVOVklST05NRU5UX0lTX1dPUktFUj10eXBlb2YgV29ya2VyR2xvYmFsU2NvcGUhPSJ1bmRlZmluZWQiO3ZhciBFTlZJUk9OTUVOVF9JU19OT0RFPXR5cGVvZiBwcm9jZXNzPT0ib2JqZWN0IiYmcHJvY2Vzcy52ZXJzaW9ucz8ubm9kZSYmcHJvY2Vzcy50eXBlIT0icmVuZGVyZXIiO2lmKEVOVklST05NRU5UX0lTX05PREUpe2NvbnN0e2NyZWF0ZVJlcXVpcmV9PWF3YWl0IGltcG9ydCgibW9kdWxlIik7dmFyIHJlcXVpcmU9Y3JlYXRlUmVxdWlyZShpbXBvcnQubWV0YS51cmwuc3RhcnRzV2l0aCgiZmlsZToiKT9pbXBvcnQubWV0YS51cmw6cHJvY2Vzcy5jd2QoKSsiLyIpfXZhciBhcmd1bWVudHNfPVtdO3ZhciB0aGlzUHJvZ3JhbT0iLi90aGlzLnByb2dyYW0iO3ZhciBxdWl0Xz0oc3RhdHVzLHRvVGhyb3cpPT57dGhyb3cgdG9UaHJvd307dmFyIF9zY3JpcHROYW1lPWltcG9ydC5tZXRhLnVybDt2YXIgc2NyaXB0RGlyZWN0b3J5PSIiO2Z1bmN0aW9uIGxvY2F0ZUZpbGUocGF0aCl7aWYoTW9kdWxlWyJsb2NhdGVGaWxlIl0pe3JldHVybiBNb2R1bGVbImxvY2F0ZUZpbGUiXShwYXRoLHNjcmlwdERpcmVjdG9yeSl9cmV0dXJuIHNjcmlwdERpcmVjdG9yeStwYXRofXZhciByZWFkQXN5bmMscmVhZEJpbmFyeTtpZihFTlZJUk9OTUVOVF9JU19OT0RFKXt2YXIgZnM9cmVxdWlyZSgiZnMiKTtpZihfc2NyaXB0TmFtZS5zdGFydHNXaXRoKCJmaWxlOiIpKXtzY3JpcHREaXJlY3Rvcnk9cmVxdWlyZSgicGF0aCIpLmRpcm5hbWUocmVxdWlyZSgidXJsIikuZmlsZVVSTFRvUGF0aChfc2NyaXB0TmFtZSkpKyIvIn1yZWFkQmluYXJ5PWZpbGVuYW1lPT57ZmlsZW5hbWU9aXNGaWxlVVJJKGZpbGVuYW1lKT9uZXcgVVJMKGZpbGVuYW1lKTpmaWxlbmFtZTt2YXIgcmV0PWZzLnJlYWRGaWxlU3luYyhmaWxlbmFtZSk7cmV0dXJuIHJldH07cmVhZEFzeW5jPWFzeW5jKGZpbGVuYW1lLGJpbmFyeT10cnVlKT0+e2ZpbGVuYW1lPWlzRmlsZVVSSShmaWxlbmFtZSk/bmV3IFVSTChmaWxlbmFtZSk6ZmlsZW5hbWU7dmFyIHJldD1mcy5yZWFkRmlsZVN5bmMoZmlsZW5hbWUsYmluYXJ5P3VuZGVmaW5lZDoidXRmOCIpO3JldHVybiByZXR9O2lmKHByb2Nlc3MuYXJndi5sZW5ndGg+MSl7dGhpc1Byb2dyYW09cHJvY2Vzcy5hcmd2WzFdLnJlcGxhY2UoL1xcL2csIi8iKX1hcmd1bWVudHNfPXByb2Nlc3MuYXJndi5zbGljZSgyKTtxdWl0Xz0oc3RhdHVzLHRvVGhyb3cpPT57cHJvY2Vzcy5leGl0Q29kZT1zdGF0dXM7dGhyb3cgdG9UaHJvd319ZWxzZSBpZihFTlZJUk9OTUVOVF9JU19XRUJ8fEVOVklST05NRU5UX0lTX1dPUktFUil7dHJ5e3NjcmlwdERpcmVjdG9yeT1uZXcgVVJMKCIuIixfc2NyaXB0TmFtZSkuaHJlZn1jYXRjaHt9e2lmKEVOVklST05NRU5UX0lTX1dPUktFUil7cmVhZEJpbmFyeT11cmw9Pnt2YXIgeGhyPW5ldyBYTUxIdHRwUmVxdWVzdDt4aHIub3BlbigiR0VUIix1cmwsZmFsc2UpO3hoci5yZXNwb25zZVR5cGU9ImFycmF5YnVmZmVyIjt4aHIuc2VuZChudWxsKTtyZXR1cm4gbmV3IFVpbnQ4QXJyYXkoeGhyLnJlc3BvbnNlKX19cmVhZEFzeW5jPWFzeW5jIHVybD0+e3ZhciByZXNwb25zZT1hd2FpdCBmZXRjaCh1cmwse2NyZWRlbnRpYWxzOiJzYW1lLW9yaWdpbiJ9KTtpZihyZXNwb25zZS5vayl7cmV0dXJuIHJlc3BvbnNlLmFycmF5QnVmZmVyKCl9dGhyb3cgbmV3IEVycm9yKHJlc3BvbnNlLnN0YXR1cysiIDogIityZXNwb25zZS51cmwpfX19ZWxzZXt9dmFyIG91dD1jb25zb2xlLmxvZy5iaW5kKGNvbnNvbGUpO3ZhciBlcnI9Y29uc29sZS5lcnJvci5iaW5kKGNvbnNvbGUpO3ZhciB3YXNtQmluYXJ5O3ZhciBBQk9SVD1mYWxzZTt2YXIgaXNGaWxlVVJJPWZpbGVuYW1lPT5maWxlbmFtZS5zdGFydHNXaXRoKCJmaWxlOi8vIik7dmFyIHJlYWR5UHJvbWlzZVJlc29sdmUscmVhZHlQcm9taXNlUmVqZWN0O3ZhciB3YXNtTWVtb3J5O3ZhciBIRUFQOCxIRUFQVTgsSEVBUDE2LEhFQVBVMTYsSEVBUDMyLEhFQVBVMzIsSEVBUEYzMixIRUFQRjY0O3ZhciBIRUFQNjQsSEVBUFU2NDt2YXIgcnVudGltZUluaXRpYWxpemVkPWZhbHNlO2Z1bmN0aW9uIHVwZGF0ZU1lbW9yeVZpZXdzKCl7dmFyIGI9d2FzbU1lbW9yeS5idWZmZXI7SEVBUDg9bmV3IEludDhBcnJheShiKTtIRUFQMTY9bmV3IEludDE2QXJyYXkoYik7SEVBUFU4PW5ldyBVaW50OEFycmF5KGIpO0hFQVBVMTY9bmV3IFVpbnQxNkFycmF5KGIpO0hFQVAzMj1uZXcgSW50MzJBcnJheShiKTtIRUFQVTMyPW5ldyBVaW50MzJBcnJheShiKTtIRUFQRjMyPW5ldyBGbG9hdDMyQXJyYXkoYik7SEVBUEY2ND1uZXcgRmxvYXQ2NEFycmF5KGIpO0hFQVA2ND1uZXcgQmlnSW50NjRBcnJheShiKTtIRUFQVTY0PW5ldyBCaWdVaW50NjRBcnJheShiKX1mdW5jdGlvbiBwcmVSdW4oKXtpZihNb2R1bGVbInByZVJ1biJdKXtpZih0eXBlb2YgTW9kdWxlWyJwcmVSdW4iXT09ImZ1bmN0aW9uIilNb2R1bGVbInByZVJ1biJdPVtNb2R1bGVbInByZVJ1biJdXTt3aGlsZShNb2R1bGVbInByZVJ1biJdLmxlbmd0aCl7YWRkT25QcmVSdW4oTW9kdWxlWyJwcmVSdW4iXS5zaGlmdCgpKX19Y2FsbFJ1bnRpbWVDYWxsYmFja3Mob25QcmVSdW5zKX1mdW5jdGlvbiBpbml0UnVudGltZSgpe3J1bnRpbWVJbml0aWFsaXplZD10cnVlO3dhc21FeHBvcnRzWyJjIl0oKX1mdW5jdGlvbiBwb3N0UnVuKCl7aWYoTW9kdWxlWyJwb3N0UnVuIl0pe2lmKHR5cGVvZiBNb2R1bGVbInBvc3RSdW4iXT09ImZ1bmN0aW9uIilNb2R1bGVbInBvc3RSdW4iXT1bTW9kdWxlWyJwb3N0UnVuIl1dO3doaWxlKE1vZHVsZVsicG9zdFJ1biJdLmxlbmd0aCl7YWRkT25Qb3N0UnVuKE1vZHVsZVsicG9zdFJ1biJdLnNoaWZ0KCkpfX1jYWxsUnVudGltZUNhbGxiYWNrcyhvblBvc3RSdW5zKX1mdW5jdGlvbiBhYm9ydCh3aGF0KXtNb2R1bGVbIm9uQWJvcnQiXT8uKHdoYXQpO3doYXQ9IkFib3J0ZWQoIit3aGF0KyIpIjtlcnIod2hhdCk7QUJPUlQ9dHJ1ZTt3aGF0Kz0iLiBCdWlsZCB3aXRoIC1zQVNTRVJUSU9OUyBmb3IgbW9yZSBpbmZvLiI7dmFyIGU9bmV3IFdlYkFzc2VtYmx5LlJ1bnRpbWVFcnJvcih3aGF0KTtyZWFkeVByb21pc2VSZWplY3Q/LihlKTt0aHJvdyBlfXZhciB3YXNtQmluYXJ5RmlsZTtmdW5jdGlvbiBmaW5kV2FzbUJpbmFyeSgpe2lmKE1vZHVsZVsibG9jYXRlRmlsZSJdKXtyZXR1cm4gbG9jYXRlRmlsZSgic2thaW5ldF9rZXJuZWxzX3N0Lndhc20iKX10cnl7cmV0dXJuIG5ldyBVUkwoInNrYWluZXRfa2VybmVsc19zdC53YXNtIixpbXBvcnQubWV0YS51cmwpLmhyZWZ9Y2F0Y2goZSl7cmV0dXJuICJza2FpbmV0X2tlcm5lbHNfc3Qud2FzbSJ9fWZ1bmN0aW9uIGdldEJpbmFyeVN5bmMoZmlsZSl7aWYoZmlsZT09d2FzbUJpbmFyeUZpbGUmJndhc21CaW5hcnkpe3JldHVybiBuZXcgVWludDhBcnJheSh3YXNtQmluYXJ5KX1pZihyZWFkQmluYXJ5KXtyZXR1cm4gcmVhZEJpbmFyeShmaWxlKX10aHJvdyJib3RoIGFzeW5jIGFuZCBzeW5jIGZldGNoaW5nIG9mIHRoZSB3YXNtIGZhaWxlZCJ9YXN5bmMgZnVuY3Rpb24gZ2V0V2FzbUJpbmFyeShiaW5hcnlGaWxlKXtpZighd2FzbUJpbmFyeSl7dHJ5e3ZhciByZXNwb25zZT1hd2FpdCByZWFkQXN5bmMoYmluYXJ5RmlsZSk7cmV0dXJuIG5ldyBVaW50OEFycmF5KHJlc3BvbnNlKX1jYXRjaHt9fXJldHVybiBnZXRCaW5hcnlTeW5jKGJpbmFyeUZpbGUpfWFzeW5jIGZ1bmN0aW9uIGluc3RhbnRpYXRlQXJyYXlCdWZmZXIoYmluYXJ5RmlsZSxpbXBvcnRzKXt0cnl7dmFyIGJpbmFyeT1hd2FpdCBnZXRXYXNtQmluYXJ5KGJpbmFyeUZpbGUpO3ZhciBpbnN0YW5jZT1hd2FpdCBXZWJBc3NlbWJseS5pbnN0YW50aWF0ZShiaW5hcnksaW1wb3J0cyk7cmV0dXJuIGluc3RhbmNlfWNhdGNoKHJlYXNvbil7ZXJyKGBmYWlsZWQgdG8gYXN5bmNocm9ub3VzbHkgcHJlcGFyZSB3YXNtOiAke3JlYXNvbn1gKTthYm9ydChyZWFzb24pfX1hc3luYyBmdW5jdGlvbiBpbnN0YW50aWF0ZUFzeW5jKGJpbmFyeSxiaW5hcnlGaWxlLGltcG9ydHMpe2lmKCFiaW5hcnkmJiFFTlZJUk9OTUVOVF9JU19OT0RFKXt0cnl7dmFyIHJlc3BvbnNlPWZldGNoKGJpbmFyeUZpbGUse2NyZWRlbnRpYWxzOiJzYW1lLW9yaWdpbiJ9KTt2YXIgaW5zdGFudGlhdGlvblJlc3VsdD1hd2FpdCBXZWJBc3NlbWJseS5pbnN0YW50aWF0ZVN0cmVhbWluZyhyZXNwb25zZSxpbXBvcnRzKTtyZXR1cm4gaW5zdGFudGlhdGlvblJlc3VsdH1jYXRjaChyZWFzb24pe2Vycihgd2FzbSBzdHJlYW1pbmcgY29tcGlsZSBmYWlsZWQ6ICR7cmVhc29ufWApO2VycigiZmFsbGluZyBiYWNrIHRvIEFycmF5QnVmZmVyIGluc3RhbnRpYXRpb24iKX19cmV0dXJuIGluc3RhbnRpYXRlQXJyYXlCdWZmZXIoYmluYXJ5RmlsZSxpbXBvcnRzKX1mdW5jdGlvbiBnZXRXYXNtSW1wb3J0cygpe3JldHVybnthOndhc21JbXBvcnRzfX1hc3luYyBmdW5jdGlvbiBjcmVhdGVXYXNtKCl7ZnVuY3Rpb24gcmVjZWl2ZUluc3RhbmNlKGluc3RhbmNlLG1vZHVsZSl7d2FzbUV4cG9ydHM9aW5zdGFuY2UuZXhwb3J0czt3YXNtRXhwb3J0cz1hcHBseVNpZ25hdHVyZUNvbnZlcnNpb25zKHdhc21FeHBvcnRzKTt3YXNtTWVtb3J5PXdhc21FeHBvcnRzWyJiIl07TW9kdWxlWyJ3YXNtTWVtb3J5Il09d2FzbU1lbW9yeTt1cGRhdGVNZW1vcnlWaWV3cygpO2Fzc2lnbldhc21FeHBvcnRzKHdhc21FeHBvcnRzKTtyZXR1cm4gd2FzbUV4cG9ydHN9ZnVuY3Rpb24gcmVjZWl2ZUluc3RhbnRpYXRpb25SZXN1bHQocmVzdWx0KXtyZXR1cm4gcmVjZWl2ZUluc3RhbmNlKHJlc3VsdFsiaW5zdGFuY2UiXSl9dmFyIGluZm89Z2V0V2FzbUltcG9ydHMoKTtpZihNb2R1bGVbImluc3RhbnRpYXRlV2FzbSJdKXtyZXR1cm4gbmV3IFByb21pc2UoKHJlc29sdmUscmVqZWN0KT0+e01vZHVsZVsiaW5zdGFudGlhdGVXYXNtIl0oaW5mbywobW9kLGluc3QpPT57cmVzb2x2ZShyZWNlaXZlSW5zdGFuY2UobW9kLGluc3QpKX0pfSl9d2FzbUJpbmFyeUZpbGU/Pz1maW5kV2FzbUJpbmFyeSgpO3ZhciByZXN1bHQ9YXdhaXQgaW5zdGFudGlhdGVBc3luYyh3YXNtQmluYXJ5LHdhc21CaW5hcnlGaWxlLGluZm8pO3ZhciBleHBvcnRzPXJlY2VpdmVJbnN0YW50aWF0aW9uUmVzdWx0KHJlc3VsdCk7cmV0dXJuIGV4cG9ydHN9Y2xhc3MgRXhpdFN0YXR1c3tuYW1lPSJFeGl0U3RhdHVzIjtjb25zdHJ1Y3RvcihzdGF0dXMpe3RoaXMubWVzc2FnZT1gUHJvZ3JhbSB0ZXJtaW5hdGVkIHdpdGggZXhpdCgke3N0YXR1c30pYDt0aGlzLnN0YXR1cz1zdGF0dXN9fXZhciBjYWxsUnVudGltZUNhbGxiYWNrcz1jYWxsYmFja3M9Pnt3aGlsZShjYWxsYmFja3MubGVuZ3RoPjApe2NhbGxiYWNrcy5zaGlmdCgpKE1vZHVsZSl9fTt2YXIgb25Qb3N0UnVucz1bXTt2YXIgYWRkT25Qb3N0UnVuPWNiPT5vblBvc3RSdW5zLnB1c2goY2IpO3ZhciBvblByZVJ1bnM9W107dmFyIGFkZE9uUHJlUnVuPWNiPT5vblByZVJ1bnMucHVzaChjYik7dmFyIG5vRXhpdFJ1bnRpbWU9dHJ1ZTt2YXIgZ2V0SGVhcE1heD0oKT0+NDI5NDkwMTc2MDt2YXIgYWxpZ25NZW1vcnk9KHNpemUsYWxpZ25tZW50KT0+TWF0aC5jZWlsKHNpemUvYWxpZ25tZW50KSphbGlnbm1lbnQ7dmFyIGdyb3dNZW1vcnk9c2l6ZT0+e3ZhciBvbGRIZWFwU2l6ZT13YXNtTWVtb3J5LmJ1ZmZlci5ieXRlTGVuZ3RoO3ZhciBwYWdlcz0oc2l6ZS1vbGRIZWFwU2l6ZSs2NTUzNSkvNjU1MzZ8MDt0cnl7d2FzbU1lbW9yeS5ncm93KHBhZ2VzKTt1cGRhdGVNZW1vcnlWaWV3cygpO3JldHVybiAxfWNhdGNoKGUpe319O2Z1bmN0aW9uIF9lbXNjcmlwdGVuX3Jlc2l6ZV9oZWFwKHJlcXVlc3RlZFNpemUpe3JlcXVlc3RlZFNpemU+Pj49MDt2YXIgb2xkU2l6ZT1IRUFQVTgubGVuZ3RoO3ZhciBtYXhIZWFwU2l6ZT1nZXRIZWFwTWF4KCk7aWYocmVxdWVzdGVkU2l6ZT5tYXhIZWFwU2l6ZSl7cmV0dXJuIGZhbHNlfWZvcih2YXIgY3V0RG93bj0xO2N1dERvd248PTQ7Y3V0RG93bio9Mil7dmFyIG92ZXJHcm93bkhlYXBTaXplPW9sZFNpemUqKDErLjIvY3V0RG93bik7b3Zlckdyb3duSGVhcFNpemU9TWF0aC5taW4ob3Zlckdyb3duSGVhcFNpemUscmVxdWVzdGVkU2l6ZSsxMDA2NjMyOTYpO3ZhciBuZXdTaXplPU1hdGgubWluKG1heEhlYXBTaXplLGFsaWduTWVtb3J5KE1hdGgubWF4KHJlcXVlc3RlZFNpemUsb3Zlckdyb3duSGVhcFNpemUpLDY1NTM2KSk7dmFyIHJlcGxhY2VtZW50PWdyb3dNZW1vcnkobmV3U2l6ZSk7aWYocmVwbGFjZW1lbnQpe3JldHVybiB0cnVlfX1yZXR1cm4gZmFsc2V9e2lmKE1vZHVsZVsibm9FeGl0UnVudGltZSJdKW5vRXhpdFJ1bnRpbWU9TW9kdWxlWyJub0V4aXRSdW50aW1lIl07aWYoTW9kdWxlWyJwcmludCJdKW91dD1Nb2R1bGVbInByaW50Il07aWYoTW9kdWxlWyJwcmludEVyciJdKWVycj1Nb2R1bGVbInByaW50RXJyIl07aWYoTW9kdWxlWyJ3YXNtQmluYXJ5Il0pd2FzbUJpbmFyeT1Nb2R1bGVbIndhc21CaW5hcnkiXTtpZihNb2R1bGVbImFyZ3VtZW50cyJdKWFyZ3VtZW50c189TW9kdWxlWyJhcmd1bWVudHMiXTtpZihNb2R1bGVbInRoaXNQcm9ncmFtIl0pdGhpc1Byb2dyYW09TW9kdWxlWyJ0aGlzUHJvZ3JhbSJdO2lmKE1vZHVsZVsicHJlSW5pdCJdKXtpZih0eXBlb2YgTW9kdWxlWyJwcmVJbml0Il09PSJmdW5jdGlvbiIpTW9kdWxlWyJwcmVJbml0Il09W01vZHVsZVsicHJlSW5pdCJdXTt3aGlsZShNb2R1bGVbInByZUluaXQiXS5sZW5ndGg+MCl7TW9kdWxlWyJwcmVJbml0Il0uc2hpZnQoKSgpfX19TW9kdWxlWyJ3YXNtTWVtb3J5Il09d2FzbU1lbW9yeTt2YXIgX3NrYWluZXRfZnAzMl9tYXRtdWwsX3NrYWluZXRfZnAzMl9tYXRtdWxfd3QsX3NrYWluZXRfYmYxNl9tYXRtdWwsX3NrYWluZXRfZnAxNl9tYXRtdWwsX3NrYWluZXRfcTRfMF9tYXRtdWwsX3NrYWluZXRfcTRfMF9tYXRtdWxfcm0sX3NrYWluZXRfcTVfMF9tYXRtdWwsX21hbGxvYyxfZnJlZSxfc2thaW5ldF9xNV8wX21hdG11bF9ybSxfc2thaW5ldF9xNV8xX21hdG11bCxfc2thaW5ldF9xNV8xX21hdG11bF9ybSxfc2thaW5ldF9xOF8wX21hdG11bCxfc2thaW5ldF9xOF8wX21hdG11bF9ybSxfc2thaW5ldF9xNGtfbWF0bXVsLF9za2FpbmV0X3E0a19tYXRtdWxfcm0sX3NrYWluZXRfcTVrX21hdG11bCxfc2thaW5ldF9xNWtfbWF0bXVsX3JtLF9za2FpbmV0X3E2a19tYXRtdWwsX3NrYWluZXRfcTZrX21hdG11bF9ybTtmdW5jdGlvbiBhc3NpZ25XYXNtRXhwb3J0cyh3YXNtRXhwb3J0cyl7TW9kdWxlWyJfc2thaW5ldF9mcDMyX21hdG11bCJdPV9za2FpbmV0X2ZwMzJfbWF0bXVsPXdhc21FeHBvcnRzWyJkIl07TW9kdWxlWyJfc2thaW5ldF9mcDMyX21hdG11bF93dCJdPV9za2FpbmV0X2ZwMzJfbWF0bXVsX3d0PXdhc21FeHBvcnRzWyJlIl07TW9kdWxlWyJfc2thaW5ldF9iZjE2X21hdG11bCJdPV9za2FpbmV0X2JmMTZfbWF0bXVsPXdhc21FeHBvcnRzWyJmIl07TW9kdWxlWyJfc2thaW5ldF9mcDE2X21hdG11bCJdPV9za2FpbmV0X2ZwMTZfbWF0bXVsPXdhc21FeHBvcnRzWyJnIl07TW9kdWxlWyJfc2thaW5ldF9xNF8wX21hdG11bCJdPV9za2FpbmV0X3E0XzBfbWF0bXVsPXdhc21FeHBvcnRzWyJoIl07TW9kdWxlWyJfc2thaW5ldF9xNF8wX21hdG11bF9ybSJdPV9za2FpbmV0X3E0XzBfbWF0bXVsX3JtPXdhc21FeHBvcnRzWyJpIl07TW9kdWxlWyJfc2thaW5ldF9xNV8wX21hdG11bCJdPV9za2FpbmV0X3E1XzBfbWF0bXVsPXdhc21FeHBvcnRzWyJqIl07TW9kdWxlWyJfbWFsbG9jIl09X21hbGxvYz13YXNtRXhwb3J0c1siayJdO01vZHVsZVsiX2ZyZWUiXT1fZnJlZT13YXNtRXhwb3J0c1sibCJdO01vZHVsZVsiX3NrYWluZXRfcTVfMF9tYXRtdWxfcm0iXT1fc2thaW5ldF9xNV8wX21hdG11bF9ybT13YXNtRXhwb3J0c1sibSJdO01vZHVsZVsiX3NrYWluZXRfcTVfMV9tYXRtdWwiXT1fc2thaW5ldF9xNV8xX21hdG11bD13YXNtRXhwb3J0c1sibiJdO01vZHVsZVsiX3NrYWluZXRfcTVfMV9tYXRtdWxfcm0iXT1fc2thaW5ldF9xNV8xX21hdG11bF9ybT13YXNtRXhwb3J0c1sibyJdO01vZHVsZVsiX3NrYWluZXRfcThfMF9tYXRtdWwiXT1fc2thaW5ldF9xOF8wX21hdG11bD13YXNtRXhwb3J0c1sicCJdO01vZHVsZVsiX3NrYWluZXRfcThfMF9tYXRtdWxfcm0iXT1fc2thaW5ldF9xOF8wX21hdG11bF9ybT13YXNtRXhwb3J0c1sicSJdO01vZHVsZVsiX3NrYWluZXRfcTRrX21hdG11bCJdPV9za2FpbmV0X3E0a19tYXRtdWw9d2FzbUV4cG9ydHNbInIiXTtNb2R1bGVbIl9za2FpbmV0X3E0a19tYXRtdWxfcm0iXT1fc2thaW5ldF9xNGtfbWF0bXVsX3JtPXdhc21FeHBvcnRzWyJzIl07TW9kdWxlWyJfc2thaW5ldF9xNWtfbWF0bXVsIl09X3NrYWluZXRfcTVrX21hdG11bD13YXNtRXhwb3J0c1sidCJdO01vZHVsZVsiX3NrYWluZXRfcTVrX21hdG11bF9ybSJdPV9za2FpbmV0X3E1a19tYXRtdWxfcm09d2FzbUV4cG9ydHNbInUiXTtNb2R1bGVbIl9za2FpbmV0X3E2a19tYXRtdWwiXT1fc2thaW5ldF9xNmtfbWF0bXVsPXdhc21FeHBvcnRzWyJ2Il07TW9kdWxlWyJfc2thaW5ldF9xNmtfbWF0bXVsX3JtIl09X3NrYWluZXRfcTZrX21hdG11bF9ybT13YXNtRXhwb3J0c1sidyJdfXZhciB3YXNtSW1wb3J0cz17YTpfZW1zY3JpcHRlbl9yZXNpemVfaGVhcH07ZnVuY3Rpb24gYXBwbHlTaWduYXR1cmVDb252ZXJzaW9ucyh3YXNtRXhwb3J0cyl7d2FzbUV4cG9ydHM9T2JqZWN0LmFzc2lnbih7fSx3YXNtRXhwb3J0cyk7dmFyIG1ha2VXcmFwcGVyX3BwPWY9PmEwPT5mKGEwKT4+PjA7dmFyIG1ha2VXcmFwcGVyX3A9Zj0+KCk9PmYoKT4+PjA7d2FzbUV4cG9ydHNbImsiXT1tYWtlV3JhcHBlcl9wcCh3YXNtRXhwb3J0c1siayJdKTt3YXNtRXhwb3J0c1siX2Vtc2NyaXB0ZW5fc3RhY2tfYWxsb2MiXT1tYWtlV3JhcHBlcl9wcCh3YXNtRXhwb3J0c1siX2Vtc2NyaXB0ZW5fc3RhY2tfYWxsb2MiXSk7d2FzbUV4cG9ydHNbImVtc2NyaXB0ZW5fc3RhY2tfZ2V0X2N1cnJlbnQiXT1tYWtlV3JhcHBlcl9wKHdhc21FeHBvcnRzWyJlbXNjcmlwdGVuX3N0YWNrX2dldF9jdXJyZW50Il0pO3JldHVybiB3YXNtRXhwb3J0c31mdW5jdGlvbiBydW4oKXtwcmVSdW4oKTtmdW5jdGlvbiBkb1J1bigpe01vZHVsZVsiY2FsbGVkUnVuIl09dHJ1ZTtpZihBQk9SVClyZXR1cm47aW5pdFJ1bnRpbWUoKTtyZWFkeVByb21pc2VSZXNvbHZlPy4oTW9kdWxlKTtNb2R1bGVbIm9uUnVudGltZUluaXRpYWxpemVkIl0/LigpO3Bvc3RSdW4oKX1pZihNb2R1bGVbInNldFN0YXR1cyJdKXtNb2R1bGVbInNldFN0YXR1cyJdKCJSdW5uaW5nLi4uIik7c2V0VGltZW91dCgoKT0+e3NldFRpbWVvdXQoKCk9Pk1vZHVsZVsic2V0U3RhdHVzIl0oIiIpLDEpO2RvUnVuKCl9LDEpfWVsc2V7ZG9SdW4oKX19dmFyIHdhc21FeHBvcnRzO3dhc21FeHBvcnRzPWF3YWl0IChjcmVhdGVXYXNtKCkpO3J1bigpO2lmKHJ1bnRpbWVJbml0aWFsaXplZCl7bW9kdWxlUnRuPU1vZHVsZX1lbHNle21vZHVsZVJ0bj1uZXcgUHJvbWlzZSgocmVzb2x2ZSxyZWplY3QpPT57cmVhZHlQcm9taXNlUmVzb2x2ZT1yZXNvbHZlO3JlYWR5UHJvbWlzZVJlamVjdD1yZWplY3R9KX0KO3JldHVybiBtb2R1bGVSdG59ZXhwb3J0IGRlZmF1bHQgY3JlYXRlU2thaW5ldEtlcm5lbHM7Cg==" - const val ST_WASM: String = "AGFzbQEAAAABWAtgA39/fwBgCH9/f39/f39/AGABfwF/YAl/f39/f39/f38AYAx/f39/f39/f39/f38AYAAAYAF/AGAHf39/f39/fwBgAn9/AX1gA39/fQF9YAV/f39/fwACBwEBYQFhAAIDMjEAAgYCBwMDBQgJCgQABAADAgABAAEAAQABAAEAAQABAAEAAQABAAABAAEAAQABAAQFBAUBcAEVFQUHAQGACICABAYIAX8BQaCNBAsHWRYBYgIAAWMAMQFkADABZQAQAWYADgFnAAwBaAAuAWkALAFqACoBawAEAWwAAwFtACgBbgAlAW8AIwFwACEBcQAfAXIAHQFzABsBdAAZAXUAFwF2ABUBdwATCRoBAEEBCxQmDw0vLSspJyQiIB4cGhgWFBIIEQwBAQrr4wExlQMBBX8CQCACQQBKBEAgAkH/A00NAUGECCgCAEHft96aAUcEQBAIQYQIQd+33poBNgIACwJAQagJKAIAIgMEQEGkCSgCAEUNAQsMAgtBmAkgAzYCAEGICSACNgIAQYQJIAE2AgBBgAkgADYCAEGkCUEBNgIAQYwJIANBAWoiAzYCAEGUCUGUCSgCAEEBajYCAEGQCUEANgIAQZwJKAIABEBBjAkoAgAhAwsgAkGQCSgCACIESgRAIANBAXQhBgNAQZAJQcAAIAIgBGsgBm0iAyADQcAATBsgBGoiBUGQCSgCACIDIAMgBEYiBxs2AgAgBwR/IAEgBCAFIAIgAiAFShsgABEAAEGQCSgCAAUgAwsiBCACSA0ACwtBmAkoAgBBAEwhAEEAIQQCQANAIAANAUGYCSgCAEEATCIBDQEgAQ0BIAENASAEQQRqIgRBgIDAAEcNAAtBoAlBATYCAEGYCSgCAEEASgRAA0BBmAkoAgBBAEoNAAsLQaAJQQA2AgALQaQJQQA2AgALDwsgAUEAIAIgABEAAAtPAQJ/QYAIKAIAIgEgAEEHakF4cSICaiEAAkAgAkEAIAAgAU0bRQRAIAA/AEEQdE0NASAAEAANAQtBrAlBMDYCAEF/DwtBgAggADYCACABC98LAQh/AkAgAEUNACAAQQhrIgMgAEEEaygCACICQXhxIgBqIQUCQCACQQFxDQAgAkECcUUNASADIAMoAgAiBGsiA0HACSgCAEkNASAAIARqIQACQAJAAkBBxAkoAgAgA0cEQCADKAIMIQEgBEH/AU0EQCABIAMoAggiAkcNAkGwCUGwCSgCAEF+IARBA3Z3cTYCAAwFCyADKAIYIQcgASADRwRAIAMoAggiAiABNgIMIAEgAjYCCAwECyADKAIUIgIEfyADQRRqBSADKAIQIgJFDQMgA0EQagshBANAIAQhBiACIgFBFGohBCABKAIUIgINACABQRBqIQQgASgCECICDQALIAZBADYCAAwDCyAFKAIEIgJBA3FBA0cNA0G4CSAANgIAIAUgAkF+cTYCBCADIABBAXI2AgQgBSAANgIADwsgAiABNgIMIAEgAjYCCAwCC0EAIQELIAdFDQACQCADKAIcIgRBAnQiAigC4AsgA0YEQCACQeALaiABNgIAIAENAUG0CUG0CSgCAEF+IAR3cTYCAAwCCwJAIAMgBygCEEYEQCAHIAE2AhAMAQsgByABNgIUCyABRQ0BCyABIAc2AhggAygCECICBEAgASACNgIQIAIgATYCGAsgAygCFCICRQ0AIAEgAjYCFCACIAE2AhgLIAMgBU8NACAFKAIEIgRBAXFFDQACQAJAAkACQCAEQQJxRQRAQcgJKAIAIAVGBEBByAkgAzYCAEG8CUG8CSgCACAAaiIANgIAIAMgAEEBcjYCBCADQcQJKAIARw0GQbgJQQA2AgBBxAlBADYCAA8LQcQJKAIAIgcgBUYEQEHECSADNgIAQbgJQbgJKAIAIABqIgA2AgAgAyAAQQFyNgIEIAAgA2ogADYCAA8LIARBeHEgAGohACAFKAIMIQEgBEH/AU0EQCAFKAIIIgIgAUYEQEGwCUGwCSgCAEF+IARBA3Z3cTYCAAwFCyACIAE2AgwgASACNgIIDAQLIAUoAhghCCABIAVHBEAgBSgCCCICIAE2AgwgASACNgIIDAMLIAUoAhQiAgR/IAVBFGoFIAUoAhAiAkUNAiAFQRBqCyEEA0AgBCEGIAIiAUEUaiEEIAEoAhQiAg0AIAFBEGohBCABKAIQIgINAAsgBkEANgIADAILIAUgBEF+cTYCBCADIABBAXI2AgQgACADaiAANgIADAMLQQAhAQsgCEUNAAJAIAUoAhwiBEECdCICKALgCyAFRgRAIAJB4AtqIAE2AgAgAQ0BQbQJQbQJKAIAQX4gBHdxNgIADAILAkAgBSAIKAIQRgRAIAggATYCEAwBCyAIIAE2AhQLIAFFDQELIAEgCDYCGCAFKAIQIgIEQCABIAI2AhAgAiABNgIYCyAFKAIUIgJFDQAgASACNgIUIAIgATYCGAsgAyAAQQFyNgIEIAAgA2ogADYCACADIAdHDQBBuAkgADYCAA8LIABB/wFNBEAgAEH4AXFB2AlqIQICf0GwCSgCACIEQQEgAEEDdnQiAHFFBEBBsAkgACAEcjYCACACDAELIAIoAggLIQAgAiADNgIIIAAgAzYCDCADIAI2AgwgAyAANgIIDwtBHyEBIABB////B00EQCAAQSYgAEEIdmciAmt2QQFxIAJBAXRrQT5qIQELIAMgATYCHCADQgA3AhAgAUECdEHgC2ohBAJ/AkACf0G0CSgCACIGQQEgAXQiAnFFBEBBtAkgAiAGcjYCACAEIAM2AgBBGCEBQQgMAQsgAEEZIAFBAXZrQQAgAUEfRxt0IQEgBCgCACEEA0AgBCICKAIEQXhxIABGDQIgAUEddiEEIAFBAXQhASACIARBBHFqIgYoAhAiBA0ACyAGIAM2AhBBGCEBIAIhBEEICyEAIAMiAgwBCyACKAIIIgQgAzYCDCACIAM2AghBGCEAQQghAUEACyEGIAEgA2ogBDYCACADIAI2AgwgACADaiAGNgIAQdAJQdAJKAIAQQFrIgBBfyAAGzYCAAsLwScBC38jAEEQayIKJAACQAJAAkACQAJAAkACQAJAAkACQCAAQfQBTQRAQbAJKAIAIgRBECAAQQtqQfgDcSAAQQtJGyIGQQN2IgB2IgFBA3EEQAJAIAFBf3NBAXEgAGoiA0EDdCIBQdgJaiIAIAEoAuAJIgIoAggiBUYEQEGwCSAEQX4gA3dxNgIADAELIAUgADYCDCAAIAU2AggLIAJBCGohACACIAFBA3I2AgQgASACaiIBIAEoAgRBAXI2AgQMCwsgBkG4CSgCACIITQ0BIAEEQAJAQQIgAHQiAkEAIAJrciABIAB0cWgiA0EDdCIBQdgJaiICIAEoAuAJIgAoAggiBUYEQEGwCSAEQX4gA3dxIgQ2AgAMAQsgBSACNgIMIAIgBTYCCAsgACAGQQNyNgIEIAAgBmoiByABIAZrIgVBAXI2AgQgACABaiAFNgIAIAgEQCAIQXhxQdgJaiEBQcQJKAIAIQICfyAEQQEgCEEDdnQiA3FFBEBBsAkgAyAEcjYCACABDAELIAEoAggLIQMgASACNgIIIAMgAjYCDCACIAE2AgwgAiADNgIICyAAQQhqIQBBxAkgBzYCAEG4CSAFNgIADAsLQbQJKAIAIgtFDQEgC2hBAnQoAuALIgIoAgRBeHEgBmshAyACIQEDQAJAIAEoAhAiAEUEQCABKAIUIgBFDQELIAAoAgRBeHEgBmsiASADIAEgA0kiARshAyAAIAIgARshAiAAIQEMAQsLIAIoAhghCSACIAIoAgwiAEcEQCACKAIIIgEgADYCDCAAIAE2AggMCgsgAigCFCIBBH8gAkEUagUgAigCECIBRQ0DIAJBEGoLIQUDQCAFIQcgASIAQRRqIQUgACgCFCIBDQAgAEEQaiEFIAAoAhAiAQ0ACyAHQQA2AgAMCQtBfyEGIABBv39LDQAgAEELaiIBQXhxIQZBtAkoAgAiB0UNAEEfIQhBACAGayEDIABB9P//B00EQCAGQSYgAUEIdmciAGt2QQFxIABBAXRrQT5qIQgLAkACQAJAIAhBAnQoAuALIgFFBEBBACEADAELQQAhACAGQRkgCEEBdmtBACAIQR9HG3QhAgNAAkAgASgCBEF4cSAGayIEIANPDQAgASEFIAQiAw0AQQAhAyABIQAMAwsgACABKAIUIgQgBCABIAJBHXZBBHFqKAIQIgFGGyAAIAQbIQAgAkEBdCECIAENAAsLIAAgBXJFBEBBACEFQQIgCHQiAEEAIABrciAHcSIARQ0DIABoQQJ0KALgCyEACyAARQ0BCwNAIAAoAgRBeHEgBmsiAiADSSEBIAIgAyABGyEDIAAgBSABGyEFIAAoAhAiAQR/IAEFIAAoAhQLIgANAAsLIAVFDQAgA0G4CSgCACAGa08NACAFKAIYIQggBSAFKAIMIgBHBEAgBSgCCCIBIAA2AgwgACABNgIIDAgLIAUoAhQiAQR/IAVBFGoFIAUoAhAiAUUNAyAFQRBqCyECA0AgAiEEIAEiAEEUaiECIAAoAhQiAQ0AIABBEGohAiAAKAIQIgENAAsgBEEANgIADAcLIAZBuAkoAgAiBU0EQEHECSgCACEAAkAgBSAGayIBQRBPBEAgACAGaiICIAFBAXI2AgQgACAFaiABNgIAIAAgBkEDcjYCBAwBCyAAIAVBA3I2AgQgACAFaiIBIAEoAgRBAXI2AgRBACECQQAhAQtBuAkgATYCAEHECSACNgIAIABBCGohAAwJCyAGQbwJKAIAIgJJBEBBvAkgAiAGayIBNgIAQcgJQcgJKAIAIgAgBmoiAjYCACACIAFBAXI2AgQgACAGQQNyNgIEIABBCGohAAwJC0EAIQAgBkEvaiIDAn9BiA0oAgAEQEGQDSgCAAwBC0GUDUJ/NwIAQYwNQoCggICAgAQ3AgBBiA0gCkEMakFwcUHYqtWqBXM2AgBBnA1BADYCAEHsDEEANgIAQYAgCyIBaiIEQQAgAWsiB3EiASAGTQ0IQegMKAIAIgUEQEHgDCgCACIIIAFqIgkgCE0NCSAFIAlJDQkLAkBB7AwtAABBBHFFBEACQAJAAkACQEHICSgCACIFBEBB8AwhAANAIAAoAgAiCCAFTQRAIAUgCCAAKAIEakkNAwsgACgCCCIADQALC0EAEAIiAkF/Rg0DIAEhBEGMDSgCACIAQQFrIgUgAnEEQCABIAJrIAIgBWpBACAAa3FqIQQLIAQgBk0NA0HoDCgCACIABEBB4AwoAgAiBSAEaiIHIAVNDQQgACAHSQ0ECyAEEAIiACACRw0BDAULIAQgAmsgB3EiBBACIgIgACgCACAAKAIEakYNASACIQALIABBf0YNASAGQTBqIARNBEAgACECDAQLQZANKAIAIgIgAyAEa2pBACACa3EiAhACQX9GDQEgAiAEaiEEIAAhAgwDCyACQX9HDQILQewMQewMKAIAQQRyNgIACyABEAIhAkEAEAIhACACQX9GDQUgAEF/Rg0FIAAgAk0NBSAAIAJrIgQgBkEoak0NBQtB4AxB4AwoAgAgBGoiADYCAEHkDCgCACAASQRAQeQMIAA2AgALAkBByAkoAgAiAwRAQfAMIQADQCACIAAoAgAiASAAKAIEIgVqRg0CIAAoAggiAA0ACwwEC0HACSgCACIAQQAgACACTRtFBEBBwAkgAjYCAAtBACEAQfQMIAQ2AgBB8AwgAjYCAEHQCUF/NgIAQdQJQYgNKAIANgIAQfwMQQA2AgADQCAAQQN0IgEgAUHYCWoiBTYC4AkgASAFNgLkCSAAQQFqIgBBIEcNAAtBvAkgBEEoayIAQXggAmtBB3EiAWsiBTYCAEHICSABIAJqIgE2AgAgASAFQQFyNgIEIAAgAmpBKDYCBEHMCUGYDSgCADYCAAwECyACIANNDQIgASADSw0CIAAoAgxBCHENAiAAIAQgBWo2AgRByAkgA0F4IANrQQdxIgBqIgE2AgBBvAlBvAkoAgAgBGoiAiAAayIANgIAIAEgAEEBcjYCBCACIANqQSg2AgRBzAlBmA0oAgA2AgAMAwtBACEADAYLQQAhAAwEC0HACSgCACACSwRAQcAJIAI2AgALIAIgBGohBUHwDCEAAkADQCAFIAAoAgAiAUcEQCAAKAIIIgANAQwCCwsgAC0ADEEIcUUNAwtB8AwhAANAAkAgACgCACIBIANNBEAgAyABIAAoAgRqIgVJDQELIAAoAgghAAwBCwtBvAkgBEEoayIAQXggAmtBB3EiAWsiBzYCAEHICSABIAJqIgE2AgAgASAHQQFyNgIEIAAgAmpBKDYCBEHMCUGYDSgCADYCACADIAVBJyAFa0EHcWpBL2siACAAIANBEGpJGyIBQRs2AgQgAUH4DCkCADcCECABQfAMKQIANwIIQfgMIAFBCGo2AgBB9AwgBDYCAEHwDCACNgIAQfwMQQA2AgAgAUEYaiEAA0AgAEEHNgIEIABBCGogAEEEaiEAIAVJDQALIAEgA0YNACABIAEoAgRBfnE2AgQgAyABIANrIgJBAXI2AgQgASACNgIAAn8gAkH/AU0EQCACQfgBcUHYCWohAAJ/QbAJKAIAIgFBASACQQN2dCICcUUEQEGwCSABIAJyNgIAIAAMAQsgACgCCAshASAAIAM2AgggASADNgIMQQwhAkEIDAELQR8hACACQf///wdNBEAgAkEmIAJBCHZnIgBrdkEBcSAAQQF0a0E+aiEACyADIAA2AhwgA0IANwIQIABBAnRB4AtqIQECQAJAQbQJKAIAIgVBASAAdCIEcUUEQEG0CSAEIAVyNgIAIAEgAzYCAAwBCyACQRkgAEEBdmtBACAAQR9HG3QhACABKAIAIQUDQCAFIgEoAgRBeHEgAkYNAiAAQR12IQUgAEEBdCEAIAEgBUEEcWoiBCgCECIFDQALIAQgAzYCEAsgAyABNgIYQQghAiADIgEhAEEMDAELIAEoAggiACADNgIMIAEgAzYCCCADIAA2AghBACEAQRghAkEMCyADaiABNgIAIAIgA2ogADYCAAtBvAkoAgAiACAGTQ0AQbwJIAAgBmsiATYCAEHICUHICSgCACIAIAZqIgI2AgAgAiABQQFyNgIEIAAgBkEDcjYCBCAAQQhqIQAMBAtBrAlBMDYCAEEAIQAMAwsgACACNgIAIAAgACgCBCAEajYCBCACQXggAmtBB3FqIgggBkEDcjYCBCABQXggAWtBB3FqIgQgBiAIaiIDayEHAkBByAkoAgAgBEYEQEHICSADNgIAQbwJQbwJKAIAIAdqIgA2AgAgAyAAQQFyNgIEDAELQcQJKAIAIARGBEBBxAkgAzYCAEG4CUG4CSgCACAHaiIANgIAIAMgAEEBcjYCBCAAIANqIAA2AgAMAQsgBCgCBCIAQQNxQQFGBEAgAEF4cSEJIAQoAgwhAgJAIABB/wFNBEAgBCgCCCIBIAJGBEBBsAlBsAkoAgBBfiAAQQN2d3E2AgAMAgsgASACNgIMIAIgATYCCAwBCyAEKAIYIQYCQCACIARHBEAgBCgCCCIAIAI2AgwgAiAANgIIDAELAkAgBCgCFCIABH8gBEEUagUgBCgCECIARQ0BIARBEGoLIQEDQCABIQUgACICQRRqIQEgACgCFCIADQAgAkEQaiEBIAIoAhAiAA0ACyAFQQA2AgAMAQtBACECCyAGRQ0AAkAgBCgCHCIAQQJ0IgEoAuALIARGBEAgAUHgC2ogAjYCACACDQFBtAlBtAkoAgBBfiAAd3E2AgAMAgsCQCAEIAYoAhBGBEAgBiACNgIQDAELIAYgAjYCFAsgAkUNAQsgAiAGNgIYIAQoAhAiAARAIAIgADYCECAAIAI2AhgLIAQoAhQiAEUNACACIAA2AhQgACACNgIYCyAHIAlqIQcgBCAJaiIEKAIEIQALIAQgAEF+cTYCBCADIAdBAXI2AgQgAyAHaiAHNgIAIAdB/wFNBEAgB0H4AXFB2AlqIQACf0GwCSgCACIBQQEgB0EDdnQiAnFFBEBBsAkgASACcjYCACAADAELIAAoAggLIQEgACADNgIIIAEgAzYCDCADIAA2AgwgAyABNgIIDAELQR8hAiAHQf///wdNBEAgB0EmIAdBCHZnIgBrdkEBcSAAQQF0a0E+aiECCyADIAI2AhwgA0IANwIQIAJBAnRB4AtqIQACQAJAQbQJKAIAIgFBASACdCIFcUUEQEG0CSABIAVyNgIAIAAgAzYCAAwBCyAHQRkgAkEBdmtBACACQR9HG3QhAiAAKAIAIQEDQCABIgAoAgRBeHEgB0YNAiACQR12IQEgAkEBdCECIAAgAUEEcWoiBSgCECIBDQALIAUgAzYCEAsgAyAANgIYIAMgAzYCDCADIAM2AggMAQsgACgCCCIBIAM2AgwgACADNgIIIANBADYCGCADIAA2AgwgAyABNgIICyAIQQhqIQAMAgsCQCAIRQ0AAkAgBSgCHCIBQQJ0IgIoAuALIAVGBEAgAkHgC2ogADYCACAADQFBtAkgB0F+IAF3cSIHNgIADAILAkAgBSAIKAIQRgRAIAggADYCEAwBCyAIIAA2AhQLIABFDQELIAAgCDYCGCAFKAIQIgEEQCAAIAE2AhAgASAANgIYCyAFKAIUIgFFDQAgACABNgIUIAEgADYCGAsCQCADQQ9NBEAgBSADIAZqIgBBA3I2AgQgACAFaiIAIAAoAgRBAXI2AgQMAQsgBSAGQQNyNgIEIAUgBmoiBCADQQFyNgIEIAMgBGogAzYCACADQf8BTQRAIANB+AFxQdgJaiEAAn9BsAkoAgAiAUEBIANBA3Z0IgJxRQRAQbAJIAEgAnI2AgAgAAwBCyAAKAIICyEBIAAgBDYCCCABIAQ2AgwgBCAANgIMIAQgATYCCAwBC0EfIQAgA0H///8HTQRAIANBJiADQQh2ZyIAa3ZBAXEgAEEBdGtBPmohAAsgBCAANgIcIARCADcCECAAQQJ0QeALaiEBAkACQCAHQQEgAHQiAnFFBEBBtAkgAiAHcjYCACABIAQ2AgAgBCABNgIYDAELIANBGSAAQQF2a0EAIABBH0cbdCEAIAEoAgAhAQNAIAEiAigCBEF4cSADRg0CIABBHXYhASAAQQF0IQAgAiABQQRxaiIHKAIQIgENAAsgByAENgIQIAQgAjYCGAsgBCAENgIMIAQgBDYCCAwBCyACKAIIIgAgBDYCDCACIAQ2AgggBEEANgIYIAQgAjYCDCAEIAA2AggLIAVBCGohAAwBCwJAIAlFDQACQCACKAIcIgFBAnQiBSgC4AsgAkYEQCAFQeALaiAANgIAIAANAUG0CSALQX4gAXdxNgIADAILAkAgAiAJKAIQRgRAIAkgADYCEAwBCyAJIAA2AhQLIABFDQELIAAgCTYCGCACKAIQIgEEQCAAIAE2AhAgASAANgIYCyACKAIUIgFFDQAgACABNgIUIAEgADYCGAsCQCADQQ9NBEAgAiADIAZqIgBBA3I2AgQgACACaiIAIAAoAgRBAXI2AgQMAQsgAiAGQQNyNgIEIAIgBmoiBSADQQFyNgIEIAMgBWogAzYCACAIBEAgCEF4cUHYCWohAEHECSgCACEBAn9BASAIQQN2dCIHIARxRQRAQbAJIAQgB3I2AgAgAAwBCyAAKAIICyEEIAAgATYCCCAEIAE2AgwgASAANgIMIAEgBDYCCAtBxAkgBTYCAEG4CSADNgIACyACQQhqIQALIApBEGokACAAC+QJAxh/AX0BewJAIAEgAk4NACAFIAZODQAgBCADQQRqIg1OBEAgBCADIANBCGoiCCAEQQFqIgcgByAISBsgA2tBBWsiD0F8cWoiCEEFaiIHIAQgB0obIgdBAnQgD0ECdEFwcWshFSAHIAhrQQRrIhZBfHEhECAPQQRxIRkgD0ECdkEBakH+////B3EhGiADQQJ0IRcgACgCECEHIAEhEgNAIAAoAhQiCSABIBFqbEECdCIIIAAoAggiDCAVamohGyAMIBdqIAhqQRBqIRwgACgCBCAFIAdsQQJ0aiEKIAwgCSASbEECdGohDiAAKAIAIAAoAgwgEmxBAnRqIR0gBSEUA0AgHSAUQQJ0aioCACIf/RMhIEEAIQwgAyEIQQAhEyANIgchCSAPQQRPBEADQCAOIAhBAnQiCWoiCCAJIApq/QAAACAg/eYBIAj9AAAA/eQB/QsAACAOIAdBAnQiCWoiCCAJIApq/QAAACAg/eYBIAj9AAAA/eQB/QsAACAMIgtBAmohDCAHQQRqIQggB0EIaiIJIQcgE0ECaiITIBpHDQALIAtBAXIhCwsgGUUEQCAOIAhBAnQiCGoiByAIIApq/QAAACAg/eYBIAf9AAAA/eQB/QsAACAMIQsgCSEICwJAIAQgCEwNAAJAAkAgFkEISQ0AIBwgC0EEdCIHaiAKIBVqIAdqSQRAIAogF2ogB2pBEGogByAbakkNAQsgCCAQaiEHQQAhDANAIA4gCCAMakECdCITaiIJIAn9AAIAIAogE2r9AAIAICD95gH95AH9CwIAIAxBBGoiDCAQRw0ACyAQIBZHDQEMAgsgCCEHCwNAIA4gB0ECdCIJaiIIIAgqAgAgCSAKaioCACAflJI4AgAgB0EBaiIHIARIDQALCyAKIAAoAhAiB0ECdGohCiAUQQFqIhQgBkcNAAsgEUEBaiERIBJBAWoiEiACRw0ACwwBCyADIARODQAgACgCBCIJIAAoAhAiHiAFbCIHQQJ0aiEMIAAoAgwhFSAAKAIAIRYgCSAEIAdqIB4gBiAFQX9zamxqQQJ0aiEXIAAoAhQiDkECdCEZIAMgBCADayIPQXxxIhBqIQggACgCCCIRIAQgASAObCIAakECdGohGiARIAAgA2pBAnRqIRsgCSADIAdqQQJ0aiEcQQAhACAeQYCAgIACcUEARyEdIA9BBEkhEwNAIBEgASAObEECdGohGCAWIAEgFWxBAnRqIRIgGyAAIBlsIgdqIBdJIBwgByAaaklxIB1yIBNyIRQgBSEJIAwhCgNAIBIgCUECdGoqAgAhHyADIQcCQCAURQRAIB/9EyEgQQAhBwNAIBggAyAHakECdCINaiILIAv9AAIAIAogDWr9AAIAICD95gH95AH9CwIAIAdBBGoiByAQRw0ACyAIIQcgDyAQRg0BCyAHQQFqIQsgBCAHa0EBcQRAIBggB0ECdCINaiIHIAcqAgAgCiANaioCACAflJI4AgAgCyEHCyAEIAtGDQADQCAYIAdBAnQiDWoiCyALKgIAIAogDWoqAgAgH5SSOAIAIBggDUEEaiINaiILIAsqAgAgCiANaioCACAflJI4AgAgB0ECaiIHIARHDQALCyAKIB5BAnRqIQogCUEBaiIJIAZHDQALIABBAWohACABQQFqIgEgAkcNAAsLC90EAwJ7B38BfSMAQSBrIgskAAJAIARBAEwNACAFQQBMDQAgBBAEIQwgBEEIdiIOQQJ0EAQhDQJAIAxFDQAgDUUNACAEQYACTwRAIAAgAUECdGohEEEAIQADQCAAQQh0IREgECAAQQp0aiEP/QwAAAAAAAAAAAAAAAAAAAAAIQlBACEBA0AgDyABQQJ0aiIE/QACMP3gASAE/QACIP3gASAE/QACEP3gASAE/QACAP3gASAJ/esB/esB/esB/esBIQkgAUEQaiIBQYACRw0ACyAMIBFqIQEgDSAAQQJ0agJ9IAkgCSAJ/Q0ICQoLDA0ODwABAgMAAQID/esBIgkgCSAJ/Q0EBQYHAAECAwABAgMAAQID/esB/R8AIhJDAAAAAFwEQEMAAIA/IBKV/RMhCkEAIQQDQCABIARqIA8gBEECdGr9AAIA/QwAAP5CAAD+QgAA/kIAAP5C/eYBIAr95gEiCf0fAJD8AP0RIAn9HwGQ/AD9HAEgCf0fApD8AP0cAiAJ/R8DkPwA/RwD/QyB////gf///4H///+B/////bgB/Qx/AAAAfwAAAH8AAAB/AAAA/bYBIAn9DQAECAwAAAAAAAAAAAAAAAD9WgAAACAEQQRqIgRBgAJHDQALIBJDBAIBPJQMAQsgAUEAQYAC/AsAQwAAAAALOAIAIABBAWoiACAORw0ACwsgC0EANgIcIAsgBTYCGCALIA42AhQgCyANNgIMIAsgDDYCCCALIAIgA2o2AgQgCyAGIAdBAnRqNgIQIAggC0EEaiAFEAELIAwQAyANEAMLIAtBIGokAAuoAgICfwF7IwBBIGsiCSQAAkAgBUEATA0AIARBAEwEQCAFQQJ0IgBFDQEgBiAHQQJ0akEAIAD8CwAMAQsgCSAFNgIcIAkgBEEFdiIKNgIYIAkgAiADajYCCCAJIAYgB0ECdGo2AhAgCSAAIAFBAnRqIgI2AgwgCkECdBAEIgFFDQAgBEEfSwRAQQAhBwNAIAEgB0ECdGogAiAHQQd0aiIA/QACACAA/QACcP3kASAA/QACQP3kASAA/QACICAA/QACYP3kAf3kASAA/QACECAA/QACUP3kASAA/QACMP3kAf3kASIL/R8AIAv9HwGSIAv9HwKSIAv9HwOSOAIAIAdBAWoiByAKRw0ACwsgCSABNgIUIAggCUEIaiAFEAEgARADCyAJQSBqJAALFAEBfyMAQRBrIgAkACAAQRBqJAALtwkDDH8NewN9IwBBQGohBiAALQABIgVBB3YhAiAALQADQQh0IgcgAC0AAnIhBCAFQQh0IgggAC0AAHJB/wdxIQMCfQJAIAVBAnZBH3EiBUEfRwRAIAUNASACQR90viADRQ0CGiADs0MAAIAzlCIbjCAbIAjBQQBIGwwCCyACQR90QYCAgAJBACADG3JBgICA/AdyvgwBCyAFQRd0IAJBH3RyIANBDXRyQYCAgMADar4LIR0gBEH/B3EhAyAEQYCAAnFBD3YhAgJ9AkAgBEEKdkEfcSIEQR9HBEAgBA0BIAJBH3S+IANFDQIaIAOzQwAAgDOUIhuMIBsgB8FBAEgbDAILIAJBH3RBgICAAkEAIAMbckGAgID8B3K+DAELIARBF3QgAkEfdHIgA0ENdHJBgICAwANqvgshGyAGIAD9XAAEIg79DD8/Pz8/Pz8/Pz8/Pz8/Pz/9Tv2JAf2pAf0LBCAgBiAA/VwACCIP/Qw/Pz8/Pz8/Pz8/Pz8/Pz8//U79iQH9qQH9CwQAIAYgAP1cAAwiEEEE/W0gD0EC/W39DDAwMDAwMDAwMDAwMDAwMDD9Tv1Q/YkB/akB/QsEECAGIBD9DA8PDw8PDw8PDw8PDw8PDw/9TiAOQQL9bf0MMDAwMDAwMDAwMDAwMDAwMP1O/VD9iQH9qQH9CwQwIABBMGohCCAAQRBqIQlBACEEA0AgASAEQQh0aiEKIAggBEEFdGohCyABIARBAXQiBUEBciIHQQd0aiEM/QwAAAAAAAAAAAAAAAAAAAAAIg4hD/0MAAAAAAAAAAAAAAAAAAAAACEQ/QwAAAAAAAAAAAAAAAAAAAAAIRFBASEDQQAhAANAIAwgAEECdCINaiIC/QAAMCIWIAAgCWr9AAAAIhIgB/1tQQT9a/0MEBAQEBAQEBAQEBAQEBAQEP1OIAAgC2r9AAAAIhNBBP1t/VAiFP2KASIV/aoB/foB/eYBIAL9AAAgIhcgFf2pAf36Af3mASAC/QAAECIVIBT9iQEiFP2qAf36Af3mASAC/QAAACIYIBT9qQH9+gH95gEgEP3kAf3kAf3kAf3kASEQIAogDWoiAP0AADAiFCASIAX9bUEE/Wv9DBAQEBAQEBAQEBAQEBAQEBD9TiAT/QwPDw8PDw8PDw8PDw8PDw8P/U79UCIS/YoBIhP9qgH9+gH95gEgAP0AACAiGSAT/akB/foB/eYBIAD9AAAQIhMgEv2JASIS/aoB/foB/eYBIAD9AAAAIhogEv2pAf36Af3mASAO/eQB/eQB/eQB/eQBIQ4gFiAXIBUgGCAR/eQB/eQB/eQB/eQBIREgFCAZIBMgGiAP/eQB/eQB/eQB/eQBIQ8gA0EQIQBBACEDDQALIBD9HwAgEP0fAZIgEP0fApIgEP0fA5IgB0ECdCIAIAZBIGoiAmooAgCylCAO/R8AIA79HwGSIA79HwKSIA79HwOSIAIgBUECdCIDaigCALKUkiAdlCAckiAR/R8AIBH9HwGSIBH9HwKSIBH9HwOSIBuUIAAgBmooAgCylCAP/R8AIA/9HwGSIA/9HwKSIA/9HwOSIBuUIAMgBmooAgCylJKTIRwgBEEBaiIEQQRHDQALIBwL1QcEBXsHfwJ9An4jAEFAaiEMIAAtAAEiCkEHdiELIAAtAANBCHQiDSAALQACciEIIApBCHQiDiAALQAAckH/B3EhCQJ9AkAgCkECdkEfcSIKQR9HBEAgCg0BIAtBH3S+IAlFDQIaIAmzQwAAgDOUIg+MIA8gDsFBAEgbDAILIAtBH3RBgICAAkEAIAkbckGAgID8B3K+DAELIApBF3QgC0EfdHIgCUENdHJBgICAwANqvgsgCEH/B3EhCSAIQYCAAnFBD3YhCwJ9AkAgCEEKdkEfcSIIQR9HBEAgCA0BIAtBH3S+IAlFDQIaIAmzQwAAgDOUIhCMIBAgDcFBAEgbDAILIAtBH3RBgICAAkEAIAkbckGAgID8B3K+DAELIAhBF3QgC0EfdHIgCUENdHJBgICAwANqvgshECAMIAD9XAAEIgP9DD8/Pz8/Pz8/Pz8/Pz8/Pz/9Tv2JAf2pAf0LBCAgDCAA/VwACCIE/Qw/Pz8/Pz8/Pz8/Pz8/Pz8//U79iQH9qQH9CwQAIAwgAP1cAAwiBUEE/W0gBEEC/W39DDAwMDAwMDAwMDAwMDAwMDD9Tv1Q/YkB/akB/QsEECAMIAX9DA8PDw8PDw8PDw8PDw8PDw/9TiADQQL9bf0MMDAwMDAwMDAwMDAwMDAwMP1O/VD9iQH9qQH9CwQwIABBEGohCUEAIQADQCAAQQN0IgsgDEEgaiINajQCACAJIABBBXRqIgj9AAAAIgb9DA8PDw8PDw8PDw8PDw8PDw/9TiID/YoBIAEgAEEGdGoiCv0AAAAiBP2IAf26ASAD/YkBIAT9hwH9ugH9rgEgCP0AABAiB/0MDw8PDw8PDw8PDw8PDw8PD/1OIgP9igEgCv0AABAiBf2IAf26Af2uASAD/YkBIAX9hwH9ugH9rgEiA/0bACAD/RsBaiAD/RsCaiAD/RsDaqx+IBF8IAZBBP1tIgP9igEgASAAQQF0QQFyIghBBXRqIgr9AAAAIgb9iAH9ugEgA/2JASAG/YcB/boB/a4BIAdBBP1tIgP9igEgCv0AABAiB/2IAf26Af2uASAD/YkBIAf9hwH9ugH9rgEiA/0bACAD/RsBaiAD/RsCaiAD/RsDaqwgDSAIQQJ0IghqNAIAfnwhESALIAxqNAIAIAX9fP1+IAT9fP1+/a4BIgP9GwAgA/0bAWogA/0bAmogA/0bA2qsfiASfCAIIAxqNAIAIAf9fP1+IAb9fP1+/a4BIgP9GwAgA/0bAWogA/0bAmogA/0bA2qsfnwhEiAAQQFqIgBBBEcNAAsgEbSUIBAgErSUkyAClAvABwIUfwt7AkAgASACTg0AIAMgBE4NACACIAFrIghBA3EhDSAEIANrQQJ0IQkgACgCFCIHQQJ0IQwgACgCCCABIAdsIANqQQJ0aiEKIAEgAmtBfE0EQCAIQXxxIQdBACEIA0AgCUUiC0UEQCAKIAUgDGxqQQAgCfwLAAsgC0UEQCAKIAwgBUEBcmxqQQAgCfwLAAsgC0UEQCAKIAwgBUECcmxqQQAgCfwLAAsgC0UEQCAKIAwgBUEDcmxqQQAgCfwLAAsgBUEEaiEFIAhBBGoiCCAHRw0ACwsgDUUNAANAIAkEQCAKIAUgDGxqQQAgCfwLAAsgBUEBaiEFIAZBAWoiBiANRw0ACwsCQCAAKAIgIgVBAEwNACAEIANrIgZBCG1BA3QgA2ohDiACIAFrIghBBG1BAnQgAWohE0EAIQcgBkEITgRAIAhBA0ohFgNAIAUgB0GAAmoiCCAFIAdrQYACSBshDyADIQ0gFgRAA0AgASELA0AgDUECdCIKIAAoAgggACgCFCIGIAtsQQJ0amoiEP0AABAhGiAQ/QAAACEbIBAgBkECdCIGaiIR/QAAECEcIBH9AAAAIR0gBiARaiIS/QAAECEeIBL9AAAAIR8gBiASaiIU/QAAECEgIBT9AAAAISEgByAPSARAIAAoAgAgACgCDCIGIAtsQQJ0aiIXIAZBAnQiBmoiGCAGaiIJIAZqIQwgCiAAKAIEIAAoAhAiCiAHbEECdGpqIQUgByEGA0AgDCAGQQJ0IhVq/QkCACIZIAX9AAAQIiL95gEgIP3kASEgIBkgBf0AAAAiI/3mASAh/eQBISEgCSAVav0JAgAiGSAi/eYBIB795AEhHiAZICP95gEgH/3kASEfIBUgGGr9CQIAIhkgIv3mASAc/eQBIRwgGSAj/eYBIB395AEhHSAVIBdq/QkCACIZICL95gEgGv3kASEaIBkgI/3mASAb/eQBIRsgBSAKQQJ0aiEFIAZBAWoiBiAPRw0ACwsgECAa/QsAECAQIBv9CwAAIBEgHP0LABAgESAd/QsAACASIB79CwAQIBIgH/0LAAAgFCAg/QsAECAUICH9CwAAIAtBBGoiCyATSA0ACyANQQhqIg0gDkgNAAsLIAAgEyACIAMgDiAHIA8QBSAAIAEgAiAOIAQgByAPEAUgACgCICIFIAgiB0oNAAsMAQsDQCAAIBMgAiADIA4gByAFIAdBgAJqIgggBSAHa0GAAkgbIgYQBSAAIAEgAiAOIAQgByAGEAUgACgCICIFIAgiB0oNAAsLC4YBAQF/IwBBMGsiDCQAAkAgCUEATA0AIApBAEwNACAMIAo2AiggDCAJNgIkIAwgCDYCICAMIAU2AhwgDCACNgIYIAwgAyAEajYCECAMIAtBACALQQBKGzYCLCAMIAYgB0ECdGo2AhQgDCAAIAFBAnRqNgIMQQQgDEEMaiAKEAELIAxBMGokAAu8CwMSfwJ7AX0jAEGAEGsiDCQAAkAgACgCGCIGQQBMDQAgASACTg0AIAAoAgggAUECdGohBSACIAFrQQJ0IQQgACgCFEECdCEIIAZBBE8EQCAGQfz///8HcSELA0AgBEUiB0UEQCAFIAMgCGxqQQAgBPwLAAsgB0UEQCAFIAggA0EBcmxqQQAgBPwLAAsgB0UEQCAFIAggA0ECcmxqQQAgBPwLAAsgB0UEQCAFIAggA0EDcmxqQQAgBPwLAAsgA0EEaiEDIAlBBGoiCSALRw0ACwsgBkEDcSIHRQ0AA0AgBARAIAUgAyAIbGpBACAE/AsACyADQQFqIQMgCkEBaiIKIAdHDQALCwJAIAAoAiAiC0EATA0AIAZBAUYEQCABQQF0IQ0gACgCCCABQQJ0aiEHIAAoAgAhDkEAIQggAiABayIKQQhIIQ8DQCAAKAIEIAAoAhAgCGxqIA1qIQkgDiAIQQJ0aioCACEXQQAhBSAPRQRAIBf9EyEVQQAhBEEIIQYDQCAHIARBAnRqIgMgFSAJIARBAXRq/QAAACIWIBX9DQABAAECAwABBAUAAQYHAAFBEP2rAf3mASAD/QAAAP3kAf0LAAAgAyAVIBYgFf0NCAkAAQoLAAEMDQABDg8AAUEQ/asB/eYBIAP9AAAQ/eQB/QsAECAGIgUiBEEIaiIGIApMDQALCwJAIAUgCk4NAAJAIAIgASAFamsiBkEESQRAIAUhAwwBCyAFIAZBfHEiC2ohAyAX/RMhFUEAIQQDQCAHIAQgBWoiEEECdGoiESAVIAkgEEEBdGr9BAAAQRD9qwH95gEgEf0AAgD95AH9CwIAIARBBGoiBCALRw0ACyAGIAtGDQELA0AgByADQQJ0aiIEIBcgCSADQQF0ai8AAEEQdL6UIAQqAgCSOAIAIANBAWoiAyAKRw0ACwsgCEEBaiIIIAAoAiBIDQALDAELIAEgAk4NACACIAFrIRADQCALQQBKBEBBgAQgAiABayIOIA5BgAROGyEHQYAEIBAgDUEJdGsiAyADQYAEThsiD0EDcSEKIAFBAXQhEUEAIQggDkEHSiEUA0AgACgCBCAAKAIQIAhsaiARaiEJQQghBEEAIQMgFARAA0AgDCADQQJ0aiIFIAkgA0EBdGr9AAAAIhUgFf0NCAkAAQoLAAEMDQABDg8AAUEQ/asB/QsEECAFIBUgFf0NAAEAAQIDAAEEBQABBgcAAUEQ/asB/QsEACAEIgNBCGoiBCAHTA0ACwsCQCADIAdODQACQCAPIANrIgRBBEkEQCADIQQMAQsgAyAEIAprIhJqIQRBACEFA0AgDCADIAVqIhNBAnRqIAkgE0EBdGr9BAAAQRD9qwH9CwIAIAVBBGoiBSASRw0ACyAKRQ0BCwNAIAwgBEECdGogCSAEQQF0ai8AAEEQdDYCACAEQQFqIgQgB0cNAAsLQQAhCSAGQQBKBEADQCAAKAIIIAAoAhQgCWxBAnRqIAFBAnRqIQYgACgCACAAKAIMIAlsQQJ0aiAIQQJ0aioCACEXQQAhAyAOQQROBEAgF/0TIRVBACEEQQQhBQNAIAYgBEECdCIDaiIEIAMgDGr9AAQAIBX95gEgBP0AAAD95AH9CwAAIAUiAyEEIANBBGoiBSAHTA0ACwsCQCADIAdODQACQCAPIANrIgRBBEkEQCADIQQMAQsgAyAEIAprIgtqIQQgF/0TIRVBACEFA0AgBiADIAVqQQJ0IhJqIhMgE/0AAgAgDCASav0AAgAgFf3mAf3kAf0LAgAgBUEEaiIFIAtHDQALIApFDQELA0AgBiAEQQJ0IgNqIgUgBSoCACADIAxqKgIAIBeUkjgCACAEQQFqIgQgB0cNAAsLIAlBAWoiCSAAKAIYIgZIDQALIAAoAiAhCwsgCEEBaiIIIAtIDQALCyANQQFqIQ0gAUGABGoiASACSA0ACwsgDEGAEGokAAuGAQEBfyMAQTBrIgwkAAJAIAlBAEwNACAKQQBMDQAgDCAKNgIoIAwgCTYCJCAMIAg2AiAgDCAFNgIcIAwgAjYCGCAMIAMgBGo2AhAgDCALQQAgC0EAShs2AiwgDCAGIAdBAnRqNgIUIAwgACABQQJ0ajYCDEEDIAxBDGogChABCyAMQTBqJAALuAYDEn8BewF9AkAgACgCDCIKQQBMDQAgASACTg0AIAAoAhAhCyAAKAIIIQwgACgCACENIAAoAgQhDiAAKAIUIgNBBE4EQCADQfz///8HcSEFIANBBGsiD0EEcSERIA9BAnZBAWpB/v///wdxIRIDQCAMIAYgC2xBAnRqIRMgDSADIAZsQQJ0aiEHIAEhBANAIA4gAyAEbEECdGohCUEEIQD9DAAAAAAAAAAAAAAAAAAAAAAhFUEAIQhBACEQIA9BBE8EQANAIAkgAEECdCIUav0AAAAgByAUav0AAAD95gEgCSAIQQJ0Ighq/QAAACAHIAhq/QAAAP3mASAV/eQB/eQBIRUgAEEEaiEIIABBCGohACAQQQJqIhAgEkcNAAsLIBFFBEAgCSAIQQJ0IgBq/QAAACAAIAdq/QAAAP3mASAV/eQBIRULIBX9HwAgFf0fAZIgFf0fApIgFf0fA5IhFiADIAUiAEoEQANAIAkgAEECdCIIaioCACAHIAhqKgIAlCAWkiEWIABBAWoiACADRw0ACwsgEyAEQQJ0aiAWOAIAIARBAWoiBCACRw0ACyAGQQFqIgYgCkcNAAsMAQsgA0EASgRAIANBAUYhBwNAIAwgBCALbEECdGohCSANIAMgBGxBAnRqIQUgASEAA0AgCSAAQQJ0agJ9IA4gACADbEECdGoiBioCACAFKgIAlCIWIAcNABogBioCBCAFKgIElCAWkiIWIANBAkYNABogBioCCCAFKgIIlCAWkgs4AgAgAEEBaiIAIAJHDQALIARBAWoiBCAKRw0ACwwBCyALQQJ0IQUgDCABQQJ0aiEDIAIgAWtBAnQhAUEAIQAgCkEETwRAIApB/P///wdxIQcDQCABRSICRQRAIAMgACAFbGpBACAB/AsACyACRQRAIAMgBSAAQQFybGpBACAB/AsACyACRQRAIAMgBSAAQQJybGpBACAB/AsACyACRQRAIAMgBSAAQQNybGpBACAB/AsACyAAQQRqIQAgBkEEaiIGIAdHDQALCyAKQQNxIgJFDQADQCABBEAgAyAAIAVsakEAIAH8CwALIABBAWohACAEQQFqIgQgAkcNAAsLC5sBAQF/IwBBIGsiCSQAAkAgBkEATA0AIAdBAEwNACAIQQBMBEAgBiAHbCIAQQBMDQEgAEECdCIARQ0BIAQgBUECdGpBACAA/AsADAELIAkgCDYCHCAJIAc2AhggCSAGNgIUIAkgBCAFQQJ0ajYCECAJIAIgA0ECdGo2AgwgCSAAIAFBAnRqNgIIQQIgCUEIaiAHEAELIAlBIGokAAsFAQh/AAv2DQQDew5/A30BfiMAQYACayIGJAACQCABIAJODQAgACgCDCENIAAoAhAiDkEATARAIAIgAWtBAnQiAEUNASANIAFBAnRqQQAgAPwLAAwBCyAAKAIAIAEgDmxB0gFsaiEHIAAoAgghDyAAKAIEIRADQEMAAAAAIRRBACEKA0AgBy0A0QEiCEEHdiEJIAhBCHQiCyAHLQDQAXJB/wdxIQAgDyAKQQJ0agJ9AkAgCEECdkEfcSIIBEAgCEEfRw0BIAlBH3RBgICAAkEAIAAbckGAgID8B3K+DAILIAAEQCAAs0MAAIAzlCIVjCAVIAvBQQBIGwwCCyAJQR90vgwBCyAIQRd0IAlBH3RyIABBDXRyQYCAgMADar4LIRUgECAKQQh0aiELKgIAIRYgB/0AACAhBCAGIAf9AACAASID/QwwMDAwMDAwMDAwMDAwMDAw/U4gB/0AAAAiBUEE/W39UP0M4ODg4ODg4ODg4ODg4ODg4P1u/QsEQCAGIANBAv1t/QwwMDAwMDAwMDAwMDAwMDAw/U4gBEEE/W39UP0M4ODg4ODg4ODg4ODg4ODg4P1u/QsEYCAGIANBAv1r/QwwMDAwMDAwMDAwMDAwMDAw/U4gBP0MDw8PDw8PDw8PDw8PDw8PD/1O/VD9DODg4ODg4ODg4ODg4ODg4OD9bv0LBCAgBiADQQT9a/0MMDAwMDAwMDAwMDAwMDAwMP1OIAX9DA8PDw8PDw8PDw8PDw8PDw/9Tv1Q/Qzg4ODg4ODg4ODg4ODg4ODg/W79CwQAIAf9AAAwIQQgBiAH/QAAkAEiA/0MMDAwMDAwMDAwMDAwMDAwMP1OIAf9AAAQIgVBBP1t/VD9DODg4ODg4ODg4ODg4ODg4OD9bv0LBFAgBiADQQL9bf0MMDAwMDAwMDAwMDAwMDAwMP1OIARBBP1t/VD9DODg4ODg4ODg4ODg4ODg4OD9bv0LBHAgBiADQQL9a/0MMDAwMDAwMDAwMDAwMDAwMP1OIAT9DA8PDw8PDw8PDw8PDw8PDw/9Tv1Q/Qzg4ODg4ODg4ODg4ODg4ODg/W79CwQwIAYgA0EE/Wv9DDAwMDAwMDAwMDAwMDAwMDD9TiAF/QwPDw8PDw8PDw8PDw8PDw8P/U79UP0M4ODg4ODg4ODg4ODg4ODg4P1u/QsEECAH/QAAYCEEIAYgB/0AAKABIgP9DDAwMDAwMDAwMDAwMDAwMDD9TiAH/QAAQCIFQQT9bf1Q/Qzg4ODg4ODg4ODg4ODg4ODg/W79CwTAASAGIANBAv1t/QwwMDAwMDAwMDAwMDAwMDAw/U4gBEEE/W39UP0M4ODg4ODg4ODg4ODg4ODg4P1u/QsE4AEgBiADQQL9a/0MMDAwMDAwMDAwMDAwMDAwMP1OIAT9DA8PDw8PDw8PDw8PDw8PDw/9Tv1Q/Qzg4ODg4ODg4ODg4ODg4ODg/W79CwSgASAGIANBBP1r/QwwMDAwMDAwMDAwMDAwMDAw/U4gBf0MDw8PDw8PDw8PDw8PDw8PD/1O/VD9DODg4ODg4ODg4ODg4ODg4OD9bv0LBIABIAf9AABwIQQgBiAH/QAAsAEiA/0MMDAwMDAwMDAwMDAwMDAwMP1OIAf9AABQIgVBBP1t/VD9DODg4ODg4ODg4ODg4ODg4OD9bv0LBNABIAYgA0EC/W39DDAwMDAwMDAwMDAwMDAwMDD9TiAEQQT9bf1Q/Qzg4ODg4ODg4ODg4ODg4ODg/W79CwTwASAGIANBAv1r/QwwMDAwMDAwMDAwMDAwMDAw/U4gBP0MDw8PDw8PDw8PDw8PDw8PD/1O/VD9DODg4ODg4ODg4ODg4ODg4OD9bv0LBLABIAYgA0EE/Wv9DDAwMDAwMDAwMDAwMDAwMDD9TiAF/QwPDw8PDw8PDw8PDw8PDw8P/U79UP0M4ODg4ODg4ODg4ODg4ODg4P1u/QsEkAEgB0HAAWohDEEAIQBBASEJQgAhFwNAIABBB3QhESAMIABBA3RqIRJBACEAA0AgEiAAQQF0aiITMAAAIAYgAEEFdCARaiIIav0ABAAiA/2HASAIIAtq/QAAACIE/YcB/boBIAP9iAEgBP2IAf26Af2uASID/RsAIAP9GwFqIAP9GwJqIAP9GwNqrH4gF3wgEzAAASAGIAhBEHIiCGr9AAQAIgP9hwEgCCALav0AAAAiBP2HAf26ASAD/YgBIAT9iAH9ugH9rgEiA/0bACAD/RsBaiAD/RsCaiAD/RsDaqx+fCEXIABBAWoiAEEERw0AC0EBIQAgCUEAIQkNAAsgB0HSAWohByAVIBaUIBe0lCAUkiEUIApBAWoiCiAORw0ACyANIAFBAnRqIBQ4AgAgAUEBaiIBIAJHDQALCyAGQYACaiQACxYAIAAgASACIAMgBCAFIAYgB0ESEAYL/g0EA3sRfwJ9AX4jAEGAAmsiBiQAAkAgASACTg0AIAIgAWtBAnQiCgRAIAAoAgwgAUECdGpBACAK/AsACyAAKAIQIg5BAEwNACAAKAIMIQ8gACgCFCEQIAAoAgAhESAAKAIIIRIgACgCBCETA0AgEyALQQh0aiENIBEgCyAQbCABakHSAWxqIQcgEiALQQJ0aioCACEYIAEhCgNAIActANEBIghBB3YhCSAIQQh0IgwgBy0A0AFyQf8HcSEAAn0CQCAIQQJ2QR9xIggEQCAIQR9HDQEgCUEfdEGAgIACQQAgABtyQYCAgPwHcr4MAgsgAARAIACzQwAAgDOUIheMIBcgDMFBAEgbDAILIAlBH3S+DAELIAhBF3QgCUEfdHIgAEENdHJBgICAwANqvgshFyAH/QAAICEEIAYgB/0AAIABIgP9DDAwMDAwMDAwMDAwMDAwMDD9TiAH/QAAACIFQQT9bf1Q/Qzg4ODg4ODg4ODg4ODg4ODg/W79CwRAIAYgA0EC/W39DDAwMDAwMDAwMDAwMDAwMDD9TiAEQQT9bf1Q/Qzg4ODg4ODg4ODg4ODg4ODg/W79CwRgIAYgA0EC/Wv9DDAwMDAwMDAwMDAwMDAwMDD9TiAE/QwPDw8PDw8PDw8PDw8PDw8P/U79UP0M4ODg4ODg4ODg4ODg4ODg4P1u/QsEICAGIANBBP1r/QwwMDAwMDAwMDAwMDAwMDAw/U4gBf0MDw8PDw8PDw8PDw8PDw8PD/1O/VD9DODg4ODg4ODg4ODg4ODg4OD9bv0LBAAgB/0AADAhBCAGIAf9AACQASID/QwwMDAwMDAwMDAwMDAwMDAw/U4gB/0AABAiBUEE/W39UP0M4ODg4ODg4ODg4ODg4ODg4P1u/QsEUCAGIANBAv1t/QwwMDAwMDAwMDAwMDAwMDAw/U4gBEEE/W39UP0M4ODg4ODg4ODg4ODg4ODg4P1u/QsEcCAGIANBAv1r/QwwMDAwMDAwMDAwMDAwMDAw/U4gBP0MDw8PDw8PDw8PDw8PDw8PD/1O/VD9DODg4ODg4ODg4ODg4ODg4OD9bv0LBDAgBiADQQT9a/0MMDAwMDAwMDAwMDAwMDAwMP1OIAX9DA8PDw8PDw8PDw8PDw8PDw/9Tv1Q/Qzg4ODg4ODg4ODg4ODg4ODg/W79CwQQIAf9AABgIQQgBiAH/QAAoAEiA/0MMDAwMDAwMDAwMDAwMDAwMP1OIAf9AABAIgVBBP1t/VD9DODg4ODg4ODg4ODg4ODg4OD9bv0LBMABIAYgA0EC/W39DDAwMDAwMDAwMDAwMDAwMDD9TiAEQQT9bf1Q/Qzg4ODg4ODg4ODg4ODg4ODg/W79CwTgASAGIANBAv1r/QwwMDAwMDAwMDAwMDAwMDAw/U4gBP0MDw8PDw8PDw8PDw8PDw8PD/1O/VD9DODg4ODg4ODg4ODg4ODg4OD9bv0LBKABIAYgA0EE/Wv9DDAwMDAwMDAwMDAwMDAwMDD9TiAF/QwPDw8PDw8PDw8PDw8PDw8P/U79UP0M4ODg4ODg4ODg4ODg4ODg4P1u/QsEgAEgB/0AAHAhBCAGIAf9AACwASID/QwwMDAwMDAwMDAwMDAwMDAw/U4gB/0AAFAiBUEE/W39UP0M4ODg4ODg4ODg4ODg4ODg4P1u/QsE0AEgBiADQQL9bf0MMDAwMDAwMDAwMDAwMDAwMP1OIARBBP1t/VD9DODg4ODg4ODg4ODg4ODg4OD9bv0LBPABIAYgA0EC/Wv9DDAwMDAwMDAwMDAwMDAwMDD9TiAE/QwPDw8PDw8PDw8PDw8PDw8P/U79UP0M4ODg4ODg4ODg4ODg4ODg4P1u/QsEsAEgBiADQQT9a/0MMDAwMDAwMDAwMDAwMDAwMP1OIAX9DA8PDw8PDw8PDw8PDw8PDw/9Tv1Q/Qzg4ODg4ODg4ODg4ODg4ODg/W79CwSQASAHQcABaiEMQQAhAEEBIQlCACEZA0AgAEEHdCEUIAwgAEEDdGohFUEAIQADQCAVIABBAXRqIhYwAAAgBiAAQQV0IBRqIghq/QAEACID/YcBIAggDWr9AAAAIgT9hwH9ugEgA/2IASAE/YgB/boB/a4BIgP9GwAgA/0bAWogA/0bAmogA/0bA2qsfiAZfCAWMAABIAYgCEEQciIIav0ABAAiA/2HASAIIA1q/QAAACIE/YcB/boBIAP9iAEgBP2IAf26Af2uASID/RsAIAP9GwFqIAP9GwJqIAP9GwNqrH58IRkgAEEBaiIAQQRHDQALQQEhACAJQQAhCQ0ACyAPIApBAnRqIgAgACoCACAXIBiUIBm0lJI4AgAgB0HSAWohByAKQQFqIgogAkcNAAsgC0EBaiILIA5HDQALCyAGQYACaiQACxYAIAAgASACIAMgBCAFIAYgB0EREAYLqAECBH8BfQJAIAEgAk4NACAAKAIMIgVBAEoEQCAAKAIAIAEgBWxBsAFsaiEDIAAoAgQhBgNAQwAAAAAhB0EAIQQDQCADIAYgBEEKdGoQCSAHkiEHIANBsAFqIQMgBEEBaiIEIAVHDQALIAAoAgggAUECdGogBzgCACABQQFqIgEgAkcNAAsMAQsgAiABa0ECdCICRQ0AIAAoAgggAUECdGpBACAC/AsACwtlAQF/IwBBIGsiCCQAAkAgBEEATA0AIAVBAEwNACAIIAU2AhwgCCAEQQh2NgIYIAggAiADajYCDCAIIAYgB0ECdGo2AhQgCCAAIAFBAnRqNgIQQRAgCEEMaiAFEAELIAhBIGokAAuvAQEJfwJAIAEgAk4NACACIAFrQQJ0IgMEQCAAKAIIIAFBAnRqQQAgA/wLAAsgACgCDCIGQQBMDQAgACgCECEHIAAoAgAhCCAAKAIEIQkDQCAJIARBCnRqIQogCCAEIAdsIAFqQbABbGohBSABIQMDQCAAKAIIIANBAnRqIgsgCyoCACAFIAoQCZI4AgAgBUGwAWohBSADQQFqIgMgAkcNAAsgBEEBaiIEIAZHDQALCwtlAQF/IwBBIGsiCCQAAkAgBEEATA0AIAVBAEwNACAIIAU2AhwgCCAEQQh2NgIYIAggAiADajYCDCAIIAYgB0ECdGo2AhQgCCAAIAFBAnRqNgIQQQ8gCEEMaiAFEAELIAhBIGokAAu6AQIFfwF9AkAgASACTg0AIAAoAhAiBUEASgRAIAAoAgAgASAFbEGQAWxqIQQgACgCCCEGIAAoAgQhBwNAQwAAAAAhCEEAIQMDQCAEIAcgA0EIdGogBiADQQJ0aioCABAKIAiSIQggBEGQAWohBCADQQFqIgMgBUcNAAsgACgCDCABQQJ0aiAIOAIAIAFBAWoiASACRw0ACwwBCyACIAFrQQJ0IgJFDQAgACgCDCABQQJ0akEAIAL8CwALCxYAIAAgASACIAMgBCAFIAYgB0EOEAYLywECCn8CfQJAIAEgAk4NACACIAFrQQJ0IgMEQCAAKAIMIAFBAnRqQQAgA/wLAAsgACgCECIGQQBMDQAgACgCFCEHIAAoAgAhCCAAKAIIIQkgACgCBCEKA0AgCiAEQQh0aiELIAggBCAHbCABakGQAWxqIQUgCSAEQQJ0aioCACENIAEhAwNAIAUgCyANEAohDiAAKAIMIANBAnRqIgwgDiAMKgIAkjgCACAFQZABaiEFIANBAWoiAyACRw0ACyAEQQFqIgQgBkcNAAsLCxYAIAAgASACIAMgBCAFIAYgB0ENEAYLswQDAnsLfwF9AkAgASACTg0AIAAoAgghDSAAKAIMIglBAEwEQCACIAFrQQJ0IgBFDQEgDSABQQJ0akEAIAD8CwAPCyAAKAIAIAEgCWxBImxqIQogCUEibCEOIAAoAgQhDwNAQwAAAAAhEEEAIQsgCiEAA0AgDyALQQd0aiEGIAAtAAEiB0EYdEGAgICAeHEhBSAALQAAIAdBCHRyIghB/wdxIQwgCEH//wNxIQgCQAJAIAdBAnZBH3EiBwRAIAdBH0cNASAIQQ10IAVyQYCAgPwHciEFDAILIAxFDQEgBUGAgICYBHIgDGciBUEXdGsgCEEJIAVBH3NrdEEOdEGAgP8DcXIhBQwBCyAHQRd0IAxBDXRyIAVqQYCAgMADaiEFCyAG/QAAACAA/QAAAiID/YcBIgT9pwH9+gH95gEgBv0AABAgBP2oAf36Af3mAf3kASAG/QAAICAD/YgBIgP9pwH9+gH95gH95AEgBv0AADAgA/2oAf36Af3mAf3kASAG/QAAQCAA/QAAEiID/YcBIgT9pwH9+gH95gH95AEgBv0AAFAgBP2oAf36Af3mAf3kASAG/QAAYCAD/YgBIgP9pwH9+gH95gH95AEgBv0AAHAgA/2oAf36Af3mAf3kASID/R8AIAP9HwGSIAP9HwKSIAP9HwOSIAW+lCAQkiEQIABBImohACALQQFqIgsgCUcNAAsgDSABQQJ0aiAQOAIAIAogDmohCiABQQFqIgEgAkcNAAsLC4EBAQF/IwBBIGsiCCQAAkAgBUEATA0AIARBAEwEQCAFQQJ0IgBFDQEgBiAHQQJ0akEAIAD8CwAMAQsgCCAFNgIcIAggBEEFdjYCGCAIIAIgA2o2AgwgCCAGIAdBAnRqNgIUIAggACABQQJ0ajYCEEEMIAhBDGogBRABCyAIQSBqJAALqAQCAnsMfwJAIAEgAk4NACACIAFrQQJ0IgUEQCAAKAIIIAFBAnRqQQAgBfwLAAsgACgCDCIMQQBMDQAgACgCCCENIAAoAhAhDiAAKAIAIQ8gACgCBCEQA0AgECAKQQd0aiEFIA8gCiAObCABakEibGohByABIQADQCAHLQABIghBGHRBgICAgHhxIQYgBy0AACAIQQh0ciILQf8HcSEJIAtB//8DcSELAkACQCAIQQJ2QR9xIggEQCAIQR9HDQEgC0ENdCAGckGAgID8B3IhBgwCCyAJRQ0BIAZBgICAmARyIAlnIgZBF3RrIAtBCSAGQR9za3RBDnRBgID/A3FyIQYMAQsgCEEXdCAJQQ10ciAGakGAgIDAA2ohBgsgDSAAQQJ0aiIJIAX9AAAAIAf9AAACIgP9hwEiBP2nAf36Af3mASAF/QAAECAE/agB/foB/eYB/eQBIAX9AAAgIAP9iAEiA/2nAf36Af3mAf3kASAF/QAAMCAD/agB/foB/eYB/eQBIAX9AABAIAf9AAASIgP9hwEiBP2nAf36Af3mAf3kASAF/QAAUCAE/agB/foB/eYB/eQBIAX9AABgIAP9iAEiA/2nAf36Af3mAf3kASAF/QAAcCAD/agB/foB/eYB/eQBIgP9HwAgA/0fAZIgA/0fApIgA/0fA5IgBr6UIAkqAgCSOAIAIAdBImohByAAQQFqIgAgAkcNAAsgCkEBaiIKIAxHDQALCwuBAQEBfyMAQSBrIggkAAJAIAVBAEwNACAEQQBMBEAgBUECdCIARQ0BIAYgB0ECdGpBACAA/AsADAELIAggBTYCHCAIIARBBXY2AhggCCACIANqNgIMIAggBiAHQQJ0ajYCFCAIIAAgAUECdGo2AhBBCyAIQQxqIAUQAQsgCEEgaiQAC6oHAw9/BHsCfQJAIAEgAk4NACAAKAIIIQ0gACgCECIKQQBMBEAgAiABa0ECdCIARQ0BIA0gAUECdGpBACAA/AsADwsgACgCACABIApsQRhsaiELIApBGGwhDyAAKAIMIRAgACgCBCERA0BDAAAAACEWQQAhCSALIQYDQCAGLQABIg5BGHRBgICAgHhxIQcgBi0AA0EIdCIFIAYtAAJyIQAgBi0AACAOQQh0ciIDQf8HcSEMIANB//8DcSEEIBAgCUECdGoCQAJAIA5BAnZBH3EiAwRAIANBH0cNASAEQQ10IAdyQYCAgPwHciEHDAILIAxFDQEgB0GAgICYBHIgDGciA0EXdGsgBEEJIANBH3NrdEEOdEGAgP8DcXIhBwwBCyADQRd0IAxBDXRyIAdqQYCAgMADaiEHCyARIAlBB3RqIQQqAgAgAEH/B3EhCCAFQRB0QYCAgIB4cSEFAkACQCAAQQp2QR9xIgMEQCADQR9HDQEgAEENdCAFckGAgID8B3IhBQwCCyAIRQ0BIAVBgICAmARyIAhnIgNBF3RrIABBCSADQR9za3RBDnRBgID/A3FyIQUMAQsgA0EXdCAIQQ10ciAFakGAgIDAA2ohBQsgBb6UIBaSIAT9AABQIAb9CQAEIhT9DAICAgICAgICAwMDAwMDAwP9Dv0MAQIECBAgQIABAgQIECBAgP1O/QwAAAAAAAAAAAAAAAAAAAAA/ST9DBAQEBAQEBAQEBAQEBAQEBD9TiAG/QAACCIVQQT9bf1QIhP9iQEiEv2qAf36Af3mASAE/QAAQCAS/akB/foB/eYB/eQBIAT9AABgIBP9igEiEv2pAf36Af3mAf3kASAE/QAAcCAS/aoB/foB/eYB/eQBIAT9AAAQIBX9DA8PDw8PDw8PDw8PDw8PDw/9TiAU/QwAAAAAAAAAAAEBAQEBAQEB/Q79DAECBAgQIECAAQIECBAgQID9Tv0MAAAAAAAAAAAAAAAAAAAAAP0k/QwQEBAQEBAQEBAQEBAQEBAQ/U79UCIT/YkBIhL9qgH9+gH95gH95AEgBP0AAAAgEv2pAf36Af3mAf3kASAE/QAAICAT/YoBIhL9qQH9+gH95gH95AEgBP0AADAgEv2qAf36Af3mAf3kASIS/R8AIBL9HwGSIBL9HwKSIBL9HwOSIAe+lJIhFiAGQRhqIQYgCUEBaiIJIApHDQALIA0gAUECdGogFjgCACALIA9qIQsgAUEBaiIBIAJHDQALCwsWACAAIAEgAiADIAQgBSAGIAdBChAHC6MHAhB/BHsCQCABIAJODQAgAiABa0ECdCIJBEAgACgCCCABQQJ0akEAIAn8CwALIAAoAhAiC0EATA0AIAAoAgghDCAAKAIMIQ0gACgCFCEOIAAoAgAhDyAAKAIEIRADQCANIApBAnRqIREgECAKQQd0aiEGIA8gCiAObCABakEYbGohByABIQkDQCAHLQABIgRBGHRBgICAgHhxIQUgBy0AA0EIdCISIActAAJyIQAgBy0AACAEQQh0ciIIQf8HcSEDIAhB//8DcSEIAkACQCAEQQJ2QR9xIgQEQCAEQR9HDQEgCEENdCAFckGAgID8B3IhBQwCCyADRQ0BIAVBgICAmARyIANnIgVBF3RrIAhBCSAFQR9za3RBDnRBgID/A3FyIQUMAQsgBEEXdCADQQ10ciAFakGAgIDAA2ohBQsgAEH/B3EhBCASQRB0QYCAgIB4cSEDAkACQCAAQQp2QR9xIggEQCAIQR9HDQEgAEENdCADckGAgID8B3IhAwwCCyAERQ0BIANBgICAmARyIARnIgNBF3RrIABBCSADQR9za3RBDnRBgID/A3FyIQMMAQsgCEEXdCAEQQ10ciADakGAgIDAA2ohAwsgDCAJQQJ0aiIAIAAqAgAgESoCACADvpSSIAb9AABQIAf9CQAEIhP9DAICAgICAgICAwMDAwMDAwP9Dv0MAQIECBAgQIABAgQIECBAgP1O/QwAAAAAAAAAAAAAAAAAAAAA/ST9DBAQEBAQEBAQEBAQEBAQEBD9TiAH/QAACCIUQQT9bf1QIhX9iQEiFv2qAf36Af3mASAG/QAAQCAW/akB/foB/eYB/eQBIAb9AABgIBX9igEiFf2pAf36Af3mAf3kASAG/QAAcCAV/aoB/foB/eYB/eQBIAb9AAAQIBT9DA8PDw8PDw8PDw8PDw8PDw/9TiAT/QwAAAAAAAAAAAEBAQEBAQEB/Q79DAECBAgQIECAAQIECBAgQID9Tv0MAAAAAAAAAAAAAAAAAAAAAP0k/QwQEBAQEBAQEBAQEBAQEBAQ/U79UCIT/YkBIhT9qgH9+gH95gH95AEgBv0AAAAgFP2pAf36Af3mAf3kASAG/QAAICAT/YoBIhP9qQH9+gH95gH95AEgBv0AADAgE/2qAf36Af3mAf3kASIT/R8AIBP9HwGSIBP9HwKSIBP9HwOSIAW+lJI4AgAgB0EYaiEHIAlBAWoiCSACRw0ACyAKQQFqIgogC0cNAAsLCxYAIAAgASACIAMgBCAFIAYgB0EJEAcLKQAgACgCJARAIAAgASACQQAgACgCHBALDwsgAEEAIAAoAhggASACEAsLlAYDDH8EewJ9AkAgASACTg0AIAAoAgghCyAAKAIQIghBAEwEQCACIAFrQQJ0IgBFDQEgCyABQQJ0akEAIAD8CwAPCyAAKAIAIAEgCGxBFmxqIQkgCEEWbCEMIAAoAgwhDSAAKAIEIQ4DQEMAAAAAIRNBACEGIAkhAANAIA4gBkEHdGohBCAALQABIgVBGHRBgICAgHhxIQMgAC0AACAFQQh0ciIHQf8HcSEKIAdB//8DcSEHIA0gBkECdGoqAgAhFAJAAkAgBUECdkEfcSIFBEAgBUEfRw0BIAdBDXQgA3JBgICA/AdyIQMMAgsgCkUNASADQYCAgJgEciAKZyIDQRd0ayAHQQkgA0Efc2t0QQ50QYCA/wNxciEDDAELIAVBF3QgCkENdHIgA2pBgICAwANqIQMLIAT9AABQIAD9CQACIg/9DAICAgICAgICAwMDAwMDAwP9Dv0MAQIECBAgQIABAgQIECBAgP1O/QwAAAAAAAAAAAAAAAAAAAAA/ST9DBAQEBAQEBAQEBAQEBAQEBD9TiAA/QAABiIQQQT9bf1QIhH9iQEiEv2qAf36Af3mASAE/QAAQCAS/akB/foB/eYB/eQBIAT9AABgIBH9igEiEf2pAf36Af3mAf3kASAE/QAAcCAR/aoB/foB/eYB/eQBIAT9AAAQIBD9DA8PDw8PDw8PDw8PDw8PDw/9TiAP/QwAAAAAAAAAAAEBAQEBAQEB/Q79DAECBAgQIECAAQIECBAgQID9Tv0MAAAAAAAAAAAAAAAAAAAAAP0k/QwQEBAQEBAQEBAQEBAQEBAQ/U79UCIP/YkBIhD9qgH9+gH95gH95AEgBP0AAAAgEP2pAf36Af3mAf3kASAE/QAAICAP/YoBIg/9qQH9+gH95gH95AEgBP0AADAgD/2qAf36Af3mAf3kASIP/R8BIBRDAACAwZSSIA/9HwCSIA/9HwKSIA/9HwOSIAO+lCATkiETIABBFmohACAGQQFqIgYgCEcNAAsgCyABQQJ0aiATOAIAIAkgDGohCSABQQFqIgEgAkcNAAsLCxYAIAAgASACIAMgBCAFIAYgB0EIEAcLiQYCDn8EewJAIAEgAk4NACACIAFrQQJ0IgMEQCAAKAIIIAFBAnRqQQAgA/wLAAsgACgCECIKQQBMDQAgACgCCCELIAAoAgwhDCAAKAIUIQ0gACgCACEOIAAoAgQhDwNAIAwgBkECdGohECAPIAZBB3RqIQMgDiAGIA1sIAFqQRZsaiEFIAEhAANAIAUtAAEiB0EYdEGAgICAeHEhBCAFLQAAIAdBCHRyIglB/wdxIQggCUH//wNxIQkCQAJAIAdBAnZBH3EiBwRAIAdBH0cNASAJQQ10IARyQYCAgPwHciEEDAILIAhFDQEgBEGAgICYBHIgCGciBEEXdGsgCUEJIARBH3NrdEEOdEGAgP8DcXIhBAwBCyAHQRd0IAhBDXRyIARqQYCAgMADaiEECyALIABBAnRqIgggA/0AAFAgBf0JAAIiEf0MAgICAgICAgIDAwMDAwMDA/0O/QwBAgQIECBAgAECBAgQIECA/U79DAAAAAAAAAAAAAAAAAAAAAD9JP0MEBAQEBAQEBAQEBAQEBAQEP1OIAX9AAAGIhJBBP1t/VAiE/2JASIU/aoB/foB/eYBIAP9AABAIBT9qQH9+gH95gH95AEgA/0AAGAgE/2KASIT/akB/foB/eYB/eQBIAP9AABwIBP9qgH9+gH95gH95AEgA/0AABAgEv0MDw8PDw8PDw8PDw8PDw8PD/1OIBH9DAAAAAAAAAAAAQEBAQEBAQH9Dv0MAQIECBAgQIABAgQIECBAgP1O/QwAAAAAAAAAAAAAAAAAAAAA/ST9DBAQEBAQEBAQEBAQEBAQEBD9Tv1QIhH9iQEiEv2qAf36Af3mAf3kASAD/QAAACAS/akB/foB/eYB/eQBIAP9AAAgIBH9igEiEf2pAf36Af3mAf3kASAD/QAAMCAR/aoB/foB/eYB/eQBIhH9HwEgECoCAEMAAIDBlJIgEf0fAJIgEf0fApIgEf0fA5IgBL6UIAgqAgCSOAIAIAVBFmohBSAAQQFqIgAgAkcNAAsgBkEBaiIGIApHDQALCwsWACAAIAEgAiADIAQgBSAGIAdBBxAHC/EEAwt/A3sBfQJAIAEgAk4NACAAKAIIIQsgACgCDCIHQQBMBEAgAiABa0ECdCIARQ0BIAsgAUECdGpBACAA/AsADwsgACgCACABIAdsQRJsaiEIIAdBEmwhDCAAKAIEIQ0DQEMAAAAAIRFBACEJIAghAANAIA0gCUEHdGohBCAALQABIgVBGHRBgICAgHhxIQMgAC0AACAFQQh0ciIGQf8HcSEKIAZB//8DcSEGAkACQCAFQQJ2QR9xIgUEQCAFQR9HDQEgBkENdCADckGAgID8B3IhAwwCCyAKRQ0BIANBgICAmARyIApnIgNBF3RrIAZBCSADQR9za3RBDnRBgID/A3FyIQMMAQsgBUEXdCAKQQ10ciADakGAgIDAA2ohAwsgBP0AAAAgAP0AAAIiDv0MDw8PDw8PDw8PDw8PDw8PD/1O/Qz4+Pj4+Pj4+Pj4+Pj4+Pj4/W4iD/2HASIQ/acB/foB/eYBIAT9AAAQIBD9qAH9+gH95gH95AEgBP0AACAgD/2IASIP/acB/foB/eYB/eQBIAT9AAAwIA/9qAH9+gH95gH95AEgBP0AAEAgDkEE/W39DPj4+Pj4+Pj4+Pj4+Pj4+Pj9biIO/YcBIg/9pwH9+gH95gH95AEgBP0AAFAgD/2oAf36Af3mAf3kASAE/QAAYCAO/YgBIg79pwH9+gH95gH95AEgBP0AAHAgDv2oAf36Af3mAf3kASIO/R8AIA79HwGSIA79HwKSIA79HwOSIAO+lCARkiERIABBEmohACAJQQFqIgkgB0cNAAsgCyABQQJ0aiAROAIAIAggDGohCCABQQFqIgEgAkcNAAsLC4EBAQF/IwBBIGsiCCQAAkAgBUEATA0AIARBAEwEQCAFQQJ0IgBFDQEgBiAHQQJ0akEAIAD8CwAMAQsgCCAFNgIcIAggBEEFdjYCGCAIIAIgA2o2AgwgCCAGIAdBAnRqNgIUIAggACABQQJ0ajYCEEEGIAhBDGogBRABCyAIQSBqJAAL5gQCDH8DewJAIAEgAk4NACACIAFrQQJ0IgMEQCAAKAIIIAFBAnRqQQAgA/wLAAsgACgCDCIKQQBMDQAgACgCCCELIAAoAhAhDCAAKAIAIQ0gACgCBCEOA0AgDiAIQQd0aiEDIA0gCCAMbCABakESbGohBSABIQADQCAFLQABIgZBGHRBgICAgHhxIQQgBS0AACAGQQh0ciIJQf8HcSEHIAlB//8DcSEJAkACQCAGQQJ2QR9xIgYEQCAGQR9HDQEgCUENdCAEckGAgID8B3IhBAwCCyAHRQ0BIARBgICAmARyIAdnIgRBF3RrIAlBCSAEQR9za3RBDnRBgID/A3FyIQQMAQsgBkEXdCAHQQ10ciAEakGAgIDAA2ohBAsgCyAAQQJ0aiIHIAP9AAAAIAX9AAACIg/9DA8PDw8PDw8PDw8PDw8PDw/9Tv0M+Pj4+Pj4+Pj4+Pj4+Pj4+P1uIhD9hwEiEf2nAf36Af3mASAD/QAAECAR/agB/foB/eYB/eQBIAP9AAAgIBD9iAEiEP2nAf36Af3mAf3kASAD/QAAMCAQ/agB/foB/eYB/eQBIAP9AABAIA9BBP1t/Qz4+Pj4+Pj4+Pj4+Pj4+Pj4/W4iD/2HASIQ/acB/foB/eYB/eQBIAP9AABQIBD9qAH9+gH95gH95AEgA/0AAGAgD/2IASIP/acB/foB/eYB/eQBIAP9AABwIA/9qAH9+gH95gH95AEiD/0fACAP/R8BkiAP/R8CkiAP/R8DkiAEvpQgByoCAJI4AgAgBUESaiEFIABBAWoiACACRw0ACyAIQQFqIgggCkcNAAsLC4EBAQF/IwBBIGsiCCQAAkAgBUEATA0AIARBAEwEQCAFQQJ0IgBFDQEgBiAHQQJ0akEAIAD8CwAMAQsgCCAFNgIcIAggBEEFdjYCGCAIIAIgA2o2AgwgCCAGIAdBAnRqNgIUIAggACABQQJ0ajYCEEEFIAhBDGogBRABCyAIQSBqJAAL+hUDEn8EewF9IwBBgBBrIgwkAAJAIAAoAhgiCUEATA0AIAEgAk4NACAAKAIIIAFBAnRqIQUgAiABa0ECdCEEIAAoAhRBAnQhByAJQQRPBEAgCUH8////B3EhCwNAIARFIgpFBEAgBSADIAdsakEAIAT8CwALIApFBEAgBSAHIANBAXJsakEAIAT8CwALIApFBEAgBSAHIANBAnJsakEAIAT8CwALIApFBEAgBSAHIANBA3JsakEAIAT8CwALIANBBGohAyAIQQRqIgggC0cNAAsLIAlBA3EiCkUNAANAIAQEQCAFIAMgB2xqQQAgBPwLAAsgA0EBaiEDIAZBAWoiBiAKRw0ACwsCQCAAKAIgIgtBAEwNACAJQQFGBEAgAUEBdCENIAAoAgggAUECdGohCSAAKAIAIQ4gAiABayIKQQhIIRBBACEHA0AgACgCBCAAKAIQIAdsaiANaiEIIA4gB0ECdGoqAgAhGUEAIQUgEEUEQCAZ/RMhFkEAIQRBCCEGA0AgCSAEQQJ0aiIDIBYgCCAEQQF0av0AAAAiF/2pAUEN/asBIhX9DADg/w8A4P8PAOD/DwDg/w/9TiAV/QwAAIAPAACADwAAgA8AAIAP/U4iFf0MAACADwAAgA8AAIAPAACAD/03/QwAAAA4AAAAOAAAADgAAAA4/U79rgEiGP0MAACAOAAAgDgAAIA4AACAOP2uAf0MAACAuAAAgLgAAIC4AACAuP3kASAY/QwAAAA4AAAAOAAAADgAAAA4/a4BIBX9DAAAAAAAAAAAAAAAAAAAAAD9N/1SIBf9pwH9DAAAAIAAAACAAAAAgAAAAID9Tv1Q/eYBIAP9AAAA/eQB/QsAACADIBYgF/2qAUEN/asBIhX9DADg/w8A4P8PAOD/DwDg/w/9TiAV/QwAAIAPAACADwAAgA8AAIAP/U4iFf0MAACADwAAgA8AAIAPAACAD/03/QwAAAA4AAAAOAAAADgAAAA4/U79rgEiGP0MAACAOAAAgDgAAIA4AACAOP2uAf0MAACAuAAAgLgAAIC4AACAuP3kASAY/QwAAAA4AAAAOAAAADgAAAA4/a4BIBX9DAAAAAAAAAAAAAAAAAAAAAD9N/1SIBf9qAH9DAAAAIAAAACAAAAAgAAAAID9Tv1Q/eYBIAP9AAAQ/eQB/QsAECAGIgUiBEEIaiIGIApMDQALCwJAIAUgCk4NAAJAIAIgASAFamsiBkEESQRAIAUhAwwBCyAFIAZBfHEiC2ohAyAZ/RMhF0EAIQQDQCAJIAQgBWoiEUECdGoiEiAXIAggEUEBdGr9XQAAIhb9DP9//3//f/9//3//f/9//3/9Tv2pAUEN/asBIhUgFf0MAACADwAAgA8AAIAPAACAD/1OIhX9DAAAgA8AAIAPAACADwAAgA/9N/0MAAAAOAAAADgAAAA4AAAAOP1O/a4BIhj9DAAAgDgAAIA4AACAOAAAgDj9rgH9DAAAgLgAAIC4AACAuAAAgLj95AEgGP0MAAAAOAAAADgAAAA4AAAAOP2uASAV/QwAAAAAAAAAAAAAAAAAAAAA/Tf9UiAW/acB/QwAAACAAAAAgAAAAIAAAACA/U79UP3mASAS/QACAP3kAf0LAgAgBEEEaiIEIAtHDQALIAYgC0YNAQsDQCAJIANBAnRqIgQgGSAIIANBAXRqLwAAIgVB//8BcUENdCIGQYCAgMADQQAgBkGAgID8AHEiBkGAgID8AEYbaiILQYCAgMADaiALQYCAgMQDar5DAACAuJK8IAYbIAVBEHRBgICAgHhxcr6UIAQqAgCSOAIAIANBAWoiAyAKRw0ACwsgB0EBaiIHIAAoAiBIDQALDAELIAEgAk4NACACIAFrIREDQCALQQBKBEBBgAQgAiABayIOIA5BgAROGyEGQYAEIBEgDUEJdGsiAyADQYAEThsiEEEDcSEKIAFBAXQhEkEAIQcgDkEHSiEUA0AgACgCBCAAKAIQIAdsaiASaiEIQQghBEEAIQMgFARAA0AgDCADQQJ0aiIFIAggA0EBdGr9AAAAIhf9qgFBDf2rASIW/QwA4P8PAOD/DwDg/w8A4P8P/U4gFv0MAACADwAAgA8AAIAPAACAD/1OIhb9DAAAgA8AAIAPAACADwAAgA/9N/0MAAAAOAAAADgAAAA4AAAAOP1O/a4BIhX9DAAAgDgAAIA4AACAOAAAgDj9rgH9DAAAgLgAAIC4AACAuAAAgLj95AEgFf0MAAAAOAAAADgAAAA4AAAAOP2uASAW/QwAAAAAAAAAAAAAAAAAAAAA/Tf9UiAX/agB/QwAAACAAAAAgAAAAIAAAACA/U79UP0LBBAgBSAX/akBQQ39qwEiFv0MAOD/DwDg/w8A4P8PAOD/D/1OIBb9DAAAgA8AAIAPAACADwAAgA/9TiIW/QwAAIAPAACADwAAgA8AAIAP/Tf9DAAAADgAAAA4AAAAOAAAADj9Tv2uASIV/QwAAIA4AACAOAAAgDgAAIA4/a4B/QwAAIC4AACAuAAAgLgAAIC4/eQBIBX9DAAAADgAAAA4AAAAOAAAADj9rgEgFv0MAAAAAAAAAAAAAAAAAAAAAP03/VIgF/2nAf0MAAAAgAAAAIAAAACAAAAAgP1O/VD9CwQAIAQiA0EIaiIEIAZMDQALCwJAIAMgBk4NAAJAIBAgA2siBEEESQRAIAMhBAwBCyADIAQgCmsiD2ohBEEAIQUDQCAMIAMgBWoiE0ECdGogCCATQQF0av1dAAAiF/0M/3//f/9//3//f/9//3//f/1O/akBQQ39qwEiFiAW/QwAAIAPAACADwAAgA8AAIAP/U4iFv0MAACADwAAgA8AAIAPAACAD/03/QwAAAA4AAAAOAAAADgAAAA4/U79rgEiFf0MAACAOAAAgDgAAIA4AACAOP2uAf0MAACAuAAAgLgAAIC4AACAuP3kASAV/QwAAAA4AAAAOAAAADgAAAA4/a4BIBb9DAAAAAAAAAAAAAAAAAAAAAD9N/1SIBf9pwH9DAAAAIAAAACAAAAAgAAAAID9Tv1Q/QsCACAFQQRqIgUgD0cNAAsgCkUNAQsDQCAMIARBAnRqIAggBEEBdGovAAAiA0H//wFxQQ10IgVBgICAwANBACAFQYCAgPwAcSIFQYCAgPwARhtqIg9BgICAwANqIA9BgICAxANqvkMAAIC4krwgBRsgA0EQdEGAgICAeHFyNgIAIARBAWoiBCAGRw0ACwtBACEIIAlBAEoEQANAIAAoAgggACgCFCAIbEECdGogAUECdGohCSAAKAIAIAAoAgwgCGxBAnRqIAdBAnRqKgIAIRlBACEDIA5BBE4EQCAZ/RMhF0EAIQRBBCEFA0AgCSAEQQJ0IgNqIgQgAyAMav0ABAAgF/3mASAE/QAAAP3kAf0LAAAgBSIDIQQgA0EEaiIFIAZMDQALCwJAIAMgBk4NAAJAIBAgA2siBEEESQRAIAMhBAwBCyADIAQgCmsiC2ohBCAZ/RMhF0EAIQUDQCAJIAMgBWpBAnQiD2oiEyAT/QACACAMIA9q/QACACAX/eYB/eQB/QsCACAFQQRqIgUgC0cNAAsgCkUNAQsDQCAJIARBAnQiA2oiBSAFKgIAIAMgDGoqAgAgGZSSOAIAIARBAWoiBCAGRw0ACwsgCEEBaiIIIAAoAhgiCUgNAAsgACgCICELCyAHQQFqIgcgC0gNAAsLIA1BAWohDSABQYAEaiIBIAJIDQALCyAMQYAQaiQAC5sBAQF/IwBBMGsiDCQAAkAgCUEATA0AIApBAEwNACAMIAg2AhwgDCAFNgIYIAwgAjYCFCAMIAk2AiAgDCAKNgIkIAwgCSAKTzYCLCAMIAtBACALQQBKGzYCKCAMIAYgB0ECdGo2AhAgDCADIARBAnRqNgIMIAwgACABQQJ0ajYCCEEBIAxBCGogCSAKIAkgCksbEAELIAxBMGokAAsCAAsLCgEAQYAICwOgBgE=" - const val MT_MJS: String = "YXN5bmMgZnVuY3Rpb24gY3JlYXRlU2thaW5ldEtlcm5lbHMobW9kdWxlQXJnPXt9KXt2YXIgbW9kdWxlUnRuO3ZhciBNb2R1bGU9bW9kdWxlQXJnO3ZhciBFTlZJUk9OTUVOVF9JU19XRUI9dHlwZW9mIHdpbmRvdz09Im9iamVjdCI7dmFyIEVOVklST05NRU5UX0lTX1dPUktFUj10eXBlb2YgV29ya2VyR2xvYmFsU2NvcGUhPSJ1bmRlZmluZWQiO3ZhciBFTlZJUk9OTUVOVF9JU19OT0RFPXR5cGVvZiBwcm9jZXNzPT0ib2JqZWN0IiYmcHJvY2Vzcy52ZXJzaW9ucz8ubm9kZSYmcHJvY2Vzcy50eXBlIT0icmVuZGVyZXIiO3ZhciBFTlZJUk9OTUVOVF9JU19QVEhSRUFEPUVOVklST05NRU5UX0lTX1dPUktFUiYmc2VsZi5uYW1lPy5zdGFydHNXaXRoKCJlbS1wdGhyZWFkIik7aWYoRU5WSVJPTk1FTlRfSVNfTk9ERSl7Y29uc3R7Y3JlYXRlUmVxdWlyZX09YXdhaXQgaW1wb3J0KCJtb2R1bGUiKTt2YXIgcmVxdWlyZT1jcmVhdGVSZXF1aXJlKGltcG9ydC5tZXRhLnVybC5zdGFydHNXaXRoKCJmaWxlOiIpP2ltcG9ydC5tZXRhLnVybDpwcm9jZXNzLmN3ZCgpKyIvIik7dmFyIHdvcmtlcl90aHJlYWRzPXJlcXVpcmUoIndvcmtlcl90aHJlYWRzIik7Z2xvYmFsLldvcmtlcj13b3JrZXJfdGhyZWFkcy5Xb3JrZXI7RU5WSVJPTk1FTlRfSVNfV09SS0VSPSF3b3JrZXJfdGhyZWFkcy5pc01haW5UaHJlYWQ7RU5WSVJPTk1FTlRfSVNfUFRIUkVBRD1FTlZJUk9OTUVOVF9JU19XT1JLRVImJndvcmtlcl90aHJlYWRzWyJ3b3JrZXJEYXRhIl09PSJlbS1wdGhyZWFkIn12YXIgYXJndW1lbnRzXz1bXTt2YXIgdGhpc1Byb2dyYW09Ii4vdGhpcy5wcm9ncmFtIjt2YXIgcXVpdF89KHN0YXR1cyx0b1Rocm93KT0+e3Rocm93IHRvVGhyb3d9O3ZhciBfc2NyaXB0TmFtZT1pbXBvcnQubWV0YS51cmw7dmFyIHNjcmlwdERpcmVjdG9yeT0iIjtmdW5jdGlvbiBsb2NhdGVGaWxlKHBhdGgpe2lmKE1vZHVsZVsibG9jYXRlRmlsZSJdKXtyZXR1cm4gTW9kdWxlWyJsb2NhdGVGaWxlIl0ocGF0aCxzY3JpcHREaXJlY3RvcnkpfXJldHVybiBzY3JpcHREaXJlY3RvcnkrcGF0aH12YXIgcmVhZEFzeW5jLHJlYWRCaW5hcnk7aWYoRU5WSVJPTk1FTlRfSVNfTk9ERSl7dmFyIGZzPXJlcXVpcmUoImZzIik7aWYoX3NjcmlwdE5hbWUuc3RhcnRzV2l0aCgiZmlsZToiKSl7c2NyaXB0RGlyZWN0b3J5PXJlcXVpcmUoInBhdGgiKS5kaXJuYW1lKHJlcXVpcmUoInVybCIpLmZpbGVVUkxUb1BhdGgoX3NjcmlwdE5hbWUpKSsiLyJ9cmVhZEJpbmFyeT1maWxlbmFtZT0+e2ZpbGVuYW1lPWlzRmlsZVVSSShmaWxlbmFtZSk/bmV3IFVSTChmaWxlbmFtZSk6ZmlsZW5hbWU7dmFyIHJldD1mcy5yZWFkRmlsZVN5bmMoZmlsZW5hbWUpO3JldHVybiByZXR9O3JlYWRBc3luYz1hc3luYyhmaWxlbmFtZSxiaW5hcnk9dHJ1ZSk9PntmaWxlbmFtZT1pc0ZpbGVVUkkoZmlsZW5hbWUpP25ldyBVUkwoZmlsZW5hbWUpOmZpbGVuYW1lO3ZhciByZXQ9ZnMucmVhZEZpbGVTeW5jKGZpbGVuYW1lLGJpbmFyeT91bmRlZmluZWQ6InV0ZjgiKTtyZXR1cm4gcmV0fTtpZihwcm9jZXNzLmFyZ3YubGVuZ3RoPjEpe3RoaXNQcm9ncmFtPXByb2Nlc3MuYXJndlsxXS5yZXBsYWNlKC9cXC9nLCIvIil9YXJndW1lbnRzXz1wcm9jZXNzLmFyZ3Yuc2xpY2UoMik7cXVpdF89KHN0YXR1cyx0b1Rocm93KT0+e3Byb2Nlc3MuZXhpdENvZGU9c3RhdHVzO3Rocm93IHRvVGhyb3d9fWVsc2UgaWYoRU5WSVJPTk1FTlRfSVNfV0VCfHxFTlZJUk9OTUVOVF9JU19XT1JLRVIpe3RyeXtzY3JpcHREaXJlY3Rvcnk9bmV3IFVSTCgiLiIsX3NjcmlwdE5hbWUpLmhyZWZ9Y2F0Y2h7fWlmKCFFTlZJUk9OTUVOVF9JU19OT0RFKXtpZihFTlZJUk9OTUVOVF9JU19XT1JLRVIpe3JlYWRCaW5hcnk9dXJsPT57dmFyIHhocj1uZXcgWE1MSHR0cFJlcXVlc3Q7eGhyLm9wZW4oIkdFVCIsdXJsLGZhbHNlKTt4aHIucmVzcG9uc2VUeXBlPSJhcnJheWJ1ZmZlciI7eGhyLnNlbmQobnVsbCk7cmV0dXJuIG5ldyBVaW50OEFycmF5KHhoci5yZXNwb25zZSl9fXJlYWRBc3luYz1hc3luYyB1cmw9Pnt2YXIgcmVzcG9uc2U9YXdhaXQgZmV0Y2godXJsLHtjcmVkZW50aWFsczoic2FtZS1vcmlnaW4ifSk7aWYocmVzcG9uc2Uub2spe3JldHVybiByZXNwb25zZS5hcnJheUJ1ZmZlcigpfXRocm93IG5ldyBFcnJvcihyZXNwb25zZS5zdGF0dXMrIiA6ICIrcmVzcG9uc2UudXJsKX19fWVsc2V7fXZhciBkZWZhdWx0UHJpbnQ9Y29uc29sZS5sb2cuYmluZChjb25zb2xlKTt2YXIgZGVmYXVsdFByaW50RXJyPWNvbnNvbGUuZXJyb3IuYmluZChjb25zb2xlKTtpZihFTlZJUk9OTUVOVF9JU19OT0RFKXt2YXIgdXRpbHM9cmVxdWlyZSgidXRpbCIpO3ZhciBzdHJpbmdpZnk9YT0+dHlwZW9mIGE9PSJvYmplY3QiP3V0aWxzLmluc3BlY3QoYSk6YTtkZWZhdWx0UHJpbnQ9KC4uLmFyZ3MpPT5mcy53cml0ZVN5bmMoMSxhcmdzLm1hcChzdHJpbmdpZnkpLmpvaW4oIiAiKSsiXG4iKTtkZWZhdWx0UHJpbnRFcnI9KC4uLmFyZ3MpPT5mcy53cml0ZVN5bmMoMixhcmdzLm1hcChzdHJpbmdpZnkpLmpvaW4oIiAiKSsiXG4iKX12YXIgb3V0PWRlZmF1bHRQcmludDt2YXIgZXJyPWRlZmF1bHRQcmludEVycjt2YXIgd2FzbUJpbmFyeTt2YXIgd2FzbU1vZHVsZTt2YXIgQUJPUlQ9ZmFsc2U7dmFyIEVYSVRTVEFUVVM7dmFyIGlzRmlsZVVSST1maWxlbmFtZT0+ZmlsZW5hbWUuc3RhcnRzV2l0aCgiZmlsZTovLyIpO2Z1bmN0aW9uIGdyb3dNZW1WaWV3cygpe2lmKHdhc21NZW1vcnkuYnVmZmVyIT1IRUFQOC5idWZmZXIpe3VwZGF0ZU1lbW9yeVZpZXdzKCl9fXZhciByZWFkeVByb21pc2VSZXNvbHZlLHJlYWR5UHJvbWlzZVJlamVjdDtpZihFTlZJUk9OTUVOVF9JU19OT0RFJiZFTlZJUk9OTUVOVF9JU19QVEhSRUFEKXt2YXIgcGFyZW50UG9ydD13b3JrZXJfdGhyZWFkc1sicGFyZW50UG9ydCJdO3BhcmVudFBvcnQub24oIm1lc3NhZ2UiLG1zZz0+Z2xvYmFsLm9ubWVzc2FnZT8uKHtkYXRhOm1zZ30pKTtPYmplY3QuYXNzaWduKGdsb2JhbFRoaXMse3NlbGY6Z2xvYmFsLHBvc3RNZXNzYWdlOm1zZz0+cGFyZW50UG9ydFsicG9zdE1lc3NhZ2UiXShtc2cpfSk7cHJvY2Vzcy5vbigidW5jYXVnaHRFeGNlcHRpb24iLGVycj0+e3Bvc3RNZXNzYWdlKHtjbWQ6InVuY2F1Z2h0RXhjZXB0aW9uIixlcnJvcjplcnJ9KTtwcm9jZXNzLmV4aXQoMSl9KX12YXIgc3RhcnRXb3JrZXI7aWYoRU5WSVJPTk1FTlRfSVNfUFRIUkVBRCl7dmFyIGluaXRpYWxpemVkSlM9ZmFsc2U7c2VsZi5vbnVuaGFuZGxlZHJlamVjdGlvbj1lPT57dGhyb3cgZS5yZWFzb258fGV9O2Z1bmN0aW9uIGhhbmRsZU1lc3NhZ2UoZSl7dHJ5e3ZhciBtc2dEYXRhPWVbImRhdGEiXTt2YXIgY21kPW1zZ0RhdGEuY21kO2lmKGNtZD09PSJsb2FkIil7bGV0IG1lc3NhZ2VRdWV1ZT1bXTtzZWxmLm9ubWVzc2FnZT1lPT5tZXNzYWdlUXVldWUucHVzaChlKTtzdGFydFdvcmtlcj0oKT0+e3Bvc3RNZXNzYWdlKHtjbWQ6ImxvYWRlZCJ9KTtmb3IobGV0IG1zZyBvZiBtZXNzYWdlUXVldWUpe2hhbmRsZU1lc3NhZ2UobXNnKX1zZWxmLm9ubWVzc2FnZT1oYW5kbGVNZXNzYWdlfTtmb3IoY29uc3QgaGFuZGxlciBvZiBtc2dEYXRhLmhhbmRsZXJzKXtpZighTW9kdWxlW2hhbmRsZXJdfHxNb2R1bGVbaGFuZGxlcl0ucHJveHkpe01vZHVsZVtoYW5kbGVyXT0oLi4uYXJncyk9Pntwb3N0TWVzc2FnZSh7Y21kOiJjYWxsSGFuZGxlciIsaGFuZGxlcixhcmdzfSl9O2lmKGhhbmRsZXI9PSJwcmludCIpb3V0PU1vZHVsZVtoYW5kbGVyXTtpZihoYW5kbGVyPT0icHJpbnRFcnIiKWVycj1Nb2R1bGVbaGFuZGxlcl19fXdhc21NZW1vcnk9bXNnRGF0YS53YXNtTWVtb3J5O3VwZGF0ZU1lbW9yeVZpZXdzKCk7d2FzbU1vZHVsZT1tc2dEYXRhLndhc21Nb2R1bGU7Y3JlYXRlV2FzbSgpO3J1bigpfWVsc2UgaWYoY21kPT09InJ1biIpe2VzdGFibGlzaFN0YWNrU3BhY2UobXNnRGF0YS5wdGhyZWFkX3B0cik7X19lbXNjcmlwdGVuX3RocmVhZF9pbml0KG1zZ0RhdGEucHRocmVhZF9wdHIsMCwwLDEsMCwwKTtQVGhyZWFkLnRocmVhZEluaXRUTFMoKTtfX2Vtc2NyaXB0ZW5fdGhyZWFkX21haWxib3hfYXdhaXQobXNnRGF0YS5wdGhyZWFkX3B0cik7aWYoIWluaXRpYWxpemVkSlMpe2luaXRpYWxpemVkSlM9dHJ1ZX10cnl7aW52b2tlRW50cnlQb2ludChtc2dEYXRhLnN0YXJ0X3JvdXRpbmUsbXNnRGF0YS5hcmcpfWNhdGNoKGV4KXtpZihleCE9InVud2luZCIpe3Rocm93IGV4fX19ZWxzZSBpZihtc2dEYXRhLnRhcmdldD09PSJzZXRpbW1lZGlhdGUiKXt9ZWxzZSBpZihjbWQ9PT0iY2hlY2tNYWlsYm94Iil7aWYoaW5pdGlhbGl6ZWRKUyl7Y2hlY2tNYWlsYm94KCl9fWVsc2UgaWYoY21kKXtlcnIoYHdvcmtlcjogcmVjZWl2ZWQgdW5rbm93biBjb21tYW5kICR7Y21kfWApO2Vycihtc2dEYXRhKX19Y2F0Y2goZXgpe19fZW1zY3JpcHRlbl90aHJlYWRfY3Jhc2hlZCgpO3Rocm93IGV4fX1zZWxmLm9ubWVzc2FnZT1oYW5kbGVNZXNzYWdlfXZhciB3YXNtTWVtb3J5O3ZhciBIRUFQOCxIRUFQVTgsSEVBUDE2LEhFQVBVMTYsSEVBUDMyLEhFQVBVMzIsSEVBUEYzMixIRUFQRjY0O3ZhciBIRUFQNjQsSEVBUFU2NDt2YXIgcnVudGltZUluaXRpYWxpemVkPWZhbHNlO2Z1bmN0aW9uIHVwZGF0ZU1lbW9yeVZpZXdzKCl7dmFyIGI9d2FzbU1lbW9yeS5idWZmZXI7SEVBUDg9bmV3IEludDhBcnJheShiKTtIRUFQMTY9bmV3IEludDE2QXJyYXkoYik7SEVBUFU4PW5ldyBVaW50OEFycmF5KGIpO0hFQVBVMTY9bmV3IFVpbnQxNkFycmF5KGIpO0hFQVAzMj1uZXcgSW50MzJBcnJheShiKTtIRUFQVTMyPW5ldyBVaW50MzJBcnJheShiKTtIRUFQRjMyPW5ldyBGbG9hdDMyQXJyYXkoYik7SEVBUEY2ND1uZXcgRmxvYXQ2NEFycmF5KGIpO0hFQVA2ND1uZXcgQmlnSW50NjRBcnJheShiKTtIRUFQVTY0PW5ldyBCaWdVaW50NjRBcnJheShiKX1mdW5jdGlvbiBpbml0TWVtb3J5KCl7aWYoRU5WSVJPTk1FTlRfSVNfUFRIUkVBRCl7cmV0dXJufWlmKE1vZHVsZVsid2FzbU1lbW9yeSJdKXt3YXNtTWVtb3J5PU1vZHVsZVsid2FzbU1lbW9yeSJdfWVsc2V7dmFyIElOSVRJQUxfTUVNT1JZPU1vZHVsZVsiSU5JVElBTF9NRU1PUlkiXXx8NjcxMDg4NjQ7d2FzbU1lbW9yeT1uZXcgV2ViQXNzZW1ibHkuTWVtb3J5KHtpbml0aWFsOklOSVRJQUxfTUVNT1JZLzY1NTM2LG1heGltdW06NjU1MzYsc2hhcmVkOnRydWV9KX11cGRhdGVNZW1vcnlWaWV3cygpfWZ1bmN0aW9uIHByZVJ1bigpe2lmKE1vZHVsZVsicHJlUnVuIl0pe2lmKHR5cGVvZiBNb2R1bGVbInByZVJ1biJdPT0iZnVuY3Rpb24iKU1vZHVsZVsicHJlUnVuIl09W01vZHVsZVsicHJlUnVuIl1dO3doaWxlKE1vZHVsZVsicHJlUnVuIl0ubGVuZ3RoKXthZGRPblByZVJ1bihNb2R1bGVbInByZVJ1biJdLnNoaWZ0KCkpfX1jYWxsUnVudGltZUNhbGxiYWNrcyhvblByZVJ1bnMpfWZ1bmN0aW9uIGluaXRSdW50aW1lKCl7cnVudGltZUluaXRpYWxpemVkPXRydWU7aWYoRU5WSVJPTk1FTlRfSVNfUFRIUkVBRClyZXR1cm4gc3RhcnRXb3JrZXIoKTt3YXNtRXhwb3J0c1sibyJdKCl9ZnVuY3Rpb24gcG9zdFJ1bigpe2lmKEVOVklST05NRU5UX0lTX1BUSFJFQUQpe3JldHVybn1pZihNb2R1bGVbInBvc3RSdW4iXSl7aWYodHlwZW9mIE1vZHVsZVsicG9zdFJ1biJdPT0iZnVuY3Rpb24iKU1vZHVsZVsicG9zdFJ1biJdPVtNb2R1bGVbInBvc3RSdW4iXV07d2hpbGUoTW9kdWxlWyJwb3N0UnVuIl0ubGVuZ3RoKXthZGRPblBvc3RSdW4oTW9kdWxlWyJwb3N0UnVuIl0uc2hpZnQoKSl9fWNhbGxSdW50aW1lQ2FsbGJhY2tzKG9uUG9zdFJ1bnMpfWZ1bmN0aW9uIGFib3J0KHdoYXQpe01vZHVsZVsib25BYm9ydCJdPy4od2hhdCk7d2hhdD0iQWJvcnRlZCgiK3doYXQrIikiO2Vycih3aGF0KTtBQk9SVD10cnVlO3doYXQrPSIuIEJ1aWxkIHdpdGggLXNBU1NFUlRJT05TIGZvciBtb3JlIGluZm8uIjt2YXIgZT1uZXcgV2ViQXNzZW1ibHkuUnVudGltZUVycm9yKHdoYXQpO3JlYWR5UHJvbWlzZVJlamVjdD8uKGUpO3Rocm93IGV9dmFyIHdhc21CaW5hcnlGaWxlO2Z1bmN0aW9uIGZpbmRXYXNtQmluYXJ5KCl7aWYoTW9kdWxlWyJsb2NhdGVGaWxlIl0pe3JldHVybiBsb2NhdGVGaWxlKCJza2FpbmV0X2tlcm5lbHNfbXQud2FzbSIpfXRyeXtyZXR1cm4gbmV3IFVSTCgic2thaW5ldF9rZXJuZWxzX210Lndhc20iLGltcG9ydC5tZXRhLnVybCkuaHJlZn1jYXRjaChlKXtyZXR1cm4gInNrYWluZXRfa2VybmVsc19tdC53YXNtIn19ZnVuY3Rpb24gZ2V0QmluYXJ5U3luYyhmaWxlKXtpZihmaWxlPT13YXNtQmluYXJ5RmlsZSYmd2FzbUJpbmFyeSl7cmV0dXJuIG5ldyBVaW50OEFycmF5KHdhc21CaW5hcnkpfWlmKHJlYWRCaW5hcnkpe3JldHVybiByZWFkQmluYXJ5KGZpbGUpfXRocm93ImJvdGggYXN5bmMgYW5kIHN5bmMgZmV0Y2hpbmcgb2YgdGhlIHdhc20gZmFpbGVkIn1hc3luYyBmdW5jdGlvbiBnZXRXYXNtQmluYXJ5KGJpbmFyeUZpbGUpe2lmKCF3YXNtQmluYXJ5KXt0cnl7dmFyIHJlc3BvbnNlPWF3YWl0IHJlYWRBc3luYyhiaW5hcnlGaWxlKTtyZXR1cm4gbmV3IFVpbnQ4QXJyYXkocmVzcG9uc2UpfWNhdGNoe319cmV0dXJuIGdldEJpbmFyeVN5bmMoYmluYXJ5RmlsZSl9YXN5bmMgZnVuY3Rpb24gaW5zdGFudGlhdGVBcnJheUJ1ZmZlcihiaW5hcnlGaWxlLGltcG9ydHMpe3RyeXt2YXIgYmluYXJ5PWF3YWl0IGdldFdhc21CaW5hcnkoYmluYXJ5RmlsZSk7dmFyIGluc3RhbmNlPWF3YWl0IFdlYkFzc2VtYmx5Lmluc3RhbnRpYXRlKGJpbmFyeSxpbXBvcnRzKTtyZXR1cm4gaW5zdGFuY2V9Y2F0Y2gocmVhc29uKXtlcnIoYGZhaWxlZCB0byBhc3luY2hyb25vdXNseSBwcmVwYXJlIHdhc206ICR7cmVhc29ufWApO2Fib3J0KHJlYXNvbil9fWFzeW5jIGZ1bmN0aW9uIGluc3RhbnRpYXRlQXN5bmMoYmluYXJ5LGJpbmFyeUZpbGUsaW1wb3J0cyl7aWYoIWJpbmFyeSYmIUVOVklST05NRU5UX0lTX05PREUpe3RyeXt2YXIgcmVzcG9uc2U9ZmV0Y2goYmluYXJ5RmlsZSx7Y3JlZGVudGlhbHM6InNhbWUtb3JpZ2luIn0pO3ZhciBpbnN0YW50aWF0aW9uUmVzdWx0PWF3YWl0IFdlYkFzc2VtYmx5Lmluc3RhbnRpYXRlU3RyZWFtaW5nKHJlc3BvbnNlLGltcG9ydHMpO3JldHVybiBpbnN0YW50aWF0aW9uUmVzdWx0fWNhdGNoKHJlYXNvbil7ZXJyKGB3YXNtIHN0cmVhbWluZyBjb21waWxlIGZhaWxlZDogJHtyZWFzb259YCk7ZXJyKCJmYWxsaW5nIGJhY2sgdG8gQXJyYXlCdWZmZXIgaW5zdGFudGlhdGlvbiIpfX1yZXR1cm4gaW5zdGFudGlhdGVBcnJheUJ1ZmZlcihiaW5hcnlGaWxlLGltcG9ydHMpfWZ1bmN0aW9uIGdldFdhc21JbXBvcnRzKCl7YXNzaWduV2FzbUltcG9ydHMoKTtyZXR1cm57YTp3YXNtSW1wb3J0c319YXN5bmMgZnVuY3Rpb24gY3JlYXRlV2FzbSgpe2Z1bmN0aW9uIHJlY2VpdmVJbnN0YW5jZShpbnN0YW5jZSxtb2R1bGUpe3dhc21FeHBvcnRzPWluc3RhbmNlLmV4cG9ydHM7d2FzbUV4cG9ydHM9YXBwbHlTaWduYXR1cmVDb252ZXJzaW9ucyh3YXNtRXhwb3J0cyk7cmVnaXN0ZXJUTFNJbml0KHdhc21FeHBvcnRzWyJLIl0pO3dhc21UYWJsZT13YXNtRXhwb3J0c1sicSJdO3dhc21Nb2R1bGU9bW9kdWxlO2Fzc2lnbldhc21FeHBvcnRzKHdhc21FeHBvcnRzKTtyZXR1cm4gd2FzbUV4cG9ydHN9ZnVuY3Rpb24gcmVjZWl2ZUluc3RhbnRpYXRpb25SZXN1bHQocmVzdWx0KXtyZXR1cm4gcmVjZWl2ZUluc3RhbmNlKHJlc3VsdFsiaW5zdGFuY2UiXSxyZXN1bHRbIm1vZHVsZSJdKX12YXIgaW5mbz1nZXRXYXNtSW1wb3J0cygpO2lmKE1vZHVsZVsiaW5zdGFudGlhdGVXYXNtIl0pe3JldHVybiBuZXcgUHJvbWlzZSgocmVzb2x2ZSxyZWplY3QpPT57TW9kdWxlWyJpbnN0YW50aWF0ZVdhc20iXShpbmZvLChtb2QsaW5zdCk9PntyZXNvbHZlKHJlY2VpdmVJbnN0YW5jZShtb2QsaW5zdCkpfSl9KX1pZihFTlZJUk9OTUVOVF9JU19QVEhSRUFEKXt2YXIgaW5zdGFuY2U9bmV3IFdlYkFzc2VtYmx5Lkluc3RhbmNlKHdhc21Nb2R1bGUsZ2V0V2FzbUltcG9ydHMoKSk7cmV0dXJuIHJlY2VpdmVJbnN0YW5jZShpbnN0YW5jZSx3YXNtTW9kdWxlKX13YXNtQmluYXJ5RmlsZT8/PWZpbmRXYXNtQmluYXJ5KCk7dmFyIHJlc3VsdD1hd2FpdCBpbnN0YW50aWF0ZUFzeW5jKHdhc21CaW5hcnksd2FzbUJpbmFyeUZpbGUsaW5mbyk7dmFyIGV4cG9ydHM9cmVjZWl2ZUluc3RhbnRpYXRpb25SZXN1bHQocmVzdWx0KTtyZXR1cm4gZXhwb3J0c31jbGFzcyBFeGl0U3RhdHVze25hbWU9IkV4aXRTdGF0dXMiO2NvbnN0cnVjdG9yKHN0YXR1cyl7dGhpcy5tZXNzYWdlPWBQcm9ncmFtIHRlcm1pbmF0ZWQgd2l0aCBleGl0KCR7c3RhdHVzfSlgO3RoaXMuc3RhdHVzPXN0YXR1c319dmFyIHRlcm1pbmF0ZVdvcmtlcj13b3JrZXI9Pnt3b3JrZXIudGVybWluYXRlKCk7d29ya2VyLm9ubWVzc2FnZT1lPT57fX07dmFyIGNsZWFudXBUaHJlYWQ9cHRocmVhZF9wdHI9Pnt2YXIgd29ya2VyPVBUaHJlYWQucHRocmVhZHNbcHRocmVhZF9wdHJdO1BUaHJlYWQucmV0dXJuV29ya2VyVG9Qb29sKHdvcmtlcil9O3ZhciBjYWxsUnVudGltZUNhbGxiYWNrcz1jYWxsYmFja3M9Pnt3aGlsZShjYWxsYmFja3MubGVuZ3RoPjApe2NhbGxiYWNrcy5zaGlmdCgpKE1vZHVsZSl9fTt2YXIgb25QcmVSdW5zPVtdO3ZhciBhZGRPblByZVJ1bj1jYj0+b25QcmVSdW5zLnB1c2goY2IpO3ZhciBydW5EZXBlbmRlbmNpZXM9MDt2YXIgZGVwZW5kZW5jaWVzRnVsZmlsbGVkPW51bGw7dmFyIHJlbW92ZVJ1bkRlcGVuZGVuY3k9aWQ9PntydW5EZXBlbmRlbmNpZXMtLTtNb2R1bGVbIm1vbml0b3JSdW5EZXBlbmRlbmNpZXMiXT8uKHJ1bkRlcGVuZGVuY2llcyk7aWYocnVuRGVwZW5kZW5jaWVzPT0wKXtpZihkZXBlbmRlbmNpZXNGdWxmaWxsZWQpe3ZhciBjYWxsYmFjaz1kZXBlbmRlbmNpZXNGdWxmaWxsZWQ7ZGVwZW5kZW5jaWVzRnVsZmlsbGVkPW51bGw7Y2FsbGJhY2soKX19fTt2YXIgYWRkUnVuRGVwZW5kZW5jeT1pZD0+e3J1bkRlcGVuZGVuY2llcysrO01vZHVsZVsibW9uaXRvclJ1bkRlcGVuZGVuY2llcyJdPy4ocnVuRGVwZW5kZW5jaWVzKX07dmFyIHNwYXduVGhyZWFkPXRocmVhZFBhcmFtcz0+e3ZhciB3b3JrZXI9UFRocmVhZC5nZXROZXdXb3JrZXIoKTtpZighd29ya2VyKXtyZXR1cm4gNn1QVGhyZWFkLnJ1bm5pbmdXb3JrZXJzLnB1c2god29ya2VyKTtQVGhyZWFkLnB0aHJlYWRzW3RocmVhZFBhcmFtcy5wdGhyZWFkX3B0cl09d29ya2VyO3dvcmtlci5wdGhyZWFkX3B0cj10aHJlYWRQYXJhbXMucHRocmVhZF9wdHI7dmFyIG1zZz17Y21kOiJydW4iLHN0YXJ0X3JvdXRpbmU6dGhyZWFkUGFyYW1zLnN0YXJ0Um91dGluZSxhcmc6dGhyZWFkUGFyYW1zLmFyZyxwdGhyZWFkX3B0cjp0aHJlYWRQYXJhbXMucHRocmVhZF9wdHJ9O2lmKEVOVklST05NRU5UX0lTX05PREUpe3dvcmtlci51bnJlZigpfXdvcmtlci5wb3N0TWVzc2FnZShtc2csdGhyZWFkUGFyYW1zLnRyYW5zZmVyTGlzdCk7cmV0dXJuIDB9O3ZhciBydW50aW1lS2VlcGFsaXZlQ291bnRlcj0wO3ZhciBrZWVwUnVudGltZUFsaXZlPSgpPT5ub0V4aXRSdW50aW1lfHxydW50aW1lS2VlcGFsaXZlQ291bnRlcj4wO3ZhciBzdGFja1NhdmU9KCk9Pl9lbXNjcmlwdGVuX3N0YWNrX2dldF9jdXJyZW50KCk7dmFyIHN0YWNrUmVzdG9yZT12YWw9Pl9fZW1zY3JpcHRlbl9zdGFja19yZXN0b3JlKHZhbCk7dmFyIHN0YWNrQWxsb2M9c3o9Pl9fZW1zY3JpcHRlbl9zdGFja19hbGxvYyhzeik7dmFyIHByb3h5VG9NYWluVGhyZWFkPShmdW5jSW5kZXgsZW1Bc21BZGRyLHN5bmMsLi4uY2FsbEFyZ3MpPT57dmFyIHNlcmlhbGl6ZWROdW1DYWxsQXJncz1jYWxsQXJncy5sZW5ndGgqMjt2YXIgc3A9c3RhY2tTYXZlKCk7dmFyIGFyZ3M9c3RhY2tBbGxvYyhzZXJpYWxpemVkTnVtQ2FsbEFyZ3MqOCk7dmFyIGI9YXJncz4+PjM7Zm9yKHZhciBpPTA7aTxjYWxsQXJncy5sZW5ndGg7aSsrKXt2YXIgYXJnPWNhbGxBcmdzW2ldO2lmKHR5cGVvZiBhcmc9PSJiaWdpbnQiKXsoZ3Jvd01lbVZpZXdzKCksSEVBUDY0KVtiKzIqaT4+PjBdPTFuOyhncm93TWVtVmlld3MoKSxIRUFQNjQpW2IrMippKzE+Pj4wXT1hcmd9ZWxzZXsoZ3Jvd01lbVZpZXdzKCksSEVBUDY0KVtiKzIqaT4+PjBdPTBuOyhncm93TWVtVmlld3MoKSxIRUFQRjY0KVtiKzIqaSsxPj4+MF09YXJnfX12YXIgcnRuPV9fZW1zY3JpcHRlbl9ydW5fanNfb25fbWFpbl90aHJlYWQoZnVuY0luZGV4LGVtQXNtQWRkcixzZXJpYWxpemVkTnVtQ2FsbEFyZ3MsYXJncyxzeW5jKTtzdGFja1Jlc3RvcmUoc3ApO3JldHVybiBydG59O2Z1bmN0aW9uIF9wcm9jX2V4aXQoY29kZSl7aWYoRU5WSVJPTk1FTlRfSVNfUFRIUkVBRClyZXR1cm4gcHJveHlUb01haW5UaHJlYWQoMCwwLDEsY29kZSk7RVhJVFNUQVRVUz1jb2RlO2lmKCFrZWVwUnVudGltZUFsaXZlKCkpe1BUaHJlYWQudGVybWluYXRlQWxsVGhyZWFkcygpO01vZHVsZVsib25FeGl0Il0/Lihjb2RlKTtBQk9SVD10cnVlfXF1aXRfKGNvZGUsbmV3IEV4aXRTdGF0dXMoY29kZSkpfWZ1bmN0aW9uIGV4aXRPbk1haW5UaHJlYWQocmV0dXJuQ29kZSl7aWYoRU5WSVJPTk1FTlRfSVNfUFRIUkVBRClyZXR1cm4gcHJveHlUb01haW5UaHJlYWQoMSwwLDAscmV0dXJuQ29kZSk7X2V4aXQocmV0dXJuQ29kZSl9dmFyIGV4aXRKUz0oc3RhdHVzLGltcGxpY2l0KT0+e0VYSVRTVEFUVVM9c3RhdHVzO2lmKEVOVklST05NRU5UX0lTX1BUSFJFQUQpe2V4aXRPbk1haW5UaHJlYWQoc3RhdHVzKTt0aHJvdyJ1bndpbmQifV9wcm9jX2V4aXQoc3RhdHVzKX07dmFyIF9leGl0PWV4aXRKUzt2YXIgUFRocmVhZD17dW51c2VkV29ya2VyczpbXSxydW5uaW5nV29ya2VyczpbXSx0bHNJbml0RnVuY3Rpb25zOltdLHB0aHJlYWRzOnt9LGluaXQoKXtpZighRU5WSVJPTk1FTlRfSVNfUFRIUkVBRCl7UFRocmVhZC5pbml0TWFpblRocmVhZCgpfX0saW5pdE1haW5UaHJlYWQoKXt2YXIgcHRocmVhZFBvb2xTaXplPTQ7d2hpbGUocHRocmVhZFBvb2xTaXplLS0pe1BUaHJlYWQuYWxsb2NhdGVVbnVzZWRXb3JrZXIoKX1hZGRPblByZVJ1bihhc3luYygpPT57dmFyIHB0aHJlYWRQb29sUmVhZHk9UFRocmVhZC5sb2FkV2FzbU1vZHVsZVRvQWxsV29ya2VycygpO2FkZFJ1bkRlcGVuZGVuY3koImxvYWRpbmctd29ya2VycyIpO2F3YWl0IHB0aHJlYWRQb29sUmVhZHk7cmVtb3ZlUnVuRGVwZW5kZW5jeSgibG9hZGluZy13b3JrZXJzIil9KX0sdGVybWluYXRlQWxsVGhyZWFkczooKT0+e2Zvcih2YXIgd29ya2VyIG9mIFBUaHJlYWQucnVubmluZ1dvcmtlcnMpe3Rlcm1pbmF0ZVdvcmtlcih3b3JrZXIpfWZvcih2YXIgd29ya2VyIG9mIFBUaHJlYWQudW51c2VkV29ya2Vycyl7dGVybWluYXRlV29ya2VyKHdvcmtlcil9UFRocmVhZC51bnVzZWRXb3JrZXJzPVtdO1BUaHJlYWQucnVubmluZ1dvcmtlcnM9W107UFRocmVhZC5wdGhyZWFkcz17fX0scmV0dXJuV29ya2VyVG9Qb29sOndvcmtlcj0+e3ZhciBwdGhyZWFkX3B0cj13b3JrZXIucHRocmVhZF9wdHI7ZGVsZXRlIFBUaHJlYWQucHRocmVhZHNbcHRocmVhZF9wdHJdO1BUaHJlYWQudW51c2VkV29ya2Vycy5wdXNoKHdvcmtlcik7UFRocmVhZC5ydW5uaW5nV29ya2Vycy5zcGxpY2UoUFRocmVhZC5ydW5uaW5nV29ya2Vycy5pbmRleE9mKHdvcmtlciksMSk7d29ya2VyLnB0aHJlYWRfcHRyPTA7X19lbXNjcmlwdGVuX3RocmVhZF9mcmVlX2RhdGEocHRocmVhZF9wdHIpfSx0aHJlYWRJbml0VExTKCl7UFRocmVhZC50bHNJbml0RnVuY3Rpb25zLmZvckVhY2goZj0+ZigpKX0sbG9hZFdhc21Nb2R1bGVUb1dvcmtlcjp3b3JrZXI9Pm5ldyBQcm9taXNlKG9uRmluaXNoZWRMb2FkaW5nPT57d29ya2VyLm9ubWVzc2FnZT1lPT57dmFyIGQ9ZVsiZGF0YSJdO3ZhciBjbWQ9ZC5jbWQ7aWYoZC50YXJnZXRUaHJlYWQmJmQudGFyZ2V0VGhyZWFkIT1fcHRocmVhZF9zZWxmKCkpe3ZhciB0YXJnZXRXb3JrZXI9UFRocmVhZC5wdGhyZWFkc1tkLnRhcmdldFRocmVhZF07aWYodGFyZ2V0V29ya2VyKXt0YXJnZXRXb3JrZXIucG9zdE1lc3NhZ2UoZCxkLnRyYW5zZmVyTGlzdCl9ZWxzZXtlcnIoYEludGVybmFsIGVycm9yISBXb3JrZXIgc2VudCBhIG1lc3NhZ2UgIiR7Y21kfSIgdG8gdGFyZ2V0IHB0aHJlYWQgJHtkLnRhcmdldFRocmVhZH0sIGJ1dCB0aGF0IHRocmVhZCBubyBsb25nZXIgZXhpc3RzIWApfXJldHVybn1pZihjbWQ9PT0iY2hlY2tNYWlsYm94Iil7Y2hlY2tNYWlsYm94KCl9ZWxzZSBpZihjbWQ9PT0ic3Bhd25UaHJlYWQiKXtzcGF3blRocmVhZChkKX1lbHNlIGlmKGNtZD09PSJjbGVhbnVwVGhyZWFkIil7Y2FsbFVzZXJDYWxsYmFjaygoKT0+Y2xlYW51cFRocmVhZChkLnRocmVhZCkpfWVsc2UgaWYoY21kPT09ImxvYWRlZCIpe3dvcmtlci5sb2FkZWQ9dHJ1ZTtpZihFTlZJUk9OTUVOVF9JU19OT0RFJiYhd29ya2VyLnB0aHJlYWRfcHRyKXt3b3JrZXIudW5yZWYoKX1vbkZpbmlzaGVkTG9hZGluZyh3b3JrZXIpfWVsc2UgaWYoZC50YXJnZXQ9PT0ic2V0aW1tZWRpYXRlIil7d29ya2VyLnBvc3RNZXNzYWdlKGQpfWVsc2UgaWYoY21kPT09InVuY2F1Z2h0RXhjZXB0aW9uIil7d29ya2VyLm9uZXJyb3IoZC5lcnJvcil9ZWxzZSBpZihjbWQ9PT0iY2FsbEhhbmRsZXIiKXtNb2R1bGVbZC5oYW5kbGVyXSguLi5kLmFyZ3MpfWVsc2UgaWYoY21kKXtlcnIoYHdvcmtlciBzZW50IGFuIHVua25vd24gY29tbWFuZCAke2NtZH1gKX19O3dvcmtlci5vbmVycm9yPWU9Pnt2YXIgbWVzc2FnZT0id29ya2VyIHNlbnQgYW4gZXJyb3IhIjtlcnIoYCR7bWVzc2FnZX0gJHtlLmZpbGVuYW1lfToke2UubGluZW5vfTogJHtlLm1lc3NhZ2V9YCk7dGhyb3cgZX07aWYoRU5WSVJPTk1FTlRfSVNfTk9ERSl7d29ya2VyLm9uKCJtZXNzYWdlIixkYXRhPT53b3JrZXIub25tZXNzYWdlKHtkYXRhfSkpO3dvcmtlci5vbigiZXJyb3IiLGU9Pndvcmtlci5vbmVycm9yKGUpKX12YXIgaGFuZGxlcnM9W107dmFyIGtub3duSGFuZGxlcnM9WyJvbkV4aXQiLCJvbkFib3J0IiwicHJpbnQiLCJwcmludEVyciJdO2Zvcih2YXIgaGFuZGxlciBvZiBrbm93bkhhbmRsZXJzKXtpZihNb2R1bGUucHJvcGVydHlJc0VudW1lcmFibGUoaGFuZGxlcikpe2hhbmRsZXJzLnB1c2goaGFuZGxlcil9fXdvcmtlci5wb3N0TWVzc2FnZSh7Y21kOiJsb2FkIixoYW5kbGVycyx3YXNtTWVtb3J5LHdhc21Nb2R1bGV9KX0pLGFzeW5jIGxvYWRXYXNtTW9kdWxlVG9BbGxXb3JrZXJzKCl7aWYoRU5WSVJPTk1FTlRfSVNfUFRIUkVBRCl7cmV0dXJufWxldCBwdGhyZWFkUG9vbFJlYWR5PVByb21pc2UuYWxsKFBUaHJlYWQudW51c2VkV29ya2Vycy5tYXAoUFRocmVhZC5sb2FkV2FzbU1vZHVsZVRvV29ya2VyKSk7cmV0dXJuIHB0aHJlYWRQb29sUmVhZHl9LGFsbG9jYXRlVW51c2VkV29ya2VyKCl7dmFyIHdvcmtlcjtpZihNb2R1bGVbIm1haW5TY3JpcHRVcmxPckJsb2IiXSl7dmFyIHB0aHJlYWRNYWluSnM9TW9kdWxlWyJtYWluU2NyaXB0VXJsT3JCbG9iIl07aWYodHlwZW9mIHB0aHJlYWRNYWluSnMhPSJzdHJpbmciKXtwdGhyZWFkTWFpbkpzPVVSTC5jcmVhdGVPYmplY3RVUkwocHRocmVhZE1haW5Kcyl9d29ya2VyPW5ldyBXb3JrZXIoRU5WSVJPTk1FTlRfSVNfTk9ERSYmdHlwZW9mIHB0aHJlYWRNYWluSnM9PSJzdHJpbmciJiZwdGhyZWFkTWFpbkpzLnN0YXJ0c1dpdGgoImRhdGE6Iik/bmV3IFVSTChwdGhyZWFkTWFpbkpzKTpwdGhyZWFkTWFpbkpzLHt0eXBlOiJtb2R1bGUiLHdvcmtlckRhdGE6ImVtLXB0aHJlYWQiLG5hbWU6ImVtLXB0aHJlYWQifSl9ZWxzZSB3b3JrZXI9bmV3IFdvcmtlcihuZXcgVVJMKCJza2FpbmV0X2tlcm5lbHNfbXQubWpzIixpbXBvcnQubWV0YS51cmwpLHt0eXBlOiJtb2R1bGUiLHdvcmtlckRhdGE6ImVtLXB0aHJlYWQiLG5hbWU6ImVtLXB0aHJlYWQifSk7UFRocmVhZC51bnVzZWRXb3JrZXJzLnB1c2god29ya2VyKX0sZ2V0TmV3V29ya2VyKCl7aWYoUFRocmVhZC51bnVzZWRXb3JrZXJzLmxlbmd0aD09MCl7UFRocmVhZC5hbGxvY2F0ZVVudXNlZFdvcmtlcigpO1BUaHJlYWQubG9hZFdhc21Nb2R1bGVUb1dvcmtlcihQVGhyZWFkLnVudXNlZFdvcmtlcnNbMF0pfXJldHVybiBQVGhyZWFkLnVudXNlZFdvcmtlcnMucG9wKCl9fTt2YXIgb25Qb3N0UnVucz1bXTt2YXIgYWRkT25Qb3N0UnVuPWNiPT5vblBvc3RSdW5zLnB1c2goY2IpO2Z1bmN0aW9uIGVzdGFibGlzaFN0YWNrU3BhY2UocHRocmVhZF9wdHIpe3ZhciBzdGFja0hpZ2g9KGdyb3dNZW1WaWV3cygpLEhFQVBVMzIpW3B0aHJlYWRfcHRyKzUyPj4+Mj4+PjBdO3ZhciBzdGFja1NpemU9KGdyb3dNZW1WaWV3cygpLEhFQVBVMzIpW3B0aHJlYWRfcHRyKzU2Pj4+Mj4+PjBdO3ZhciBzdGFja0xvdz1zdGFja0hpZ2gtc3RhY2tTaXplO19lbXNjcmlwdGVuX3N0YWNrX3NldF9saW1pdHMoc3RhY2tIaWdoLHN0YWNrTG93KTtzdGFja1Jlc3RvcmUoc3RhY2tIaWdoKX12YXIgd2FzbVRhYmxlTWlycm9yPVtdO3ZhciB3YXNtVGFibGU7dmFyIGdldFdhc21UYWJsZUVudHJ5PWZ1bmNQdHI9Pnt2YXIgZnVuYz13YXNtVGFibGVNaXJyb3JbZnVuY1B0cl07aWYoIWZ1bmMpe3dhc21UYWJsZU1pcnJvcltmdW5jUHRyXT1mdW5jPXdhc21UYWJsZS5nZXQoZnVuY1B0cil9cmV0dXJuIGZ1bmN9O3ZhciBpbnZva2VFbnRyeVBvaW50PShwdHIsYXJnKT0+e3J1bnRpbWVLZWVwYWxpdmVDb3VudGVyPTA7bm9FeGl0UnVudGltZT0wO3ZhciByZXN1bHQ9Z2V0V2FzbVRhYmxlRW50cnkocHRyKShhcmcpO2Z1bmN0aW9uIGZpbmlzaChyZXN1bHQpe2lmKGtlZXBSdW50aW1lQWxpdmUoKSl7RVhJVFNUQVRVUz1yZXN1bHQ7cmV0dXJufV9fZW1zY3JpcHRlbl90aHJlYWRfZXhpdChyZXN1bHQpfWZpbmlzaChyZXN1bHQpfTt2YXIgbm9FeGl0UnVudGltZT10cnVlO3ZhciByZWdpc3RlclRMU0luaXQ9dGxzSW5pdEZ1bmM9PlBUaHJlYWQudGxzSW5pdEZ1bmN0aW9ucy5wdXNoKHRsc0luaXRGdW5jKTtmdW5jdGlvbiBwdGhyZWFkQ3JlYXRlUHJveGllZChwdGhyZWFkX3B0cixhdHRyLHN0YXJ0Um91dGluZSxhcmcpe2lmKEVOVklST05NRU5UX0lTX1BUSFJFQUQpcmV0dXJuIHByb3h5VG9NYWluVGhyZWFkKDIsMCwxLHB0aHJlYWRfcHRyLGF0dHIsc3RhcnRSb3V0aW5lLGFyZyk7cmV0dXJuIF9fX3B0aHJlYWRfY3JlYXRlX2pzKHB0aHJlYWRfcHRyLGF0dHIsc3RhcnRSb3V0aW5lLGFyZyl9dmFyIF9lbXNjcmlwdGVuX2hhc190aHJlYWRpbmdfc3VwcG9ydD0oKT0+dHlwZW9mIFNoYXJlZEFycmF5QnVmZmVyIT0idW5kZWZpbmVkIjtmdW5jdGlvbiBfX19wdGhyZWFkX2NyZWF0ZV9qcyhwdGhyZWFkX3B0cixhdHRyLHN0YXJ0Um91dGluZSxhcmcpe3B0aHJlYWRfcHRyPj4+PTA7YXR0cj4+Pj0wO3N0YXJ0Um91dGluZT4+Pj0wO2FyZz4+Pj0wO2lmKCFfZW1zY3JpcHRlbl9oYXNfdGhyZWFkaW5nX3N1cHBvcnQoKSl7cmV0dXJuIDZ9dmFyIHRyYW5zZmVyTGlzdD1bXTt2YXIgZXJyb3I9MDtpZihFTlZJUk9OTUVOVF9JU19QVEhSRUFEJiYodHJhbnNmZXJMaXN0Lmxlbmd0aD09PTB8fGVycm9yKSl7cmV0dXJuIHB0aHJlYWRDcmVhdGVQcm94aWVkKHB0aHJlYWRfcHRyLGF0dHIsc3RhcnRSb3V0aW5lLGFyZyl9aWYoZXJyb3IpcmV0dXJuIGVycm9yO3ZhciB0aHJlYWRQYXJhbXM9e3N0YXJ0Um91dGluZSxwdGhyZWFkX3B0cixhcmcsdHJhbnNmZXJMaXN0fTtpZihFTlZJUk9OTUVOVF9JU19QVEhSRUFEKXt0aHJlYWRQYXJhbXMuY21kPSJzcGF3blRocmVhZCI7cG9zdE1lc3NhZ2UodGhyZWFkUGFyYW1zLHRyYW5zZmVyTGlzdCk7cmV0dXJuIDB9cmV0dXJuIHNwYXduVGhyZWFkKHRocmVhZFBhcmFtcyl9ZnVuY3Rpb24gX19lbXNjcmlwdGVuX2luaXRfbWFpbl90aHJlYWRfanModGIpe3RiPj4+PTA7X19lbXNjcmlwdGVuX3RocmVhZF9pbml0KHRiLCFFTlZJUk9OTUVOVF9JU19XT1JLRVIsMSwhRU5WSVJPTk1FTlRfSVNfV0VCLDY1NTM2LGZhbHNlKTtQVGhyZWFkLnRocmVhZEluaXRUTFMoKX12YXIgaGFuZGxlRXhjZXB0aW9uPWU9PntpZihlIGluc3RhbmNlb2YgRXhpdFN0YXR1c3x8ZT09InVud2luZCIpe3JldHVybiBFWElUU1RBVFVTfXF1aXRfKDEsZSl9O3ZhciBtYXliZUV4aXQ9KCk9PntpZigha2VlcFJ1bnRpbWVBbGl2ZSgpKXt0cnl7aWYoRU5WSVJPTk1FTlRfSVNfUFRIUkVBRCl7aWYoX3B0aHJlYWRfc2VsZigpKV9fZW1zY3JpcHRlbl90aHJlYWRfZXhpdChFWElUU1RBVFVTKTtyZXR1cm59X2V4aXQoRVhJVFNUQVRVUyl9Y2F0Y2goZSl7aGFuZGxlRXhjZXB0aW9uKGUpfX19O3ZhciBjYWxsVXNlckNhbGxiYWNrPWZ1bmM9PntpZihBQk9SVCl7cmV0dXJufXRyeXtmdW5jKCk7bWF5YmVFeGl0KCl9Y2F0Y2goZSl7aGFuZGxlRXhjZXB0aW9uKGUpfX07ZnVuY3Rpb24gX19lbXNjcmlwdGVuX3RocmVhZF9tYWlsYm94X2F3YWl0KHB0aHJlYWRfcHRyKXtwdGhyZWFkX3B0cj4+Pj0wO2lmKHR5cGVvZiBBdG9taWNzLndhaXRBc3luYz09PSJmdW5jdGlvbiIpe3ZhciB3YWl0PUF0b21pY3Mud2FpdEFzeW5jKChncm93TWVtVmlld3MoKSxIRUFQMzIpLHB0aHJlYWRfcHRyPj4+MixwdGhyZWFkX3B0cik7d2FpdC52YWx1ZS50aGVuKGNoZWNrTWFpbGJveCk7dmFyIHdhaXRpbmdBc3luYz1wdGhyZWFkX3B0cisxMjg7QXRvbWljcy5zdG9yZSgoZ3Jvd01lbVZpZXdzKCksSEVBUDMyKSx3YWl0aW5nQXN5bmM+Pj4yLDEpfX12YXIgY2hlY2tNYWlsYm94PSgpPT5jYWxsVXNlckNhbGxiYWNrKCgpPT57dmFyIHB0aHJlYWRfcHRyPV9wdGhyZWFkX3NlbGYoKTtpZihwdGhyZWFkX3B0cil7X19lbXNjcmlwdGVuX3RocmVhZF9tYWlsYm94X2F3YWl0KHB0aHJlYWRfcHRyKTtfX2Vtc2NyaXB0ZW5fY2hlY2tfbWFpbGJveCgpfX0pO2Z1bmN0aW9uIF9fZW1zY3JpcHRlbl9ub3RpZnlfbWFpbGJveF9wb3N0bWVzc2FnZSh0YXJnZXRUaHJlYWQsY3VyclRocmVhZElkKXt0YXJnZXRUaHJlYWQ+Pj49MDtjdXJyVGhyZWFkSWQ+Pj49MDtpZih0YXJnZXRUaHJlYWQ9PWN1cnJUaHJlYWRJZCl7c2V0VGltZW91dChjaGVja01haWxib3gpfWVsc2UgaWYoRU5WSVJPTk1FTlRfSVNfUFRIUkVBRCl7cG9zdE1lc3NhZ2Uoe3RhcmdldFRocmVhZCxjbWQ6ImNoZWNrTWFpbGJveCJ9KX1lbHNle3ZhciB3b3JrZXI9UFRocmVhZC5wdGhyZWFkc1t0YXJnZXRUaHJlYWRdO2lmKCF3b3JrZXIpe3JldHVybn13b3JrZXIucG9zdE1lc3NhZ2Uoe2NtZDoiY2hlY2tNYWlsYm94In0pfX12YXIgcHJveGllZEpTQ2FsbEFyZ3M9W107ZnVuY3Rpb24gX19lbXNjcmlwdGVuX3JlY2VpdmVfb25fbWFpbl90aHJlYWRfanMoZnVuY0luZGV4LGVtQXNtQWRkcixjYWxsaW5nVGhyZWFkLG51bUNhbGxBcmdzLGFyZ3Mpe2VtQXNtQWRkcj4+Pj0wO2NhbGxpbmdUaHJlYWQ+Pj49MDthcmdzPj4+PTA7bnVtQ2FsbEFyZ3MvPTI7cHJveGllZEpTQ2FsbEFyZ3MubGVuZ3RoPW51bUNhbGxBcmdzO3ZhciBiPWFyZ3M+Pj4zO2Zvcih2YXIgaT0wO2k8bnVtQ2FsbEFyZ3M7aSsrKXtpZigoZ3Jvd01lbVZpZXdzKCksSEVBUDY0KVtiKzIqaT4+PjBdKXtwcm94aWVkSlNDYWxsQXJnc1tpXT0oZ3Jvd01lbVZpZXdzKCksSEVBUDY0KVtiKzIqaSsxPj4+MF19ZWxzZXtwcm94aWVkSlNDYWxsQXJnc1tpXT0oZ3Jvd01lbVZpZXdzKCksSEVBUEY2NClbYisyKmkrMT4+PjBdfX12YXIgZnVuYz1wcm94aWVkRnVuY3Rpb25UYWJsZVtmdW5jSW5kZXhdO1BUaHJlYWQuY3VycmVudFByb3hpZWRPcGVyYXRpb25DYWxsZXJUaHJlYWQ9Y2FsbGluZ1RocmVhZDt2YXIgcnRuPWZ1bmMoLi4ucHJveGllZEpTQ2FsbEFyZ3MpO1BUaHJlYWQuY3VycmVudFByb3hpZWRPcGVyYXRpb25DYWxsZXJUaHJlYWQ9MDtyZXR1cm4gcnRufWZ1bmN0aW9uIF9fZW1zY3JpcHRlbl90aHJlYWRfY2xlYW51cCh0aHJlYWQpe3RocmVhZD4+Pj0wO2lmKCFFTlZJUk9OTUVOVF9JU19QVEhSRUFEKWNsZWFudXBUaHJlYWQodGhyZWFkKTtlbHNlIHBvc3RNZXNzYWdlKHtjbWQ6ImNsZWFudXBUaHJlYWQiLHRocmVhZH0pfWZ1bmN0aW9uIF9fZW1zY3JpcHRlbl90aHJlYWRfc2V0X3N0cm9uZ3JlZih0aHJlYWQpe3RocmVhZD4+Pj0wO2lmKEVOVklST05NRU5UX0lTX05PREUpe1BUaHJlYWQucHRocmVhZHNbdGhyZWFkXS5yZWYoKX19dmFyIF9lbXNjcmlwdGVuX2dldF9ub3c9KCk9PnBlcmZvcm1hbmNlLnRpbWVPcmlnaW4rcGVyZm9ybWFuY2Uubm93KCk7dmFyIF9lbXNjcmlwdGVuX2NoZWNrX2Jsb2NraW5nX2FsbG93ZWQ9KCk9Pnt9O3ZhciBydW50aW1lS2VlcGFsaXZlUHVzaD0oKT0+e3J1bnRpbWVLZWVwYWxpdmVDb3VudGVyKz0xfTt2YXIgX2Vtc2NyaXB0ZW5fZXhpdF93aXRoX2xpdmVfcnVudGltZT0oKT0+e3J1bnRpbWVLZWVwYWxpdmVQdXNoKCk7dGhyb3cidW53aW5kIn07dmFyIGdldEhlYXBNYXg9KCk9PjQyOTQ5MDE3NjA7dmFyIGFsaWduTWVtb3J5PShzaXplLGFsaWdubWVudCk9Pk1hdGguY2VpbChzaXplL2FsaWdubWVudCkqYWxpZ25tZW50O3ZhciBncm93TWVtb3J5PXNpemU9Pnt2YXIgb2xkSGVhcFNpemU9d2FzbU1lbW9yeS5idWZmZXIuYnl0ZUxlbmd0aDt2YXIgcGFnZXM9KHNpemUtb2xkSGVhcFNpemUrNjU1MzUpLzY1NTM2fDA7dHJ5e3dhc21NZW1vcnkuZ3JvdyhwYWdlcyk7dXBkYXRlTWVtb3J5Vmlld3MoKTtyZXR1cm4gMX1jYXRjaChlKXt9fTtmdW5jdGlvbiBfZW1zY3JpcHRlbl9yZXNpemVfaGVhcChyZXF1ZXN0ZWRTaXplKXtyZXF1ZXN0ZWRTaXplPj4+PTA7dmFyIG9sZFNpemU9KGdyb3dNZW1WaWV3cygpLEhFQVBVOCkubGVuZ3RoO2lmKHJlcXVlc3RlZFNpemU8PW9sZFNpemUpe3JldHVybiBmYWxzZX12YXIgbWF4SGVhcFNpemU9Z2V0SGVhcE1heCgpO2lmKHJlcXVlc3RlZFNpemU+bWF4SGVhcFNpemUpe3JldHVybiBmYWxzZX1mb3IodmFyIGN1dERvd249MTtjdXREb3duPD00O2N1dERvd24qPTIpe3ZhciBvdmVyR3Jvd25IZWFwU2l6ZT1vbGRTaXplKigxKy4yL2N1dERvd24pO292ZXJHcm93bkhlYXBTaXplPU1hdGgubWluKG92ZXJHcm93bkhlYXBTaXplLHJlcXVlc3RlZFNpemUrMTAwNjYzMjk2KTt2YXIgbmV3U2l6ZT1NYXRoLm1pbihtYXhIZWFwU2l6ZSxhbGlnbk1lbW9yeShNYXRoLm1heChyZXF1ZXN0ZWRTaXplLG92ZXJHcm93bkhlYXBTaXplKSw2NTUzNikpO3ZhciByZXBsYWNlbWVudD1ncm93TWVtb3J5KG5ld1NpemUpO2lmKHJlcGxhY2VtZW50KXtyZXR1cm4gdHJ1ZX19cmV0dXJuIGZhbHNlfXZhciBwcmludENoYXJCdWZmZXJzPVtudWxsLFtdLFtdXTt2YXIgVVRGOERlY29kZXI9dHlwZW9mIFRleHREZWNvZGVyIT0idW5kZWZpbmVkIj9uZXcgVGV4dERlY29kZXI6dW5kZWZpbmVkO3ZhciBmaW5kU3RyaW5nRW5kPShoZWFwT3JBcnJheSxpZHgsbWF4Qnl0ZXNUb1JlYWQsaWdub3JlTnVsKT0+e3ZhciBtYXhJZHg9aWR4K21heEJ5dGVzVG9SZWFkO2lmKGlnbm9yZU51bClyZXR1cm4gbWF4SWR4O3doaWxlKGhlYXBPckFycmF5W2lkeF0mJiEoaWR4Pj1tYXhJZHgpKSsraWR4O3JldHVybiBpZHh9O3ZhciBVVEY4QXJyYXlUb1N0cmluZz0oaGVhcE9yQXJyYXksaWR4PTAsbWF4Qnl0ZXNUb1JlYWQsaWdub3JlTnVsKT0+e2lkeD4+Pj0wO3ZhciBlbmRQdHI9ZmluZFN0cmluZ0VuZChoZWFwT3JBcnJheSxpZHgsbWF4Qnl0ZXNUb1JlYWQsaWdub3JlTnVsKTtpZihlbmRQdHItaWR4PjE2JiZoZWFwT3JBcnJheS5idWZmZXImJlVURjhEZWNvZGVyKXtyZXR1cm4gVVRGOERlY29kZXIuZGVjb2RlKGhlYXBPckFycmF5LmJ1ZmZlciBpbnN0YW5jZW9mIEFycmF5QnVmZmVyP2hlYXBPckFycmF5LnN1YmFycmF5KGlkeCxlbmRQdHIpOmhlYXBPckFycmF5LnNsaWNlKGlkeCxlbmRQdHIpKX12YXIgc3RyPSIiO3doaWxlKGlkeDxlbmRQdHIpe3ZhciB1MD1oZWFwT3JBcnJheVtpZHgrK107aWYoISh1MCYxMjgpKXtzdHIrPVN0cmluZy5mcm9tQ2hhckNvZGUodTApO2NvbnRpbnVlfXZhciB1MT1oZWFwT3JBcnJheVtpZHgrK10mNjM7aWYoKHUwJjIyNCk9PTE5Mil7c3RyKz1TdHJpbmcuZnJvbUNoYXJDb2RlKCh1MCYzMSk8PDZ8dTEpO2NvbnRpbnVlfXZhciB1Mj1oZWFwT3JBcnJheVtpZHgrK10mNjM7aWYoKHUwJjI0MCk9PTIyNCl7dTA9KHUwJjE1KTw8MTJ8dTE8PDZ8dTJ9ZWxzZXt1MD0odTAmNyk8PDE4fHUxPDwxMnx1Mjw8NnxoZWFwT3JBcnJheVtpZHgrK10mNjN9aWYodTA8NjU1MzYpe3N0cis9U3RyaW5nLmZyb21DaGFyQ29kZSh1MCl9ZWxzZXt2YXIgY2g9dTAtNjU1MzY7c3RyKz1TdHJpbmcuZnJvbUNoYXJDb2RlKDU1Mjk2fGNoPj4xMCw1NjMyMHxjaCYxMDIzKX19cmV0dXJuIHN0cn07dmFyIHByaW50Q2hhcj0oc3RyZWFtLGN1cnIpPT57dmFyIGJ1ZmZlcj1wcmludENoYXJCdWZmZXJzW3N0cmVhbV07aWYoY3Vycj09PTB8fGN1cnI9PT0xMCl7KHN0cmVhbT09PTE/b3V0OmVycikoVVRGOEFycmF5VG9TdHJpbmcoYnVmZmVyKSk7YnVmZmVyLmxlbmd0aD0wfWVsc2V7YnVmZmVyLnB1c2goY3Vycil9fTtmdW5jdGlvbiBfZmRfd3JpdGUoZmQsaW92LGlvdmNudCxwbnVtKXtpZihFTlZJUk9OTUVOVF9JU19QVEhSRUFEKXJldHVybiBwcm94eVRvTWFpblRocmVhZCgzLDAsMSxmZCxpb3YsaW92Y250LHBudW0pO2lvdj4+Pj0wO2lvdmNudD4+Pj0wO3BudW0+Pj49MDt2YXIgbnVtPTA7Zm9yKHZhciBpPTA7aTxpb3ZjbnQ7aSsrKXt2YXIgcHRyPShncm93TWVtVmlld3MoKSxIRUFQVTMyKVtpb3Y+Pj4yPj4+MF07dmFyIGxlbj0oZ3Jvd01lbVZpZXdzKCksSEVBUFUzMilbaW92KzQ+Pj4yPj4+MF07aW92Kz04O2Zvcih2YXIgaj0wO2o8bGVuO2orKyl7cHJpbnRDaGFyKGZkLChncm93TWVtVmlld3MoKSxIRUFQVTgpW3B0citqPj4+MF0pfW51bSs9bGVufShncm93TWVtVmlld3MoKSxIRUFQVTMyKVtwbnVtPj4+Mj4+PjBdPW51bTtyZXR1cm4gMH1QVGhyZWFkLmluaXQoKTt7aW5pdE1lbW9yeSgpO2lmKE1vZHVsZVsibm9FeGl0UnVudGltZSJdKW5vRXhpdFJ1bnRpbWU9TW9kdWxlWyJub0V4aXRSdW50aW1lIl07aWYoTW9kdWxlWyJwcmludCJdKW91dD1Nb2R1bGVbInByaW50Il07aWYoTW9kdWxlWyJwcmludEVyciJdKWVycj1Nb2R1bGVbInByaW50RXJyIl07aWYoTW9kdWxlWyJ3YXNtQmluYXJ5Il0pd2FzbUJpbmFyeT1Nb2R1bGVbIndhc21CaW5hcnkiXTtpZihNb2R1bGVbImFyZ3VtZW50cyJdKWFyZ3VtZW50c189TW9kdWxlWyJhcmd1bWVudHMiXTtpZihNb2R1bGVbInRoaXNQcm9ncmFtIl0pdGhpc1Byb2dyYW09TW9kdWxlWyJ0aGlzUHJvZ3JhbSJdO2lmKE1vZHVsZVsicHJlSW5pdCJdKXtpZih0eXBlb2YgTW9kdWxlWyJwcmVJbml0Il09PSJmdW5jdGlvbiIpTW9kdWxlWyJwcmVJbml0Il09W01vZHVsZVsicHJlSW5pdCJdXTt3aGlsZShNb2R1bGVbInByZUluaXQiXS5sZW5ndGg+MCl7TW9kdWxlWyJwcmVJbml0Il0uc2hpZnQoKSgpfX19TW9kdWxlWyJ3YXNtTWVtb3J5Il09d2FzbU1lbW9yeTt2YXIgcHJveGllZEZ1bmN0aW9uVGFibGU9W19wcm9jX2V4aXQsZXhpdE9uTWFpblRocmVhZCxwdGhyZWFkQ3JlYXRlUHJveGllZCxfZmRfd3JpdGVdO3ZhciBfc2thaW5ldF9mcDMyX21hdG11bCxfc2thaW5ldF9mcDMyX21hdG11bF93dCxfc2thaW5ldF9iZjE2X21hdG11bCxfc2thaW5ldF9mcDE2X21hdG11bCxfc2thaW5ldF9xNF8wX21hdG11bCxfc2thaW5ldF9xNF8wX21hdG11bF9ybSxfc2thaW5ldF9xNV8wX21hdG11bCxfbWFsbG9jLF9mcmVlLF9za2FpbmV0X3E1XzBfbWF0bXVsX3JtLF9za2FpbmV0X3E1XzFfbWF0bXVsLF9za2FpbmV0X3E1XzFfbWF0bXVsX3JtLF9za2FpbmV0X3E4XzBfbWF0bXVsLF9za2FpbmV0X3E4XzBfbWF0bXVsX3JtLF9za2FpbmV0X3E0a19tYXRtdWwsX3NrYWluZXRfcTRrX21hdG11bF9ybSxfc2thaW5ldF9xNWtfbWF0bXVsLF9za2FpbmV0X3E1a19tYXRtdWxfcm0sX3NrYWluZXRfcTZrX21hdG11bCxfc2thaW5ldF9xNmtfbWF0bXVsX3JtLF9fZW1zY3JpcHRlbl90bHNfaW5pdCxfcHRocmVhZF9zZWxmLF9fZW1zY3JpcHRlbl90aHJlYWRfaW5pdCxfX2Vtc2NyaXB0ZW5fdGhyZWFkX2NyYXNoZWQsX19lbXNjcmlwdGVuX3J1bl9qc19vbl9tYWluX3RocmVhZCxfX2Vtc2NyaXB0ZW5fdGhyZWFkX2ZyZWVfZGF0YSxfX2Vtc2NyaXB0ZW5fdGhyZWFkX2V4aXQsX19lbXNjcmlwdGVuX2NoZWNrX21haWxib3gsX2Vtc2NyaXB0ZW5fc3RhY2tfc2V0X2xpbWl0cyxfX2Vtc2NyaXB0ZW5fc3RhY2tfcmVzdG9yZSxfX2Vtc2NyaXB0ZW5fc3RhY2tfYWxsb2MsX2Vtc2NyaXB0ZW5fc3RhY2tfZ2V0X2N1cnJlbnQ7ZnVuY3Rpb24gYXNzaWduV2FzbUV4cG9ydHMod2FzbUV4cG9ydHMpe01vZHVsZVsiX3NrYWluZXRfZnAzMl9tYXRtdWwiXT1fc2thaW5ldF9mcDMyX21hdG11bD13YXNtRXhwb3J0c1sicCJdO01vZHVsZVsiX3NrYWluZXRfZnAzMl9tYXRtdWxfd3QiXT1fc2thaW5ldF9mcDMyX21hdG11bF93dD13YXNtRXhwb3J0c1siciJdO01vZHVsZVsiX3NrYWluZXRfYmYxNl9tYXRtdWwiXT1fc2thaW5ldF9iZjE2X21hdG11bD13YXNtRXhwb3J0c1sicyJdO01vZHVsZVsiX3NrYWluZXRfZnAxNl9tYXRtdWwiXT1fc2thaW5ldF9mcDE2X21hdG11bD13YXNtRXhwb3J0c1sidCJdO01vZHVsZVsiX3NrYWluZXRfcTRfMF9tYXRtdWwiXT1fc2thaW5ldF9xNF8wX21hdG11bD13YXNtRXhwb3J0c1sidSJdO01vZHVsZVsiX3NrYWluZXRfcTRfMF9tYXRtdWxfcm0iXT1fc2thaW5ldF9xNF8wX21hdG11bF9ybT13YXNtRXhwb3J0c1sidiJdO01vZHVsZVsiX3NrYWluZXRfcTVfMF9tYXRtdWwiXT1fc2thaW5ldF9xNV8wX21hdG11bD13YXNtRXhwb3J0c1sidyJdO01vZHVsZVsiX21hbGxvYyJdPV9tYWxsb2M9d2FzbUV4cG9ydHNbIngiXTtNb2R1bGVbIl9mcmVlIl09X2ZyZWU9d2FzbUV4cG9ydHNbInkiXTtNb2R1bGVbIl9za2FpbmV0X3E1XzBfbWF0bXVsX3JtIl09X3NrYWluZXRfcTVfMF9tYXRtdWxfcm09d2FzbUV4cG9ydHNbInoiXTtNb2R1bGVbIl9za2FpbmV0X3E1XzFfbWF0bXVsIl09X3NrYWluZXRfcTVfMV9tYXRtdWw9d2FzbUV4cG9ydHNbIkEiXTtNb2R1bGVbIl9za2FpbmV0X3E1XzFfbWF0bXVsX3JtIl09X3NrYWluZXRfcTVfMV9tYXRtdWxfcm09d2FzbUV4cG9ydHNbIkIiXTtNb2R1bGVbIl9za2FpbmV0X3E4XzBfbWF0bXVsIl09X3NrYWluZXRfcThfMF9tYXRtdWw9d2FzbUV4cG9ydHNbIkMiXTtNb2R1bGVbIl9za2FpbmV0X3E4XzBfbWF0bXVsX3JtIl09X3NrYWluZXRfcThfMF9tYXRtdWxfcm09d2FzbUV4cG9ydHNbIkQiXTtNb2R1bGVbIl9za2FpbmV0X3E0a19tYXRtdWwiXT1fc2thaW5ldF9xNGtfbWF0bXVsPXdhc21FeHBvcnRzWyJFIl07TW9kdWxlWyJfc2thaW5ldF9xNGtfbWF0bXVsX3JtIl09X3NrYWluZXRfcTRrX21hdG11bF9ybT13YXNtRXhwb3J0c1siRiJdO01vZHVsZVsiX3NrYWluZXRfcTVrX21hdG11bCJdPV9za2FpbmV0X3E1a19tYXRtdWw9d2FzbUV4cG9ydHNbIkciXTtNb2R1bGVbIl9za2FpbmV0X3E1a19tYXRtdWxfcm0iXT1fc2thaW5ldF9xNWtfbWF0bXVsX3JtPXdhc21FeHBvcnRzWyJIIl07TW9kdWxlWyJfc2thaW5ldF9xNmtfbWF0bXVsIl09X3NrYWluZXRfcTZrX21hdG11bD13YXNtRXhwb3J0c1siSSJdO01vZHVsZVsiX3NrYWluZXRfcTZrX21hdG11bF9ybSJdPV9za2FpbmV0X3E2a19tYXRtdWxfcm09d2FzbUV4cG9ydHNbIkoiXTtfX2Vtc2NyaXB0ZW5fdGxzX2luaXQ9d2FzbUV4cG9ydHNbIksiXTtfcHRocmVhZF9zZWxmPXdhc21FeHBvcnRzWyJMIl07X19lbXNjcmlwdGVuX3RocmVhZF9pbml0PXdhc21FeHBvcnRzWyJNIl07X19lbXNjcmlwdGVuX3RocmVhZF9jcmFzaGVkPXdhc21FeHBvcnRzWyJOIl07X19lbXNjcmlwdGVuX3J1bl9qc19vbl9tYWluX3RocmVhZD13YXNtRXhwb3J0c1siTyJdO19fZW1zY3JpcHRlbl90aHJlYWRfZnJlZV9kYXRhPXdhc21FeHBvcnRzWyJQIl07X19lbXNjcmlwdGVuX3RocmVhZF9leGl0PXdhc21FeHBvcnRzWyJRIl07X19lbXNjcmlwdGVuX2NoZWNrX21haWxib3g9d2FzbUV4cG9ydHNbIlIiXTtfZW1zY3JpcHRlbl9zdGFja19zZXRfbGltaXRzPXdhc21FeHBvcnRzWyJTIl07X19lbXNjcmlwdGVuX3N0YWNrX3Jlc3RvcmU9d2FzbUV4cG9ydHNbIlQiXTtfX2Vtc2NyaXB0ZW5fc3RhY2tfYWxsb2M9d2FzbUV4cG9ydHNbIlUiXTtfZW1zY3JpcHRlbl9zdGFja19nZXRfY3VycmVudD13YXNtRXhwb3J0c1siViJdfXZhciB3YXNtSW1wb3J0cztmdW5jdGlvbiBhc3NpZ25XYXNtSW1wb3J0cygpe3dhc21JbXBvcnRzPXtnOl9fX3B0aHJlYWRfY3JlYXRlX2pzLGs6X19lbXNjcmlwdGVuX2luaXRfbWFpbl90aHJlYWRfanMsbjpfX2Vtc2NyaXB0ZW5fbm90aWZ5X21haWxib3hfcG9zdG1lc3NhZ2UsbDpfX2Vtc2NyaXB0ZW5fcmVjZWl2ZV9vbl9tYWluX3RocmVhZF9qcyxjOl9fZW1zY3JpcHRlbl90aHJlYWRfY2xlYW51cCxqOl9fZW1zY3JpcHRlbl90aHJlYWRfbWFpbGJveF9hd2FpdCxpOl9fZW1zY3JpcHRlbl90aHJlYWRfc2V0X3N0cm9uZ3JlZixlOl9lbXNjcmlwdGVuX2NoZWNrX2Jsb2NraW5nX2FsbG93ZWQsaDpfZW1zY3JpcHRlbl9leGl0X3dpdGhfbGl2ZV9ydW50aW1lLGI6X2Vtc2NyaXB0ZW5fZ2V0X25vdyxtOl9lbXNjcmlwdGVuX3Jlc2l6ZV9oZWFwLGY6X2V4aXQsZDpfZmRfd3JpdGUsYTp3YXNtTWVtb3J5fX1mdW5jdGlvbiBhcHBseVNpZ25hdHVyZUNvbnZlcnNpb25zKHdhc21FeHBvcnRzKXt3YXNtRXhwb3J0cz1PYmplY3QuYXNzaWduKHt9LHdhc21FeHBvcnRzKTt2YXIgbWFrZVdyYXBwZXJfcHA9Zj0+YTA9PmYoYTApPj4+MDt2YXIgbWFrZVdyYXBwZXJfcD1mPT4oKT0+ZigpPj4+MDt3YXNtRXhwb3J0c1sieCJdPW1ha2VXcmFwcGVyX3BwKHdhc21FeHBvcnRzWyJ4Il0pO3dhc21FeHBvcnRzWyJMIl09bWFrZVdyYXBwZXJfcCh3YXNtRXhwb3J0c1siTCJdKTt3YXNtRXhwb3J0c1siVSJdPW1ha2VXcmFwcGVyX3BwKHdhc21FeHBvcnRzWyJVIl0pO3dhc21FeHBvcnRzWyJWIl09bWFrZVdyYXBwZXJfcCh3YXNtRXhwb3J0c1siViJdKTtyZXR1cm4gd2FzbUV4cG9ydHN9ZnVuY3Rpb24gcnVuKCl7aWYocnVuRGVwZW5kZW5jaWVzPjApe2RlcGVuZGVuY2llc0Z1bGZpbGxlZD1ydW47cmV0dXJufWlmKEVOVklST05NRU5UX0lTX1BUSFJFQUQpe3JlYWR5UHJvbWlzZVJlc29sdmU/LihNb2R1bGUpO2luaXRSdW50aW1lKCk7cmV0dXJufXByZVJ1bigpO2lmKHJ1bkRlcGVuZGVuY2llcz4wKXtkZXBlbmRlbmNpZXNGdWxmaWxsZWQ9cnVuO3JldHVybn1mdW5jdGlvbiBkb1J1bigpe01vZHVsZVsiY2FsbGVkUnVuIl09dHJ1ZTtpZihBQk9SVClyZXR1cm47aW5pdFJ1bnRpbWUoKTtyZWFkeVByb21pc2VSZXNvbHZlPy4oTW9kdWxlKTtNb2R1bGVbIm9uUnVudGltZUluaXRpYWxpemVkIl0/LigpO3Bvc3RSdW4oKX1pZihNb2R1bGVbInNldFN0YXR1cyJdKXtNb2R1bGVbInNldFN0YXR1cyJdKCJSdW5uaW5nLi4uIik7c2V0VGltZW91dCgoKT0+e3NldFRpbWVvdXQoKCk9Pk1vZHVsZVsic2V0U3RhdHVzIl0oIiIpLDEpO2RvUnVuKCl9LDEpfWVsc2V7ZG9SdW4oKX19dmFyIHdhc21FeHBvcnRzO2lmKCFFTlZJUk9OTUVOVF9JU19QVEhSRUFEKXt3YXNtRXhwb3J0cz1hd2FpdCAoY3JlYXRlV2FzbSgpKTtydW4oKX1pZihydW50aW1lSW5pdGlhbGl6ZWQpe21vZHVsZVJ0bj1Nb2R1bGV9ZWxzZXttb2R1bGVSdG49bmV3IFByb21pc2UoKHJlc29sdmUscmVqZWN0KT0+e3JlYWR5UHJvbWlzZVJlc29sdmU9cmVzb2x2ZTtyZWFkeVByb21pc2VSZWplY3Q9cmVqZWN0fSl9CjtyZXR1cm4gbW9kdWxlUnRufWV4cG9ydCBkZWZhdWx0IGNyZWF0ZVNrYWluZXRLZXJuZWxzO3ZhciBpc1B0aHJlYWQ9Z2xvYmFsVGhpcy5zZWxmPy5uYW1lPy5zdGFydHNXaXRoKCJlbS1wdGhyZWFkIik7dmFyIGlzTm9kZT10eXBlb2YgcHJvY2Vzcz09Im9iamVjdCImJnByb2Nlc3MudmVyc2lvbnM/Lm5vZGUmJnByb2Nlc3MudHlwZSE9InJlbmRlcmVyIjtpZihpc05vZGUpaXNQdGhyZWFkPShhd2FpdCBpbXBvcnQoIndvcmtlcl90aHJlYWRzIikpLndvcmtlckRhdGE9PT0iZW0tcHRocmVhZCI7aXNQdGhyZWFkJiZjcmVhdGVTa2FpbmV0S2VybmVscygpOwo=" - const val MT_WASM: String = "AGFzbQEAAAABmgEVYAF/AGADf39/AGAAAGAIf39/f39/f38AYAJ/fwBgAX8Bf2ACf38Bf2AAAX9gCX9/f39/f39/fwBgDH9/f39/f39/f39/fwBgBH9/f38Bf2AFf39/f38BfGADf39/AX9gAAF8YAN/f3wBf2AHf39/f39/fwBgBX9/f39/AGACf38BfWADf399AX1gBn9/f39/fwBgA39+fwF+AloOAWEBYgANAWEBYwAAAWEBZAAKAWEBZQACAWEBZgAAAWEBZwAKAWEBaAACAWEBaQAAAWEBagAAAWEBawAAAWEBbAALAWEBbQAFAWEBbgAEAWEBYQIDgAiAgAQDeHcABQQABQEBBQ4AAQUAAAYMDwgIBAAFAgIEBgAAAgcCBAACBAQCAAAABgYGAAQABRAREgQCAAAAAAEACQIAAAsEAAECAAAIAAAHBQIBAwEDAQMBAwEDAQEDAQMBAwEDAQMBCQMBAwEDAQMHBQATBBQMAQUCBwkCAgQFAXABJSUGNQp/AUGQogQLfwFBAAt/AUEAC38BQQALfwFBAAt/AUEAC38BQQALfwFBAAt/AUEAC38BQQALB4wBIgFvAIMBAXAAbgFxAQABcgBSAXMARwF0AIEBAXUAdQF2AHMBdwBxAXgAEQF5ABABegBvAUEAbAFCAGoBQwBoAUQAZgFFAGQBRgBhAUcAXwFIAF0BSQBbAUoAWQFLAFUBTACAAQFNAHkBTgBIAU8ASwFQAEYBUQBEAVIAfwFTAHoBVAB4AVUAdwFWAHYIAoIBCSoBAEEBCyRjTkV9dHJwbWtpZ2ViYF5cWlhXVlRTFlEyUE9NSUxKQ0J+fHsMAQIKmtICd/MBAQd/IAAoAgghBAJAAkAgACgCACICQQ9xRQRAIABBBGoiAUEA/kECACEADAELIwMiAygCGCAAKAIEIgVB/////wNxRw0BAkAgAkEDcUEBRw0AIAAoAhQiAUUNACAAIAFBAWs2AhQPCyACQYABcSIGBEAgAyAAQRBqNgJUQQBBAf4eApQNGgsgAEEEaiEBIAAoAgwiByAAKAIQIgA2AgAgA0HMAGogAEcEQCAAQQRrIAc2AgALIAEgBUEBdCACQR10cUEfdUH/////B3H+QQIAIQAgBkUNACADQQA2AlQQLgsgBEUgAEEATnENACABQQEQDwsLqAIBBn8CQCAALQAAQQ9xDQAgAEEEakEAQQr+SAIADQBBAA8LAn8CQCAAKAIAIgJBD3FFBEAgAEEEakEAQQr+SAIARQ0BIAAoAgAhAgsgABAYIgFBCkcNACAAQQhqIQQgAEEEaiEDQeQAIQEDQAJAIAFFDQAgAygCAEUNACABQQFrIQEgBCgCAEUNAQsLIAAQGCIBQQpHDQAgAkEEcUUhBSACQQNxQQJHIQIDQAJAIAMoAgAiAUH/////A3EiBiABQQBHIAVxckUNAAJAIAINACAGIwMoAhhHDQBBEAwECyAEQQH+HgIAGiADIAEgAUGAgICAeHIiAf5IAgAaIAMgARAmIQEgBEEB/iUCABogAUEbRg0AIAENAgsgABAYIgFBCkYNAAsLIAELC2QBAX8CQCAARQ0AIAFBAEgNACAAQQNxDQAgAUUEQA8LIABBACAAQQD+SAKYCyICIAAgAkYbIQICQCABQf////8HRg0AIAAgAkcNACABQQFGDQEgAUEBayEBCyAAIAH+AAIAGgsLiwwBCH8CQCAARQ0AQbQZLQAAQQJxBEBBuBkQDg0BCyAAQQhrIgMgAEEEaygCACICQXhxIgBqIQUCQAJAIAJBAXENACACQQJxRQ0BIAMgAygCACIEayIDQYgWKAIASQ0BIAAgBGohAAJAAkACQEGMFigCACADRwRAIAMoAgwhASAEQf8BTQRAIAEgAygCCCICRw0CQfgVQfgVKAIAQX4gBEEDdndxNgIADAULIAMoAhghByABIANHBEAgAygCCCICIAE2AgwgASACNgIIDAQLIAMoAhQiAgR/IANBFGoFIAMoAhAiAkUNAyADQRBqCyEEA0AgBCEGIAIiAUEUaiEEIAEoAhQiAg0AIAFBEGohBCABKAIQIgINAAsgBkEANgIADAMLIAUoAgQiAkEDcUEDRw0DQYAWIAA2AgAgBSACQX5xNgIEIAMgAEEBcjYCBCAFIAA2AgAMBAsgAiABNgIMIAEgAjYCCAwCC0EAIQELIAdFDQACQCADKAIcIgRBAnQiAigCqBggA0YEQCACQagYaiABNgIAIAENAUH8FUH8FSgCAEF+IAR3cTYCAAwCCwJAIAMgBygCEEYEQCAHIAE2AhAMAQsgByABNgIUCyABRQ0BCyABIAc2AhggAygCECICBEAgASACNgIQIAIgATYCGAsgAygCFCICRQ0AIAEgAjYCFCACIAE2AhgLIAMgBU8NACAFKAIEIgRBAXFFDQACQAJAAkACQCAEQQJxRQRAQZAWKAIAIAVGBEBBkBYgAzYCAEGEFkGEFigCACAAaiIANgIAIAMgAEEBcjYCBCADQYwWKAIARw0GQYAWQQA2AgBBjBZBADYCAAwGC0GMFigCACIHIAVGBEBBjBYgAzYCAEGAFkGAFigCACAAaiIANgIAIAMgAEEBcjYCBCAAIANqIAA2AgAMBgsgBEF4cSAAaiEAIAUoAgwhASAEQf8BTQRAIAUoAggiAiABRgRAQfgVQfgVKAIAQX4gBEEDdndxNgIADAULIAIgATYCDCABIAI2AggMBAsgBSgCGCEIIAEgBUcEQCAFKAIIIgIgATYCDCABIAI2AggMAwsgBSgCFCICBH8gBUEUagUgBSgCECICRQ0CIAVBEGoLIQQDQCAEIQYgAiIBQRRqIQQgASgCFCICDQAgAUEQaiEEIAEoAhAiAg0ACyAGQQA2AgAMAgsgBSAEQX5xNgIEIAMgAEEBcjYCBCAAIANqIAA2AgAMAwtBACEBCyAIRQ0AAkAgBSgCHCIEQQJ0IgIoAqgYIAVGBEAgAkGoGGogATYCACABDQFB/BVB/BUoAgBBfiAEd3E2AgAMAgsCQCAFIAgoAhBGBEAgCCABNgIQDAELIAggATYCFAsgAUUNAQsgASAINgIYIAUoAhAiAgRAIAEgAjYCECACIAE2AhgLIAUoAhQiAkUNACABIAI2AhQgAiABNgIYCyADIABBAXI2AgQgACADaiAANgIAIAMgB0cNAEGAFiAANgIADAELIABB/wFNBEAgAEH4AXFBoBZqIQICf0H4FSgCACIEQQEgAEEDdnQiAHFFBEBB+BUgACAEcjYCACACDAELIAIoAggLIQAgAiADNgIIIAAgAzYCDCADIAI2AgwgAyAANgIIDAELQR8hASAAQf///wdNBEAgAEEmIABBCHZnIgJrdkEBcSACQQF0a0E+aiEBCyADIAE2AhwgA0IANwIQIAFBAnRBqBhqIQQCfwJAAn9B/BUoAgAiBkEBIAF0IgJxRQRAQfwVIAIgBnI2AgAgBCADNgIAQRghAUEIDAELIABBGSABQQF2a0EAIAFBH0cbdCEBIAQoAgAhBANAIAQiAigCBEF4cSAARg0CIAFBHXYhBCABQQF0IQEgAiAEQQRxaiIGKAIQIgQNAAsgBiADNgIQQRghASACIQRBCAshACADIgIMAQsgAigCCCIEIAM2AgwgAiADNgIIQRghAEEIIQFBAAshBiABIANqIAQ2AgAgAyACNgIMIAAgA2ogBjYCAEGYFkGYFigCAEEBayIAQX8gABs2AgALQbQZLQAAQQJxRQ0AQbgZEA0LC9InAQl/QeAVKAIARQRAEEALAkBBtBktAABBAnEEQEG4GRAODQELAkACQAJAAkACQAJAAkACQCAAQfQBTQRAQfgVKAIAIgJBECAAQQtqQfgDcSAAQQtJGyIFQQN2IgB2IgFBA3EEQAJAIAFBf3NBAXEgAGoiBEEDdCIAQaAWaiIFIAAoAqgWIgEoAggiA0YEQEH4FSACQX4gBHdxNgIADAELIAMgBTYCDCAFIAM2AggLIAFBCGohBCABIABBA3I2AgQgACABaiIAIAAoAgRBAXI2AgQMCQsgBUGAFigCACIDTQ0BIAEEQAJAQQIgAHQiBEEAIARrciABIAB0cWgiB0EDdCIBQaAWaiIEIAEoAqgWIgAoAggiBkYEQEH4FSACQX4gB3dxIgI2AgAMAQsgBiAENgIMIAQgBjYCCAsgACAFQQNyNgIEIAAgBWoiByABIAVrIgZBAXI2AgQgACABaiAGNgIAIAMEQCADQXhxQaAWaiEBQYwWKAIAIQUCfyACQQEgA0EDdnQiA3FFBEBB+BUgAiADcjYCACABDAELIAEoAggLIQIgASAFNgIIIAIgBTYCDCAFIAE2AgwgBSACNgIICyAAQQhqIQRBjBYgBzYCAEGAFiAGNgIADAkLQfwVKAIAIghFDQEgCGhBAnQoAqgYIgIoAgRBeHEgBWshBiACIQADQAJAIAAoAhAiAUUEQCAAKAIUIgFFDQELIAEoAgRBeHEgBWsiACAGIAAgBkkiABshBiABIAIgABshAiABIQAMAQsLIAIoAhghBwJAIAIgAigCDCIBRwRAIAIoAggiACABNgIMIAEgADYCCAwBCwJAIAIoAhQiAAR/IAJBFGoFIAIoAhAiAEUNASACQRBqCyEDA0AgAyEEIAAiAUEUaiEDIAAoAhQiAA0AIAFBEGohAyABKAIQIgANAAsgBEEANgIADAELQQAhAQsCQCAHRQ0AAkAgAigCHCIAQQJ0IgMoAqgYIAJGBEAgA0GoGGogATYCACABDQFB/BUgCEF+IAB3cTYCAAwCCwJAIAIgBygCEEYEQCAHIAE2AhAMAQsgByABNgIUCyABRQ0BCyABIAc2AhggAigCECIABEAgASAANgIQIAAgATYCGAsgAigCFCIARQ0AIAEgADYCFCAAIAE2AhgLAkAgBkEPTQRAIAIgBSAGaiIAQQNyNgIEIAAgAmoiACAAKAIEQQFyNgIEDAELIAIgBUEDcjYCBCACIAVqIgUgBkEBcjYCBCAFIAZqIAY2AgBBgBYoAgAiAwRAIANBeHFBoBZqIQBBjBYoAgAhAQJ/QfgVKAIAIgRBASADQQN2dCIDcUUEQEH4FSADIARyNgIAIAAMAQsgACgCCAshAyAAIAE2AgggAyABNgIMIAEgADYCDCABIAM2AggLQYwWIAU2AgBBgBYgBjYCAAsgAkEIaiEEDAgLQX8hBSAAQb9/Sw0AIABBC2oiAUF4cSEFQfwVKAIAIgdFDQBBHyEIQQAgBWshBCAAQfT//wdNBEAgBUEmIAFBCHZnIgBrdkEBcSAAQQF0a0E+aiEICwJAAkACQCAIQQJ0KAKoGCIBRQRAQQAhAAwBC0EAIQAgBUEZIAhBAXZrQQAgCEEfRxt0IQIDQAJAIAEoAgRBeHEgBWsiBiAETw0AIAEhAyAGIgQNAEEAIQQgASEADAMLIAAgASgCFCIGIAYgASACQR12QQRxaigCECIBRhsgACAGGyEAIAJBAXQhAiABDQALCyAAIANyRQRAQQAhA0ECIAh0IgBBACAAa3IgB3EiAEUNAyAAaEECdCgCqBghAAsgAEUNAQsDQCAAKAIEQXhxIAVrIgIgBEkhASACIAQgARshBCAAIAMgARshAyAAKAIQIgEEfyABBSAAKAIUCyIADQALCyADRQ0AIARBgBYoAgAgBWtPDQAgAygCGCEIIAMgAygCDCIARwRAIAMoAggiASAANgIMIAAgATYCCAwHCyADKAIUIgEEfyADQRRqBSADKAIQIgFFDQIgA0EQagshAgNAIAIhBiABIgBBFGohAiAAKAIUIgENACAAQRBqIQIgACgCECIBDQALIAZBADYCAAwGCyAFQYAWKAIAIgNNBEBBjBYoAgAhAAJAIAMgBWsiAUEQTwRAIAAgBWoiAiABQQFyNgIEIAAgA2ogATYCACAAIAVBA3I2AgQMAQsgACADQQNyNgIEIAAgA2oiASABKAIEQQFyNgIEQQAhAkEAIQELQYAWIAE2AgBBjBYgAjYCACAAQQhqIQQMBwsgBUGEFigCACIASQRAQYQWIAAgBWsiATYCAEGQFkGQFigCACIAIAVqIgI2AgAgAiABQQFyNgIEIAAgBUEDcjYCBCAAQQhqIQQMBwtBACEEQeAVKAIARQRAEEALQegVKAIAIgAgBUEvaiIGakEAIABrcSIDIAVNDQZBsBkoAgAiAARAQagZKAIAIgEgA2oiAiABTQ0HIAAgAkkNBwtBtBktAABBBHFFBEACQAJAAkACQEGQFigCACIBBEBB0BkhAANAIAAoAgAiAiABTQRAIAEgAiAAKAIEakkNAwsgACgCCCIADQALC0HoGRAOGkEAEBQiAkF/Rg0DIAMhAUHkFSgCACIAQQFrIgQgAnEEQCABIAJrIAIgBGpBACAAa3FqIQELIAEgBU0NA0GwGSgCACIABEBBqBkoAgAiBCABaiIHIARNDQQgACAHSQ0ECyABEBQiACACRw0BDAYLQegZEA4aQegVKAIAIgEgBkGEFigCAGtqQQAgAWtxIgEQFCICIAAoAgAgACgCBGpGDQEgAiEACyAAQX9GDQEgBUEwaiABSwRAQegVKAIAIgIgBiABa2pBACACa3EiAhAUQX9GDQIgASACaiEBCyAAIQIMBAsgAkF/Rw0DC0G0GUG0GSgCAEEEcjYCAEHoGRANC0HoGRAOGiADEBQhAkEAEBQhAEHoGRANIAJBf0YNAyAAQX9GDQMgACACTQ0DIAAgAmsiASAFQShqTQ0DDAILQQAhAAwEC0HoGRANC0GoGUGoGSgCACABaiIANgIAQawZKAIAIABJBEBBrBkgADYCAAsCQAJAAkBBkBYoAgAiBARAQdAZIQADQCACIAAoAgAiAyAAKAIEIgZqRg0CIAAoAggiAA0ACwwCC0GIFigCACIAQQAgACACTRtFBEBBiBYgAjYCAAtBACEAQdQZIAE2AgBB0BkgAjYCAEGYFkF/NgIAQZwWQeAVKAIANgIAQdwZQQA2AgADQCAAQQN0IgMgA0GgFmoiBDYCqBYgAyAENgKsFiAAQQFqIgBBIEcNAAtBhBYgAUEoayIAQXggAmtBB3EiAWsiAzYCAEGQFiABIAJqIgE2AgAgASADQQFyNgIEIAAgAmpBKDYCBEGUFkHwFSgCADYCAAwCCyACIARNDQAgAyAESw0AIAAoAgxBCHENACAAIAEgBmo2AgRBkBYgBEF4IARrQQdxIgBqIgI2AgBBhBZBhBYoAgAgAWoiASAAayIANgIAIAIgAEEBcjYCBCABIARqQSg2AgRBlBZB8BUoAgA2AgAMAQtBiBYoAgAgAksEQEGIFiACNgIACyABIAJqIQZB0BkhAAJAA0AgBiAAKAIAIgNHBEAgACgCCCIADQEMAgsLIAAtAAxBCHFFDQMLQdAZIQADQAJAIAAoAgAiAyAETQRAIAQgAyAAKAIEaiIGSQ0BCyAAKAIIIQAMAQsLQYQWIAFBKGsiAEF4IAJrQQdxIgNrIgc2AgBBkBYgAiADaiIDNgIAIAMgB0EBcjYCBCAAIAJqQSg2AgRBlBZB8BUoAgA2AgAgBCAGQScgBmtBB3FqQS9rIgAgACAEQRBqSRsiA0EbNgIEIANB2BkpAgA3AhAgA0HQGSkCADcCCEHYGSADQQhqNgIAQdQZIAE2AgBB0BkgAjYCAEHcGUEANgIAIANBGGohAANAIABBBzYCBCAAQQhqIABBBGohACAGSQ0ACyADIARGDQAgAyADKAIEQX5xNgIEIAQgAyAEayICQQFyNgIEIAMgAjYCAAJ/IAJB/wFNBEAgAkH4AXFBoBZqIQACf0H4FSgCACIBQQEgAkEDdnQiAnFFBEBB+BUgASACcjYCACAADAELIAAoAggLIQEgACAENgIIIAEgBDYCDEEIIQNBDAwBC0EfIQAgAkH///8HTQRAIAJBJiACQQh2ZyIAa3ZBAXEgAEEBdGtBPmohAAsgBCAANgIcIARCADcCECAAQQJ0QagYaiEBAkACQEH8FSgCACIDQQEgAHQiBnFFBEBB/BUgAyAGcjYCACABIAQ2AgAMAQsgAkEZIABBAXZrQQAgAEEfRxt0IQAgASgCACEDA0AgAyIBKAIEQXhxIAJGDQIgAEEddiEDIABBAXQhACABIANBBHFqIgYoAhAiAw0ACyAGIAQ2AhALIAQgATYCGEEMIQMgBCIBIQBBCAwBCyABKAIIIgAgBDYCDCABIAQ2AgggBCAANgIIQQAhAEEMIQNBGAsgAyAEaiABNgIAIARqIAA2AgALQYQWKAIAIgAgBU0NAEGEFiAAIAVrIgE2AgBBkBZBkBYoAgAiACAFaiICNgIAIAIgAUEBcjYCBCAAIAVBA3I2AgQgAEEIaiEEDAMLIwFBMDYCAEEAIQQMAgsgACACNgIAIAAgACgCBCABajYCBCACQXggAmtBB3FqIgggBUEDcjYCBCADQXggA2tBB3FqIgQgBSAIaiIDayEGAkBBkBYoAgAgBEYEQEGQFiADNgIAQYQWQYQWKAIAIAZqIgA2AgAgAyAAQQFyNgIEDAELQYwWKAIAIARGBEBBjBYgAzYCAEGAFkGAFigCACAGaiIANgIAIAMgAEEBcjYCBCAAIANqIAA2AgAMAQsgBCgCBCIAQQNxQQFGBEAgAEF4cSEJIAQoAgwhAgJAIABB/wFNBEAgBCgCCCIBIAJGBEBB+BVB+BUoAgBBfiAAQQN2d3E2AgAMAgsgASACNgIMIAIgATYCCAwBCyAEKAIYIQcCQCACIARHBEAgBCgCCCIAIAI2AgwgAiAANgIIDAELAkAgBCgCFCIABH8gBEEUagUgBCgCECIARQ0BIARBEGoLIQEDQCABIQUgACICQRRqIQEgACgCFCIADQAgAkEQaiEBIAIoAhAiAA0ACyAFQQA2AgAMAQtBACECCyAHRQ0AAkAgBCgCHCIAQQJ0IgEoAqgYIARGBEAgAUGoGGogAjYCACACDQFB/BVB/BUoAgBBfiAAd3E2AgAMAgsCQCAEIAcoAhBGBEAgByACNgIQDAELIAcgAjYCFAsgAkUNAQsgAiAHNgIYIAQoAhAiAARAIAIgADYCECAAIAI2AhgLIAQoAhQiAEUNACACIAA2AhQgACACNgIYCyAGIAlqIQYgBCAJaiIEKAIEIQALIAQgAEF+cTYCBCADIAZBAXI2AgQgAyAGaiAGNgIAIAZB/wFNBEAgBkH4AXFBoBZqIQACf0H4FSgCACIBQQEgBkEDdnQiAnFFBEBB+BUgASACcjYCACAADAELIAAoAggLIQEgACADNgIIIAEgAzYCDCADIAA2AgwgAyABNgIIDAELQR8hAiAGQf///wdNBEAgBkEmIAZBCHZnIgBrdkEBcSAAQQF0a0E+aiECCyADIAI2AhwgA0IANwIQIAJBAnRBqBhqIQACQAJAQfwVKAIAIgFBASACdCIFcUUEQEH8FSABIAVyNgIAIAAgAzYCAAwBCyAGQRkgAkEBdmtBACACQR9HG3QhAiAAKAIAIQEDQCABIgAoAgRBeHEgBkYNAiACQR12IQEgAkEBdCECIAAgAUEEcWoiBSgCECIBDQALIAUgAzYCEAsgAyAANgIYIAMgAzYCDCADIAM2AggMAQsgACgCCCIBIAM2AgwgACADNgIIIANBADYCGCADIAA2AgwgAyABNgIICyAIQQhqIQQMAQsCQCAIRQ0AAkAgAygCHCIBQQJ0IgIoAqgYIANGBEAgAkGoGGogADYCACAADQFB/BUgB0F+IAF3cSIHNgIADAILAkAgAyAIKAIQRgRAIAggADYCEAwBCyAIIAA2AhQLIABFDQELIAAgCDYCGCADKAIQIgEEQCAAIAE2AhAgASAANgIYCyADKAIUIgFFDQAgACABNgIUIAEgADYCGAsCQCAEQQ9NBEAgAyAEIAVqIgBBA3I2AgQgACADaiIAIAAoAgRBAXI2AgQMAQsgAyAFQQNyNgIEIAMgBWoiAiAEQQFyNgIEIAIgBGogBDYCACAEQf8BTQRAIARB+AFxQaAWaiEAAn9B+BUoAgAiAUEBIARBA3Z0IgVxRQRAQfgVIAEgBXI2AgAgAAwBCyAAKAIICyEBIAAgAjYCCCABIAI2AgwgAiAANgIMIAIgATYCCAwBC0EfIQAgBEH///8HTQRAIARBJiAEQQh2ZyIAa3ZBAXEgAEEBdGtBPmohAAsgAiAANgIcIAJCADcCECAAQQJ0QagYaiEBAkACQCAHQQEgAHQiBXFFBEBB/BUgBSAHcjYCACABIAI2AgAMAQsgBEEZIABBAXZrQQAgAEEfRxt0IQAgASgCACEFA0AgBSIBKAIEQXhxIARGDQIgAEEddiEFIABBAXQhACABIAVBBHFqIgYoAhAiBQ0ACyAGIAI2AhALIAIgATYCGCACIAI2AgwgAiACNgIIDAELIAEoAggiACACNgIMIAEgAjYCCCACQQA2AhggAiABNgIMIAIgADYCCAsgA0EIaiEEC0G0GS0AAEECcUUNAEG4GRANCyAEC+0DAQR/AkAgAkEASgRAIAJB/wNNDQFB8AlBExAsQfQJEA4aAkBBlAsoAgAiAwRAQZALKAIARQ0BC0H0CRANDAILQfQKIAI2AgBB8AogATYCAEHsCiAANgIAQZALQQE2AgBB+AogA0EBaiIENgIAQfwKQQD+FwIAQYQLIAP+FwIAQYALQYAL/hACAEEBav4XAgBBiAv+EAIABEACQEGMCigCAEUEQEGMCkF/EDAMAQtBmAooAgAEQEGUCkEB/h4CABpBlApB/////wcQDwsLQfgKKAIAIQQLQfQJEA0gAkH8Cv4QAgAiA0oEQCAEQQF0IQYDQCADQQAgA0HAACACIANrIAZtIgQgBEHAAEwbIANqIgX+SAL8CiIERgR/IAEgAyAFIAIgAiAFShsgABEBAEH8Cv4QAgAFIAQLIgMgAkgNAAsLQQAhAwJAA0BBhAv+EAIAQQBMDQFBhAv+EAIAQQBMDQFBhAv+EAIAQQBMDQFBhAv+EAIAQQBMDQEgA0EEaiIDQYCAwABHDQALQfQJEA4aQYwLQQH+FwIAQYQL/hACAEEASgRAA0BBvApB9AkQJUGEC/4QAgBBAEoNAAsLQYwLQQD+FwIAQfQJEA0LQfQJEA4aQZALQQA2AgBB9AkQDQsPCyABQQAgAiAAEQEAC9IBAgN/AXxB5AAhAwJAAn8CQANAIAMEQCABBEAgASgCAA0DCyADQQFrIQMgACgCACACRg0BDAQLCyABDQBBAQwBCyABQQH+HgIAGkEACyMFIQMCQCAAKAIAIAJHDQBEAAAAAAAA8D9EAAAAAAAAWUAgAxshBiMDIQQDQAJAAkAgA0UEQCAELQApQQFHDQELA0AgBCgCJA0EIAAgAiAGEBVBt39GDQALDAELIAAgAkQAAAAAAADwfxAVGgsgACgCACACRg0ACwsNACABQQH+JQIAGgsLWgECfyAAQQdqQXhxIQIDQAJAIAJBAEHkCf4QAgAiACACaiIBIABNG0UEQCABPwBBEHRNDQEgARALDQELIwFBMDYCAEF/DwtBACAAIAH+SALkCSAARw0ACyAAC9UBAgF8AX8gAEEDcQR/QWQFEDEjBkUEQAJ/EAAhA0EAQQAgAP5IApgLGgJAEAAgAiADoCICZA0AAn8DQEEAIABBACAAQQD+SAKYCyIEIAAgBEYbRQ0BGhAxIAEgAP4QAgBGBEBBAEEAIAD+SAKYCxoQACACZA0DDAELC0F6CwwBC0EAIABBAP5IApgLGkG3fwsPC0F6Qbd/QQAgACABQn8gAkQAAAAAAECPQKJEAAAAAABAj0Ci/AYgAkQAAAAAAADwf2Eb/gECACIAQQJGGyAAQQFGGwsLCgAgABAzIAAQEAuHBAECfyACQYAETwRAIAIEQCAAIAEgAvwKAAALDwsgACACaiEDAkAgACABc0EDcUUEQAJAIABBA3FFBEAgACECDAELIAJFBEAgACECDAELIAAhAgNAIAIgAS0AADoAACABQQFqIQEgAkEBaiICQQNxRQ0BIAIgA0kNAAsLIANBfHEhAAJAIANBwABJDQAgAiAAQUBqIgRLDQADQCACIAEoAgA2AgAgAiABKAIENgIEIAIgASgCCDYCCCACIAEoAgw2AgwgAiABKAIQNgIQIAIgASgCFDYCFCACIAEoAhg2AhggAiABKAIcNgIcIAIgASgCIDYCICACIAEoAiQ2AiQgAiABKAIoNgIoIAIgASgCLDYCLCACIAEoAjA2AjAgAiABKAI0NgI0IAIgASgCODYCOCACIAEoAjw2AjwgAUFAayEBIAJBQGsiAiAETQ0ACwsgACACTQ0BA0AgAiABKAIANgIAIAFBBGohASACQQRqIgIgAEkNAAsMAQsgA0EESQRAIAAhAgwBCyACQQRJBEAgACECDAELIANBBGshBCAAIQIDQCACIAEtAAA6AAAgAiABLQABOgABIAIgAS0AAjoAAiACIAEtAAM6AAMgAUEEaiEBIAJBBGoiAiAETQ0ACwsgAiADSQRAA0AgAiABLQAAOgAAIAFBAWohASACQQFqIgIgA0cNAAsLC5wDAQZ/IAAtAABBD3FFBEAgAEEEakEAQQr+SAIAQQpxDwsCfyAAKAIAIQICQAJAAkACQAJAIwMiASgCGCIEIAAoAgQiA0H/////A3EiBkcNAAJAIAJBCHFFDQAgACgCFEEATg0AIABBADYCFCADQYCAgIAEcSEDDAILIAJBA3FBAUcNAEEGIAAoAhQiAUH+////B0sNBRogACABQQFqNgIUDAILQTghBSAGQf////8DRg0DAkAgBg0AQQAgAyACQQRxGw0AIAMgAEEEaiADIAJBgAFxBH8gASgCUEUEQCABQXQ2AlALIAAoAgghBiABIABBEGo2AlQgBEGAgICAeHIgBCAGGwUgBAsgA0GAgICABHFy/kgCAEYNASABQQA2AlQgAkEMcUEMRw0AIAAoAggNBAtBCgwECyACQQ9xDQELQQAMAgsgACABKAJMIgI2AhAgACABQcwAaiIFNgIMIABBEGohBCACIAVHBEAgAkEEayAENgIACyABIAQ2AkxBACEFIAFBADYCVCADRQ0AIABBADYCFEE+IQULIAULCxYAIABBAP5BAgBBAkYEQCAAQQEQDwsLMQAgAEEAQQH+SAIABEAgAEEBQQL+SAIAGgNAIABBAEECEBMgAEEAQQL+SAIADQALCwvwAQIDfwN8IwBBEGsiBCQAAkACfwJAAkAjBSICDQAjAyIDLQAoQQFHDQAgAy0AKUUNAQtEAAAAAAAA8D9EAAAAAAAAWUAgAhshBRAARAAAAAAAAPB/oCEHIwMhAgNAAkAgAigCJEUEQCAHEAChIgZEAAAAAAAAAABlRQ0BQckADAQLQQshAAwECyAAIAEgBSAGIAUgBmMbEBUiA0G3f0YNAAtBACADawwBC0EAIAAgAUQAAAAAAADwfxAVawsiAEEAIABBb3FBC0YbIAAgAEHJAEcbIgBBG0cNAEEbQQBBiA0oAgAbIQALIARBEGokACAAC8kDAQN/IwBBEGsiBCQAIAAQDhoCQCAAQZwLRwRAIAAgARA1IQEMAQsjASIDQQE6AARBnAsgARA1IQEgA0EAOgAECyAAEA0gAQR/IAQgAigCCDYCCCAEIAIpAgA3AwAjAEEwayIAJAACQAJ/IAEoAhwiA/4QAnwhAgNAQQAgAkUNARogAiADIAIgAkEBav5IAnwiAkcNAAtBAQtFBEBBACEDDAELIAFBBGoiAhAOGiAAIAQoAgg2AiAgACAEKQIANwMYIAEgAEEYahA3IQMgAhANAn8gAwR/IAFBAv5BAgAhBUEBIQMgASgCHCICIAVBAkYNARogACABNgIsIAAgATYCECAAQRU2AiggAEEWNgIkIAAgACkCJDcDCCMAQRBrIgMkACACKAJ4QQRqEA4aIAIoAnggAyAAKAIQNgIIIAMgACkCCDcDACADEDcaIAIoAnhBBGoQDQJAIAIoAnhBAv5BAgBBAkYNACAC/hACgAEEQCACQX/+AAIAGgwBCyACIwMQDAsgA0EQaiQAQQEFQQALIQMgASgCHAsiAUEB/iUCfEEBRgRAIAFB/ABqQf////8HEA8LCyAAQTBqJAAgAwVBAAsgBEEQaiQAC+QJAxh/AX0BewJAIAEgAk4NACAFIAZODQAgBCADQQRqIg1OBEAgBCADIANBCGoiCCAEQQFqIgcgByAISBsgA2tBBWsiD0F8cWoiCEEFaiIHIAQgB0obIgdBAnQgD0ECdEFwcWshFSAHIAhrQQRrIhZBfHEhECAPQQRxIRkgD0ECdkEBakH+////B3EhGiADQQJ0IRcgACgCECEHIAEhEgNAIAAoAhQiCSABIBFqbEECdCIIIAAoAggiDCAVamohGyAMIBdqIAhqQRBqIRwgACgCBCAFIAdsQQJ0aiEKIAwgCSASbEECdGohDiAAKAIAIAAoAgwgEmxBAnRqIR0gBSEUA0AgHSAUQQJ0aioCACIf/RMhIEEAIQwgAyEIQQAhEyANIgchCSAPQQRPBEADQCAOIAhBAnQiCWoiCCAJIApq/QAAACAg/eYBIAj9AAAA/eQB/QsAACAOIAdBAnQiCWoiCCAJIApq/QAAACAg/eYBIAj9AAAA/eQB/QsAACAMIgtBAmohDCAHQQRqIQggB0EIaiIJIQcgE0ECaiITIBpHDQALIAtBAXIhCwsgGUUEQCAOIAhBAnQiCGoiByAIIApq/QAAACAg/eYBIAf9AAAA/eQB/QsAACAMIQsgCSEICwJAIAQgCEwNAAJAAkAgFkEISQ0AIBwgC0EEdCIHaiAKIBVqIAdqSQRAIAogF2ogB2pBEGogByAbakkNAQsgCCAQaiEHQQAhDANAIA4gCCAMakECdCITaiIJIAn9AAIAIAogE2r9AAIAICD95gH95AH9CwIAIAxBBGoiDCAQRw0ACyAQIBZHDQEMAgsgCCEHCwNAIA4gB0ECdCIJaiIIIAgqAgAgCSAKaioCACAflJI4AgAgB0EBaiIHIARIDQALCyAKIAAoAhAiB0ECdGohCiAUQQFqIhQgBkcNAAsgEUEBaiERIBJBAWoiEiACRw0ACwwBCyADIARODQAgACgCBCIJIAAoAhAiHiAFbCIHQQJ0aiEMIAAoAgwhFSAAKAIAIRYgCSAEIAdqIB4gBiAFQX9zamxqQQJ0aiEXIAAoAhQiDkECdCEZIAMgBCADayIPQXxxIhBqIQggACgCCCIRIAQgASAObCIAakECdGohGiARIAAgA2pBAnRqIRsgCSADIAdqQQJ0aiEcQQAhACAeQYCAgIACcUEARyEdIA9BBEkhEwNAIBEgASAObEECdGohGCAWIAEgFWxBAnRqIRIgGyAAIBlsIgdqIBdJIBwgByAaaklxIB1yIBNyIRQgBSEJIAwhCgNAIBIgCUECdGoqAgAhHyADIQcCQCAURQRAIB/9EyEgQQAhBwNAIBggAyAHakECdCINaiILIAv9AAIAIAogDWr9AAIAICD95gH95AH9CwIAIAdBBGoiByAQRw0ACyAIIQcgDyAQRg0BCyAHQQFqIQsgBCAHa0EBcQRAIBggB0ECdCINaiIHIAcqAgAgCiANaioCACAflJI4AgAgCyEHCyAEIAtGDQADQCAYIAdBAnQiDWoiCyALKgIAIAogDWoqAgAgH5SSOAIAIBggDUEEaiINaiILIAsqAgAgCiANaioCACAflJI4AgAgB0ECaiIHIARHDQALCyAKIB5BAnRqIQogCUEBaiIJIAZHDQALIABBAWohACABQQFqIgEgAkcNAAsLC90EAwJ7B38BfSMAQSBrIgskAAJAIARBAEwNACAFQQBMDQAgBBARIQwgBEEIdiIOQQJ0EBEhDQJAIAxFDQAgDUUNACAEQYACTwRAIAAgAUECdGohEEEAIQADQCAAQQh0IREgECAAQQp0aiEP/QwAAAAAAAAAAAAAAAAAAAAAIQlBACEBA0AgDyABQQJ0aiIE/QACMP3gASAE/QACIP3gASAE/QACEP3gASAE/QACAP3gASAJ/esB/esB/esB/esBIQkgAUEQaiIBQYACRw0ACyAMIBFqIQEgDSAAQQJ0agJ9IAkgCSAJ/Q0ICQoLDA0ODwABAgMAAQID/esBIgkgCSAJ/Q0EBQYHAAECAwABAgMAAQID/esB/R8AIhJDAAAAAFwEQEMAAIA/IBKV/RMhCkEAIQQDQCABIARqIA8gBEECdGr9AAIA/QwAAP5CAAD+QgAA/kIAAP5C/eYBIAr95gEiCf0fAJD8AP0RIAn9HwGQ/AD9HAEgCf0fApD8AP0cAiAJ/R8DkPwA/RwD/QyB////gf///4H///+B/////bgB/Qx/AAAAfwAAAH8AAAB/AAAA/bYBIAn9DQAECAwAAAAAAAAAAAAAAAD9WgAAACAEQQRqIgRBgAJHDQALIBJDBAIBPJQMAQsgAUEAQYAC/AsAQwAAAAALOAIAIABBAWoiACAORw0ACwsgC0EANgIcIAsgBTYCGCALIA42AhQgCyANNgIMIAsgDDYCCCALIAIgA2o2AgQgCyAGIAdBAnRqNgIQIAggC0EEaiAFEBILIAwQECANEBALIAtBIGokAAuoAgICfwF7IwBBIGsiCSQAAkAgBUEATA0AIARBAEwEQCAFQQJ0IgBFDQEgBiAHQQJ0akEAIAD8CwAMAQsgCSAFNgIcIAkgBEEFdiIKNgIYIAkgAiADajYCCCAJIAYgB0ECdGo2AhAgCSAAIAFBAnRqIgI2AgwgCkECdBARIgFFDQAgBEEfSwRAQQAhBwNAIAEgB0ECdGogAiAHQQd0aiIA/QACACAA/QACcP3kASAA/QACQP3kASAA/QACICAA/QACYP3kAf3kASAA/QACECAA/QACUP3kASAA/QACMP3kAf3kASIL/R8AIAv9HwGSIAv9HwKSIAv9HwOSOAIAIAdBAWoiByAKRw0ACwsgCSABNgIUIAggCUEIaiAFEBIgARAQCyAJQSBqJAALMAEBfyABIwMiAigCSCAAQQJ0aiIAKAIARwRAIAAgATYCACACIAItACpBAXI6ACoLC8QDAQZ/IwBBEGsiBSQAAkAgACAAKAIARw0AAkACQCAAQSBqQQJBA/5IAgBBAmsOAgECAAsjAyEBIAVBDGoEQCAFIAEtACg2AgwLIAFBAToAKBADIwBBEGsiASQAAkAgACAAKAIARw0AAkAgACgCIEEDRg0AIAAjA0cNAAwBCyMDIQQgAUEMaiIDBEAgAyAELQAoNgIACyAEQQE6ACggASgCDEUEQCMDQQA6ACgLAkACQAJAIABBIGoiBigCACIDBEADQCACQRxGDQIgAkHJAEYNAiADQQNOBEAgASgCDCIAQQJNBH8jAyAAOgAoQQAFQRwLGgwGC0EBIQQgBiADEBshAiAGKAIAIgMNAAsMAgsgASgCDCICQQJNBH8jAyACOgAoQQAFQRwLGkEBIQQMAgtBACEECyABKAIMIgNBAk0EfyMDIAM6AChBAAVBHAsaIAJBHEYNASACQckARg0BCyMAQRBrQQA2AgwCQEGgDSgCACICRQ0AQaANQagNIAIQE0GoDSgCAEUNAEGgDUEBEA8LIARFDQAgABABCyABQRBqJAAgBSgCDCIAQQJNBH8jAyAAOgAoQQAFQRwLGgsLIAVBEGokAAvVBgEFfyMAQTBrIgQkAAJAIABFBEBBHCECDAELQawNKAIARQRAQawNQSs2AgALQckLLQAARQRAAkBBywssAAAiA0UNAEGMDUEAQYGAgIB4/kgCACEBIANBAEgEQEHLC0EAOgAACyABRQ0AA0BBjA0gAUH/////B2ogASABQQBIGyIDIANB/////wdr/kgCACIBIANGDQEgAkEBaiICQQpHDQALQYwNQQH+HgIAQQFqIQEDQCABQQBIBEBBjA1BACABEBMgAUH/////B2ohAQsgAUGMDSABIAFBgICAgHhy/kgCACIBRw0ACwtBkA0oAgAiAQRAA0ACQCABRQ0AIAEoAkxBAE4NACABQQA2AkwLIAEoAjgiAQ0ACwsCQEGMDSgCAEEATg0AQYwNQf////8H/h4CAEGBgICAeEYNAEGMDUEBEA8LAkBBnA0oAgAiAUUNACABKAJMQQBODQAgAUEANgJMCwJAQeAJKAIAIgFFDQAgASgCTEEATg0AIAFBADYCTAsCQEGcDSgCACIBRQ0AIAEoAkxBAE4NACABQQA2AkwLQckLQQE6AAALIARBBGpBAEEs/AsAIARBxAgoAgAiATYCBEHICCgCAEGPAWoiAkEAIAFBD2ogBCgCDBtqIgMQESIBIAIQLyABIAM2AjAgASABNgIsIAEgATYCAEGsDUGsDSgCACICQQFqNgIAIAEgAUHMAGo2AkwgASACNgIYIAFB6As2AmAgAUEDQQIgBCgCEBs2AiAgASAEKAIEIgM2AjggASABQYcBakF8cSICNgJ0IAJBBWohAkHICCgCAARAIAEgAkEDakF8cSICNgJIQcgIKAIAIAJqIQILIAEgBCgCDCIFIAIgA2pBD2pBcHEgBRs2AjQgARBBIwMhAhAkIAIoAgwhAyABIAI2AgggASADNgIMIAMgATYCCCABKAIIIAE2AgwQI0HMC0HMCygCACICQQFqNgIAIAJFBEBBywtBAToAAAsgASAEQQRqQRRBABAFIgIEQEHMC0HMCygCAEEBayIANgIAIABFBEBBywtBADoAAAsQJCABKAIMIgAgASgCCCIDNgIIIAMgADYCDCABIAE2AgwgASABNgIIECMMAQsgACABNgIACyAEQTBqJAAgAgs0AQF/QaQNKAIAIgAEQEGkDSAAQQFrNgIADwtBoA1BAP4XAgBBqA0oAgAEQEGgDUEBEA8LC1EBAn8jAygCGCIAQaANKAIARwRAQaANQQAgAP5IAgAiAQRAA0BBoA1BqA0gARATQaANQQAgAP5IAgAiAQ0ACwsPC0GkDUGkDSgCAEEBajYCAAuPBgEGfyMAQSBrIgIkACACQQA2AhggAkIANwMQIAJCADcDCCAAKAIQGiMEBEAQAwsCQAJAIAEtAABBD3FFDQAjAygCGCABKAIEQf////8HcUYNAAwBCwJ/IAAoAgAiBwRAIAAoAgghAyAAQQxqQQH+HgIAGiAAQQhqDAELIABBIGoiAxAaIAJBAjYCFCACQQA2AhAgAiAAKAIEIgQ2AgwgACACQQhqIgU2AgQCQCAAKAIURQRAIAAgBTYCFAwBCyAEIAJBCGo2AgALIAMQGUECIQMgAkEUagshBSABEA0jAyEGIAJBBGoiBARAIAQgBi0AKDYCAAsgBkECOgAoIAIoAgRBAUYEQCMDQQE6ACgLIAUgAxAbIQQCQCAFKAIAIANHDQADQCAEQRtHQQAgBBsNASAFIAMQGyEEIAUoAgAgA0YNAAsLIARBACAEQRtHGyEEAn8CQAJAIAcEQCAEQQtGBEBBC0EAIAAoAgggA0YbIQQLIABBDGoiA0F//h4CAEGBgICAeEcNAgwBCyACQRBqQQBBAv5IAgBFBEAgAEEgaiIDEBoCQCAAKAIEIAJBCGpGBEAgACACKAIMNgIEDAELIAIoAggiBUUNACAFIAIoAgw2AgQLAkAgACgCFCACQQhqRgRAIAAgAigCCDYCFAwBCyACKAIMIgBFDQAgACACKAIINgIACyADEBkgAigCGCIARQ0CIABBf/4eAgBBAUcNAiACKAIYIQMMAQsgAkEUahAaIAEQDhoCQCACKAIMDQAgAS0AAEEIcQ0AIAFBCGpBAf4eAgAaCwJAIAIoAggiAwRAIAEoAgQiAEEASgRAIAFBBGogACAAQYCAgIB4cv5IAgAaCyADQQxqIgBBAP4XAgAgAEH/////BxAPDAELIAEtAABBCHENACABQQhqQQH+JQIAGgsgAigCBAwCCyADQQEQDwsgARAOIQAgAigCBCIBQQJNBH8jAyABOgAoQQAFQRwLGiAAIAQgABtBC0cNAUEBCyIAQQJNBH8jAyAAOgAoQQAFQRwLGgsgAkEgaiQAC1MBA38jAEEQayICJAAjAyEDIAJBDGoiBARAIAQgAy0AKDYCAAsgA0EBOgAoIAAgARAbIAIoAgwiAUECTQR/IwMgAToAKEEABUEcCxogAkEQaiQAC20BAn8jAEEQayIBJAAgAEEBNgIgIABBBGoiAhAOGiAAKAIsIAAoAjBHBEADQCABQQRqIAAQOSACEA0gASgCDCABKAIEEQAAIAIQDhogACgCLCAAKAIwRw0ACwsgAhANIABBADYCICABQRBqJAALLgAgACgCAEUEQCAAQQEQMA8LIAAoAgwEQCAAQQhqIgBBAf4eAgAaIABBARAPCwt3AQR/QbwRKAIAIwMoAhhGBEBBvBFBADYCAAsDQEG0ESgCACECQbARQbARKAIAIgAgAEEBa0EAIABB/////wdxIgFBAUcbQQAgAUH/////B0cbIgP+SAIAIABHDQALAkAgAw0AIAJFIABBAE5xDQBBsBEgARAPCwtLAQJ/AkACQANAQQYhAQJAQbARKAIAIgBB/////wdxQf7///8Haw4CAwIACyAAQbARIAAgAEEBav5IAgBHDQALQQAPC0EKIQELIAELnwEBAX8CQBAqQQpHDQBB5AAhAANAAkAgAEUNAEGwESgCAEUNACAAQQFrIQBBtBEoAgBFDQELCxAqQQpHDQADQAJAQbARKAIAIgBB/////wdxQf////8HRw0AQbQRQQH+HgIAGkGwESAAQX/+SAIAGkG4ESgCABpBsBFBfxAmIQBBtBFBAf4lAgAaIABFDQAgAEEbRw0CCxAqQQpGDQALCwu0AQEDfwJAIAAoAgBBAkYEQCMAQRBrQQA2AgwMAQsjAEEQayICJAADQAJAAkACQAJAAkAgAEEAQQH+SAIADgQAAgEDBQsgAkEEaiIDIAA2AgQgAkEhNgIEIAIjAyIEKAJENgIMIAQgAzYCRCABEQIAIwMgAigCDDYCRCAAQQL+QQIAQQNHDQAgAEH/////BxAPCyACQRBqJAAMAgsgAEEBQQP+SAIAGgsgAEEAQQMQEwwBCwsLCzAAIAAoAgBBgQFOBEBBlA0oAgAiAARAA0BBlA1BmA0gABATQZQNKAIAIgANAAsLCwsnAAJAQQBBf/4eApQNQQFHDQBBmA0oAgBFDQBBlA1B/////wcQDwsL1gIBAX8CQCABRQ0AIABBADoAACAAIAFqIgJBAWtBADoAACABQQNJDQAgAEEAOgACIABBADoAASACQQNrQQA6AAAgAkECa0EAOgAAIAFBB0kNACAAQQA6AAMgAkEEa0EAOgAAIAFBCUkNACAAQQAgAGtBA3EiAmoiAEEANgIAIAAgASACa0F8cSICaiIBQQRrQQA2AgAgAkEJSQ0AIABBADYCCCAAQQA2AgQgAUEIa0EANgIAIAFBDGtBADYCACACQRlJDQAgAEEANgIYIABBADYCFCAAQQA2AhAgAEEANgIMIAFBEGtBADYCACABQRRrQQA2AgAgAUEYa0EANgIAIAFBHGtBADYCACACIABBBHFBGHIiAmsiAUEgSQ0AIAAgAmohAANAIABCADcDGCAAQgA3AxAgAEIANwMIIABCADcDACAAQSBqIQAgAUEgayIBQR9LDQALCwvyAQEFfyMAQRBrIgMkACADQQA2AgwgAEEgaiIGEBogACgCFCICQQBHIQUCQCABRQ0AIAJFDQADQAJAIAJBCGpBAEEB/kgCAARAIAMgAygCDEEBajYCDCACIANBDGo2AhAMAQsgBCACIAQbIQQgAUEBayEBCyACKAIAIgJBAEchBSABRQ0BIAINAAsLAkAgBQRAIAIoAgQiAQRAIAFBADYCAAsgAkEANgIEDAELIABBADYCBAsgACACNgIUIAYQGSADKAIMIgIEQANAIANBDGpBACACEBMgAygCDCICDQALCyAEBEAgBEEMahAZCyADQRBqJAALYgEBfwJAIwUEQEGEDf4QAgANASMBIgAtAARFBEAgAEEBOgAEQZwLEA4aQZwLIwMQNiEAQZwLEA0CQCAARQ0AIAAoAiANACAAECcLIwFBADoABAsLDwtBhA3+EAIAEAcQBgALhwEBA38jAEEgayIBJAACQCAAKAIIRQRAIABBEGoiAhAOGiAAQQI2AgwgAhANIABBKGoQKAwBCyAAKAIYBEAgACgCECECIAAoAgwgASAANgIcIAEgADYCECABQRc2AhggAUEcNgIUIAEgASkCFDcDCCACIAFBCGoQHA0BCyAAEBYLIAFBIGokAAuCAQECfyAAKAIIRQRAIABBEGoQLQJAIABBKGoiACgCAEUNACAAKAIMRQ0AIABBDGoiAUGAgICAeP4zAgAaIABBCGoiAkEB/h4CABogAkH/////BxAPIAAoAgwiAEH/////B3FFDQADQCABQQAgABATIAEoAgAiAEH/////B3ENAAsLCwtlAQJ/IAAgACgCWEYEQCAAQgA3AlhBwAsoAgBBABAgDwsjAygCSEHACygCACIBQQJ0aigCACICIABGBEAgASACKAJYECALIAAoAlwiAiAAKAJYIgE2AlggASACNgJcIABCADcCWAueCQEOfwJAIAAgARA2IgJFBEAgACgCHCICIAAoAiBGBEACfyACQQF0QQEgAhsiD0ECdCEJIAAoAhgiDEUEQCAJEBEMAQsgCUFATwRAIwFBMDYCAEEADAELAkBBtBktAABBAnEEQEG4GRAODQELAn9BECAJQQtqQXhxIAlBC0kbIQZBACECIAxBCGsiBCgCBCIKQXhxIQgCQCAKQQNxRQRAIAZBgAJJDQEgBkEEaiAITQRAIAQhAiAIIAZrQegVKAIAQQF0TQ0CC0EADAILIAQgCGohBwJAIAYgCE0EQCAIIAZrIgNBEEkNASAEIAYgCkEBcXJBAnI2AgQgBCAGaiICIANBA3I2AgQgByAHKAIEQQFyNgIEIAIgAxA/DAELQZAWKAIAIAdGBEBBhBYoAgAgCGoiCCAGTQ0CIAQgBiAKQQFxckECcjYCBCAEIAZqIgMgCCAGayICQQFyNgIEQYQWIAI2AgBBkBYgAzYCAAwBC0GMFigCACAHRgRAQYAWKAIAIAhqIgMgBkkNAgJAIAMgBmsiAkEQTwRAIAQgBiAKQQFxckECcjYCBCAEIAZqIgggAkEBcjYCBCADIARqIgMgAjYCACADIAMoAgRBfnE2AgQMAQsgBCAKQQFxIANyQQJyNgIEIAMgBGoiAiACKAIEQQFyNgIEQQAhAkEAIQgLQYwWIAg2AgBBgBYgAjYCAAwBCyAHKAIEIgNBAnENASADQXhxIAhqIg0gBkkNASANIAZrIQ4gBygCDCEFAkAgA0H/AU0EQCAHKAIIIgIgBUYEQEH4FUH4FSgCAEF+IANBA3Z3cTYCAAwCCyACIAU2AgwgBSACNgIIDAELIAcoAhghCwJAIAUgB0cEQCAHKAIIIgIgBTYCDCAFIAI2AggMAQsCQCAHKAIUIgIEfyAHQRRqBSAHKAIQIgJFDQEgB0EQagshCANAIAghAyACIgVBFGohCCACKAIUIgINACAFQRBqIQggBSgCECICDQALIANBADYCAAwBC0EAIQULIAtFDQACQCAHKAIcIgNBAnQiAigCqBggB0YEQCACQagYaiAFNgIAIAUNAUH8FUH8FSgCAEF+IAN3cTYCAAwCCwJAIAcgCygCEEYEQCALIAU2AhAMAQsgCyAFNgIUCyAFRQ0BCyAFIAs2AhggBygCECICBEAgBSACNgIQIAIgBTYCGAsgBygCFCICRQ0AIAUgAjYCFCACIAU2AhgLIA5BD00EQCAEIApBAXEgDXJBAnI2AgQgBCANaiICIAIoAgRBAXI2AgQMAQsgBCAGIApBAXFyQQJyNgIEIAQgBmoiAyAOQQNyNgIEIAQgDWoiAiACKAIEQQFyNgIEIAMgDhA/CyAEIQILIAILIQJBtBktAABBAnEEQEG4GRANCyACQQhqIAINARpBACAJEBEiBEUNARogBCAMQXxBeCAMQQRrKAIAIgJBA3EbIAJBeHFqIgIgCSACIAlJGxAXIAwQEAsgBAsiAkUNAiAAIA82AiAgACACNgIYCyABEDsiAkUNASAAIAAoAhwiAUEBajYCHCAAKAIYIAFBAnRqIAI2AgALIAIPC0EAC0oBA38CQCAAKAIcIgJBAEwNACAAKAIYIQNBACEAA0AgASADIABBAnRqKAIAIgQoAhxHBEAgAiAAQQFqIgBHDQEMAgsLIAQPC0EAC/wBAQV/AkAgACgCLCAAKAIwQQFqIAAoAigiAm9HDQAgAkEYbBARIgMEfyACQQF0IQUCQCAAKAIwIgQgACgCLCICTgRAIAMgACgCJCACQQxsaiAEIAJrIgJBDGwQFwwBCyADIAAoAiQgAkEMbGogACgCKCACayICQQxsIgYQFyADIAZqIAAoAiQgBEEMbBAXIAIgBGohAgsgACgCJBAQIAAgAjYCMCAAQQA2AiwgACAFNgIoIAAgAzYCJEEBBUEACw0AQQAPCyAAKAIkIAAoAjBBDGxqIgMgASkCADcCACADIAEoAgg2AgggACAAKAIwQQFqIAAoAihvNgIwQQELZQEDfyMAQRBrIgEkACAAQQRqIgIQDhogACgCLCAAKAIwRwRAA0AgAUEEaiAAEDkgASgCCCIDBEAgASgCDCADEQAACyAAKAIsIAAoAjBHDQALCyACEA0gAEEA/hcCACABQRBqJAALOAECfyAAIAEoAiQgASgCLCICQQxsaiIDKQIANwIAIAAgAygCCDYCCCABIAJBAWogASgCKG82AiwLFAAgAEEEahAtIAAoAiQQECAAEBAL1wIBBn8jAEFAaiIBJABBjAgQGEUEQEHACCgCACICQYgIRwRAA0AgAigCOCAC/hACAEUEQCACKAI0IgQgAigCOCIFNgI4IAUgBDYCNCACEDoLIgJBiAhHDQALC0GMCBANCwJAQTwQESICRQ0AQYAMEBEiA0UEQCACEBAMAQsgAUIANwMoIAFCADcDMCABQQA2AjwgAUIANwMgIAEgADYCHCABQQA2AhggASADNgIUIAFBgAE2AhAgAUEANgIMIAFBADYCCCABQQA2AgQgAUEANgIAIAIgASgCPDYCACACIAEpAzA3AhQgAiABKQMoNwIMIAIgASkDIDcCBCACIAEoAhw2AhwgAiABKAIYNgIgIAIgASgCFDYCJCACIAEoAhA2AiggAiABKAIMNgIsIAIgASgCCDYCMCACIAEoAgQ2AjQgAiABKAIANgI4IAIhBgsgAUFAayQAIAYLwAcCFH8LewJAIAEgAk4NACADIARODQAgAiABayIIQQNxIQ0gBCADa0ECdCEJIAAoAhQiB0ECdCEMIAAoAgggASAHbCADakECdGohCiABIAJrQXxNBEAgCEF8cSEHQQAhCANAIAlFIgtFBEAgCiAFIAxsakEAIAn8CwALIAtFBEAgCiAMIAVBAXJsakEAIAn8CwALIAtFBEAgCiAMIAVBAnJsakEAIAn8CwALIAtFBEAgCiAMIAVBA3JsakEAIAn8CwALIAVBBGohBSAIQQRqIgggB0cNAAsLIA1FDQADQCAJBEAgCiAFIAxsakEAIAn8CwALIAVBAWohBSAGQQFqIgYgDUcNAAsLAkAgACgCICIFQQBMDQAgBCADayIGQQhtQQN0IANqIQ4gAiABayIIQQRtQQJ0IAFqIRNBACEHIAZBCE4EQCAIQQNKIRYDQCAFIAdBgAJqIgggBSAHa0GAAkgbIQ8gAyENIBYEQANAIAEhCwNAIA1BAnQiCiAAKAIIIAAoAhQiBiALbEECdGpqIhD9AAAQIRogEP0AAAAhGyAQIAZBAnQiBmoiEf0AABAhHCAR/QAAACEdIAYgEWoiEv0AABAhHiAS/QAAACEfIAYgEmoiFP0AABAhICAU/QAAACEhIAcgD0gEQCAAKAIAIAAoAgwiBiALbEECdGoiFyAGQQJ0IgZqIhggBmoiCSAGaiEMIAogACgCBCAAKAIQIgogB2xBAnRqaiEFIAchBgNAIAwgBkECdCIVav0JAgAiGSAF/QAAECIi/eYBICD95AEhICAZIAX9AAAAIiP95gEgIf3kASEhIAkgFWr9CQIAIhkgIv3mASAe/eQBIR4gGSAj/eYBIB/95AEhHyAVIBhq/QkCACIZICL95gEgHP3kASEcIBkgI/3mASAd/eQBIR0gFSAXav0JAgAiGSAi/eYBIBr95AEhGiAZICP95gEgG/3kASEbIAUgCkECdGohBSAGQQFqIgYgD0cNAAsLIBAgGv0LABAgECAb/QsAACARIBz9CwAQIBEgHf0LAAAgEiAe/QsAECASIB/9CwAAIBQgIP0LABAgFCAh/QsAACALQQRqIgsgE0gNAAsgDUEIaiINIA5IDQALCyAAIBMgAiADIA4gByAPEB0gACABIAIgDiAEIAcgDxAdIAAoAiAiBSAIIgdKDQALDAELA0AgACATIAIgAyAOIAcgBSAHQYACaiIIIAUgB2tBgAJIGyIGEB0gACABIAIgDiAEIAcgBhAdIAAoAiAiBSAIIgdKDQALCwu3CQMMfw17A30jAEFAaiEGIAAtAAEiBUEHdiECIAAtAANBCHQiByAALQACciEEIAVBCHQiCCAALQAAckH/B3EhAwJ9AkAgBUECdkEfcSIFQR9HBEAgBQ0BIAJBH3S+IANFDQIaIAOzQwAAgDOUIhuMIBsgCMFBAEgbDAILIAJBH3RBgICAAkEAIAMbckGAgID8B3K+DAELIAVBF3QgAkEfdHIgA0ENdHJBgICAwANqvgshHSAEQf8HcSEDIARBgIACcUEPdiECAn0CQCAEQQp2QR9xIgRBH0cEQCAEDQEgAkEfdL4gA0UNAhogA7NDAACAM5QiG4wgGyAHwUEASBsMAgsgAkEfdEGAgIACQQAgAxtyQYCAgPwHcr4MAQsgBEEXdCACQR90ciADQQ10ckGAgIDAA2q+CyEbIAYgAP1cAAQiDv0MPz8/Pz8/Pz8/Pz8/Pz8/P/1O/YkB/akB/QsEICAGIAD9XAAIIg/9DD8/Pz8/Pz8/Pz8/Pz8/Pz/9Tv2JAf2pAf0LBAAgBiAA/VwADCIQQQT9bSAPQQL9bf0MMDAwMDAwMDAwMDAwMDAwMP1O/VD9iQH9qQH9CwQQIAYgEP0MDw8PDw8PDw8PDw8PDw8PD/1OIA5BAv1t/QwwMDAwMDAwMDAwMDAwMDAw/U79UP2JAf2pAf0LBDAgAEEwaiEIIABBEGohCUEAIQQDQCABIARBCHRqIQogCCAEQQV0aiELIAEgBEEBdCIFQQFyIgdBB3RqIQz9DAAAAAAAAAAAAAAAAAAAAAAiDiEP/QwAAAAAAAAAAAAAAAAAAAAAIRD9DAAAAAAAAAAAAAAAAAAAAAAhEUEBIQNBACEAA0AgDCAAQQJ0Ig1qIgL9AAAwIhYgACAJav0AAAAiEiAH/W1BBP1r/QwQEBAQEBAQEBAQEBAQEBAQ/U4gACALav0AAAAiE0EE/W39UCIU/YoBIhX9qgH9+gH95gEgAv0AACAiFyAV/akB/foB/eYBIAL9AAAQIhUgFP2JASIU/aoB/foB/eYBIAL9AAAAIhggFP2pAf36Af3mASAQ/eQB/eQB/eQB/eQBIRAgCiANaiIA/QAAMCIUIBIgBf1tQQT9a/0MEBAQEBAQEBAQEBAQEBAQEP1OIBP9DA8PDw8PDw8PDw8PDw8PDw/9Tv1QIhL9igEiE/2qAf36Af3mASAA/QAAICIZIBP9qQH9+gH95gEgAP0AABAiEyAS/YkBIhL9qgH9+gH95gEgAP0AAAAiGiAS/akB/foB/eYBIA795AH95AH95AH95AEhDiAWIBcgFSAYIBH95AH95AH95AH95AEhESAUIBkgEyAaIA/95AH95AH95AH95AEhDyADQRAhAEEAIQMNAAsgEP0fACAQ/R8BkiAQ/R8CkiAQ/R8DkiAHQQJ0IgAgBkEgaiICaigCALKUIA79HwAgDv0fAZIgDv0fApIgDv0fA5IgAiAFQQJ0IgNqKAIAspSSIB2UIBySIBH9HwAgEf0fAZIgEf0fApIgEf0fA5IgG5QgACAGaigCALKUIA/9HwAgD/0fAZIgD/0fApIgD/0fA5IgG5QgAyAGaigCALKUkpMhHCAEQQFqIgRBBEcNAAsgHAvVBwQFewd/An0CfiMAQUBqIQwgAC0AASIKQQd2IQsgAC0AA0EIdCINIAAtAAJyIQggCkEIdCIOIAAtAAByQf8HcSEJAn0CQCAKQQJ2QR9xIgpBH0cEQCAKDQEgC0EfdL4gCUUNAhogCbNDAACAM5QiD4wgDyAOwUEASBsMAgsgC0EfdEGAgIACQQAgCRtyQYCAgPwHcr4MAQsgCkEXdCALQR90ciAJQQ10ckGAgIDAA2q+CyAIQf8HcSEJIAhBgIACcUEPdiELAn0CQCAIQQp2QR9xIghBH0cEQCAIDQEgC0EfdL4gCUUNAhogCbNDAACAM5QiEIwgECANwUEASBsMAgsgC0EfdEGAgIACQQAgCRtyQYCAgPwHcr4MAQsgCEEXdCALQR90ciAJQQ10ckGAgIDAA2q+CyEQIAwgAP1cAAQiA/0MPz8/Pz8/Pz8/Pz8/Pz8/P/1O/YkB/akB/QsEICAMIAD9XAAIIgT9DD8/Pz8/Pz8/Pz8/Pz8/Pz/9Tv2JAf2pAf0LBAAgDCAA/VwADCIFQQT9bSAEQQL9bf0MMDAwMDAwMDAwMDAwMDAwMP1O/VD9iQH9qQH9CwQQIAwgBf0MDw8PDw8PDw8PDw8PDw8PD/1OIANBAv1t/QwwMDAwMDAwMDAwMDAwMDAw/U79UP2JAf2pAf0LBDAgAEEQaiEJQQAhAANAIABBA3QiCyAMQSBqIg1qNAIAIAkgAEEFdGoiCP0AAAAiBv0MDw8PDw8PDw8PDw8PDw8PD/1OIgP9igEgASAAQQZ0aiIK/QAAACIE/YgB/boBIAP9iQEgBP2HAf26Af2uASAI/QAAECIH/QwPDw8PDw8PDw8PDw8PDw8P/U4iA/2KASAK/QAAECIF/YgB/boB/a4BIAP9iQEgBf2HAf26Af2uASID/RsAIAP9GwFqIAP9GwJqIAP9GwNqrH4gEXwgBkEE/W0iA/2KASABIABBAXRBAXIiCEEFdGoiCv0AAAAiBv2IAf26ASAD/YkBIAb9hwH9ugH9rgEgB0EE/W0iA/2KASAK/QAAECIH/YgB/boB/a4BIAP9iQEgB/2HAf26Af2uASID/RsAIAP9GwFqIAP9GwJqIAP9GwNqrCANIAhBAnQiCGo0AgB+fCERIAsgDGo0AgAgBf18/X4gBP18/X79rgEiA/0bACAD/RsBaiAD/RsCaiAD/RsDaqx+IBJ8IAggDGo0AgAgB/18/X4gBv18/X79rgEiA/0bACAD/RsBaiAD/RsCaiAD/RsDaqx+fCESIABBAWoiAEEERw0ACyARtJQgECAStJSTIAKUC40LAQd/IAAgAWohBQJAAkAgACgCBCICQQFxDQAgAkECcUUNASAAKAIAIgIgAWohAQJAAkACQCAAIAJrIgBBjBYoAgBHBEAgACgCDCEDIAJB/wFNBEAgAyAAKAIIIgRHDQJB+BVB+BUoAgBBfiACQQN2d3E2AgAMBQsgACgCGCEGIAAgA0cEQCAAKAIIIgIgAzYCDCADIAI2AggMBAsgACgCFCIEBH8gAEEUagUgACgCECIERQ0DIABBEGoLIQIDQCACIQcgBCIDQRRqIQIgAygCFCIEDQAgA0EQaiECIAMoAhAiBA0ACyAHQQA2AgAMAwsgBSgCBCICQQNxQQNHDQNBgBYgATYCACAFIAJBfnE2AgQgACABQQFyNgIEIAUgATYCAA8LIAQgAzYCDCADIAQ2AggMAgtBACEDCyAGRQ0AAkAgACgCHCICQQJ0IgQoAqgYIABGBEAgBEGoGGogAzYCACADDQFB/BVB/BUoAgBBfiACd3E2AgAMAgsCQCAAIAYoAhBGBEAgBiADNgIQDAELIAYgAzYCFAsgA0UNAQsgAyAGNgIYIAAoAhAiAgRAIAMgAjYCECACIAM2AhgLIAAoAhQiAkUNACADIAI2AhQgAiADNgIYCwJAAkACQAJAIAUoAgQiAkECcUUEQEGQFigCACAFRgRAQZAWIAA2AgBBhBZBhBYoAgAgAWoiATYCACAAIAFBAXI2AgQgAEGMFigCAEcNBkGAFkEANgIAQYwWQQA2AgAPC0GMFigCACIIIAVGBEBBjBYgADYCAEGAFkGAFigCACABaiIBNgIAIAAgAUEBcjYCBCAAIAFqIAE2AgAPCyACQXhxIAFqIQEgBSgCDCEDIAJB/wFNBEAgBSgCCCIEIANGBEBB+BVB+BUoAgBBfiACQQN2d3E2AgAMBQsgBCADNgIMIAMgBDYCCAwECyAFKAIYIQYgAyAFRwRAIAUoAggiAiADNgIMIAMgAjYCCAwDCyAFKAIUIgQEfyAFQRRqBSAFKAIQIgRFDQIgBUEQagshAgNAIAIhByAEIgNBFGohAiADKAIUIgQNACADQRBqIQIgAygCECIEDQALIAdBADYCAAwCCyAFIAJBfnE2AgQgACABQQFyNgIEIAAgAWogATYCAAwDC0EAIQMLIAZFDQACQCAFKAIcIgJBAnQiBCgCqBggBUYEQCAEQagYaiADNgIAIAMNAUH8FUH8FSgCAEF+IAJ3cTYCAAwCCwJAIAUgBigCEEYEQCAGIAM2AhAMAQsgBiADNgIUCyADRQ0BCyADIAY2AhggBSgCECICBEAgAyACNgIQIAIgAzYCGAsgBSgCFCICRQ0AIAMgAjYCFCACIAM2AhgLIAAgAUEBcjYCBCAAIAFqIAE2AgAgACAIRw0AQYAWIAE2AgAPCyABQf8BTQRAIAFB+AFxQaAWaiECAn9B+BUoAgAiA0EBIAFBA3Z0IgFxRQRAQfgVIAEgA3I2AgAgAgwBCyACKAIICyEBIAIgADYCCCABIAA2AgwgACACNgIMIAAgATYCCA8LQR8hAyABQf///wdNBEAgAUEmIAFBCHZnIgJrdkEBcSACQQF0a0E+aiEDCyAAIAM2AhwgAEIANwIQIANBAnRBqBhqIQICQAJAQfwVKAIAIgRBASADdCIHcUUEQEH8FSAEIAdyNgIAIAIgADYCACAAIAI2AhgMAQsgAUEZIANBAXZrQQAgA0EfRxt0IQMgAigCACECA0AgAiIEKAIEQXhxIAFGDQIgA0EddiECIANBAXQhAyAEIAJBBHFqIgcoAhAiAg0ACyAHIAA2AhAgACAENgIYCyAAIAA2AgwgACAANgIIDwsgBCgCCCIBIAA2AgwgBCAANgIIIABBADYCGCAAIAQ2AgwgACABNgIICwu7AQEDfyMAQRBrIgEkAEHoGRAOGkHgFSgCAEUEQEH0FUECNgIAQewVQn83AgBB5BVCgKCAgICABDcCAEG0GUECNgIAIAFBDGoiAkEANgIAIwBBIGsiAEIANwMYIABCADcDECAAQgA3AwhBuBkgACkDCDcCAEHIGSAAKQMYNwIAQcAZIAApAxA3AgAgAgRAQbgZIAIoAgA2AgALQeAVIAFBCGpBcHFB2KrVqgVzNgIAC0HoGRANIAFBEGokAAscACAAIAAQOzYCeCAAQQH+FwJ8IABBAP4XAoABCxoAIABBAP5BAgBBA0YEQCAAQf////8HEA8LCwIAC+gEAQZ/IwMiAUEBOgAoIAEgADYCQCABQQA6ACkgAUEB/iUCfEEBayIABEAgAUH8AGohAgNAIAIgAEQAAAAAAADwfxAVGiAC/hACACIADQALCyABKAJ4EDgCQCABKAJ4IgD+EAIARQRAIAAQOgwBC0GMCBAOGiAAQYgINgI4IABBvAgoAgA2AjRBvAggADYCACAAKAI0IAA2AjhBjAgQDQsjAyECA0AgAigCRCIABEAgACgCBCAAKAIAIAIgACgCCDYCRBEAAAwBCwsCQCMDIgAtACpBAXFFDQADQBArIAAgAC0AKkH+AXE6ACpBACECA0AgAkECdCIDKALgESEEIAAoAkggA2oiBigCACEDIAZBADYCAAJAIANFDQAgBEUNACAEQSBGDQAQKSADIAQRAAAQKwsgAkEBaiICQYABRw0ACxApIAAtACpBAXFFDQEgBUEDSSAFQQFqIQUNAAsLECRBAEEB/h4ClA0aAkAgASgCTCIARQ0AIAAgAUHMAGoiAkYNAANAIABBCGsoAgAgASAANgJUIAEgACgCADYCTCAAQQxrIgBBgICAgAT+QQIAIQMgAUEANgJURSADQQBOcUUEQCAAQQEQDwsgAigCACIARQ0BIAAgAkcNAAsLEC5BzAtBzAsoAgBBAWsiADYCACAARQRAQcsLQQA6AAALIAEoAgwiACABKAIIIgI2AgggAiAANgIMIAEgATYCCCABIAE2AgwQIyMFRQRAQQAkA0EAJARBACQFQQEkBiABQSBqIgBBAkEB/kgCAEEDRgRAIAEQAQ8LIABBAP4XAgAgAEEBEA8PC0EAEAQAC7wLAxJ/AnsBfSMAQYAQayIMJAACQCAAKAIYIgZBAEwNACABIAJODQAgACgCCCABQQJ0aiEFIAIgAWtBAnQhBCAAKAIUQQJ0IQggBkEETwRAIAZB/P///wdxIQsDQCAERSIHRQRAIAUgAyAIbGpBACAE/AsACyAHRQRAIAUgCCADQQFybGpBACAE/AsACyAHRQRAIAUgCCADQQJybGpBACAE/AsACyAHRQRAIAUgCCADQQNybGpBACAE/AsACyADQQRqIQMgCUEEaiIJIAtHDQALCyAGQQNxIgdFDQADQCAEBEAgBSADIAhsakEAIAT8CwALIANBAWohAyAKQQFqIgogB0cNAAsLAkAgACgCICILQQBMDQAgBkEBRgRAIAFBAXQhDSAAKAIIIAFBAnRqIQcgACgCACEOQQAhCCACIAFrIgpBCEghDwNAIAAoAgQgACgCECAIbGogDWohCSAOIAhBAnRqKgIAIRdBACEFIA9FBEAgF/0TIRVBACEEQQghBgNAIAcgBEECdGoiAyAVIAkgBEEBdGr9AAAAIhYgFf0NAAEAAQIDAAEEBQABBgcAAUEQ/asB/eYBIAP9AAAA/eQB/QsAACADIBUgFiAV/Q0ICQABCgsAAQwNAAEODwABQRD9qwH95gEgA/0AABD95AH9CwAQIAYiBSIEQQhqIgYgCkwNAAsLAkAgBSAKTg0AAkAgAiABIAVqayIGQQRJBEAgBSEDDAELIAUgBkF8cSILaiEDIBf9EyEVQQAhBANAIAcgBCAFaiIQQQJ0aiIRIBUgCSAQQQF0av0EAABBEP2rAf3mASAR/QACAP3kAf0LAgAgBEEEaiIEIAtHDQALIAYgC0YNAQsDQCAHIANBAnRqIgQgFyAJIANBAXRqLwAAQRB0vpQgBCoCAJI4AgAgA0EBaiIDIApHDQALCyAIQQFqIgggACgCIEgNAAsMAQsgASACTg0AIAIgAWshEANAIAtBAEoEQEGABCACIAFrIg4gDkGABE4bIQdBgAQgECANQQl0ayIDIANBgAROGyIPQQNxIQogAUEBdCERQQAhCCAOQQdKIRQDQCAAKAIEIAAoAhAgCGxqIBFqIQlBCCEEQQAhAyAUBEADQCAMIANBAnRqIgUgCSADQQF0av0AAAAiFSAV/Q0ICQABCgsAAQwNAAEODwABQRD9qwH9CwQQIAUgFSAV/Q0AAQABAgMAAQQFAAEGBwABQRD9qwH9CwQAIAQiA0EIaiIEIAdMDQALCwJAIAMgB04NAAJAIA8gA2siBEEESQRAIAMhBAwBCyADIAQgCmsiEmohBEEAIQUDQCAMIAMgBWoiE0ECdGogCSATQQF0av0EAABBEP2rAf0LAgAgBUEEaiIFIBJHDQALIApFDQELA0AgDCAEQQJ0aiAJIARBAXRqLwAAQRB0NgIAIARBAWoiBCAHRw0ACwtBACEJIAZBAEoEQANAIAAoAgggACgCFCAJbEECdGogAUECdGohBiAAKAIAIAAoAgwgCWxBAnRqIAhBAnRqKgIAIRdBACEDIA5BBE4EQCAX/RMhFUEAIQRBBCEFA0AgBiAEQQJ0IgNqIgQgAyAMav0ABAAgFf3mASAE/QAAAP3kAf0LAAAgBSIDIQQgA0EEaiIFIAdMDQALCwJAIAMgB04NAAJAIA8gA2siBEEESQRAIAMhBAwBCyADIAQgCmsiC2ohBCAX/RMhFUEAIQUDQCAGIAMgBWpBAnQiEmoiEyAT/QACACAMIBJq/QACACAV/eYB/eQB/QsCACAFQQRqIgUgC0cNAAsgCkUNAQsDQCAGIARBAnQiA2oiBSAFKgIAIAMgDGoqAgAgF5SSOAIAIARBAWoiBCAHRw0ACwsgCUEBaiIJIAAoAhgiBkgNAAsgACgCICELCyAIQQFqIgggC0gNAAsLIA1BAWohDSABQYAEaiIBIAJIDQALCyAMQYAQaiQACxIAIAAoAiwiAEGEARAvIAAQEAuGAQEBfyMAQTBrIgwkAAJAIAlBAEwNACAKQQBMDQAgDCAKNgIoIAwgCTYCJCAMIAg2AiAgDCAFNgIcIAwgAjYCGCAMIAMgBGo2AhAgDCALQQAgC0EAShs2AiwgDCAGIAdBAnRqNgIUIAwgACABQQJ0ajYCDEEDIAxBDGogChASCyAMQTBqJAALCwBBhA0jA/4XAgALJQECf0HACygCAEEAECAgACEBA0AgASgCWCABEDIiASAARw0ACws4ACAAIAAoAgAgACgCBCAAKAIIIAAoAgwgACgCEBAKOQMYIAAtACBBAUYEQCAAKAIQEBAgABAQCwulBAIBfwF8IwBBMGsiBSQAIAUgATYCDCAFIAA2AgggBUEANgAsIAVBADYAKSAFQQA6ACggBUIANwMgIAVBADYCHCAFIAM2AhggBSACNgIUIAUjAzYCEAJ8IAQEQCMAQRBrIgMkACADIAVBCGo2AgwgA0EANgIIIANBHzYCBCMAQYABayIAJAAjAEHgAGsiASQAQcQLQRsQLCABQQBB0AD8CwAgAUEeNgJcIAEgA0EEajYCWCABQQA2AlQgAUEANgJQIABBIGoiAiABKAJcNgIAIAIgASgCWDYCBCACIAEoAlQ2AgggAiABKAJQNgIMIAJBEGogAUHQAPwKAAAgAUHgAGokACAAQRk2AhggAEEaNgIUIAAgAjYCHCAAIAI2AhAgACAAKQIUNwMIQZwLQYAMIABBCGoQHAR/IABBMGoiARAOGiAAKAIsRQRAIABByABqIQIDQCACIAEQJSAAKAIsRQ0ACwsgARANIAAoAixBAUYFQQALIQEgAEEgahAzIABBgAFqJAAgA0EQaiQAIAUrAyBEAAAAAAAAAAAgARsMAQtBKBARIgEgBUEIakEo/AoAACABQQE6ACAgASACQQN0IgAQESICNgIQIAIgAyAAEBcjAEEgayIAJAAgACABNgIcIAAgATYCECAAQQA2AhggAEEfNgIUIAAgACkCFDcDCEGcC0GADCAAQQhqEBwaIABBIGokAEQAAAAAAAAAAAsgBUEwaiQAC5QBAQJ/IAEoAgggASgCABEAACMAQSBrIgEkAAJAIAAoAghFBEAgAEEQaiICEA4aIABBATYCDCAAEDQgAhANIABBKGoQKAwBCyAAEDQgACgCECECIAAoAgwgASAANgIcIAEgADYCECABQRc2AhggAUEYNgIUIAEgASkCFDcDCCACIAFBCGoQHA0AIAAQFgsgAUEgaiQACxMAIAAoAgQgACgCGBEAACAAEBYLuAYDEn8BewF9AkAgACgCDCIKQQBMDQAgASACTg0AIAAoAhAhCyAAKAIIIQwgACgCACENIAAoAgQhDiAAKAIUIgNBBE4EQCADQfz///8HcSEFIANBBGsiD0EEcSERIA9BAnZBAWpB/v///wdxIRIDQCAMIAYgC2xBAnRqIRMgDSADIAZsQQJ0aiEHIAEhBANAIA4gAyAEbEECdGohCUEEIQD9DAAAAAAAAAAAAAAAAAAAAAAhFUEAIQhBACEQIA9BBE8EQANAIAkgAEECdCIUav0AAAAgByAUav0AAAD95gEgCSAIQQJ0Ighq/QAAACAHIAhq/QAAAP3mASAV/eQB/eQBIRUgAEEEaiEIIABBCGohACAQQQJqIhAgEkcNAAsLIBFFBEAgCSAIQQJ0IgBq/QAAACAAIAdq/QAAAP3mASAV/eQBIRULIBX9HwAgFf0fAZIgFf0fApIgFf0fA5IhFiADIAUiAEoEQANAIAkgAEECdCIIaioCACAHIAhqKgIAlCAWkiEWIABBAWoiACADRw0ACwsgEyAEQQJ0aiAWOAIAIARBAWoiBCACRw0ACyAGQQFqIgYgCkcNAAsMAQsgA0EASgRAIANBAUYhBwNAIAwgBCALbEECdGohCSANIAMgBGxBAnRqIQUgASEAA0AgCSAAQQJ0agJ9IA4gACADbEECdGoiBioCACAFKgIAlCIWIAcNABogBioCBCAFKgIElCAWkiIWIANBAkYNABogBioCCCAFKgIIlCAWkgs4AgAgAEEBaiIAIAJHDQALIARBAWoiBCAKRw0ACwwBCyALQQJ0IQUgDCABQQJ0aiEDIAIgAWtBAnQhAUEAIQAgCkEETwRAIApB/P///wdxIQcDQCABRSICRQRAIAMgACAFbGpBACAB/AsACyACRQRAIAMgBSAAQQFybGpBACAB/AsACyACRQRAIAMgBSAAQQJybGpBACAB/AsACyACRQRAIAMgBSAAQQNybGpBACAB/AsACyAAQQRqIQAgBkEEaiIGIAdHDQALCyAKQQNxIgJFDQADQCABBEAgAyAAIAVsakEAIAH8CwALIABBAWohACAEQQFqIgQgAkcNAAsLC/YCAQN/IwMiACgCSEUEQCAAQbANNgJICwJAQbwRKAIAIwMoAhhGDQBBsBFBAEH/////B/5IAgAEf0EKBUG8ESMDKAIYNgIAQQALQQpHDQBB5AAhAANAAkAgAEUNAEGwESgCAEUNACAAQQFrIQBBtBEoAgBFDQELC0GwEUEAQf////8H/kgCAAR/QQoFQbwRIwMoAhg2AgBBAAtBCkYEQANAAkBBsBEoAgAiAEUNAEG0EUEB/h4CABpBsBEgACAAQYCAgIB4ciIA/kgCABpBuBEoAgAaQbARIAAQJiEAQbQRQQH+JQIAGiAARQ0AIABBG0cNAwtBsBFBAEH/////B/5IAgAEf0EKBUG8ESMDKAIYNgIAQQALQQpGDQALC0G8ESMDKAIYNgIAC0HQESgCACIBIQADQAJAIABBAnQiAigC4BFFBEBBwAsgADYCAEHQESAANgIAIAJB4BFqQR02AgAMAQsgAEEBakH/AHEiACABRw0BCwsQKQtqAQF/AkAjAygCSEHACygCAEECdGooAgAiAUUEQCAAIAA2AlggACAANgJcQcALKAIAIAAQIAwBCyAAIAE2AlggACABKAJcNgJcIAEgADYCXCAAKAJcIAA2AlgLIAAgACgCBCAAKAIAEQQACxMAIAAoAgQgACgCFBEAACAAEBYLmwEBAX8jAEEgayIJJAACQCAGQQBMDQAgB0EATA0AIAhBAEwEQCAGIAdsIgBBAEwNASAAQQJ0IgBFDQEgBCAFQQJ0akEAIAD8CwAMAQsgCSAINgIcIAkgBzYCGCAJIAY2AhQgCSAEIAVBAnRqNgIQIAkgAiADQQJ0ajYCDCAJIAAgAUECdGo2AghBAiAJQQhqIAcQEgsgCUEgaiQACxkAIABBAf4XAgAgABAnIABBAUEA/kgCABoLBgAgABA4C04BA38jASEAIwMiAigCdCIBBEAgAkEANgJ0IAEkASABQQBBBfwLACABDwsjAkEBIAAbBEBBASQCQQUQESEACyAAJAEgAEEAQQX8CwAgAAvOAgEHfwNAQQAhAAJAA0BBgAv+EAIAIAFHDQFBgAv+EAIAIAFHDQFBgAv+EAIAIAFHDQFBgAv+EAIAIAFHDQEgAEEEaiIAQYCAwABHDQALQfQJEA4aQQBBAf4eAogLGiABQYAL/hACAEYEQANAQYwKQfQJECVBgAv+EAIAIAFGDQALC0EAQQH+JQKICxpB9AkQDQtBgAv+EAIAIQFB+AooAgAhAkHwCigCACEFQewKKAIAIQZB9AooAgAhAyADQfwK/hACACIASgRAIAJBAXQhBwNAIAMgAEEAIABBwAAgAyAAayAHbSICIAJBwABMGyAAaiIE/kgC/AoiAkYEfyAFIAAgBCADIAMgBEobIAYRAQBB/Ar+EAIABSACCyIASg0ACwtBAEEB/iUChAtBAUcNAEH0CRAOGkGMC/4QAgAEQEG8ChAoC0H0CRANDAALAAtwAQJ/IwBBEGsiACQAAkAgAEEMaiIBECINACAAKAIMECFBlAtBlAsoAgBBAWo2AgAgARAiDQAgACgCDBAhQZQLQZQLKAIAQQFqNgIAIAEQIg0AIAAoAgwQIUGUC0GUCygCAEEBajYCAAsgAEEQaiQAC/YNBAN7Dn8DfQF+IwBBgAJrIgYkAAJAIAEgAk4NACAAKAIMIQ0gACgCECIOQQBMBEAgAiABa0ECdCIARQ0BIA0gAUECdGpBACAA/AsADAELIAAoAgAgASAObEHSAWxqIQcgACgCCCEPIAAoAgQhEANAQwAAAAAhFEEAIQoDQCAHLQDRASIIQQd2IQkgCEEIdCILIActANABckH/B3EhACAPIApBAnRqAn0CQCAIQQJ2QR9xIggEQCAIQR9HDQEgCUEfdEGAgIACQQAgABtyQYCAgPwHcr4MAgsgAARAIACzQwAAgDOUIhWMIBUgC8FBAEgbDAILIAlBH3S+DAELIAhBF3QgCUEfdHIgAEENdHJBgICAwANqvgshFSAQIApBCHRqIQsqAgAhFiAH/QAAICEEIAYgB/0AAIABIgP9DDAwMDAwMDAwMDAwMDAwMDD9TiAH/QAAACIFQQT9bf1Q/Qzg4ODg4ODg4ODg4ODg4ODg/W79CwRAIAYgA0EC/W39DDAwMDAwMDAwMDAwMDAwMDD9TiAEQQT9bf1Q/Qzg4ODg4ODg4ODg4ODg4ODg/W79CwRgIAYgA0EC/Wv9DDAwMDAwMDAwMDAwMDAwMDD9TiAE/QwPDw8PDw8PDw8PDw8PDw8P/U79UP0M4ODg4ODg4ODg4ODg4ODg4P1u/QsEICAGIANBBP1r/QwwMDAwMDAwMDAwMDAwMDAw/U4gBf0MDw8PDw8PDw8PDw8PDw8PD/1O/VD9DODg4ODg4ODg4ODg4ODg4OD9bv0LBAAgB/0AADAhBCAGIAf9AACQASID/QwwMDAwMDAwMDAwMDAwMDAw/U4gB/0AABAiBUEE/W39UP0M4ODg4ODg4ODg4ODg4ODg4P1u/QsEUCAGIANBAv1t/QwwMDAwMDAwMDAwMDAwMDAw/U4gBEEE/W39UP0M4ODg4ODg4ODg4ODg4ODg4P1u/QsEcCAGIANBAv1r/QwwMDAwMDAwMDAwMDAwMDAw/U4gBP0MDw8PDw8PDw8PDw8PDw8PD/1O/VD9DODg4ODg4ODg4ODg4ODg4OD9bv0LBDAgBiADQQT9a/0MMDAwMDAwMDAwMDAwMDAwMP1OIAX9DA8PDw8PDw8PDw8PDw8PDw/9Tv1Q/Qzg4ODg4ODg4ODg4ODg4ODg/W79CwQQIAf9AABgIQQgBiAH/QAAoAEiA/0MMDAwMDAwMDAwMDAwMDAwMP1OIAf9AABAIgVBBP1t/VD9DODg4ODg4ODg4ODg4ODg4OD9bv0LBMABIAYgA0EC/W39DDAwMDAwMDAwMDAwMDAwMDD9TiAEQQT9bf1Q/Qzg4ODg4ODg4ODg4ODg4ODg/W79CwTgASAGIANBAv1r/QwwMDAwMDAwMDAwMDAwMDAw/U4gBP0MDw8PDw8PDw8PDw8PDw8PD/1O/VD9DODg4ODg4ODg4ODg4ODg4OD9bv0LBKABIAYgA0EE/Wv9DDAwMDAwMDAwMDAwMDAwMDD9TiAF/QwPDw8PDw8PDw8PDw8PDw8P/U79UP0M4ODg4ODg4ODg4ODg4ODg4P1u/QsEgAEgB/0AAHAhBCAGIAf9AACwASID/QwwMDAwMDAwMDAwMDAwMDAw/U4gB/0AAFAiBUEE/W39UP0M4ODg4ODg4ODg4ODg4ODg4P1u/QsE0AEgBiADQQL9bf0MMDAwMDAwMDAwMDAwMDAwMP1OIARBBP1t/VD9DODg4ODg4ODg4ODg4ODg4OD9bv0LBPABIAYgA0EC/Wv9DDAwMDAwMDAwMDAwMDAwMDD9TiAE/QwPDw8PDw8PDw8PDw8PDw8P/U79UP0M4ODg4ODg4ODg4ODg4ODg4P1u/QsEsAEgBiADQQT9a/0MMDAwMDAwMDAwMDAwMDAwMP1OIAX9DA8PDw8PDw8PDw8PDw8PDw/9Tv1Q/Qzg4ODg4ODg4ODg4ODg4ODg/W79CwSQASAHQcABaiEMQQAhAEEBIQlCACEXA0AgAEEHdCERIAwgAEEDdGohEkEAIQADQCASIABBAXRqIhMwAAAgBiAAQQV0IBFqIghq/QAEACID/YcBIAggC2r9AAAAIgT9hwH9ugEgA/2IASAE/YgB/boB/a4BIgP9GwAgA/0bAWogA/0bAmogA/0bA2qsfiAXfCATMAABIAYgCEEQciIIav0ABAAiA/2HASAIIAtq/QAAACIE/YcB/boBIAP9iAEgBP2IAf26Af2uASID/RsAIAP9GwFqIAP9GwJqIAP9GwNqrH58IRcgAEEBaiIAQQRHDQALQQEhACAJQQAhCQ0ACyAHQdIBaiEHIBUgFpQgF7SUIBSSIRQgCkEBaiIKIA5HDQALIA0gAUECdGogFDgCACABQQFqIgEgAkcNAAsLIAZBgAJqJAALFgAgACABIAIgAyAEIAUgBiAHQRIQHgv+DQQDexF/An0BfiMAQYACayIGJAACQCABIAJODQAgAiABa0ECdCIKBEAgACgCDCABQQJ0akEAIAr8CwALIAAoAhAiDkEATA0AIAAoAgwhDyAAKAIUIRAgACgCACERIAAoAgghEiAAKAIEIRMDQCATIAtBCHRqIQ0gESALIBBsIAFqQdIBbGohByASIAtBAnRqKgIAIRggASEKA0AgBy0A0QEiCEEHdiEJIAhBCHQiDCAHLQDQAXJB/wdxIQACfQJAIAhBAnZBH3EiCARAIAhBH0cNASAJQR90QYCAgAJBACAAG3JBgICA/AdyvgwCCyAABEAgALNDAACAM5QiF4wgFyAMwUEASBsMAgsgCUEfdL4MAQsgCEEXdCAJQR90ciAAQQ10ckGAgIDAA2q+CyEXIAf9AAAgIQQgBiAH/QAAgAEiA/0MMDAwMDAwMDAwMDAwMDAwMP1OIAf9AAAAIgVBBP1t/VD9DODg4ODg4ODg4ODg4ODg4OD9bv0LBEAgBiADQQL9bf0MMDAwMDAwMDAwMDAwMDAwMP1OIARBBP1t/VD9DODg4ODg4ODg4ODg4ODg4OD9bv0LBGAgBiADQQL9a/0MMDAwMDAwMDAwMDAwMDAwMP1OIAT9DA8PDw8PDw8PDw8PDw8PDw/9Tv1Q/Qzg4ODg4ODg4ODg4ODg4ODg/W79CwQgIAYgA0EE/Wv9DDAwMDAwMDAwMDAwMDAwMDD9TiAF/QwPDw8PDw8PDw8PDw8PDw8P/U79UP0M4ODg4ODg4ODg4ODg4ODg4P1u/QsEACAH/QAAMCEEIAYgB/0AAJABIgP9DDAwMDAwMDAwMDAwMDAwMDD9TiAH/QAAECIFQQT9bf1Q/Qzg4ODg4ODg4ODg4ODg4ODg/W79CwRQIAYgA0EC/W39DDAwMDAwMDAwMDAwMDAwMDD9TiAEQQT9bf1Q/Qzg4ODg4ODg4ODg4ODg4ODg/W79CwRwIAYgA0EC/Wv9DDAwMDAwMDAwMDAwMDAwMDD9TiAE/QwPDw8PDw8PDw8PDw8PDw8P/U79UP0M4ODg4ODg4ODg4ODg4ODg4P1u/QsEMCAGIANBBP1r/QwwMDAwMDAwMDAwMDAwMDAw/U4gBf0MDw8PDw8PDw8PDw8PDw8PD/1O/VD9DODg4ODg4ODg4ODg4ODg4OD9bv0LBBAgB/0AAGAhBCAGIAf9AACgASID/QwwMDAwMDAwMDAwMDAwMDAw/U4gB/0AAEAiBUEE/W39UP0M4ODg4ODg4ODg4ODg4ODg4P1u/QsEwAEgBiADQQL9bf0MMDAwMDAwMDAwMDAwMDAwMP1OIARBBP1t/VD9DODg4ODg4ODg4ODg4ODg4OD9bv0LBOABIAYgA0EC/Wv9DDAwMDAwMDAwMDAwMDAwMDD9TiAE/QwPDw8PDw8PDw8PDw8PDw8P/U79UP0M4ODg4ODg4ODg4ODg4ODg4P1u/QsEoAEgBiADQQT9a/0MMDAwMDAwMDAwMDAwMDAwMP1OIAX9DA8PDw8PDw8PDw8PDw8PDw/9Tv1Q/Qzg4ODg4ODg4ODg4ODg4ODg/W79CwSAASAH/QAAcCEEIAYgB/0AALABIgP9DDAwMDAwMDAwMDAwMDAwMDD9TiAH/QAAUCIFQQT9bf1Q/Qzg4ODg4ODg4ODg4ODg4ODg/W79CwTQASAGIANBAv1t/QwwMDAwMDAwMDAwMDAwMDAw/U4gBEEE/W39UP0M4ODg4ODg4ODg4ODg4ODg4P1u/QsE8AEgBiADQQL9a/0MMDAwMDAwMDAwMDAwMDAwMP1OIAT9DA8PDw8PDw8PDw8PDw8PDw/9Tv1Q/Qzg4ODg4ODg4ODg4ODg4ODg/W79CwSwASAGIANBBP1r/QwwMDAwMDAwMDAwMDAwMDAw/U4gBf0MDw8PDw8PDw8PDw8PDw8PD/1O/VD9DODg4ODg4ODg4ODg4ODg4OD9bv0LBJABIAdBwAFqIQxBACEAQQEhCUIAIRkDQCAAQQd0IRQgDCAAQQN0aiEVQQAhAANAIBUgAEEBdGoiFjAAACAGIABBBXQgFGoiCGr9AAQAIgP9hwEgCCANav0AAAAiBP2HAf26ASAD/YgBIAT9iAH9ugH9rgEiA/0bACAD/RsBaiAD/RsCaiAD/RsDaqx+IBl8IBYwAAEgBiAIQRByIghq/QAEACID/YcBIAggDWr9AAAAIgT9hwH9ugEgA/2IASAE/YgB/boB/a4BIgP9GwAgA/0bAWogA/0bAmogA/0bA2qsfnwhGSAAQQFqIgBBBEcNAAtBASEAIAlBACEJDQALIA8gCkECdGoiACAAKgIAIBcgGJQgGbSUkjgCACAHQdIBaiEHIApBAWoiCiACRw0ACyALQQFqIgsgDkcNAAsLIAZBgAJqJAALFgAgACABIAIgAyAEIAUgBiAHQREQHguoAQIEfwF9AkAgASACTg0AIAAoAgwiBUEASgRAIAAoAgAgASAFbEGwAWxqIQMgACgCBCEGA0BDAAAAACEHQQAhBANAIAMgBiAEQQp0ahA9IAeSIQcgA0GwAWohAyAEQQFqIgQgBUcNAAsgACgCCCABQQJ0aiAHOAIAIAFBAWoiASACRw0ACwwBCyACIAFrQQJ0IgJFDQAgACgCCCABQQJ0akEAIAL8CwALC2UBAX8jAEEgayIIJAACQCAEQQBMDQAgBUEATA0AIAggBTYCHCAIIARBCHY2AhggCCACIANqNgIMIAggBiAHQQJ0ajYCFCAIIAAgAUECdGo2AhBBECAIQQxqIAUQEgsgCEEgaiQAC68BAQl/AkAgASACTg0AIAIgAWtBAnQiAwRAIAAoAgggAUECdGpBACAD/AsACyAAKAIMIgZBAEwNACAAKAIQIQcgACgCACEIIAAoAgQhCQNAIAkgBEEKdGohCiAIIAQgB2wgAWpBsAFsaiEFIAEhAwNAIAAoAgggA0ECdGoiCyALKgIAIAUgChA9kjgCACAFQbABaiEFIANBAWoiAyACRw0ACyAEQQFqIgQgBkcNAAsLC2UBAX8jAEEgayIIJAACQCAEQQBMDQAgBUEATA0AIAggBTYCHCAIIARBCHY2AhggCCACIANqNgIMIAggBiAHQQJ0ajYCFCAIIAAgAUECdGo2AhBBDyAIQQxqIAUQEgsgCEEgaiQAC7oBAgV/AX0CQCABIAJODQAgACgCECIFQQBKBEAgACgCACABIAVsQZABbGohBCAAKAIIIQYgACgCBCEHA0BDAAAAACEIQQAhAwNAIAQgByADQQh0aiAGIANBAnRqKgIAED4gCJIhCCAEQZABaiEEIANBAWoiAyAFRw0ACyAAKAIMIAFBAnRqIAg4AgAgAUEBaiIBIAJHDQALDAELIAIgAWtBAnQiAkUNACAAKAIMIAFBAnRqQQAgAvwLAAsLFgAgACABIAIgAyAEIAUgBiAHQQ4QHgvLAQIKfwJ9AkAgASACTg0AIAIgAWtBAnQiAwRAIAAoAgwgAUECdGpBACAD/AsACyAAKAIQIgZBAEwNACAAKAIUIQcgACgCACEIIAAoAgghCSAAKAIEIQoDQCAKIARBCHRqIQsgCCAEIAdsIAFqQZABbGohBSAJIARBAnRqKgIAIQ0gASEDA0AgBSALIA0QPiEOIAAoAgwgA0ECdGoiDCAOIAwqAgCSOAIAIAVBkAFqIQUgA0EBaiIDIAJHDQALIARBAWoiBCAGRw0ACwsLKQAgACgCJARAIAAgASACQQAgACgCHBA8DwsgAEEAIAAoAhggASACEDwLFgAgACABIAIgAyAEIAUgBiAHQQ0QHguzBAMCewt/AX0CQCABIAJODQAgACgCCCENIAAoAgwiCUEATARAIAIgAWtBAnQiAEUNASANIAFBAnRqQQAgAPwLAA8LIAAoAgAgASAJbEEibGohCiAJQSJsIQ4gACgCBCEPA0BDAAAAACEQQQAhCyAKIQADQCAPIAtBB3RqIQYgAC0AASIHQRh0QYCAgIB4cSEFIAAtAAAgB0EIdHIiCEH/B3EhDCAIQf//A3EhCAJAAkAgB0ECdkEfcSIHBEAgB0EfRw0BIAhBDXQgBXJBgICA/AdyIQUMAgsgDEUNASAFQYCAgJgEciAMZyIFQRd0ayAIQQkgBUEfc2t0QQ50QYCA/wNxciEFDAELIAdBF3QgDEENdHIgBWpBgICAwANqIQULIAb9AAAAIAD9AAACIgP9hwEiBP2nAf36Af3mASAG/QAAECAE/agB/foB/eYB/eQBIAb9AAAgIAP9iAEiA/2nAf36Af3mAf3kASAG/QAAMCAD/agB/foB/eYB/eQBIAb9AABAIAD9AAASIgP9hwEiBP2nAf36Af3mAf3kASAG/QAAUCAE/agB/foB/eYB/eQBIAb9AABgIAP9iAEiA/2nAf36Af3mAf3kASAG/QAAcCAD/agB/foB/eYB/eQBIgP9HwAgA/0fAZIgA/0fApIgA/0fA5IgBb6UIBCSIRAgAEEiaiEAIAtBAWoiCyAJRw0ACyANIAFBAnRqIBA4AgAgCiAOaiEKIAFBAWoiASACRw0ACwsLgQEBAX8jAEEgayIIJAACQCAFQQBMDQAgBEEATARAIAVBAnQiAEUNASAGIAdBAnRqQQAgAPwLAAwBCyAIIAU2AhwgCCAEQQV2NgIYIAggAiADajYCDCAIIAYgB0ECdGo2AhQgCCAAIAFBAnRqNgIQQQwgCEEMaiAFEBILIAhBIGokAAuoBAICewx/AkAgASACTg0AIAIgAWtBAnQiBQRAIAAoAgggAUECdGpBACAF/AsACyAAKAIMIgxBAEwNACAAKAIIIQ0gACgCECEOIAAoAgAhDyAAKAIEIRADQCAQIApBB3RqIQUgDyAKIA5sIAFqQSJsaiEHIAEhAANAIActAAEiCEEYdEGAgICAeHEhBiAHLQAAIAhBCHRyIgtB/wdxIQkgC0H//wNxIQsCQAJAIAhBAnZBH3EiCARAIAhBH0cNASALQQ10IAZyQYCAgPwHciEGDAILIAlFDQEgBkGAgICYBHIgCWciBkEXdGsgC0EJIAZBH3NrdEEOdEGAgP8DcXIhBgwBCyAIQRd0IAlBDXRyIAZqQYCAgMADaiEGCyANIABBAnRqIgkgBf0AAAAgB/0AAAIiA/2HASIE/acB/foB/eYBIAX9AAAQIAT9qAH9+gH95gH95AEgBf0AACAgA/2IASID/acB/foB/eYB/eQBIAX9AAAwIAP9qAH9+gH95gH95AEgBf0AAEAgB/0AABIiA/2HASIE/acB/foB/eYB/eQBIAX9AABQIAT9qAH9+gH95gH95AEgBf0AAGAgA/2IASID/acB/foB/eYB/eQBIAX9AABwIAP9qAH9+gH95gH95AEiA/0fACAD/R8BkiAD/R8CkiAD/R8DkiAGvpQgCSoCAJI4AgAgB0EiaiEHIABBAWoiACACRw0ACyAKQQFqIgogDEcNAAsLC4EBAQF/IwBBIGsiCCQAAkAgBUEATA0AIARBAEwEQCAFQQJ0IgBFDQEgBiAHQQJ0akEAIAD8CwAMAQsgCCAFNgIcIAggBEEFdjYCGCAIIAIgA2o2AgwgCCAGIAdBAnRqNgIUIAggACABQQJ0ajYCEEELIAhBDGogBRASCyAIQSBqJAALqgcDD38EewJ9AkAgASACTg0AIAAoAgghDSAAKAIQIgpBAEwEQCACIAFrQQJ0IgBFDQEgDSABQQJ0akEAIAD8CwAPCyAAKAIAIAEgCmxBGGxqIQsgCkEYbCEPIAAoAgwhECAAKAIEIREDQEMAAAAAIRZBACEJIAshBgNAIAYtAAEiDkEYdEGAgICAeHEhByAGLQADQQh0IgUgBi0AAnIhACAGLQAAIA5BCHRyIgNB/wdxIQwgA0H//wNxIQQgECAJQQJ0agJAAkAgDkECdkEfcSIDBEAgA0EfRw0BIARBDXQgB3JBgICA/AdyIQcMAgsgDEUNASAHQYCAgJgEciAMZyIDQRd0ayAEQQkgA0Efc2t0QQ50QYCA/wNxciEHDAELIANBF3QgDEENdHIgB2pBgICAwANqIQcLIBEgCUEHdGohBCoCACAAQf8HcSEIIAVBEHRBgICAgHhxIQUCQAJAIABBCnZBH3EiAwRAIANBH0cNASAAQQ10IAVyQYCAgPwHciEFDAILIAhFDQEgBUGAgICYBHIgCGciA0EXdGsgAEEJIANBH3NrdEEOdEGAgP8DcXIhBQwBCyADQRd0IAhBDXRyIAVqQYCAgMADaiEFCyAFvpQgFpIgBP0AAFAgBv0JAAQiFP0MAgICAgICAgIDAwMDAwMDA/0O/QwBAgQIECBAgAECBAgQIECA/U79DAAAAAAAAAAAAAAAAAAAAAD9JP0MEBAQEBAQEBAQEBAQEBAQEP1OIAb9AAAIIhVBBP1t/VAiE/2JASIS/aoB/foB/eYBIAT9AABAIBL9qQH9+gH95gH95AEgBP0AAGAgE/2KASIS/akB/foB/eYB/eQBIAT9AABwIBL9qgH9+gH95gH95AEgBP0AABAgFf0MDw8PDw8PDw8PDw8PDw8PD/1OIBT9DAAAAAAAAAAAAQEBAQEBAQH9Dv0MAQIECBAgQIABAgQIECBAgP1O/QwAAAAAAAAAAAAAAAAAAAAA/ST9DBAQEBAQEBAQEBAQEBAQEBD9Tv1QIhP9iQEiEv2qAf36Af3mAf3kASAE/QAAACAS/akB/foB/eYB/eQBIAT9AAAgIBP9igEiEv2pAf36Af3mAf3kASAE/QAAMCAS/aoB/foB/eYB/eQBIhL9HwAgEv0fAZIgEv0fApIgEv0fA5IgB76UkiEWIAZBGGohBiAJQQFqIgkgCkcNAAsgDSABQQJ0aiAWOAIAIAsgD2ohCyABQQFqIgEgAkcNAAsLCxYAIAAgASACIAMgBCAFIAYgB0EKEB8LowcCEH8EewJAIAEgAk4NACACIAFrQQJ0IgkEQCAAKAIIIAFBAnRqQQAgCfwLAAsgACgCECILQQBMDQAgACgCCCEMIAAoAgwhDSAAKAIUIQ4gACgCACEPIAAoAgQhEANAIA0gCkECdGohESAQIApBB3RqIQYgDyAKIA5sIAFqQRhsaiEHIAEhCQNAIActAAEiBEEYdEGAgICAeHEhBSAHLQADQQh0IhIgBy0AAnIhACAHLQAAIARBCHRyIghB/wdxIQMgCEH//wNxIQgCQAJAIARBAnZBH3EiBARAIARBH0cNASAIQQ10IAVyQYCAgPwHciEFDAILIANFDQEgBUGAgICYBHIgA2ciBUEXdGsgCEEJIAVBH3NrdEEOdEGAgP8DcXIhBQwBCyAEQRd0IANBDXRyIAVqQYCAgMADaiEFCyAAQf8HcSEEIBJBEHRBgICAgHhxIQMCQAJAIABBCnZBH3EiCARAIAhBH0cNASAAQQ10IANyQYCAgPwHciEDDAILIARFDQEgA0GAgICYBHIgBGciA0EXdGsgAEEJIANBH3NrdEEOdEGAgP8DcXIhAwwBCyAIQRd0IARBDXRyIANqQYCAgMADaiEDCyAMIAlBAnRqIgAgACoCACARKgIAIAO+lJIgBv0AAFAgB/0JAAQiE/0MAgICAgICAgIDAwMDAwMDA/0O/QwBAgQIECBAgAECBAgQIECA/U79DAAAAAAAAAAAAAAAAAAAAAD9JP0MEBAQEBAQEBAQEBAQEBAQEP1OIAf9AAAIIhRBBP1t/VAiFf2JASIW/aoB/foB/eYBIAb9AABAIBb9qQH9+gH95gH95AEgBv0AAGAgFf2KASIV/akB/foB/eYB/eQBIAb9AABwIBX9qgH9+gH95gH95AEgBv0AABAgFP0MDw8PDw8PDw8PDw8PDw8PD/1OIBP9DAAAAAAAAAAAAQEBAQEBAQH9Dv0MAQIECBAgQIABAgQIECBAgP1O/QwAAAAAAAAAAAAAAAAAAAAA/ST9DBAQEBAQEBAQEBAQEBAQEBD9Tv1QIhP9iQEiFP2qAf36Af3mAf3kASAG/QAAACAU/akB/foB/eYB/eQBIAb9AAAgIBP9igEiE/2pAf36Af3mAf3kASAG/QAAMCAT/aoB/foB/eYB/eQBIhP9HwAgE/0fAZIgE/0fApIgE/0fA5IgBb6UkjgCACAHQRhqIQcgCUEBaiIJIAJHDQALIApBAWoiCiALRw0ACwsLFgAgACABIAIgAyAEIAUgBiAHQQkQHwuUBgMMfwR7An0CQCABIAJODQAgACgCCCELIAAoAhAiCEEATARAIAIgAWtBAnQiAEUNASALIAFBAnRqQQAgAPwLAA8LIAAoAgAgASAIbEEWbGohCSAIQRZsIQwgACgCDCENIAAoAgQhDgNAQwAAAAAhE0EAIQYgCSEAA0AgDiAGQQd0aiEEIAAtAAEiBUEYdEGAgICAeHEhAyAALQAAIAVBCHRyIgdB/wdxIQogB0H//wNxIQcgDSAGQQJ0aioCACEUAkACQCAFQQJ2QR9xIgUEQCAFQR9HDQEgB0ENdCADckGAgID8B3IhAwwCCyAKRQ0BIANBgICAmARyIApnIgNBF3RrIAdBCSADQR9za3RBDnRBgID/A3FyIQMMAQsgBUEXdCAKQQ10ciADakGAgIDAA2ohAwsgBP0AAFAgAP0JAAIiD/0MAgICAgICAgIDAwMDAwMDA/0O/QwBAgQIECBAgAECBAgQIECA/U79DAAAAAAAAAAAAAAAAAAAAAD9JP0MEBAQEBAQEBAQEBAQEBAQEP1OIAD9AAAGIhBBBP1t/VAiEf2JASIS/aoB/foB/eYBIAT9AABAIBL9qQH9+gH95gH95AEgBP0AAGAgEf2KASIR/akB/foB/eYB/eQBIAT9AABwIBH9qgH9+gH95gH95AEgBP0AABAgEP0MDw8PDw8PDw8PDw8PDw8PD/1OIA/9DAAAAAAAAAAAAQEBAQEBAQH9Dv0MAQIECBAgQIABAgQIECBAgP1O/QwAAAAAAAAAAAAAAAAAAAAA/ST9DBAQEBAQEBAQEBAQEBAQEBD9Tv1QIg/9iQEiEP2qAf36Af3mAf3kASAE/QAAACAQ/akB/foB/eYB/eQBIAT9AAAgIA/9igEiD/2pAf36Af3mAf3kASAE/QAAMCAP/aoB/foB/eYB/eQBIg/9HwEgFEMAAIDBlJIgD/0fAJIgD/0fApIgD/0fA5IgA76UIBOSIRMgAEEWaiEAIAZBAWoiBiAIRw0ACyALIAFBAnRqIBM4AgAgCSAMaiEJIAFBAWoiASACRw0ACwsLmwEBAX8jAEEwayIMJAACQCAJQQBMDQAgCkEATA0AIAwgCDYCHCAMIAU2AhggDCACNgIUIAwgCTYCICAMIAo2AiQgDCAJIApPNgIsIAwgC0EAIAtBAEobNgIoIAwgBiAHQQJ0ajYCECAMIAMgBEECdGo2AgwgDCAAIAFBAnRqNgIIQQEgDEEIaiAJIAogCSAKSxsQEgsgDEEwaiQACxYAIAAgASACIAMgBCAFIAYgB0EIEB8LiQYCDn8EewJAIAEgAk4NACACIAFrQQJ0IgMEQCAAKAIIIAFBAnRqQQAgA/wLAAsgACgCECIKQQBMDQAgACgCCCELIAAoAgwhDCAAKAIUIQ0gACgCACEOIAAoAgQhDwNAIAwgBkECdGohECAPIAZBB3RqIQMgDiAGIA1sIAFqQRZsaiEFIAEhAANAIAUtAAEiB0EYdEGAgICAeHEhBCAFLQAAIAdBCHRyIglB/wdxIQggCUH//wNxIQkCQAJAIAdBAnZBH3EiBwRAIAdBH0cNASAJQQ10IARyQYCAgPwHciEEDAILIAhFDQEgBEGAgICYBHIgCGciBEEXdGsgCUEJIARBH3NrdEEOdEGAgP8DcXIhBAwBCyAHQRd0IAhBDXRyIARqQYCAgMADaiEECyALIABBAnRqIgggA/0AAFAgBf0JAAIiEf0MAgICAgICAgIDAwMDAwMDA/0O/QwBAgQIECBAgAECBAgQIECA/U79DAAAAAAAAAAAAAAAAAAAAAD9JP0MEBAQEBAQEBAQEBAQEBAQEP1OIAX9AAAGIhJBBP1t/VAiE/2JASIU/aoB/foB/eYBIAP9AABAIBT9qQH9+gH95gH95AEgA/0AAGAgE/2KASIT/akB/foB/eYB/eQBIAP9AABwIBP9qgH9+gH95gH95AEgA/0AABAgEv0MDw8PDw8PDw8PDw8PDw8PD/1OIBH9DAAAAAAAAAAAAQEBAQEBAQH9Dv0MAQIECBAgQIABAgQIECBAgP1O/QwAAAAAAAAAAAAAAAAAAAAA/ST9DBAQEBAQEBAQEBAQEBAQEBD9Tv1QIhH9iQEiEv2qAf36Af3mAf3kASAD/QAAACAS/akB/foB/eYB/eQBIAP9AAAgIBH9igEiEf2pAf36Af3mAf3kASAD/QAAMCAR/aoB/foB/eYB/eQBIhH9HwEgECoCAEMAAIDBlJIgEf0fAJIgEf0fApIgEf0fA5IgBL6UIAgqAgCSOAIAIAVBFmohBSAAQQFqIgAgAkcNAAsgBkEBaiIGIApHDQALCwsWACAAIAEgAiADIAQgBSAGIAdBBxAfC/EEAwt/A3sBfQJAIAEgAk4NACAAKAIIIQsgACgCDCIHQQBMBEAgAiABa0ECdCIARQ0BIAsgAUECdGpBACAA/AsADwsgACgCACABIAdsQRJsaiEIIAdBEmwhDCAAKAIEIQ0DQEMAAAAAIRFBACEJIAghAANAIA0gCUEHdGohBCAALQABIgVBGHRBgICAgHhxIQMgAC0AACAFQQh0ciIGQf8HcSEKIAZB//8DcSEGAkACQCAFQQJ2QR9xIgUEQCAFQR9HDQEgBkENdCADckGAgID8B3IhAwwCCyAKRQ0BIANBgICAmARyIApnIgNBF3RrIAZBCSADQR9za3RBDnRBgID/A3FyIQMMAQsgBUEXdCAKQQ10ciADakGAgIDAA2ohAwsgBP0AAAAgAP0AAAIiDv0MDw8PDw8PDw8PDw8PDw8PD/1O/Qz4+Pj4+Pj4+Pj4+Pj4+Pj4/W4iD/2HASIQ/acB/foB/eYBIAT9AAAQIBD9qAH9+gH95gH95AEgBP0AACAgD/2IASIP/acB/foB/eYB/eQBIAT9AAAwIA/9qAH9+gH95gH95AEgBP0AAEAgDkEE/W39DPj4+Pj4+Pj4+Pj4+Pj4+Pj9biIO/YcBIg/9pwH9+gH95gH95AEgBP0AAFAgD/2oAf36Af3mAf3kASAE/QAAYCAO/YgBIg79pwH9+gH95gH95AEgBP0AAHAgDv2oAf36Af3mAf3kASIO/R8AIA79HwGSIA79HwKSIA79HwOSIAO+lCARkiERIABBEmohACAJQQFqIgkgB0cNAAsgCyABQQJ0aiAROAIAIAggDGohCCABQQFqIgEgAkcNAAsLC4EBAQF/IwBBIGsiCCQAAkAgBUEATA0AIARBAEwEQCAFQQJ0IgBFDQEgBiAHQQJ0akEAIAD8CwAMAQsgCCAFNgIcIAggBEEFdjYCGCAIIAIgA2o2AgwgCCAGIAdBAnRqNgIUIAggACABQQJ0ajYCEEEGIAhBDGogBRASCyAIQSBqJAAL5gQCDH8DewJAIAEgAk4NACACIAFrQQJ0IgMEQCAAKAIIIAFBAnRqQQAgA/wLAAsgACgCDCIKQQBMDQAgACgCCCELIAAoAhAhDCAAKAIAIQ0gACgCBCEOA0AgDiAIQQd0aiEDIA0gCCAMbCABakESbGohBSABIQADQCAFLQABIgZBGHRBgICAgHhxIQQgBS0AACAGQQh0ciIJQf8HcSEHIAlB//8DcSEJAkACQCAGQQJ2QR9xIgYEQCAGQR9HDQEgCUENdCAEckGAgID8B3IhBAwCCyAHRQ0BIARBgICAmARyIAdnIgRBF3RrIAlBCSAEQR9za3RBDnRBgID/A3FyIQQMAQsgBkEXdCAHQQ10ciAEakGAgIDAA2ohBAsgCyAAQQJ0aiIHIAP9AAAAIAX9AAACIg/9DA8PDw8PDw8PDw8PDw8PDw/9Tv0M+Pj4+Pj4+Pj4+Pj4+Pj4+P1uIhD9hwEiEf2nAf36Af3mASAD/QAAECAR/agB/foB/eYB/eQBIAP9AAAgIBD9iAEiEP2nAf36Af3mAf3kASAD/QAAMCAQ/agB/foB/eYB/eQBIAP9AABAIA9BBP1t/Qz4+Pj4+Pj4+Pj4+Pj4+Pj4/W4iD/2HASIQ/acB/foB/eYB/eQBIAP9AABQIBD9qAH9+gH95gH95AEgA/0AAGAgD/2IASIP/acB/foB/eYB/eQBIAP9AABwIA/9qAH9+gH95gH95AEiD/0fACAP/R8BkiAP/R8CkiAP/R8DkiAEvpQgByoCAJI4AgAgBUESaiEFIABBAWoiACACRw0ACyAIQQFqIgggCkcNAAsLC4EBAQF/IwBBIGsiCCQAAkAgBUEATA0AIARBAEwEQCAFQQJ0IgBFDQEgBiAHQQJ0akEAIAD8CwAMAQsgCCAFNgIcIAggBEEFdjYCGCAIIAIgA2o2AgwgCCAGIAdBAnRqNgIUIAggACABQQJ0ajYCEEEFIAhBDGogBRASCyAIQSBqJAALBAAjAAsQACMAIABrQXBxIgAkACAACwYAIAAkAAsnACAAJAMgASQEIAIkBSADJAYCQCACRQ0AIARFDQBBxAggBDYCAAsLCgAgACQIIAEkBwsEAEIAC/ACAQd/IwBBIGsiAyQAIAMgACgCHCIENgIQIAAoAhQhBSADIAI2AhwgAyABNgIYIAMgBSAEayIBNgIUIAEgAmohBUECIQcCfwJAAkACQCAAKAI8IANBEGoiAUECIANBDGoQAiIEBH8jASAENgIAQX8FQQALBEAgASEEDAELA0AgBSADKAIMIgZGDQIgBkEASARAIAEhBAwECyABQQhBACAGIAEoAgQiCEsiCRtqIgQgBiAIQQAgCRtrIgggBCgCAGo2AgAgAUEMQQQgCRtqIgEgASgCACAIazYCACAFIAZrIQUgACgCPCAEIgEgByAJayIHIANBDGoQAiIGBH8jASAGNgIAQX8FQQALRQ0ACwsgBUF/Rw0BCyAAIAAoAiwiATYCHCAAIAE2AhQgACABIAAoAjBqNgIQIAIMAQsgAEEANgIcIABCADcDECAAIAAoAgBBIHI2AgBBACAHQQJGDQAaIAIgBCgCBGsLIANBIGokAAv6FQMSfwR7AX0jAEGAEGsiDCQAAkAgACgCGCIJQQBMDQAgASACTg0AIAAoAgggAUECdGohBSACIAFrQQJ0IQQgACgCFEECdCEHIAlBBE8EQCAJQfz///8HcSELA0AgBEUiCkUEQCAFIAMgB2xqQQAgBPwLAAsgCkUEQCAFIAcgA0EBcmxqQQAgBPwLAAsgCkUEQCAFIAcgA0ECcmxqQQAgBPwLAAsgCkUEQCAFIAcgA0EDcmxqQQAgBPwLAAsgA0EEaiEDIAhBBGoiCCALRw0ACwsgCUEDcSIKRQ0AA0AgBARAIAUgAyAHbGpBACAE/AsACyADQQFqIQMgBkEBaiIGIApHDQALCwJAIAAoAiAiC0EATA0AIAlBAUYEQCABQQF0IQ0gACgCCCABQQJ0aiEJIAAoAgAhDiACIAFrIgpBCEghEEEAIQcDQCAAKAIEIAAoAhAgB2xqIA1qIQggDiAHQQJ0aioCACEZQQAhBSAQRQRAIBn9EyEWQQAhBEEIIQYDQCAJIARBAnRqIgMgFiAIIARBAXRq/QAAACIX/akBQQ39qwEiFf0MAOD/DwDg/w8A4P8PAOD/D/1OIBX9DAAAgA8AAIAPAACADwAAgA/9TiIV/QwAAIAPAACADwAAgA8AAIAP/Tf9DAAAADgAAAA4AAAAOAAAADj9Tv2uASIY/QwAAIA4AACAOAAAgDgAAIA4/a4B/QwAAIC4AACAuAAAgLgAAIC4/eQBIBj9DAAAADgAAAA4AAAAOAAAADj9rgEgFf0MAAAAAAAAAAAAAAAAAAAAAP03/VIgF/2nAf0MAAAAgAAAAIAAAACAAAAAgP1O/VD95gEgA/0AAAD95AH9CwAAIAMgFiAX/aoBQQ39qwEiFf0MAOD/DwDg/w8A4P8PAOD/D/1OIBX9DAAAgA8AAIAPAACADwAAgA/9TiIV/QwAAIAPAACADwAAgA8AAIAP/Tf9DAAAADgAAAA4AAAAOAAAADj9Tv2uASIY/QwAAIA4AACAOAAAgDgAAIA4/a4B/QwAAIC4AACAuAAAgLgAAIC4/eQBIBj9DAAAADgAAAA4AAAAOAAAADj9rgEgFf0MAAAAAAAAAAAAAAAAAAAAAP03/VIgF/2oAf0MAAAAgAAAAIAAAACAAAAAgP1O/VD95gEgA/0AABD95AH9CwAQIAYiBSIEQQhqIgYgCkwNAAsLAkAgBSAKTg0AAkAgAiABIAVqayIGQQRJBEAgBSEDDAELIAUgBkF8cSILaiEDIBn9EyEXQQAhBANAIAkgBCAFaiIRQQJ0aiISIBcgCCARQQF0av1dAAAiFv0M/3//f/9//3//f/9//3//f/1O/akBQQ39qwEiFSAV/QwAAIAPAACADwAAgA8AAIAP/U4iFf0MAACADwAAgA8AAIAPAACAD/03/QwAAAA4AAAAOAAAADgAAAA4/U79rgEiGP0MAACAOAAAgDgAAIA4AACAOP2uAf0MAACAuAAAgLgAAIC4AACAuP3kASAY/QwAAAA4AAAAOAAAADgAAAA4/a4BIBX9DAAAAAAAAAAAAAAAAAAAAAD9N/1SIBb9pwH9DAAAAIAAAACAAAAAgAAAAID9Tv1Q/eYBIBL9AAIA/eQB/QsCACAEQQRqIgQgC0cNAAsgBiALRg0BCwNAIAkgA0ECdGoiBCAZIAggA0EBdGovAAAiBUH//wFxQQ10IgZBgICAwANBACAGQYCAgPwAcSIGQYCAgPwARhtqIgtBgICAwANqIAtBgICAxANqvkMAAIC4krwgBhsgBUEQdEGAgICAeHFyvpQgBCoCAJI4AgAgA0EBaiIDIApHDQALCyAHQQFqIgcgACgCIEgNAAsMAQsgASACTg0AIAIgAWshEQNAIAtBAEoEQEGABCACIAFrIg4gDkGABE4bIQZBgAQgESANQQl0ayIDIANBgAROGyIQQQNxIQogAUEBdCESQQAhByAOQQdKIRQDQCAAKAIEIAAoAhAgB2xqIBJqIQhBCCEEQQAhAyAUBEADQCAMIANBAnRqIgUgCCADQQF0av0AAAAiF/2qAUEN/asBIhb9DADg/w8A4P8PAOD/DwDg/w/9TiAW/QwAAIAPAACADwAAgA8AAIAP/U4iFv0MAACADwAAgA8AAIAPAACAD/03/QwAAAA4AAAAOAAAADgAAAA4/U79rgEiFf0MAACAOAAAgDgAAIA4AACAOP2uAf0MAACAuAAAgLgAAIC4AACAuP3kASAV/QwAAAA4AAAAOAAAADgAAAA4/a4BIBb9DAAAAAAAAAAAAAAAAAAAAAD9N/1SIBf9qAH9DAAAAIAAAACAAAAAgAAAAID9Tv1Q/QsEECAFIBf9qQFBDf2rASIW/QwA4P8PAOD/DwDg/w8A4P8P/U4gFv0MAACADwAAgA8AAIAPAACAD/1OIhb9DAAAgA8AAIAPAACADwAAgA/9N/0MAAAAOAAAADgAAAA4AAAAOP1O/a4BIhX9DAAAgDgAAIA4AACAOAAAgDj9rgH9DAAAgLgAAIC4AACAuAAAgLj95AEgFf0MAAAAOAAAADgAAAA4AAAAOP2uASAW/QwAAAAAAAAAAAAAAAAAAAAA/Tf9UiAX/acB/QwAAACAAAAAgAAAAIAAAACA/U79UP0LBAAgBCIDQQhqIgQgBkwNAAsLAkAgAyAGTg0AAkAgECADayIEQQRJBEAgAyEEDAELIAMgBCAKayIPaiEEQQAhBQNAIAwgAyAFaiITQQJ0aiAIIBNBAXRq/V0AACIX/Qz/f/9//3//f/9//3//f/9//U79qQFBDf2rASIWIBb9DAAAgA8AAIAPAACADwAAgA/9TiIW/QwAAIAPAACADwAAgA8AAIAP/Tf9DAAAADgAAAA4AAAAOAAAADj9Tv2uASIV/QwAAIA4AACAOAAAgDgAAIA4/a4B/QwAAIC4AACAuAAAgLgAAIC4/eQBIBX9DAAAADgAAAA4AAAAOAAAADj9rgEgFv0MAAAAAAAAAAAAAAAAAAAAAP03/VIgF/2nAf0MAAAAgAAAAIAAAACAAAAAgP1O/VD9CwIAIAVBBGoiBSAPRw0ACyAKRQ0BCwNAIAwgBEECdGogCCAEQQF0ai8AACIDQf//AXFBDXQiBUGAgIDAA0EAIAVBgICA/ABxIgVBgICA/ABGG2oiD0GAgIDAA2ogD0GAgIDEA2q+QwAAgLiSvCAFGyADQRB0QYCAgIB4cXI2AgAgBEEBaiIEIAZHDQALC0EAIQggCUEASgRAA0AgACgCCCAAKAIUIAhsQQJ0aiABQQJ0aiEJIAAoAgAgACgCDCAIbEECdGogB0ECdGoqAgAhGUEAIQMgDkEETgRAIBn9EyEXQQAhBEEEIQUDQCAJIARBAnQiA2oiBCADIAxq/QAEACAX/eYBIAT9AAAA/eQB/QsAACAFIgMhBCADQQRqIgUgBkwNAAsLAkAgAyAGTg0AAkAgECADayIEQQRJBEAgAyEEDAELIAMgBCAKayILaiEEIBn9EyEXQQAhBQNAIAkgAyAFakECdCIPaiITIBP9AAIAIAwgD2r9AAIAIBf95gH95AH9CwIAIAVBBGoiBSALRw0ACyAKRQ0BCwNAIAkgBEECdCIDaiIFIAUqAgAgAyAMaioCACAZlJI4AgAgBEEBaiIEIAZHDQALCyAIQQFqIgggACgCGCIJSA0ACyAAKAIgIQsLIAdBAWoiByALSA0ACwsgDUEBaiENIAFBgARqIgEgAkgNAAsLIAxBgBBqJAALBABBAAsgAQF/IwMoAngiAEEB/hcCACAAECcgAEEBQQD+SAIAGgsEACMDC4YBAQF/IwBBMGsiDCQAAkAgCUEATA0AIApBAEwNACAMIAo2AiggDCAJNgIkIAwgCDYCICAMIAU2AhwgDCACNgIYIAwgAyAEajYCECAMIAtBACALQQBKGzYCLCAMIAYgB0ECdGo2AhQgDCAAIAFBAnRqNgIMQQQgDEEMaiAKEBILIAxBMGokAAuRAQACQAJAAkBBiCJBAEEB/kgCAA4CAAECC0GACCQBQYAIQQBBBfwLACMJBEAAC0GICEEAQTT8CwBBvAhBAEHlAPwIAABBoQlBAEE//AsAQeAJQQBBCPwIAQBB8AlBAEGYGPwLAEGIIkEC/hcCAEGIIkF//gACABoMAQtBiCJBAUJ//gECABoLQQEkCfwJAPwJAQt1AEGQogQkCEGQIiQHQaAMQQI2AgBBgAxBgAw2AgBBzAxBzAw2AgBB4AxB6As2AgBBmAxBKjYCAEHIDEGwDTYCAEGMDEGADDYCAEGIDEGADDYCAEGADBAJQbQMIwg2AgBBuAwjCCMHazYCAEGADBBBQYAMEAgLC3ICAWUIBAAACAQAAAAAAQAAAgAAAAAAAAUAAAAAAAAAAAAAACIAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACMAAAAkAAAACA0AAAAEAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAD/////CgEIUAQAABARAQA=" + const val ST_MJS: String = "YXN5bmMgZnVuY3Rpb24gY3JlYXRlU2thaW5ldEtlcm5lbHMobW9kdWxlQXJnPXt9KXt2YXIgbW9kdWxlUnRuO3ZhciBNb2R1bGU9bW9kdWxlQXJnO3ZhciBFTlZJUk9OTUVOVF9JU19XRUI9dHlwZW9mIHdpbmRvdz09Im9iamVjdCI7dmFyIEVOVklST05NRU5UX0lTX1dPUktFUj10eXBlb2YgV29ya2VyR2xvYmFsU2NvcGUhPSJ1bmRlZmluZWQiO3ZhciBFTlZJUk9OTUVOVF9JU19OT0RFPXR5cGVvZiBwcm9jZXNzPT0ib2JqZWN0IiYmcHJvY2Vzcy52ZXJzaW9ucz8ubm9kZSYmcHJvY2Vzcy50eXBlIT0icmVuZGVyZXIiO2lmKEVOVklST05NRU5UX0lTX05PREUpe2NvbnN0e2NyZWF0ZVJlcXVpcmV9PWF3YWl0IGltcG9ydCgibW9kdWxlIik7dmFyIHJlcXVpcmU9Y3JlYXRlUmVxdWlyZShpbXBvcnQubWV0YS51cmwuc3RhcnRzV2l0aCgiZmlsZToiKT9pbXBvcnQubWV0YS51cmw6cHJvY2Vzcy5jd2QoKSsiLyIpfXZhciBhcmd1bWVudHNfPVtdO3ZhciB0aGlzUHJvZ3JhbT0iLi90aGlzLnByb2dyYW0iO3ZhciBxdWl0Xz0oc3RhdHVzLHRvVGhyb3cpPT57dGhyb3cgdG9UaHJvd307dmFyIF9zY3JpcHROYW1lPWltcG9ydC5tZXRhLnVybDt2YXIgc2NyaXB0RGlyZWN0b3J5PSIiO2Z1bmN0aW9uIGxvY2F0ZUZpbGUocGF0aCl7aWYoTW9kdWxlWyJsb2NhdGVGaWxlIl0pe3JldHVybiBNb2R1bGVbImxvY2F0ZUZpbGUiXShwYXRoLHNjcmlwdERpcmVjdG9yeSl9cmV0dXJuIHNjcmlwdERpcmVjdG9yeStwYXRofXZhciByZWFkQXN5bmMscmVhZEJpbmFyeTtpZihFTlZJUk9OTUVOVF9JU19OT0RFKXt2YXIgZnM9cmVxdWlyZSgiZnMiKTtpZihfc2NyaXB0TmFtZS5zdGFydHNXaXRoKCJmaWxlOiIpKXtzY3JpcHREaXJlY3Rvcnk9cmVxdWlyZSgicGF0aCIpLmRpcm5hbWUocmVxdWlyZSgidXJsIikuZmlsZVVSTFRvUGF0aChfc2NyaXB0TmFtZSkpKyIvIn1yZWFkQmluYXJ5PWZpbGVuYW1lPT57ZmlsZW5hbWU9aXNGaWxlVVJJKGZpbGVuYW1lKT9uZXcgVVJMKGZpbGVuYW1lKTpmaWxlbmFtZTt2YXIgcmV0PWZzLnJlYWRGaWxlU3luYyhmaWxlbmFtZSk7cmV0dXJuIHJldH07cmVhZEFzeW5jPWFzeW5jKGZpbGVuYW1lLGJpbmFyeT10cnVlKT0+e2ZpbGVuYW1lPWlzRmlsZVVSSShmaWxlbmFtZSk/bmV3IFVSTChmaWxlbmFtZSk6ZmlsZW5hbWU7dmFyIHJldD1mcy5yZWFkRmlsZVN5bmMoZmlsZW5hbWUsYmluYXJ5P3VuZGVmaW5lZDoidXRmOCIpO3JldHVybiByZXR9O2lmKHByb2Nlc3MuYXJndi5sZW5ndGg+MSl7dGhpc1Byb2dyYW09cHJvY2Vzcy5hcmd2WzFdLnJlcGxhY2UoL1xcL2csIi8iKX1hcmd1bWVudHNfPXByb2Nlc3MuYXJndi5zbGljZSgyKTtxdWl0Xz0oc3RhdHVzLHRvVGhyb3cpPT57cHJvY2Vzcy5leGl0Q29kZT1zdGF0dXM7dGhyb3cgdG9UaHJvd319ZWxzZSBpZihFTlZJUk9OTUVOVF9JU19XRUJ8fEVOVklST05NRU5UX0lTX1dPUktFUil7dHJ5e3NjcmlwdERpcmVjdG9yeT1uZXcgVVJMKCIuIixfc2NyaXB0TmFtZSkuaHJlZn1jYXRjaHt9e2lmKEVOVklST05NRU5UX0lTX1dPUktFUil7cmVhZEJpbmFyeT11cmw9Pnt2YXIgeGhyPW5ldyBYTUxIdHRwUmVxdWVzdDt4aHIub3BlbigiR0VUIix1cmwsZmFsc2UpO3hoci5yZXNwb25zZVR5cGU9ImFycmF5YnVmZmVyIjt4aHIuc2VuZChudWxsKTtyZXR1cm4gbmV3IFVpbnQ4QXJyYXkoeGhyLnJlc3BvbnNlKX19cmVhZEFzeW5jPWFzeW5jIHVybD0+e3ZhciByZXNwb25zZT1hd2FpdCBmZXRjaCh1cmwse2NyZWRlbnRpYWxzOiJzYW1lLW9yaWdpbiJ9KTtpZihyZXNwb25zZS5vayl7cmV0dXJuIHJlc3BvbnNlLmFycmF5QnVmZmVyKCl9dGhyb3cgbmV3IEVycm9yKHJlc3BvbnNlLnN0YXR1cysiIDogIityZXNwb25zZS51cmwpfX19ZWxzZXt9dmFyIG91dD1jb25zb2xlLmxvZy5iaW5kKGNvbnNvbGUpO3ZhciBlcnI9Y29uc29sZS5lcnJvci5iaW5kKGNvbnNvbGUpO3ZhciB3YXNtQmluYXJ5O3ZhciBBQk9SVD1mYWxzZTt2YXIgaXNGaWxlVVJJPWZpbGVuYW1lPT5maWxlbmFtZS5zdGFydHNXaXRoKCJmaWxlOi8vIik7dmFyIHJlYWR5UHJvbWlzZVJlc29sdmUscmVhZHlQcm9taXNlUmVqZWN0O3ZhciB3YXNtTWVtb3J5O3ZhciBIRUFQOCxIRUFQVTgsSEVBUDE2LEhFQVBVMTYsSEVBUDMyLEhFQVBVMzIsSEVBUEYzMixIRUFQRjY0O3ZhciBIRUFQNjQsSEVBUFU2NDt2YXIgcnVudGltZUluaXRpYWxpemVkPWZhbHNlO2Z1bmN0aW9uIHVwZGF0ZU1lbW9yeVZpZXdzKCl7dmFyIGI9d2FzbU1lbW9yeS5idWZmZXI7SEVBUDg9bmV3IEludDhBcnJheShiKTtIRUFQMTY9bmV3IEludDE2QXJyYXkoYik7SEVBUFU4PW5ldyBVaW50OEFycmF5KGIpO0hFQVBVMTY9bmV3IFVpbnQxNkFycmF5KGIpO0hFQVAzMj1uZXcgSW50MzJBcnJheShiKTtIRUFQVTMyPW5ldyBVaW50MzJBcnJheShiKTtIRUFQRjMyPW5ldyBGbG9hdDMyQXJyYXkoYik7SEVBUEY2ND1uZXcgRmxvYXQ2NEFycmF5KGIpO0hFQVA2ND1uZXcgQmlnSW50NjRBcnJheShiKTtIRUFQVTY0PW5ldyBCaWdVaW50NjRBcnJheShiKX1mdW5jdGlvbiBwcmVSdW4oKXtpZihNb2R1bGVbInByZVJ1biJdKXtpZih0eXBlb2YgTW9kdWxlWyJwcmVSdW4iXT09ImZ1bmN0aW9uIilNb2R1bGVbInByZVJ1biJdPVtNb2R1bGVbInByZVJ1biJdXTt3aGlsZShNb2R1bGVbInByZVJ1biJdLmxlbmd0aCl7YWRkT25QcmVSdW4oTW9kdWxlWyJwcmVSdW4iXS5zaGlmdCgpKX19Y2FsbFJ1bnRpbWVDYWxsYmFja3Mob25QcmVSdW5zKX1mdW5jdGlvbiBpbml0UnVudGltZSgpe3J1bnRpbWVJbml0aWFsaXplZD10cnVlO3dhc21FeHBvcnRzWyJjIl0oKX1mdW5jdGlvbiBwb3N0UnVuKCl7aWYoTW9kdWxlWyJwb3N0UnVuIl0pe2lmKHR5cGVvZiBNb2R1bGVbInBvc3RSdW4iXT09ImZ1bmN0aW9uIilNb2R1bGVbInBvc3RSdW4iXT1bTW9kdWxlWyJwb3N0UnVuIl1dO3doaWxlKE1vZHVsZVsicG9zdFJ1biJdLmxlbmd0aCl7YWRkT25Qb3N0UnVuKE1vZHVsZVsicG9zdFJ1biJdLnNoaWZ0KCkpfX1jYWxsUnVudGltZUNhbGxiYWNrcyhvblBvc3RSdW5zKX1mdW5jdGlvbiBhYm9ydCh3aGF0KXtNb2R1bGVbIm9uQWJvcnQiXT8uKHdoYXQpO3doYXQ9IkFib3J0ZWQoIit3aGF0KyIpIjtlcnIod2hhdCk7QUJPUlQ9dHJ1ZTt3aGF0Kz0iLiBCdWlsZCB3aXRoIC1zQVNTRVJUSU9OUyBmb3IgbW9yZSBpbmZvLiI7dmFyIGU9bmV3IFdlYkFzc2VtYmx5LlJ1bnRpbWVFcnJvcih3aGF0KTtyZWFkeVByb21pc2VSZWplY3Q/LihlKTt0aHJvdyBlfXZhciB3YXNtQmluYXJ5RmlsZTtmdW5jdGlvbiBmaW5kV2FzbUJpbmFyeSgpe2lmKE1vZHVsZVsibG9jYXRlRmlsZSJdKXtyZXR1cm4gbG9jYXRlRmlsZSgic2thaW5ldF9rZXJuZWxzX3N0Lndhc20iKX10cnl7cmV0dXJuIG5ldyBVUkwoInNrYWluZXRfa2VybmVsc19zdC53YXNtIixpbXBvcnQubWV0YS51cmwpLmhyZWZ9Y2F0Y2goZSl7cmV0dXJuICJza2FpbmV0X2tlcm5lbHNfc3Qud2FzbSJ9fWZ1bmN0aW9uIGdldEJpbmFyeVN5bmMoZmlsZSl7aWYoZmlsZT09d2FzbUJpbmFyeUZpbGUmJndhc21CaW5hcnkpe3JldHVybiBuZXcgVWludDhBcnJheSh3YXNtQmluYXJ5KX1pZihyZWFkQmluYXJ5KXtyZXR1cm4gcmVhZEJpbmFyeShmaWxlKX10aHJvdyJib3RoIGFzeW5jIGFuZCBzeW5jIGZldGNoaW5nIG9mIHRoZSB3YXNtIGZhaWxlZCJ9YXN5bmMgZnVuY3Rpb24gZ2V0V2FzbUJpbmFyeShiaW5hcnlGaWxlKXtpZighd2FzbUJpbmFyeSl7dHJ5e3ZhciByZXNwb25zZT1hd2FpdCByZWFkQXN5bmMoYmluYXJ5RmlsZSk7cmV0dXJuIG5ldyBVaW50OEFycmF5KHJlc3BvbnNlKX1jYXRjaHt9fXJldHVybiBnZXRCaW5hcnlTeW5jKGJpbmFyeUZpbGUpfWFzeW5jIGZ1bmN0aW9uIGluc3RhbnRpYXRlQXJyYXlCdWZmZXIoYmluYXJ5RmlsZSxpbXBvcnRzKXt0cnl7dmFyIGJpbmFyeT1hd2FpdCBnZXRXYXNtQmluYXJ5KGJpbmFyeUZpbGUpO3ZhciBpbnN0YW5jZT1hd2FpdCBXZWJBc3NlbWJseS5pbnN0YW50aWF0ZShiaW5hcnksaW1wb3J0cyk7cmV0dXJuIGluc3RhbmNlfWNhdGNoKHJlYXNvbil7ZXJyKGBmYWlsZWQgdG8gYXN5bmNocm9ub3VzbHkgcHJlcGFyZSB3YXNtOiAke3JlYXNvbn1gKTthYm9ydChyZWFzb24pfX1hc3luYyBmdW5jdGlvbiBpbnN0YW50aWF0ZUFzeW5jKGJpbmFyeSxiaW5hcnlGaWxlLGltcG9ydHMpe2lmKCFiaW5hcnkmJiFFTlZJUk9OTUVOVF9JU19OT0RFKXt0cnl7dmFyIHJlc3BvbnNlPWZldGNoKGJpbmFyeUZpbGUse2NyZWRlbnRpYWxzOiJzYW1lLW9yaWdpbiJ9KTt2YXIgaW5zdGFudGlhdGlvblJlc3VsdD1hd2FpdCBXZWJBc3NlbWJseS5pbnN0YW50aWF0ZVN0cmVhbWluZyhyZXNwb25zZSxpbXBvcnRzKTtyZXR1cm4gaW5zdGFudGlhdGlvblJlc3VsdH1jYXRjaChyZWFzb24pe2Vycihgd2FzbSBzdHJlYW1pbmcgY29tcGlsZSBmYWlsZWQ6ICR7cmVhc29ufWApO2VycigiZmFsbGluZyBiYWNrIHRvIEFycmF5QnVmZmVyIGluc3RhbnRpYXRpb24iKX19cmV0dXJuIGluc3RhbnRpYXRlQXJyYXlCdWZmZXIoYmluYXJ5RmlsZSxpbXBvcnRzKX1mdW5jdGlvbiBnZXRXYXNtSW1wb3J0cygpe3JldHVybnthOndhc21JbXBvcnRzfX1hc3luYyBmdW5jdGlvbiBjcmVhdGVXYXNtKCl7ZnVuY3Rpb24gcmVjZWl2ZUluc3RhbmNlKGluc3RhbmNlLG1vZHVsZSl7d2FzbUV4cG9ydHM9aW5zdGFuY2UuZXhwb3J0czt3YXNtRXhwb3J0cz1hcHBseVNpZ25hdHVyZUNvbnZlcnNpb25zKHdhc21FeHBvcnRzKTt3YXNtTWVtb3J5PXdhc21FeHBvcnRzWyJiIl07TW9kdWxlWyJ3YXNtTWVtb3J5Il09d2FzbU1lbW9yeTt1cGRhdGVNZW1vcnlWaWV3cygpO2Fzc2lnbldhc21FeHBvcnRzKHdhc21FeHBvcnRzKTtyZXR1cm4gd2FzbUV4cG9ydHN9ZnVuY3Rpb24gcmVjZWl2ZUluc3RhbnRpYXRpb25SZXN1bHQocmVzdWx0KXtyZXR1cm4gcmVjZWl2ZUluc3RhbmNlKHJlc3VsdFsiaW5zdGFuY2UiXSl9dmFyIGluZm89Z2V0V2FzbUltcG9ydHMoKTtpZihNb2R1bGVbImluc3RhbnRpYXRlV2FzbSJdKXtyZXR1cm4gbmV3IFByb21pc2UoKHJlc29sdmUscmVqZWN0KT0+e01vZHVsZVsiaW5zdGFudGlhdGVXYXNtIl0oaW5mbywobW9kLGluc3QpPT57cmVzb2x2ZShyZWNlaXZlSW5zdGFuY2UobW9kLGluc3QpKX0pfSl9d2FzbUJpbmFyeUZpbGU/Pz1maW5kV2FzbUJpbmFyeSgpO3ZhciByZXN1bHQ9YXdhaXQgaW5zdGFudGlhdGVBc3luYyh3YXNtQmluYXJ5LHdhc21CaW5hcnlGaWxlLGluZm8pO3ZhciBleHBvcnRzPXJlY2VpdmVJbnN0YW50aWF0aW9uUmVzdWx0KHJlc3VsdCk7cmV0dXJuIGV4cG9ydHN9Y2xhc3MgRXhpdFN0YXR1c3tuYW1lPSJFeGl0U3RhdHVzIjtjb25zdHJ1Y3RvcihzdGF0dXMpe3RoaXMubWVzc2FnZT1gUHJvZ3JhbSB0ZXJtaW5hdGVkIHdpdGggZXhpdCgke3N0YXR1c30pYDt0aGlzLnN0YXR1cz1zdGF0dXN9fXZhciBjYWxsUnVudGltZUNhbGxiYWNrcz1jYWxsYmFja3M9Pnt3aGlsZShjYWxsYmFja3MubGVuZ3RoPjApe2NhbGxiYWNrcy5zaGlmdCgpKE1vZHVsZSl9fTt2YXIgb25Qb3N0UnVucz1bXTt2YXIgYWRkT25Qb3N0UnVuPWNiPT5vblBvc3RSdW5zLnB1c2goY2IpO3ZhciBvblByZVJ1bnM9W107dmFyIGFkZE9uUHJlUnVuPWNiPT5vblByZVJ1bnMucHVzaChjYik7dmFyIG5vRXhpdFJ1bnRpbWU9dHJ1ZTt2YXIgZ2V0SGVhcE1heD0oKT0+NDI5NDkwMTc2MDt2YXIgYWxpZ25NZW1vcnk9KHNpemUsYWxpZ25tZW50KT0+TWF0aC5jZWlsKHNpemUvYWxpZ25tZW50KSphbGlnbm1lbnQ7dmFyIGdyb3dNZW1vcnk9c2l6ZT0+e3ZhciBvbGRIZWFwU2l6ZT13YXNtTWVtb3J5LmJ1ZmZlci5ieXRlTGVuZ3RoO3ZhciBwYWdlcz0oc2l6ZS1vbGRIZWFwU2l6ZSs2NTUzNSkvNjU1MzZ8MDt0cnl7d2FzbU1lbW9yeS5ncm93KHBhZ2VzKTt1cGRhdGVNZW1vcnlWaWV3cygpO3JldHVybiAxfWNhdGNoKGUpe319O2Z1bmN0aW9uIF9lbXNjcmlwdGVuX3Jlc2l6ZV9oZWFwKHJlcXVlc3RlZFNpemUpe3JlcXVlc3RlZFNpemU+Pj49MDt2YXIgb2xkU2l6ZT1IRUFQVTgubGVuZ3RoO3ZhciBtYXhIZWFwU2l6ZT1nZXRIZWFwTWF4KCk7aWYocmVxdWVzdGVkU2l6ZT5tYXhIZWFwU2l6ZSl7cmV0dXJuIGZhbHNlfWZvcih2YXIgY3V0RG93bj0xO2N1dERvd248PTQ7Y3V0RG93bio9Mil7dmFyIG92ZXJHcm93bkhlYXBTaXplPW9sZFNpemUqKDErLjIvY3V0RG93bik7b3Zlckdyb3duSGVhcFNpemU9TWF0aC5taW4ob3Zlckdyb3duSGVhcFNpemUscmVxdWVzdGVkU2l6ZSsxMDA2NjMyOTYpO3ZhciBuZXdTaXplPU1hdGgubWluKG1heEhlYXBTaXplLGFsaWduTWVtb3J5KE1hdGgubWF4KHJlcXVlc3RlZFNpemUsb3Zlckdyb3duSGVhcFNpemUpLDY1NTM2KSk7dmFyIHJlcGxhY2VtZW50PWdyb3dNZW1vcnkobmV3U2l6ZSk7aWYocmVwbGFjZW1lbnQpe3JldHVybiB0cnVlfX1yZXR1cm4gZmFsc2V9e2lmKE1vZHVsZVsibm9FeGl0UnVudGltZSJdKW5vRXhpdFJ1bnRpbWU9TW9kdWxlWyJub0V4aXRSdW50aW1lIl07aWYoTW9kdWxlWyJwcmludCJdKW91dD1Nb2R1bGVbInByaW50Il07aWYoTW9kdWxlWyJwcmludEVyciJdKWVycj1Nb2R1bGVbInByaW50RXJyIl07aWYoTW9kdWxlWyJ3YXNtQmluYXJ5Il0pd2FzbUJpbmFyeT1Nb2R1bGVbIndhc21CaW5hcnkiXTtpZihNb2R1bGVbImFyZ3VtZW50cyJdKWFyZ3VtZW50c189TW9kdWxlWyJhcmd1bWVudHMiXTtpZihNb2R1bGVbInRoaXNQcm9ncmFtIl0pdGhpc1Byb2dyYW09TW9kdWxlWyJ0aGlzUHJvZ3JhbSJdO2lmKE1vZHVsZVsicHJlSW5pdCJdKXtpZih0eXBlb2YgTW9kdWxlWyJwcmVJbml0Il09PSJmdW5jdGlvbiIpTW9kdWxlWyJwcmVJbml0Il09W01vZHVsZVsicHJlSW5pdCJdXTt3aGlsZShNb2R1bGVbInByZUluaXQiXS5sZW5ndGg+MCl7TW9kdWxlWyJwcmVJbml0Il0uc2hpZnQoKSgpfX19TW9kdWxlWyJ3YXNtTWVtb3J5Il09d2FzbU1lbW9yeTt2YXIgX3NrYWluZXRfZnAzMl9tYXRtdWwsX3NrYWluZXRfZnAzMl9tYXRtdWxfd3QsX3NrYWluZXRfYmYxNl9tYXRtdWwsX3NrYWluZXRfZnAxNl9tYXRtdWwsX3NrYWluZXRfcTRfMF9tYXRtdWwsX3NrYWluZXRfcTRfMF9tYXRtdWxfcm0sX3NrYWluZXRfcTVfMF9tYXRtdWwsX21hbGxvYyxfZnJlZSxfc2thaW5ldF9xNV8wX21hdG11bF9ybSxfc2thaW5ldF9xNV8xX21hdG11bCxfc2thaW5ldF9xNV8xX21hdG11bF9ybSxfc2thaW5ldF9xOF8wX21hdG11bCxfc2thaW5ldF9xOF8wX21hdG11bF9ybSxfc2thaW5ldF9xNGtfbWF0bXVsLF9za2FpbmV0X3E0a19tYXRtdWxfcm0sX3NrYWluZXRfcTVrX21hdG11bCxfc2thaW5ldF9xNWtfbWF0bXVsX3JtLF9za2FpbmV0X3E2a19tYXRtdWwsX3NrYWluZXRfcTZrX21hdG11bF9ybSxfc2thaW5ldF90ZXJuYXJ5X2YzMl9nZW12LF9za2FpbmV0X3Rlcm5hcnlfbG1oZWFkX3N0YWdlMSxfc2thaW5ldF9iaXRuZXRfZ2Vtdl90cTJfMCxfc2thaW5ldF9mMzJfYmluYXJ5LF9za2FpbmV0X2YzMl91bmFyeSxfc2thaW5ldF9mMzJfcmVkdWNlX2FsbCxfc2thaW5ldF9mMzJfcmVkdWNlX2F4aXMsX3NrYWluZXRfZjMyX3NvZnRtYXhfcm93cyxfc2thaW5ldF9mMzJfbGF5ZXJfbm9ybSxfc2thaW5ldF9mMzJfcm1zX25vcm0sX3NrYWluZXRfZjMyX3NkcGEsX3NrYWluZXRfZjMyX2NvbnY7ZnVuY3Rpb24gYXNzaWduV2FzbUV4cG9ydHMod2FzbUV4cG9ydHMpe01vZHVsZVsiX3NrYWluZXRfZnAzMl9tYXRtdWwiXT1fc2thaW5ldF9mcDMyX21hdG11bD13YXNtRXhwb3J0c1siZCJdO01vZHVsZVsiX3NrYWluZXRfZnAzMl9tYXRtdWxfd3QiXT1fc2thaW5ldF9mcDMyX21hdG11bF93dD13YXNtRXhwb3J0c1siZSJdO01vZHVsZVsiX3NrYWluZXRfYmYxNl9tYXRtdWwiXT1fc2thaW5ldF9iZjE2X21hdG11bD13YXNtRXhwb3J0c1siZiJdO01vZHVsZVsiX3NrYWluZXRfZnAxNl9tYXRtdWwiXT1fc2thaW5ldF9mcDE2X21hdG11bD13YXNtRXhwb3J0c1siZyJdO01vZHVsZVsiX3NrYWluZXRfcTRfMF9tYXRtdWwiXT1fc2thaW5ldF9xNF8wX21hdG11bD13YXNtRXhwb3J0c1siaCJdO01vZHVsZVsiX3NrYWluZXRfcTRfMF9tYXRtdWxfcm0iXT1fc2thaW5ldF9xNF8wX21hdG11bF9ybT13YXNtRXhwb3J0c1siaSJdO01vZHVsZVsiX3NrYWluZXRfcTVfMF9tYXRtdWwiXT1fc2thaW5ldF9xNV8wX21hdG11bD13YXNtRXhwb3J0c1siaiJdO01vZHVsZVsiX21hbGxvYyJdPV9tYWxsb2M9d2FzbUV4cG9ydHNbImsiXTtNb2R1bGVbIl9mcmVlIl09X2ZyZWU9d2FzbUV4cG9ydHNbImwiXTtNb2R1bGVbIl9za2FpbmV0X3E1XzBfbWF0bXVsX3JtIl09X3NrYWluZXRfcTVfMF9tYXRtdWxfcm09d2FzbUV4cG9ydHNbIm0iXTtNb2R1bGVbIl9za2FpbmV0X3E1XzFfbWF0bXVsIl09X3NrYWluZXRfcTVfMV9tYXRtdWw9d2FzbUV4cG9ydHNbIm4iXTtNb2R1bGVbIl9za2FpbmV0X3E1XzFfbWF0bXVsX3JtIl09X3NrYWluZXRfcTVfMV9tYXRtdWxfcm09d2FzbUV4cG9ydHNbIm8iXTtNb2R1bGVbIl9za2FpbmV0X3E4XzBfbWF0bXVsIl09X3NrYWluZXRfcThfMF9tYXRtdWw9d2FzbUV4cG9ydHNbInAiXTtNb2R1bGVbIl9za2FpbmV0X3E4XzBfbWF0bXVsX3JtIl09X3NrYWluZXRfcThfMF9tYXRtdWxfcm09d2FzbUV4cG9ydHNbInEiXTtNb2R1bGVbIl9za2FpbmV0X3E0a19tYXRtdWwiXT1fc2thaW5ldF9xNGtfbWF0bXVsPXdhc21FeHBvcnRzWyJyIl07TW9kdWxlWyJfc2thaW5ldF9xNGtfbWF0bXVsX3JtIl09X3NrYWluZXRfcTRrX21hdG11bF9ybT13YXNtRXhwb3J0c1sicyJdO01vZHVsZVsiX3NrYWluZXRfcTVrX21hdG11bCJdPV9za2FpbmV0X3E1a19tYXRtdWw9d2FzbUV4cG9ydHNbInQiXTtNb2R1bGVbIl9za2FpbmV0X3E1a19tYXRtdWxfcm0iXT1fc2thaW5ldF9xNWtfbWF0bXVsX3JtPXdhc21FeHBvcnRzWyJ1Il07TW9kdWxlWyJfc2thaW5ldF9xNmtfbWF0bXVsIl09X3NrYWluZXRfcTZrX21hdG11bD13YXNtRXhwb3J0c1sidiJdO01vZHVsZVsiX3NrYWluZXRfcTZrX21hdG11bF9ybSJdPV9za2FpbmV0X3E2a19tYXRtdWxfcm09d2FzbUV4cG9ydHNbInciXTtNb2R1bGVbIl9za2FpbmV0X3Rlcm5hcnlfZjMyX2dlbXYiXT1fc2thaW5ldF90ZXJuYXJ5X2YzMl9nZW12PXdhc21FeHBvcnRzWyJ4Il07TW9kdWxlWyJfc2thaW5ldF90ZXJuYXJ5X2xtaGVhZF9zdGFnZTEiXT1fc2thaW5ldF90ZXJuYXJ5X2xtaGVhZF9zdGFnZTE9d2FzbUV4cG9ydHNbInkiXTtNb2R1bGVbIl9za2FpbmV0X2JpdG5ldF9nZW12X3RxMl8wIl09X3NrYWluZXRfYml0bmV0X2dlbXZfdHEyXzA9d2FzbUV4cG9ydHNbInoiXTtNb2R1bGVbIl9za2FpbmV0X2YzMl9iaW5hcnkiXT1fc2thaW5ldF9mMzJfYmluYXJ5PXdhc21FeHBvcnRzWyJBIl07TW9kdWxlWyJfc2thaW5ldF9mMzJfdW5hcnkiXT1fc2thaW5ldF9mMzJfdW5hcnk9d2FzbUV4cG9ydHNbIkIiXTtNb2R1bGVbIl9za2FpbmV0X2YzMl9yZWR1Y2VfYWxsIl09X3NrYWluZXRfZjMyX3JlZHVjZV9hbGw9d2FzbUV4cG9ydHNbIkMiXTtNb2R1bGVbIl9za2FpbmV0X2YzMl9yZWR1Y2VfYXhpcyJdPV9za2FpbmV0X2YzMl9yZWR1Y2VfYXhpcz13YXNtRXhwb3J0c1siRCJdO01vZHVsZVsiX3NrYWluZXRfZjMyX3NvZnRtYXhfcm93cyJdPV9za2FpbmV0X2YzMl9zb2Z0bWF4X3Jvd3M9d2FzbUV4cG9ydHNbIkUiXTtNb2R1bGVbIl9za2FpbmV0X2YzMl9sYXllcl9ub3JtIl09X3NrYWluZXRfZjMyX2xheWVyX25vcm09d2FzbUV4cG9ydHNbIkYiXTtNb2R1bGVbIl9za2FpbmV0X2YzMl9ybXNfbm9ybSJdPV9za2FpbmV0X2YzMl9ybXNfbm9ybT13YXNtRXhwb3J0c1siRyJdO01vZHVsZVsiX3NrYWluZXRfZjMyX3NkcGEiXT1fc2thaW5ldF9mMzJfc2RwYT13YXNtRXhwb3J0c1siSCJdO01vZHVsZVsiX3NrYWluZXRfZjMyX2NvbnYiXT1fc2thaW5ldF9mMzJfY29udj13YXNtRXhwb3J0c1siSSJdfXZhciB3YXNtSW1wb3J0cz17YTpfZW1zY3JpcHRlbl9yZXNpemVfaGVhcH07ZnVuY3Rpb24gYXBwbHlTaWduYXR1cmVDb252ZXJzaW9ucyh3YXNtRXhwb3J0cyl7d2FzbUV4cG9ydHM9T2JqZWN0LmFzc2lnbih7fSx3YXNtRXhwb3J0cyk7dmFyIG1ha2VXcmFwcGVyX3BwPWY9PmEwPT5mKGEwKT4+PjA7dmFyIG1ha2VXcmFwcGVyX3A9Zj0+KCk9PmYoKT4+PjA7d2FzbUV4cG9ydHNbImsiXT1tYWtlV3JhcHBlcl9wcCh3YXNtRXhwb3J0c1siayJdKTt3YXNtRXhwb3J0c1siX2Vtc2NyaXB0ZW5fc3RhY2tfYWxsb2MiXT1tYWtlV3JhcHBlcl9wcCh3YXNtRXhwb3J0c1siX2Vtc2NyaXB0ZW5fc3RhY2tfYWxsb2MiXSk7d2FzbUV4cG9ydHNbImVtc2NyaXB0ZW5fc3RhY2tfZ2V0X2N1cnJlbnQiXT1tYWtlV3JhcHBlcl9wKHdhc21FeHBvcnRzWyJlbXNjcmlwdGVuX3N0YWNrX2dldF9jdXJyZW50Il0pO3JldHVybiB3YXNtRXhwb3J0c31mdW5jdGlvbiBydW4oKXtwcmVSdW4oKTtmdW5jdGlvbiBkb1J1bigpe01vZHVsZVsiY2FsbGVkUnVuIl09dHJ1ZTtpZihBQk9SVClyZXR1cm47aW5pdFJ1bnRpbWUoKTtyZWFkeVByb21pc2VSZXNvbHZlPy4oTW9kdWxlKTtNb2R1bGVbIm9uUnVudGltZUluaXRpYWxpemVkIl0/LigpO3Bvc3RSdW4oKX1pZihNb2R1bGVbInNldFN0YXR1cyJdKXtNb2R1bGVbInNldFN0YXR1cyJdKCJSdW5uaW5nLi4uIik7c2V0VGltZW91dCgoKT0+e3NldFRpbWVvdXQoKCk9Pk1vZHVsZVsic2V0U3RhdHVzIl0oIiIpLDEpO2RvUnVuKCl9LDEpfWVsc2V7ZG9SdW4oKX19dmFyIHdhc21FeHBvcnRzO3dhc21FeHBvcnRzPWF3YWl0IChjcmVhdGVXYXNtKCkpO3J1bigpO2lmKHJ1bnRpbWVJbml0aWFsaXplZCl7bW9kdWxlUnRuPU1vZHVsZX1lbHNle21vZHVsZVJ0bj1uZXcgUHJvbWlzZSgocmVzb2x2ZSxyZWplY3QpPT57cmVhZHlQcm9taXNlUmVzb2x2ZT1yZXNvbHZlO3JlYWR5UHJvbWlzZVJlamVjdD1yZWplY3R9KX0KO3JldHVybiBtb2R1bGVSdG59ZXhwb3J0IGRlZmF1bHQgY3JlYXRlU2thaW5ldEtlcm5lbHM7Cg==" + const val ST_WASM: String = "AGFzbQEAAAABtgEUYAN/f38AYAh/f39/f39/fwBgAX8Bf2ABfQF9YAl/f39/f39/f38AYAx/f39/f39/f39/f38AYAF8AXxgAABgBX9/f39/AGAGf39/f39/AGAEf39/fwBgAX8AYAd/f39/f39/AGACf38BfWADf399AX1gFH9/f39/f39/f399f39/f39/f39/AGAGf39/f399AGAHf39/f39/fQBgCX9/fX9/f39/fwBgC39/f39/f39/f39/AAIHAQFhAWEAAgNOTQMAAwILAgQMBAQDAwYGBw0OCAUABQIACQAPEAAAEQAIAAkKAAoAAQASABMAAQABAAEAAQABAAEAAQABAAEAAQABAAABAAEAAQABAAUHBAUBcAEfHwUHAQGACICABAYIAX8BQYDDBAsHiQEiAWICAAFjAE0BZABMAWUABwFmABUBZwATAWgASgFpAEgBagBGAWsABgFsAAUBbQBEAW4AQQFvAD8BcAA9AXEAOwFyADkBcwA3AXQANQF1ADMBdgAxAXcALwF4AC0BeQArAXoAKQFBACcBQgAlAUMAIwFEACIBRQAgAUYAHgFHABsBSAAaAUkAGAkkAQBBAQseQhwUS0lHRUNAPjw6ODY0MjAuLCooJiQhHx0ZFw8WDAEzCqKrA03rAQMCfAF/AX4CfQJAIAC8QRR2Qf8PcSIDQasISQ0AQwAAAAAgAEMAAID/Ww0BGiADQfgPTwRAIAAgAJIPCyAAQxdysUJeBEAjAEEQayIDQwAAAHA4AgwgAyoCDEMAAABwlA8LIABDtPHPwl1FDQAjAEEQayIDQwAAABA4AgwgAyoCDEMAAAAQlA8LQaA7KwMAQZg7KwMAIAC7oiIBIAFBkDsrAwAiAaAiAiABoaEiAaJBqDsrAwCgIAEgAaKiQbA7KwMAIAGiRAAAAAAAAPA/oKAgAr0iBEIvhiAEp0EfcUEDdCkD8Dh8v6K2CwuVAwEFfwJAIAJBAEoEQCACQf8DTQ0BQdw9KAIAQd+33poBRwRAEA9B3D1B37femgE2AgALAkBBgD8oAgAiAwRAQfw+KAIARQ0BCwwCC0HwPiADNgIAQeA+IAI2AgBB3D4gATYCAEHYPiAANgIAQfw+QQE2AgBB5D4gA0EBaiIDNgIAQew+Qew+KAIAQQFqNgIAQeg+QQA2AgBB9D4oAgAEQEHkPigCACEDCyACQeg+KAIAIgRKBEAgA0EBdCEGA0BB6D5BwAAgAiAEayAGbSIDIANBwABMGyAEaiIFQeg+KAIAIgMgAyAERiIHGzYCACAHBH8gASAEIAUgAiACIAVKGyAAEQAAQeg+KAIABSADCyIEIAJIDQALC0HwPigCAEEATCEAQQAhBAJAA0AgAA0BQfA+KAIAQQBMIgENASABDQEgAQ0BIARBBGoiBEGAgMAARw0AC0H4PkEBNgIAQfA+KAIAQQBKBEADQEHwPigCAEEASg0ACwtB+D5BADYCAAtB/D5BADYCAAsPCyABQQAgAiAAEQAAC6YBAgF9AX8CQCAAiyIBvCICQdW+svgDTwRAIAJBgYCAiQRPBEBDAAAAACABlUMAAIA/kiEBDAILQwAAgD9DAAAAQCABIAGSEAtDAAAAQJKVkyEBDAELIAJB+YqL9ANPBEAgASABkhALIgEgAUMAAABAkpUhAQwBCyACQYCAgARJDQAgAUMAAADAlBALIgGMIAFDAAAAQJKVIQELIAGMIAEgALxBAEgbC08BAn9B2D0oAgAiASAAQQdqQXhxIgJqIQACQCACQQAgACABTRtFBEAgAD8AQRB0TQ0BIAAQAA0BC0GEP0EwNgIAQX8PC0HYPSAANgIAIAEL4gsBCH8CQCAARQ0AIABBCGsiAyAAQQRrKAIAIgJBeHEiAGohBQJAIAJBAXENACACQQJxRQ0BIAMgAygCACIEayIDQZg/KAIASQ0BIAAgBGohAAJAAkACQEGcPygCACADRwRAIAMoAgwhASAEQf8BTQRAIAEgAygCCCICRw0CQYg/QYg/KAIAQX4gBEEDdndxNgIADAULIAMoAhghByABIANHBEAgAygCCCICIAE2AgwgASACNgIIDAQLIAMoAhQiAgR/IANBFGoFIAMoAhAiAkUNAyADQRBqCyEEA0AgBCEGIAIiAUEUaiEEIAEoAhQiAg0AIAFBEGohBCABKAIQIgINAAsgBkEANgIADAMLIAUoAgQiAkEDcUEDRw0DQZA/IAA2AgAgBSACQX5xNgIEIAMgAEEBcjYCBCAFIAA2AgAPCyACIAE2AgwgASACNgIIDAILQQAhAQsgB0UNAAJAIAMoAhwiBEECdCICKAK4QSADRgRAIAJBuMEAaiABNgIAIAENAUGMP0GMPygCAEF+IAR3cTYCAAwCCwJAIAMgBygCEEYEQCAHIAE2AhAMAQsgByABNgIUCyABRQ0BCyABIAc2AhggAygCECICBEAgASACNgIQIAIgATYCGAsgAygCFCICRQ0AIAEgAjYCFCACIAE2AhgLIAMgBU8NACAFKAIEIgRBAXFFDQACQAJAAkACQCAEQQJxRQRAQaA/KAIAIAVGBEBBoD8gAzYCAEGUP0GUPygCACAAaiIANgIAIAMgAEEBcjYCBCADQZw/KAIARw0GQZA/QQA2AgBBnD9BADYCAA8LQZw/KAIAIgcgBUYEQEGcPyADNgIAQZA/QZA/KAIAIABqIgA2AgAgAyAAQQFyNgIEIAAgA2ogADYCAA8LIARBeHEgAGohACAFKAIMIQEgBEH/AU0EQCAFKAIIIgIgAUYEQEGIP0GIPygCAEF+IARBA3Z3cTYCAAwFCyACIAE2AgwgASACNgIIDAQLIAUoAhghCCABIAVHBEAgBSgCCCICIAE2AgwgASACNgIIDAMLIAUoAhQiAgR/IAVBFGoFIAUoAhAiAkUNAiAFQRBqCyEEA0AgBCEGIAIiAUEUaiEEIAEoAhQiAg0AIAFBEGohBCABKAIQIgINAAsgBkEANgIADAILIAUgBEF+cTYCBCADIABBAXI2AgQgACADaiAANgIADAMLQQAhAQsgCEUNAAJAIAUoAhwiBEECdCICKAK4QSAFRgRAIAJBuMEAaiABNgIAIAENAUGMP0GMPygCAEF+IAR3cTYCAAwCCwJAIAUgCCgCEEYEQCAIIAE2AhAMAQsgCCABNgIUCyABRQ0BCyABIAg2AhggBSgCECICBEAgASACNgIQIAIgATYCGAsgBSgCFCICRQ0AIAEgAjYCFCACIAE2AhgLIAMgAEEBcjYCBCAAIANqIAA2AgAgAyAHRw0AQZA/IAA2AgAPCyAAQf8BTQRAIABB+AFxQbA/aiECAn9BiD8oAgAiBEEBIABBA3Z0IgBxRQRAQYg/IAAgBHI2AgAgAgwBCyACKAIICyEAIAIgAzYCCCAAIAM2AgwgAyACNgIMIAMgADYCCA8LQR8hASAAQf///wdNBEAgAEEmIABBCHZnIgJrdkEBcSACQQF0a0E+aiEBCyADIAE2AhwgA0IANwIQIAFBAnRBuMEAaiEEAn8CQAJ/QYw/KAIAIgZBASABdCICcUUEQEGMPyACIAZyNgIAIAQgAzYCAEEYIQFBCAwBCyAAQRkgAUEBdmtBACABQR9HG3QhASAEKAIAIQQDQCAEIgIoAgRBeHEgAEYNAiABQR12IQQgAUEBdCEBIAIgBEEEcWoiBigCECIEDQALIAYgAzYCEEEYIQEgAiEEQQgLIQAgAyICDAELIAIoAggiBCADNgIMIAIgAzYCCEEYIQBBCCEBQQALIQYgASADaiAENgIAIAMgAjYCDCAAIANqIAY2AgBBqD9BqD8oAgBBAWsiAEF/IAAbNgIACwvsJwELfyMAQRBrIgokAAJAAkACQAJAAkACQAJAAkACQAJAIABB9AFNBEBBiD8oAgAiBEEQIABBC2pB+ANxIABBC0kbIgZBA3YiAHYiAUEDcQRAAkAgAUF/c0EBcSAAaiIDQQN0IgFBsD9qIgAgASgCuD8iAigCCCIFRgRAQYg/IARBfiADd3E2AgAMAQsgBSAANgIMIAAgBTYCCAsgAkEIaiEAIAIgAUEDcjYCBCABIAJqIgEgASgCBEEBcjYCBAwLCyAGQZA/KAIAIghNDQEgAQRAAkBBAiAAdCICQQAgAmtyIAEgAHRxaCIDQQN0IgFBsD9qIgIgASgCuD8iACgCCCIFRgRAQYg/IARBfiADd3EiBDYCAAwBCyAFIAI2AgwgAiAFNgIICyAAIAZBA3I2AgQgACAGaiIHIAEgBmsiBUEBcjYCBCAAIAFqIAU2AgAgCARAIAhBeHFBsD9qIQFBnD8oAgAhAgJ/IARBASAIQQN2dCIDcUUEQEGIPyADIARyNgIAIAEMAQsgASgCCAshAyABIAI2AgggAyACNgIMIAIgATYCDCACIAM2AggLIABBCGohAEGcPyAHNgIAQZA/IAU2AgAMCwtBjD8oAgAiC0UNASALaEECdCgCuEEiAigCBEF4cSAGayEDIAIhAQNAAkAgASgCECIARQRAIAEoAhQiAEUNAQsgACgCBEF4cSAGayIBIAMgASADSSIBGyEDIAAgAiABGyECIAAhAQwBCwsgAigCGCEJIAIgAigCDCIARwRAIAIoAggiASAANgIMIAAgATYCCAwKCyACKAIUIgEEfyACQRRqBSACKAIQIgFFDQMgAkEQagshBQNAIAUhByABIgBBFGohBSAAKAIUIgENACAAQRBqIQUgACgCECIBDQALIAdBADYCAAwJC0F/IQYgAEG/f0sNACAAQQtqIgFBeHEhBkGMPygCACIHRQ0AQR8hCEEAIAZrIQMgAEH0//8HTQRAIAZBJiABQQh2ZyIAa3ZBAXEgAEEBdGtBPmohCAsCQAJAAkAgCEECdCgCuEEiAUUEQEEAIQAMAQtBACEAIAZBGSAIQQF2a0EAIAhBH0cbdCECA0ACQCABKAIEQXhxIAZrIgQgA08NACABIQUgBCIDDQBBACEDIAEhAAwDCyAAIAEoAhQiBCAEIAEgAkEddkEEcWooAhAiAUYbIAAgBBshACACQQF0IQIgAQ0ACwsgACAFckUEQEEAIQVBAiAIdCIAQQAgAGtyIAdxIgBFDQMgAGhBAnQoArhBIQALIABFDQELA0AgACgCBEF4cSAGayICIANJIQEgAiADIAEbIQMgACAFIAEbIQUgACgCECIBBH8gAQUgACgCFAsiAA0ACwsgBUUNACADQZA/KAIAIAZrTw0AIAUoAhghCCAFIAUoAgwiAEcEQCAFKAIIIgEgADYCDCAAIAE2AggMCAsgBSgCFCIBBH8gBUEUagUgBSgCECIBRQ0DIAVBEGoLIQIDQCACIQQgASIAQRRqIQIgACgCFCIBDQAgAEEQaiECIAAoAhAiAQ0ACyAEQQA2AgAMBwsgBkGQPygCACIFTQRAQZw/KAIAIQACQCAFIAZrIgFBEE8EQCAAIAZqIgIgAUEBcjYCBCAAIAVqIAE2AgAgACAGQQNyNgIEDAELIAAgBUEDcjYCBCAAIAVqIgEgASgCBEEBcjYCBEEAIQJBACEBC0GQPyABNgIAQZw/IAI2AgAgAEEIaiEADAkLIAZBlD8oAgAiAkkEQEGUPyACIAZrIgE2AgBBoD9BoD8oAgAiACAGaiICNgIAIAIgAUEBcjYCBCAAIAZBA3I2AgQgAEEIaiEADAkLQQAhACAGQS9qIgMCf0HgwgAoAgAEQEHowgAoAgAMAQtB7MIAQn83AgBB5MIAQoCggICAgAQ3AgBB4MIAIApBDGpBcHFB2KrVqgVzNgIAQfTCAEEANgIAQcTCAEEANgIAQYAgCyIBaiIEQQAgAWsiB3EiASAGTQ0IQcDCACgCACIFBEBBuMIAKAIAIgggAWoiCSAITQ0JIAUgCUkNCQsCQEHEwgAtAABBBHFFBEACQAJAAkACQEGgPygCACIFBEBByMIAIQADQCAAKAIAIgggBU0EQCAFIAggACgCBGpJDQMLIAAoAggiAA0ACwtBABAEIgJBf0YNAyABIQRB5MIAKAIAIgBBAWsiBSACcQRAIAEgAmsgAiAFakEAIABrcWohBAsgBCAGTQ0DQcDCACgCACIABEBBuMIAKAIAIgUgBGoiByAFTQ0EIAAgB0kNBAsgBBAEIgAgAkcNAQwFCyAEIAJrIAdxIgQQBCICIAAoAgAgACgCBGpGDQEgAiEACyAAQX9GDQEgBkEwaiAETQRAIAAhAgwEC0HowgAoAgAiAiADIARrakEAIAJrcSICEARBf0YNASACIARqIQQgACECDAMLIAJBf0cNAgtBxMIAQcTCACgCAEEEcjYCAAsgARAEIQJBABAEIQAgAkF/Rg0FIABBf0YNBSAAIAJNDQUgACACayIEIAZBKGpNDQULQbjCAEG4wgAoAgAgBGoiADYCAEG8wgAoAgAgAEkEQEG8wgAgADYCAAsCQEGgPygCACIDBEBByMIAIQADQCACIAAoAgAiASAAKAIEIgVqRg0CIAAoAggiAA0ACwwEC0GYPygCACIAQQAgACACTRtFBEBBmD8gAjYCAAtBACEAQczCACAENgIAQcjCACACNgIAQag/QX82AgBBrD9B4MIAKAIANgIAQdTCAEEANgIAA0AgAEEDdCIBIAFBsD9qIgU2Arg/IAEgBTYCvD8gAEEBaiIAQSBHDQALQZQ/IARBKGsiAEF4IAJrQQdxIgFrIgU2AgBBoD8gASACaiIBNgIAIAEgBUEBcjYCBCAAIAJqQSg2AgRBpD9B8MIAKAIANgIADAQLIAIgA00NAiABIANLDQIgACgCDEEIcQ0CIAAgBCAFajYCBEGgPyADQXggA2tBB3EiAGoiATYCAEGUP0GUPygCACAEaiICIABrIgA2AgAgASAAQQFyNgIEIAIgA2pBKDYCBEGkP0HwwgAoAgA2AgAMAwtBACEADAYLQQAhAAwEC0GYPygCACACSwRAQZg/IAI2AgALIAIgBGohBUHIwgAhAAJAA0AgBSAAKAIAIgFHBEAgACgCCCIADQEMAgsLIAAtAAxBCHFFDQMLQcjCACEAA0ACQCAAKAIAIgEgA00EQCADIAEgACgCBGoiBUkNAQsgACgCCCEADAELC0GUPyAEQShrIgBBeCACa0EHcSIBayIHNgIAQaA/IAEgAmoiATYCACABIAdBAXI2AgQgACACakEoNgIEQaQ/QfDCACgCADYCACADIAVBJyAFa0EHcWpBL2siACAAIANBEGpJGyIBQRs2AgQgAUHQwgApAgA3AhAgAUHIwgApAgA3AghB0MIAIAFBCGo2AgBBzMIAIAQ2AgBByMIAIAI2AgBB1MIAQQA2AgAgAUEYaiEAA0AgAEEHNgIEIABBCGogAEEEaiEAIAVJDQALIAEgA0YNACABIAEoAgRBfnE2AgQgAyABIANrIgJBAXI2AgQgASACNgIAAn8gAkH/AU0EQCACQfgBcUGwP2ohAAJ/QYg/KAIAIgFBASACQQN2dCICcUUEQEGIPyABIAJyNgIAIAAMAQsgACgCCAshASAAIAM2AgggASADNgIMQQwhAkEIDAELQR8hACACQf///wdNBEAgAkEmIAJBCHZnIgBrdkEBcSAAQQF0a0E+aiEACyADIAA2AhwgA0IANwIQIABBAnRBuMEAaiEBAkACQEGMPygCACIFQQEgAHQiBHFFBEBBjD8gBCAFcjYCACABIAM2AgAMAQsgAkEZIABBAXZrQQAgAEEfRxt0IQAgASgCACEFA0AgBSIBKAIEQXhxIAJGDQIgAEEddiEFIABBAXQhACABIAVBBHFqIgQoAhAiBQ0ACyAEIAM2AhALIAMgATYCGEEIIQIgAyIBIQBBDAwBCyABKAIIIgAgAzYCDCABIAM2AgggAyAANgIIQQAhAEEYIQJBDAsgA2ogATYCACACIANqIAA2AgALQZQ/KAIAIgAgBk0NAEGUPyAAIAZrIgE2AgBBoD9BoD8oAgAiACAGaiICNgIAIAIgAUEBcjYCBCAAIAZBA3I2AgQgAEEIaiEADAQLQYQ/QTA2AgBBACEADAMLIAAgAjYCACAAIAAoAgQgBGo2AgQgAkF4IAJrQQdxaiIIIAZBA3I2AgQgAUF4IAFrQQdxaiIEIAYgCGoiA2shBwJAQaA/KAIAIARGBEBBoD8gAzYCAEGUP0GUPygCACAHaiIANgIAIAMgAEEBcjYCBAwBC0GcPygCACAERgRAQZw/IAM2AgBBkD9BkD8oAgAgB2oiADYCACADIABBAXI2AgQgACADaiAANgIADAELIAQoAgQiAEEDcUEBRgRAIABBeHEhCSAEKAIMIQICQCAAQf8BTQRAIAQoAggiASACRgRAQYg/QYg/KAIAQX4gAEEDdndxNgIADAILIAEgAjYCDCACIAE2AggMAQsgBCgCGCEGAkAgAiAERwRAIAQoAggiACACNgIMIAIgADYCCAwBCwJAIAQoAhQiAAR/IARBFGoFIAQoAhAiAEUNASAEQRBqCyEBA0AgASEFIAAiAkEUaiEBIAAoAhQiAA0AIAJBEGohASACKAIQIgANAAsgBUEANgIADAELQQAhAgsgBkUNAAJAIAQoAhwiAEECdCIBKAK4QSAERgRAIAFBuMEAaiACNgIAIAINAUGMP0GMPygCAEF+IAB3cTYCAAwCCwJAIAQgBigCEEYEQCAGIAI2AhAMAQsgBiACNgIUCyACRQ0BCyACIAY2AhggBCgCECIABEAgAiAANgIQIAAgAjYCGAsgBCgCFCIARQ0AIAIgADYCFCAAIAI2AhgLIAcgCWohByAEIAlqIgQoAgQhAAsgBCAAQX5xNgIEIAMgB0EBcjYCBCADIAdqIAc2AgAgB0H/AU0EQCAHQfgBcUGwP2ohAAJ/QYg/KAIAIgFBASAHQQN2dCICcUUEQEGIPyABIAJyNgIAIAAMAQsgACgCCAshASAAIAM2AgggASADNgIMIAMgADYCDCADIAE2AggMAQtBHyECIAdB////B00EQCAHQSYgB0EIdmciAGt2QQFxIABBAXRrQT5qIQILIAMgAjYCHCADQgA3AhAgAkECdEG4wQBqIQACQAJAQYw/KAIAIgFBASACdCIFcUUEQEGMPyABIAVyNgIAIAAgAzYCAAwBCyAHQRkgAkEBdmtBACACQR9HG3QhAiAAKAIAIQEDQCABIgAoAgRBeHEgB0YNAiACQR12IQEgAkEBdCECIAAgAUEEcWoiBSgCECIBDQALIAUgAzYCEAsgAyAANgIYIAMgAzYCDCADIAM2AggMAQsgACgCCCIBIAM2AgwgACADNgIIIANBADYCGCADIAA2AgwgAyABNgIICyAIQQhqIQAMAgsCQCAIRQ0AAkAgBSgCHCIBQQJ0IgIoArhBIAVGBEAgAkG4wQBqIAA2AgAgAA0BQYw/IAdBfiABd3EiBzYCAAwCCwJAIAUgCCgCEEYEQCAIIAA2AhAMAQsgCCAANgIUCyAARQ0BCyAAIAg2AhggBSgCECIBBEAgACABNgIQIAEgADYCGAsgBSgCFCIBRQ0AIAAgATYCFCABIAA2AhgLAkAgA0EPTQRAIAUgAyAGaiIAQQNyNgIEIAAgBWoiACAAKAIEQQFyNgIEDAELIAUgBkEDcjYCBCAFIAZqIgQgA0EBcjYCBCADIARqIAM2AgAgA0H/AU0EQCADQfgBcUGwP2ohAAJ/QYg/KAIAIgFBASADQQN2dCICcUUEQEGIPyABIAJyNgIAIAAMAQsgACgCCAshASAAIAQ2AgggASAENgIMIAQgADYCDCAEIAE2AggMAQtBHyEAIANB////B00EQCADQSYgA0EIdmciAGt2QQFxIABBAXRrQT5qIQALIAQgADYCHCAEQgA3AhAgAEECdEG4wQBqIQECQAJAIAdBASAAdCICcUUEQEGMPyACIAdyNgIAIAEgBDYCACAEIAE2AhgMAQsgA0EZIABBAXZrQQAgAEEfRxt0IQAgASgCACEBA0AgASICKAIEQXhxIANGDQIgAEEddiEBIABBAXQhACACIAFBBHFqIgcoAhAiAQ0ACyAHIAQ2AhAgBCACNgIYCyAEIAQ2AgwgBCAENgIIDAELIAIoAggiACAENgIMIAIgBDYCCCAEQQA2AhggBCACNgIMIAQgADYCCAsgBUEIaiEADAELAkAgCUUNAAJAIAIoAhwiAUECdCIFKAK4QSACRgRAIAVBuMEAaiAANgIAIAANAUGMPyALQX4gAXdxNgIADAILAkAgAiAJKAIQRgRAIAkgADYCEAwBCyAJIAA2AhQLIABFDQELIAAgCTYCGCACKAIQIgEEQCAAIAE2AhAgASAANgIYCyACKAIUIgFFDQAgACABNgIUIAEgADYCGAsCQCADQQ9NBEAgAiADIAZqIgBBA3I2AgQgACACaiIAIAAoAgRBAXI2AgQMAQsgAiAGQQNyNgIEIAIgBmoiBSADQQFyNgIEIAMgBWogAzYCACAIBEAgCEF4cUGwP2ohAEGcPygCACEBAn9BASAIQQN2dCIHIARxRQRAQYg/IAQgB3I2AgAgAAwBCyAAKAIICyEEIAAgATYCCCAEIAE2AgwgASAANgIMIAEgBDYCCAtBnD8gBTYCAEGQPyADNgIACyACQQhqIQALIApBEGokACAAC5sBAQF/IwBBIGsiCSQAAkAgBkEATA0AIAdBAEwNACAIQQBMBEAgBiAHbCIAQQBMDQEgAEECdCIARQ0BIAQgBUECdGpBACAA/AsADAELIAkgCDYCHCAJIAc2AhggCSAGNgIUIAkgBCAFQQJ0ajYCECAJIAIgA0ECdGo2AgwgCSAAIAFBAnRqNgIIQQIgCUEIaiAHEAILIAlBIGokAAvkCQMYfwF9AXsCQCABIAJODQAgBSAGTg0AIAQgA0EEaiINTgRAIAQgAyADQQhqIgggBEEBaiIHIAcgCEgbIANrQQVrIg9BfHFqIghBBWoiByAEIAdKGyIHQQJ0IA9BAnRBcHFrIRUgByAIa0EEayIWQXxxIRAgD0EEcSEZIA9BAnZBAWpB/v///wdxIRogA0ECdCEXIAAoAhAhByABIRIDQCAAKAIUIgkgASARamxBAnQiCCAAKAIIIgwgFWpqIRsgDCAXaiAIakEQaiEcIAAoAgQgBSAHbEECdGohCiAMIAkgEmxBAnRqIQ4gACgCACAAKAIMIBJsQQJ0aiEdIAUhFANAIB0gFEECdGoqAgAiH/0TISBBACEMIAMhCEEAIRMgDSIHIQkgD0EETwRAA0AgDiAIQQJ0IglqIgggCSAKav0AAAAgIP3mASAI/QAAAP3kAf0LAAAgDiAHQQJ0IglqIgggCSAKav0AAAAgIP3mASAI/QAAAP3kAf0LAAAgDCILQQJqIQwgB0EEaiEIIAdBCGoiCSEHIBNBAmoiEyAaRw0ACyALQQFyIQsLIBlFBEAgDiAIQQJ0IghqIgcgCCAKav0AAAAgIP3mASAH/QAAAP3kAf0LAAAgDCELIAkhCAsCQCAEIAhMDQACQAJAIBZBCEkNACAcIAtBBHQiB2ogCiAVaiAHakkEQCAKIBdqIAdqQRBqIAcgG2pJDQELIAggEGohB0EAIQwDQCAOIAggDGpBAnQiE2oiCSAJ/QACACAKIBNq/QACACAg/eYB/eQB/QsCACAMQQRqIgwgEEcNAAsgECAWRw0BDAILIAghBwsDQCAOIAdBAnQiCWoiCCAIKgIAIAkgCmoqAgAgH5SSOAIAIAdBAWoiByAESA0ACwsgCiAAKAIQIgdBAnRqIQogFEEBaiIUIAZHDQALIBFBAWohESASQQFqIhIgAkcNAAsMAQsgAyAETg0AIAAoAgQiCSAAKAIQIh4gBWwiB0ECdGohDCAAKAIMIRUgACgCACEWIAkgBCAHaiAeIAYgBUF/c2psakECdGohFyAAKAIUIg5BAnQhGSADIAQgA2siD0F8cSIQaiEIIAAoAggiESAEIAEgDmwiAGpBAnRqIRogESAAIANqQQJ0aiEbIAkgAyAHakECdGohHEEAIQAgHkGAgICAAnFBAEchHSAPQQRJIRMDQCARIAEgDmxBAnRqIRggFiABIBVsQQJ0aiESIBsgACAZbCIHaiAXSSAcIAcgGmpJcSAdciATciEUIAUhCSAMIQoDQCASIAlBAnRqKgIAIR8gAyEHAkAgFEUEQCAf/RMhIEEAIQcDQCAYIAMgB2pBAnQiDWoiCyAL/QACACAKIA1q/QACACAg/eYB/eQB/QsCACAHQQRqIgcgEEcNAAsgCCEHIA8gEEYNAQsgB0EBaiELIAQgB2tBAXEEQCAYIAdBAnQiDWoiByAHKgIAIAogDWoqAgAgH5SSOAIAIAshBwsgBCALRg0AA0AgGCAHQQJ0Ig1qIgsgCyoCACAKIA1qKgIAIB+UkjgCACAYIA1BBGoiDWoiCyALKgIAIAogDWoqAgAgH5SSOAIAIAdBAmoiByAERw0ACwsgCiAeQQJ0aiEKIAlBAWoiCSAGRw0ACyAAQQFqIQAgAUEBaiIBIAJHDQALCwvdBAMCewd/AX0jAEEgayILJAACQCAEQQBMDQAgBUEATA0AIAQQBiEMIARBCHYiDkECdBAGIQ0CQCAMRQ0AIA1FDQAgBEGAAk8EQCAAIAFBAnRqIRBBACEAA0AgAEEIdCERIBAgAEEKdGohD/0MAAAAAAAAAAAAAAAAAAAAACEJQQAhAQNAIA8gAUECdGoiBP0AAjD94AEgBP0AAiD94AEgBP0AAhD94AEgBP0AAgD94AEgCf3rAf3rAf3rAf3rASEJIAFBEGoiAUGAAkcNAAsgDCARaiEBIA0gAEECdGoCfSAJIAkgCf0NCAkKCwwNDg8AAQIDAAECA/3rASIJIAkgCf0NBAUGBwABAgMAAQIDAAECA/3rAf0fACISQwAAAABcBEBDAACAPyASlf0TIQpBACEEA0AgASAEaiAPIARBAnRq/QACAP0MAAD+QgAA/kIAAP5CAAD+Qv3mASAK/eYBIgn9HwCQ/AD9ESAJ/R8BkPwA/RwBIAn9HwKQ/AD9HAIgCf0fA5D8AP0cA/0Mgf///4H///+B////gf////24Af0MfwAAAH8AAAB/AAAAfwAAAP22ASAJ/Q0ABAgMAAAAAAAAAAAAAAAA/VoAAAAgBEEEaiIEQYACRw0ACyASQwQCATyUDAELIAFBAEGAAvwLAEMAAAAACzgCACAAQQFqIgAgDkcNAAsLIAtBADYCHCALIAU2AhggCyAONgIUIAsgDTYCDCALIAw2AgggCyACIANqNgIEIAsgBiAHQQJ0ajYCECAIIAtBBGogBRACCyAMEAUgDRAFCyALQSBqJAALqAICAn8BeyMAQSBrIgkkAAJAIAVBAEwNACAEQQBMBEAgBUECdCIARQ0BIAYgB0ECdGpBACAA/AsADAELIAkgBTYCHCAJIARBBXYiCjYCGCAJIAIgA2o2AgggCSAGIAdBAnRqNgIQIAkgACABQQJ0aiICNgIMIApBAnQQBiIBRQ0AIARBH0sEQEEAIQcDQCABIAdBAnRqIAIgB0EHdGoiAP0AAgAgAP0AAnD95AEgAP0AAkD95AEgAP0AAiAgAP0AAmD95AH95AEgAP0AAhAgAP0AAlD95AEgAP0AAjD95AH95AEiC/0fACAL/R8BkiAL/R8CkiAL/R8DkjgCACAHQQFqIgcgCkcNAAsLIAkgATYCFCAIIAlBCGogBRACIAEQBQsgCUEgaiQAC6IEAgJ/BH0CQAJAAkACfQJAIAC8IgJB/////wdxIgFBxPDWjARPBEAgAUGAgID8B0sNBSACQQBIBEBDAACAvw8LIAFBmOTFlQRJDQEgAEMAAAB/lA8LIAFBmeTF9QNJDQIgAUGRq5T8A0sNACACQQBOBEBBASEBQ9H3FzchAyAAQ4BxMb+SDAILQX8hAUPR9xe3IQMgAEOAcTE/kgwBCyAAQzuquD+UQwAAAD8gAJiS/AAiAbIiBEPR9xc3lCEDIAAgBEOAcTG/lJILIgAgACADkyIAkyADkyEDDAELIAFBgICAmANJDQFBACEBCyAAIABDAAAAP5QiBZQiBCAEIARDEDDPOpRDaIgIvZKUQwAAgD+SIgZDAABAQCAGIAWUkyIFk0MAAMBAIAAgBZSTlZQhBSABRQRAIAAgACAFlCAEk5MPCyAAIAUgA5OUIAOTIASTIQMCQAJAAkAgAUEBag4DAAIBAgsgACADk0MAAAA/lEMAAAC/kg8LIABDAACAvl0EQCADIABDAAAAP5KTQwAAAMCUDwsgACADkyIAIACSQwAAgD+SDwsgAUEXdCICQYCAgPwDar4hBCABQTlPBEAgACADk0MAAIA/kiIAIACSQwAAAH+UIAAgBJQgAUGAAUYbQwAAgL+SDwtBgICA/AMgAmu+IQUgAUEWTQR9QwAAgD8gBZMgACADk5IFIAAgAyAFkpNDAACAP5ILIASUIQALIAALgAICAn8CfCAAvCIBQYCAgPwDRgRAQwAAAAAPCwJAIAFBgICA/AdrQf///4d4TQRAIAFBAXQiAkUEQCMAQRBrIgFDAACAvzgCDCABKgIMQwAAAACVDwsgAUGAgID8B0YNASACQYCAgHhJIAFBAE5xRQRAIAAgAJMiACAAlQ8LIABDAAAAS5S8QYCAgNwAayEBC0HAPSsDACABIAFBgIDM+QNrIgFBgICAfHFrvrsgAUEPdkHwAXEiAisDuDuiRAAAAAAAAPC/oCIDIAOiIgSiQcg9KwMAIAOiQdA9KwMAoKAgBKIgAUEXdbdBuD0rAwCiIAIrA8A7oCADoKC2IQALIAALswQDA3wDfwJ+AnwCQCAAvUI0iKdB/w9xIgVByQdrQT9JBEAgBSEEDAELIAVByQdJBEAgAEQAAAAAAADwP6APCyAFQYkISQ0ARAAAAAAAAAAAIAC9IgdCgICAgICAgHhRDQEaIAVB/w9PBEAgAEQAAAAAAADwP6APCyAHQgBTBEAjAEEQayIERAAAAAAAAAAQOQMIIAQrAwhEAAAAAAAAABCiDwsjAEEQayIERAAAAAAAAABwOQMIIAQrAwhEAAAAAAAAAHCiDwsgAEGAKCsDAKJBiCgrAwAiAaAiAiABoSIBQZgoKwMAoiABQZAoKwMAoiAAoKAiASABoiIAIACiIAFBuCgrAwCiQbAoKwMAoKIgACABQagoKwMAokGgKCsDAKCiIAK9IgenQQR0QfAPcSIFKwPwKCABoKCgIQEgBSkD+CggB0IthnwhCCAERQRAAnwgB0KAgICACINQBEAgCEKAgICAgICAiD99vyIAIAGiIACgRAAAAAAAAAB/ogwBCyAIQoCAgICAgIDwP3y/IgIgAaIiASACoCIDRAAAAAAAAPA/YwR8IwBBEGsiBCAEQoCAgICAgIAINwMIIAQrAwhEAAAAAAAAEACiOQMIRAAAAAAAAAAAIANEAAAAAAAA8D+gIgAgASACIAOhoCADRAAAAAAAAPA/IAChoKCgRAAAAAAAAPC/oCIAIABEAAAAAAAAAABhGwUgAwtEAAAAAAAAEACiCw8LIAi/IgAgAaIgAKALC/YHAwR8An8BfiAAvSIHQiCIpyIGQf////8HcSIFQYCAwP8HTwRARAAAAAAAAPA/IACjQQEgBkEedkECcWu3oA8LIAVB//+r/wNNBEAgBUH//7/xA00EQCAARAAAAAAAACBAoiAARGnbFIK6bvA/oqBEAAAAAAAAwD+iDwsgACAAIACiIgEgASABIAFErBYAEtbq+L6iRORoZiORone/oKJETxnX21Eqnb+gokQTuRxpfc3Uv6CiRGjbFIK6bsA/oCABIAEgASABIAFEIGGiQkOc0L6iRBAaHCLJXSE/oKJED2vTxCLQdD+gokS6zjZVTKWwP6CiRAnc2s15d9k/oKJEAAAAAAAA8D+go6IgAKAPC0QAAAAAAADwPyEBIAVB///fgARNBEBEAAAAAAAA8D8CfCAFQf//z/8DTQRARAAAAID61MM/IACZRAAAAAAAAPC/oCIAIAAgACAAIAAgAEQ/B5YKOL9hv6JE65WXWTYqoj+gokTsKD49mGO8v6CiROQgUYDKX9Q/oKJE8cO4+0DS17+gokRNs5KtAI3aP6CiRDh19764WWO/oCAAIAAgACAAIAAgAEQdFTVXVIuIP6JEHN1Ra8Ltiz+gokQfNWPnYCbAP6CiRKfpn9lcY7I/oKJEM2/rkvBK4T+gokQj4+4YZj67P6CiRAAAAAAAAPA/oKOhDAELRAAAAAAAAPA/IACZIgEgAaKjIQACfCAFQey2m4AETQRAIAAgACAAIAAgACAAIABEXMKaxu+gI8CiRPLS5FdlUlTAoKJEsqvM61wTZ8CgokRmIiiEsUxkwKCiRI2jy+QKME/AoKJEJuewQQQeJcCgokRgc7rkFjTmv6CiRDVkDWASNIS/oCEDIAAgACAAIAAgACAAIABEYpp07vLvrr+iRJNKSI7vRxpAoKJELK5I7qMoW0CgokQUA3BXIdB6QKCiRGgo7CEZK4RAoKJEcRqK1Q0pe0CgokQh52pSDDVhQKCiRId2cL25pjNAoAwBCyAAIAAgACAAIAAgAEQ/ONybTjh+wKJEklkuamEEkMCgokQo8nUTiOyDwKCiRJjtxUNdFGTAoKJEWplfVQnCMcCgokTehcJwupPpv6CiREpv6DkSNIS/oCEDIAAgACAAIAAgACAARGItcULicDbAokRj55/ndKh9QKCiROY7384Z8qNAoKJEaiSMaLf/qECgokQYUZ0Y6wKYQKCiRAqfGyKuXHRAoKJEkFEdJotWPkCgCyEERAAAAAAAAOK/IAG9QoCAgIBwg78iAiACoqEQDSACIAGhIAEgAqCiIAMgACAEokQAAAAAAADwP6CjoBANoiABowuhIQELIAGaIAEgB0IAUxsLFAEBfyMAQRBrIgAkACAAQRBqJAALtwkDDH8NewN9IwBBQGohBiAALQABIgVBB3YhAiAALQADQQh0IgcgAC0AAnIhBCAFQQh0IgggAC0AAHJB/wdxIQMCfQJAIAVBAnZBH3EiBUEfRwRAIAUNASACQR90viADRQ0CGiADs0MAAIAzlCIbjCAbIAjBQQBIGwwCCyACQR90QYCAgAJBACADG3JBgICA/AdyvgwBCyAFQRd0IAJBH3RyIANBDXRyQYCAgMADar4LIR0gBEH/B3EhAyAEQYCAAnFBD3YhAgJ9AkAgBEEKdkEfcSIEQR9HBEAgBA0BIAJBH3S+IANFDQIaIAOzQwAAgDOUIhuMIBsgB8FBAEgbDAILIAJBH3RBgICAAkEAIAMbckGAgID8B3K+DAELIARBF3QgAkEfdHIgA0ENdHJBgICAwANqvgshGyAGIAD9XAAEIg79DD8/Pz8/Pz8/Pz8/Pz8/Pz/9Tv2JAf2pAf0LBCAgBiAA/VwACCIP/Qw/Pz8/Pz8/Pz8/Pz8/Pz8//U79iQH9qQH9CwQAIAYgAP1cAAwiEEEE/W0gD0EC/W39DDAwMDAwMDAwMDAwMDAwMDD9Tv1Q/YkB/akB/QsEECAGIBD9DA8PDw8PDw8PDw8PDw8PDw/9TiAOQQL9bf0MMDAwMDAwMDAwMDAwMDAwMP1O/VD9iQH9qQH9CwQwIABBMGohCCAAQRBqIQlBACEEA0AgASAEQQh0aiEKIAggBEEFdGohCyABIARBAXQiBUEBciIHQQd0aiEM/QwAAAAAAAAAAAAAAAAAAAAAIg4hD/0MAAAAAAAAAAAAAAAAAAAAACEQ/QwAAAAAAAAAAAAAAAAAAAAAIRFBASEDQQAhAANAIAwgAEECdCINaiIC/QAAMCIWIAAgCWr9AAAAIhIgB/1tQQT9a/0MEBAQEBAQEBAQEBAQEBAQEP1OIAAgC2r9AAAAIhNBBP1t/VAiFP2KASIV/aoB/foB/eYBIAL9AAAgIhcgFf2pAf36Af3mASAC/QAAECIVIBT9iQEiFP2qAf36Af3mASAC/QAAACIYIBT9qQH9+gH95gEgEP3kAf3kAf3kAf3kASEQIAogDWoiAP0AADAiFCASIAX9bUEE/Wv9DBAQEBAQEBAQEBAQEBAQEBD9TiAT/QwPDw8PDw8PDw8PDw8PDw8P/U79UCIS/YoBIhP9qgH9+gH95gEgAP0AACAiGSAT/akB/foB/eYBIAD9AAAQIhMgEv2JASIS/aoB/foB/eYBIAD9AAAAIhogEv2pAf36Af3mASAO/eQB/eQB/eQB/eQBIQ4gFiAXIBUgGCAR/eQB/eQB/eQB/eQBIREgFCAZIBMgGiAP/eQB/eQB/eQB/eQBIQ8gA0EQIQBBACEDDQALIBD9HwAgEP0fAZIgEP0fApIgEP0fA5IgB0ECdCIAIAZBIGoiAmooAgCylCAO/R8AIA79HwGSIA79HwKSIA79HwOSIAIgBUECdCIDaigCALKUkiAdlCAckiAR/R8AIBH9HwGSIBH9HwKSIBH9HwOSIBuUIAAgBmooAgCylCAP/R8AIA/9HwGSIA/9HwKSIA/9HwOSIBuUIAMgBmooAgCylJKTIRwgBEEBaiIEQQRHDQALIBwL1QcEBXsHfwJ9An4jAEFAaiEMIAAtAAEiCkEHdiELIAAtAANBCHQiDSAALQACciEIIApBCHQiDiAALQAAckH/B3EhCQJ9AkAgCkECdkEfcSIKQR9HBEAgCg0BIAtBH3S+IAlFDQIaIAmzQwAAgDOUIg+MIA8gDsFBAEgbDAILIAtBH3RBgICAAkEAIAkbckGAgID8B3K+DAELIApBF3QgC0EfdHIgCUENdHJBgICAwANqvgsgCEH/B3EhCSAIQYCAAnFBD3YhCwJ9AkAgCEEKdkEfcSIIQR9HBEAgCA0BIAtBH3S+IAlFDQIaIAmzQwAAgDOUIhCMIBAgDcFBAEgbDAILIAtBH3RBgICAAkEAIAkbckGAgID8B3K+DAELIAhBF3QgC0EfdHIgCUENdHJBgICAwANqvgshECAMIAD9XAAEIgP9DD8/Pz8/Pz8/Pz8/Pz8/Pz/9Tv2JAf2pAf0LBCAgDCAA/VwACCIE/Qw/Pz8/Pz8/Pz8/Pz8/Pz8//U79iQH9qQH9CwQAIAwgAP1cAAwiBUEE/W0gBEEC/W39DDAwMDAwMDAwMDAwMDAwMDD9Tv1Q/YkB/akB/QsEECAMIAX9DA8PDw8PDw8PDw8PDw8PDw/9TiADQQL9bf0MMDAwMDAwMDAwMDAwMDAwMP1O/VD9iQH9qQH9CwQwIABBEGohCUEAIQADQCAAQQN0IgsgDEEgaiINajQCACAJIABBBXRqIgj9AAAAIgb9DA8PDw8PDw8PDw8PDw8PDw/9TiID/YoBIAEgAEEGdGoiCv0AAAAiBP2IAf26ASAD/YkBIAT9hwH9ugH9rgEgCP0AABAiB/0MDw8PDw8PDw8PDw8PDw8PD/1OIgP9igEgCv0AABAiBf2IAf26Af2uASAD/YkBIAX9hwH9ugH9rgEiA/0bACAD/RsBaiAD/RsCaiAD/RsDaqx+IBF8IAZBBP1tIgP9igEgASAAQQF0QQFyIghBBXRqIgr9AAAAIgb9iAH9ugEgA/2JASAG/YcB/boB/a4BIAdBBP1tIgP9igEgCv0AABAiB/2IAf26Af2uASAD/YkBIAf9hwH9ugH9rgEiA/0bACAD/RsBaiAD/RsCaiAD/RsDaqwgDSAIQQJ0IghqNAIAfnwhESALIAxqNAIAIAX9fP1+IAT9fP1+/a4BIgP9GwAgA/0bAWogA/0bAmogA/0bA2qsfiASfCAIIAxqNAIAIAf9fP1+IAb9fP1+/a4BIgP9GwAgA/0bAWogA/0bAmogA/0bA2qsfnwhEiAAQQFqIgBBBEcNAAsgEbSUIBAgErSUkyAClAvABwIUfwt7AkAgASACTg0AIAMgBE4NACACIAFrIghBA3EhDSAEIANrQQJ0IQkgACgCFCIHQQJ0IQwgACgCCCABIAdsIANqQQJ0aiEKIAEgAmtBfE0EQCAIQXxxIQdBACEIA0AgCUUiC0UEQCAKIAUgDGxqQQAgCfwLAAsgC0UEQCAKIAwgBUEBcmxqQQAgCfwLAAsgC0UEQCAKIAwgBUECcmxqQQAgCfwLAAsgC0UEQCAKIAwgBUEDcmxqQQAgCfwLAAsgBUEEaiEFIAhBBGoiCCAHRw0ACwsgDUUNAANAIAkEQCAKIAUgDGxqQQAgCfwLAAsgBUEBaiEFIAZBAWoiBiANRw0ACwsCQCAAKAIgIgVBAEwNACAEIANrIgZBCG1BA3QgA2ohDiACIAFrIghBBG1BAnQgAWohE0EAIQcgBkEITgRAIAhBA0ohFgNAIAUgB0GAAmoiCCAFIAdrQYACSBshDyADIQ0gFgRAA0AgASELA0AgDUECdCIKIAAoAgggACgCFCIGIAtsQQJ0amoiEP0AABAhGiAQ/QAAACEbIBAgBkECdCIGaiIR/QAAECEcIBH9AAAAIR0gBiARaiIS/QAAECEeIBL9AAAAIR8gBiASaiIU/QAAECEgIBT9AAAAISEgByAPSARAIAAoAgAgACgCDCIGIAtsQQJ0aiIXIAZBAnQiBmoiGCAGaiIJIAZqIQwgCiAAKAIEIAAoAhAiCiAHbEECdGpqIQUgByEGA0AgDCAGQQJ0IhVq/QkCACIZIAX9AAAQIiL95gEgIP3kASEgIBkgBf0AAAAiI/3mASAh/eQBISEgCSAVav0JAgAiGSAi/eYBIB795AEhHiAZICP95gEgH/3kASEfIBUgGGr9CQIAIhkgIv3mASAc/eQBIRwgGSAj/eYBIB395AEhHSAVIBdq/QkCACIZICL95gEgGv3kASEaIBkgI/3mASAb/eQBIRsgBSAKQQJ0aiEFIAZBAWoiBiAPRw0ACwsgECAa/QsAECAQIBv9CwAAIBEgHP0LABAgESAd/QsAACASIB79CwAQIBIgH/0LAAAgFCAg/QsAECAUICH9CwAAIAtBBGoiCyATSA0ACyANQQhqIg0gDkgNAAsLIAAgEyACIAMgDiAHIA8QCCAAIAEgAiAOIAQgByAPEAggACgCICIFIAgiB0oNAAsMAQsDQCAAIBMgAiADIA4gByAFIAdBgAJqIgggBSAHa0GAAkgbIgYQCCAAIAEgAiAOIAQgByAGEAggACgCICIFIAgiB0oNAAsLC4YBAQF/IwBBMGsiDCQAAkAgCUEATA0AIApBAEwNACAMIAo2AiggDCAJNgIkIAwgCDYCICAMIAU2AhwgDCACNgIYIAwgAyAEajYCECAMIAtBACALQQBKGzYCLCAMIAYgB0ECdGo2AhQgDCAAIAFBAnRqNgIMQQQgDEEMaiAKEAILIAxBMGokAAu8CwMSfwJ7AX0jAEGAEGsiDCQAAkAgACgCGCIGQQBMDQAgASACTg0AIAAoAgggAUECdGohBSACIAFrQQJ0IQQgACgCFEECdCEIIAZBBE8EQCAGQfz///8HcSELA0AgBEUiB0UEQCAFIAMgCGxqQQAgBPwLAAsgB0UEQCAFIAggA0EBcmxqQQAgBPwLAAsgB0UEQCAFIAggA0ECcmxqQQAgBPwLAAsgB0UEQCAFIAggA0EDcmxqQQAgBPwLAAsgA0EEaiEDIAlBBGoiCSALRw0ACwsgBkEDcSIHRQ0AA0AgBARAIAUgAyAIbGpBACAE/AsACyADQQFqIQMgCkEBaiIKIAdHDQALCwJAIAAoAiAiC0EATA0AIAZBAUYEQCABQQF0IQ0gACgCCCABQQJ0aiEHIAAoAgAhDkEAIQggAiABayIKQQhIIQ8DQCAAKAIEIAAoAhAgCGxqIA1qIQkgDiAIQQJ0aioCACEXQQAhBSAPRQRAIBf9EyEVQQAhBEEIIQYDQCAHIARBAnRqIgMgFSAJIARBAXRq/QAAACIWIBX9DQABAAECAwABBAUAAQYHAAFBEP2rAf3mASAD/QAAAP3kAf0LAAAgAyAVIBYgFf0NCAkAAQoLAAEMDQABDg8AAUEQ/asB/eYBIAP9AAAQ/eQB/QsAECAGIgUiBEEIaiIGIApMDQALCwJAIAUgCk4NAAJAIAIgASAFamsiBkEESQRAIAUhAwwBCyAFIAZBfHEiC2ohAyAX/RMhFUEAIQQDQCAHIAQgBWoiEEECdGoiESAVIAkgEEEBdGr9BAAAQRD9qwH95gEgEf0AAgD95AH9CwIAIARBBGoiBCALRw0ACyAGIAtGDQELA0AgByADQQJ0aiIEIBcgCSADQQF0ai8AAEEQdL6UIAQqAgCSOAIAIANBAWoiAyAKRw0ACwsgCEEBaiIIIAAoAiBIDQALDAELIAEgAk4NACACIAFrIRADQCALQQBKBEBBgAQgAiABayIOIA5BgAROGyEHQYAEIBAgDUEJdGsiAyADQYAEThsiD0EDcSEKIAFBAXQhEUEAIQggDkEHSiEUA0AgACgCBCAAKAIQIAhsaiARaiEJQQghBEEAIQMgFARAA0AgDCADQQJ0aiIFIAkgA0EBdGr9AAAAIhUgFf0NCAkAAQoLAAEMDQABDg8AAUEQ/asB/QsEECAFIBUgFf0NAAEAAQIDAAEEBQABBgcAAUEQ/asB/QsEACAEIgNBCGoiBCAHTA0ACwsCQCADIAdODQACQCAPIANrIgRBBEkEQCADIQQMAQsgAyAEIAprIhJqIQRBACEFA0AgDCADIAVqIhNBAnRqIAkgE0EBdGr9BAAAQRD9qwH9CwIAIAVBBGoiBSASRw0ACyAKRQ0BCwNAIAwgBEECdGogCSAEQQF0ai8AAEEQdDYCACAEQQFqIgQgB0cNAAsLQQAhCSAGQQBKBEADQCAAKAIIIAAoAhQgCWxBAnRqIAFBAnRqIQYgACgCACAAKAIMIAlsQQJ0aiAIQQJ0aioCACEXQQAhAyAOQQROBEAgF/0TIRVBACEEQQQhBQNAIAYgBEECdCIDaiIEIAMgDGr9AAQAIBX95gEgBP0AAAD95AH9CwAAIAUiAyEEIANBBGoiBSAHTA0ACwsCQCADIAdODQACQCAPIANrIgRBBEkEQCADIQQMAQsgAyAEIAprIgtqIQQgF/0TIRVBACEFA0AgBiADIAVqQQJ0IhJqIhMgE/0AAgAgDCASav0AAgAgFf3mAf3kAf0LAgAgBUEEaiIFIAtHDQALIApFDQELA0AgBiAEQQJ0IgNqIgUgBSoCACADIAxqKgIAIBeUkjgCACAEQQFqIgQgB0cNAAsLIAlBAWoiCSAAKAIYIgZIDQALIAAoAiAhCwsgCEEBaiIIIAtIDQALCyANQQFqIQ0gAUGABGoiASACSA0ACwsgDEGAEGokAAuGAQEBfyMAQTBrIgwkAAJAIAlBAEwNACAKQQBMDQAgDCAKNgIoIAwgCTYCJCAMIAg2AiAgDCAFNgIcIAwgAjYCGCAMIAMgBGo2AhAgDCALQQAgC0EAShs2AiwgDCAGIAdBAnRqNgIUIAwgACABQQJ0ajYCDEEDIAxBDGogChACCyAMQTBqJAALBQEIfwALigcCGH8BfQJAIAEgAk4NACAAKAIIIgVBAEwNACAAKAIUIAAoAhBsIhMgACgCDGwhFCAAKAJMIAAoAkhsIREgACgCICIGQQJ0IQkgBkH+////B3EhFSAGQQFxIRYDQCABIAEgEW0iByARbGsiCCAAKAJMIgptIQQCQCAFQQBMDQAgACgCGCIDQQBMDQAgACgCJCAHbCAAKAIwayEXIAAoAiggBGwgACgCNGshGCAAKAIsIAggBCAKbGtsIAAoAjhrIgdBH3VBACAHa3EiCEECdCEKIAAoAgQgACgCUCABbEECdGohBEEAIQ0DQCADQQBKBEAgACgCACANIBRsQQJ0aiEZQQAhDgNAIAAoAhwhAwJAAkAgACgCPCAObCAXaiIFQQBOBEAgA0EATA0CIAUgACgCDE4NASAZIAUgE2xBAnRqIRpBACEPA0ACQAJAAkAgACgCQCAPbCAYaiIDQQBOBEAgAyAAKAIQSA0BCyAJRQ0BIARBACAJ/AsADAELIBogACgCFCIFIANsQQJ0aiEMIAAoAkQiEEEBRwRAIAZBAEwNAkEAIQNBACESIAZBAUcEQANAQwAAAAAhGyAEAn1DAAAAACADIBBsIAdqIgtBAEgNABpDAAAAACAFIAtMDQAaIAwgC0ECdGoqAgALOAIAAkAgA0EBciAQbCAHaiILQQBIDQAgBSALTA0AIAwgC0ECdGoqAgAhGwsgBCAbOAIEIANBAmohAyAEQQhqIQQgEkECaiISIBVHDQALCyAWRQ0CQwAAAAAhGwJAIAMgEGwgB2oiA0EASA0AIAMgBU4NACAMIANBAnRqKgIAIRsLIAQgGzgCACAEQQRqIQQMAgsgBSAHayIDIAYgAyAGSBsiBSAISiEDAkAgCEEATA0AIApFDQAgBEEAIAr8CwALIAUgCCADGyEDAkAgBSAITA0AIAMgCGtBAnQiBUUNACAEIApqIAwgB0ECdGogCmogBfwKAAALIAMgBk4NACAGIANrQQJ0IgVFDQAgBCADQQJ0akEAIAX8CwALIAQgCWohBAsgD0EBaiIPIAAoAhxIDQALDAILIANBAEwNAQtBACEDA0AgCQRAIARBACAJ/AsACyAEIAlqIQQgA0EBaiIDIAAoAhxIDQALCyAOQQFqIg4gACgCGCIDSA0ACyAAKAIIIQULIA1BAWoiDSAFSA0ACwsgAUEBaiIBIAJHDQALCwv9BgMWfwJ7AX0jAEHgAGsiByQAAkAgBSgCACISQQBMDQAgBSgCCCIMQQBMDQAgBSgCDCIOQQBMDQAgBSgCVCIPIAUoAlAiECAFKAJMbGwiCEEATA0AIAUoAgQhBiAFKAIQIQkgBSgCFCELIAcgBSgCGCIRNgIgIAcgCzYCHCAHIAk2AhggByAENgIQIAcgBiAObSINNgIUIAUoAhwhCiAHIAX9AAIcIhz9CwIkIAcgBf0AAiz9CwI0IAX9AAI8IR0gByAKIA1sIBz9GwFsIBz9GwJsIgo2AlwgByAPNgJYIAcgEDYCVCAHIB39CwJEIBEgCSALbGwiBSANbCERIAwgDk8iFyAKQQBKcSEUIAggDGwhDyAMIA5uIQkgBSAGbCEVIAIEQCAIQfz///8HcSEMIAhBBEkhGEEAIQsDQCAAIAsgFWxBAnRqIRkgAyALIA9sQQJ0aiEaQQAhDQNAIBogCSANbCIFIAhsQQJ0aiEQIAEgBSAKbEECdGohBgJAAkAgFARAIAcgGSANIBFsQQJ0ajYCDEEcIAdBDGogCBACIAZBACAEQQAgEEEAIAkgCCAKEAcMAQsgBkEAIARBACAQQQAgCSAIIAoQByAXRQ0BCyACIAVBAnRqIRtBACEGA0AgECAGIAhsQQJ0aiEWIBsgBkECdGoqAgAhHkEAIQUCQCAYRQRAIB79EyEcA0AgFiAFQQJ0aiITIBP9AAIAIBz95AH9CwIAIAVBBGoiBSAMRw0ACyAMIgUgCEYNAQsDQCAWIAVBAnRqIhMgEyoCACAekjgCACAFQQFqIgUgCEcNAAsLIAZBAWoiBiAJSA0ACwsgDUEBaiINIA5HDQALIAtBAWoiCyASRw0ACwwBC0EAIQYgFARAA0AgACAGIBVsQQJ0aiECIAMgBiAPbEECdGohDEEAIQUDQCAHIAIgBSARbEECdGo2AgxBHCAHQQxqIAgQAiABIAUgCWwiCyAKbEECdGpBACAEQQAgDCAIIAtsQQJ0akEAIAkgCCAKEAcgBUEBaiIFIA5HDQALIAZBAWoiBiASRw0ADAILAAsDQCADIAYgD2xBAnRqIQBBACEFA0AgASAFIAlsIgIgCmxBAnRqQQAgBEEAIAAgAiAIbEECdGpBACAJIAggChAHIAVBAWoiBSAORw0ACyAGQQFqIgYgEkcNAAsLIAdB4ABqJAALoRcDHH8EewN9IAAoAhxBAnQQBiIJBEAgASACSARAIAAoAhggACgCEGwhFgNAIAAoAhAiEyABIBZtIhBsIAEgECAWbGsiBCAAKAIYIgNtIhJqIANsIQYgBCADIBJsayIXIAAoAhwiBSADa2ohByASIBMgACgCFCIIbW0hGAJ/IAAoAiwiC0EATARAIAdBAWohBCAFIQNBAAwBCyAHQQFqIgQgACgCMGoiAyAFIAMgBUgbIQNBfyAHIAtrIgcgB0EASBtBAWoLIREgACgCICIKIAVsIAAoAgwhGyAAKAIAIQ4gACgCCCEcIAAoAgQhDSAEIAMgAyAEShsgAyAAKAIoGyEMIAogBiAXamwhDyAIIBBsIBhqbCAAKAI0IgMEQCADIBBBAnRqKAIAIgMgDCADIAxIGyEMCwJAIBEgDEEAIAxBAEobIgcgByARSxsiCEUgBSAHTHENAAJAIAhFDQBBACEDIAhBBE8EQCAIQfz///8HcSEDQQAhBANAIAkgBEECdGr9DMrySfHK8knxyvJJ8crySfH9CwIAIARBBGoiBCADRw0ACyADIAhGDQELA0AgCSADQQJ0akHK5aeKfzYCACADQQFqIgMgCEcNAAsLIAUgB0wNACAFIAciA2siBkEETwRAIAkgA0ECdGohFSADIAZBfHEiC2ohA0EAIQQDQCAVIARBAnRq/QzK8knxyvJJ8crySfHK8knx/QsCACAEQQRqIgQgC0cNAAsgBiALRg0BCwNAIAkgA0ECdGpByuWnin82AgAgA0EBaiIDIAVHDQALCyAPQQJ0IRVBAnQhFCAHIBFLBEAgDiAVaiEPIA0gFGohHSAKQfD///8HcSEOIAAqAiQhJCAIIQsDQCAdIAogC2xBAnRqIQ1BECEGQQAhA/0MAAAAAAAAAAAAAAAAAAAAACIfISD9DAAAAAAAAAAAAAAAAAAAAAAhIf0MAAAAAAAAAAAAAAAAAAAAACEiIApBEE4EQANAIA0gA0ECdCIEaiID/QAAMCAEIA9qIgT9AAAw/eYBIB/95AEhHyAD/QAAICAE/QAAIP3mASAg/eQBISAgA/0AABAgBP0AABD95gEgIf3kASEhIAP9AAAAIAT9AAAA/eYBICL95AEhIiAGIgNBEGoiBiAKTA0ACyAhICL95AEgIP3kASAf/eQBIR8gDiEDCyAKIANBBHIiBE4EQANAIA0gA0ECdCIDav0AAAAgAyAPav0AAAD95gEgH/3kASEfIAQiA0EEaiIEIApMDQALCyAf/R8AIB/9HwGSIB/9HwKSIB/9HwOSISMCQCADIApODQACQCAKIANrIhlBBEkEQCADIQQMAQv9DAAAAAAAAAAAAAAAAAAAAAAgI/0gACEfIAMgGUF8cSIaaiEEQQAhBgNAIA0gAyAGakECdCIeav0AAgAgDyAeav0AAgD95gEgH/3kASEfIAZBBGoiBiAaRw0ACyAfIB8gH/0NBAUGBwABAgMMDQ4PAAECA/3kASIfIB8gH/0NCAkKCwABAgMAAQIDAAECA/3kAf0fACEjIBkgGkYNAQsDQCANIARBAnQiA2oqAgAgAyAPaioCAJQgI5IhIyAEQQFqIgQgCkcNAAsLIAkgC0ECdGogIyAklDgCACAMIAtBAWoiC0cNAAsLAn0gACgCOCIEBH8gBCAAKAJIIgYgACgCRCIHIBIgGCAAKAJAIgMgE0YbQQAgA0EBRxsgEEEAIAAoAjxBAUcbIANsamwgF0EAIAdBAUcbamxBAnRqIQcCQAJAAkAgBkEBRwRAIAVBAEwNAkEAIQMgBUEESQ0BIAVB/P///wdxIQNBACEEA0AgCSAEQQJ0IgZqIgggCP0AAgAgBiAHav0AAgD95AH9CwIAIARBBGoiBCADRw0ACyADIAVHDQEMAwsgBUEATA0BIAcqAgAhI0EAIQMgBUEETwRAIAVB/P///wdxIQMgI/0TIR9BACEEA0AgCSAEQQJ0aiIHIAf9AAIAIB/95AH9CwIAIARBBGoiBCADRw0ACyADIAVGDQMLA0AgCSADQQJ0aiIEIAQqAgAgI5I4AgAgA0EBaiIDIAVHDQALDAILA0AgCSADQQJ0IgRqIgYgBioCACAEIAdqKgIAkjgCACADQQFqIgMgBUcNAAsMAQtBASEDQQAhDCAFIQcgCSoCAAwCC0EAIQggBQUgBwsgBSAMIBFKIgMbIQcgCEEAIAMbIQwgBUEESARAQQEhAyAJKgIADAELQQghBCAJ/QAAACEfQQQhAyAFQQhPBEADQCAfIAkgA0ECdGr9AAAA/ekBIR8gBCIDQQRqIgQgBUwNAAsgBUH8////B3EhAwsgH/0fACAf/R8BlyAf/R8CIB/9HwOXlwshIwJAAkAgAyAFSARAIAUgA2siBkEETwRAIAkgA0ECdGohCyADIAZBfHEiCGohAyAj/RMhH0EAIQQDQCALIARBAnRq/QACACAf/esBIR8gBEEEaiIEIAhHDQALIB8gHyAf/Q0ICQoLDA0ODwABAgMAAQID/esBIh8gHyAf/Q0EBQYHAAECAwABAgMAAQID/esB/R8AISMgBiAIRg0CCwNAIAkgA0ECdGoqAgAiJCAjICMgJF0bISMgA0EBaiIDIAVHDQALDAELIAVBAEwNAQsCQAJAIAVBBEkiBgRAQQAhA0MAAAAAISQMAQsgBUF8cSEDICP9EyEh/QwAAAAAAAAAAAAAAAAAAAAAISBBACEEA0AgCSAEQQJ0aiIIIAj9AAIAICH95QEiH/0fABAB/RMgH/0fARAB/SABIB/9HwIQAf0gAiAf/R8DEAH9IAMiH/0LAgAgHyAg/eQBISAgBEEEaiIEIANHDQALICAgICAf/Q0EBQYHAAECAwwNDg8AAQID/eQBIh8gHyAf/Q0ICQoLAAECAwABAgMAAQID/eQB/R8AISQgAyAFRg0BCwNAIAkgA0ECdGoiBCAEKgIAICOTEAEiJTgCACAlICSSISQgA0EBaiIDIAVHDQALCyAkQwAAAABeRQ0AQwAAgD8gJJUhI0EAIQMgBkUEQCAj/RMhH0EAIQRBBCEGA0AgCSAEQQJ0aiIDIAP9AAAAIB/95gH9CwAAIAYiAyEEIANBBGoiBiAFTA0ACwsgAyAFTw0AIAUgA2siBEEETwRAIAkgA0ECdGohBiADIAQgBUEDcSIIayILaiEDICP9EyEfQQAhBANAIAYgBEECdGoiDiAO/QACACAf/eYB/QsCACAEQQRqIgQgC0cNAAsgCEUNAQsDQCAJIANBAnRqIgQgBCoCACAjlDgCACADQQFqIgMgBUcNAAsLIBUgG2ohBQJAIApBAEwNACAKQQJ0IgNFDQAgBUEAIAP8CwALIAcgDEoEQCAUIBxqIREgCkEDcSELIApBBGsiDkEEcSEQIA5BAnZBAWpB/v///wdxIQ8DQCARIAogDGxBAnRqIQggCSAMQQJ0aioCACEjQQAhBAJAIApBBEgNACAj/RMhH0EEIQNBACEGIA5BA0sEQANAIAUgBEECdCIEaiINIAQgCGr9AAAAIB/95gEgDf0AAAD95AH9CwAAIAUgA0ECdCIEaiINIAQgCGr9AAAAIB/95gEgDf0AAAD95AH9CwAAIANBBGohBCADQQhqIQMgBkECaiIGIA9HDQALCyAQDQAgBSAEQQJ0IgRqIgYgBCAIav0AAAAgH/3mASAG/QAAAP3kAf0LAAAgAyEECwJAIAQgCk4NAAJAIAogBGsiA0EESQRAIAQhAwwBCyAEIAMgC2siDWohAyAj/RMhH0EAIQYDQCAFIAQgBmpBAnQiEmoiEyAT/QACACAIIBJq/QACACAf/eYB/eQB/QsCACAGQQRqIgYgDUcNAAsgC0UNAQsDQCAFIANBAnQiBGoiBiAGKgIAIAQgCGoqAgAgI5SSOAIAIANBAWoiAyAKRw0ACwsgDEEBaiIMIAdHDQALCyABQQFqIgEgAkcNAAsLIAkQBQsL+AEBAX8jAEHQAGsiFCQAAkAgBEEATA0AIAVBAEwNACAGQQBMDQAgB0EATA0AIAlBAEwNACAIQQBMBEAgByAJbCAFbCAEbEECdCIARQ0BIANBACAA/AsADAELIBQgEzYCTCAUIBI2AkggFCARNgJEIBQgEDYCQCAUIA82AjwgFCAONgI4IBQgDTYCNCAUIAw2AjAgFCALNgIsIBQgCjgCKCAUIAk2AiQgFCAINgIgIBQgBjYCGCAUIAM2AhAgFCACNgIMIBQgATYCCCAUIAA2AgQgFCAFNgIUIBQgBzYCHEEbIBRBBGogBCAFbCAHbBACCyAUQdAAaiQAC2EBAX8jAEEgayIGJAACQCADQQBMDQAgBEEATA0AIAZBATYCHCAGIAU4AhggBiAENgIUIAYgAjYCECAGQQA2AgwgBiABNgIIIAYgADYCBEEaIAZBBGogAxACCyAGQSBqJAALuAYDEn8BewF9AkAgACgCDCIKQQBMDQAgASACTg0AIAAoAhAhCyAAKAIIIQwgACgCACENIAAoAgQhDiAAKAIUIgNBBE4EQCADQfz///8HcSEFIANBBGsiD0EEcSERIA9BAnZBAWpB/v///wdxIRIDQCAMIAYgC2xBAnRqIRMgDSADIAZsQQJ0aiEHIAEhBANAIA4gAyAEbEECdGohCUEEIQD9DAAAAAAAAAAAAAAAAAAAAAAhFUEAIQhBACEQIA9BBE8EQANAIAkgAEECdCIUav0AAAAgByAUav0AAAD95gEgCSAIQQJ0Ighq/QAAACAHIAhq/QAAAP3mASAV/eQB/eQBIRUgAEEEaiEIIABBCGohACAQQQJqIhAgEkcNAAsLIBFFBEAgCSAIQQJ0IgBq/QAAACAAIAdq/QAAAP3mASAV/eQBIRULIBX9HwAgFf0fAZIgFf0fApIgFf0fA5IhFiADIAUiAEoEQANAIAkgAEECdCIIaioCACAHIAhqKgIAlCAWkiEWIABBAWoiACADRw0ACwsgEyAEQQJ0aiAWOAIAIARBAWoiBCACRw0ACyAGQQFqIgYgCkcNAAsMAQsgA0EASgRAIANBAUYhBwNAIAwgBCALbEECdGohCSANIAMgBGxBAnRqIQUgASEAA0AgCSAAQQJ0agJ9IA4gACADbEECdGoiBioCACAFKgIAlCIWIAcNABogBioCBCAFKgIElCAWkiIWIANBAkYNABogBioCCCAFKgIIlCAWkgs4AgAgAEEBaiIAIAJHDQALIARBAWoiBCAKRw0ACwwBCyALQQJ0IQUgDCABQQJ0aiEDIAIgAWtBAnQhAUEAIQAgCkEETwRAIApB/P///wdxIQcDQCABRSICRQRAIAMgACAFbGpBACAB/AsACyACRQRAIAMgBSAAQQFybGpBACAB/AsACyACRQRAIAMgBSAAQQJybGpBACAB/AsACyACRQRAIAMgBSAAQQNybGpBACAB/AsACyAAQQRqIQAgBkEEaiIGIAdHDQALCyAKQQNxIgJFDQADQCABBEAgAyAAIAVsakEAIAH8CwALIABBAWohACAEQQFqIgQgAkcNAAsLC4YdAxJ/A3sEfSABIAJIBEBDAACAPyAAKAIQIgmylSEaIAlBA3EhCyAJQfz///8HcSEOIAlBBGsiDUEEcSEQIA1BAnZBAWoiA0H+////B3EhESADQfz///8HcSESIANBA3EhDyANQQxJIRMDQCABIAlsQQJ0IgMgACgCDGohCiAAKAIAIANqIQcCQCAAKAIYBEACfyAJQQRIIgwEQP0MAAAAAAAAAAAAAAAAAAAAACEVQQAMAQv9DAAAAAAAAAAAAAAAAAAAAAAhFUEAIQVBBCEDQQAhBkEAIQQgE0UEQANAIAcgA0ECdGoiCP0AACAiFiAW/eYBIAj9AAAQIhYgFv3mASAI/QAAACIWIBb95gEgByAGQQJ0av0AAAAiFiAW/eYBIBX95AH95AH95AH95AEhFSADQQxqIQYgA0EQaiEDIARBBGoiBCASRw0ACwsgDiAPRQ0AGgNAIAMiBEEEaiEDIAcgBkECdGr9AAAAIhYgFv3mASAV/eQBIRUgBCEGIAVBAWoiBSAPRw0ACyAOCyEEIBX9HwAgFf0fAZIgFf0fApIgFf0fA5IhGAJAIAQgCU4NACAJIARrIgNBBE8EQP0MAAAAAAAAAAAAAAAAAAAAACAY/SAAIRUgByAEQQJ0aiEGIAQgAyALayIFaiEEQQAhAwNAIAYgA0ECdGr9AAIAIhYgFv3mASAV/eQBIRUgA0EEaiIDIAVHDQALIBUgFSAV/Q0EBQYHAAECAwwNDg8AAQID/eQBIhUgFSAV/Q0ICQoLAAECAwABAgMAAQID/eQB/R8AIRggC0UNAQsDQCAHIARBAnRqKgIAIhkgGZQgGJIhGCAEQQFqIgQgCUcNAAsLQwAAgD8gGCAalCAAKgIUkpGVIRggACgCBCEEQQAhAwJAAkACQAJAAkACQAJAIAwNACAY/RMhFSAERQ0BQQQhBkEAIQUgDUEETwRAA0AgCiADQQJ0IgNqIAMgB2r9AAAAIBX95gEgAyAEav0AAAD95gH9CwAAIAogBkECdCIDaiADIAdq/QAAACAV/eYBIAMgBGr9AAAA/eYB/QsAACAGQQRqIQMgBkEIaiEGIAVBAmoiBSARRw0ACwsgEA0AIAogA0ECdCIDaiADIAdq/QAAACAV/eYBIAMgBGr9AAAA/eYB/QsAACAGIQMLIAMgCU4NByAERQ0BIAkgA2siBkEETw0CIAMhBgwDC0EEIQRBACEGIA1BBE8EQANAIAogA0ECdCIDaiADIAdq/QAAACAV/eYB/QsAACAKIARBAnQiA2ogAyAHav0AAAAgFf3mAf0LAAAgBEEEaiEDIARBCGohBCAGQQJqIgYgEUcNAAsLIBBFBEAgCiADQQJ0IgNqIAMgB2r9AAAAIBX95gH9CwAAIAQhAwsgAyAJTw0GCyAJIANrIgVBBE8NAiADIQQMAwsgAyAGIAtrIgxqIQYgGP0TIRVBACEFA0AgCiADIAVqQQJ0IghqIAcgCGr9AAIAIBX95gEgBCAIav0AAgD95gH9CwIAIAVBBGoiBSAMRw0ACyALRQ0ECwNAIAogBkECdCIDaiADIAdqKgIAIBiUIAMgBGoqAgCUOAIAIAZBAWoiBiAJRw0ACwwDCyADIAVBfHEiCGohBCAY/RMhFUEAIQYDQCAKIAMgBmpBAnQiDGogByAMav0AAgAgFf3mAf0LAgAgBkEEaiIGIAhHDQALIAUgCEYNAgsDQCAKIARBAnQiA2ogAyAHaioCACAYlDgCACAEQQFqIgQgCUcNAAsMAQsCfyAJQQRIIgwEQP0MAAAAAAAAAAAAAAAAAAAAACEVQQAMAQtBACEF/QwAAAAAAAAAAAAAAAAAAAAAIRVBBCEDQQAhBkEAIQQgE0UEQANAIAcgA0ECdGoiCP0AACAgCP0AABAgCP0AAAAgByAGQQJ0av0AAAAgFf3kAf3kAf3kAf3kASEVIANBDGohBiADQRBqIQMgBEEEaiIEIBJHDQALCyAOIA9FDQAaA0AgAyIEQQRqIQMgByAGQQJ0av0AAAAgFf3kASEVIAQhBiAFQQFqIgUgD0cNAAsgDgshBCAV/R8AIBX9HwGSIBX9HwKSIBX9HwOSIRgCQCAEIAlODQAgCSAEayIDQQRPBED9DAAAAAAAAAAAAAAAAAAAAAAgGP0gACEVIAcgBEECdGohBiAEIAMgC2siBWohBEEAIQMDQCAGIANBAnRq/QACACAV/eQBIRUgA0EEaiIDIAVHDQALIBUgFSAV/Q0EBQYHAAECAwwNDg8AAQID/eQBIhUgFSAV/Q0ICQoLAAECAwABAgMAAQID/eQB/R8AIRggC0UNAQsDQCAHIARBAnRqKgIAIBiSIRggBEEBaiIEIAlHDQALCyAYIBqUIRgCfyAMBED9DAAAAAAAAAAAAAAAAAAAAAAhFUEADAELIBj9EyEW/QwAAAAAAAAAAAAAAAAAAAAAIRVBACEFQQQhA0EAIQZBACEEIBNFBEADQCAHIANBAnRqIgj9AAAgIBb95QEiFyAX/eYBIAj9AAAQIBb95QEiFyAX/eYBIAj9AAAAIBb95QEiFyAX/eYBIAcgBkECdGr9AAAAIBb95QEiFyAX/eYBIBX95AH95AH95AH95AEhFSADQQxqIQYgA0EQaiEDIARBBGoiBCASRw0ACwsgDiAPRQ0AGgNAIAMiBEEEaiEDIAcgBkECdGr9AAAAIBb95QEiFyAX/eYBIBX95AEhFSAEIQYgBUEBaiIFIA9HDQALIA4LIQQgFf0fACAV/R8BkiAV/R8CkiAV/R8DkiEZAkAgBCAJTg0AIAkgBGsiA0EETwRA/QwAAAAAAAAAAAAAAAAAAAAAIBn9IAAhFSAHIARBAnRqIQYgBCADIAtrIgVqIQQgGP0TIRZBACEDA0AgBiADQQJ0av0AAgAgFv3lASIXIBf95gEgFf3kASEVIANBBGoiAyAFRw0ACyAVIBUgFf0NBAUGBwABAgMMDQ4PAAECA/3kASIVIBUgFf0NCAkKCwABAgMAAQIDAAECA/3kAf0fACEZIAtFDQELA0AgByAEQQJ0aioCACAYkyIbIBuUIBmSIRkgBEEBaiIEIAlHDQALC0MAAIA/IBkgGpQgACoCFJKRlSEZIAAoAgghCCAAKAIEIQZBACEDAkAgDA0AIBn9EyEVIBj9EyEWIAZFBEAgCARAQQQhBUEAIQQgDUEETwRAA0AgCiADQQJ0IgNqIAMgB2r9AAAAIBb95QEgFf3mASADIAhq/QAAAP3kAf0LAAAgCiAFQQJ0IgNqIAMgB2r9AAAAIBb95QEgFf3mASADIAhq/QAAAP3kAf0LAAAgBUEEaiEDIAVBCGohBSAEQQJqIgQgEUcNAAsLIBANAiAKIANBAnQiA2ogAyAHav0AAAAgFv3lASAV/eYBIAMgCGr9AAAA/eQB/QsAACAFIQMMAgtBBCEFQQAhBCANQQRPBEADQCAKIANBAnQiA2ogAyAHav0AAAAgFv3lASAV/eYB/QsAACAKIAVBAnQiA2ogAyAHav0AAAAgFv3lASAV/eYB/QsAACAFQQRqIQMgBUEIaiEFIARBAmoiBCARRw0ACwsgEA0BIAogA0ECdCIDaiADIAdq/QAAACAW/eUBIBX95gH9CwAAIAUhAwwBCyAIBEBBACEFQQQhBANAIAogBUECdCIDaiADIAdq/QAAACAW/eUBIBX95gEgAyAGav0AAAD95gEgAyAIav0AAAD95AH9CwAAIAQiAyEFIANBBGoiBCAJTA0ACwwBC0EEIQVBACEEIA1BBE8EQANAIAogA0ECdCIDaiADIAdq/QAAACAW/eUBIBX95gEgAyAGav0AAAD95gH9CwAAIAogBUECdCIDaiADIAdq/QAAACAW/eUBIBX95gEgAyAGav0AAAD95gH9CwAAIAVBBGohAyAFQQhqIQUgBEECaiIEIBFHDQALCyAQDQAgCiADQQJ0IgNqIAMgB2r9AAAAIBb95QEgFf3mASADIAZq/QAAAP3mAf0LAAAgBSEDCyADIAlODQAgCSADayEEIAZFBEAgCARAAkAgBEEESQRAIAMhBgwBCyADIAQgC2siDGohBiAZ/RMhFSAY/RMhFkEAIQUDQCAKIAMgBWpBAnQiBGogBCAHav0AAgAgFv3lASAV/eYBIAQgCGr9AAIA/eQB/QsCACAFQQRqIgUgDEcNAAsgC0UNAwsDQCAKIAZBAnQiA2ogAyAHaioCACAYkyAZlCADIAhqKgIAkjgCACAGQQFqIgYgCUcNAAsMAgsCQCAEQQRJBEAgAyEEDAELIAMgBCALayIFaiEEIBn9EyEVIBj9EyEWQQAhBgNAIAogAyAGakECdCIIaiAHIAhq/QACACAW/eUBIBX95gH9CwIAIAZBBGoiBiAFRw0ACyALRQ0CCwNAIAogBEECdCIDaiADIAdqKgIAIBiTIBmUOAIAIARBAWoiBCAJRw0ACwwBCyAIBEACQCAEQQRJBEAgAyEFDAELIAMgBCALayIUaiEFIBn9EyEVIBj9EyEWQQAhBANAIAogAyAEakECdCIMaiAHIAxq/QACACAW/eUBIBX95gEgBiAMav0AAgD95gEgCCAMav0AAgD95AH9CwIAIARBBGoiBCAURw0ACyALRQ0CCwNAIAogBUECdCIDaiADIAdqKgIAIBiTIBmUIAMgBmoqAgCUIAMgCGoqAgCSOAIAIAVBAWoiBSAJRw0ACwwBCwJAIARBBEkEQCADIQQMAQsgAyAEIAtrIgxqIQQgGf0TIRUgGP0TIRZBACEFA0AgCiADIAVqQQJ0IghqIAcgCGr9AAIAIBb95QEgFf3mASAGIAhq/QACAP3mAf0LAgAgBUEEaiIFIAxHDQALIAtFDQELA0AgCiAEQQJ0IgNqIAMgB2oqAgAgGJMgGZQgAyAGaioCAJQ4AgAgBEEBaiIEIAlHDQALCyABQQFqIgEgAkcNAAsLC2EBAX8jAEEgayIHJAACQCAEQQBMDQAgBUEATA0AIAdBADYCHCAHIAY4AhggByAFNgIUIAcgAzYCECAHIAI2AgwgByABNgIIIAcgADYCBEEaIAdBBGogBBACCyAHQSBqJAALlg0DCX8DewV9IAEgAkgEQEMAAAAAEAwhEgNAIAAoAgAiCiAAKAIIIgUgAWxBAnQiCWohBwJ9IAVBBEgEQEEBIQQgByoCAAwBC0EIIQMgB/0AAAAhDEEEIQQgBUEITwRAA0AgDCAHIARBAnRq/QAAAP3pASEMIAMiBEEEaiIDIAVMDQALIAVB/P///wdxIQQLIAz9HwAgDP0fAZcgDP0fAiAM/R8Dl5cLIQ8CQCAEIAVODQAgBSAEayIGQQRPBEAgByAEQQJ0aiELIAQgBkF8cSIIaiEEIA/9EyEMQQAhAwNAIAsgA0ECdGr9AAIAIAz96wEhDCADQQRqIgMgCEcNAAsgDCAMIAz9DQgJCgsMDQ4PAAECAwABAgP96wEiDCAMIAz9DQQFBgcAAQIDAAECAwABAgP96wH9HwAhDyAGIAhGDQELA0AgByAEQQJ0aioCACIQIA8gDyAQXRshDyAEQQFqIgQgBUcNAAsLIAkgACgCBCIJaiEIAkACQAJAAkACQAJAAkAgACgCDARAIAVBAEwNASAFQQRJIgQEQEEAIQNDAAAAACEQDAULIAVB/P///wdxIQMgD/0TIQ79DAAAAAAAAAAAAAAAAAAAAAAhDEEAIQYDQCAHIAZBAnRq/QACACAO/eUBIg39HwAQAf0TIA39HwEQAf0gASAN/R8CEAH9IAIgDf0fAxAB/SADIAz95AEhDCAGQQRqIgYgA0cNAAsgDCAMIA39DQQFBgcAAQIDDA0ODwABAgP95AEiDCAMIAz9DQgJCgsAAQIDAAECAwABAgP95AH9HwAhECADIAVHDQQMBQsgBUEATA0GQQAhA0MAAAAAIRACQCAFQQRJIgsNACAJIAprQRBJDQAgBUH8////B3EhAyAP/RMhDv0MAAAAAAAAAAAAAAAAAAAAACEMQQAhBgNAIAggBkECdCIEaiAEIAdq/QACACAO/eUBIg39HwAQAf0TIA39HwEQAf0gASAN/R8CEAH9IAIgDf0fAxAB/SADIg39CwIAIA0gDP3kASEMIAZBBGoiBiADRw0ACyAMIAwgDf0NBAUGBwABAgMMDQ4PAAECA/3kASIMIAwgDP0NCAkKCwABAgMAAQIDAAECA/3kAf0fACEQIAMgBUYNAwsgA0EBciEEIAVBAXEEQCAIIANBAnQiA2ogAyAHaioCACAPkxABIhE4AgAgESAQkiEQIAQhAwsgBCAFRw0BDAILIA8gEpIhD0EAIQMMBAsDQCAIIANBAnQiBGogBCAHaioCACAPkxABIhE4AgAgCCAEQQRqIgRqIAQgB2oqAgAgD5MQASITOAIAIBMgESAQkpIhECADQQJqIgMgBUcNAAsLIBBDAAAAAF5FDQNDAACAPyAQlSEPQQAhBCALRQRAIA/9EyEMQQAhA0EEIQYDQCAIIANBAnRqIgMgA/0AAAAgDP3mAf0LAAAgBiIEIgNBBGoiBiAFTA0ACwsgBCAFTw0DIAUgBGsiA0EETwRAIAggBEECdGohBiAEIAMgBUEDcSIHayIKaiEEIA/9EyEMQQAhAwNAIAYgA0ECdGoiCSAJ/QACACAM/eYB/QsCACADQQRqIgMgCkcNAAsgB0UNBAsDQCAIIARBAnRqIgMgAyoCACAPlDgCACAEQQFqIgQgBUcNAAsMAwsDQCAHIANBAnRqKgIAIA+TEAEgEJIhECADQQFqIgMgBUcNAAsLIBAQDCAPkiEPQQAhAyAEDQAgD/0TIQxBACEGQQQhBANAIAggBkECdCIDaiADIAdq/QAAACAM/eUB/QsAACAEIgMhBiADQQRqIgQgBUwNAAsLIAMgBU4NAAJAIAUgA2siBEEESQRAIAMhBAwBCyAJIAprQRBJBEAgAyEEDAELIAMgBCAFQQNxIgprIglqIQQgD/0TIQxBACEGA0AgCCADIAZqQQJ0IgtqIAcgC2r9AAIAIAz95QH9CwIAIAZBBGoiBiAJRw0ACyAKRQ0BC0EAIQYgBSAEIgNrQQNxIgoEQANAIAggA0ECdCIJaiAHIAlqKgIAIA+TOAIAIANBAWohAyAGQQFqIgYgCkcNAAsLIAQgBWtBfEsNAANAIAggA0ECdCIEaiAEIAdqKgIAIA+TOAIAIAggBEEEaiIGaiAGIAdqKgIAIA+TOAIAIAggBEEIaiIGaiAGIAdqKgIAIA+TOAIAIAggBEEMaiIEaiAEIAdqKgIAIA+TOAIAIANBBGoiAyAFRw0ACwsgAUEBaiIBIAJHDQALCwtJAQF/IwBBEGsiBSQAAkAgAkEATA0AIANBAEwNACAFIAQ2AgwgBSADNgIIIAUgATYCBCAFIAA2AgBBGSAFIAIQAgsgBUEQaiQAC+0JAx9/AXsBfQJAIAEgAk4NACAAKAIMIQYgACgCCCILQQBKBEBDAACAPyALs5UiI/0TISIgBkEDcSEMIAZBAnQhDSAGQQRrIg9BBHEhFCAGIAtsIhJBAnQhFSABIAZsQQJ0IRYgASASbEECdCEXIAYgAUECdEEEamwhGCAPQQJ2QQFqIgNB/P///wdxIRkgA0EDcSETIANB/v///wdxIRogBiABIAtsQQJ0QQRqbCEbA0AgASASbEECdCEFIAAoAgAhAyAAKAIEIgQgASAGbEECdGohBwJAIAZBAEwNACANRQ0AIAdBACAN/AsACyADIAVqIRwgDiAVbCIFIAMgG2pqIR0gAyAXaiAFaiEeIA0gDmwiAyAEIBhqaiEfIAQgFmogA2ohIEEAIQoDQCAcIAYgCmxBAnRqIQhBACEDAkAgBkEESCIhDQBBBCEEQQAhBSAPQQRPBEADQCAHIANBAnQiA2oiCSADIAhq/QAAACAJ/QAAAP3kAf0LAAAgByAEQQJ0IgNqIgkgAyAIav0AAAAgCf0AAAD95AH9CwAAIARBBGohAyAEQQhqIQQgBUECaiIFIBpHDQALCyAUDQAgByADQQJ0IgNqIgUgAyAIav0AAAAgBf0AAAD95AH9CwAAIAQhAwsCQCADIAZODQACQCAGIANrIgRBBEkEQCADIQUMAQsCQCAgIANBAnQiBWogHSAKIA1sIglqTw0AIAkgHmogBWogH08NACADIQUMAQsgAyAEIAxrIglqIQVBACEEA0AgByADIARqQQJ0IhBqIhEgEf0AAgAgCCAQav0AAgD95AH9CwIAIARBBGoiBCAJRw0ACyAMRQ0BC0EAIQMgBiAFIgRrQQNxIgkEQANAIAcgBEECdCIQaiIRIBEqAgAgCCAQaioCAJI4AgAgBEEBaiEEIANBAWoiAyAJRw0ACwsgBSAGa0F8Sw0AA0AgByAEQQJ0IgNqIgUgBSoCACADIAhqKgIAkjgCACAHIANBBGoiBWoiCSAJKgIAIAUgCGoqAgCSOAIAIAcgA0EIaiIFaiIJIAkqAgAgBSAIaioCAJI4AgAgByADQQxqIgNqIgUgBSoCACADIAhqKgIAkjgCACAEQQRqIgQgBkcNAAsLIApBAWoiCiALRw0ACwJAIAAoAhBFDQBBACEDAkAgIQ0AQQAhBUEEIQRBACEIIA9BDE8EQANAIAcgA0ECdGoiAyAD/QAAACAi/eYB/QsAACAHIARBAnRqIgMgA/0AAAAgIv3mAf0LAAAgAyAD/QAAECAi/eYB/QsAECADIAP9AAAgICL95gH9CwAgIARBDGohAyAEQRBqIQQgCEEEaiIIIBlHDQALCyATRQ0AA0AgByADQQJ0aiIDIAP9AAAAICL95gH9CwAAIAQiA0EEaiEEIAVBAWoiBSATRw0ACwsgAyAGTg0AIAYgA2siBEEETwRAIAcgA0ECdGohBSADIAQgDGsiCGohA0EAIQQDQCAFIARBAnRqIgogCv0AAgAgIv3mAf0LAgAgBEEEaiIEIAhHDQALIAxFDQELA0AgByADQQJ0aiIEIAQqAgAgI5Q4AgAgA0EBaiIDIAZHDQALCyAOQQFqIQ4gAUEBaiIBIAJHDQALDAELIAZBAEwNACAGIAIgAWtsQQJ0IgJFDQAgACgCBCABIAZsQQJ0akEAIAL8CwALC3wBAX8jAEEgayIGJAACQCABQQBMDQAgA0EATA0AIAJBAEwEQCABIANsIgBBAEwNASAAQQJ0IgBFDQEgBUEAIAD8CwAMAQsgBiAENgIcIAYgAzYCGCAGIAI2AhQgBiAFNgIQIAYgADYCDEEYIAZBDGogARACCyAGQSBqJAALiwMDBHsFfwF9IAFBAEwEQCADQwAAAAA4AgAPC0EQIQkgAUEQTwRAA0AgACAIQQJ0aiII/QAAMCAE/eQBIQQgCP0AACAgBf3kASEFIAj9AAAQIAb95AEhBiAI/QAAACAH/eQBIQcgCSIIQRBqIgkgAUwNAAsgBiAH/eQBIAX95AEgBP3kASEEIAFB8P///wdxIQgLIAT9HwAgBP0fAZIgBP0fApIgBP0fA5IhDQJAIAEgCE0NACABIAhrIglBBE8EQP0MAAAAAAAAAAAAAAAAAAAAACAN/SAAIQQgACAIQQJ0aiEKIAggCSABQQNxIgtrIgxqIQhBACEJA0AgCiAJQQJ0av0AAgAgBP3kASEEIAlBBGoiCSAMRw0ACyAEIAQgBP0NBAUGBwABAgMMDQ4PAAECA/3kASIEIAQgBP0NCAkKCwABAgMAAQIDAAECA/3kAf0fACENIAtFDQELA0AgACAIQQJ0aioCACANkiENIAhBAWoiCCABRw0ACwsgAyANIAGzlSANIAIbOAIAC+4QBAZ/AXsBfQF8IAJBCHQiAiAAKAIIIgMgAiADSBshAyABQQh0IQEgACgCBCEEIAAoAgAhBQJAAkACQAJAAkACQAJAAkACQAJAIAAoAgwOBgUEAwIBAAYLIAEgA04NCANAIAQgAUECdCIAaiAAIAVqKgIAuxAOtjgCACABQQFqIgEgA0cNAAsMCAsgASADTg0HA0AgBCABQQJ0IgBqIAAgBWoqAgC7IgtEAAAAAAAA4D+iIAtEzTt/Zp6g5j+iEA5EAAAAAAAA8D+gorY4AgAgAUEBaiIBIANHDQALDAcLIAEgA04NBgJAIAMgAWsiAEEESQRAIAEhAgwBCyAEIAVrQRBJBEAgASECDAELIAEgACADQQNxIgZrIgdqIQJBACEAA0AgBCAAIAFqQQJ0IghqIAUgCGr9AAIAIgn9DAAAAD8AAAA/AAAAPwAAAD/95gEgCSAJ/eYB/QwTJzc9Eyc3PRMnNz0TJzc9/eYBIAn95gEgCf3kAf0MKkJMPypCTD8qQkw/KkJMP/3mASIJ/R8AEAP9EyAJ/R8BEAP9IAEgCf0fAhAD/SACIAn9HwMQA/0gA/0MAACAPwAAgD8AAIA/AACAP/3kAf3mAf0LAgAgAEEEaiIAIAdHDQALIAZFDQcLIAJBAWohACADIAJrQQFxBEAgBCACQQJ0IgFqIAEgBWoqAgAiCkMAAAA/lCAKIAqUQxMnNz2UIAqUIAqSQypCTD+UEANDAACAP5KUOAIAIAAhAgsgACADRg0GA0AgBCACQQJ0IgBqIAAgBWoqAgAiCkMAAAA/lCAKIAqUQxMnNz2UIAqUIAqSQypCTD+UEANDAACAP5KUOAIAIAQgAEEEaiIAaiAAIAVqKgIAIgpDAAAAP5QgCiAKlEMTJzc9lCAKlCAKkkMqQkw/lBADQwAAgD+SlDgCACACQQJqIgIgA0cNAAsMBgsgASADTg0FAkAgAyABayIAQQRJBEAgASECDAELIAQgBWtBEEkEQCABIQIMAQsgASAAIANBA3EiBmsiB2ohAkEAIQADQCAFIAAgAWpBAnQiCGr9AAIAIQkgBCAIaiAJIAn94QEiCf0fABAB/RMgCf0fARAB/SABIAn9HwIQAf0gAiAJ/R8DEAH9IAP9DAAAgD8AAIA/AACAPwAAgD/95AH95wH9CwIAIABBBGoiACAHRw0ACyAGRQ0GCyACQQFqIQAgAyACa0EBcQRAIAUgAkECdCIBaioCACEKIAEgBGogCiAKjBABQwAAgD+SlTgCACAAIQILIAAgA0YNBQNAIAUgAkECdCIAaioCACEKIAAgBGogCiAKjBABQwAAgD+SlTgCACAFIABBBGoiAGoqAgAhCiAAIARqIAogCowQAUMAAIA/kpU4AgAgAkECaiICIANHDQALDAULIAEgA04NBAJAIAMgAWsiAEEESQRAIAEhAgwBCyAEIAVrQRBJBEAgASECDAELIAEgACADQQNxIgZrIgdqIQJBACEAA0AgBCAAIAFqQQJ0Ighq/QwAAIA/AACAPwAAgD8AAIA//QwAAIA/AACAPwAAgD8AAIA/IAUgCGr9AAIA/eEBIgn9HwAQAf0TIAn9HwEQAf0gASAJ/R8CEAH9IAIgCf0fAxAB/SAD/eQB/ecB/QsCACAAQQRqIgAgB0cNAAsgBkUNBQsgAkEBaiEAIAMgAmtBAXEEQCAEIAJBAnQiAWpDAACAPyABIAVqKgIAjBABQwAAgD+SlTgCACAAIQILIAAgA0YNBANAIAQgAkECdCIAakMAAIA/IAAgBWoqAgCMEAFDAACAP5KVOAIAIAQgAEEEaiIAakMAAIA/IAAgBWoqAgCMEAFDAACAP5KVOAIAIAJBAmoiAiADRw0ACwwECyABQQRyIgIgA0wNASABIQAMAgsgASADTg0CAkAgAyABayIAQQRJBEAgASECDAELIAQgBWtBEEkEQCABIQIMAQsgASAAIANBA3EiBmsiB2ohAkEAIQADQCAEIAAgAWpBAnQiCGogBSAIav0AAgAiCf0fABAD/RMgCf0fARAD/SABIAn9HwIQA/0gAiAJ/R8DEAP9IAP9CwIAIABBBGoiACAHRw0ACyAGRQ0DCyADIAIiAWtBA3EiBgRAQQAhAANAIAQgAUECdCIHaiAFIAdqKgIAEAM4AgAgAUEBaiEBIABBAWoiACAGRw0ACwsgAiADa0F8Sw0CA0AgBCABQQJ0IgBqIAAgBWoqAgAQAzgCACAEIABBBGoiAmogAiAFaioCABADOAIAIAQgAEEIaiICaiACIAVqKgIAEAM4AgAgBCAAQQxqIgBqIAAgBWoqAgAQAzgCACABQQRqIgEgA0cNAAsMAgsDQCAEIAFBAnQiAGogACAFav0AAAD9DAAAAAAAAAAAAAAAAAAAAAD96QH9CwAAIAIiACIBQQRqIgIgA0wNAAsLIAAgA04NAAJAIAMgAGsiBkEESQRAIAAhAgwBCyAEIAVrQRBJBEAgACECDAELIAAgBkF8cSIHaiECQQAhAQNAIAQgACABakECdCIIaiAFIAhq/QACAP0MAAAAAAAAAAAAAAAAAAAAAP3pAf0LAgAgAUEEaiIBIAdHDQALIAYgB0YNAQsgAyACIgFrQQNxIgYEQEEAIQADQCAEIAFBAnQiB2ogBSAHaioCAEMAAAAAlzgCACABQQFqIQEgAEEBaiIAIAZHDQALCyACIANrQXxLDQADQCAEIAFBAnQiAGogACAFaioCAEMAAAAAlzgCACAEIABBBGoiAmogAiAFaioCAEMAAAAAlzgCACAEIABBCGoiAmogAiAFaioCAEMAAAAAlzgCACAEIABBDGoiAGogACAFaioCAEMAAAAAlzgCACABQQRqIgEgA0cNAAsLC0cBAX8jAEEQayIEJAAgA0EASgRAIAQgADYCDCAEIAI2AgQgBCABNgIAIAQgAzYCCEEXIAQgA0H/AWpBCHYQAgsgBEEQaiQAC4oiAwt/A30EeyACQQp0IgIgACgCDCIDIAIgA0gbIQggAUEKdCECAkACQAJAAkACQAJAAkACQAJAAkAgACgCGCIJDgMAAQMECwJAAkAgACgCHA4DAAEDBQsgACgCCCEDIAAoAgQhCSAAKAIAIQQgACgCFCEFIAggAkEEciIASARAIAIhAQwJCwJAAkACQCAFDgMAAQIICwNAIAMgAkECdCIBaiABIAlq/QAAACABIARq/QAAAP3kAf0LAAAgACIBIQIgAUEEaiIAIAhMDQALDAoLA0AgAyACQQJ0IgFqIAEgBGr9AAAAIAEgCWr9AAAA/eUB/QsAACAAIgEhAiABQQRqIgAgCEwNAAsMCQsDQCADIAJBAnQiAWogASAJav0AAAAgASAEav0AAAD95gH9CwAAIAAiASECIAFBBGoiACAITA0ACwwICyAAKAIIIQMgACgCACEJIAAoAhQhBCAAKAIEKgIAIQ4CQCAIIAJBBHIiAEgEQCACIQEMAQtDAACAPyAOlf0TIRMgDv0TIRIDQCAAIQEgAyACQQJ0IgBqIAAgCWr9AAAAIRECewJAAkACQAJAIAQOAwIBAAMLIBEgEv3mAQwDCyARIBL95QEMAgsgESAS/eQBDAELIBEgE/3mAQv9CwAAIAEhAiABQQRqIgAgCEwNAAsLIAEgCE4NCAJAIAggAWsiBUEESQRAIAEhAgwBCyABIAVBfHEiBmohAkMAAIA/IA6V/RMhFCAO/RMhESAE/REhEkEAIQADQCADIAAgAWpBAnQiB2ogByAJav0AAgAiEyAR/eYBIBMgEf3lASATIBH95AEgEyAU/eYBIBL9DAAAAAAAAAAAAAAAAAAAAAD9N/1SIBL9DAEAAAABAAAAAQAAAAEAAAD9N/1SIBL9DAIAAAACAAAAAgAAAAIAAAD9N/1S/QsCACAAQQRqIgAgBkcNAAsgBSAGRg0JC0MAAIA/IA6VIRADQCAJIAJBAnQiAGoqAgAhDyAAIANqAn0CQAJAAkACQCAEDgMCAQADCyAPIA6UDAMLIA8gDpMMAgsgDyAOkgwBCyAPIBCUCzgCACACQQFqIgIgCEcNAAsMCAsgACgCHA0CIAAoAgghAyAAKAIEIQkgACgCFCEEIAAoAgAqAgAhDiAIIAJBBHIiAEgEQCACIQEMBgsgDv0TIRECQAJAAkAgBA4DAAECBwsDQCADIAJBAnQiAWogASAJav0AAAAgEf3kAf0LAAAgACIBIQIgAUEEaiIAIAhMDQALDAcLA0AgAyACQQJ0IgFqIBEgASAJav0AAAD95QH9CwAAIAAiASECIAFBBGoiACAITA0ACwwGCwNAIAMgAkECdCIBaiABIAlq/QAAACAR/eYB/QsAACAAIgEhAiABQQRqIgAgCEwNAAsMBQsgAiAITg0GA0AgAkECdCIBIAAoAghqIQUgACgCACABaiEGIAAoAgQgAiAAKAIQIgFvIgRBAnRqIQcgACgCFCEJQQAhAwJAIAEgBGsiASAIIAJrIgQgASAESBsiC0EESA0AAkACQAJAAkAgCQ4DAgEAAwtBBCEBIAtBBGsiCkEETwRAIApBAnZBAWpB/v///wdxIQxBACEEA0AgBSADQQJ0IgNqIAMgB2r9AAAAIAMgBmr9AAAA/eYB/QsAACAFIAFBAnQiA2ogAyAHav0AAAAgAyAGav0AAAD95gH9CwAAIAFBBGohAyABQQhqIQEgBEECaiIEIAxHDQALCyAKQQRxDQMgBSADQQJ0IgNqIAMgB2r9AAAAIAMgBmr9AAAA/eYB/QsAACABIQMMAwtBBCEBIAtBBGsiCkEETwRAIApBAnZBAWpB/v///wdxIQxBACEEA0AgBSADQQJ0IgNqIAMgBmr9AAAAIAMgB2r9AAAA/eUB/QsAACAFIAFBAnQiA2ogAyAGav0AAAAgAyAHav0AAAD95QH9CwAAIAFBBGohAyABQQhqIQEgBEECaiIEIAxHDQALCyAKQQRxDQIgBSADQQJ0IgNqIAMgBmr9AAAAIAMgB2r9AAAA/eUB/QsAACABIQMMAgtBBCEBIAkhAyALQQRrIgpBBE8EQCAKQQJ2QQFqQf7///8HcSEMQQAhBANAIAUgA0ECdCIDaiADIAdq/QAAACADIAZq/QAAAP3kAf0LAAAgBSABQQJ0IgNqIAMgB2r9AAAAIAMgBmr9AAAA/eQB/QsAACABQQRqIQMgAUEIaiEBIARBAmoiBCAMRw0ACwsgCkEEcQ0BIAUgA0ECdCIDaiADIAdq/QAAACADIAZq/QAAAP3kAf0LAAAgASEDDAELQQQhASALQQRrIgpBBE8EQCAKQQJ2QQFqQf7///8HcSEMQQAhBANAIAUgA0ECdCIDaiADIAZq/QAAACADIAdq/QAAAP3nAf0LAAAgBSABQQJ0IgNqIAMgBmr9AAAAIAMgB2r9AAAA/ecB/QsAACABQQRqIQMgAUEIaiEBIARBAmoiBCAMRw0ACwsgCkEEcQ0AIAUgA0ECdCIDaiADIAZq/QAAACADIAdq/QAAAP3nAf0LAAAgASEDCwJAIAMgC04NAAJAIAsgA2siAUEESQRAIAMhAQwBCyADIAEgC0EDcSIMayINaiEBIAn9ESERQQAhBANAIAUgAyAEakECdCIKaiAHIApq/QACACISIAYgCmr9AAIAIhP95AEgEyAS/eUBIBIgE/3mASATIBL95wEgEf0MAgAAAAIAAAACAAAAAgAAAP03/VIgEf0MAQAAAAEAAAABAAAAAQAAAP03/VIgEf0MAAAAAAAAAAAAAAAAAAAAAP03/VL9CwIAIARBBGoiBCANRw0ACyAMRQ0BCwNAIAcgAUECdCIDaioCACEPIAMgBmoqAgAhDiADIAVqAn0CQAJAAkACQCAJDgMAAQIDCyAPIA6SDAMLIA4gD5MMAgsgDyAOlAwBCyAOIA+VCzgCACABQQFqIgEgC0cNAAsLIAIgC2oiAiAISA0ACwwGCyAAKAIcDQAgAiAITg0FA0AgAkECdCIBIAAoAghqIQUgACgCBCABaiEGIAAoAgAgAiAAKAIQIgFvIgRBAnRqIQcgACgCFCEJQQAhAwJAIAEgBGsiASAIIAJrIgQgASAESBsiC0EESA0AAkACQAJAAkAgCQ4DAgEAAwtBBCEBIAtBBGsiCkEETwRAIApBAnZBAWpB/v///wdxIQxBACEEA0AgBSADQQJ0IgNqIAMgBmr9AAAAIAMgB2r9AAAA/eYB/QsAACAFIAFBAnQiA2ogAyAGav0AAAAgAyAHav0AAAD95gH9CwAAIAFBBGohAyABQQhqIQEgBEECaiIEIAxHDQALCyAKQQRxDQMgBSADQQJ0IgNqIAMgBmr9AAAAIAMgB2r9AAAA/eYB/QsAACABIQMMAwtBBCEBIAtBBGsiCkEETwRAIApBAnZBAWpB/v///wdxIQxBACEEA0AgBSADQQJ0IgNqIAMgB2r9AAAAIAMgBmr9AAAA/eUB/QsAACAFIAFBAnQiA2ogAyAHav0AAAAgAyAGav0AAAD95QH9CwAAIAFBBGohAyABQQhqIQEgBEECaiIEIAxHDQALCyAKQQRxDQIgBSADQQJ0IgNqIAMgB2r9AAAAIAMgBmr9AAAA/eUB/QsAACABIQMMAgtBBCEBIAkhAyALQQRrIgpBBE8EQCAKQQJ2QQFqQf7///8HcSEMQQAhBANAIAUgA0ECdCIDaiADIAZq/QAAACADIAdq/QAAAP3kAf0LAAAgBSABQQJ0IgNqIAMgBmr9AAAAIAMgB2r9AAAA/eQB/QsAACABQQRqIQMgAUEIaiEBIARBAmoiBCAMRw0ACwsgCkEEcQ0BIAUgA0ECdCIDaiADIAZq/QAAACADIAdq/QAAAP3kAf0LAAAgASEDDAELQQQhASALQQRrIgpBBE8EQCAKQQJ2QQFqQf7///8HcSEMQQAhBANAIAUgA0ECdCIDaiADIAdq/QAAACADIAZq/QAAAP3nAf0LAAAgBSABQQJ0IgNqIAMgB2r9AAAAIAMgBmr9AAAA/ecB/QsAACABQQRqIQMgAUEIaiEBIARBAmoiBCAMRw0ACwsgCkEEcQ0AIAUgA0ECdCIDaiADIAdq/QAAACADIAZq/QAAAP3nAf0LAAAgASEDCwJAIAMgC04NAAJAIAsgA2siAUEESQRAIAMhAQwBCyADIAEgC0EDcSIMayINaiEBIAn9ESERQQAhBANAIAUgAyAEakECdCIKaiAGIApq/QACACISIAcgCmr9AAIAIhP95AEgEyAS/eUBIBIgE/3mASATIBL95wEgEf0MAgAAAAIAAAACAAAAAgAAAP03/VIgEf0MAQAAAAEAAAABAAAAAQAAAP03/VIgEf0MAAAAAAAAAAAAAAAAAAAAAP03/VL9CwIAIARBBGoiBCANRw0ACyAMRQ0BCwNAIAYgAUECdCIDaioCACEPIAMgB2oqAgAhDiADIAVqAn0CQAJAAkACQCAJDgMAAQIDCyAPIA6SDAMLIA4gD5MMAgsgDyAOlAwBCyAOIA+VCzgCACABQQFqIgEgC0cNAAsLIAIgC2oiAiAISA0ACwwFCyACIAhODQQgACgCCCEEIAAoAgQhBSAAKAIAIQYgACgCFCEHIAAoAhwhCwNAIAIhAQJAAkACQCAJDgICAQALIAIgACgCEG8hAQwBC0EAIQELIAIhAwJAAkACQCALDgICAQALIAIgACgCEG8hAwwBC0EAIQMLIAUgA0ECdGoqAgAhDyAGIAFBAnRqKgIAIQ4gBCACQQJ0agJ9AkACQAJAAkAgBw4DAAECAwsgDyAOkgwDCyAOIA+TDAILIA8gDpQMAQsgDiAPlQs4AgAgAkEBaiICIAhHDQALDAQLA0AgAyACQQJ0IgFqIAEgBGr9AAAAIAEgCWr9AAAA/ecB/QsAACAAIgEhAiABQQRqIgAgCEwNAAsMAgsDQCADIAJBAnQiAWogESABIAlq/QAAAP3nAf0LAAAgACIBIQIgAUEEaiIAIAhMDQALCyABIAhODQECQCAIIAFrIgVBBEkEQCABIQIMAQsgASAFQXxxIgZqIQIgDv0TIREgBP0RIRJBACEAA0AgAyAAIAFqQQJ0IgdqIAcgCWr9AAIAIhMgEf3kASARIBP95QEgEyAR/eYBIBEgE/3nASAS/QwCAAAAAgAAAAIAAAACAAAA/Tf9UiAS/QwBAAAAAQAAAAEAAAABAAAA/Tf9UiAS/QwAAAAAAAAAAAAAAAAAAAAA/Tf9Uv0LAgAgAEEEaiIAIAZHDQALIAUgBkYNAgsDQCAJIAJBAnQiAGoqAgAhDyAAIANqAn0CQAJAAkACQCAEDgMAAQIDCyAPIA6SDAMLIA4gD5MMAgsgDyAOlAwBCyAOIA+VCzgCACACQQFqIgIgCEcNAAsMAQsgASAITg0AAkAgCCABayIGQQRJBEAgASECDAELIAEgBkF8cSIHaiECIAX9ESERQQAhAANAIAMgACABakECdCILaiAJIAtq/QACACISIAQgC2r9AAIAIhP95AEgEyAS/eUBIBIgE/3mASATIBL95wEgEf0MAgAAAAIAAAACAAAAAgAAAP03/VIgEf0MAQAAAAEAAAABAAAAAQAAAP03/VIgEf0MAAAAAAAAAAAAAAAAAAAAAP03/VL9CwIAIABBBGoiACAHRw0ACyAGIAdGDQELA0AgCSACQQJ0IgBqKgIAIQ8gACAEaioCACEOIAAgA2oCfQJAAkACQAJAIAUOAwABAgMLIA8gDpIMAwsgDiAPkwwCCyAPIA6UDAELIA4gD5ULOAIAIAJBAWoiAiAIRw0ACwsLawEBfyMAQSBrIggkACAGQQBKBEAgCCAENgIcIAggAjYCGCAIIAA2AhQgCCAFNgIIIAggAzYCBCAIIAE2AgAgCEEBIAcgB0EBTBs2AhAgCCAGNgIMQRYgCCAGQf8HakEKdhACCyAIQSBqJAALxQoDDn8FewJ9AkAgASACTg0AIAAoAgwhCyAAKAIYIgxBAEwEQCACIAFrQQJ0IgBFDQEgCyABQQJ0akEAIAD8CwAPCyAAKAIUIQ0gACgCACEOIAAoAgQhDwNAIAEgDGwhEEEAIQpDAAAAACEWA0AgDSAKQQh0IghrIQcgDyAKIBBqQcIAbGoiCS4AQCIEQf8HcSEGIARBgICAgHhxIQMgBEH//wNxIQUCQAJAIARBCnZBH3EiBARAIARBH0cNASADIAVBDXRyQYCAgPwHciEDDAILIAZFDQEgAyAFQQkgBmciA0Efc2t0QQ50QYCA/wNxIANBF3RrakGAgICYBGohAwwBCyAEQRd0IAZBDXRyIANqQYCAgMADaiEDCyAIIA5qIQYgA74hFwJAIAdB/wFMBEBBACEEQQAhAwNAAkAgAyAHTg0AIAMgCWotAABBA3EiBUEBTQRAIAUNASAEIAMgBmosAABrIQQMAQsgBCADIAZqLAAAaiEECyADQQFqIgNBIEcNAAtBACEDA0ACQCADQSBqIgUgB04NACADIAlqLQAAQQJ2QQNxIghBAU0EQCAIDQEgBCAFIAZqLAAAayEEDAELIAQgBSAGaiwAAGohBAsgA0EBaiIDQSBHDQALQQAhAwNAAkAgA0FAayIFIAdODQAgAyAJai0AAEEEdkEDcSIIQQFNBEAgCA0BIAQgBSAGaiwAAGshBAwBCyAEIAUgBmosAABqIQQLIANBAWoiA0EgRw0AC0EAIQMDQAJAIANB4ABqIgUgB04NACADIAlqLAAAIghBAE4EQCAIQT9LDQEgBCAFIAZqLAAAayEEDAELIAQgBSAGaiwAAGohBAsgA0EBaiIDQSBHDQALIAlBIGohCUEAIQMDQAJAIANBgAFqIgUgB04NACADIAlqLQAAQQNxIghBAU0EQCAIDQEgBCAFIAZqLAAAayEEDAELIAQgBSAGaiwAAGohBAsgA0EBaiIDQSBHDQALQQAhAwNAAkAgA0GgAWoiBSAHTg0AIAMgCWotAABBAnZBA3EiCEEBTQRAIAgNASAEIAUgBmosAABrIQQMAQsgBCAFIAZqLAAAaiEECyADQQFqIgNBIEcNAAtBACEDA0ACQCADQcABaiIFIAdODQAgAyAJai0AAEEEdkEDcSIIQQFNBEAgCA0BIAQgBSAGaiwAAGshBAwBCyAEIAUgBmosAABqIQQLIANBAWoiA0EgRw0AC0EAIQMDQAJAIANB4AFqIgUgB04NACADIAlqLAAAIghBAE4EQCAIQT9LDQEgBCAFIAZqLAAAayEEDAELIAQgBSAGaiwAAGohBAsgA0EBaiIDQSBHDQALDAELQQAhB0EBIQP9DAAAAAAAAAAAAAAAAAAAAAAhEQNAIAYgB0EHdGohBCAJIAdBBXRqIgf9AAAQIRMgB/0AAAAhFEEAIQcDQCAUIAdBAXQiBf1t/QwDAwMDAwMDAwMDAwMDAwMD/U4iEv2KASAEIAdBBXRqIgj9AAAAIhX9iAH9ugEgEf2uASAS/YkBIBX9hwH9ugH9rgEgEyAF/W39DAMDAwMDAwMDAwMDAwMDAwP9TiIR/YoBIAj9AAAQIhL9iAH9ugH9rgEgEf2JASAS/YcB/boB/a4BIREgB0EBaiIHQQRHDQALQQEhByADQQAhAw0ACyAR/RsAIBH9GwFqIBH9GwJqIBH9GwNqIAAoAgggCkECdGooAgBrIQQLIASyIBeUIBaSIRYgCkEBaiIKIAxHDQALIAsgAUECdGogACoCECAWlDgCACABQQFqIgEgAkcNAAsLC64DAgh/AXsjAEGgBGsiCSQAAkAgBUEATA0AIAZBAEwNACAFQf8BakEIdiELIAlBIGohDCAFQYGAAk8EQCALQQJ0EAYiDSEMIA1FDQELIAMgBGohECAAIAFqIQ4gBSEAA0BBACEDAkAgBSAKQQh0IgRMDQBBgAJBASAAIABBAUwbIgEgAUGAAk4bIQEgBCAOaiEPQQAhBCAAQQROBEAgAUH8A3EhBP0MAAAAAAAAAAAAAAAAAAAAACERA0AgESADIA9q/VwAAP2HAf2nAf2uASERIANBBGoiAyAERw0ACyARIBEgEf0NCAkKCwwNDg8AAQIDAAECA/2uASIRIBEgEf0NBAUGBwABAgMAAQIDAAECA/2uAf0bACEDIAEgBEYNAQsDQCADIAQgD2osAABqIQMgBEEBaiIEIAFHDQALCyAMIApBAnRqIAM2AgAgAEGAAmshACAKQQFqIgogC0cNAAsgCSALNgIcIAkgBTYCGCAJIAI4AhQgCSAMNgIMIAkgEDYCCCAJIA42AgQgCSAHIAhBAnRqNgIQQRUgCUEEaiAGEAIgDRAFCyAJQaAEaiQAC/MFAgJ7EH8gASACSARAIAAoAgQhCiAAKAIcIgVBfnEhDCAAKAIAIQ0gACgCGCEOIAAoAhQhDyAAKAIQIRAgACgCDCERIAAoAgghEiAFQQFKIRMDQCAPIAEgBWwiAGohBiAAIBBqIQcgACARaiEIIAAgEmohCUEAIQD9DAAAAAAAAAAAAAAAAAAAAAAiBCEDQQIhCyATBEADQCAAIAhqLQAAQQR0/QAEgAj9DKuqqj6rqqo+q6qqPquqqj795gEgACAJai0AAEEEdP0ABIAI/eQBIAAgB2otAABBBHT9AASACP0MOY7jPTmO4z05juM9OY7jPf3mAf3kASAAIAZqLQAAQQR0/QAEgAj9DCa0Fz0mtBc9JrQXPSa0Fz395gH95AEgCiAAQQR0aiIU/QAAAP3mASAD/eQBIQMgCCAAQQFyIgBqLQAAQQR0/QAEgAj9DKuqqj6rqqo+q6qqPquqqj795gEgACAJai0AAEEEdP0ABIAI/eQBIAAgB2otAABBBHT9AASACP0MOY7jPTmO4z05juM9OY7jPf3mAf3kASAAIAZqLQAAQQR0/QAEgAj9DCa0Fz0mtBc9JrQXPSa0Fz395gH95AEgFP0AABD95gEgBP3kASEEIAsiAEECaiILIAVMDQALIAwhAAsgACAFSARAA0AgACAIai0AAEEEdP0ABIAI/Qyrqqo+q6qqPquqqj6rqqo+/eYBIAAgCWotAABBBHT9AASACP3kASAAIAdqLQAAQQR0/QAEgAj9DDmO4z05juM9OY7jPTmO4z395gH95AEgACAGai0AAEEEdP0ABIAI/QwmtBc9JrQXPSa0Fz0mtBc9/eYB/eQBIAogAEEEdGr9AAAA/eYBIAP95AEhAyAAQQFqIgAgBUcNAAsLIA0gAUECdGogAyAE/eQBIgP9HwAgA/0fAZIgA/0fApIgA/0fA5IgDiABQQF0ai8BAEENdEGAwP//AHFBgICAwANqvpQ4AgAgAUEBaiIBIAJHDQALCwuFAQEBfyMAQSBrIgskACAIQQBKBEAgCyAHQQRtNgIcIAsgBSAGQQF0ajYCGCALIAAgAUECdGo2AgQgCyAJIApBAnRqNgIAIAsgAiADaiIANgIIIAsgACAEQQNsajYCFCALIAAgBEEBdGo2AhAgCyAAIARqNgIMQRQgCyAIEAILIAtBIGokAAvaBQIEewx/AkAgASACTg0AIAAoAgQhCCAAKAIAIQwgACgCCCEOIAAoAgwiB0EETgRAIAdBAXEhDyAIIAdB/P///wdxIglBBHRqIRAgByAJQQFyIhFGIRIDQCAOIAEgB2xqIQpBACEAQQQhC/0MAAAAAAAAAAAAAAAAAAAAACIEIQX9DAAAAAAAAAAAAAAAAAAAAAAhBv0MAAAAAAAAAAAAAAAAAAAAACEDA0AgCCAAQQR0aiIN/QAAMCAAIApqIgAtAANBBHT9AASACP3mASAE/eQBIQQgDf0AACAgAC0AAkEEdP0ABIAI/eYBIAX95AEhBSAN/QAAECAALQABQQR0/QAEgAj95gEgBv3kASEGIA39AAAAIAAtAABBBHT9AASACP3mASAD/eQBIQMgCyIAQQRqIgsgB0wNAAsCQCAHIAlMDQAgDwR/IBD9AAAAIAkgCmotAABBBHT9AASACP3mASAD/eQBIQMgEQUgCQshACASDQADQCAIIABBAWoiC0EEdGr9AAAAIAogC2otAABBBHT9AASACP3mASAIIABBBHRq/QAAACAAIApqLQAAQQR0/QAEgAj95gEgA/3kAf3kASEDIABBAmoiACAHRw0ACwsgDCABQQJ0aiAFIAb95AEgBP3kASAD/eQBIgP9HwAgA/0fAZIgA/0fApIgA/0fA5I4AgAgAUEBaiIBIAJHDQALDAELIAdBAEoEQCAHQQFGIQkDQCAMIAFBAnRqAnsgCP0AAAAgDiABIAdsaiIALQAAQQR0/QAEgAj95gEiAyAJDQAaIAj9AAAQIAAtAAFBBHT9AASACP3mASAD/eQBIgMgB0ECRg0AGiAI/QAAICAALQACQQR0/QAEgAj95gEgA/3kAQsiA/0fACAD/R8BkiAD/R8CkiAD/R8DkjgCACABQQFqIgEgAkcNAAsMAQsgAiABa0ECdCIARQ0AIAwgAUECdGpBACAA/AsACwtSAQF/IwBBEGsiCCQAIAVBAEoEQCAIIARBBG02AgwgCCACIANqNgIIIAggACABQQJ0ajYCBCAIIAYgB0ECdGo2AgBBEyAIIAUQAgsgCEEQaiQAC/YNBAN7Dn8DfQF+IwBBgAJrIgYkAAJAIAEgAk4NACAAKAIMIQ0gACgCECIOQQBMBEAgAiABa0ECdCIARQ0BIA0gAUECdGpBACAA/AsADAELIAAoAgAgASAObEHSAWxqIQcgACgCCCEPIAAoAgQhEANAQwAAAAAhFEEAIQoDQCAHLQDRASIIQQd2IQkgCEEIdCILIActANABckH/B3EhACAPIApBAnRqAn0CQCAIQQJ2QR9xIggEQCAIQR9HDQEgCUEfdEGAgIACQQAgABtyQYCAgPwHcr4MAgsgAARAIACzQwAAgDOUIhWMIBUgC8FBAEgbDAILIAlBH3S+DAELIAhBF3QgCUEfdHIgAEENdHJBgICAwANqvgshFSAQIApBCHRqIQsqAgAhFiAH/QAAICEEIAYgB/0AAIABIgP9DDAwMDAwMDAwMDAwMDAwMDD9TiAH/QAAACIFQQT9bf1Q/Qzg4ODg4ODg4ODg4ODg4ODg/W79CwRAIAYgA0EC/W39DDAwMDAwMDAwMDAwMDAwMDD9TiAEQQT9bf1Q/Qzg4ODg4ODg4ODg4ODg4ODg/W79CwRgIAYgA0EC/Wv9DDAwMDAwMDAwMDAwMDAwMDD9TiAE/QwPDw8PDw8PDw8PDw8PDw8P/U79UP0M4ODg4ODg4ODg4ODg4ODg4P1u/QsEICAGIANBBP1r/QwwMDAwMDAwMDAwMDAwMDAw/U4gBf0MDw8PDw8PDw8PDw8PDw8PD/1O/VD9DODg4ODg4ODg4ODg4ODg4OD9bv0LBAAgB/0AADAhBCAGIAf9AACQASID/QwwMDAwMDAwMDAwMDAwMDAw/U4gB/0AABAiBUEE/W39UP0M4ODg4ODg4ODg4ODg4ODg4P1u/QsEUCAGIANBAv1t/QwwMDAwMDAwMDAwMDAwMDAw/U4gBEEE/W39UP0M4ODg4ODg4ODg4ODg4ODg4P1u/QsEcCAGIANBAv1r/QwwMDAwMDAwMDAwMDAwMDAw/U4gBP0MDw8PDw8PDw8PDw8PDw8PD/1O/VD9DODg4ODg4ODg4ODg4ODg4OD9bv0LBDAgBiADQQT9a/0MMDAwMDAwMDAwMDAwMDAwMP1OIAX9DA8PDw8PDw8PDw8PDw8PDw/9Tv1Q/Qzg4ODg4ODg4ODg4ODg4ODg/W79CwQQIAf9AABgIQQgBiAH/QAAoAEiA/0MMDAwMDAwMDAwMDAwMDAwMP1OIAf9AABAIgVBBP1t/VD9DODg4ODg4ODg4ODg4ODg4OD9bv0LBMABIAYgA0EC/W39DDAwMDAwMDAwMDAwMDAwMDD9TiAEQQT9bf1Q/Qzg4ODg4ODg4ODg4ODg4ODg/W79CwTgASAGIANBAv1r/QwwMDAwMDAwMDAwMDAwMDAw/U4gBP0MDw8PDw8PDw8PDw8PDw8PD/1O/VD9DODg4ODg4ODg4ODg4ODg4OD9bv0LBKABIAYgA0EE/Wv9DDAwMDAwMDAwMDAwMDAwMDD9TiAF/QwPDw8PDw8PDw8PDw8PDw8P/U79UP0M4ODg4ODg4ODg4ODg4ODg4P1u/QsEgAEgB/0AAHAhBCAGIAf9AACwASID/QwwMDAwMDAwMDAwMDAwMDAw/U4gB/0AAFAiBUEE/W39UP0M4ODg4ODg4ODg4ODg4ODg4P1u/QsE0AEgBiADQQL9bf0MMDAwMDAwMDAwMDAwMDAwMP1OIARBBP1t/VD9DODg4ODg4ODg4ODg4ODg4OD9bv0LBPABIAYgA0EC/Wv9DDAwMDAwMDAwMDAwMDAwMDD9TiAE/QwPDw8PDw8PDw8PDw8PDw8P/U79UP0M4ODg4ODg4ODg4ODg4ODg4P1u/QsEsAEgBiADQQT9a/0MMDAwMDAwMDAwMDAwMDAwMP1OIAX9DA8PDw8PDw8PDw8PDw8PDw/9Tv1Q/Qzg4ODg4ODg4ODg4ODg4ODg/W79CwSQASAHQcABaiEMQQAhAEEBIQlCACEXA0AgAEEHdCERIAwgAEEDdGohEkEAIQADQCASIABBAXRqIhMwAAAgBiAAQQV0IBFqIghq/QAEACID/YcBIAggC2r9AAAAIgT9hwH9ugEgA/2IASAE/YgB/boB/a4BIgP9GwAgA/0bAWogA/0bAmogA/0bA2qsfiAXfCATMAABIAYgCEEQciIIav0ABAAiA/2HASAIIAtq/QAAACIE/YcB/boBIAP9iAEgBP2IAf26Af2uASID/RsAIAP9GwFqIAP9GwJqIAP9GwNqrH58IRcgAEEBaiIAQQRHDQALQQEhACAJQQAhCQ0ACyAHQdIBaiEHIBUgFpQgF7SUIBSSIRQgCkEBaiIKIA5HDQALIA0gAUECdGogFDgCACABQQFqIgEgAkcNAAsLIAZBgAJqJAALFgAgACABIAIgAyAEIAUgBiAHQRIQCQv+DQQDexF/An0BfiMAQYACayIGJAACQCABIAJODQAgAiABa0ECdCIKBEAgACgCDCABQQJ0akEAIAr8CwALIAAoAhAiDkEATA0AIAAoAgwhDyAAKAIUIRAgACgCACERIAAoAgghEiAAKAIEIRMDQCATIAtBCHRqIQ0gESALIBBsIAFqQdIBbGohByASIAtBAnRqKgIAIRggASEKA0AgBy0A0QEiCEEHdiEJIAhBCHQiDCAHLQDQAXJB/wdxIQACfQJAIAhBAnZBH3EiCARAIAhBH0cNASAJQR90QYCAgAJBACAAG3JBgICA/AdyvgwCCyAABEAgALNDAACAM5QiF4wgFyAMwUEASBsMAgsgCUEfdL4MAQsgCEEXdCAJQR90ciAAQQ10ckGAgIDAA2q+CyEXIAf9AAAgIQQgBiAH/QAAgAEiA/0MMDAwMDAwMDAwMDAwMDAwMP1OIAf9AAAAIgVBBP1t/VD9DODg4ODg4ODg4ODg4ODg4OD9bv0LBEAgBiADQQL9bf0MMDAwMDAwMDAwMDAwMDAwMP1OIARBBP1t/VD9DODg4ODg4ODg4ODg4ODg4OD9bv0LBGAgBiADQQL9a/0MMDAwMDAwMDAwMDAwMDAwMP1OIAT9DA8PDw8PDw8PDw8PDw8PDw/9Tv1Q/Qzg4ODg4ODg4ODg4ODg4ODg/W79CwQgIAYgA0EE/Wv9DDAwMDAwMDAwMDAwMDAwMDD9TiAF/QwPDw8PDw8PDw8PDw8PDw8P/U79UP0M4ODg4ODg4ODg4ODg4ODg4P1u/QsEACAH/QAAMCEEIAYgB/0AAJABIgP9DDAwMDAwMDAwMDAwMDAwMDD9TiAH/QAAECIFQQT9bf1Q/Qzg4ODg4ODg4ODg4ODg4ODg/W79CwRQIAYgA0EC/W39DDAwMDAwMDAwMDAwMDAwMDD9TiAEQQT9bf1Q/Qzg4ODg4ODg4ODg4ODg4ODg/W79CwRwIAYgA0EC/Wv9DDAwMDAwMDAwMDAwMDAwMDD9TiAE/QwPDw8PDw8PDw8PDw8PDw8P/U79UP0M4ODg4ODg4ODg4ODg4ODg4P1u/QsEMCAGIANBBP1r/QwwMDAwMDAwMDAwMDAwMDAw/U4gBf0MDw8PDw8PDw8PDw8PDw8PD/1O/VD9DODg4ODg4ODg4ODg4ODg4OD9bv0LBBAgB/0AAGAhBCAGIAf9AACgASID/QwwMDAwMDAwMDAwMDAwMDAw/U4gB/0AAEAiBUEE/W39UP0M4ODg4ODg4ODg4ODg4ODg4P1u/QsEwAEgBiADQQL9bf0MMDAwMDAwMDAwMDAwMDAwMP1OIARBBP1t/VD9DODg4ODg4ODg4ODg4ODg4OD9bv0LBOABIAYgA0EC/Wv9DDAwMDAwMDAwMDAwMDAwMDD9TiAE/QwPDw8PDw8PDw8PDw8PDw8P/U79UP0M4ODg4ODg4ODg4ODg4ODg4P1u/QsEoAEgBiADQQT9a/0MMDAwMDAwMDAwMDAwMDAwMP1OIAX9DA8PDw8PDw8PDw8PDw8PDw/9Tv1Q/Qzg4ODg4ODg4ODg4ODg4ODg/W79CwSAASAH/QAAcCEEIAYgB/0AALABIgP9DDAwMDAwMDAwMDAwMDAwMDD9TiAH/QAAUCIFQQT9bf1Q/Qzg4ODg4ODg4ODg4ODg4ODg/W79CwTQASAGIANBAv1t/QwwMDAwMDAwMDAwMDAwMDAw/U4gBEEE/W39UP0M4ODg4ODg4ODg4ODg4ODg4P1u/QsE8AEgBiADQQL9a/0MMDAwMDAwMDAwMDAwMDAwMP1OIAT9DA8PDw8PDw8PDw8PDw8PDw/9Tv1Q/Qzg4ODg4ODg4ODg4ODg4ODg/W79CwSwASAGIANBBP1r/QwwMDAwMDAwMDAwMDAwMDAw/U4gBf0MDw8PDw8PDw8PDw8PDw8PD/1O/VD9DODg4ODg4ODg4ODg4ODg4OD9bv0LBJABIAdBwAFqIQxBACEAQQEhCUIAIRkDQCAAQQd0IRQgDCAAQQN0aiEVQQAhAANAIBUgAEEBdGoiFjAAACAGIABBBXQgFGoiCGr9AAQAIgP9hwEgCCANav0AAAAiBP2HAf26ASAD/YgBIAT9iAH9ugH9rgEiA/0bACAD/RsBaiAD/RsCaiAD/RsDaqx+IBl8IBYwAAEgBiAIQRByIghq/QAEACID/YcBIAggDWr9AAAAIgT9hwH9ugEgA/2IASAE/YgB/boB/a4BIgP9GwAgA/0bAWogA/0bAmogA/0bA2qsfnwhGSAAQQFqIgBBBEcNAAtBASEAIAlBACEJDQALIA8gCkECdGoiACAAKgIAIBcgGJQgGbSUkjgCACAHQdIBaiEHIApBAWoiCiACRw0ACyALQQFqIgsgDkcNAAsLIAZBgAJqJAALFgAgACABIAIgAyAEIAUgBiAHQREQCQuoAQIEfwF9AkAgASACTg0AIAAoAgwiBUEASgRAIAAoAgAgASAFbEGwAWxqIQMgACgCBCEGA0BDAAAAACEHQQAhBANAIAMgBiAEQQp0ahAQIAeSIQcgA0GwAWohAyAEQQFqIgQgBUcNAAsgACgCCCABQQJ0aiAHOAIAIAFBAWoiASACRw0ACwwBCyACIAFrQQJ0IgJFDQAgACgCCCABQQJ0akEAIAL8CwALC2UBAX8jAEEgayIIJAACQCAEQQBMDQAgBUEATA0AIAggBTYCHCAIIARBCHY2AhggCCACIANqNgIMIAggBiAHQQJ0ajYCFCAIIAAgAUECdGo2AhBBECAIQQxqIAUQAgsgCEEgaiQAC68BAQl/AkAgASACTg0AIAIgAWtBAnQiAwRAIAAoAgggAUECdGpBACAD/AsACyAAKAIMIgZBAEwNACAAKAIQIQcgACgCACEIIAAoAgQhCQNAIAkgBEEKdGohCiAIIAQgB2wgAWpBsAFsaiEFIAEhAwNAIAAoAgggA0ECdGoiCyALKgIAIAUgChAQkjgCACAFQbABaiEFIANBAWoiAyACRw0ACyAEQQFqIgQgBkcNAAsLC2UBAX8jAEEgayIIJAACQCAEQQBMDQAgBUEATA0AIAggBTYCHCAIIARBCHY2AhggCCACIANqNgIMIAggBiAHQQJ0ajYCFCAIIAAgAUECdGo2AhBBDyAIQQxqIAUQAgsgCEEgaiQAC7oBAgV/AX0CQCABIAJODQAgACgCECIFQQBKBEAgACgCACABIAVsQZABbGohBCAAKAIIIQYgACgCBCEHA0BDAAAAACEIQQAhAwNAIAQgByADQQh0aiAGIANBAnRqKgIAEBEgCJIhCCAEQZABaiEEIANBAWoiAyAFRw0ACyAAKAIMIAFBAnRqIAg4AgAgAUEBaiIBIAJHDQALDAELIAIgAWtBAnQiAkUNACAAKAIMIAFBAnRqQQAgAvwLAAsLFgAgACABIAIgAyAEIAUgBiAHQQ4QCQvLAQIKfwJ9AkAgASACTg0AIAIgAWtBAnQiAwRAIAAoAgwgAUECdGpBACAD/AsACyAAKAIQIgZBAEwNACAAKAIUIQcgACgCACEIIAAoAgghCSAAKAIEIQoDQCAKIARBCHRqIQsgCCAEIAdsIAFqQZABbGohBSAJIARBAnRqKgIAIQ0gASEDA0AgBSALIA0QESEOIAAoAgwgA0ECdGoiDCAOIAwqAgCSOAIAIAVBkAFqIQUgA0EBaiIDIAJHDQALIARBAWoiBCAGRw0ACwsLFgAgACABIAIgAyAEIAUgBiAHQQ0QCQuzBAMCewt/AX0CQCABIAJODQAgACgCCCENIAAoAgwiCUEATARAIAIgAWtBAnQiAEUNASANIAFBAnRqQQAgAPwLAA8LIAAoAgAgASAJbEEibGohCiAJQSJsIQ4gACgCBCEPA0BDAAAAACEQQQAhCyAKIQADQCAPIAtBB3RqIQYgAC0AASIHQRh0QYCAgIB4cSEFIAAtAAAgB0EIdHIiCEH/B3EhDCAIQf//A3EhCAJAAkAgB0ECdkEfcSIHBEAgB0EfRw0BIAhBDXQgBXJBgICA/AdyIQUMAgsgDEUNASAFQYCAgJgEciAMZyIFQRd0ayAIQQkgBUEfc2t0QQ50QYCA/wNxciEFDAELIAdBF3QgDEENdHIgBWpBgICAwANqIQULIAb9AAAAIAD9AAACIgP9hwEiBP2nAf36Af3mASAG/QAAECAE/agB/foB/eYB/eQBIAb9AAAgIAP9iAEiA/2nAf36Af3mAf3kASAG/QAAMCAD/agB/foB/eYB/eQBIAb9AABAIAD9AAASIgP9hwEiBP2nAf36Af3mAf3kASAG/QAAUCAE/agB/foB/eYB/eQBIAb9AABgIAP9iAEiA/2nAf36Af3mAf3kASAG/QAAcCAD/agB/foB/eYB/eQBIgP9HwAgA/0fAZIgA/0fApIgA/0fA5IgBb6UIBCSIRAgAEEiaiEAIAtBAWoiCyAJRw0ACyANIAFBAnRqIBA4AgAgCiAOaiEKIAFBAWoiASACRw0ACwsLgQEBAX8jAEEgayIIJAACQCAFQQBMDQAgBEEATARAIAVBAnQiAEUNASAGIAdBAnRqQQAgAPwLAAwBCyAIIAU2AhwgCCAEQQV2NgIYIAggAiADajYCDCAIIAYgB0ECdGo2AhQgCCAAIAFBAnRqNgIQQQwgCEEMaiAFEAILIAhBIGokAAuoBAICewx/AkAgASACTg0AIAIgAWtBAnQiBQRAIAAoAgggAUECdGpBACAF/AsACyAAKAIMIgxBAEwNACAAKAIIIQ0gACgCECEOIAAoAgAhDyAAKAIEIRADQCAQIApBB3RqIQUgDyAKIA5sIAFqQSJsaiEHIAEhAANAIActAAEiCEEYdEGAgICAeHEhBiAHLQAAIAhBCHRyIgtB/wdxIQkgC0H//wNxIQsCQAJAIAhBAnZBH3EiCARAIAhBH0cNASALQQ10IAZyQYCAgPwHciEGDAILIAlFDQEgBkGAgICYBHIgCWciBkEXdGsgC0EJIAZBH3NrdEEOdEGAgP8DcXIhBgwBCyAIQRd0IAlBDXRyIAZqQYCAgMADaiEGCyANIABBAnRqIgkgBf0AAAAgB/0AAAIiA/2HASIE/acB/foB/eYBIAX9AAAQIAT9qAH9+gH95gH95AEgBf0AACAgA/2IASID/acB/foB/eYB/eQBIAX9AAAwIAP9qAH9+gH95gH95AEgBf0AAEAgB/0AABIiA/2HASIE/acB/foB/eYB/eQBIAX9AABQIAT9qAH9+gH95gH95AEgBf0AAGAgA/2IASID/acB/foB/eYB/eQBIAX9AABwIAP9qAH9+gH95gH95AEiA/0fACAD/R8BkiAD/R8CkiAD/R8DkiAGvpQgCSoCAJI4AgAgB0EiaiEHIABBAWoiACACRw0ACyAKQQFqIgogDEcNAAsLC4EBAQF/IwBBIGsiCCQAAkAgBUEATA0AIARBAEwEQCAFQQJ0IgBFDQEgBiAHQQJ0akEAIAD8CwAMAQsgCCAFNgIcIAggBEEFdjYCGCAIIAIgA2o2AgwgCCAGIAdBAnRqNgIUIAggACABQQJ0ajYCEEELIAhBDGogBRACCyAIQSBqJAALqgcDD38EewJ9AkAgASACTg0AIAAoAgghDSAAKAIQIgpBAEwEQCACIAFrQQJ0IgBFDQEgDSABQQJ0akEAIAD8CwAPCyAAKAIAIAEgCmxBGGxqIQsgCkEYbCEPIAAoAgwhECAAKAIEIREDQEMAAAAAIRZBACEJIAshBgNAIAYtAAEiDkEYdEGAgICAeHEhByAGLQADQQh0IgUgBi0AAnIhACAGLQAAIA5BCHRyIgNB/wdxIQwgA0H//wNxIQQgECAJQQJ0agJAAkAgDkECdkEfcSIDBEAgA0EfRw0BIARBDXQgB3JBgICA/AdyIQcMAgsgDEUNASAHQYCAgJgEciAMZyIDQRd0ayAEQQkgA0Efc2t0QQ50QYCA/wNxciEHDAELIANBF3QgDEENdHIgB2pBgICAwANqIQcLIBEgCUEHdGohBCoCACAAQf8HcSEIIAVBEHRBgICAgHhxIQUCQAJAIABBCnZBH3EiAwRAIANBH0cNASAAQQ10IAVyQYCAgPwHciEFDAILIAhFDQEgBUGAgICYBHIgCGciA0EXdGsgAEEJIANBH3NrdEEOdEGAgP8DcXIhBQwBCyADQRd0IAhBDXRyIAVqQYCAgMADaiEFCyAFvpQgFpIgBP0AAFAgBv0JAAQiFP0MAgICAgICAgIDAwMDAwMDA/0O/QwBAgQIECBAgAECBAgQIECA/U79DAAAAAAAAAAAAAAAAAAAAAD9JP0MEBAQEBAQEBAQEBAQEBAQEP1OIAb9AAAIIhVBBP1t/VAiE/2JASIS/aoB/foB/eYBIAT9AABAIBL9qQH9+gH95gH95AEgBP0AAGAgE/2KASIS/akB/foB/eYB/eQBIAT9AABwIBL9qgH9+gH95gH95AEgBP0AABAgFf0MDw8PDw8PDw8PDw8PDw8PD/1OIBT9DAAAAAAAAAAAAQEBAQEBAQH9Dv0MAQIECBAgQIABAgQIECBAgP1O/QwAAAAAAAAAAAAAAAAAAAAA/ST9DBAQEBAQEBAQEBAQEBAQEBD9Tv1QIhP9iQEiEv2qAf36Af3mAf3kASAE/QAAACAS/akB/foB/eYB/eQBIAT9AAAgIBP9igEiEv2pAf36Af3mAf3kASAE/QAAMCAS/aoB/foB/eYB/eQBIhL9HwAgEv0fAZIgEv0fApIgEv0fA5IgB76UkiEWIAZBGGohBiAJQQFqIgkgCkcNAAsgDSABQQJ0aiAWOAIAIAsgD2ohCyABQQFqIgEgAkcNAAsLCxYAIAAgASACIAMgBCAFIAYgB0EKEAoLowcCEH8EewJAIAEgAk4NACACIAFrQQJ0IgkEQCAAKAIIIAFBAnRqQQAgCfwLAAsgACgCECILQQBMDQAgACgCCCEMIAAoAgwhDSAAKAIUIQ4gACgCACEPIAAoAgQhEANAIA0gCkECdGohESAQIApBB3RqIQYgDyAKIA5sIAFqQRhsaiEHIAEhCQNAIActAAEiBEEYdEGAgICAeHEhBSAHLQADQQh0IhIgBy0AAnIhACAHLQAAIARBCHRyIghB/wdxIQMgCEH//wNxIQgCQAJAIARBAnZBH3EiBARAIARBH0cNASAIQQ10IAVyQYCAgPwHciEFDAILIANFDQEgBUGAgICYBHIgA2ciBUEXdGsgCEEJIAVBH3NrdEEOdEGAgP8DcXIhBQwBCyAEQRd0IANBDXRyIAVqQYCAgMADaiEFCyAAQf8HcSEEIBJBEHRBgICAgHhxIQMCQAJAIABBCnZBH3EiCARAIAhBH0cNASAAQQ10IANyQYCAgPwHciEDDAILIARFDQEgA0GAgICYBHIgBGciA0EXdGsgAEEJIANBH3NrdEEOdEGAgP8DcXIhAwwBCyAIQRd0IARBDXRyIANqQYCAgMADaiEDCyAMIAlBAnRqIgAgACoCACARKgIAIAO+lJIgBv0AAFAgB/0JAAQiE/0MAgICAgICAgIDAwMDAwMDA/0O/QwBAgQIECBAgAECBAgQIECA/U79DAAAAAAAAAAAAAAAAAAAAAD9JP0MEBAQEBAQEBAQEBAQEBAQEP1OIAf9AAAIIhRBBP1t/VAiFf2JASIW/aoB/foB/eYBIAb9AABAIBb9qQH9+gH95gH95AEgBv0AAGAgFf2KASIV/akB/foB/eYB/eQBIAb9AABwIBX9qgH9+gH95gH95AEgBv0AABAgFP0MDw8PDw8PDw8PDw8PDw8PD/1OIBP9DAAAAAAAAAAAAQEBAQEBAQH9Dv0MAQIECBAgQIABAgQIECBAgP1O/QwAAAAAAAAAAAAAAAAAAAAA/ST9DBAQEBAQEBAQEBAQEBAQEBD9Tv1QIhP9iQEiFP2qAf36Af3mAf3kASAG/QAAACAU/akB/foB/eYB/eQBIAb9AAAgIBP9igEiE/2pAf36Af3mAf3kASAG/QAAMCAT/aoB/foB/eYB/eQBIhP9HwAgE/0fAZIgE/0fApIgE/0fA5IgBb6UkjgCACAHQRhqIQcgCUEBaiIJIAJHDQALIApBAWoiCiALRw0ACwsLFgAgACABIAIgAyAEIAUgBiAHQQkQCgspACAAKAIkBEAgACABIAJBACAAKAIcEBIPCyAAQQAgACgCGCABIAIQEguUBgMMfwR7An0CQCABIAJODQAgACgCCCELIAAoAhAiCEEATARAIAIgAWtBAnQiAEUNASALIAFBAnRqQQAgAPwLAA8LIAAoAgAgASAIbEEWbGohCSAIQRZsIQwgACgCDCENIAAoAgQhDgNAQwAAAAAhE0EAIQYgCSEAA0AgDiAGQQd0aiEEIAAtAAEiBUEYdEGAgICAeHEhAyAALQAAIAVBCHRyIgdB/wdxIQogB0H//wNxIQcgDSAGQQJ0aioCACEUAkACQCAFQQJ2QR9xIgUEQCAFQR9HDQEgB0ENdCADckGAgID8B3IhAwwCCyAKRQ0BIANBgICAmARyIApnIgNBF3RrIAdBCSADQR9za3RBDnRBgID/A3FyIQMMAQsgBUEXdCAKQQ10ciADakGAgIDAA2ohAwsgBP0AAFAgAP0JAAIiD/0MAgICAgICAgIDAwMDAwMDA/0O/QwBAgQIECBAgAECBAgQIECA/U79DAAAAAAAAAAAAAAAAAAAAAD9JP0MEBAQEBAQEBAQEBAQEBAQEP1OIAD9AAAGIhBBBP1t/VAiEf2JASIS/aoB/foB/eYBIAT9AABAIBL9qQH9+gH95gH95AEgBP0AAGAgEf2KASIR/akB/foB/eYB/eQBIAT9AABwIBH9qgH9+gH95gH95AEgBP0AABAgEP0MDw8PDw8PDw8PDw8PDw8PD/1OIA/9DAAAAAAAAAAAAQEBAQEBAQH9Dv0MAQIECBAgQIABAgQIECBAgP1O/QwAAAAAAAAAAAAAAAAAAAAA/ST9DBAQEBAQEBAQEBAQEBAQEBD9Tv1QIg/9iQEiEP2qAf36Af3mAf3kASAE/QAAACAQ/akB/foB/eYB/eQBIAT9AAAgIA/9igEiD/2pAf36Af3mAf3kASAE/QAAMCAP/aoB/foB/eYB/eQBIg/9HwEgFEMAAIDBlJIgD/0fAJIgD/0fApIgD/0fA5IgA76UIBOSIRMgAEEWaiEAIAZBAWoiBiAIRw0ACyALIAFBAnRqIBM4AgAgCSAMaiEJIAFBAWoiASACRw0ACwsLFgAgACABIAIgAyAEIAUgBiAHQQgQCguJBgIOfwR7AkAgASACTg0AIAIgAWtBAnQiAwRAIAAoAgggAUECdGpBACAD/AsACyAAKAIQIgpBAEwNACAAKAIIIQsgACgCDCEMIAAoAhQhDSAAKAIAIQ4gACgCBCEPA0AgDCAGQQJ0aiEQIA8gBkEHdGohAyAOIAYgDWwgAWpBFmxqIQUgASEAA0AgBS0AASIHQRh0QYCAgIB4cSEEIAUtAAAgB0EIdHIiCUH/B3EhCCAJQf//A3EhCQJAAkAgB0ECdkEfcSIHBEAgB0EfRw0BIAlBDXQgBHJBgICA/AdyIQQMAgsgCEUNASAEQYCAgJgEciAIZyIEQRd0ayAJQQkgBEEfc2t0QQ50QYCA/wNxciEEDAELIAdBF3QgCEENdHIgBGpBgICAwANqIQQLIAsgAEECdGoiCCAD/QAAUCAF/QkAAiIR/QwCAgICAgICAgMDAwMDAwMD/Q79DAECBAgQIECAAQIECBAgQID9Tv0MAAAAAAAAAAAAAAAAAAAAAP0k/QwQEBAQEBAQEBAQEBAQEBAQ/U4gBf0AAAYiEkEE/W39UCIT/YkBIhT9qgH9+gH95gEgA/0AAEAgFP2pAf36Af3mAf3kASAD/QAAYCAT/YoBIhP9qQH9+gH95gH95AEgA/0AAHAgE/2qAf36Af3mAf3kASAD/QAAECAS/QwPDw8PDw8PDw8PDw8PDw8P/U4gEf0MAAAAAAAAAAABAQEBAQEBAf0O/QwBAgQIECBAgAECBAgQIECA/U79DAAAAAAAAAAAAAAAAAAAAAD9JP0MEBAQEBAQEBAQEBAQEBAQEP1O/VAiEf2JASIS/aoB/foB/eYB/eQBIAP9AAAAIBL9qQH9+gH95gH95AEgA/0AACAgEf2KASIR/akB/foB/eYB/eQBIAP9AAAwIBH9qgH9+gH95gH95AEiEf0fASAQKgIAQwAAgMGUkiAR/R8AkiAR/R8CkiAR/R8DkiAEvpQgCCoCAJI4AgAgBUEWaiEFIABBAWoiACACRw0ACyAGQQFqIgYgCkcNAAsLCxYAIAAgASACIAMgBCAFIAYgB0EHEAoL8QQDC38DewF9AkAgASACTg0AIAAoAgghCyAAKAIMIgdBAEwEQCACIAFrQQJ0IgBFDQEgCyABQQJ0akEAIAD8CwAPCyAAKAIAIAEgB2xBEmxqIQggB0ESbCEMIAAoAgQhDQNAQwAAAAAhEUEAIQkgCCEAA0AgDSAJQQd0aiEEIAAtAAEiBUEYdEGAgICAeHEhAyAALQAAIAVBCHRyIgZB/wdxIQogBkH//wNxIQYCQAJAIAVBAnZBH3EiBQRAIAVBH0cNASAGQQ10IANyQYCAgPwHciEDDAILIApFDQEgA0GAgICYBHIgCmciA0EXdGsgBkEJIANBH3NrdEEOdEGAgP8DcXIhAwwBCyAFQRd0IApBDXRyIANqQYCAgMADaiEDCyAE/QAAACAA/QAAAiIO/QwPDw8PDw8PDw8PDw8PDw8P/U79DPj4+Pj4+Pj4+Pj4+Pj4+Pj9biIP/YcBIhD9pwH9+gH95gEgBP0AABAgEP2oAf36Af3mAf3kASAE/QAAICAP/YgBIg/9pwH9+gH95gH95AEgBP0AADAgD/2oAf36Af3mAf3kASAE/QAAQCAOQQT9bf0M+Pj4+Pj4+Pj4+Pj4+Pj4+P1uIg79hwEiD/2nAf36Af3mAf3kASAE/QAAUCAP/agB/foB/eYB/eQBIAT9AABgIA79iAEiDv2nAf36Af3mAf3kASAE/QAAcCAO/agB/foB/eYB/eQBIg79HwAgDv0fAZIgDv0fApIgDv0fA5IgA76UIBGSIREgAEESaiEAIAlBAWoiCSAHRw0ACyALIAFBAnRqIBE4AgAgCCAMaiEIIAFBAWoiASACRw0ACwsLgQEBAX8jAEEgayIIJAACQCAFQQBMDQAgBEEATARAIAVBAnQiAEUNASAGIAdBAnRqQQAgAPwLAAwBCyAIIAU2AhwgCCAEQQV2NgIYIAggAiADajYCDCAIIAYgB0ECdGo2AhQgCCAAIAFBAnRqNgIQQQYgCEEMaiAFEAILIAhBIGokAAvmBAIMfwN7AkAgASACTg0AIAIgAWtBAnQiAwRAIAAoAgggAUECdGpBACAD/AsACyAAKAIMIgpBAEwNACAAKAIIIQsgACgCECEMIAAoAgAhDSAAKAIEIQ4DQCAOIAhBB3RqIQMgDSAIIAxsIAFqQRJsaiEFIAEhAANAIAUtAAEiBkEYdEGAgICAeHEhBCAFLQAAIAZBCHRyIglB/wdxIQcgCUH//wNxIQkCQAJAIAZBAnZBH3EiBgRAIAZBH0cNASAJQQ10IARyQYCAgPwHciEEDAILIAdFDQEgBEGAgICYBHIgB2ciBEEXdGsgCUEJIARBH3NrdEEOdEGAgP8DcXIhBAwBCyAGQRd0IAdBDXRyIARqQYCAgMADaiEECyALIABBAnRqIgcgA/0AAAAgBf0AAAIiD/0MDw8PDw8PDw8PDw8PDw8PD/1O/Qz4+Pj4+Pj4+Pj4+Pj4+Pj4/W4iEP2HASIR/acB/foB/eYBIAP9AAAQIBH9qAH9+gH95gH95AEgA/0AACAgEP2IASIQ/acB/foB/eYB/eQBIAP9AAAwIBD9qAH9+gH95gH95AEgA/0AAEAgD0EE/W39DPj4+Pj4+Pj4+Pj4+Pj4+Pj9biIP/YcBIhD9pwH9+gH95gH95AEgA/0AAFAgEP2oAf36Af3mAf3kASAD/QAAYCAP/YgBIg/9pwH9+gH95gH95AEgA/0AAHAgD/2oAf36Af3mAf3kASIP/R8AIA/9HwGSIA/9HwKSIA/9HwOSIAS+lCAHKgIAkjgCACAFQRJqIQUgAEEBaiIAIAJHDQALIAhBAWoiCCAKRw0ACwsLgQEBAX8jAEEgayIIJAACQCAFQQBMDQAgBEEATARAIAVBAnQiAEUNASAGIAdBAnRqQQAgAPwLAAwBCyAIIAU2AhwgCCAEQQV2NgIYIAggAiADajYCDCAIIAYgB0ECdGo2AhQgCCAAIAFBAnRqNgIQQQUgCEEMaiAFEAILIAhBIGokAAv6FQMSfwR7AX0jAEGAEGsiDCQAAkAgACgCGCIJQQBMDQAgASACTg0AIAAoAgggAUECdGohBSACIAFrQQJ0IQQgACgCFEECdCEHIAlBBE8EQCAJQfz///8HcSELA0AgBEUiCkUEQCAFIAMgB2xqQQAgBPwLAAsgCkUEQCAFIAcgA0EBcmxqQQAgBPwLAAsgCkUEQCAFIAcgA0ECcmxqQQAgBPwLAAsgCkUEQCAFIAcgA0EDcmxqQQAgBPwLAAsgA0EEaiEDIAhBBGoiCCALRw0ACwsgCUEDcSIKRQ0AA0AgBARAIAUgAyAHbGpBACAE/AsACyADQQFqIQMgBkEBaiIGIApHDQALCwJAIAAoAiAiC0EATA0AIAlBAUYEQCABQQF0IQ0gACgCCCABQQJ0aiEJIAAoAgAhDiACIAFrIgpBCEghEEEAIQcDQCAAKAIEIAAoAhAgB2xqIA1qIQggDiAHQQJ0aioCACEZQQAhBSAQRQRAIBn9EyEWQQAhBEEIIQYDQCAJIARBAnRqIgMgFiAIIARBAXRq/QAAACIX/akBQQ39qwEiFf0MAOD/DwDg/w8A4P8PAOD/D/1OIBX9DAAAgA8AAIAPAACADwAAgA/9TiIV/QwAAIAPAACADwAAgA8AAIAP/Tf9DAAAADgAAAA4AAAAOAAAADj9Tv2uASIY/QwAAIA4AACAOAAAgDgAAIA4/a4B/QwAAIC4AACAuAAAgLgAAIC4/eQBIBj9DAAAADgAAAA4AAAAOAAAADj9rgEgFf0MAAAAAAAAAAAAAAAAAAAAAP03/VIgF/2nAf0MAAAAgAAAAIAAAACAAAAAgP1O/VD95gEgA/0AAAD95AH9CwAAIAMgFiAX/aoBQQ39qwEiFf0MAOD/DwDg/w8A4P8PAOD/D/1OIBX9DAAAgA8AAIAPAACADwAAgA/9TiIV/QwAAIAPAACADwAAgA8AAIAP/Tf9DAAAADgAAAA4AAAAOAAAADj9Tv2uASIY/QwAAIA4AACAOAAAgDgAAIA4/a4B/QwAAIC4AACAuAAAgLgAAIC4/eQBIBj9DAAAADgAAAA4AAAAOAAAADj9rgEgFf0MAAAAAAAAAAAAAAAAAAAAAP03/VIgF/2oAf0MAAAAgAAAAIAAAACAAAAAgP1O/VD95gEgA/0AABD95AH9CwAQIAYiBSIEQQhqIgYgCkwNAAsLAkAgBSAKTg0AAkAgAiABIAVqayIGQQRJBEAgBSEDDAELIAUgBkF8cSILaiEDIBn9EyEXQQAhBANAIAkgBCAFaiIRQQJ0aiISIBcgCCARQQF0av1dAAAiFv0M/3//f/9//3//f/9//3//f/1O/akBQQ39qwEiFSAV/QwAAIAPAACADwAAgA8AAIAP/U4iFf0MAACADwAAgA8AAIAPAACAD/03/QwAAAA4AAAAOAAAADgAAAA4/U79rgEiGP0MAACAOAAAgDgAAIA4AACAOP2uAf0MAACAuAAAgLgAAIC4AACAuP3kASAY/QwAAAA4AAAAOAAAADgAAAA4/a4BIBX9DAAAAAAAAAAAAAAAAAAAAAD9N/1SIBb9pwH9DAAAAIAAAACAAAAAgAAAAID9Tv1Q/eYBIBL9AAIA/eQB/QsCACAEQQRqIgQgC0cNAAsgBiALRg0BCwNAIAkgA0ECdGoiBCAZIAggA0EBdGovAAAiBUH//wFxQQ10IgZBgICAwANBACAGQYCAgPwAcSIGQYCAgPwARhtqIgtBgICAwANqIAtBgICAxANqvkMAAIC4krwgBhsgBUEQdEGAgICAeHFyvpQgBCoCAJI4AgAgA0EBaiIDIApHDQALCyAHQQFqIgcgACgCIEgNAAsMAQsgASACTg0AIAIgAWshEQNAIAtBAEoEQEGABCACIAFrIg4gDkGABE4bIQZBgAQgESANQQl0ayIDIANBgAROGyIQQQNxIQogAUEBdCESQQAhByAOQQdKIRQDQCAAKAIEIAAoAhAgB2xqIBJqIQhBCCEEQQAhAyAUBEADQCAMIANBAnRqIgUgCCADQQF0av0AAAAiF/2qAUEN/asBIhb9DADg/w8A4P8PAOD/DwDg/w/9TiAW/QwAAIAPAACADwAAgA8AAIAP/U4iFv0MAACADwAAgA8AAIAPAACAD/03/QwAAAA4AAAAOAAAADgAAAA4/U79rgEiFf0MAACAOAAAgDgAAIA4AACAOP2uAf0MAACAuAAAgLgAAIC4AACAuP3kASAV/QwAAAA4AAAAOAAAADgAAAA4/a4BIBb9DAAAAAAAAAAAAAAAAAAAAAD9N/1SIBf9qAH9DAAAAIAAAACAAAAAgAAAAID9Tv1Q/QsEECAFIBf9qQFBDf2rASIW/QwA4P8PAOD/DwDg/w8A4P8P/U4gFv0MAACADwAAgA8AAIAPAACAD/1OIhb9DAAAgA8AAIAPAACADwAAgA/9N/0MAAAAOAAAADgAAAA4AAAAOP1O/a4BIhX9DAAAgDgAAIA4AACAOAAAgDj9rgH9DAAAgLgAAIC4AACAuAAAgLj95AEgFf0MAAAAOAAAADgAAAA4AAAAOP2uASAW/QwAAAAAAAAAAAAAAAAAAAAA/Tf9UiAX/acB/QwAAACAAAAAgAAAAIAAAACA/U79UP0LBAAgBCIDQQhqIgQgBkwNAAsLAkAgAyAGTg0AAkAgECADayIEQQRJBEAgAyEEDAELIAMgBCAKayIPaiEEQQAhBQNAIAwgAyAFaiITQQJ0aiAIIBNBAXRq/V0AACIX/Qz/f/9//3//f/9//3//f/9//U79qQFBDf2rASIWIBb9DAAAgA8AAIAPAACADwAAgA/9TiIW/QwAAIAPAACADwAAgA8AAIAP/Tf9DAAAADgAAAA4AAAAOAAAADj9Tv2uASIV/QwAAIA4AACAOAAAgDgAAIA4/a4B/QwAAIC4AACAuAAAgLgAAIC4/eQBIBX9DAAAADgAAAA4AAAAOAAAADj9rgEgFv0MAAAAAAAAAAAAAAAAAAAAAP03/VIgF/2nAf0MAAAAgAAAAIAAAACAAAAAgP1O/VD9CwIAIAVBBGoiBSAPRw0ACyAKRQ0BCwNAIAwgBEECdGogCCAEQQF0ai8AACIDQf//AXFBDXQiBUGAgIDAA0EAIAVBgICA/ABxIgVBgICA/ABGG2oiD0GAgIDAA2ogD0GAgIDEA2q+QwAAgLiSvCAFGyADQRB0QYCAgIB4cXI2AgAgBEEBaiIEIAZHDQALC0EAIQggCUEASgRAA0AgACgCCCAAKAIUIAhsQQJ0aiABQQJ0aiEJIAAoAgAgACgCDCAIbEECdGogB0ECdGoqAgAhGUEAIQMgDkEETgRAIBn9EyEXQQAhBEEEIQUDQCAJIARBAnQiA2oiBCADIAxq/QAEACAX/eYBIAT9AAAA/eQB/QsAACAFIgMhBCADQQRqIgUgBkwNAAsLAkAgAyAGTg0AAkAgECADayIEQQRJBEAgAyEEDAELIAMgBCAKayILaiEEIBn9EyEXQQAhBQNAIAkgAyAFakECdCIPaiITIBP9AAIAIAwgD2r9AAIAIBf95gH95AH9CwIAIAVBBGoiBSALRw0ACyAKRQ0BCwNAIAkgBEECdCIDaiIFIAUqAgAgAyAMaioCACAZlJI4AgAgBEEBaiIEIAZHDQALCyAIQQFqIgggACgCGCIJSA0ACyAAKAIgIQsLIAdBAWoiByALSA0ACwsgDUEBaiENIAFBgARqIgEgAkgNAAsLIAxBgBBqJAALmwEBAX8jAEEwayIMJAACQCAJQQBMDQAgCkEATA0AIAwgCDYCHCAMIAU2AhggDCACNgIUIAwgCTYCICAMIAo2AiQgDCAJIApPNgIsIAwgC0EAIAtBAEobNgIoIAwgBiAHQQJ0ajYCECAMIAMgBEECdGo2AgwgDCAAIAFBAnRqNgIIQQEgDEEIaiAJIAogCSAKSxsQAgsgDEEwaiQACwIACwvbMzMAQYIIC06AvwAAgL8AAIC/AACAvwAAAAAAAIC/AACAvwAAgL8AAIA/AACAvwAAgL8AAIC/AAAAQAAAgL8AAIC/AACAvwAAgL8AAAAAAACAvwAAgL8AQdoIC+oBgL8AAIC/AACAPwAAAAAAAIC/AACAvwAAAEAAAAAAAACAvwAAgL8AAIC/AACAPwAAgL8AAIC/AAAAAAAAgD8AAIC/AACAvwAAgD8AAIA/AACAvwAAgL8AAABAAACAPwAAgL8AAIC/AACAvwAAAEAAAIC/AACAvwAAAAAAAABAAACAvwAAgL8AAIA/AAAAQAAAgL8AAIC/AAAAQAAAAEAAAIC/AACAvwAAgL8AAIC/AAAAAAAAgL8AAAAAAACAvwAAAAAAAIC/AACAPwAAgL8AAAAAAACAvwAAAEAAAIC/AAAAAAAAgL8AAIC/AEHOCgsCgL8AQd4KCwaAvwAAgD8AQe4KCwaAvwAAAEAAQf4KC9IBgL8AAIC/AACAPwAAAAAAAIC/AAAAAAAAgD8AAAAAAACAvwAAgD8AAIA/AAAAAAAAgL8AAABAAACAPwAAAAAAAIC/AACAvwAAAEAAAAAAAACAvwAAAAAAAABAAAAAAAAAgL8AAIA/AAAAQAAAAAAAAIC/AAAAQAAAAEAAAAAAAACAvwAAgL8AAIC/AACAPwAAgL8AAAAAAACAvwAAgD8AAIC/AACAPwAAgL8AAIA/AACAvwAAAEAAAIC/AACAPwAAgL8AAIC/AAAAAAAAgD8AAIC/AEHaDAv2AYA/AACAvwAAgD8AAAAAAACAPwAAgL8AAABAAAAAAAAAgD8AAIC/AACAvwAAgD8AAIA/AACAvwAAAAAAAIA/AACAPwAAgL8AAIA/AACAPwAAgD8AAIC/AAAAQAAAgD8AAIA/AACAvwAAgL8AAABAAACAPwAAgL8AAAAAAAAAQAAAgD8AAIC/AACAPwAAAEAAAIA/AACAvwAAAEAAAABAAACAPwAAgL8AAIC/AACAvwAAAEAAAIC/AAAAAAAAgL8AAABAAACAvwAAgD8AAIC/AAAAQAAAgL8AAABAAACAvwAAAEAAAIC/AACAvwAAAAAAAABAAACAvwBB2w4LsQFAAACAvwAAgD8AAAAAAAAAQAAAgL8AAABAAAAAAAAAAEAAAIC/AACAvwAAgD8AAABAAACAvwAAAAAAAIA/AAAAQAAAgL8AAIA/AACAPwAAAEAAAIC/AAAAQAAAgD8AAABAAACAvwAAgL8AAABAAAAAQAAAgL8AAAAAAAAAQAAAAEAAAIC/AACAPwAAAEAAAABAAACAvwAAAEAAAABAAAAAQAAAgL8AAIC/AACAvwAAgL8AQZYQCzaAvwAAgL8AAAAAAACAPwAAgL8AAIC/AAAAAAAAAEAAAIC/AACAvwAAAAAAAIC/AAAAAAAAgL8AQdoQCzKAvwAAAAAAAIA/AAAAAAAAgL8AAAAAAAAAQAAAAAAAAIC/AAAAAAAAgL8AAIA/AACAvwBBlhELNoA/AACAvwAAAAAAAIA/AACAPwAAgL8AAAAAAAAAQAAAgD8AAIC/AAAAAAAAgL8AAABAAACAvwBB1xELMUAAAIC/AAAAAAAAgD8AAABAAACAvwAAAAAAAABAAAAAQAAAgL8AAAAAAACAvwAAgL8AQZYSCwKAvwBBohILBoA/AACAvwBBsxILBUAAAIC/AEHCEgsCgL8AQeISCwKAPwBB8xILAUAAQYITCwaAvwAAgD8AQZYTCwKAPwBBohMLBoA/AACAPwBBsxMLBUAAAIA/AEHCEwsGgL8AAABAAEHXEwsBQABB4hMLBoA/AAAAQABB8xMLBUAAAABAAEGCFAsKgL8AAIC/AACAPwBBlhQLNoC/AACAPwAAAAAAAIA/AACAvwAAgD8AAAAAAAAAQAAAgL8AAIA/AAAAAAAAgL8AAAAAAACAPwBB2hQLMoA/AAAAAAAAgD8AAAAAAACAPwAAAAAAAABAAAAAAAAAgD8AAAAAAACAvwAAgD8AAIA/AEGWFQs2gD8AAIA/AAAAAAAAgD8AAIA/AACAPwAAAAAAAABAAACAPwAAgD8AAAAAAACAvwAAAEAAAIA/AEHXFQs1QAAAgD8AAAAAAACAPwAAAEAAAIA/AAAAAAAAAEAAAABAAACAPwAAAAAAAIC/AACAvwAAAEAAQZYWCzaAvwAAAEAAAAAAAACAPwAAgL8AAABAAAAAAAAAAEAAAIC/AAAAQAAAAAAAAIC/AAAAAAAAAEAAQdsWCzFAAAAAAAAAgD8AAAAAAAAAQAAAAAAAAABAAAAAAAAAAEAAAAAAAACAvwAAgD8AAABAAEGWFws2gD8AAABAAAAAAAAAgD8AAIA/AAAAQAAAAAAAAABAAACAPwAAAEAAAAAAAACAvwAAAEAAAABAAEHXFwt5QAAAAEAAAAAAAACAPwAAAEAAAABAAAAAAAAAAEAAAABAAAAAQAAAAAAAAIC/AACAvwAAgL8AAIA/AAAAAAAAgL8AAIC/AACAPwAAgD8AAIC/AACAvwAAgD8AAABAAACAvwAAgL8AAIA/AACAvwAAAAAAAIC/AACAPwBB2hgL6gGAvwAAgD8AAIA/AAAAAAAAgL8AAIA/AAAAQAAAAAAAAIC/AACAPwAAgL8AAIA/AACAvwAAgD8AAAAAAACAPwAAgL8AAIA/AACAPwAAgD8AAIC/AACAPwAAAEAAAIA/AACAvwAAgD8AAIC/AAAAQAAAgL8AAIA/AAAAAAAAAEAAAIC/AACAPwAAgD8AAABAAACAvwAAgD8AAABAAAAAQAAAgL8AAIA/AACAvwAAgL8AAAAAAACAPwAAAAAAAIC/AAAAAAAAgD8AAIA/AACAvwAAAAAAAIA/AAAAQAAAgL8AAAAAAACAPwAAgL8AQc4aCwKAPwBB3hoLBoA/AACAPwBB7hoLBoA/AAAAQABB/hoL0gGAPwAAgL8AAIA/AAAAAAAAgD8AAAAAAACAPwAAAAAAAIA/AACAPwAAgD8AAAAAAACAPwAAAEAAAIA/AAAAAAAAgD8AAIC/AAAAQAAAAAAAAIA/AAAAAAAAAEAAAAAAAACAPwAAgD8AAABAAAAAAAAAgD8AAABAAAAAQAAAAAAAAIA/AACAvwAAgL8AAIA/AACAPwAAAAAAAIC/AACAPwAAgD8AAIA/AACAvwAAgD8AAIA/AAAAQAAAgL8AAIA/AACAPwAAgL8AAAAAAACAPwAAgD8AQdocC/YBgD8AAIA/AACAPwAAAAAAAIA/AACAPwAAAEAAAAAAAACAPwAAgD8AAIC/AACAPwAAgD8AAIA/AAAAAAAAgD8AAIA/AACAPwAAgD8AAIA/AACAPwAAgD8AAABAAACAPwAAgD8AAIA/AACAvwAAAEAAAIA/AACAPwAAAAAAAABAAACAPwAAgD8AAIA/AAAAQAAAgD8AAIA/AAAAQAAAAEAAAIA/AACAPwAAgL8AAIC/AAAAQAAAgD8AAAAAAACAvwAAAEAAAIA/AACAPwAAgL8AAABAAACAPwAAAEAAAIC/AAAAQAAAgD8AAIC/AAAAAAAAAEAAAIA/AEHbHgv1AUAAAIA/AACAPwAAAAAAAABAAACAPwAAAEAAAAAAAAAAQAAAgD8AAIC/AACAPwAAAEAAAIA/AAAAAAAAgD8AAABAAACAPwAAgD8AAIA/AAAAQAAAgD8AAABAAACAPwAAAEAAAIA/AACAvwAAAEAAAABAAACAPwAAAAAAAABAAAAAQAAAgD8AAIA/AAAAQAAAAEAAAIA/AAAAQAAAAEAAAABAAACAPwAAgL8AAIC/AACAvwAAAEAAAAAAAACAvwAAgL8AAABAAACAPwAAgL8AAIC/AAAAQAAAAEAAAIC/AACAvwAAAEAAAIC/AAAAAAAAgL8AAABAAEHaIAvqAYC/AAAAQAAAgD8AAAAAAACAvwAAAEAAAABAAAAAAAAAgL8AAABAAACAvwAAgD8AAIC/AAAAQAAAAAAAAIA/AACAvwAAAEAAAIA/AACAPwAAgL8AAABAAAAAQAAAgD8AAIC/AAAAQAAAgL8AAABAAACAvwAAAEAAAAAAAAAAQAAAgL8AAABAAACAPwAAAEAAAIC/AAAAQAAAAEAAAABAAACAvwAAAEAAAIC/AACAvwAAAAAAAABAAAAAAAAAgL8AAAAAAAAAQAAAgD8AAIC/AAAAAAAAAEAAAABAAACAvwAAAAAAAABAAACAvwBBzyILAUAAQd8iCwVAAACAPwBB7yILBUAAAABAAEH/IgvRAUAAAIC/AACAPwAAAAAAAABAAAAAAAAAgD8AAAAAAAAAQAAAgD8AAIA/AAAAAAAAAEAAAABAAACAPwAAAAAAAABAAACAvwAAAEAAAAAAAAAAQAAAAAAAAABAAAAAAAAAAEAAAIA/AAAAQAAAAAAAAABAAAAAQAAAAEAAAAAAAAAAQAAAgL8AAIC/AACAPwAAAEAAAAAAAACAvwAAgD8AAABAAACAPwAAgL8AAIA/AAAAQAAAAEAAAIC/AACAPwAAAEAAAIC/AAAAAAAAgD8AAABAAEHaJAv2AYA/AAAAQAAAgD8AAAAAAACAPwAAAEAAAABAAAAAAAAAgD8AAABAAACAvwAAgD8AAIA/AAAAQAAAAAAAAIA/AACAPwAAAEAAAIA/AACAPwAAgD8AAABAAAAAQAAAgD8AAIA/AAAAQAAAgL8AAABAAACAPwAAAEAAAAAAAAAAQAAAgD8AAABAAACAPwAAAEAAAIA/AAAAQAAAAEAAAABAAACAPwAAAEAAAIC/AACAvwAAAEAAAABAAAAAAAAAgL8AAABAAAAAQAAAgD8AAIC/AAAAQAAAAEAAAABAAACAvwAAAEAAAABAAACAvwAAAAAAAABAAAAAQABB2yYLlQJAAAAAQAAAgD8AAAAAAAAAQAAAAEAAAABAAAAAAAAAAEAAAABAAACAvwAAgD8AAABAAAAAQAAAAAAAAIA/AAAAQAAAAEAAAIA/AACAPwAAAEAAAABAAAAAQAAAgD8AAABAAAAAQAAAgL8AAABAAAAAQAAAAEAAAAAAAAAAQAAAAEAAAABAAACAPwAAAEAAAABAAAAAQAAAAEAAAABAAAAAQAAAAED+gitlRxVnQAAAAAAAADhDAAD6/kIudr86O568mvcMvb39/////98/PFRVVVVVxT+RKxfPVVWlPxfQpGcREYE/AAAAAAAAyELvOfr+Qi7mPyTEgv+9v84/tfQM1whrrD/MUEbSq7KDP4Q6Tpvg11U/AEH+KAvaFPA/br+IGk87mzw1M/upPfbvP13c2JwTYHG8YYB3Pprs7z/RZocQel6QvIV/bugV4+8/E/ZnNVLSjDx0hRXTsNnvP/qO+SOAzou83vbdKWvQ7z9hyOZhTvdgPMibdRhFx+8/mdMzW+SjkDyD88bKPr7vP217g12mmpc8D4n5bFi17z/87/2SGrWOPPdHciuSrO8/0ZwvcD2+Pjyi0dMy7KPvPwtukIk0A2q8G9P+r2ab7z8OvS8qUlaVvFFbEtABk+8/VepOjO+AULzMMWzAvYrvPxb01bkjyZG84C2prpqC7z+vVVzp49OAPFGOpciYeu8/SJOl6hUbgLx7UX08uHLvPz0y3lXwH4+86o2MOPlq7z+/UxM/jImLPHXLb+tbY+8/JusRdpzZlrzUXASE4FvvP2AvOj737Jo8qrloMYdU7z+dOIbLguePvB3Z/CJQTe8/jcOmREFvijzWjGKIO0bvP30E5LAFeoA8ltx9kUk/7z+UqKjj/Y6WPDhidW56OO8/fUh08hhehzw/prJPzjHvP/LnH5grR4A83XziZUUr7z9eCHE/e7iWvIFj9eHfJO8/MasJbeH3gjzh3h/1nR7vP/q/bxqbIT28kNna0H8Y7z+0CgxygjeLPAsD5KaFEu8/j8vOiZIUbjxWLz6prwzvP7arsE11TYM8FbcxCv4G7z9MdKziAUKGPDHYTPxwAe8/SvjTXTndjzz/FmSyCPzuPwRbjjuAo4a88Z+SX8X27j9oUEvM7UqSvMupOjen8e4/ji1RG/gHmbxm2AVtruzuP9I2lD7o0XG895/lNNvn7j8VG86zGRmZvOWoE8Mt4+4/bUwqp0ifhTwiNBJMpt7uP4ppKHpgEpO8HICsBEXa7j9biRdIj6dYvCou9yEK1u4/G5pJZ5ssfLyXqFDZ9dHuPxGswmDtY0M8LYlhYAjO7j/vZAY7CWaWPFcAHe1Byu4/eQOh2uHMbjzQPMG1osbuPzASDz+O/5M83tPX8CrD7j+wr3q7zpB2PCcqNtXav+4/d+BU670dkzwN3f2ZsrzuP46jcQA0lI+8pyyddrK57j9Jo5PczN6HvEJmz6Latu4/XzgPvcbeeLyCT51WK7TuP/Zce+xGEoa8D5JdyqSx7j+O1/0YBTWTPNontTZHr+4/BZuKL7eYezz9x5fUEq3uPwlUHOLhY5A8KVRI3Qer7j/qxhlQhcc0PLdGWYomqe4/NcBkK+YylDxIIa0Vb6fuP592mWFK5Iy8Cdx2ueGl7j+oTe87xTOMvIVVOrB+pO4/rukriXhThLwgw8w0RqPuP1hYVnjdzpO8JSJVgjii7j9kGX6AqhBXPHOpTNRVoe4/KCJev++zk7zNO39mnqDuP4K5NIetEmq8v9oLdRKg7j/uqW2472djvC8aZTyyn+4/UYjgVD3cgLyElFH5fZ/uP88+Wn5kH3i8dF/s6HWf7j+wfYvASu6GvHSBpUian+4/iuZVHjIZhrzJZ0JW65/uP9PUCV7LnJA8P13eT2mg7j8dpU253DJ7vIcB63MUoe4/a8BnVP3slDwywTAB7aHuP1Vs1qvh62U8Yk7PNvOi7j9Cz7MvxaGIvBIaPlQnpO4/NDc78bZpk7wTzkyZiaXuPx7/GTqEXoC8rccjRhqn7j9uV3LYUNSUvO2SRJvZqO4/AIoOW2etkDyZZorZx6ruP7Tq8MEvt40826AqQuWs7j//58WcYLZlvIxEtRYyr+4/RF/zWYP2ezw2dxWZrrHuP4M9HqcfCZO8xv+RC1u07j8pHmyLuKldvOXFzbA3t+4/WbmQfPkjbLwPUsjLRLruP6r59CJDQ5K8UE7en4K97j9LjmbXbMqFvLoHynDxwO4/J86RK/yvcTyQ8KOCkcTuP7tzCuE10m08IyPjGWPI7j9jImIiBMWHvGXlXXtmzO4/1THi44YcizwzLUrsm9DuPxW7vNPRu5G8XSU+sgPV7j/SMe6cMcyQPFizMBOe2e4/s1pzboRphDy//XlVa97uP7SdjpfN34K8evPTv2vj7j+HM8uSdxqMPK3TWpmf6O4/+tnRSo97kLxmto0pB+7uP7qu3FbZw1W8+xVPuKLz7j9A9qY9DqSQvDpZ5Y1y+e4/NJOtOPTWaLxHXvvydv/uPzWKWGvi7pG8SgahMLAF7z/N3V8K1/90PNLBS5AeDO8/rJiS+vu9kbwJHtdbwhLvP7MMrzCubnM8nFKF3ZsZ7z+U/Z9cMuOOPHrQ/1+rIO8/rFkJ0Y/ghDxL0Vcu8SfvP2caTjivzWM8tecGlG0v7z9oGZJsLGtnPGmQ79wgN+8/0rXMgxiKgLz6w11VCz/vP2/6/z9drY+8fIkHSi1H7z9JqXU4rg2QvPKJDQiHT+8/pwc9poWjdDyHpPvcGFjvPw8iQCCekYK8mIPJFuNg7z+sksHVUFqOPIUy2wPmae8/S2sBrFk6hDxgtAHzIXPvPx8+tAch1YK8X5t7M5d87z/JDUc7uSqJvCmh9RRGhu8/04g6YAS2dDz2P4vnLpDvP3FynVHsxYM8g0zH+1Ga7z/wkdOPEvePvNqQpKKvpO8/fXQj4piujbzxZ44tSK/vPwggqkG8w448J1ph7hu67z8y66nDlCuEPJe6azcrxe8/7oXRMalkijxARW5bdtDvP+3jO+S6N468FL6crf3b7z+dzZFNO4l3PNiQnoHB5+8/icxgQcEFUzzxcY8rwvPvPwAAAAAAAPA/dIUV07DZ7z8PiflsWLXvP1FbEtABk+8/e1F9PLhy7z+quWgxh1TvPzhidW56OO8/4d4f9Z0e7z8VtzEK/gbvP8upOjen8e4/IjQSTKbe7j8tiWFgCM7uPycqNtXav+4/gk+dViu07j8pVEjdB6vuP4VVOrB+pO4/zTt/Zp6g7j90X+zodZ/uP4cB63MUoe4/E85MmYml7j/boCpC5azuP+XFzbA3t+4/kPCjgpHE7j9dJT6yA9XuP63TWpmf6O4/R1778nb/7j+cUoXdmxnvP2mQ79wgN+8/h6T73BhY7z9fm3szl3zvP9qQpKKvpO8/QEVuW3bQ7z8AAAAAAADoQpQjkUv4aqw/88T6UM6/zj/WUgz/Qi7mPwAAAAAAADhD/oIrZUcVR0CUI5FL+Gq8PvPE+lDOvy4/1lIM/0Iulj++8/h57GH2P96qjID3e9W/PYivSu1x9T/bbcCn8L7Sv7AQ8PA5lfQ/ZzpRf64e0L+FA7iwlcnzP+kkgqbYMcu/pWSIDBkN8z9Yd8AKT1fGv6COC3siXvI/AIGcxyuqwb8/NBpKSrvxP14OjM52Trq/uuWK8Fgj8T/MHGFaPJexv6cAmUE/lfA/HgzhOPRSor8AAAAAAADwPwAAAAAAAAAArEea/Yxg7j+EWfJdqqWqP6BqAh+zpOw/tC42qlNevD/m/GpXNiDrPwjbIHflJsU/LaqhY9HC6T9wRyINhsLLP+1BeAPmhug/4X6gyIsF0T9iSFP13GfnPwnutlcwBNQ/7zn6/kIu5j80g7hIow7Qv2oL4AtbV9U/I0EK8v7/378AQdg9CwOAIQE=" + const val MT_MJS: String = "YXN5bmMgZnVuY3Rpb24gY3JlYXRlU2thaW5ldEtlcm5lbHMobW9kdWxlQXJnPXt9KXt2YXIgbW9kdWxlUnRuO3ZhciBNb2R1bGU9bW9kdWxlQXJnO3ZhciBFTlZJUk9OTUVOVF9JU19XRUI9dHlwZW9mIHdpbmRvdz09Im9iamVjdCI7dmFyIEVOVklST05NRU5UX0lTX1dPUktFUj10eXBlb2YgV29ya2VyR2xvYmFsU2NvcGUhPSJ1bmRlZmluZWQiO3ZhciBFTlZJUk9OTUVOVF9JU19OT0RFPXR5cGVvZiBwcm9jZXNzPT0ib2JqZWN0IiYmcHJvY2Vzcy52ZXJzaW9ucz8ubm9kZSYmcHJvY2Vzcy50eXBlIT0icmVuZGVyZXIiO3ZhciBFTlZJUk9OTUVOVF9JU19QVEhSRUFEPUVOVklST05NRU5UX0lTX1dPUktFUiYmc2VsZi5uYW1lPy5zdGFydHNXaXRoKCJlbS1wdGhyZWFkIik7aWYoRU5WSVJPTk1FTlRfSVNfTk9ERSl7Y29uc3R7Y3JlYXRlUmVxdWlyZX09YXdhaXQgaW1wb3J0KCJtb2R1bGUiKTt2YXIgcmVxdWlyZT1jcmVhdGVSZXF1aXJlKGltcG9ydC5tZXRhLnVybC5zdGFydHNXaXRoKCJmaWxlOiIpP2ltcG9ydC5tZXRhLnVybDpwcm9jZXNzLmN3ZCgpKyIvIik7dmFyIHdvcmtlcl90aHJlYWRzPXJlcXVpcmUoIndvcmtlcl90aHJlYWRzIik7Z2xvYmFsLldvcmtlcj13b3JrZXJfdGhyZWFkcy5Xb3JrZXI7RU5WSVJPTk1FTlRfSVNfV09SS0VSPSF3b3JrZXJfdGhyZWFkcy5pc01haW5UaHJlYWQ7RU5WSVJPTk1FTlRfSVNfUFRIUkVBRD1FTlZJUk9OTUVOVF9JU19XT1JLRVImJndvcmtlcl90aHJlYWRzWyJ3b3JrZXJEYXRhIl09PSJlbS1wdGhyZWFkIn12YXIgYXJndW1lbnRzXz1bXTt2YXIgdGhpc1Byb2dyYW09Ii4vdGhpcy5wcm9ncmFtIjt2YXIgcXVpdF89KHN0YXR1cyx0b1Rocm93KT0+e3Rocm93IHRvVGhyb3d9O3ZhciBfc2NyaXB0TmFtZT1pbXBvcnQubWV0YS51cmw7dmFyIHNjcmlwdERpcmVjdG9yeT0iIjtmdW5jdGlvbiBsb2NhdGVGaWxlKHBhdGgpe2lmKE1vZHVsZVsibG9jYXRlRmlsZSJdKXtyZXR1cm4gTW9kdWxlWyJsb2NhdGVGaWxlIl0ocGF0aCxzY3JpcHREaXJlY3RvcnkpfXJldHVybiBzY3JpcHREaXJlY3RvcnkrcGF0aH12YXIgcmVhZEFzeW5jLHJlYWRCaW5hcnk7aWYoRU5WSVJPTk1FTlRfSVNfTk9ERSl7dmFyIGZzPXJlcXVpcmUoImZzIik7aWYoX3NjcmlwdE5hbWUuc3RhcnRzV2l0aCgiZmlsZToiKSl7c2NyaXB0RGlyZWN0b3J5PXJlcXVpcmUoInBhdGgiKS5kaXJuYW1lKHJlcXVpcmUoInVybCIpLmZpbGVVUkxUb1BhdGgoX3NjcmlwdE5hbWUpKSsiLyJ9cmVhZEJpbmFyeT1maWxlbmFtZT0+e2ZpbGVuYW1lPWlzRmlsZVVSSShmaWxlbmFtZSk/bmV3IFVSTChmaWxlbmFtZSk6ZmlsZW5hbWU7dmFyIHJldD1mcy5yZWFkRmlsZVN5bmMoZmlsZW5hbWUpO3JldHVybiByZXR9O3JlYWRBc3luYz1hc3luYyhmaWxlbmFtZSxiaW5hcnk9dHJ1ZSk9PntmaWxlbmFtZT1pc0ZpbGVVUkkoZmlsZW5hbWUpP25ldyBVUkwoZmlsZW5hbWUpOmZpbGVuYW1lO3ZhciByZXQ9ZnMucmVhZEZpbGVTeW5jKGZpbGVuYW1lLGJpbmFyeT91bmRlZmluZWQ6InV0ZjgiKTtyZXR1cm4gcmV0fTtpZihwcm9jZXNzLmFyZ3YubGVuZ3RoPjEpe3RoaXNQcm9ncmFtPXByb2Nlc3MuYXJndlsxXS5yZXBsYWNlKC9cXC9nLCIvIil9YXJndW1lbnRzXz1wcm9jZXNzLmFyZ3Yuc2xpY2UoMik7cXVpdF89KHN0YXR1cyx0b1Rocm93KT0+e3Byb2Nlc3MuZXhpdENvZGU9c3RhdHVzO3Rocm93IHRvVGhyb3d9fWVsc2UgaWYoRU5WSVJPTk1FTlRfSVNfV0VCfHxFTlZJUk9OTUVOVF9JU19XT1JLRVIpe3RyeXtzY3JpcHREaXJlY3Rvcnk9bmV3IFVSTCgiLiIsX3NjcmlwdE5hbWUpLmhyZWZ9Y2F0Y2h7fWlmKCFFTlZJUk9OTUVOVF9JU19OT0RFKXtpZihFTlZJUk9OTUVOVF9JU19XT1JLRVIpe3JlYWRCaW5hcnk9dXJsPT57dmFyIHhocj1uZXcgWE1MSHR0cFJlcXVlc3Q7eGhyLm9wZW4oIkdFVCIsdXJsLGZhbHNlKTt4aHIucmVzcG9uc2VUeXBlPSJhcnJheWJ1ZmZlciI7eGhyLnNlbmQobnVsbCk7cmV0dXJuIG5ldyBVaW50OEFycmF5KHhoci5yZXNwb25zZSl9fXJlYWRBc3luYz1hc3luYyB1cmw9Pnt2YXIgcmVzcG9uc2U9YXdhaXQgZmV0Y2godXJsLHtjcmVkZW50aWFsczoic2FtZS1vcmlnaW4ifSk7aWYocmVzcG9uc2Uub2spe3JldHVybiByZXNwb25zZS5hcnJheUJ1ZmZlcigpfXRocm93IG5ldyBFcnJvcihyZXNwb25zZS5zdGF0dXMrIiA6ICIrcmVzcG9uc2UudXJsKX19fWVsc2V7fXZhciBkZWZhdWx0UHJpbnQ9Y29uc29sZS5sb2cuYmluZChjb25zb2xlKTt2YXIgZGVmYXVsdFByaW50RXJyPWNvbnNvbGUuZXJyb3IuYmluZChjb25zb2xlKTtpZihFTlZJUk9OTUVOVF9JU19OT0RFKXt2YXIgdXRpbHM9cmVxdWlyZSgidXRpbCIpO3ZhciBzdHJpbmdpZnk9YT0+dHlwZW9mIGE9PSJvYmplY3QiP3V0aWxzLmluc3BlY3QoYSk6YTtkZWZhdWx0UHJpbnQ9KC4uLmFyZ3MpPT5mcy53cml0ZVN5bmMoMSxhcmdzLm1hcChzdHJpbmdpZnkpLmpvaW4oIiAiKSsiXG4iKTtkZWZhdWx0UHJpbnRFcnI9KC4uLmFyZ3MpPT5mcy53cml0ZVN5bmMoMixhcmdzLm1hcChzdHJpbmdpZnkpLmpvaW4oIiAiKSsiXG4iKX12YXIgb3V0PWRlZmF1bHRQcmludDt2YXIgZXJyPWRlZmF1bHRQcmludEVycjt2YXIgd2FzbUJpbmFyeTt2YXIgd2FzbU1vZHVsZTt2YXIgQUJPUlQ9ZmFsc2U7dmFyIEVYSVRTVEFUVVM7dmFyIGlzRmlsZVVSST1maWxlbmFtZT0+ZmlsZW5hbWUuc3RhcnRzV2l0aCgiZmlsZTovLyIpO2Z1bmN0aW9uIGdyb3dNZW1WaWV3cygpe2lmKHdhc21NZW1vcnkuYnVmZmVyIT1IRUFQOC5idWZmZXIpe3VwZGF0ZU1lbW9yeVZpZXdzKCl9fXZhciByZWFkeVByb21pc2VSZXNvbHZlLHJlYWR5UHJvbWlzZVJlamVjdDtpZihFTlZJUk9OTUVOVF9JU19OT0RFJiZFTlZJUk9OTUVOVF9JU19QVEhSRUFEKXt2YXIgcGFyZW50UG9ydD13b3JrZXJfdGhyZWFkc1sicGFyZW50UG9ydCJdO3BhcmVudFBvcnQub24oIm1lc3NhZ2UiLG1zZz0+Z2xvYmFsLm9ubWVzc2FnZT8uKHtkYXRhOm1zZ30pKTtPYmplY3QuYXNzaWduKGdsb2JhbFRoaXMse3NlbGY6Z2xvYmFsLHBvc3RNZXNzYWdlOm1zZz0+cGFyZW50UG9ydFsicG9zdE1lc3NhZ2UiXShtc2cpfSk7cHJvY2Vzcy5vbigidW5jYXVnaHRFeGNlcHRpb24iLGVycj0+e3Bvc3RNZXNzYWdlKHtjbWQ6InVuY2F1Z2h0RXhjZXB0aW9uIixlcnJvcjplcnJ9KTtwcm9jZXNzLmV4aXQoMSl9KX12YXIgc3RhcnRXb3JrZXI7aWYoRU5WSVJPTk1FTlRfSVNfUFRIUkVBRCl7dmFyIGluaXRpYWxpemVkSlM9ZmFsc2U7c2VsZi5vbnVuaGFuZGxlZHJlamVjdGlvbj1lPT57dGhyb3cgZS5yZWFzb258fGV9O2Z1bmN0aW9uIGhhbmRsZU1lc3NhZ2UoZSl7dHJ5e3ZhciBtc2dEYXRhPWVbImRhdGEiXTt2YXIgY21kPW1zZ0RhdGEuY21kO2lmKGNtZD09PSJsb2FkIil7bGV0IG1lc3NhZ2VRdWV1ZT1bXTtzZWxmLm9ubWVzc2FnZT1lPT5tZXNzYWdlUXVldWUucHVzaChlKTtzdGFydFdvcmtlcj0oKT0+e3Bvc3RNZXNzYWdlKHtjbWQ6ImxvYWRlZCJ9KTtmb3IobGV0IG1zZyBvZiBtZXNzYWdlUXVldWUpe2hhbmRsZU1lc3NhZ2UobXNnKX1zZWxmLm9ubWVzc2FnZT1oYW5kbGVNZXNzYWdlfTtmb3IoY29uc3QgaGFuZGxlciBvZiBtc2dEYXRhLmhhbmRsZXJzKXtpZighTW9kdWxlW2hhbmRsZXJdfHxNb2R1bGVbaGFuZGxlcl0ucHJveHkpe01vZHVsZVtoYW5kbGVyXT0oLi4uYXJncyk9Pntwb3N0TWVzc2FnZSh7Y21kOiJjYWxsSGFuZGxlciIsaGFuZGxlcixhcmdzfSl9O2lmKGhhbmRsZXI9PSJwcmludCIpb3V0PU1vZHVsZVtoYW5kbGVyXTtpZihoYW5kbGVyPT0icHJpbnRFcnIiKWVycj1Nb2R1bGVbaGFuZGxlcl19fXdhc21NZW1vcnk9bXNnRGF0YS53YXNtTWVtb3J5O3VwZGF0ZU1lbW9yeVZpZXdzKCk7d2FzbU1vZHVsZT1tc2dEYXRhLndhc21Nb2R1bGU7Y3JlYXRlV2FzbSgpO3J1bigpfWVsc2UgaWYoY21kPT09InJ1biIpe2VzdGFibGlzaFN0YWNrU3BhY2UobXNnRGF0YS5wdGhyZWFkX3B0cik7X19lbXNjcmlwdGVuX3RocmVhZF9pbml0KG1zZ0RhdGEucHRocmVhZF9wdHIsMCwwLDEsMCwwKTtQVGhyZWFkLnRocmVhZEluaXRUTFMoKTtfX2Vtc2NyaXB0ZW5fdGhyZWFkX21haWxib3hfYXdhaXQobXNnRGF0YS5wdGhyZWFkX3B0cik7aWYoIWluaXRpYWxpemVkSlMpe2luaXRpYWxpemVkSlM9dHJ1ZX10cnl7aW52b2tlRW50cnlQb2ludChtc2dEYXRhLnN0YXJ0X3JvdXRpbmUsbXNnRGF0YS5hcmcpfWNhdGNoKGV4KXtpZihleCE9InVud2luZCIpe3Rocm93IGV4fX19ZWxzZSBpZihtc2dEYXRhLnRhcmdldD09PSJzZXRpbW1lZGlhdGUiKXt9ZWxzZSBpZihjbWQ9PT0iY2hlY2tNYWlsYm94Iil7aWYoaW5pdGlhbGl6ZWRKUyl7Y2hlY2tNYWlsYm94KCl9fWVsc2UgaWYoY21kKXtlcnIoYHdvcmtlcjogcmVjZWl2ZWQgdW5rbm93biBjb21tYW5kICR7Y21kfWApO2Vycihtc2dEYXRhKX19Y2F0Y2goZXgpe19fZW1zY3JpcHRlbl90aHJlYWRfY3Jhc2hlZCgpO3Rocm93IGV4fX1zZWxmLm9ubWVzc2FnZT1oYW5kbGVNZXNzYWdlfXZhciB3YXNtTWVtb3J5O3ZhciBIRUFQOCxIRUFQVTgsSEVBUDE2LEhFQVBVMTYsSEVBUDMyLEhFQVBVMzIsSEVBUEYzMixIRUFQRjY0O3ZhciBIRUFQNjQsSEVBUFU2NDt2YXIgcnVudGltZUluaXRpYWxpemVkPWZhbHNlO2Z1bmN0aW9uIHVwZGF0ZU1lbW9yeVZpZXdzKCl7dmFyIGI9d2FzbU1lbW9yeS5idWZmZXI7SEVBUDg9bmV3IEludDhBcnJheShiKTtIRUFQMTY9bmV3IEludDE2QXJyYXkoYik7SEVBUFU4PW5ldyBVaW50OEFycmF5KGIpO0hFQVBVMTY9bmV3IFVpbnQxNkFycmF5KGIpO0hFQVAzMj1uZXcgSW50MzJBcnJheShiKTtIRUFQVTMyPW5ldyBVaW50MzJBcnJheShiKTtIRUFQRjMyPW5ldyBGbG9hdDMyQXJyYXkoYik7SEVBUEY2ND1uZXcgRmxvYXQ2NEFycmF5KGIpO0hFQVA2ND1uZXcgQmlnSW50NjRBcnJheShiKTtIRUFQVTY0PW5ldyBCaWdVaW50NjRBcnJheShiKX1mdW5jdGlvbiBpbml0TWVtb3J5KCl7aWYoRU5WSVJPTk1FTlRfSVNfUFRIUkVBRCl7cmV0dXJufWlmKE1vZHVsZVsid2FzbU1lbW9yeSJdKXt3YXNtTWVtb3J5PU1vZHVsZVsid2FzbU1lbW9yeSJdfWVsc2V7dmFyIElOSVRJQUxfTUVNT1JZPU1vZHVsZVsiSU5JVElBTF9NRU1PUlkiXXx8NjcxMDg4NjQ7d2FzbU1lbW9yeT1uZXcgV2ViQXNzZW1ibHkuTWVtb3J5KHtpbml0aWFsOklOSVRJQUxfTUVNT1JZLzY1NTM2LG1heGltdW06NjU1MzYsc2hhcmVkOnRydWV9KX11cGRhdGVNZW1vcnlWaWV3cygpfWZ1bmN0aW9uIHByZVJ1bigpe2lmKE1vZHVsZVsicHJlUnVuIl0pe2lmKHR5cGVvZiBNb2R1bGVbInByZVJ1biJdPT0iZnVuY3Rpb24iKU1vZHVsZVsicHJlUnVuIl09W01vZHVsZVsicHJlUnVuIl1dO3doaWxlKE1vZHVsZVsicHJlUnVuIl0ubGVuZ3RoKXthZGRPblByZVJ1bihNb2R1bGVbInByZVJ1biJdLnNoaWZ0KCkpfX1jYWxsUnVudGltZUNhbGxiYWNrcyhvblByZVJ1bnMpfWZ1bmN0aW9uIGluaXRSdW50aW1lKCl7cnVudGltZUluaXRpYWxpemVkPXRydWU7aWYoRU5WSVJPTk1FTlRfSVNfUFRIUkVBRClyZXR1cm4gc3RhcnRXb3JrZXIoKTt3YXNtRXhwb3J0c1sibyJdKCl9ZnVuY3Rpb24gcG9zdFJ1bigpe2lmKEVOVklST05NRU5UX0lTX1BUSFJFQUQpe3JldHVybn1pZihNb2R1bGVbInBvc3RSdW4iXSl7aWYodHlwZW9mIE1vZHVsZVsicG9zdFJ1biJdPT0iZnVuY3Rpb24iKU1vZHVsZVsicG9zdFJ1biJdPVtNb2R1bGVbInBvc3RSdW4iXV07d2hpbGUoTW9kdWxlWyJwb3N0UnVuIl0ubGVuZ3RoKXthZGRPblBvc3RSdW4oTW9kdWxlWyJwb3N0UnVuIl0uc2hpZnQoKSl9fWNhbGxSdW50aW1lQ2FsbGJhY2tzKG9uUG9zdFJ1bnMpfWZ1bmN0aW9uIGFib3J0KHdoYXQpe01vZHVsZVsib25BYm9ydCJdPy4od2hhdCk7d2hhdD0iQWJvcnRlZCgiK3doYXQrIikiO2Vycih3aGF0KTtBQk9SVD10cnVlO3doYXQrPSIuIEJ1aWxkIHdpdGggLXNBU1NFUlRJT05TIGZvciBtb3JlIGluZm8uIjt2YXIgZT1uZXcgV2ViQXNzZW1ibHkuUnVudGltZUVycm9yKHdoYXQpO3JlYWR5UHJvbWlzZVJlamVjdD8uKGUpO3Rocm93IGV9dmFyIHdhc21CaW5hcnlGaWxlO2Z1bmN0aW9uIGZpbmRXYXNtQmluYXJ5KCl7aWYoTW9kdWxlWyJsb2NhdGVGaWxlIl0pe3JldHVybiBsb2NhdGVGaWxlKCJza2FpbmV0X2tlcm5lbHNfbXQud2FzbSIpfXRyeXtyZXR1cm4gbmV3IFVSTCgic2thaW5ldF9rZXJuZWxzX210Lndhc20iLGltcG9ydC5tZXRhLnVybCkuaHJlZn1jYXRjaChlKXtyZXR1cm4gInNrYWluZXRfa2VybmVsc19tdC53YXNtIn19ZnVuY3Rpb24gZ2V0QmluYXJ5U3luYyhmaWxlKXtpZihmaWxlPT13YXNtQmluYXJ5RmlsZSYmd2FzbUJpbmFyeSl7cmV0dXJuIG5ldyBVaW50OEFycmF5KHdhc21CaW5hcnkpfWlmKHJlYWRCaW5hcnkpe3JldHVybiByZWFkQmluYXJ5KGZpbGUpfXRocm93ImJvdGggYXN5bmMgYW5kIHN5bmMgZmV0Y2hpbmcgb2YgdGhlIHdhc20gZmFpbGVkIn1hc3luYyBmdW5jdGlvbiBnZXRXYXNtQmluYXJ5KGJpbmFyeUZpbGUpe2lmKCF3YXNtQmluYXJ5KXt0cnl7dmFyIHJlc3BvbnNlPWF3YWl0IHJlYWRBc3luYyhiaW5hcnlGaWxlKTtyZXR1cm4gbmV3IFVpbnQ4QXJyYXkocmVzcG9uc2UpfWNhdGNoe319cmV0dXJuIGdldEJpbmFyeVN5bmMoYmluYXJ5RmlsZSl9YXN5bmMgZnVuY3Rpb24gaW5zdGFudGlhdGVBcnJheUJ1ZmZlcihiaW5hcnlGaWxlLGltcG9ydHMpe3RyeXt2YXIgYmluYXJ5PWF3YWl0IGdldFdhc21CaW5hcnkoYmluYXJ5RmlsZSk7dmFyIGluc3RhbmNlPWF3YWl0IFdlYkFzc2VtYmx5Lmluc3RhbnRpYXRlKGJpbmFyeSxpbXBvcnRzKTtyZXR1cm4gaW5zdGFuY2V9Y2F0Y2gocmVhc29uKXtlcnIoYGZhaWxlZCB0byBhc3luY2hyb25vdXNseSBwcmVwYXJlIHdhc206ICR7cmVhc29ufWApO2Fib3J0KHJlYXNvbil9fWFzeW5jIGZ1bmN0aW9uIGluc3RhbnRpYXRlQXN5bmMoYmluYXJ5LGJpbmFyeUZpbGUsaW1wb3J0cyl7aWYoIWJpbmFyeSYmIUVOVklST05NRU5UX0lTX05PREUpe3RyeXt2YXIgcmVzcG9uc2U9ZmV0Y2goYmluYXJ5RmlsZSx7Y3JlZGVudGlhbHM6InNhbWUtb3JpZ2luIn0pO3ZhciBpbnN0YW50aWF0aW9uUmVzdWx0PWF3YWl0IFdlYkFzc2VtYmx5Lmluc3RhbnRpYXRlU3RyZWFtaW5nKHJlc3BvbnNlLGltcG9ydHMpO3JldHVybiBpbnN0YW50aWF0aW9uUmVzdWx0fWNhdGNoKHJlYXNvbil7ZXJyKGB3YXNtIHN0cmVhbWluZyBjb21waWxlIGZhaWxlZDogJHtyZWFzb259YCk7ZXJyKCJmYWxsaW5nIGJhY2sgdG8gQXJyYXlCdWZmZXIgaW5zdGFudGlhdGlvbiIpfX1yZXR1cm4gaW5zdGFudGlhdGVBcnJheUJ1ZmZlcihiaW5hcnlGaWxlLGltcG9ydHMpfWZ1bmN0aW9uIGdldFdhc21JbXBvcnRzKCl7YXNzaWduV2FzbUltcG9ydHMoKTtyZXR1cm57YTp3YXNtSW1wb3J0c319YXN5bmMgZnVuY3Rpb24gY3JlYXRlV2FzbSgpe2Z1bmN0aW9uIHJlY2VpdmVJbnN0YW5jZShpbnN0YW5jZSxtb2R1bGUpe3dhc21FeHBvcnRzPWluc3RhbmNlLmV4cG9ydHM7d2FzbUV4cG9ydHM9YXBwbHlTaWduYXR1cmVDb252ZXJzaW9ucyh3YXNtRXhwb3J0cyk7cmVnaXN0ZXJUTFNJbml0KHdhc21FeHBvcnRzWyJXIl0pO3dhc21UYWJsZT13YXNtRXhwb3J0c1sicSJdO3dhc21Nb2R1bGU9bW9kdWxlO2Fzc2lnbldhc21FeHBvcnRzKHdhc21FeHBvcnRzKTtyZXR1cm4gd2FzbUV4cG9ydHN9ZnVuY3Rpb24gcmVjZWl2ZUluc3RhbnRpYXRpb25SZXN1bHQocmVzdWx0KXtyZXR1cm4gcmVjZWl2ZUluc3RhbmNlKHJlc3VsdFsiaW5zdGFuY2UiXSxyZXN1bHRbIm1vZHVsZSJdKX12YXIgaW5mbz1nZXRXYXNtSW1wb3J0cygpO2lmKE1vZHVsZVsiaW5zdGFudGlhdGVXYXNtIl0pe3JldHVybiBuZXcgUHJvbWlzZSgocmVzb2x2ZSxyZWplY3QpPT57TW9kdWxlWyJpbnN0YW50aWF0ZVdhc20iXShpbmZvLChtb2QsaW5zdCk9PntyZXNvbHZlKHJlY2VpdmVJbnN0YW5jZShtb2QsaW5zdCkpfSl9KX1pZihFTlZJUk9OTUVOVF9JU19QVEhSRUFEKXt2YXIgaW5zdGFuY2U9bmV3IFdlYkFzc2VtYmx5Lkluc3RhbmNlKHdhc21Nb2R1bGUsZ2V0V2FzbUltcG9ydHMoKSk7cmV0dXJuIHJlY2VpdmVJbnN0YW5jZShpbnN0YW5jZSx3YXNtTW9kdWxlKX13YXNtQmluYXJ5RmlsZT8/PWZpbmRXYXNtQmluYXJ5KCk7dmFyIHJlc3VsdD1hd2FpdCBpbnN0YW50aWF0ZUFzeW5jKHdhc21CaW5hcnksd2FzbUJpbmFyeUZpbGUsaW5mbyk7dmFyIGV4cG9ydHM9cmVjZWl2ZUluc3RhbnRpYXRpb25SZXN1bHQocmVzdWx0KTtyZXR1cm4gZXhwb3J0c31jbGFzcyBFeGl0U3RhdHVze25hbWU9IkV4aXRTdGF0dXMiO2NvbnN0cnVjdG9yKHN0YXR1cyl7dGhpcy5tZXNzYWdlPWBQcm9ncmFtIHRlcm1pbmF0ZWQgd2l0aCBleGl0KCR7c3RhdHVzfSlgO3RoaXMuc3RhdHVzPXN0YXR1c319dmFyIHRlcm1pbmF0ZVdvcmtlcj13b3JrZXI9Pnt3b3JrZXIudGVybWluYXRlKCk7d29ya2VyLm9ubWVzc2FnZT1lPT57fX07dmFyIGNsZWFudXBUaHJlYWQ9cHRocmVhZF9wdHI9Pnt2YXIgd29ya2VyPVBUaHJlYWQucHRocmVhZHNbcHRocmVhZF9wdHJdO1BUaHJlYWQucmV0dXJuV29ya2VyVG9Qb29sKHdvcmtlcil9O3ZhciBjYWxsUnVudGltZUNhbGxiYWNrcz1jYWxsYmFja3M9Pnt3aGlsZShjYWxsYmFja3MubGVuZ3RoPjApe2NhbGxiYWNrcy5zaGlmdCgpKE1vZHVsZSl9fTt2YXIgb25QcmVSdW5zPVtdO3ZhciBhZGRPblByZVJ1bj1jYj0+b25QcmVSdW5zLnB1c2goY2IpO3ZhciBydW5EZXBlbmRlbmNpZXM9MDt2YXIgZGVwZW5kZW5jaWVzRnVsZmlsbGVkPW51bGw7dmFyIHJlbW92ZVJ1bkRlcGVuZGVuY3k9aWQ9PntydW5EZXBlbmRlbmNpZXMtLTtNb2R1bGVbIm1vbml0b3JSdW5EZXBlbmRlbmNpZXMiXT8uKHJ1bkRlcGVuZGVuY2llcyk7aWYocnVuRGVwZW5kZW5jaWVzPT0wKXtpZihkZXBlbmRlbmNpZXNGdWxmaWxsZWQpe3ZhciBjYWxsYmFjaz1kZXBlbmRlbmNpZXNGdWxmaWxsZWQ7ZGVwZW5kZW5jaWVzRnVsZmlsbGVkPW51bGw7Y2FsbGJhY2soKX19fTt2YXIgYWRkUnVuRGVwZW5kZW5jeT1pZD0+e3J1bkRlcGVuZGVuY2llcysrO01vZHVsZVsibW9uaXRvclJ1bkRlcGVuZGVuY2llcyJdPy4ocnVuRGVwZW5kZW5jaWVzKX07dmFyIHNwYXduVGhyZWFkPXRocmVhZFBhcmFtcz0+e3ZhciB3b3JrZXI9UFRocmVhZC5nZXROZXdXb3JrZXIoKTtpZighd29ya2VyKXtyZXR1cm4gNn1QVGhyZWFkLnJ1bm5pbmdXb3JrZXJzLnB1c2god29ya2VyKTtQVGhyZWFkLnB0aHJlYWRzW3RocmVhZFBhcmFtcy5wdGhyZWFkX3B0cl09d29ya2VyO3dvcmtlci5wdGhyZWFkX3B0cj10aHJlYWRQYXJhbXMucHRocmVhZF9wdHI7dmFyIG1zZz17Y21kOiJydW4iLHN0YXJ0X3JvdXRpbmU6dGhyZWFkUGFyYW1zLnN0YXJ0Um91dGluZSxhcmc6dGhyZWFkUGFyYW1zLmFyZyxwdGhyZWFkX3B0cjp0aHJlYWRQYXJhbXMucHRocmVhZF9wdHJ9O2lmKEVOVklST05NRU5UX0lTX05PREUpe3dvcmtlci51bnJlZigpfXdvcmtlci5wb3N0TWVzc2FnZShtc2csdGhyZWFkUGFyYW1zLnRyYW5zZmVyTGlzdCk7cmV0dXJuIDB9O3ZhciBydW50aW1lS2VlcGFsaXZlQ291bnRlcj0wO3ZhciBrZWVwUnVudGltZUFsaXZlPSgpPT5ub0V4aXRSdW50aW1lfHxydW50aW1lS2VlcGFsaXZlQ291bnRlcj4wO3ZhciBzdGFja1NhdmU9KCk9Pl9lbXNjcmlwdGVuX3N0YWNrX2dldF9jdXJyZW50KCk7dmFyIHN0YWNrUmVzdG9yZT12YWw9Pl9fZW1zY3JpcHRlbl9zdGFja19yZXN0b3JlKHZhbCk7dmFyIHN0YWNrQWxsb2M9c3o9Pl9fZW1zY3JpcHRlbl9zdGFja19hbGxvYyhzeik7dmFyIHByb3h5VG9NYWluVGhyZWFkPShmdW5jSW5kZXgsZW1Bc21BZGRyLHN5bmMsLi4uY2FsbEFyZ3MpPT57dmFyIHNlcmlhbGl6ZWROdW1DYWxsQXJncz1jYWxsQXJncy5sZW5ndGgqMjt2YXIgc3A9c3RhY2tTYXZlKCk7dmFyIGFyZ3M9c3RhY2tBbGxvYyhzZXJpYWxpemVkTnVtQ2FsbEFyZ3MqOCk7dmFyIGI9YXJncz4+PjM7Zm9yKHZhciBpPTA7aTxjYWxsQXJncy5sZW5ndGg7aSsrKXt2YXIgYXJnPWNhbGxBcmdzW2ldO2lmKHR5cGVvZiBhcmc9PSJiaWdpbnQiKXsoZ3Jvd01lbVZpZXdzKCksSEVBUDY0KVtiKzIqaT4+PjBdPTFuOyhncm93TWVtVmlld3MoKSxIRUFQNjQpW2IrMippKzE+Pj4wXT1hcmd9ZWxzZXsoZ3Jvd01lbVZpZXdzKCksSEVBUDY0KVtiKzIqaT4+PjBdPTBuOyhncm93TWVtVmlld3MoKSxIRUFQRjY0KVtiKzIqaSsxPj4+MF09YXJnfX12YXIgcnRuPV9fZW1zY3JpcHRlbl9ydW5fanNfb25fbWFpbl90aHJlYWQoZnVuY0luZGV4LGVtQXNtQWRkcixzZXJpYWxpemVkTnVtQ2FsbEFyZ3MsYXJncyxzeW5jKTtzdGFja1Jlc3RvcmUoc3ApO3JldHVybiBydG59O2Z1bmN0aW9uIF9wcm9jX2V4aXQoY29kZSl7aWYoRU5WSVJPTk1FTlRfSVNfUFRIUkVBRClyZXR1cm4gcHJveHlUb01haW5UaHJlYWQoMCwwLDEsY29kZSk7RVhJVFNUQVRVUz1jb2RlO2lmKCFrZWVwUnVudGltZUFsaXZlKCkpe1BUaHJlYWQudGVybWluYXRlQWxsVGhyZWFkcygpO01vZHVsZVsib25FeGl0Il0/Lihjb2RlKTtBQk9SVD10cnVlfXF1aXRfKGNvZGUsbmV3IEV4aXRTdGF0dXMoY29kZSkpfWZ1bmN0aW9uIGV4aXRPbk1haW5UaHJlYWQocmV0dXJuQ29kZSl7aWYoRU5WSVJPTk1FTlRfSVNfUFRIUkVBRClyZXR1cm4gcHJveHlUb01haW5UaHJlYWQoMSwwLDAscmV0dXJuQ29kZSk7X2V4aXQocmV0dXJuQ29kZSl9dmFyIGV4aXRKUz0oc3RhdHVzLGltcGxpY2l0KT0+e0VYSVRTVEFUVVM9c3RhdHVzO2lmKEVOVklST05NRU5UX0lTX1BUSFJFQUQpe2V4aXRPbk1haW5UaHJlYWQoc3RhdHVzKTt0aHJvdyJ1bndpbmQifV9wcm9jX2V4aXQoc3RhdHVzKX07dmFyIF9leGl0PWV4aXRKUzt2YXIgUFRocmVhZD17dW51c2VkV29ya2VyczpbXSxydW5uaW5nV29ya2VyczpbXSx0bHNJbml0RnVuY3Rpb25zOltdLHB0aHJlYWRzOnt9LGluaXQoKXtpZighRU5WSVJPTk1FTlRfSVNfUFRIUkVBRCl7UFRocmVhZC5pbml0TWFpblRocmVhZCgpfX0saW5pdE1haW5UaHJlYWQoKXt2YXIgcHRocmVhZFBvb2xTaXplPTQ7d2hpbGUocHRocmVhZFBvb2xTaXplLS0pe1BUaHJlYWQuYWxsb2NhdGVVbnVzZWRXb3JrZXIoKX1hZGRPblByZVJ1bihhc3luYygpPT57dmFyIHB0aHJlYWRQb29sUmVhZHk9UFRocmVhZC5sb2FkV2FzbU1vZHVsZVRvQWxsV29ya2VycygpO2FkZFJ1bkRlcGVuZGVuY3koImxvYWRpbmctd29ya2VycyIpO2F3YWl0IHB0aHJlYWRQb29sUmVhZHk7cmVtb3ZlUnVuRGVwZW5kZW5jeSgibG9hZGluZy13b3JrZXJzIil9KX0sdGVybWluYXRlQWxsVGhyZWFkczooKT0+e2Zvcih2YXIgd29ya2VyIG9mIFBUaHJlYWQucnVubmluZ1dvcmtlcnMpe3Rlcm1pbmF0ZVdvcmtlcih3b3JrZXIpfWZvcih2YXIgd29ya2VyIG9mIFBUaHJlYWQudW51c2VkV29ya2Vycyl7dGVybWluYXRlV29ya2VyKHdvcmtlcil9UFRocmVhZC51bnVzZWRXb3JrZXJzPVtdO1BUaHJlYWQucnVubmluZ1dvcmtlcnM9W107UFRocmVhZC5wdGhyZWFkcz17fX0scmV0dXJuV29ya2VyVG9Qb29sOndvcmtlcj0+e3ZhciBwdGhyZWFkX3B0cj13b3JrZXIucHRocmVhZF9wdHI7ZGVsZXRlIFBUaHJlYWQucHRocmVhZHNbcHRocmVhZF9wdHJdO1BUaHJlYWQudW51c2VkV29ya2Vycy5wdXNoKHdvcmtlcik7UFRocmVhZC5ydW5uaW5nV29ya2Vycy5zcGxpY2UoUFRocmVhZC5ydW5uaW5nV29ya2Vycy5pbmRleE9mKHdvcmtlciksMSk7d29ya2VyLnB0aHJlYWRfcHRyPTA7X19lbXNjcmlwdGVuX3RocmVhZF9mcmVlX2RhdGEocHRocmVhZF9wdHIpfSx0aHJlYWRJbml0VExTKCl7UFRocmVhZC50bHNJbml0RnVuY3Rpb25zLmZvckVhY2goZj0+ZigpKX0sbG9hZFdhc21Nb2R1bGVUb1dvcmtlcjp3b3JrZXI9Pm5ldyBQcm9taXNlKG9uRmluaXNoZWRMb2FkaW5nPT57d29ya2VyLm9ubWVzc2FnZT1lPT57dmFyIGQ9ZVsiZGF0YSJdO3ZhciBjbWQ9ZC5jbWQ7aWYoZC50YXJnZXRUaHJlYWQmJmQudGFyZ2V0VGhyZWFkIT1fcHRocmVhZF9zZWxmKCkpe3ZhciB0YXJnZXRXb3JrZXI9UFRocmVhZC5wdGhyZWFkc1tkLnRhcmdldFRocmVhZF07aWYodGFyZ2V0V29ya2VyKXt0YXJnZXRXb3JrZXIucG9zdE1lc3NhZ2UoZCxkLnRyYW5zZmVyTGlzdCl9ZWxzZXtlcnIoYEludGVybmFsIGVycm9yISBXb3JrZXIgc2VudCBhIG1lc3NhZ2UgIiR7Y21kfSIgdG8gdGFyZ2V0IHB0aHJlYWQgJHtkLnRhcmdldFRocmVhZH0sIGJ1dCB0aGF0IHRocmVhZCBubyBsb25nZXIgZXhpc3RzIWApfXJldHVybn1pZihjbWQ9PT0iY2hlY2tNYWlsYm94Iil7Y2hlY2tNYWlsYm94KCl9ZWxzZSBpZihjbWQ9PT0ic3Bhd25UaHJlYWQiKXtzcGF3blRocmVhZChkKX1lbHNlIGlmKGNtZD09PSJjbGVhbnVwVGhyZWFkIil7Y2FsbFVzZXJDYWxsYmFjaygoKT0+Y2xlYW51cFRocmVhZChkLnRocmVhZCkpfWVsc2UgaWYoY21kPT09ImxvYWRlZCIpe3dvcmtlci5sb2FkZWQ9dHJ1ZTtpZihFTlZJUk9OTUVOVF9JU19OT0RFJiYhd29ya2VyLnB0aHJlYWRfcHRyKXt3b3JrZXIudW5yZWYoKX1vbkZpbmlzaGVkTG9hZGluZyh3b3JrZXIpfWVsc2UgaWYoZC50YXJnZXQ9PT0ic2V0aW1tZWRpYXRlIil7d29ya2VyLnBvc3RNZXNzYWdlKGQpfWVsc2UgaWYoY21kPT09InVuY2F1Z2h0RXhjZXB0aW9uIil7d29ya2VyLm9uZXJyb3IoZC5lcnJvcil9ZWxzZSBpZihjbWQ9PT0iY2FsbEhhbmRsZXIiKXtNb2R1bGVbZC5oYW5kbGVyXSguLi5kLmFyZ3MpfWVsc2UgaWYoY21kKXtlcnIoYHdvcmtlciBzZW50IGFuIHVua25vd24gY29tbWFuZCAke2NtZH1gKX19O3dvcmtlci5vbmVycm9yPWU9Pnt2YXIgbWVzc2FnZT0id29ya2VyIHNlbnQgYW4gZXJyb3IhIjtlcnIoYCR7bWVzc2FnZX0gJHtlLmZpbGVuYW1lfToke2UubGluZW5vfTogJHtlLm1lc3NhZ2V9YCk7dGhyb3cgZX07aWYoRU5WSVJPTk1FTlRfSVNfTk9ERSl7d29ya2VyLm9uKCJtZXNzYWdlIixkYXRhPT53b3JrZXIub25tZXNzYWdlKHtkYXRhfSkpO3dvcmtlci5vbigiZXJyb3IiLGU9Pndvcmtlci5vbmVycm9yKGUpKX12YXIgaGFuZGxlcnM9W107dmFyIGtub3duSGFuZGxlcnM9WyJvbkV4aXQiLCJvbkFib3J0IiwicHJpbnQiLCJwcmludEVyciJdO2Zvcih2YXIgaGFuZGxlciBvZiBrbm93bkhhbmRsZXJzKXtpZihNb2R1bGUucHJvcGVydHlJc0VudW1lcmFibGUoaGFuZGxlcikpe2hhbmRsZXJzLnB1c2goaGFuZGxlcil9fXdvcmtlci5wb3N0TWVzc2FnZSh7Y21kOiJsb2FkIixoYW5kbGVycyx3YXNtTWVtb3J5LHdhc21Nb2R1bGV9KX0pLGFzeW5jIGxvYWRXYXNtTW9kdWxlVG9BbGxXb3JrZXJzKCl7aWYoRU5WSVJPTk1FTlRfSVNfUFRIUkVBRCl7cmV0dXJufWxldCBwdGhyZWFkUG9vbFJlYWR5PVByb21pc2UuYWxsKFBUaHJlYWQudW51c2VkV29ya2Vycy5tYXAoUFRocmVhZC5sb2FkV2FzbU1vZHVsZVRvV29ya2VyKSk7cmV0dXJuIHB0aHJlYWRQb29sUmVhZHl9LGFsbG9jYXRlVW51c2VkV29ya2VyKCl7dmFyIHdvcmtlcjtpZihNb2R1bGVbIm1haW5TY3JpcHRVcmxPckJsb2IiXSl7dmFyIHB0aHJlYWRNYWluSnM9TW9kdWxlWyJtYWluU2NyaXB0VXJsT3JCbG9iIl07aWYodHlwZW9mIHB0aHJlYWRNYWluSnMhPSJzdHJpbmciKXtwdGhyZWFkTWFpbkpzPVVSTC5jcmVhdGVPYmplY3RVUkwocHRocmVhZE1haW5Kcyl9d29ya2VyPW5ldyBXb3JrZXIoRU5WSVJPTk1FTlRfSVNfTk9ERSYmdHlwZW9mIHB0aHJlYWRNYWluSnM9PSJzdHJpbmciJiZwdGhyZWFkTWFpbkpzLnN0YXJ0c1dpdGgoImRhdGE6Iik/bmV3IFVSTChwdGhyZWFkTWFpbkpzKTpwdGhyZWFkTWFpbkpzLHt0eXBlOiJtb2R1bGUiLHdvcmtlckRhdGE6ImVtLXB0aHJlYWQiLG5hbWU6ImVtLXB0aHJlYWQifSl9ZWxzZSB3b3JrZXI9bmV3IFdvcmtlcihuZXcgVVJMKCJza2FpbmV0X2tlcm5lbHNfbXQubWpzIixpbXBvcnQubWV0YS51cmwpLHt0eXBlOiJtb2R1bGUiLHdvcmtlckRhdGE6ImVtLXB0aHJlYWQiLG5hbWU6ImVtLXB0aHJlYWQifSk7UFRocmVhZC51bnVzZWRXb3JrZXJzLnB1c2god29ya2VyKX0sZ2V0TmV3V29ya2VyKCl7aWYoUFRocmVhZC51bnVzZWRXb3JrZXJzLmxlbmd0aD09MCl7UFRocmVhZC5hbGxvY2F0ZVVudXNlZFdvcmtlcigpO1BUaHJlYWQubG9hZFdhc21Nb2R1bGVUb1dvcmtlcihQVGhyZWFkLnVudXNlZFdvcmtlcnNbMF0pfXJldHVybiBQVGhyZWFkLnVudXNlZFdvcmtlcnMucG9wKCl9fTt2YXIgb25Qb3N0UnVucz1bXTt2YXIgYWRkT25Qb3N0UnVuPWNiPT5vblBvc3RSdW5zLnB1c2goY2IpO2Z1bmN0aW9uIGVzdGFibGlzaFN0YWNrU3BhY2UocHRocmVhZF9wdHIpe3ZhciBzdGFja0hpZ2g9KGdyb3dNZW1WaWV3cygpLEhFQVBVMzIpW3B0aHJlYWRfcHRyKzUyPj4+Mj4+PjBdO3ZhciBzdGFja1NpemU9KGdyb3dNZW1WaWV3cygpLEhFQVBVMzIpW3B0aHJlYWRfcHRyKzU2Pj4+Mj4+PjBdO3ZhciBzdGFja0xvdz1zdGFja0hpZ2gtc3RhY2tTaXplO19lbXNjcmlwdGVuX3N0YWNrX3NldF9saW1pdHMoc3RhY2tIaWdoLHN0YWNrTG93KTtzdGFja1Jlc3RvcmUoc3RhY2tIaWdoKX12YXIgd2FzbVRhYmxlTWlycm9yPVtdO3ZhciB3YXNtVGFibGU7dmFyIGdldFdhc21UYWJsZUVudHJ5PWZ1bmNQdHI9Pnt2YXIgZnVuYz13YXNtVGFibGVNaXJyb3JbZnVuY1B0cl07aWYoIWZ1bmMpe3dhc21UYWJsZU1pcnJvcltmdW5jUHRyXT1mdW5jPXdhc21UYWJsZS5nZXQoZnVuY1B0cil9cmV0dXJuIGZ1bmN9O3ZhciBpbnZva2VFbnRyeVBvaW50PShwdHIsYXJnKT0+e3J1bnRpbWVLZWVwYWxpdmVDb3VudGVyPTA7bm9FeGl0UnVudGltZT0wO3ZhciByZXN1bHQ9Z2V0V2FzbVRhYmxlRW50cnkocHRyKShhcmcpO2Z1bmN0aW9uIGZpbmlzaChyZXN1bHQpe2lmKGtlZXBSdW50aW1lQWxpdmUoKSl7RVhJVFNUQVRVUz1yZXN1bHQ7cmV0dXJufV9fZW1zY3JpcHRlbl90aHJlYWRfZXhpdChyZXN1bHQpfWZpbmlzaChyZXN1bHQpfTt2YXIgbm9FeGl0UnVudGltZT10cnVlO3ZhciByZWdpc3RlclRMU0luaXQ9dGxzSW5pdEZ1bmM9PlBUaHJlYWQudGxzSW5pdEZ1bmN0aW9ucy5wdXNoKHRsc0luaXRGdW5jKTtmdW5jdGlvbiBwdGhyZWFkQ3JlYXRlUHJveGllZChwdGhyZWFkX3B0cixhdHRyLHN0YXJ0Um91dGluZSxhcmcpe2lmKEVOVklST05NRU5UX0lTX1BUSFJFQUQpcmV0dXJuIHByb3h5VG9NYWluVGhyZWFkKDIsMCwxLHB0aHJlYWRfcHRyLGF0dHIsc3RhcnRSb3V0aW5lLGFyZyk7cmV0dXJuIF9fX3B0aHJlYWRfY3JlYXRlX2pzKHB0aHJlYWRfcHRyLGF0dHIsc3RhcnRSb3V0aW5lLGFyZyl9dmFyIF9lbXNjcmlwdGVuX2hhc190aHJlYWRpbmdfc3VwcG9ydD0oKT0+dHlwZW9mIFNoYXJlZEFycmF5QnVmZmVyIT0idW5kZWZpbmVkIjtmdW5jdGlvbiBfX19wdGhyZWFkX2NyZWF0ZV9qcyhwdGhyZWFkX3B0cixhdHRyLHN0YXJ0Um91dGluZSxhcmcpe3B0aHJlYWRfcHRyPj4+PTA7YXR0cj4+Pj0wO3N0YXJ0Um91dGluZT4+Pj0wO2FyZz4+Pj0wO2lmKCFfZW1zY3JpcHRlbl9oYXNfdGhyZWFkaW5nX3N1cHBvcnQoKSl7cmV0dXJuIDZ9dmFyIHRyYW5zZmVyTGlzdD1bXTt2YXIgZXJyb3I9MDtpZihFTlZJUk9OTUVOVF9JU19QVEhSRUFEJiYodHJhbnNmZXJMaXN0Lmxlbmd0aD09PTB8fGVycm9yKSl7cmV0dXJuIHB0aHJlYWRDcmVhdGVQcm94aWVkKHB0aHJlYWRfcHRyLGF0dHIsc3RhcnRSb3V0aW5lLGFyZyl9aWYoZXJyb3IpcmV0dXJuIGVycm9yO3ZhciB0aHJlYWRQYXJhbXM9e3N0YXJ0Um91dGluZSxwdGhyZWFkX3B0cixhcmcsdHJhbnNmZXJMaXN0fTtpZihFTlZJUk9OTUVOVF9JU19QVEhSRUFEKXt0aHJlYWRQYXJhbXMuY21kPSJzcGF3blRocmVhZCI7cG9zdE1lc3NhZ2UodGhyZWFkUGFyYW1zLHRyYW5zZmVyTGlzdCk7cmV0dXJuIDB9cmV0dXJuIHNwYXduVGhyZWFkKHRocmVhZFBhcmFtcyl9ZnVuY3Rpb24gX19lbXNjcmlwdGVuX2luaXRfbWFpbl90aHJlYWRfanModGIpe3RiPj4+PTA7X19lbXNjcmlwdGVuX3RocmVhZF9pbml0KHRiLCFFTlZJUk9OTUVOVF9JU19XT1JLRVIsMSwhRU5WSVJPTk1FTlRfSVNfV0VCLDY1NTM2LGZhbHNlKTtQVGhyZWFkLnRocmVhZEluaXRUTFMoKX12YXIgaGFuZGxlRXhjZXB0aW9uPWU9PntpZihlIGluc3RhbmNlb2YgRXhpdFN0YXR1c3x8ZT09InVud2luZCIpe3JldHVybiBFWElUU1RBVFVTfXF1aXRfKDEsZSl9O3ZhciBtYXliZUV4aXQ9KCk9PntpZigha2VlcFJ1bnRpbWVBbGl2ZSgpKXt0cnl7aWYoRU5WSVJPTk1FTlRfSVNfUFRIUkVBRCl7aWYoX3B0aHJlYWRfc2VsZigpKV9fZW1zY3JpcHRlbl90aHJlYWRfZXhpdChFWElUU1RBVFVTKTtyZXR1cm59X2V4aXQoRVhJVFNUQVRVUyl9Y2F0Y2goZSl7aGFuZGxlRXhjZXB0aW9uKGUpfX19O3ZhciBjYWxsVXNlckNhbGxiYWNrPWZ1bmM9PntpZihBQk9SVCl7cmV0dXJufXRyeXtmdW5jKCk7bWF5YmVFeGl0KCl9Y2F0Y2goZSl7aGFuZGxlRXhjZXB0aW9uKGUpfX07ZnVuY3Rpb24gX19lbXNjcmlwdGVuX3RocmVhZF9tYWlsYm94X2F3YWl0KHB0aHJlYWRfcHRyKXtwdGhyZWFkX3B0cj4+Pj0wO2lmKHR5cGVvZiBBdG9taWNzLndhaXRBc3luYz09PSJmdW5jdGlvbiIpe3ZhciB3YWl0PUF0b21pY3Mud2FpdEFzeW5jKChncm93TWVtVmlld3MoKSxIRUFQMzIpLHB0aHJlYWRfcHRyPj4+MixwdGhyZWFkX3B0cik7d2FpdC52YWx1ZS50aGVuKGNoZWNrTWFpbGJveCk7dmFyIHdhaXRpbmdBc3luYz1wdGhyZWFkX3B0cisxMjg7QXRvbWljcy5zdG9yZSgoZ3Jvd01lbVZpZXdzKCksSEVBUDMyKSx3YWl0aW5nQXN5bmM+Pj4yLDEpfX12YXIgY2hlY2tNYWlsYm94PSgpPT5jYWxsVXNlckNhbGxiYWNrKCgpPT57dmFyIHB0aHJlYWRfcHRyPV9wdGhyZWFkX3NlbGYoKTtpZihwdGhyZWFkX3B0cil7X19lbXNjcmlwdGVuX3RocmVhZF9tYWlsYm94X2F3YWl0KHB0aHJlYWRfcHRyKTtfX2Vtc2NyaXB0ZW5fY2hlY2tfbWFpbGJveCgpfX0pO2Z1bmN0aW9uIF9fZW1zY3JpcHRlbl9ub3RpZnlfbWFpbGJveF9wb3N0bWVzc2FnZSh0YXJnZXRUaHJlYWQsY3VyclRocmVhZElkKXt0YXJnZXRUaHJlYWQ+Pj49MDtjdXJyVGhyZWFkSWQ+Pj49MDtpZih0YXJnZXRUaHJlYWQ9PWN1cnJUaHJlYWRJZCl7c2V0VGltZW91dChjaGVja01haWxib3gpfWVsc2UgaWYoRU5WSVJPTk1FTlRfSVNfUFRIUkVBRCl7cG9zdE1lc3NhZ2Uoe3RhcmdldFRocmVhZCxjbWQ6ImNoZWNrTWFpbGJveCJ9KX1lbHNle3ZhciB3b3JrZXI9UFRocmVhZC5wdGhyZWFkc1t0YXJnZXRUaHJlYWRdO2lmKCF3b3JrZXIpe3JldHVybn13b3JrZXIucG9zdE1lc3NhZ2Uoe2NtZDoiY2hlY2tNYWlsYm94In0pfX12YXIgcHJveGllZEpTQ2FsbEFyZ3M9W107ZnVuY3Rpb24gX19lbXNjcmlwdGVuX3JlY2VpdmVfb25fbWFpbl90aHJlYWRfanMoZnVuY0luZGV4LGVtQXNtQWRkcixjYWxsaW5nVGhyZWFkLG51bUNhbGxBcmdzLGFyZ3Mpe2VtQXNtQWRkcj4+Pj0wO2NhbGxpbmdUaHJlYWQ+Pj49MDthcmdzPj4+PTA7bnVtQ2FsbEFyZ3MvPTI7cHJveGllZEpTQ2FsbEFyZ3MubGVuZ3RoPW51bUNhbGxBcmdzO3ZhciBiPWFyZ3M+Pj4zO2Zvcih2YXIgaT0wO2k8bnVtQ2FsbEFyZ3M7aSsrKXtpZigoZ3Jvd01lbVZpZXdzKCksSEVBUDY0KVtiKzIqaT4+PjBdKXtwcm94aWVkSlNDYWxsQXJnc1tpXT0oZ3Jvd01lbVZpZXdzKCksSEVBUDY0KVtiKzIqaSsxPj4+MF19ZWxzZXtwcm94aWVkSlNDYWxsQXJnc1tpXT0oZ3Jvd01lbVZpZXdzKCksSEVBUEY2NClbYisyKmkrMT4+PjBdfX12YXIgZnVuYz1wcm94aWVkRnVuY3Rpb25UYWJsZVtmdW5jSW5kZXhdO1BUaHJlYWQuY3VycmVudFByb3hpZWRPcGVyYXRpb25DYWxsZXJUaHJlYWQ9Y2FsbGluZ1RocmVhZDt2YXIgcnRuPWZ1bmMoLi4ucHJveGllZEpTQ2FsbEFyZ3MpO1BUaHJlYWQuY3VycmVudFByb3hpZWRPcGVyYXRpb25DYWxsZXJUaHJlYWQ9MDtyZXR1cm4gcnRufWZ1bmN0aW9uIF9fZW1zY3JpcHRlbl90aHJlYWRfY2xlYW51cCh0aHJlYWQpe3RocmVhZD4+Pj0wO2lmKCFFTlZJUk9OTUVOVF9JU19QVEhSRUFEKWNsZWFudXBUaHJlYWQodGhyZWFkKTtlbHNlIHBvc3RNZXNzYWdlKHtjbWQ6ImNsZWFudXBUaHJlYWQiLHRocmVhZH0pfWZ1bmN0aW9uIF9fZW1zY3JpcHRlbl90aHJlYWRfc2V0X3N0cm9uZ3JlZih0aHJlYWQpe3RocmVhZD4+Pj0wO2lmKEVOVklST05NRU5UX0lTX05PREUpe1BUaHJlYWQucHRocmVhZHNbdGhyZWFkXS5yZWYoKX19dmFyIF9lbXNjcmlwdGVuX2dldF9ub3c9KCk9PnBlcmZvcm1hbmNlLnRpbWVPcmlnaW4rcGVyZm9ybWFuY2Uubm93KCk7dmFyIF9lbXNjcmlwdGVuX2NoZWNrX2Jsb2NraW5nX2FsbG93ZWQ9KCk9Pnt9O3ZhciBydW50aW1lS2VlcGFsaXZlUHVzaD0oKT0+e3J1bnRpbWVLZWVwYWxpdmVDb3VudGVyKz0xfTt2YXIgX2Vtc2NyaXB0ZW5fZXhpdF93aXRoX2xpdmVfcnVudGltZT0oKT0+e3J1bnRpbWVLZWVwYWxpdmVQdXNoKCk7dGhyb3cidW53aW5kIn07dmFyIGdldEhlYXBNYXg9KCk9PjQyOTQ5MDE3NjA7dmFyIGFsaWduTWVtb3J5PShzaXplLGFsaWdubWVudCk9Pk1hdGguY2VpbChzaXplL2FsaWdubWVudCkqYWxpZ25tZW50O3ZhciBncm93TWVtb3J5PXNpemU9Pnt2YXIgb2xkSGVhcFNpemU9d2FzbU1lbW9yeS5idWZmZXIuYnl0ZUxlbmd0aDt2YXIgcGFnZXM9KHNpemUtb2xkSGVhcFNpemUrNjU1MzUpLzY1NTM2fDA7dHJ5e3dhc21NZW1vcnkuZ3JvdyhwYWdlcyk7dXBkYXRlTWVtb3J5Vmlld3MoKTtyZXR1cm4gMX1jYXRjaChlKXt9fTtmdW5jdGlvbiBfZW1zY3JpcHRlbl9yZXNpemVfaGVhcChyZXF1ZXN0ZWRTaXplKXtyZXF1ZXN0ZWRTaXplPj4+PTA7dmFyIG9sZFNpemU9KGdyb3dNZW1WaWV3cygpLEhFQVBVOCkubGVuZ3RoO2lmKHJlcXVlc3RlZFNpemU8PW9sZFNpemUpe3JldHVybiBmYWxzZX12YXIgbWF4SGVhcFNpemU9Z2V0SGVhcE1heCgpO2lmKHJlcXVlc3RlZFNpemU+bWF4SGVhcFNpemUpe3JldHVybiBmYWxzZX1mb3IodmFyIGN1dERvd249MTtjdXREb3duPD00O2N1dERvd24qPTIpe3ZhciBvdmVyR3Jvd25IZWFwU2l6ZT1vbGRTaXplKigxKy4yL2N1dERvd24pO292ZXJHcm93bkhlYXBTaXplPU1hdGgubWluKG92ZXJHcm93bkhlYXBTaXplLHJlcXVlc3RlZFNpemUrMTAwNjYzMjk2KTt2YXIgbmV3U2l6ZT1NYXRoLm1pbihtYXhIZWFwU2l6ZSxhbGlnbk1lbW9yeShNYXRoLm1heChyZXF1ZXN0ZWRTaXplLG92ZXJHcm93bkhlYXBTaXplKSw2NTUzNikpO3ZhciByZXBsYWNlbWVudD1ncm93TWVtb3J5KG5ld1NpemUpO2lmKHJlcGxhY2VtZW50KXtyZXR1cm4gdHJ1ZX19cmV0dXJuIGZhbHNlfXZhciBwcmludENoYXJCdWZmZXJzPVtudWxsLFtdLFtdXTt2YXIgVVRGOERlY29kZXI9dHlwZW9mIFRleHREZWNvZGVyIT0idW5kZWZpbmVkIj9uZXcgVGV4dERlY29kZXI6dW5kZWZpbmVkO3ZhciBmaW5kU3RyaW5nRW5kPShoZWFwT3JBcnJheSxpZHgsbWF4Qnl0ZXNUb1JlYWQsaWdub3JlTnVsKT0+e3ZhciBtYXhJZHg9aWR4K21heEJ5dGVzVG9SZWFkO2lmKGlnbm9yZU51bClyZXR1cm4gbWF4SWR4O3doaWxlKGhlYXBPckFycmF5W2lkeF0mJiEoaWR4Pj1tYXhJZHgpKSsraWR4O3JldHVybiBpZHh9O3ZhciBVVEY4QXJyYXlUb1N0cmluZz0oaGVhcE9yQXJyYXksaWR4PTAsbWF4Qnl0ZXNUb1JlYWQsaWdub3JlTnVsKT0+e2lkeD4+Pj0wO3ZhciBlbmRQdHI9ZmluZFN0cmluZ0VuZChoZWFwT3JBcnJheSxpZHgsbWF4Qnl0ZXNUb1JlYWQsaWdub3JlTnVsKTtpZihlbmRQdHItaWR4PjE2JiZoZWFwT3JBcnJheS5idWZmZXImJlVURjhEZWNvZGVyKXtyZXR1cm4gVVRGOERlY29kZXIuZGVjb2RlKGhlYXBPckFycmF5LmJ1ZmZlciBpbnN0YW5jZW9mIEFycmF5QnVmZmVyP2hlYXBPckFycmF5LnN1YmFycmF5KGlkeCxlbmRQdHIpOmhlYXBPckFycmF5LnNsaWNlKGlkeCxlbmRQdHIpKX12YXIgc3RyPSIiO3doaWxlKGlkeDxlbmRQdHIpe3ZhciB1MD1oZWFwT3JBcnJheVtpZHgrK107aWYoISh1MCYxMjgpKXtzdHIrPVN0cmluZy5mcm9tQ2hhckNvZGUodTApO2NvbnRpbnVlfXZhciB1MT1oZWFwT3JBcnJheVtpZHgrK10mNjM7aWYoKHUwJjIyNCk9PTE5Mil7c3RyKz1TdHJpbmcuZnJvbUNoYXJDb2RlKCh1MCYzMSk8PDZ8dTEpO2NvbnRpbnVlfXZhciB1Mj1oZWFwT3JBcnJheVtpZHgrK10mNjM7aWYoKHUwJjI0MCk9PTIyNCl7dTA9KHUwJjE1KTw8MTJ8dTE8PDZ8dTJ9ZWxzZXt1MD0odTAmNyk8PDE4fHUxPDwxMnx1Mjw8NnxoZWFwT3JBcnJheVtpZHgrK10mNjN9aWYodTA8NjU1MzYpe3N0cis9U3RyaW5nLmZyb21DaGFyQ29kZSh1MCl9ZWxzZXt2YXIgY2g9dTAtNjU1MzY7c3RyKz1TdHJpbmcuZnJvbUNoYXJDb2RlKDU1Mjk2fGNoPj4xMCw1NjMyMHxjaCYxMDIzKX19cmV0dXJuIHN0cn07dmFyIHByaW50Q2hhcj0oc3RyZWFtLGN1cnIpPT57dmFyIGJ1ZmZlcj1wcmludENoYXJCdWZmZXJzW3N0cmVhbV07aWYoY3Vycj09PTB8fGN1cnI9PT0xMCl7KHN0cmVhbT09PTE/b3V0OmVycikoVVRGOEFycmF5VG9TdHJpbmcoYnVmZmVyKSk7YnVmZmVyLmxlbmd0aD0wfWVsc2V7YnVmZmVyLnB1c2goY3Vycil9fTtmdW5jdGlvbiBfZmRfd3JpdGUoZmQsaW92LGlvdmNudCxwbnVtKXtpZihFTlZJUk9OTUVOVF9JU19QVEhSRUFEKXJldHVybiBwcm94eVRvTWFpblRocmVhZCgzLDAsMSxmZCxpb3YsaW92Y250LHBudW0pO2lvdj4+Pj0wO2lvdmNudD4+Pj0wO3BudW0+Pj49MDt2YXIgbnVtPTA7Zm9yKHZhciBpPTA7aTxpb3ZjbnQ7aSsrKXt2YXIgcHRyPShncm93TWVtVmlld3MoKSxIRUFQVTMyKVtpb3Y+Pj4yPj4+MF07dmFyIGxlbj0oZ3Jvd01lbVZpZXdzKCksSEVBUFUzMilbaW92KzQ+Pj4yPj4+MF07aW92Kz04O2Zvcih2YXIgaj0wO2o8bGVuO2orKyl7cHJpbnRDaGFyKGZkLChncm93TWVtVmlld3MoKSxIRUFQVTgpW3B0citqPj4+MF0pfW51bSs9bGVufShncm93TWVtVmlld3MoKSxIRUFQVTMyKVtwbnVtPj4+Mj4+PjBdPW51bTtyZXR1cm4gMH1QVGhyZWFkLmluaXQoKTt7aW5pdE1lbW9yeSgpO2lmKE1vZHVsZVsibm9FeGl0UnVudGltZSJdKW5vRXhpdFJ1bnRpbWU9TW9kdWxlWyJub0V4aXRSdW50aW1lIl07aWYoTW9kdWxlWyJwcmludCJdKW91dD1Nb2R1bGVbInByaW50Il07aWYoTW9kdWxlWyJwcmludEVyciJdKWVycj1Nb2R1bGVbInByaW50RXJyIl07aWYoTW9kdWxlWyJ3YXNtQmluYXJ5Il0pd2FzbUJpbmFyeT1Nb2R1bGVbIndhc21CaW5hcnkiXTtpZihNb2R1bGVbImFyZ3VtZW50cyJdKWFyZ3VtZW50c189TW9kdWxlWyJhcmd1bWVudHMiXTtpZihNb2R1bGVbInRoaXNQcm9ncmFtIl0pdGhpc1Byb2dyYW09TW9kdWxlWyJ0aGlzUHJvZ3JhbSJdO2lmKE1vZHVsZVsicHJlSW5pdCJdKXtpZih0eXBlb2YgTW9kdWxlWyJwcmVJbml0Il09PSJmdW5jdGlvbiIpTW9kdWxlWyJwcmVJbml0Il09W01vZHVsZVsicHJlSW5pdCJdXTt3aGlsZShNb2R1bGVbInByZUluaXQiXS5sZW5ndGg+MCl7TW9kdWxlWyJwcmVJbml0Il0uc2hpZnQoKSgpfX19TW9kdWxlWyJ3YXNtTWVtb3J5Il09d2FzbU1lbW9yeTt2YXIgcHJveGllZEZ1bmN0aW9uVGFibGU9W19wcm9jX2V4aXQsZXhpdE9uTWFpblRocmVhZCxwdGhyZWFkQ3JlYXRlUHJveGllZCxfZmRfd3JpdGVdO3ZhciBfc2thaW5ldF9mcDMyX21hdG11bCxfc2thaW5ldF9mcDMyX21hdG11bF93dCxfc2thaW5ldF9iZjE2X21hdG11bCxfc2thaW5ldF9mcDE2X21hdG11bCxfc2thaW5ldF9xNF8wX21hdG11bCxfc2thaW5ldF9xNF8wX21hdG11bF9ybSxfc2thaW5ldF9xNV8wX21hdG11bCxfbWFsbG9jLF9mcmVlLF9za2FpbmV0X3E1XzBfbWF0bXVsX3JtLF9za2FpbmV0X3E1XzFfbWF0bXVsLF9za2FpbmV0X3E1XzFfbWF0bXVsX3JtLF9za2FpbmV0X3E4XzBfbWF0bXVsLF9za2FpbmV0X3E4XzBfbWF0bXVsX3JtLF9za2FpbmV0X3E0a19tYXRtdWwsX3NrYWluZXRfcTRrX21hdG11bF9ybSxfc2thaW5ldF9xNWtfbWF0bXVsLF9za2FpbmV0X3E1a19tYXRtdWxfcm0sX3NrYWluZXRfcTZrX21hdG11bCxfc2thaW5ldF9xNmtfbWF0bXVsX3JtLF9za2FpbmV0X3Rlcm5hcnlfZjMyX2dlbXYsX3NrYWluZXRfdGVybmFyeV9sbWhlYWRfc3RhZ2UxLF9za2FpbmV0X2JpdG5ldF9nZW12X3RxMl8wLF9za2FpbmV0X2YzMl9iaW5hcnksX3NrYWluZXRfZjMyX3VuYXJ5LF9za2FpbmV0X2YzMl9yZWR1Y2VfYWxsLF9za2FpbmV0X2YzMl9yZWR1Y2VfYXhpcyxfc2thaW5ldF9mMzJfc29mdG1heF9yb3dzLF9za2FpbmV0X2YzMl9sYXllcl9ub3JtLF9za2FpbmV0X2YzMl9ybXNfbm9ybSxfc2thaW5ldF9mMzJfc2RwYSxfc2thaW5ldF9mMzJfY29udixfX2Vtc2NyaXB0ZW5fdGxzX2luaXQsX3B0aHJlYWRfc2VsZixfX2Vtc2NyaXB0ZW5fdGhyZWFkX2luaXQsX19lbXNjcmlwdGVuX3RocmVhZF9jcmFzaGVkLF9fZW1zY3JpcHRlbl9ydW5fanNfb25fbWFpbl90aHJlYWQsX19lbXNjcmlwdGVuX3RocmVhZF9mcmVlX2RhdGEsX19lbXNjcmlwdGVuX3RocmVhZF9leGl0LF9fZW1zY3JpcHRlbl9jaGVja19tYWlsYm94LF9lbXNjcmlwdGVuX3N0YWNrX3NldF9saW1pdHMsX19lbXNjcmlwdGVuX3N0YWNrX3Jlc3RvcmUsX19lbXNjcmlwdGVuX3N0YWNrX2FsbG9jLF9lbXNjcmlwdGVuX3N0YWNrX2dldF9jdXJyZW50O2Z1bmN0aW9uIGFzc2lnbldhc21FeHBvcnRzKHdhc21FeHBvcnRzKXtNb2R1bGVbIl9za2FpbmV0X2ZwMzJfbWF0bXVsIl09X3NrYWluZXRfZnAzMl9tYXRtdWw9d2FzbUV4cG9ydHNbInAiXTtNb2R1bGVbIl9za2FpbmV0X2ZwMzJfbWF0bXVsX3d0Il09X3NrYWluZXRfZnAzMl9tYXRtdWxfd3Q9d2FzbUV4cG9ydHNbInIiXTtNb2R1bGVbIl9za2FpbmV0X2JmMTZfbWF0bXVsIl09X3NrYWluZXRfYmYxNl9tYXRtdWw9d2FzbUV4cG9ydHNbInMiXTtNb2R1bGVbIl9za2FpbmV0X2ZwMTZfbWF0bXVsIl09X3NrYWluZXRfZnAxNl9tYXRtdWw9d2FzbUV4cG9ydHNbInQiXTtNb2R1bGVbIl9za2FpbmV0X3E0XzBfbWF0bXVsIl09X3NrYWluZXRfcTRfMF9tYXRtdWw9d2FzbUV4cG9ydHNbInUiXTtNb2R1bGVbIl9za2FpbmV0X3E0XzBfbWF0bXVsX3JtIl09X3NrYWluZXRfcTRfMF9tYXRtdWxfcm09d2FzbUV4cG9ydHNbInYiXTtNb2R1bGVbIl9za2FpbmV0X3E1XzBfbWF0bXVsIl09X3NrYWluZXRfcTVfMF9tYXRtdWw9d2FzbUV4cG9ydHNbInciXTtNb2R1bGVbIl9tYWxsb2MiXT1fbWFsbG9jPXdhc21FeHBvcnRzWyJ4Il07TW9kdWxlWyJfZnJlZSJdPV9mcmVlPXdhc21FeHBvcnRzWyJ5Il07TW9kdWxlWyJfc2thaW5ldF9xNV8wX21hdG11bF9ybSJdPV9za2FpbmV0X3E1XzBfbWF0bXVsX3JtPXdhc21FeHBvcnRzWyJ6Il07TW9kdWxlWyJfc2thaW5ldF9xNV8xX21hdG11bCJdPV9za2FpbmV0X3E1XzFfbWF0bXVsPXdhc21FeHBvcnRzWyJBIl07TW9kdWxlWyJfc2thaW5ldF9xNV8xX21hdG11bF9ybSJdPV9za2FpbmV0X3E1XzFfbWF0bXVsX3JtPXdhc21FeHBvcnRzWyJCIl07TW9kdWxlWyJfc2thaW5ldF9xOF8wX21hdG11bCJdPV9za2FpbmV0X3E4XzBfbWF0bXVsPXdhc21FeHBvcnRzWyJDIl07TW9kdWxlWyJfc2thaW5ldF9xOF8wX21hdG11bF9ybSJdPV9za2FpbmV0X3E4XzBfbWF0bXVsX3JtPXdhc21FeHBvcnRzWyJEIl07TW9kdWxlWyJfc2thaW5ldF9xNGtfbWF0bXVsIl09X3NrYWluZXRfcTRrX21hdG11bD13YXNtRXhwb3J0c1siRSJdO01vZHVsZVsiX3NrYWluZXRfcTRrX21hdG11bF9ybSJdPV9za2FpbmV0X3E0a19tYXRtdWxfcm09d2FzbUV4cG9ydHNbIkYiXTtNb2R1bGVbIl9za2FpbmV0X3E1a19tYXRtdWwiXT1fc2thaW5ldF9xNWtfbWF0bXVsPXdhc21FeHBvcnRzWyJHIl07TW9kdWxlWyJfc2thaW5ldF9xNWtfbWF0bXVsX3JtIl09X3NrYWluZXRfcTVrX21hdG11bF9ybT13YXNtRXhwb3J0c1siSCJdO01vZHVsZVsiX3NrYWluZXRfcTZrX21hdG11bCJdPV9za2FpbmV0X3E2a19tYXRtdWw9d2FzbUV4cG9ydHNbIkkiXTtNb2R1bGVbIl9za2FpbmV0X3E2a19tYXRtdWxfcm0iXT1fc2thaW5ldF9xNmtfbWF0bXVsX3JtPXdhc21FeHBvcnRzWyJKIl07TW9kdWxlWyJfc2thaW5ldF90ZXJuYXJ5X2YzMl9nZW12Il09X3NrYWluZXRfdGVybmFyeV9mMzJfZ2Vtdj13YXNtRXhwb3J0c1siSyJdO01vZHVsZVsiX3NrYWluZXRfdGVybmFyeV9sbWhlYWRfc3RhZ2UxIl09X3NrYWluZXRfdGVybmFyeV9sbWhlYWRfc3RhZ2UxPXdhc21FeHBvcnRzWyJMIl07TW9kdWxlWyJfc2thaW5ldF9iaXRuZXRfZ2Vtdl90cTJfMCJdPV9za2FpbmV0X2JpdG5ldF9nZW12X3RxMl8wPXdhc21FeHBvcnRzWyJNIl07TW9kdWxlWyJfc2thaW5ldF9mMzJfYmluYXJ5Il09X3NrYWluZXRfZjMyX2JpbmFyeT13YXNtRXhwb3J0c1siTiJdO01vZHVsZVsiX3NrYWluZXRfZjMyX3VuYXJ5Il09X3NrYWluZXRfZjMyX3VuYXJ5PXdhc21FeHBvcnRzWyJPIl07TW9kdWxlWyJfc2thaW5ldF9mMzJfcmVkdWNlX2FsbCJdPV9za2FpbmV0X2YzMl9yZWR1Y2VfYWxsPXdhc21FeHBvcnRzWyJQIl07TW9kdWxlWyJfc2thaW5ldF9mMzJfcmVkdWNlX2F4aXMiXT1fc2thaW5ldF9mMzJfcmVkdWNlX2F4aXM9d2FzbUV4cG9ydHNbIlEiXTtNb2R1bGVbIl9za2FpbmV0X2YzMl9zb2Z0bWF4X3Jvd3MiXT1fc2thaW5ldF9mMzJfc29mdG1heF9yb3dzPXdhc21FeHBvcnRzWyJSIl07TW9kdWxlWyJfc2thaW5ldF9mMzJfbGF5ZXJfbm9ybSJdPV9za2FpbmV0X2YzMl9sYXllcl9ub3JtPXdhc21FeHBvcnRzWyJTIl07TW9kdWxlWyJfc2thaW5ldF9mMzJfcm1zX25vcm0iXT1fc2thaW5ldF9mMzJfcm1zX25vcm09d2FzbUV4cG9ydHNbIlQiXTtNb2R1bGVbIl9za2FpbmV0X2YzMl9zZHBhIl09X3NrYWluZXRfZjMyX3NkcGE9d2FzbUV4cG9ydHNbIlUiXTtNb2R1bGVbIl9za2FpbmV0X2YzMl9jb252Il09X3NrYWluZXRfZjMyX2NvbnY9d2FzbUV4cG9ydHNbIlYiXTtfX2Vtc2NyaXB0ZW5fdGxzX2luaXQ9d2FzbUV4cG9ydHNbIlciXTtfcHRocmVhZF9zZWxmPXdhc21FeHBvcnRzWyJYIl07X19lbXNjcmlwdGVuX3RocmVhZF9pbml0PXdhc21FeHBvcnRzWyJZIl07X19lbXNjcmlwdGVuX3RocmVhZF9jcmFzaGVkPXdhc21FeHBvcnRzWyJaIl07X19lbXNjcmlwdGVuX3J1bl9qc19vbl9tYWluX3RocmVhZD13YXNtRXhwb3J0c1siXyJdO19fZW1zY3JpcHRlbl90aHJlYWRfZnJlZV9kYXRhPXdhc21FeHBvcnRzWyIkIl07X19lbXNjcmlwdGVuX3RocmVhZF9leGl0PXdhc21FeHBvcnRzWyJhYSJdO19fZW1zY3JpcHRlbl9jaGVja19tYWlsYm94PXdhc21FeHBvcnRzWyJiYSJdO19lbXNjcmlwdGVuX3N0YWNrX3NldF9saW1pdHM9d2FzbUV4cG9ydHNbImNhIl07X19lbXNjcmlwdGVuX3N0YWNrX3Jlc3RvcmU9d2FzbUV4cG9ydHNbImRhIl07X19lbXNjcmlwdGVuX3N0YWNrX2FsbG9jPXdhc21FeHBvcnRzWyJlYSJdO19lbXNjcmlwdGVuX3N0YWNrX2dldF9jdXJyZW50PXdhc21FeHBvcnRzWyJmYSJdfXZhciB3YXNtSW1wb3J0cztmdW5jdGlvbiBhc3NpZ25XYXNtSW1wb3J0cygpe3dhc21JbXBvcnRzPXtnOl9fX3B0aHJlYWRfY3JlYXRlX2pzLGs6X19lbXNjcmlwdGVuX2luaXRfbWFpbl90aHJlYWRfanMsbjpfX2Vtc2NyaXB0ZW5fbm90aWZ5X21haWxib3hfcG9zdG1lc3NhZ2UsbDpfX2Vtc2NyaXB0ZW5fcmVjZWl2ZV9vbl9tYWluX3RocmVhZF9qcyxjOl9fZW1zY3JpcHRlbl90aHJlYWRfY2xlYW51cCxqOl9fZW1zY3JpcHRlbl90aHJlYWRfbWFpbGJveF9hd2FpdCxpOl9fZW1zY3JpcHRlbl90aHJlYWRfc2V0X3N0cm9uZ3JlZixlOl9lbXNjcmlwdGVuX2NoZWNrX2Jsb2NraW5nX2FsbG93ZWQsaDpfZW1zY3JpcHRlbl9leGl0X3dpdGhfbGl2ZV9ydW50aW1lLGI6X2Vtc2NyaXB0ZW5fZ2V0X25vdyxtOl9lbXNjcmlwdGVuX3Jlc2l6ZV9oZWFwLGY6X2V4aXQsZDpfZmRfd3JpdGUsYTp3YXNtTWVtb3J5fX1mdW5jdGlvbiBhcHBseVNpZ25hdHVyZUNvbnZlcnNpb25zKHdhc21FeHBvcnRzKXt3YXNtRXhwb3J0cz1PYmplY3QuYXNzaWduKHt9LHdhc21FeHBvcnRzKTt2YXIgbWFrZVdyYXBwZXJfcHA9Zj0+YTA9PmYoYTApPj4+MDt2YXIgbWFrZVdyYXBwZXJfcD1mPT4oKT0+ZigpPj4+MDt3YXNtRXhwb3J0c1sieCJdPW1ha2VXcmFwcGVyX3BwKHdhc21FeHBvcnRzWyJ4Il0pO3dhc21FeHBvcnRzWyJYIl09bWFrZVdyYXBwZXJfcCh3YXNtRXhwb3J0c1siWCJdKTt3YXNtRXhwb3J0c1siZWEiXT1tYWtlV3JhcHBlcl9wcCh3YXNtRXhwb3J0c1siZWEiXSk7d2FzbUV4cG9ydHNbImZhIl09bWFrZVdyYXBwZXJfcCh3YXNtRXhwb3J0c1siZmEiXSk7cmV0dXJuIHdhc21FeHBvcnRzfWZ1bmN0aW9uIHJ1bigpe2lmKHJ1bkRlcGVuZGVuY2llcz4wKXtkZXBlbmRlbmNpZXNGdWxmaWxsZWQ9cnVuO3JldHVybn1pZihFTlZJUk9OTUVOVF9JU19QVEhSRUFEKXtyZWFkeVByb21pc2VSZXNvbHZlPy4oTW9kdWxlKTtpbml0UnVudGltZSgpO3JldHVybn1wcmVSdW4oKTtpZihydW5EZXBlbmRlbmNpZXM+MCl7ZGVwZW5kZW5jaWVzRnVsZmlsbGVkPXJ1bjtyZXR1cm59ZnVuY3Rpb24gZG9SdW4oKXtNb2R1bGVbImNhbGxlZFJ1biJdPXRydWU7aWYoQUJPUlQpcmV0dXJuO2luaXRSdW50aW1lKCk7cmVhZHlQcm9taXNlUmVzb2x2ZT8uKE1vZHVsZSk7TW9kdWxlWyJvblJ1bnRpbWVJbml0aWFsaXplZCJdPy4oKTtwb3N0UnVuKCl9aWYoTW9kdWxlWyJzZXRTdGF0dXMiXSl7TW9kdWxlWyJzZXRTdGF0dXMiXSgiUnVubmluZy4uLiIpO3NldFRpbWVvdXQoKCk9PntzZXRUaW1lb3V0KCgpPT5Nb2R1bGVbInNldFN0YXR1cyJdKCIiKSwxKTtkb1J1bigpfSwxKX1lbHNle2RvUnVuKCl9fXZhciB3YXNtRXhwb3J0cztpZighRU5WSVJPTk1FTlRfSVNfUFRIUkVBRCl7d2FzbUV4cG9ydHM9YXdhaXQgKGNyZWF0ZVdhc20oKSk7cnVuKCl9aWYocnVudGltZUluaXRpYWxpemVkKXttb2R1bGVSdG49TW9kdWxlfWVsc2V7bW9kdWxlUnRuPW5ldyBQcm9taXNlKChyZXNvbHZlLHJlamVjdCk9PntyZWFkeVByb21pc2VSZXNvbHZlPXJlc29sdmU7cmVhZHlQcm9taXNlUmVqZWN0PXJlamVjdH0pfQo7cmV0dXJuIG1vZHVsZVJ0bn1leHBvcnQgZGVmYXVsdCBjcmVhdGVTa2FpbmV0S2VybmVsczt2YXIgaXNQdGhyZWFkPWdsb2JhbFRoaXMuc2VsZj8ubmFtZT8uc3RhcnRzV2l0aCgiZW0tcHRocmVhZCIpO3ZhciBpc05vZGU9dHlwZW9mIHByb2Nlc3M9PSJvYmplY3QiJiZwcm9jZXNzLnZlcnNpb25zPy5ub2RlJiZwcm9jZXNzLnR5cGUhPSJyZW5kZXJlciI7aWYoaXNOb2RlKWlzUHRocmVhZD0oYXdhaXQgaW1wb3J0KCJ3b3JrZXJfdGhyZWFkcyIpKS53b3JrZXJEYXRhPT09ImVtLXB0aHJlYWQiO2lzUHRocmVhZCYmY3JlYXRlU2thaW5ldEtlcm5lbHMoKTsK" + const val MT_WASM: String = "AGFzbQEAAAAB7wEdYAF/AGADf39/AGAAAGAIf39/f39/f38AYAJ/fwBgAX8Bf2ACf38Bf2ABfQF9YAABf2AJf39/f39/f39/AGAMf39/f39/f39/f39/AGAGf39/f39/AGAEf39/fwF/YAV/f39/fwF8YAN/f38Bf2ABfAF8YAV/f39/fwBgBH9/f38AYAABfGADf398AX9gB39/f39/f38AYAJ/fwF9YAN/f30BfWAUf39/f39/f39/f31/f39/f39/f38AYAZ/f39/f30AYAd/f39/f399AGAJf399f39/f39/AGALf39/f39/f39/f38AYAN/fn8BfgJaDgFhAWIAEgFhAWMAAAFhAWQADAFhAWUAAgFhAWYAAAFhAWcADAFhAWgAAgFhAWkAAAFhAWoAAAFhAWsAAAFhAWwADQFhAW0ABQFhAW4ABAFhAWECA4AIgIAEA5UBkwEHAAUBBAcABQEFAAEJEw4UCQkEBQAABgAABwIIAAUCAgQGAAAABgYGAAQABQcPDxAVFgQCAAIEAAIEBAIADQQAAgAAAQAACgEIBQIBCwEXGAEZARABCxEBEQEDARoBGwEDAQMBAwEDAQMBAwEBAwEDAQMBAwEDAQoDAQMBCAUACwQcAw4FAgEIAAMAAAABAgAKAgIEBQFwAS8vBjUKfwFB8NcEC38BQQALfwFBAAt/AUEAC38BQQALfwFBAAt/AUEAC38BQQALfwFBAAt/AUEACwfSAS4BbwCfAQFwAIQBAXEBAAFyABkBcwBTAXQAnQEBdQCWAQF2AI8BAXcAhwEBeAAUAXkAEwF6AIUBAUEAggEBQgCAAQFDAH4BRAB8AUUAegFGAHcBRwB1AUgAcwFJAHEBSgBvAUsAbQFMAGsBTQBpAU4AZwFPAGUBUABjAVEAYgFSAGABUwBeAVQAXAFVAFsBVgBZAVcAVQFYAJQBAVkAjAEBWgCbAQFfAEoBJACZAQJhYQCYAQJiYQCSAQJjYQCNAQJkYQCLAQJlYQCKAQJmYQCJAQgCngEJQAEAQQELLnlUUJoBkwGIAYYBgwGBAX99e3h2dHJwbmxqaGZkYV9dWlhXVlJRF08vTk1MnAFLSZcBlQGRAZABjgEMAQQK45wEkwHrAQMCfAF/AX4CfQJAIAC8QRR2Qf8PcSIDQasISQ0AQwAAAAAgAEMAAID/Ww0BGiADQfgPTwRAIAAgAJIPCyAAQxdysUJeBEAjAEEQayIDQwAAAHA4AgwgAyoCDEMAAABwlA8LIABDtPHPwl1FDQAjAEEQayIDQwAAABA4AgwgAyoCDEMAAAAQlA8LQbA7KwMAQag7KwMAIAC7oiIBIAFBoDsrAwAiAaAiAiABoaEiAaJBuDsrAwCgIAEgAaKiQcA7KwMAIAGiRAAAAAAAAPA/oKAgAr0iBEIvhiAEp0EfcUEDdCkDgDl8v6K2CwvzAQEHfyAAKAIIIQQCQAJAIAAoAgAiAkEPcUUEQCAAQQRqIgFBAP5BAgAhAAwBCyMDIgMoAhggACgCBCIFQf////8DcUcNAQJAIAJBA3FBAUcNACAAKAIUIgFFDQAgACABQQFrNgIUDwsgAkGAAXEiBgRAIAMgAEEQajYCVEEAQQH+HgL0QhoLIABBBGohASAAKAIMIgcgACgCECIANgIAIANBzABqIABHBEAgAEEEayAHNgIACyABIAVBAXQgAkEddHFBH3VB/////wdx/kECACEAIAZFDQAgA0EANgJUEEULIARFIABBAE5xDQAgAUEBEBELC6gCAQZ/AkAgAC0AAEEPcQ0AIABBBGpBAEEK/kgCAA0AQQAPCwJ/AkAgACgCACICQQ9xRQRAIABBBGpBAEEK/kgCAEUNASAAKAIAIQILIAAQICIBQQpHDQAgAEEIaiEEIABBBGohA0HkACEBA0ACQCABRQ0AIAMoAgBFDQAgAUEBayEBIAQoAgBFDQELCyAAECAiAUEKRw0AIAJBBHFFIQUgAkEDcUECRyECA0ACQCADKAIAIgFB/////wNxIgYgAUEARyAFcXJFDQACQCACDQAgBiMDKAIYRw0AQRAMBAsgBEEB/h4CABogAyABIAFBgICAgHhyIgH+SAIAGiADIAEQLiEBIARBAf4lAgAaIAFBG0YNACABDQILIAAQICIBQQpGDQALCyABCwuGBAEEfwJAIAJBAEoEQCACQf8DTQ0BQdA/QR0QQ0HUPxAPGgJAQfTAACgCACIDBEBB8MAAKAIARQ0BC0HUPxAODAILQdTAACACNgIAQdDAACABNgIAQczAACAANgIAQfDAAEEBNgIAQdjAACADQQFqIgQ2AgBB3MAAQQD+FwIAQeTAACAD/hcCAEHgwABB4MAA/hACAEEBav4XAgBB6MAA/hACAARAAkBB7D8oAgBFBEBB7D9BfxBHDAELQfg/KAIABEBB9D9BAf4eAgAaQfQ/Qf////8HEBELC0HYwAAoAgAhBAtB1D8QDiACQdzAAP4QAgAiA0oEQCAEQQF0IQYDQCADQQAgA0HAACACIANrIAZtIgQgBEHAAEwbIANqIgX+SALcQCIERgR/IAEgAyAFIAIgAiAFShsgABEBAEHcwAD+EAIABSAECyIDIAJIDQALC0EAIQMCQANAQeTAAP4QAgBBAEwNAUHkwAD+EAIAQQBMDQFB5MAA/hACAEEATA0BQeTAAP4QAgBBAEwNASADQQRqIgNBgIDAAEcNAAtB1D8QDxpB7MAAQQH+FwIAQeTAAP4QAgBBAEoEQANAQZzAAEHUPxAtQeTAAP4QAgBBAEoNAAsLQezAAEEA/hcCAEHUPxAOC0HUPxAPGkHwwABBADYCAEHUPxAOCw8LIAFBACACIAARAQALZAEBfwJAIABFDQAgAUEASA0AIABBA3ENACABRQRADwsgAEEAIABBAP5IAvhAIgIgACACRhshAgJAIAFB/////wdGDQAgACACRw0AIAFBAUYNASABQQFrIQELIAAgAf4AAgAaCwumAQIBfQF/AkAgAIsiAbwiAkHVvrL4A08EQCACQYGAgIkETwRAQwAAAAAgAZVDAACAP5IhAQwCC0MAAIA/QwAAAEAgASABkhAmQwAAAECSlZMhAQwBCyACQfmKi/QDTwRAIAEgAZIQJiIBIAFDAAAAQJKVIQEMAQsgAkGAgIAESQ0AIAFDAAAAwJQQJiIBjCABQwAAAECSlSEBCyABjCABIAC8QQBIGwuwDAEIfwJAIABFDQBBlM8ALQAAQQJxBEBBmM8AEA8NAQsgAEEIayIDIABBBGsoAgAiAkF4cSIAaiEFAkACQCACQQFxDQAgAkECcUUNASADIAMoAgAiBGsiA0HoywAoAgBJDQEgACAEaiEAAkACQAJAQezLACgCACADRwRAIAMoAgwhASAEQf8BTQRAIAEgAygCCCICRw0CQdjLAEHYywAoAgBBfiAEQQN2d3E2AgAMBQsgAygCGCEHIAEgA0cEQCADKAIIIgIgATYCDCABIAI2AggMBAsgAygCFCICBH8gA0EUagUgAygCECICRQ0DIANBEGoLIQQDQCAEIQYgAiIBQRRqIQQgASgCFCICDQAgAUEQaiEEIAEoAhAiAg0ACyAGQQA2AgAMAwsgBSgCBCICQQNxQQNHDQNB4MsAIAA2AgAgBSACQX5xNgIEIAMgAEEBcjYCBCAFIAA2AgAMBAsgAiABNgIMIAEgAjYCCAwCC0EAIQELIAdFDQACQCADKAIcIgRBAnQiAigCiE4gA0YEQCACQYjOAGogATYCACABDQFB3MsAQdzLACgCAEF+IAR3cTYCAAwCCwJAIAMgBygCEEYEQCAHIAE2AhAMAQsgByABNgIUCyABRQ0BCyABIAc2AhggAygCECICBEAgASACNgIQIAIgATYCGAsgAygCFCICRQ0AIAEgAjYCFCACIAE2AhgLIAMgBU8NACAFKAIEIgRBAXFFDQACQAJAAkACQCAEQQJxRQRAQfDLACgCACAFRgRAQfDLACADNgIAQeTLAEHkywAoAgAgAGoiADYCACADIABBAXI2AgQgA0HsywAoAgBHDQZB4MsAQQA2AgBB7MsAQQA2AgAMBgtB7MsAKAIAIgcgBUYEQEHsywAgAzYCAEHgywBB4MsAKAIAIABqIgA2AgAgAyAAQQFyNgIEIAAgA2ogADYCAAwGCyAEQXhxIABqIQAgBSgCDCEBIARB/wFNBEAgBSgCCCICIAFGBEBB2MsAQdjLACgCAEF+IARBA3Z3cTYCAAwFCyACIAE2AgwgASACNgIIDAQLIAUoAhghCCABIAVHBEAgBSgCCCICIAE2AgwgASACNgIIDAMLIAUoAhQiAgR/IAVBFGoFIAUoAhAiAkUNAiAFQRBqCyEEA0AgBCEGIAIiAUEUaiEEIAEoAhQiAg0AIAFBEGohBCABKAIQIgINAAsgBkEANgIADAILIAUgBEF+cTYCBCADIABBAXI2AgQgACADaiAANgIADAMLQQAhAQsgCEUNAAJAIAUoAhwiBEECdCICKAKITiAFRgRAIAJBiM4AaiABNgIAIAENAUHcywBB3MsAKAIAQX4gBHdxNgIADAILAkAgBSAIKAIQRgRAIAggATYCEAwBCyAIIAE2AhQLIAFFDQELIAEgCDYCGCAFKAIQIgIEQCABIAI2AhAgAiABNgIYCyAFKAIUIgJFDQAgASACNgIUIAIgATYCGAsgAyAAQQFyNgIEIAAgA2ogADYCACADIAdHDQBB4MsAIAA2AgAMAQsgAEH/AU0EQCAAQfgBcUGAzABqIQICf0HYywAoAgAiBEEBIABBA3Z0IgBxRQRAQdjLACAAIARyNgIAIAIMAQsgAigCCAshACACIAM2AgggACADNgIMIAMgAjYCDCADIAA2AggMAQtBHyEBIABB////B00EQCAAQSYgAEEIdmciAmt2QQFxIAJBAXRrQT5qIQELIAMgATYCHCADQgA3AhAgAUECdEGIzgBqIQQCfwJAAn9B3MsAKAIAIgZBASABdCICcUUEQEHcywAgAiAGcjYCACAEIAM2AgBBGCEBQQgMAQsgAEEZIAFBAXZrQQAgAUEfRxt0IQEgBCgCACEEA0AgBCICKAIEQXhxIABGDQIgAUEddiEEIAFBAXQhASACIARBBHFqIgYoAhAiBA0ACyAGIAM2AhBBGCEBIAIhBEEICyEAIAMiAgwBCyACKAIIIgQgAzYCDCACIAM2AghBGCEAQQghAUEACyEGIAEgA2ogBDYCACADIAI2AgwgACADaiAGNgIAQfjLAEH4ywAoAgBBAWsiAEF/IAAbNgIAC0GUzwAtAABBAnFFDQBBmM8AEA4LC9UoAQl/QcDLACgCAEUEQBBACwJAQZTPAC0AAEECcQRAQZjPABAPDQELAkACQAJAAkACQAJAAkACQCAAQfQBTQRAQdjLACgCACICQRAgAEELakH4A3EgAEELSRsiBUEDdiIAdiIBQQNxBEACQCABQX9zQQFxIABqIgRBA3QiAEGAzABqIgUgACgCiEwiASgCCCIDRgRAQdjLACACQX4gBHdxNgIADAELIAMgBTYCDCAFIAM2AggLIAFBCGohBCABIABBA3I2AgQgACABaiIAIAAoAgRBAXI2AgQMCQsgBUHgywAoAgAiA00NASABBEACQEECIAB0IgRBACAEa3IgASAAdHFoIgdBA3QiAUGAzABqIgQgASgCiEwiACgCCCIGRgRAQdjLACACQX4gB3dxIgI2AgAMAQsgBiAENgIMIAQgBjYCCAsgACAFQQNyNgIEIAAgBWoiByABIAVrIgZBAXI2AgQgACABaiAGNgIAIAMEQCADQXhxQYDMAGohAUHsywAoAgAhBQJ/IAJBASADQQN2dCIDcUUEQEHYywAgAiADcjYCACABDAELIAEoAggLIQIgASAFNgIIIAIgBTYCDCAFIAE2AgwgBSACNgIICyAAQQhqIQRB7MsAIAc2AgBB4MsAIAY2AgAMCQtB3MsAKAIAIghFDQEgCGhBAnQoAohOIgIoAgRBeHEgBWshBiACIQADQAJAIAAoAhAiAUUEQCAAKAIUIgFFDQELIAEoAgRBeHEgBWsiACAGIAAgBkkiABshBiABIAIgABshAiABIQAMAQsLIAIoAhghBwJAIAIgAigCDCIBRwRAIAIoAggiACABNgIMIAEgADYCCAwBCwJAIAIoAhQiAAR/IAJBFGoFIAIoAhAiAEUNASACQRBqCyEDA0AgAyEEIAAiAUEUaiEDIAAoAhQiAA0AIAFBEGohAyABKAIQIgANAAsgBEEANgIADAELQQAhAQsCQCAHRQ0AAkAgAigCHCIAQQJ0IgMoAohOIAJGBEAgA0GIzgBqIAE2AgAgAQ0BQdzLACAIQX4gAHdxNgIADAILAkAgAiAHKAIQRgRAIAcgATYCEAwBCyAHIAE2AhQLIAFFDQELIAEgBzYCGCACKAIQIgAEQCABIAA2AhAgACABNgIYCyACKAIUIgBFDQAgASAANgIUIAAgATYCGAsCQCAGQQ9NBEAgAiAFIAZqIgBBA3I2AgQgACACaiIAIAAoAgRBAXI2AgQMAQsgAiAFQQNyNgIEIAIgBWoiBSAGQQFyNgIEIAUgBmogBjYCAEHgywAoAgAiAwRAIANBeHFBgMwAaiEAQezLACgCACEBAn9B2MsAKAIAIgRBASADQQN2dCIDcUUEQEHYywAgAyAEcjYCACAADAELIAAoAggLIQMgACABNgIIIAMgATYCDCABIAA2AgwgASADNgIIC0HsywAgBTYCAEHgywAgBjYCAAsgAkEIaiEEDAgLQX8hBSAAQb9/Sw0AIABBC2oiAUF4cSEFQdzLACgCACIHRQ0AQR8hCEEAIAVrIQQgAEH0//8HTQRAIAVBJiABQQh2ZyIAa3ZBAXEgAEEBdGtBPmohCAsCQAJAAkAgCEECdCgCiE4iAUUEQEEAIQAMAQtBACEAIAVBGSAIQQF2a0EAIAhBH0cbdCECA0ACQCABKAIEQXhxIAVrIgYgBE8NACABIQMgBiIEDQBBACEEIAEhAAwDCyAAIAEoAhQiBiAGIAEgAkEddkEEcWooAhAiAUYbIAAgBhshACACQQF0IQIgAQ0ACwsgACADckUEQEEAIQNBAiAIdCIAQQAgAGtyIAdxIgBFDQMgAGhBAnQoAohOIQALIABFDQELA0AgACgCBEF4cSAFayICIARJIQEgAiAEIAEbIQQgACADIAEbIQMgACgCECIBBH8gAQUgACgCFAsiAA0ACwsgA0UNACAEQeDLACgCACAFa08NACADKAIYIQggAyADKAIMIgBHBEAgAygCCCIBIAA2AgwgACABNgIIDAcLIAMoAhQiAQR/IANBFGoFIAMoAhAiAUUNAiADQRBqCyECA0AgAiEGIAEiAEEUaiECIAAoAhQiAQ0AIABBEGohAiAAKAIQIgENAAsgBkEANgIADAYLIAVB4MsAKAIAIgNNBEBB7MsAKAIAIQACQCADIAVrIgFBEE8EQCAAIAVqIgIgAUEBcjYCBCAAIANqIAE2AgAgACAFQQNyNgIEDAELIAAgA0EDcjYCBCAAIANqIgEgASgCBEEBcjYCBEEAIQJBACEBC0HgywAgATYCAEHsywAgAjYCACAAQQhqIQQMBwsgBUHkywAoAgAiAEkEQEHkywAgACAFayIBNgIAQfDLAEHwywAoAgAiACAFaiICNgIAIAIgAUEBcjYCBCAAIAVBA3I2AgQgAEEIaiEEDAcLQQAhBEHAywAoAgBFBEAQQAtByMsAKAIAIgAgBUEvaiIGakEAIABrcSIDIAVNDQZBkM8AKAIAIgAEQEGIzwAoAgAiASADaiICIAFNDQcgACACSQ0HC0GUzwAtAABBBHFFBEACQAJAAkACQEHwywAoAgAiAQRAQbDPACEAA0AgACgCACICIAFNBEAgASACIAAoAgRqSQ0DCyAAKAIIIgANAAsLQcjPABAPGkEAEBYiAkF/Rg0DIAMhAUHEywAoAgAiAEEBayIEIAJxBEAgASACayACIARqQQAgAGtxaiEBCyABIAVNDQNBkM8AKAIAIgAEQEGIzwAoAgAiBCABaiIHIARNDQQgACAHSQ0ECyABEBYiACACRw0BDAYLQcjPABAPGkHIywAoAgAiASAGQeTLACgCAGtqQQAgAWtxIgEQFiICIAAoAgAgACgCBGpGDQEgAiEACyAAQX9GDQEgBUEwaiABSwRAQcjLACgCACICIAYgAWtqQQAgAmtxIgIQFkF/Rg0CIAEgAmohAQsgACECDAQLIAJBf0cNAwtBlM8AQZTPACgCAEEEcjYCAEHIzwAQDgtByM8AEA8aIAMQFiECQQAQFiEAQcjPABAOIAJBf0YNAyAAQX9GDQMgACACTQ0DIAAgAmsiASAFQShqTQ0DDAILQQAhAAwEC0HIzwAQDgtBiM8AQYjPACgCACABaiIANgIAQYzPACgCACAASQRAQYzPACAANgIACwJAAkACQEHwywAoAgAiBARAQbDPACEAA0AgAiAAKAIAIgMgACgCBCIGakYNAiAAKAIIIgANAAsMAgtB6MsAKAIAIgBBACAAIAJNG0UEQEHoywAgAjYCAAtBACEAQbTPACABNgIAQbDPACACNgIAQfjLAEF/NgIAQfzLAEHAywAoAgA2AgBBvM8AQQA2AgADQCAAQQN0IgMgA0GAzABqIgQ2AohMIAMgBDYCjEwgAEEBaiIAQSBHDQALQeTLACABQShrIgBBeCACa0EHcSIBayIDNgIAQfDLACABIAJqIgE2AgAgASADQQFyNgIEIAAgAmpBKDYCBEH0ywBB0MsAKAIANgIADAILIAIgBE0NACADIARLDQAgACgCDEEIcQ0AIAAgASAGajYCBEHwywAgBEF4IARrQQdxIgBqIgI2AgBB5MsAQeTLACgCACABaiIBIABrIgA2AgAgAiAAQQFyNgIEIAEgBGpBKDYCBEH0ywBB0MsAKAIANgIADAELQejLACgCACACSwRAQejLACACNgIACyABIAJqIQZBsM8AIQACQANAIAYgACgCACIDRwRAIAAoAggiAA0BDAILCyAALQAMQQhxRQ0DC0GwzwAhAANAAkAgACgCACIDIARNBEAgBCADIAAoAgRqIgZJDQELIAAoAgghAAwBCwtB5MsAIAFBKGsiAEF4IAJrQQdxIgNrIgc2AgBB8MsAIAIgA2oiAzYCACADIAdBAXI2AgQgACACakEoNgIEQfTLAEHQywAoAgA2AgAgBCAGQScgBmtBB3FqQS9rIgAgACAEQRBqSRsiA0EbNgIEIANBuM8AKQIANwIQIANBsM8AKQIANwIIQbjPACADQQhqNgIAQbTPACABNgIAQbDPACACNgIAQbzPAEEANgIAIANBGGohAANAIABBBzYCBCAAQQhqIABBBGohACAGSQ0ACyADIARGDQAgAyADKAIEQX5xNgIEIAQgAyAEayICQQFyNgIEIAMgAjYCAAJ/IAJB/wFNBEAgAkH4AXFBgMwAaiEAAn9B2MsAKAIAIgFBASACQQN2dCICcUUEQEHYywAgASACcjYCACAADAELIAAoAggLIQEgACAENgIIIAEgBDYCDEEIIQNBDAwBC0EfIQAgAkH///8HTQRAIAJBJiACQQh2ZyIAa3ZBAXEgAEEBdGtBPmohAAsgBCAANgIcIARCADcCECAAQQJ0QYjOAGohAQJAAkBB3MsAKAIAIgNBASAAdCIGcUUEQEHcywAgAyAGcjYCACABIAQ2AgAMAQsgAkEZIABBAXZrQQAgAEEfRxt0IQAgASgCACEDA0AgAyIBKAIEQXhxIAJGDQIgAEEddiEDIABBAXQhACABIANBBHFqIgYoAhAiAw0ACyAGIAQ2AhALIAQgATYCGEEMIQMgBCIBIQBBCAwBCyABKAIIIgAgBDYCDCABIAQ2AgggBCAANgIIQQAhAEEMIQNBGAsgAyAEaiABNgIAIARqIAA2AgALQeTLACgCACIAIAVNDQBB5MsAIAAgBWsiATYCAEHwywBB8MsAKAIAIgAgBWoiAjYCACACIAFBAXI2AgQgACAFQQNyNgIEIABBCGohBAwDCyMBQTA2AgBBACEEDAILIAAgAjYCACAAIAAoAgQgAWo2AgQgAkF4IAJrQQdxaiIIIAVBA3I2AgQgA0F4IANrQQdxaiIEIAUgCGoiA2shBgJAQfDLACgCACAERgRAQfDLACADNgIAQeTLAEHkywAoAgAgBmoiADYCACADIABBAXI2AgQMAQtB7MsAKAIAIARGBEBB7MsAIAM2AgBB4MsAQeDLACgCACAGaiIANgIAIAMgAEEBcjYCBCAAIANqIAA2AgAMAQsgBCgCBCIAQQNxQQFGBEAgAEF4cSEJIAQoAgwhAgJAIABB/wFNBEAgBCgCCCIBIAJGBEBB2MsAQdjLACgCAEF+IABBA3Z3cTYCAAwCCyABIAI2AgwgAiABNgIIDAELIAQoAhghBwJAIAIgBEcEQCAEKAIIIgAgAjYCDCACIAA2AggMAQsCQCAEKAIUIgAEfyAEQRRqBSAEKAIQIgBFDQEgBEEQagshAQNAIAEhBSAAIgJBFGohASAAKAIUIgANACACQRBqIQEgAigCECIADQALIAVBADYCAAwBC0EAIQILIAdFDQACQCAEKAIcIgBBAnQiASgCiE4gBEYEQCABQYjOAGogAjYCACACDQFB3MsAQdzLACgCAEF+IAB3cTYCAAwCCwJAIAQgBygCEEYEQCAHIAI2AhAMAQsgByACNgIUCyACRQ0BCyACIAc2AhggBCgCECIABEAgAiAANgIQIAAgAjYCGAsgBCgCFCIARQ0AIAIgADYCFCAAIAI2AhgLIAYgCWohBiAEIAlqIgQoAgQhAAsgBCAAQX5xNgIEIAMgBkEBcjYCBCADIAZqIAY2AgAgBkH/AU0EQCAGQfgBcUGAzABqIQACf0HYywAoAgAiAUEBIAZBA3Z0IgJxRQRAQdjLACABIAJyNgIAIAAMAQsgACgCCAshASAAIAM2AgggASADNgIMIAMgADYCDCADIAE2AggMAQtBHyECIAZB////B00EQCAGQSYgBkEIdmciAGt2QQFxIABBAXRrQT5qIQILIAMgAjYCHCADQgA3AhAgAkECdEGIzgBqIQACQAJAQdzLACgCACIBQQEgAnQiBXFFBEBB3MsAIAEgBXI2AgAgACADNgIADAELIAZBGSACQQF2a0EAIAJBH0cbdCECIAAoAgAhAQNAIAEiACgCBEF4cSAGRg0CIAJBHXYhASACQQF0IQIgACABQQRxaiIFKAIQIgENAAsgBSADNgIQCyADIAA2AhggAyADNgIMIAMgAzYCCAwBCyAAKAIIIgEgAzYCDCAAIAM2AgggA0EANgIYIAMgADYCDCADIAE2AggLIAhBCGohBAwBCwJAIAhFDQACQCADKAIcIgFBAnQiAigCiE4gA0YEQCACQYjOAGogADYCACAADQFB3MsAIAdBfiABd3EiBzYCAAwCCwJAIAMgCCgCEEYEQCAIIAA2AhAMAQsgCCAANgIUCyAARQ0BCyAAIAg2AhggAygCECIBBEAgACABNgIQIAEgADYCGAsgAygCFCIBRQ0AIAAgATYCFCABIAA2AhgLAkAgBEEPTQRAIAMgBCAFaiIAQQNyNgIEIAAgA2oiACAAKAIEQQFyNgIEDAELIAMgBUEDcjYCBCADIAVqIgIgBEEBcjYCBCACIARqIAQ2AgAgBEH/AU0EQCAEQfgBcUGAzABqIQACf0HYywAoAgAiAUEBIARBA3Z0IgVxRQRAQdjLACABIAVyNgIAIAAMAQsgACgCCAshASAAIAI2AgggASACNgIMIAIgADYCDCACIAE2AggMAQtBHyEAIARB////B00EQCAEQSYgBEEIdmciAGt2QQFxIABBAXRrQT5qIQALIAIgADYCHCACQgA3AhAgAEECdEGIzgBqIQECQAJAIAdBASAAdCIFcUUEQEHcywAgBSAHcjYCACABIAI2AgAMAQsgBEEZIABBAXZrQQAgAEEfRxt0IQAgASgCACEFA0AgBSIBKAIEQXhxIARGDQIgAEEddiEFIABBAXQhACABIAVBBHFqIgYoAhAiBQ0ACyAGIAI2AhALIAIgATYCGCACIAI2AgwgAiACNgIIDAELIAEoAggiACACNgIMIAEgAjYCCCACQQA2AhggAiABNgIMIAIgADYCCAsgA0EIaiEEC0GUzwAtAABBAnFFDQBBmM8AEA4LIAQL0gECA38BfEHkACEDAkACfwJAA0AgAwRAIAEEQCABKAIADQMLIANBAWshAyAAKAIAIAJGDQEMBAsLIAENAEEBDAELIAFBAf4eAgAaQQALIwUhAwJAIAAoAgAgAkcNAEQAAAAAAADwP0QAAAAAAABZQCADGyEGIwMhBANAAkACQCADRQRAIAQtAClBAUcNAQsDQCAEKAIkDQQgACACIAYQGkG3f0YNAAsMAQsgACACRAAAAAAAAPB/EBoaCyAAKAIAIAJGDQALCw0AIAFBAf4lAgAaCwtaAQJ/IABBB2pBeHEhAgNAAkAgAkEAQcQ//hACACIAIAJqIgEgAE0bRQRAIAE/AEEQdE0NASABEAsNAQsjAUEwNgIAQX8PC0EAIAAgAf5IAsQ/IABHDQALIAALCgAgABAwIAAQEwuHBAECfyACQYAETwRAIAIEQCAAIAEgAvwKAAALDwsgACACaiEDAkAgACABc0EDcUUEQAJAIABBA3FFBEAgACECDAELIAJFBEAgACECDAELIAAhAgNAIAIgAS0AADoAACABQQFqIQEgAkEBaiICQQNxRQ0BIAIgA0kNAAsLIANBfHEhAAJAIANBwABJDQAgAiAAQUBqIgRLDQADQCACIAEoAgA2AgAgAiABKAIENgIEIAIgASgCCDYCCCACIAEoAgw2AgwgAiABKAIQNgIQIAIgASgCFDYCFCACIAEoAhg2AhggAiABKAIcNgIcIAIgASgCIDYCICACIAEoAiQ2AiQgAiABKAIoNgIoIAIgASgCLDYCLCACIAEoAjA2AjAgAiABKAI0NgI0IAIgASgCODYCOCACIAEoAjw2AjwgAUFAayEBIAJBQGsiAiAETQ0ACwsgACACTQ0BA0AgAiABKAIANgIAIAFBBGohASACQQRqIgIgAEkNAAsMAQsgA0EESQRAIAAhAgwBCyACQQRJBEAgACECDAELIANBBGshBCAAIQIDQCACIAEtAAA6AAAgAiABLQABOgABIAIgAS0AAjoAAiACIAEtAAM6AAMgAUEEaiEBIAJBBGoiAiAETQ0ACwsgAiADSQRAA0AgAiABLQAAOgAAIAFBAWohASACQQFqIgIgA0cNAAsLC5sBAQF/IwBBIGsiCSQAAkAgBkEATA0AIAdBAEwNACAIQQBMBEAgBiAHbCIAQQBMDQEgAEECdCIARQ0BIAQgBUECdGpBACAA/AsADAELIAkgCDYCHCAJIAc2AhggCSAGNgIUIAkgBCAFQQJ0ajYCECAJIAIgA0ECdGo2AgwgCSAAIAFBAnRqNgIIQQIgCUEIaiAHEBALIAlBIGokAAvVAQIBfAF/IABBA3EEf0FkBRBIIwZFBEACfxAAIQNBAEEAIAD+SAL4QBoCQBAAIAIgA6AiAmQNAAJ/A0BBACAAQQAgAEEA/kgC+EAiBCAAIARGG0UNARoQSCABIAD+EAIARgRAQQBBACAA/kgC+EAaEAAgAmQNAwwBCwtBegsMAQtBACAAQQD+SAL4QBpBt38LDwtBekG3f0EAIAAgAUJ/IAJEAAAAAABAj0CiRAAAAAAAQI9AovwGIAJEAAAAAAAA8H9hG/4BAgAiAEECRhsgAEEBRhsLC8sDAQN/IwBBEGsiBCQAIAAQDxoCQCAAQfzAAEcEQCAAIAEQMiEBDAELIwEiA0EBOgAEQfzAACABEDIhASADQQA6AAQLIAAQDiABBH8gBCACKAIINgIIIAQgAikCADcDACMAQTBrIgAkAAJAAn8gASgCHCID/hACfCECA0BBACACRQ0BGiACIAMgAiACQQFq/kgCfCICRw0AC0EBC0UEQEEAIQMMAQsgAUEEaiICEA8aIAAgBCgCCDYCICAAIAQpAgA3AxggASAAQRhqEDQhAyACEA4CfyADBH8gAUEC/kECACEFQQEhAyABKAIcIgIgBUECRg0BGiAAIAE2AiwgACABNgIQIABBHzYCKCAAQSA2AiQgACAAKQIkNwMIIwBBEGsiAyQAIAIoAnhBBGoQDxogAigCeCADIAAoAhA2AgggAyAAKQIINwMAIAMQNBogAigCeEEEahAOAkAgAigCeEEC/kECAEECRg0AIAL+EAKAAQRAIAJBf/4AAgAaDAELIAIjAxAMCyADQRBqJABBAQVBAAshAyABKAIcCyIBQQH+JQJ8QQFGBEAgAUH8AGpB/////wcQEQsLIABBMGokACADBUEACyAEQRBqJAAL5AkDGH8BfQF7AkAgASACTg0AIAUgBk4NACAEIANBBGoiDU4EQCAEIAMgA0EIaiIIIARBAWoiByAHIAhIGyADa0EFayIPQXxxaiIIQQVqIgcgBCAHShsiB0ECdCAPQQJ0QXBxayEVIAcgCGtBBGsiFkF8cSEQIA9BBHEhGSAPQQJ2QQFqQf7///8HcSEaIANBAnQhFyAAKAIQIQcgASESA0AgACgCFCIJIAEgEWpsQQJ0IgggACgCCCIMIBVqaiEbIAwgF2ogCGpBEGohHCAAKAIEIAUgB2xBAnRqIQogDCAJIBJsQQJ0aiEOIAAoAgAgACgCDCASbEECdGohHSAFIRQDQCAdIBRBAnRqKgIAIh/9EyEgQQAhDCADIQhBACETIA0iByEJIA9BBE8EQANAIA4gCEECdCIJaiIIIAkgCmr9AAAAICD95gEgCP0AAAD95AH9CwAAIA4gB0ECdCIJaiIIIAkgCmr9AAAAICD95gEgCP0AAAD95AH9CwAAIAwiC0ECaiEMIAdBBGohCCAHQQhqIgkhByATQQJqIhMgGkcNAAsgC0EBciELCyAZRQRAIA4gCEECdCIIaiIHIAggCmr9AAAAICD95gEgB/0AAAD95AH9CwAAIAwhCyAJIQgLAkAgBCAITA0AAkACQCAWQQhJDQAgHCALQQR0IgdqIAogFWogB2pJBEAgCiAXaiAHakEQaiAHIBtqSQ0BCyAIIBBqIQdBACEMA0AgDiAIIAxqQQJ0IhNqIgkgCf0AAgAgCiATav0AAgAgIP3mAf3kAf0LAgAgDEEEaiIMIBBHDQALIBAgFkcNAQwCCyAIIQcLA0AgDiAHQQJ0IglqIgggCCoCACAJIApqKgIAIB+UkjgCACAHQQFqIgcgBEgNAAsLIAogACgCECIHQQJ0aiEKIBRBAWoiFCAGRw0ACyARQQFqIREgEkEBaiISIAJHDQALDAELIAMgBE4NACAAKAIEIgkgACgCECIeIAVsIgdBAnRqIQwgACgCDCEVIAAoAgAhFiAJIAQgB2ogHiAGIAVBf3NqbGpBAnRqIRcgACgCFCIOQQJ0IRkgAyAEIANrIg9BfHEiEGohCCAAKAIIIhEgBCABIA5sIgBqQQJ0aiEaIBEgACADakECdGohGyAJIAMgB2pBAnRqIRxBACEAIB5BgICAgAJxQQBHIR0gD0EESSETA0AgESABIA5sQQJ0aiEYIBYgASAVbEECdGohEiAbIAAgGWwiB2ogF0kgHCAHIBpqSXEgHXIgE3IhFCAFIQkgDCEKA0AgEiAJQQJ0aioCACEfIAMhBwJAIBRFBEAgH/0TISBBACEHA0AgGCADIAdqQQJ0Ig1qIgsgC/0AAgAgCiANav0AAgAgIP3mAf3kAf0LAgAgB0EEaiIHIBBHDQALIAghByAPIBBGDQELIAdBAWohCyAEIAdrQQFxBEAgGCAHQQJ0Ig1qIgcgByoCACAKIA1qKgIAIB+UkjgCACALIQcLIAQgC0YNAANAIBggB0ECdCINaiILIAsqAgAgCiANaioCACAflJI4AgAgGCANQQRqIg1qIgsgCyoCACAKIA1qKgIAIB+UkjgCACAHQQJqIgcgBEcNAAsLIAogHkECdGohCiAJQQFqIgkgBkcNAAsgAEEBaiEAIAFBAWoiASACRw0ACwsL3QQDAnsHfwF9IwBBIGsiCyQAAkAgBEEATA0AIAVBAEwNACAEEBQhDCAEQQh2Ig5BAnQQFCENAkAgDEUNACANRQ0AIARBgAJPBEAgACABQQJ0aiEQQQAhAANAIABBCHQhESAQIABBCnRqIQ/9DAAAAAAAAAAAAAAAAAAAAAAhCUEAIQEDQCAPIAFBAnRqIgT9AAIw/eABIAT9AAIg/eABIAT9AAIQ/eABIAT9AAIA/eABIAn96wH96wH96wH96wEhCSABQRBqIgFBgAJHDQALIAwgEWohASANIABBAnRqAn0gCSAJIAn9DQgJCgsMDQ4PAAECAwABAgP96wEiCSAJIAn9DQQFBgcAAQIDAAECAwABAgP96wH9HwAiEkMAAAAAXARAQwAAgD8gEpX9EyEKQQAhBANAIAEgBGogDyAEQQJ0av0AAgD9DAAA/kIAAP5CAAD+QgAA/kL95gEgCv3mASIJ/R8AkPwA/REgCf0fAZD8AP0cASAJ/R8CkPwA/RwCIAn9HwOQ/AD9HAP9DIH///+B////gf///4H////9uAH9DH8AAAB/AAAAfwAAAH8AAAD9tgEgCf0NAAQIDAAAAAAAAAAAAAAAAP1aAAAAIARBBGoiBEGAAkcNAAsgEkMEAgE8lAwBCyABQQBBgAL8CwBDAAAAAAs4AgAgAEEBaiIAIA5HDQALCyALQQA2AhwgCyAFNgIYIAsgDjYCFCALIA02AgwgCyAMNgIIIAsgAiADajYCBCALIAYgB0ECdGo2AhAgCCALQQRqIAUQEAsgDBATIA0QEwsgC0EgaiQAC6gCAgJ/AXsjAEEgayIJJAACQCAFQQBMDQAgBEEATARAIAVBAnQiAEUNASAGIAdBAnRqQQAgAPwLAAwBCyAJIAU2AhwgCSAEQQV2Igo2AhggCSACIANqNgIIIAkgBiAHQQJ0ajYCECAJIAAgAUECdGoiAjYCDCAKQQJ0EBQiAUUNACAEQR9LBEBBACEHA0AgASAHQQJ0aiACIAdBB3RqIgD9AAIAIAD9AAJw/eQBIAD9AAJA/eQBIAD9AAIgIAD9AAJg/eQB/eQBIAD9AAIQIAD9AAJQ/eQBIAD9AAIw/eQB/eQBIgv9HwAgC/0fAZIgC/0fApIgC/0fA5I4AgAgB0EBaiIHIApHDQALCyAJIAE2AhQgCCAJQQhqIAUQECABEBMLIAlBIGokAAswAQF/IAEjAyICKAJIIABBAnRqIgAoAgBHBEAgACABNgIAIAIgAi0AKkEBcjoAKgsLnAMBBn8gAC0AAEEPcUUEQCAAQQRqQQBBCv5IAgBBCnEPCwJ/IAAoAgAhAgJAAkACQAJAAkAjAyIBKAIYIgQgACgCBCIDQf////8DcSIGRw0AAkAgAkEIcUUNACAAKAIUQQBODQAgAEEANgIUIANBgICAgARxIQMMAgsgAkEDcUEBRw0AQQYgACgCFCIBQf7///8HSw0FGiAAIAFBAWo2AhQMAgtBOCEFIAZB/////wNGDQMCQCAGDQBBACADIAJBBHEbDQAgAyAAQQRqIAMgAkGAAXEEfyABKAJQRQRAIAFBdDYCUAsgACgCCCEGIAEgAEEQajYCVCAEQYCAgIB4ciAEIAYbBSAECyADQYCAgIAEcXL+SAIARg0BIAFBADYCVCACQQxxQQxHDQAgACgCCA0EC0EKDAQLIAJBD3ENAQtBAAwCCyAAIAEoAkwiAjYCECAAIAFBzABqIgU2AgwgAEEQaiEEIAIgBUcEQCACQQRrIAQ2AgALIAEgBDYCTEEAIQUgAUEANgJUIANFDQAgAEEANgIUQT4hBQsgBQsLFgAgAEEA/kECAEECRgRAIABBARARCwsxACAAQQBBAf5IAgAEQCAAQQFBAv5IAgAaA0AgAEEAQQIQFSAAQQBBAv5IAgANAAsLC/EBAgN/A3wjAEEQayIEJAACQAJ/AkACQCMFIgINACMDIgMtAChBAUcNACADLQApRQ0BC0QAAAAAAADwP0QAAAAAAABZQCACGyEFEABEAAAAAAAA8H+gIQcjAyECA0ACQCACKAIkRQRAIAcQAKEiBkQAAAAAAAAAAGVFDQFByQAMBAtBCyEADAQLIAAgASAFIAYgBSAGYxsQGiIDQbd/Rg0AC0EAIANrDAELQQAgACABRAAAAAAAAPB/EBprCyIAQQAgAEFvcUELRhsgACAAQckARxsiAEEbRw0AQRtBAEHowgAoAgAbIQALIARBEGokACAAC20BAn8jAEEQayIBJAAgAEEBNgIgIABBBGoiAhAPGiAAKAIsIAAoAjBHBEADQCABQQRqIAAQNiACEA4gASgCDCABKAIEEQAAIAIQDxogACgCLCAAKAIwRw0ACwsgAhAOIABBADYCICABQRBqJAALLgAgACgCAEUEQCAAQQEQRw8LIAAoAgwEQCAAQQhqIgBBAf4eAgAaIABBARARCwuiBAICfwR9AkACQAJAAn0CQCAAvCICQf////8HcSIBQcTw1owETwRAIAFBgICA/AdLDQUgAkEASARAQwAAgL8PCyABQZjkxZUESQ0BIABDAAAAf5QPCyABQZnkxfUDSQ0CIAFBkauU/ANLDQAgAkEATgRAQQEhAUPR9xc3IQMgAEOAcTG/kgwCC0F/IQFD0fcXtyEDIABDgHExP5IMAQsgAEM7qrg/lEMAAAA/IACYkvwAIgGyIgRD0fcXN5QhAyAAIARDgHExv5SSCyIAIAAgA5MiAJMgA5MhAwwBCyABQYCAgJgDSQ0BQQAhAQsgACAAQwAAAD+UIgWUIgQgBCAEQxAwzzqUQ2iICL2SlEMAAIA/kiIGQwAAQEAgBiAFlJMiBZNDAADAQCAAIAWUk5WUIQUgAUUEQCAAIAAgBZQgBJOTDwsgACAFIAOTlCADkyAEkyEDAkACQAJAIAFBAWoOAwACAQILIAAgA5NDAAAAP5RDAAAAv5IPCyAAQwAAgL5dBEAgAyAAQwAAAD+Sk0MAAADAlA8LIAAgA5MiACAAkkMAAIA/kg8LIAFBF3QiAkGAgID8A2q+IQQgAUE5TwRAIAAgA5NDAACAP5IiACAAkkMAAAB/lCAAIASUIAFBgAFGG0MAAIC/kg8LQYCAgPwDIAJrviEFIAFBFk0EfUMAAIA/IAWTIAAgA5OSBSAAIAMgBZKTQwAAgD+SCyAElCEACyAAC30BBH9BnMcAKAIAIwMoAhhGBEBBnMcAQQA2AgALA0BBlMcAKAIAIQJBkMcAQZDHACgCACIAIABBAWtBACAAQf////8HcSIBQQFHG0EAIAFB/////wdHGyID/kgCACAARw0ACwJAIAMNACACRSAAQQBOcQ0AQZDHACABEBELC00BAn8CQAJAA0BBBiEBAkBBkMcAKAIAIgBB/////wdxQf7///8Haw4CAwIACyAAQZDHACAAIABBAWr+SAIARw0AC0EADwtBCiEBCyABC8kDAQZ/IwBBEGsiBSQAAkAgACAAKAIARw0AAkACQCAAQSBqQQJBA/5IAgBBAmsOAgECAAsjAyEBIAVBDGoEQCAFIAEtACg2AgwLIAFBAToAKBADIwBBEGsiASQAAkAgACAAKAIARw0AAkAgACgCIEEDRg0AIAAjA0cNAAwBCyMDIQQgAUEMaiIDBEAgAyAELQAoNgIACyAEQQE6ACggASgCDEUEQCMDQQA6ACgLAkACQAJAIABBIGoiBigCACIDBEADQCACQRxGDQIgAkHJAEYNAiADQQNOBEAgASgCDCIAQQJNBH8jAyAAOgAoQQAFQRwLGgwGC0EBIQQgBiADECMhAiAGKAIAIgMNAAsMAgsgASgCDCICQQJNBH8jAyACOgAoQQAFQRwLGkEBIQQMAgtBACEECyABKAIMIgNBAk0EfyMDIAM6AChBAAVBHAsaIAJBHEYNASACQckARg0BCyMAQRBrQQA2AgwCQEGAwwAoAgAiAkUNAEGAwwBBiMMAIAIQFUGIwwAoAgBFDQBBgMMAQQEQEQsgBEUNACAAEAELIAFBEGokACAFKAIMIgBBAk0EfyMDIAA6AChBAAVBHAsaCwsgBUEQaiQAC+8GAQV/IwBBMGsiBCQAAkAgAEUEQEEcIQIMAQtBjMMAKAIARQRAQYzDAEErNgIAC0GpwQAtAABFBEACQEGrwQAsAAAiA0UNAEHswgBBAEGBgICAeP5IAgAhASADQQBIBEBBq8EAQQA6AAALIAFFDQADQEHswgAgAUH/////B2ogASABQQBIGyIDIANB/////wdr/kgCACIBIANGDQEgAkEBaiICQQpHDQALQezCAEEB/h4CAEEBaiEBA0AgAUEASARAQezCAEEAIAEQFSABQf////8HaiEBCyABQezCACABIAFBgICAgHhy/kgCACIBRw0ACwtB8MIAKAIAIgEEQANAAkAgAUUNACABKAJMQQBODQAgAUEANgJMCyABKAI4IgENAAsLAkBB7MIAKAIAQQBODQBB7MIAQf////8H/h4CAEGBgICAeEYNAEHswgBBARARCwJAQfzCACgCACIBRQ0AIAEoAkxBAE4NACABQQA2AkwLAkBBwD8oAgAiAUUNACABKAJMQQBODQAgAUEANgJMCwJAQfzCACgCACIBRQ0AIAEoAkxBAE4NACABQQA2AkwLQanBAEEBOgAACyAEQQRqQQBBLPwLACAEQaQ+KAIAIgE2AgRBqD4oAgBBjwFqIgJBACABQQ9qIAQoAgwbaiIDEBQiASACEEYgASADNgIwIAEgATYCLCABIAE2AgBBjMMAQYzDACgCACICQQFqNgIAIAEgAUHMAGo2AkwgASACNgIYIAFByMEANgJgIAFBA0ECIAQoAhAbNgIgIAEgBCgCBCIDNgI4IAEgAUGHAWpBfHEiAjYCdCACQQVqIQJBqD4oAgAEQCABIAJBA2pBfHEiAjYCSEGoPigCACACaiECCyABIAQoAgwiBSACIANqQQ9qQXBxIAUbNgI0IAEQQSMDIQIQLCACKAIMIQMgASACNgIIIAEgAzYCDCADIAE2AgggASgCCCABNgIMECtBrMEAQazBACgCACICQQFqNgIAIAJFBEBBq8EAQQE6AAALIAEgBEEEakEeQQAQBSICBEBBrMEAQazBACgCAEEBayIANgIAIABFBEBBq8EAQQA6AAALECwgASgCDCIAIAEoAggiAzYCCCADIAA2AgwgASABNgIMIAEgATYCCBArDAELIAAgATYCAAsgBEEwaiQAIAILOQEBf0GEwwAoAgAiAARAQYTDACAAQQFrNgIADwtBgMMAQQD+FwIAQYjDACgCAARAQYDDAEEBEBELC1gBAn8jAygCGCIAQYDDACgCAEcEQEGAwwBBACAA/kgCACIBBEADQEGAwwBBiMMAIAEQFUGAwwBBACAA/kgCACIBDQALCw8LQYTDAEGEwwAoAgBBAWo2AgALjwYBBn8jAEEgayICJAAgAkEANgIYIAJCADcDECACQgA3AwggACgCEBojBARAEAMLAkACQCABLQAAQQ9xRQ0AIwMoAhggASgCBEH/////B3FGDQAMAQsCfyAAKAIAIgcEQCAAKAIIIQMgAEEMakEB/h4CABogAEEIagwBCyAAQSBqIgMQIiACQQI2AhQgAkEANgIQIAIgACgCBCIENgIMIAAgAkEIaiIFNgIEAkAgACgCFEUEQCAAIAU2AhQMAQsgBCACQQhqNgIACyADECFBAiEDIAJBFGoLIQUgARAOIwMhBiACQQRqIgQEQCAEIAYtACg2AgALIAZBAjoAKCACKAIEQQFGBEAjA0EBOgAoCyAFIAMQIyEEAkAgBSgCACADRw0AA0AgBEEbR0EAIAQbDQEgBSADECMhBCAFKAIAIANGDQALCyAEQQAgBEEbRxshBAJ/AkACQCAHBEAgBEELRgRAQQtBACAAKAIIIANGGyEECyAAQQxqIgNBf/4eAgBBgYCAgHhHDQIMAQsgAkEQakEAQQL+SAIARQRAIABBIGoiAxAiAkAgACgCBCACQQhqRgRAIAAgAigCDDYCBAwBCyACKAIIIgVFDQAgBSACKAIMNgIECwJAIAAoAhQgAkEIakYEQCAAIAIoAgg2AhQMAQsgAigCDCIARQ0AIAAgAigCCDYCAAsgAxAhIAIoAhgiAEUNAiAAQX/+HgIAQQFHDQIgAigCGCEDDAELIAJBFGoQIiABEA8aAkAgAigCDA0AIAEtAABBCHENACABQQhqQQH+HgIAGgsCQCACKAIIIgMEQCABKAIEIgBBAEoEQCABQQRqIAAgAEGAgICAeHL+SAIAGgsgA0EMaiIAQQD+FwIAIABB/////wcQEQwBCyABLQAAQQhxDQAgAUEIakEB/iUCABoLIAIoAgQMAgsgA0EBEBELIAEQDyEAIAIoAgQiAUECTQR/IwMgAToAKEEABUEcCxogACAEIAAbQQtHDQFBAQsiAEECTQR/IwMgADoAKEEABUEcCxoLIAJBIGokAAtTAQN/IwBBEGsiAiQAIwMhAyACQQxqIgQEQCAEIAMtACg2AgALIANBAToAKCAAIAEQIyACKAIMIgFBAk0EfyMDIAE6AChBAAVBHAsaIAJBEGokAAuHAQEDfyMAQSBrIgEkAAJAIAAoAghFBEAgAEEQaiICEA8aIABBAjYCDCACEA4gAEEoahAlDAELIAAoAhgEQCAAKAIQIQIgACgCDCABIAA2AhwgASAANgIQIAFBITYCGCABQSY2AhQgASABKQIUNwMIIAIgAUEIahAbDQELIAAQFwsgAUEgaiQAC4IBAQJ/IAAoAghFBEAgAEEQahBEAkAgAEEoaiIAKAIARQ0AIAAoAgxFDQAgAEEMaiIBQYCAgIB4/jMCABogAEEIaiICQQH+HgIAGiACQf////8HEBEgACgCDCIAQf////8HcUUNAANAIAFBACAAEBUgASgCACIAQf////8HcQ0ACwsLC2cBAn8gACAAKAJYRgRAIABCADcCWEGgwQAoAgBBABAfDwsjAygCSEGgwQAoAgAiAUECdGooAgAiAiAARgRAIAEgAigCWBAfCyAAKAJcIgIgACgCWCIBNgJYIAEgAjYCXCAAQgA3AlgLsAkBDn8CQCAAIAEQMyICRQRAIAAoAhwiAiAAKAIgRgRAAn8gAkEBdEEBIAIbIg9BAnQhCSAAKAIYIgxFBEAgCRAUDAELIAlBQE8EQCMBQTA2AgBBAAwBCwJAQZTPAC0AAEECcQRAQZjPABAPDQELAn9BECAJQQtqQXhxIAlBC0kbIQZBACECIAxBCGsiBCgCBCIKQXhxIQgCQCAKQQNxRQRAIAZBgAJJDQEgBkEEaiAITQRAIAQhAiAIIAZrQcjLACgCAEEBdE0NAgtBAAwCCyAEIAhqIQcCQCAGIAhNBEAgCCAGayIDQRBJDQEgBCAGIApBAXFyQQJyNgIEIAQgBmoiAiADQQNyNgIEIAcgBygCBEEBcjYCBCACIAMQPwwBC0HwywAoAgAgB0YEQEHkywAoAgAgCGoiCCAGTQ0CIAQgBiAKQQFxckECcjYCBCAEIAZqIgMgCCAGayICQQFyNgIEQeTLACACNgIAQfDLACADNgIADAELQezLACgCACAHRgRAQeDLACgCACAIaiIDIAZJDQICQCADIAZrIgJBEE8EQCAEIAYgCkEBcXJBAnI2AgQgBCAGaiIIIAJBAXI2AgQgAyAEaiIDIAI2AgAgAyADKAIEQX5xNgIEDAELIAQgCkEBcSADckECcjYCBCADIARqIgIgAigCBEEBcjYCBEEAIQJBACEIC0HsywAgCDYCAEHgywAgAjYCAAwBCyAHKAIEIgNBAnENASADQXhxIAhqIg0gBkkNASANIAZrIQ4gBygCDCEFAkAgA0H/AU0EQCAHKAIIIgIgBUYEQEHYywBB2MsAKAIAQX4gA0EDdndxNgIADAILIAIgBTYCDCAFIAI2AggMAQsgBygCGCELAkAgBSAHRwRAIAcoAggiAiAFNgIMIAUgAjYCCAwBCwJAIAcoAhQiAgR/IAdBFGoFIAcoAhAiAkUNASAHQRBqCyEIA0AgCCEDIAIiBUEUaiEIIAIoAhQiAg0AIAVBEGohCCAFKAIQIgINAAsgA0EANgIADAELQQAhBQsgC0UNAAJAIAcoAhwiA0ECdCICKAKITiAHRgRAIAJBiM4AaiAFNgIAIAUNAUHcywBB3MsAKAIAQX4gA3dxNgIADAILAkAgByALKAIQRgRAIAsgBTYCEAwBCyALIAU2AhQLIAVFDQELIAUgCzYCGCAHKAIQIgIEQCAFIAI2AhAgAiAFNgIYCyAHKAIUIgJFDQAgBSACNgIUIAIgBTYCGAsgDkEPTQRAIAQgCkEBcSANckECcjYCBCAEIA1qIgIgAigCBEEBcjYCBAwBCyAEIAYgCkEBcXJBAnI2AgQgBCAGaiIDIA5BA3I2AgQgBCANaiICIAIoAgRBAXI2AgQgAyAOED8LIAQhAgsgAgshAkGUzwAtAABBAnEEQEGYzwAQDgsgAkEIaiACDQEaQQAgCRAUIgRFDQEaIAQgDEF8QXggDEEEaygCACICQQNxGyACQXhxaiICIAkgAiAJSRsQGCAMEBMLIAQLIgJFDQIgACAPNgIgIAAgAjYCGAsgARA4IgJFDQEgACAAKAIcIgFBAWo2AhwgACgCGCABQQJ0aiACNgIACyACDwtBAAtKAQN/AkAgACgCHCICQQBMDQAgACgCGCEDQQAhAANAIAEgAyAAQQJ0aigCACIEKAIcRwRAIAIgAEEBaiIARw0BDAILCyAEDwtBAAv8AQEFfwJAIAAoAiwgACgCMEEBaiAAKAIoIgJvRw0AIAJBGGwQFCIDBH8gAkEBdCEFAkAgACgCMCIEIAAoAiwiAk4EQCADIAAoAiQgAkEMbGogBCACayICQQxsEBgMAQsgAyAAKAIkIAJBDGxqIAAoAiggAmsiAkEMbCIGEBggAyAGaiAAKAIkIARBDGwQGCACIARqIQILIAAoAiQQEyAAIAI2AjAgAEEANgIsIAAgBTYCKCAAIAM2AiRBAQVBAAsNAEEADwsgACgCJCAAKAIwQQxsaiIDIAEpAgA3AgAgAyABKAIINgIIIAAgACgCMEEBaiAAKAIobzYCMEEBC2UBA38jAEEQayIBJAAgAEEEaiICEA8aIAAoAiwgACgCMEcEQANAIAFBBGogABA2IAEoAggiAwRAIAEoAgwgAxEAAAsgACgCLCAAKAIwRw0ACwsgAhAOIABBAP4XAgAgAUEQaiQACzgBAn8gACABKAIkIAEoAiwiAkEMbGoiAykCADcCACAAIAMoAgg2AgggASACQQFqIAEoAihvNgIsCxQAIABBBGoQRCAAKAIkEBMgABATC9cCAQZ/IwBBQGoiASQAQew9ECBFBEBBoD4oAgAiAkHoPUcEQANAIAIoAjggAv4QAgBFBEAgAigCNCIEIAIoAjgiBTYCOCAFIAQ2AjQgAhA3CyICQeg9Rw0ACwtB7D0QDgsCQEE8EBQiAkUNAEGADBAUIgNFBEAgAhATDAELIAFCADcDKCABQgA3AzAgAUEANgI8IAFCADcDICABIAA2AhwgAUEANgIYIAEgAzYCFCABQYABNgIQIAFBADYCDCABQQA2AgggAUEANgIEIAFBADYCACACIAEoAjw2AgAgAiABKQMwNwIUIAIgASkDKDcCDCACIAEpAyA3AgQgAiABKAIcNgIcIAIgASgCGDYCICACIAEoAhQ2AiQgAiABKAIQNgIoIAIgASgCDDYCLCACIAEoAgg2AjAgAiABKAIENgI0IAIgASgCADYCOCACIQYLIAFBQGskACAGC4ACAgJ/AnwgALwiAUGAgID8A0YEQEMAAAAADwsCQCABQYCAgPwHa0H///+HeE0EQCABQQF0IgJFBEAjAEEQayIBQwAAgL84AgwgASoCDEMAAAAAlQ8LIAFBgICA/AdGDQEgAkGAgIB4SSABQQBOcUUEQCAAIACTIgAgAJUPCyAAQwAAAEuUvEGAgIDcAGshAQtB0D0rAwAgASABQYCAzPkDayIBQYCAgHxxa767IAFBD3ZB8AFxIgIrA8g7okQAAAAAAADwv6AiAyADoiIEokHYPSsDACADokHgPSsDAKCgIASiIAFBF3W3Qcg9KwMAoiACKwPQO6AgA6CgtiEACyAAC7MEAwN8A38CfgJ8AkAgAL1CNIinQf8PcSIFQckHa0E/SQRAIAUhBAwBCyAFQckHSQRAIABEAAAAAAAA8D+gDwsgBUGJCEkNAEQAAAAAAAAAACAAvSIHQoCAgICAgIB4UQ0BGiAFQf8PTwRAIABEAAAAAAAA8D+gDwsgB0IAUwRAIwBBEGsiBEQAAAAAAAAAEDkDCCAEKwMIRAAAAAAAAAAQog8LIwBBEGsiBEQAAAAAAAAAcDkDCCAEKwMIRAAAAAAAAABwog8LIABBkCgrAwCiQZgoKwMAIgGgIgIgAaEiAUGoKCsDAKIgAUGgKCsDAKIgAKCgIgEgAaIiACAAoiABQcgoKwMAokHAKCsDAKCiIAAgAUG4KCsDAKJBsCgrAwCgoiACvSIHp0EEdEHwD3EiBSsDgCkgAaCgoCEBIAUpA4gpIAdCLYZ8IQggBEUEQAJ8IAdCgICAgAiDUARAIAhCgICAgICAgIg/fb8iACABoiAAoEQAAAAAAAAAf6IMAQsgCEKAgICAgICA8D98vyICIAGiIgEgAqAiA0QAAAAAAADwP2MEfCMAQRBrIgQgBEKAgICAgICACDcDCCAEKwMIRAAAAAAAABAAojkDCEQAAAAAAAAAACADRAAAAAAAAPA/oCIAIAEgAiADoaAgA0QAAAAAAADwPyAAoaCgoEQAAAAAAADwv6AiACAARAAAAAAAAAAAYRsFIAMLRAAAAAAAABAAogsPCyAIvyIAIAGiIACgCwv2BwMEfAJ/AX4gAL0iB0IgiKciBkH/////B3EiBUGAgMD/B08EQEQAAAAAAADwPyAAo0EBIAZBHnZBAnFrt6APCyAFQf//q/8DTQRAIAVB//+/8QNNBEAgAEQAAAAAAAAgQKIgAERp2xSCum7wP6KgRAAAAAAAAMA/og8LIAAgACAAoiIBIAEgASABRKwWABLW6vi+okTkaGYjkaJ3v6CiRE8Z19tRKp2/oKJEE7kcaX3N1L+gokRo2xSCum7AP6AgASABIAEgASABRCBhokJDnNC+okQQGhwiyV0hP6CiRA9r08Qi0HQ/oKJEus42VUylsD+gokQJ3NrNeXfZP6CiRAAAAAAAAPA/oKOiIACgDwtEAAAAAAAA8D8hASAFQf//34AETQRARAAAAAAAAPA/AnwgBUH//8//A00EQEQAAACA+tTDPyAAmUQAAAAAAADwv6AiACAAIAAgACAAIABEPweWCji/Yb+iROuVl1k2KqI/oKJE7Cg+PZhjvL+gokTkIFGAyl/UP6CiRPHDuPtA0te/oKJETbOSrQCN2j+gokQ4dfe+uFljv6AgACAAIAAgACAAIABEHRU1V1SLiD+iRBzdUWvC7Ys/oKJEHzVj52AmwD+gokSn6Z/ZXGOyP6CiRDNv65LwSuE/oKJEI+PuGGY+uz+gokQAAAAAAADwP6CjoQwBC0QAAAAAAADwPyAAmSIBIAGioyEAAnwgBUHstpuABE0EQCAAIAAgACAAIAAgACAARFzCmsbvoCPAokTy0uRXZVJUwKCiRLKrzOtcE2fAoKJEZiIohLFMZMCgokSNo8vkCjBPwKCiRCbnsEEEHiXAoKJEYHO65BY05r+gokQ1ZA1gEjSEv6AhAyAAIAAgACAAIAAgACAARGKadO7y766/okSTSkiO70caQKCiRCyuSO6jKFtAoKJEFANwVyHQekCgokRoKOwhGSuEQKCiRHEaitUNKXtAoKJEIedqUgw1YUCgokSHdnC9uaYzQKAMAQsgACAAIAAgACAAIABEPzjcm044fsCiRJJZLmphBJDAoKJEKPJ1E4jsg8CgokSY7cVDXRRkwKCiRFqZX1UJwjHAoKJE3oXCcLqT6b+gokRKb+g5EjSEv6AhAyAAIAAgACAAIAAgAERiLXFC4nA2wKJEY+ef53SofUCgokTmO9/OGfKjQKCiRGokjGi3/6hAoKJEGFGdGOsCmECgokQKnxsirlx0QKCiRJBRHSaLVj5AoAshBEQAAAAAAADivyABvUKAgICAcIO/IgIgAqKhEDogAiABoSABIAKgoiADIAAgBKJEAAAAAAAA8D+go6AQOqIgAaMLoSEBCyABmiABIAdCAFMbC8AHAhR/C3sCQCABIAJODQAgAyAETg0AIAIgAWsiCEEDcSENIAQgA2tBAnQhCSAAKAIUIgdBAnQhDCAAKAIIIAEgB2wgA2pBAnRqIQogASACa0F8TQRAIAhBfHEhB0EAIQgDQCAJRSILRQRAIAogBSAMbGpBACAJ/AsACyALRQRAIAogDCAFQQFybGpBACAJ/AsACyALRQRAIAogDCAFQQJybGpBACAJ/AsACyALRQRAIAogDCAFQQNybGpBACAJ/AsACyAFQQRqIQUgCEEEaiIIIAdHDQALCyANRQ0AA0AgCQRAIAogBSAMbGpBACAJ/AsACyAFQQFqIQUgBkEBaiIGIA1HDQALCwJAIAAoAiAiBUEATA0AIAQgA2siBkEIbUEDdCADaiEOIAIgAWsiCEEEbUECdCABaiETQQAhByAGQQhOBEAgCEEDSiEWA0AgBSAHQYACaiIIIAUgB2tBgAJIGyEPIAMhDSAWBEADQCABIQsDQCANQQJ0IgogACgCCCAAKAIUIgYgC2xBAnRqaiIQ/QAAECEaIBD9AAAAIRsgECAGQQJ0IgZqIhH9AAAQIRwgEf0AAAAhHSAGIBFqIhL9AAAQIR4gEv0AAAAhHyAGIBJqIhT9AAAQISAgFP0AAAAhISAHIA9IBEAgACgCACAAKAIMIgYgC2xBAnRqIhcgBkECdCIGaiIYIAZqIgkgBmohDCAKIAAoAgQgACgCECIKIAdsQQJ0amohBSAHIQYDQCAMIAZBAnQiFWr9CQIAIhkgBf0AABAiIv3mASAg/eQBISAgGSAF/QAAACIj/eYBICH95AEhISAJIBVq/QkCACIZICL95gEgHv3kASEeIBkgI/3mASAf/eQBIR8gFSAYav0JAgAiGSAi/eYBIBz95AEhHCAZICP95gEgHf3kASEdIBUgF2r9CQIAIhkgIv3mASAa/eQBIRogGSAj/eYBIBv95AEhGyAFIApBAnRqIQUgBkEBaiIGIA9HDQALCyAQIBr9CwAQIBAgG/0LAAAgESAc/QsAECARIB39CwAAIBIgHv0LABAgEiAf/QsAACAUICD9CwAQIBQgIf0LAAAgC0EEaiILIBNIDQALIA1BCGoiDSAOSA0ACwsgACATIAIgAyAOIAcgDxAcIAAgASACIA4gBCAHIA8QHCAAKAIgIgUgCCIHSg0ACwwBCwNAIAAgEyACIAMgDiAHIAUgB0GAAmoiCCAFIAdrQYACSBsiBhAcIAAgASACIA4gBCAHIAYQHCAAKAIgIgUgCCIHSg0ACwsLtwkDDH8NewN9IwBBQGohBiAALQABIgVBB3YhAiAALQADQQh0IgcgAC0AAnIhBCAFQQh0IgggAC0AAHJB/wdxIQMCfQJAIAVBAnZBH3EiBUEfRwRAIAUNASACQR90viADRQ0CGiADs0MAAIAzlCIbjCAbIAjBQQBIGwwCCyACQR90QYCAgAJBACADG3JBgICA/AdyvgwBCyAFQRd0IAJBH3RyIANBDXRyQYCAgMADar4LIR0gBEH/B3EhAyAEQYCAAnFBD3YhAgJ9AkAgBEEKdkEfcSIEQR9HBEAgBA0BIAJBH3S+IANFDQIaIAOzQwAAgDOUIhuMIBsgB8FBAEgbDAILIAJBH3RBgICAAkEAIAMbckGAgID8B3K+DAELIARBF3QgAkEfdHIgA0ENdHJBgICAwANqvgshGyAGIAD9XAAEIg79DD8/Pz8/Pz8/Pz8/Pz8/Pz/9Tv2JAf2pAf0LBCAgBiAA/VwACCIP/Qw/Pz8/Pz8/Pz8/Pz8/Pz8//U79iQH9qQH9CwQAIAYgAP1cAAwiEEEE/W0gD0EC/W39DDAwMDAwMDAwMDAwMDAwMDD9Tv1Q/YkB/akB/QsEECAGIBD9DA8PDw8PDw8PDw8PDw8PDw/9TiAOQQL9bf0MMDAwMDAwMDAwMDAwMDAwMP1O/VD9iQH9qQH9CwQwIABBMGohCCAAQRBqIQlBACEEA0AgASAEQQh0aiEKIAggBEEFdGohCyABIARBAXQiBUEBciIHQQd0aiEM/QwAAAAAAAAAAAAAAAAAAAAAIg4hD/0MAAAAAAAAAAAAAAAAAAAAACEQ/QwAAAAAAAAAAAAAAAAAAAAAIRFBASEDQQAhAANAIAwgAEECdCINaiIC/QAAMCIWIAAgCWr9AAAAIhIgB/1tQQT9a/0MEBAQEBAQEBAQEBAQEBAQEP1OIAAgC2r9AAAAIhNBBP1t/VAiFP2KASIV/aoB/foB/eYBIAL9AAAgIhcgFf2pAf36Af3mASAC/QAAECIVIBT9iQEiFP2qAf36Af3mASAC/QAAACIYIBT9qQH9+gH95gEgEP3kAf3kAf3kAf3kASEQIAogDWoiAP0AADAiFCASIAX9bUEE/Wv9DBAQEBAQEBAQEBAQEBAQEBD9TiAT/QwPDw8PDw8PDw8PDw8PDw8P/U79UCIS/YoBIhP9qgH9+gH95gEgAP0AACAiGSAT/akB/foB/eYBIAD9AAAQIhMgEv2JASIS/aoB/foB/eYBIAD9AAAAIhogEv2pAf36Af3mASAO/eQB/eQB/eQB/eQBIQ4gFiAXIBUgGCAR/eQB/eQB/eQB/eQBIREgFCAZIBMgGiAP/eQB/eQB/eQB/eQBIQ8gA0EQIQBBACEDDQALIBD9HwAgEP0fAZIgEP0fApIgEP0fA5IgB0ECdCIAIAZBIGoiAmooAgCylCAO/R8AIA79HwGSIA79HwKSIA79HwOSIAIgBUECdCIDaigCALKUkiAdlCAckiAR/R8AIBH9HwGSIBH9HwKSIBH9HwOSIBuUIAAgBmooAgCylCAP/R8AIA/9HwGSIA/9HwKSIA/9HwOSIBuUIAMgBmooAgCylJKTIRwgBEEBaiIEQQRHDQALIBwL1QcEBXsHfwJ9An4jAEFAaiEMIAAtAAEiCkEHdiELIAAtAANBCHQiDSAALQACciEIIApBCHQiDiAALQAAckH/B3EhCQJ9AkAgCkECdkEfcSIKQR9HBEAgCg0BIAtBH3S+IAlFDQIaIAmzQwAAgDOUIg+MIA8gDsFBAEgbDAILIAtBH3RBgICAAkEAIAkbckGAgID8B3K+DAELIApBF3QgC0EfdHIgCUENdHJBgICAwANqvgsgCEH/B3EhCSAIQYCAAnFBD3YhCwJ9AkAgCEEKdkEfcSIIQR9HBEAgCA0BIAtBH3S+IAlFDQIaIAmzQwAAgDOUIhCMIBAgDcFBAEgbDAILIAtBH3RBgICAAkEAIAkbckGAgID8B3K+DAELIAhBF3QgC0EfdHIgCUENdHJBgICAwANqvgshECAMIAD9XAAEIgP9DD8/Pz8/Pz8/Pz8/Pz8/Pz/9Tv2JAf2pAf0LBCAgDCAA/VwACCIE/Qw/Pz8/Pz8/Pz8/Pz8/Pz8//U79iQH9qQH9CwQAIAwgAP1cAAwiBUEE/W0gBEEC/W39DDAwMDAwMDAwMDAwMDAwMDD9Tv1Q/YkB/akB/QsEECAMIAX9DA8PDw8PDw8PDw8PDw8PDw/9TiADQQL9bf0MMDAwMDAwMDAwMDAwMDAwMP1O/VD9iQH9qQH9CwQwIABBEGohCUEAIQADQCAAQQN0IgsgDEEgaiINajQCACAJIABBBXRqIgj9AAAAIgb9DA8PDw8PDw8PDw8PDw8PDw/9TiID/YoBIAEgAEEGdGoiCv0AAAAiBP2IAf26ASAD/YkBIAT9hwH9ugH9rgEgCP0AABAiB/0MDw8PDw8PDw8PDw8PDw8PD/1OIgP9igEgCv0AABAiBf2IAf26Af2uASAD/YkBIAX9hwH9ugH9rgEiA/0bACAD/RsBaiAD/RsCaiAD/RsDaqx+IBF8IAZBBP1tIgP9igEgASAAQQF0QQFyIghBBXRqIgr9AAAAIgb9iAH9ugEgA/2JASAG/YcB/boB/a4BIAdBBP1tIgP9igEgCv0AABAiB/2IAf26Af2uASAD/YkBIAf9hwH9ugH9rgEiA/0bACAD/RsBaiAD/RsCaiAD/RsDaqwgDSAIQQJ0IghqNAIAfnwhESALIAxqNAIAIAX9fP1+IAT9fP1+/a4BIgP9GwAgA/0bAWogA/0bAmogA/0bA2qsfiASfCAIIAxqNAIAIAf9fP1+IAb9fP1+/a4BIgP9GwAgA/0bAWogA/0bAmogA/0bA2qsfnwhEiAAQQFqIgBBBEcNAAsgEbSUIBAgErSUkyAClAurCwEHfyAAIAFqIQUCQAJAIAAoAgQiAkEBcQ0AIAJBAnFFDQEgACgCACICIAFqIQECQAJAAkAgACACayIAQezLACgCAEcEQCAAKAIMIQMgAkH/AU0EQCADIAAoAggiBEcNAkHYywBB2MsAKAIAQX4gAkEDdndxNgIADAULIAAoAhghBiAAIANHBEAgACgCCCICIAM2AgwgAyACNgIIDAQLIAAoAhQiBAR/IABBFGoFIAAoAhAiBEUNAyAAQRBqCyECA0AgAiEHIAQiA0EUaiECIAMoAhQiBA0AIANBEGohAiADKAIQIgQNAAsgB0EANgIADAMLIAUoAgQiAkEDcUEDRw0DQeDLACABNgIAIAUgAkF+cTYCBCAAIAFBAXI2AgQgBSABNgIADwsgBCADNgIMIAMgBDYCCAwCC0EAIQMLIAZFDQACQCAAKAIcIgJBAnQiBCgCiE4gAEYEQCAEQYjOAGogAzYCACADDQFB3MsAQdzLACgCAEF+IAJ3cTYCAAwCCwJAIAAgBigCEEYEQCAGIAM2AhAMAQsgBiADNgIUCyADRQ0BCyADIAY2AhggACgCECICBEAgAyACNgIQIAIgAzYCGAsgACgCFCICRQ0AIAMgAjYCFCACIAM2AhgLAkACQAJAAkAgBSgCBCICQQJxRQRAQfDLACgCACAFRgRAQfDLACAANgIAQeTLAEHkywAoAgAgAWoiATYCACAAIAFBAXI2AgQgAEHsywAoAgBHDQZB4MsAQQA2AgBB7MsAQQA2AgAPC0HsywAoAgAiCCAFRgRAQezLACAANgIAQeDLAEHgywAoAgAgAWoiATYCACAAIAFBAXI2AgQgACABaiABNgIADwsgAkF4cSABaiEBIAUoAgwhAyACQf8BTQRAIAUoAggiBCADRgRAQdjLAEHYywAoAgBBfiACQQN2d3E2AgAMBQsgBCADNgIMIAMgBDYCCAwECyAFKAIYIQYgAyAFRwRAIAUoAggiAiADNgIMIAMgAjYCCAwDCyAFKAIUIgQEfyAFQRRqBSAFKAIQIgRFDQIgBUEQagshAgNAIAIhByAEIgNBFGohAiADKAIUIgQNACADQRBqIQIgAygCECIEDQALIAdBADYCAAwCCyAFIAJBfnE2AgQgACABQQFyNgIEIAAgAWogATYCAAwDC0EAIQMLIAZFDQACQCAFKAIcIgJBAnQiBCgCiE4gBUYEQCAEQYjOAGogAzYCACADDQFB3MsAQdzLACgCAEF+IAJ3cTYCAAwCCwJAIAUgBigCEEYEQCAGIAM2AhAMAQsgBiADNgIUCyADRQ0BCyADIAY2AhggBSgCECICBEAgAyACNgIQIAIgAzYCGAsgBSgCFCICRQ0AIAMgAjYCFCACIAM2AhgLIAAgAUEBcjYCBCAAIAFqIAE2AgAgACAIRw0AQeDLACABNgIADwsgAUH/AU0EQCABQfgBcUGAzABqIQICf0HYywAoAgAiA0EBIAFBA3Z0IgFxRQRAQdjLACABIANyNgIAIAIMAQsgAigCCAshASACIAA2AgggASAANgIMIAAgAjYCDCAAIAE2AggPC0EfIQMgAUH///8HTQRAIAFBJiABQQh2ZyICa3ZBAXEgAkEBdGtBPmohAwsgACADNgIcIABCADcCECADQQJ0QYjOAGohAgJAAkBB3MsAKAIAIgRBASADdCIHcUUEQEHcywAgBCAHcjYCACACIAA2AgAgACACNgIYDAELIAFBGSADQQF2a0EAIANBH0cbdCEDIAIoAgAhAgNAIAIiBCgCBEF4cSABRg0CIANBHXYhAiADQQF0IQMgBCACQQRxaiIHKAIQIgINAAsgByAANgIQIAAgBDYCGAsgACAANgIMIAAgADYCCA8LIAQoAggiASAANgIMIAQgADYCCCAAQQA2AhggACAENgIMIAAgATYCCAsLxwEBA38jAEEQayIBJABByM8AEA8aQcDLACgCAEUEQEHUywBBAjYCAEHMywBCfzcCAEHEywBCgKCAgICABDcCAEGUzwBBAjYCACABQQxqIgJBADYCACMAQSBrIgBCADcDGCAAQgA3AxAgAEIANwMIQZjPACAAKQMINwIAQajPACAAKQMYNwIAQaDPACAAKQMQNwIAIAIEQEGYzwAgAigCADYCAAtBwMsAIAFBCGpBcHFB2KrVqgVzNgIAC0HIzwAQDiABQRBqJAALHAAgACAAEDg2AnggAEEB/hcCfCAAQQD+FwKAAQunAQEBfwJAEChBCkcNAEHkACEAA0ACQCAARQ0AQZDHACgCAEUNACAAQQFrIQBBlMcAKAIARQ0BCwsQKEEKRw0AA0ACQEGQxwAoAgAiAEH/////B3FB/////wdHDQBBlMcAQQH+HgIAGkGQxwAgAEF//kgCABpBmMcAKAIAGkGQxwBBfxAuIQBBlMcAQQH+JQIAGiAARQ0AIABBG0cNAgsQKEEKRg0ACwsLtAEBA38CQCAAKAIAQQJGBEAjAEEQa0EANgIMDAELIwBBEGsiAiQAA0ACQAJAAkACQAJAIABBAEEB/kgCAA4EAAIBAwULIAJBBGoiAyAANgIEIAJBKzYCBCACIwMiBCgCRDYCDCAEIAM2AkQgARECACMDIAIoAgw2AkQgAEEC/kECAEEDRw0AIABB/////wcQEQsgAkEQaiQADAILIABBAUED/kgCABoLIABBAEEDEBUMAQsLCws0ACAAKAIAQYEBTgRAQfTCACgCACIABEADQEH0wgBB+MIAIAAQFUH0wgAoAgAiAA0ACwsLCykAAkBBAEF//h4C9EJBAUcNAEH4wgAoAgBFDQBB9MIAQf////8HEBELC9YCAQF/AkAgAUUNACAAQQA6AAAgACABaiICQQFrQQA6AAAgAUEDSQ0AIABBADoAAiAAQQA6AAEgAkEDa0EAOgAAIAJBAmtBADoAACABQQdJDQAgAEEAOgADIAJBBGtBADoAACABQQlJDQAgAEEAIABrQQNxIgJqIgBBADYCACAAIAEgAmtBfHEiAmoiAUEEa0EANgIAIAJBCUkNACAAQQA2AgggAEEANgIEIAFBCGtBADYCACABQQxrQQA2AgAgAkEZSQ0AIABBADYCGCAAQQA2AhQgAEEANgIQIABBADYCDCABQRBrQQA2AgAgAUEUa0EANgIAIAFBGGtBADYCACABQRxrQQA2AgAgAiAAQQRxQRhyIgJrIgFBIEkNACAAIAJqIQADQCAAQgA3AxggAEIANwMQIABCADcDCCAAQgA3AwAgAEEgaiEAIAFBIGsiAUEfSw0ACwsL8gEBBX8jAEEQayIDJAAgA0EANgIMIABBIGoiBhAiIAAoAhQiAkEARyEFAkAgAUUNACACRQ0AA0ACQCACQQhqQQBBAf5IAgAEQCADIAMoAgxBAWo2AgwgAiADQQxqNgIQDAELIAQgAiAEGyEEIAFBAWshAQsgAigCACICQQBHIQUgAUUNASACDQALCwJAIAUEQCACKAIEIgEEQCABQQA2AgALIAJBADYCBAwBCyAAQQA2AgQLIAAgAjYCFCAGECEgAygCDCICBEADQCADQQxqQQAgAhAVIAMoAgwiAg0ACwsgBARAIARBDGoQIQsgA0EQaiQAC2cBAX8CQCMFBEBB5MIA/hACAA0BIwEiAC0ABEUEQCAAQQE6AARB/MAAEA8aQfzAACMDEDMhAEH8wAAQDgJAIABFDQAgACgCIA0AIAAQJAsjAUEAOgAECwsPC0HkwgD+EAIAEAcQBgALOAAgACAAKAIAIAAoAgQgACgCCCAAKAIMIAAoAhAQCjkDGCAALQAgQQFGBEAgACgCEBATIAAQEwsLqgQCAX8BfCMAQTBrIgUkACAFIAE2AgwgBSAANgIIIAVBADYALCAFQQA2ACkgBUEAOgAoIAVCADcDICAFQQA2AhwgBSADNgIYIAUgAjYCFCAFIwM2AhACfCAEBEAjAEEQayIDJAAgAyAFQQhqNgIMIANBADYCCCADQSk2AgQjAEGAAWsiACQAIwBB4ABrIgEkAEGkwQBBJRBDIAFBAEHQAPwLACABQSg2AlwgASADQQRqNgJYIAFBADYCVCABQQA2AlAgAEEgaiICIAEoAlw2AgAgAiABKAJYNgIEIAIgASgCVDYCCCACIAEoAlA2AgwgAkEQaiABQdAA/AoAACABQeAAaiQAIABBIzYCGCAAQSQ2AhQgACACNgIcIAAgAjYCECAAIAApAhQ3AwhB/MAAQeDBACAAQQhqEBsEfyAAQTBqIgEQDxogACgCLEUEQCAAQcgAaiECA0AgAiABEC0gACgCLEUNAAsLIAEQDiAAKAIsQQFGBUEACyEBIABBIGoQMCAAQYABaiQAIANBEGokACAFKwMgRAAAAAAAAAAAIAEbDAELQSgQFCIBIAVBCGpBKPwKAAAgAUEBOgAgIAEgAkEDdCIAEBQiAjYCECACIAMgABAYIwBBIGsiACQAIAAgATYCHCAAIAE2AhAgAEEANgIYIABBKTYCFCAAIAApAhQ3AwhB/MAAQeDBACAAQQhqEBsaIABBIGokAEQAAAAAAAAAAAsgBUEwaiQAC5QBAQJ/IAEoAgggASgCABEAACMAQSBrIgEkAAJAIAAoAghFBEAgAEEQaiICEA8aIABBATYCDCAAEDEgAhAOIABBKGoQJQwBCyAAEDEgACgCECECIAAoAgwgASAANgIcIAEgADYCECABQSE2AhggAUEiNgIUIAEgASkCFDcDCCACIAFBCGoQGw0AIAAQFwsgAUEgaiQACxMAIAAoAgQgACgCGBEAACAAEBcLiwMBA38jAyIAKAJIRQRAIABBkMMANgJICwJAQZzHACgCACMDKAIYRg0AQZDHAEEAQf////8H/kgCAAR/QQoFQZzHACMDKAIYNgIAQQALQQpHDQBB5AAhAANAAkAgAEUNAEGQxwAoAgBFDQAgAEEBayEAQZTHACgCAEUNAQsLQZDHAEEAQf////8H/kgCAAR/QQoFQZzHACMDKAIYNgIAQQALQQpGBEADQAJAQZDHACgCACIARQ0AQZTHAEEB/h4CABpBkMcAIAAgAEGAgICAeHIiAP5IAgAaQZjHACgCABpBkMcAIAAQLiEAQZTHAEEB/iUCABogAEUNACAAQRtHDQMLQZDHAEEAQf////8H/kgCAAR/QQoFQZzHACMDKAIYNgIAQQALQQpGDQALC0GcxwAjAygCGDYCAAtBsMcAKAIAIgEhAANAAkAgAEECdCICKALAR0UEQEGgwQAgADYCAEGwxwAgADYCACACQcDHAGpBJzYCAAwBCyAAQQFqQf8AcSIAIAFHDQELCxAnC2wBAX8CQCMDKAJIQaDBACgCAEECdGooAgAiAUUEQCAAIAA2AlggACAANgJcQaDBACgCACAAEB8MAQsgACABNgJYIAAgASgCXDYCXCABIAA2AlwgACgCXCAANgJYCyAAIAAoAgQgACgCABEEAAsTACAAKAIEIAAoAhQRAAAgABAXC7wLAxJ/AnsBfSMAQYAQayIMJAACQCAAKAIYIgZBAEwNACABIAJODQAgACgCCCABQQJ0aiEFIAIgAWtBAnQhBCAAKAIUQQJ0IQggBkEETwRAIAZB/P///wdxIQsDQCAERSIHRQRAIAUgAyAIbGpBACAE/AsACyAHRQRAIAUgCCADQQFybGpBACAE/AsACyAHRQRAIAUgCCADQQJybGpBACAE/AsACyAHRQRAIAUgCCADQQNybGpBACAE/AsACyADQQRqIQMgCUEEaiIJIAtHDQALCyAGQQNxIgdFDQADQCAEBEAgBSADIAhsakEAIAT8CwALIANBAWohAyAKQQFqIgogB0cNAAsLAkAgACgCICILQQBMDQAgBkEBRgRAIAFBAXQhDSAAKAIIIAFBAnRqIQcgACgCACEOQQAhCCACIAFrIgpBCEghDwNAIAAoAgQgACgCECAIbGogDWohCSAOIAhBAnRqKgIAIRdBACEFIA9FBEAgF/0TIRVBACEEQQghBgNAIAcgBEECdGoiAyAVIAkgBEEBdGr9AAAAIhYgFf0NAAEAAQIDAAEEBQABBgcAAUEQ/asB/eYBIAP9AAAA/eQB/QsAACADIBUgFiAV/Q0ICQABCgsAAQwNAAEODwABQRD9qwH95gEgA/0AABD95AH9CwAQIAYiBSIEQQhqIgYgCkwNAAsLAkAgBSAKTg0AAkAgAiABIAVqayIGQQRJBEAgBSEDDAELIAUgBkF8cSILaiEDIBf9EyEVQQAhBANAIAcgBCAFaiIQQQJ0aiIRIBUgCSAQQQF0av0EAABBEP2rAf3mASAR/QACAP3kAf0LAgAgBEEEaiIEIAtHDQALIAYgC0YNAQsDQCAHIANBAnRqIgQgFyAJIANBAXRqLwAAQRB0vpQgBCoCAJI4AgAgA0EBaiIDIApHDQALCyAIQQFqIgggACgCIEgNAAsMAQsgASACTg0AIAIgAWshEANAIAtBAEoEQEGABCACIAFrIg4gDkGABE4bIQdBgAQgECANQQl0ayIDIANBgAROGyIPQQNxIQogAUEBdCERQQAhCCAOQQdKIRQDQCAAKAIEIAAoAhAgCGxqIBFqIQlBCCEEQQAhAyAUBEADQCAMIANBAnRqIgUgCSADQQF0av0AAAAiFSAV/Q0ICQABCgsAAQwNAAEODwABQRD9qwH9CwQQIAUgFSAV/Q0AAQABAgMAAQQFAAEGBwABQRD9qwH9CwQAIAQiA0EIaiIEIAdMDQALCwJAIAMgB04NAAJAIA8gA2siBEEESQRAIAMhBAwBCyADIAQgCmsiEmohBEEAIQUDQCAMIAMgBWoiE0ECdGogCSATQQF0av0EAABBEP2rAf0LAgAgBUEEaiIFIBJHDQALIApFDQELA0AgDCAEQQJ0aiAJIARBAXRqLwAAQRB0NgIAIARBAWoiBCAHRw0ACwtBACEJIAZBAEoEQANAIAAoAgggACgCFCAJbEECdGogAUECdGohBiAAKAIAIAAoAgwgCWxBAnRqIAhBAnRqKgIAIRdBACEDIA5BBE4EQCAX/RMhFUEAIQRBBCEFA0AgBiAEQQJ0IgNqIgQgAyAMav0ABAAgFf3mASAE/QAAAP3kAf0LAAAgBSIDIQQgA0EEaiIFIAdMDQALCwJAIAMgB04NAAJAIA8gA2siBEEESQRAIAMhBAwBCyADIAQgCmsiC2ohBCAX/RMhFUEAIQUDQCAGIAMgBWpBAnQiEmoiEyAT/QACACAMIBJq/QACACAV/eYB/eQB/QsCACAFQQRqIgUgC0cNAAsgCkUNAQsDQCAGIARBAnQiA2oiBSAFKgIAIAMgDGoqAgAgF5SSOAIAIARBAWoiBCAHRw0ACwsgCUEBaiIJIAAoAhgiBkgNAAsgACgCICELCyAIQQFqIgggC0gNAAsLIA1BAWohDSABQYAEaiIBIAJIDQALCyAMQYAQaiQACxkAIABBAf4XAgAgABAkIABBAUEA/kgCABoLBgAgABA1C4YBAQF/IwBBMGsiDCQAAkAgCUEATA0AIApBAEwNACAMIAo2AiggDCAJNgIkIAwgCDYCICAMIAU2AhwgDCACNgIYIAwgAyAEajYCECAMIAtBACALQQBKGzYCLCAMIAYgB0ECdGo2AhQgDCAAIAFBAnRqNgIMQQMgDEEMaiAKEBALIAxBMGokAAu4BgMSfwF7AX0CQCAAKAIMIgpBAEwNACABIAJODQAgACgCECELIAAoAgghDCAAKAIAIQ0gACgCBCEOIAAoAhQiA0EETgRAIANB/P///wdxIQUgA0EEayIPQQRxIREgD0ECdkEBakH+////B3EhEgNAIAwgBiALbEECdGohEyANIAMgBmxBAnRqIQcgASEEA0AgDiADIARsQQJ0aiEJQQQhAP0MAAAAAAAAAAAAAAAAAAAAACEVQQAhCEEAIRAgD0EETwRAA0AgCSAAQQJ0IhRq/QAAACAHIBRq/QAAAP3mASAJIAhBAnQiCGr9AAAAIAcgCGr9AAAA/eYBIBX95AH95AEhFSAAQQRqIQggAEEIaiEAIBBBAmoiECASRw0ACwsgEUUEQCAJIAhBAnQiAGr9AAAAIAAgB2r9AAAA/eYBIBX95AEhFQsgFf0fACAV/R8BkiAV/R8CkiAV/R8DkiEWIAMgBSIASgRAA0AgCSAAQQJ0IghqKgIAIAcgCGoqAgCUIBaSIRYgAEEBaiIAIANHDQALCyATIARBAnRqIBY4AgAgBEEBaiIEIAJHDQALIAZBAWoiBiAKRw0ACwwBCyADQQBKBEAgA0EBRiEHA0AgDCAEIAtsQQJ0aiEJIA0gAyAEbEECdGohBSABIQADQCAJIABBAnRqAn0gDiAAIANsQQJ0aiIGKgIAIAUqAgCUIhYgBw0AGiAGKgIEIAUqAgSUIBaSIhYgA0ECRg0AGiAGKgIIIAUqAgiUIBaSCzgCACAAQQFqIgAgAkcNAAsgBEEBaiIEIApHDQALDAELIAtBAnQhBSAMIAFBAnRqIQMgAiABa0ECdCEBQQAhACAKQQRPBEAgCkH8////B3EhBwNAIAFFIgJFBEAgAyAAIAVsakEAIAH8CwALIAJFBEAgAyAFIABBAXJsakEAIAH8CwALIAJFBEAgAyAFIABBAnJsakEAIAH8CwALIAJFBEAgAyAFIABBA3JsakEAIAH8CwALIABBBGohACAGQQRqIgYgB0cNAAsLIApBA3EiAkUNAANAIAEEQCADIAAgBWxqQQAgAfwLAAsgAEEBaiEAIARBAWoiBCACRw0ACwsLTgEDfyMBIQAjAyICKAJ0IgEEQCACQQA2AnQgASQBIAFBAEEF/AsAIAEPCyMCQQEgABsEQEEBJAJBBRAUIQALIAAkASAAQQBBBfwLACAAC90CAQd/A0BBACEAAkADQEHgwAD+EAIAIAFHDQFB4MAA/hACACABRw0BQeDAAP4QAgAgAUcNAUHgwAD+EAIAIAFHDQEgAEEEaiIAQYCAwABHDQALQdQ/EA8aQQBBAf4eAuhAGiABQeDAAP4QAgBGBEADQEHsP0HUPxAtQeDAAP4QAgAgAUYNAAsLQQBBAf4lAuhAGkHUPxAOC0HgwAD+EAIAIQFB2MAAKAIAIQJB0MAAKAIAIQVBzMAAKAIAIQZB1MAAKAIAIQMgA0HcwAD+EAIAIgBKBEAgAkEBdCEHA0AgAyAAQQAgAEHAACADIABrIAdtIgIgAkHAAEwbIABqIgT+SALcQCICRgR/IAUgACAEIAMgAyAEShsgBhEBAEHcwAD+EAIABSACCyIASg0ACwtBAEEB/iUC5EBBAUcNAEHUPxAPGkHswAD+EAIABEBBnMAAECULQdQ/EA4MAAsAC3YBAn8jAEEQayIAJAACQCAAQQxqIgEQKg0AIAAoAgwQKUH0wABB9MAAKAIAQQFqNgIAIAEQKg0AIAAoAgwQKUH0wABB9MAAKAIAQQFqNgIAIAEQKg0AIAAoAgwQKUH0wABB9MAAKAIAQQFqNgIACyAAQRBqJAALigcCGH8BfQJAIAEgAk4NACAAKAIIIgVBAEwNACAAKAIUIAAoAhBsIhMgACgCDGwhFCAAKAJMIAAoAkhsIREgACgCICIGQQJ0IQkgBkH+////B3EhFSAGQQFxIRYDQCABIAEgEW0iByARbGsiCCAAKAJMIgptIQQCQCAFQQBMDQAgACgCGCIDQQBMDQAgACgCJCAHbCAAKAIwayEXIAAoAiggBGwgACgCNGshGCAAKAIsIAggBCAKbGtsIAAoAjhrIgdBH3VBACAHa3EiCEECdCEKIAAoAgQgACgCUCABbEECdGohBEEAIQ0DQCADQQBKBEAgACgCACANIBRsQQJ0aiEZQQAhDgNAIAAoAhwhAwJAAkAgACgCPCAObCAXaiIFQQBOBEAgA0EATA0CIAUgACgCDE4NASAZIAUgE2xBAnRqIRpBACEPA0ACQAJAAkAgACgCQCAPbCAYaiIDQQBOBEAgAyAAKAIQSA0BCyAJRQ0BIARBACAJ/AsADAELIBogACgCFCIFIANsQQJ0aiEMIAAoAkQiEEEBRwRAIAZBAEwNAkEAIQNBACESIAZBAUcEQANAQwAAAAAhGyAEAn1DAAAAACADIBBsIAdqIgtBAEgNABpDAAAAACAFIAtMDQAaIAwgC0ECdGoqAgALOAIAAkAgA0EBciAQbCAHaiILQQBIDQAgBSALTA0AIAwgC0ECdGoqAgAhGwsgBCAbOAIEIANBAmohAyAEQQhqIQQgEkECaiISIBVHDQALCyAWRQ0CQwAAAAAhGwJAIAMgEGwgB2oiA0EASA0AIAMgBU4NACAMIANBAnRqKgIAIRsLIAQgGzgCACAEQQRqIQQMAgsgBSAHayIDIAYgAyAGSBsiBSAISiEDAkAgCEEATA0AIApFDQAgBEEAIAr8CwALIAUgCCADGyEDAkAgBSAITA0AIAMgCGtBAnQiBUUNACAEIApqIAwgB0ECdGogCmogBfwKAAALIAMgBk4NACAGIANrQQJ0IgVFDQAgBCADQQJ0akEAIAX8CwALIAQgCWohBAsgD0EBaiIPIAAoAhxIDQALDAILIANBAEwNAQtBACEDA0AgCQRAIARBACAJ/AsACyAEIAlqIQQgA0EBaiIDIAAoAhxIDQALCyAOQQFqIg4gACgCGCIDSA0ACyAAKAIIIQULIA1BAWoiDSAFSA0ACwsgAUEBaiIBIAJHDQALCwv9BgMWfwJ7AX0jAEHgAGsiByQAAkAgBSgCACISQQBMDQAgBSgCCCIMQQBMDQAgBSgCDCIOQQBMDQAgBSgCVCIPIAUoAlAiECAFKAJMbGwiCEEATA0AIAUoAgQhBiAFKAIQIQkgBSgCFCELIAcgBSgCGCIRNgIgIAcgCzYCHCAHIAk2AhggByAENgIQIAcgBiAObSINNgIUIAUoAhwhCiAHIAX9AAIcIhz9CwIkIAcgBf0AAiz9CwI0IAX9AAI8IR0gByAKIA1sIBz9GwFsIBz9GwJsIgo2AlwgByAPNgJYIAcgEDYCVCAHIB39CwJEIBEgCSALbGwiBSANbCERIAwgDk8iFyAKQQBKcSEUIAggDGwhDyAMIA5uIQkgBSAGbCEVIAIEQCAIQfz///8HcSEMIAhBBEkhGEEAIQsDQCAAIAsgFWxBAnRqIRkgAyALIA9sQQJ0aiEaQQAhDQNAIBogCSANbCIFIAhsQQJ0aiEQIAEgBSAKbEECdGohBgJAAkAgFARAIAcgGSANIBFsQQJ0ajYCDEEcIAdBDGogCBAQIAZBACAEQQAgEEEAIAkgCCAKEBkMAQsgBkEAIARBACAQQQAgCSAIIAoQGSAXRQ0BCyACIAVBAnRqIRtBACEGA0AgECAGIAhsQQJ0aiEWIBsgBkECdGoqAgAhHkEAIQUCQCAYRQRAIB79EyEcA0AgFiAFQQJ0aiITIBP9AAIAIBz95AH9CwIAIAVBBGoiBSAMRw0ACyAMIgUgCEYNAQsDQCAWIAVBAnRqIhMgEyoCACAekjgCACAFQQFqIgUgCEcNAAsLIAZBAWoiBiAJSA0ACwsgDUEBaiINIA5HDQALIAtBAWoiCyASRw0ACwwBC0EAIQYgFARAA0AgACAGIBVsQQJ0aiECIAMgBiAPbEECdGohDEEAIQUDQCAHIAIgBSARbEECdGo2AgxBHCAHQQxqIAgQECABIAUgCWwiCyAKbEECdGpBACAEQQAgDCAIIAtsQQJ0akEAIAkgCCAKEBkgBUEBaiIFIA5HDQALIAZBAWoiBiASRw0ADAILAAsDQCADIAYgD2xBAnRqIQBBACEFA0AgASAFIAlsIgIgCmxBAnRqQQAgBEEAIAAgAiAIbEECdGpBACAJIAggChAZIAVBAWoiBSAORw0ACyAGQQFqIgYgEkcNAAsLIAdB4ABqJAALoRcDHH8EewN9IAAoAhxBAnQQFCIJBEAgASACSARAIAAoAhggACgCEGwhFgNAIAAoAhAiEyABIBZtIhBsIAEgECAWbGsiBCAAKAIYIgNtIhJqIANsIQYgBCADIBJsayIXIAAoAhwiBSADa2ohByASIBMgACgCFCIIbW0hGAJ/IAAoAiwiC0EATARAIAdBAWohBCAFIQNBAAwBCyAHQQFqIgQgACgCMGoiAyAFIAMgBUgbIQNBfyAHIAtrIgcgB0EASBtBAWoLIREgACgCICIKIAVsIAAoAgwhGyAAKAIAIQ4gACgCCCEcIAAoAgQhDSAEIAMgAyAEShsgAyAAKAIoGyEMIAogBiAXamwhDyAIIBBsIBhqbCAAKAI0IgMEQCADIBBBAnRqKAIAIgMgDCADIAxIGyEMCwJAIBEgDEEAIAxBAEobIgcgByARSxsiCEUgBSAHTHENAAJAIAhFDQBBACEDIAhBBE8EQCAIQfz///8HcSEDQQAhBANAIAkgBEECdGr9DMrySfHK8knxyvJJ8crySfH9CwIAIARBBGoiBCADRw0ACyADIAhGDQELA0AgCSADQQJ0akHK5aeKfzYCACADQQFqIgMgCEcNAAsLIAUgB0wNACAFIAciA2siBkEETwRAIAkgA0ECdGohFSADIAZBfHEiC2ohA0EAIQQDQCAVIARBAnRq/QzK8knxyvJJ8crySfHK8knx/QsCACAEQQRqIgQgC0cNAAsgBiALRg0BCwNAIAkgA0ECdGpByuWnin82AgAgA0EBaiIDIAVHDQALCyAPQQJ0IRVBAnQhFCAHIBFLBEAgDiAVaiEPIA0gFGohHSAKQfD///8HcSEOIAAqAiQhJCAIIQsDQCAdIAogC2xBAnRqIQ1BECEGQQAhA/0MAAAAAAAAAAAAAAAAAAAAACIfISD9DAAAAAAAAAAAAAAAAAAAAAAhIf0MAAAAAAAAAAAAAAAAAAAAACEiIApBEE4EQANAIA0gA0ECdCIEaiID/QAAMCAEIA9qIgT9AAAw/eYBIB/95AEhHyAD/QAAICAE/QAAIP3mASAg/eQBISAgA/0AABAgBP0AABD95gEgIf3kASEhIAP9AAAAIAT9AAAA/eYBICL95AEhIiAGIgNBEGoiBiAKTA0ACyAhICL95AEgIP3kASAf/eQBIR8gDiEDCyAKIANBBHIiBE4EQANAIA0gA0ECdCIDav0AAAAgAyAPav0AAAD95gEgH/3kASEfIAQiA0EEaiIEIApMDQALCyAf/R8AIB/9HwGSIB/9HwKSIB/9HwOSISMCQCADIApODQACQCAKIANrIhlBBEkEQCADIQQMAQv9DAAAAAAAAAAAAAAAAAAAAAAgI/0gACEfIAMgGUF8cSIaaiEEQQAhBgNAIA0gAyAGakECdCIeav0AAgAgDyAeav0AAgD95gEgH/3kASEfIAZBBGoiBiAaRw0ACyAfIB8gH/0NBAUGBwABAgMMDQ4PAAECA/3kASIfIB8gH/0NCAkKCwABAgMAAQIDAAECA/3kAf0fACEjIBkgGkYNAQsDQCANIARBAnQiA2oqAgAgAyAPaioCAJQgI5IhIyAEQQFqIgQgCkcNAAsLIAkgC0ECdGogIyAklDgCACAMIAtBAWoiC0cNAAsLAn0gACgCOCIEBH8gBCAAKAJIIgYgACgCRCIHIBIgGCAAKAJAIgMgE0YbQQAgA0EBRxsgEEEAIAAoAjxBAUcbIANsamwgF0EAIAdBAUcbamxBAnRqIQcCQAJAAkAgBkEBRwRAIAVBAEwNAkEAIQMgBUEESQ0BIAVB/P///wdxIQNBACEEA0AgCSAEQQJ0IgZqIgggCP0AAgAgBiAHav0AAgD95AH9CwIAIARBBGoiBCADRw0ACyADIAVHDQEMAwsgBUEATA0BIAcqAgAhI0EAIQMgBUEETwRAIAVB/P///wdxIQMgI/0TIR9BACEEA0AgCSAEQQJ0aiIHIAf9AAIAIB/95AH9CwIAIARBBGoiBCADRw0ACyADIAVGDQMLA0AgCSADQQJ0aiIEIAQqAgAgI5I4AgAgA0EBaiIDIAVHDQALDAILA0AgCSADQQJ0IgRqIgYgBioCACAEIAdqKgIAkjgCACADQQFqIgMgBUcNAAsMAQtBASEDQQAhDCAFIQcgCSoCAAwCC0EAIQggBQUgBwsgBSAMIBFKIgMbIQcgCEEAIAMbIQwgBUEESARAQQEhAyAJKgIADAELQQghBCAJ/QAAACEfQQQhAyAFQQhPBEADQCAfIAkgA0ECdGr9AAAA/ekBIR8gBCIDQQRqIgQgBUwNAAsgBUH8////B3EhAwsgH/0fACAf/R8BlyAf/R8CIB/9HwOXlwshIwJAAkAgAyAFSARAIAUgA2siBkEETwRAIAkgA0ECdGohCyADIAZBfHEiCGohAyAj/RMhH0EAIQQDQCALIARBAnRq/QACACAf/esBIR8gBEEEaiIEIAhHDQALIB8gHyAf/Q0ICQoLDA0ODwABAgMAAQID/esBIh8gHyAf/Q0EBQYHAAECAwABAgMAAQID/esB/R8AISMgBiAIRg0CCwNAIAkgA0ECdGoqAgAiJCAjICMgJF0bISMgA0EBaiIDIAVHDQALDAELIAVBAEwNAQsCQAJAIAVBBEkiBgRAQQAhA0MAAAAAISQMAQsgBUF8cSEDICP9EyEh/QwAAAAAAAAAAAAAAAAAAAAAISBBACEEA0AgCSAEQQJ0aiIIIAj9AAIAICH95QEiH/0fABAN/RMgH/0fARAN/SABIB/9HwIQDf0gAiAf/R8DEA39IAMiH/0LAgAgHyAg/eQBISAgBEEEaiIEIANHDQALICAgICAf/Q0EBQYHAAECAwwNDg8AAQID/eQBIh8gHyAf/Q0ICQoLAAECAwABAgMAAQID/eQB/R8AISQgAyAFRg0BCwNAIAkgA0ECdGoiBCAEKgIAICOTEA0iJTgCACAlICSSISQgA0EBaiIDIAVHDQALCyAkQwAAAABeRQ0AQwAAgD8gJJUhI0EAIQMgBkUEQCAj/RMhH0EAIQRBBCEGA0AgCSAEQQJ0aiIDIAP9AAAAIB/95gH9CwAAIAYiAyEEIANBBGoiBiAFTA0ACwsgAyAFTw0AIAUgA2siBEEETwRAIAkgA0ECdGohBiADIAQgBUEDcSIIayILaiEDICP9EyEfQQAhBANAIAYgBEECdGoiDiAO/QACACAf/eYB/QsCACAEQQRqIgQgC0cNAAsgCEUNAQsDQCAJIANBAnRqIgQgBCoCACAjlDgCACADQQFqIgMgBUcNAAsLIBUgG2ohBQJAIApBAEwNACAKQQJ0IgNFDQAgBUEAIAP8CwALIAcgDEoEQCAUIBxqIREgCkEDcSELIApBBGsiDkEEcSEQIA5BAnZBAWpB/v///wdxIQ8DQCARIAogDGxBAnRqIQggCSAMQQJ0aioCACEjQQAhBAJAIApBBEgNACAj/RMhH0EEIQNBACEGIA5BA0sEQANAIAUgBEECdCIEaiINIAQgCGr9AAAAIB/95gEgDf0AAAD95AH9CwAAIAUgA0ECdCIEaiINIAQgCGr9AAAAIB/95gEgDf0AAAD95AH9CwAAIANBBGohBCADQQhqIQMgBkECaiIGIA9HDQALCyAQDQAgBSAEQQJ0IgRqIgYgBCAIav0AAAAgH/3mASAG/QAAAP3kAf0LAAAgAyEECwJAIAQgCk4NAAJAIAogBGsiA0EESQRAIAQhAwwBCyAEIAMgC2siDWohAyAj/RMhH0EAIQYDQCAFIAQgBmpBAnQiEmoiEyAT/QACACAIIBJq/QACACAf/eYB/eQB/QsCACAGQQRqIgYgDUcNAAsgC0UNAQsDQCAFIANBAnQiBGoiBiAGKgIAIAQgCGoqAgAgI5SSOAIAIANBAWoiAyAKRw0ACwsgDEEBaiIMIAdHDQALCyABQQFqIgEgAkcNAAsLIAkQEwsL+AEBAX8jAEHQAGsiFCQAAkAgBEEATA0AIAVBAEwNACAGQQBMDQAgB0EATA0AIAlBAEwNACAIQQBMBEAgByAJbCAFbCAEbEECdCIARQ0BIANBACAA/AsADAELIBQgEzYCTCAUIBI2AkggFCARNgJEIBQgEDYCQCAUIA82AjwgFCAONgI4IBQgDTYCNCAUIAw2AjAgFCALNgIsIBQgCjgCKCAUIAk2AiQgFCAINgIgIBQgBjYCGCAUIAM2AhAgFCACNgIMIBQgATYCCCAUIAA2AgQgFCAFNgIUIBQgBzYCHEEbIBRBBGogBCAFbCAHbBAQCyAUQdAAaiQAC2EBAX8jAEEgayIGJAACQCADQQBMDQAgBEEATA0AIAZBATYCHCAGIAU4AhggBiAENgIUIAYgAjYCECAGQQA2AgwgBiABNgIIIAYgADYCBEEaIAZBBGogAxAQCyAGQSBqJAALhh0DEn8DewR9IAEgAkgEQEMAAIA/IAAoAhAiCbKVIRogCUEDcSELIAlB/P///wdxIQ4gCUEEayINQQRxIRAgDUECdkEBaiIDQf7///8HcSERIANB/P///wdxIRIgA0EDcSEPIA1BDEkhEwNAIAEgCWxBAnQiAyAAKAIMaiEKIAAoAgAgA2ohBwJAIAAoAhgEQAJ/IAlBBEgiDARA/QwAAAAAAAAAAAAAAAAAAAAAIRVBAAwBC/0MAAAAAAAAAAAAAAAAAAAAACEVQQAhBUEEIQNBACEGQQAhBCATRQRAA0AgByADQQJ0aiII/QAAICIWIBb95gEgCP0AABAiFiAW/eYBIAj9AAAAIhYgFv3mASAHIAZBAnRq/QAAACIWIBb95gEgFf3kAf3kAf3kAf3kASEVIANBDGohBiADQRBqIQMgBEEEaiIEIBJHDQALCyAOIA9FDQAaA0AgAyIEQQRqIQMgByAGQQJ0av0AAAAiFiAW/eYBIBX95AEhFSAEIQYgBUEBaiIFIA9HDQALIA4LIQQgFf0fACAV/R8BkiAV/R8CkiAV/R8DkiEYAkAgBCAJTg0AIAkgBGsiA0EETwRA/QwAAAAAAAAAAAAAAAAAAAAAIBj9IAAhFSAHIARBAnRqIQYgBCADIAtrIgVqIQRBACEDA0AgBiADQQJ0av0AAgAiFiAW/eYBIBX95AEhFSADQQRqIgMgBUcNAAsgFSAVIBX9DQQFBgcAAQIDDA0ODwABAgP95AEiFSAVIBX9DQgJCgsAAQIDAAECAwABAgP95AH9HwAhGCALRQ0BCwNAIAcgBEECdGoqAgAiGSAZlCAYkiEYIARBAWoiBCAJRw0ACwtDAACAPyAYIBqUIAAqAhSSkZUhGCAAKAIEIQRBACEDAkACQAJAAkACQAJAAkAgDA0AIBj9EyEVIARFDQFBBCEGQQAhBSANQQRPBEADQCAKIANBAnQiA2ogAyAHav0AAAAgFf3mASADIARq/QAAAP3mAf0LAAAgCiAGQQJ0IgNqIAMgB2r9AAAAIBX95gEgAyAEav0AAAD95gH9CwAAIAZBBGohAyAGQQhqIQYgBUECaiIFIBFHDQALCyAQDQAgCiADQQJ0IgNqIAMgB2r9AAAAIBX95gEgAyAEav0AAAD95gH9CwAAIAYhAwsgAyAJTg0HIARFDQEgCSADayIGQQRPDQIgAyEGDAMLQQQhBEEAIQYgDUEETwRAA0AgCiADQQJ0IgNqIAMgB2r9AAAAIBX95gH9CwAAIAogBEECdCIDaiADIAdq/QAAACAV/eYB/QsAACAEQQRqIQMgBEEIaiEEIAZBAmoiBiARRw0ACwsgEEUEQCAKIANBAnQiA2ogAyAHav0AAAAgFf3mAf0LAAAgBCEDCyADIAlPDQYLIAkgA2siBUEETw0CIAMhBAwDCyADIAYgC2siDGohBiAY/RMhFUEAIQUDQCAKIAMgBWpBAnQiCGogByAIav0AAgAgFf3mASAEIAhq/QACAP3mAf0LAgAgBUEEaiIFIAxHDQALIAtFDQQLA0AgCiAGQQJ0IgNqIAMgB2oqAgAgGJQgAyAEaioCAJQ4AgAgBkEBaiIGIAlHDQALDAMLIAMgBUF8cSIIaiEEIBj9EyEVQQAhBgNAIAogAyAGakECdCIMaiAHIAxq/QACACAV/eYB/QsCACAGQQRqIgYgCEcNAAsgBSAIRg0CCwNAIAogBEECdCIDaiADIAdqKgIAIBiUOAIAIARBAWoiBCAJRw0ACwwBCwJ/IAlBBEgiDARA/QwAAAAAAAAAAAAAAAAAAAAAIRVBAAwBC0EAIQX9DAAAAAAAAAAAAAAAAAAAAAAhFUEEIQNBACEGQQAhBCATRQRAA0AgByADQQJ0aiII/QAAICAI/QAAECAI/QAAACAHIAZBAnRq/QAAACAV/eQB/eQB/eQB/eQBIRUgA0EMaiEGIANBEGohAyAEQQRqIgQgEkcNAAsLIA4gD0UNABoDQCADIgRBBGohAyAHIAZBAnRq/QAAACAV/eQBIRUgBCEGIAVBAWoiBSAPRw0ACyAOCyEEIBX9HwAgFf0fAZIgFf0fApIgFf0fA5IhGAJAIAQgCU4NACAJIARrIgNBBE8EQP0MAAAAAAAAAAAAAAAAAAAAACAY/SAAIRUgByAEQQJ0aiEGIAQgAyALayIFaiEEQQAhAwNAIAYgA0ECdGr9AAIAIBX95AEhFSADQQRqIgMgBUcNAAsgFSAVIBX9DQQFBgcAAQIDDA0ODwABAgP95AEiFSAVIBX9DQgJCgsAAQIDAAECAwABAgP95AH9HwAhGCALRQ0BCwNAIAcgBEECdGoqAgAgGJIhGCAEQQFqIgQgCUcNAAsLIBggGpQhGAJ/IAwEQP0MAAAAAAAAAAAAAAAAAAAAACEVQQAMAQsgGP0TIRb9DAAAAAAAAAAAAAAAAAAAAAAhFUEAIQVBBCEDQQAhBkEAIQQgE0UEQANAIAcgA0ECdGoiCP0AACAgFv3lASIXIBf95gEgCP0AABAgFv3lASIXIBf95gEgCP0AAAAgFv3lASIXIBf95gEgByAGQQJ0av0AAAAgFv3lASIXIBf95gEgFf3kAf3kAf3kAf3kASEVIANBDGohBiADQRBqIQMgBEEEaiIEIBJHDQALCyAOIA9FDQAaA0AgAyIEQQRqIQMgByAGQQJ0av0AAAAgFv3lASIXIBf95gEgFf3kASEVIAQhBiAFQQFqIgUgD0cNAAsgDgshBCAV/R8AIBX9HwGSIBX9HwKSIBX9HwOSIRkCQCAEIAlODQAgCSAEayIDQQRPBED9DAAAAAAAAAAAAAAAAAAAAAAgGf0gACEVIAcgBEECdGohBiAEIAMgC2siBWohBCAY/RMhFkEAIQMDQCAGIANBAnRq/QACACAW/eUBIhcgF/3mASAV/eQBIRUgA0EEaiIDIAVHDQALIBUgFSAV/Q0EBQYHAAECAwwNDg8AAQID/eQBIhUgFSAV/Q0ICQoLAAECAwABAgMAAQID/eQB/R8AIRkgC0UNAQsDQCAHIARBAnRqKgIAIBiTIhsgG5QgGZIhGSAEQQFqIgQgCUcNAAsLQwAAgD8gGSAalCAAKgIUkpGVIRkgACgCCCEIIAAoAgQhBkEAIQMCQCAMDQAgGf0TIRUgGP0TIRYgBkUEQCAIBEBBBCEFQQAhBCANQQRPBEADQCAKIANBAnQiA2ogAyAHav0AAAAgFv3lASAV/eYBIAMgCGr9AAAA/eQB/QsAACAKIAVBAnQiA2ogAyAHav0AAAAgFv3lASAV/eYBIAMgCGr9AAAA/eQB/QsAACAFQQRqIQMgBUEIaiEFIARBAmoiBCARRw0ACwsgEA0CIAogA0ECdCIDaiADIAdq/QAAACAW/eUBIBX95gEgAyAIav0AAAD95AH9CwAAIAUhAwwCC0EEIQVBACEEIA1BBE8EQANAIAogA0ECdCIDaiADIAdq/QAAACAW/eUBIBX95gH9CwAAIAogBUECdCIDaiADIAdq/QAAACAW/eUBIBX95gH9CwAAIAVBBGohAyAFQQhqIQUgBEECaiIEIBFHDQALCyAQDQEgCiADQQJ0IgNqIAMgB2r9AAAAIBb95QEgFf3mAf0LAAAgBSEDDAELIAgEQEEAIQVBBCEEA0AgCiAFQQJ0IgNqIAMgB2r9AAAAIBb95QEgFf3mASADIAZq/QAAAP3mASADIAhq/QAAAP3kAf0LAAAgBCIDIQUgA0EEaiIEIAlMDQALDAELQQQhBUEAIQQgDUEETwRAA0AgCiADQQJ0IgNqIAMgB2r9AAAAIBb95QEgFf3mASADIAZq/QAAAP3mAf0LAAAgCiAFQQJ0IgNqIAMgB2r9AAAAIBb95QEgFf3mASADIAZq/QAAAP3mAf0LAAAgBUEEaiEDIAVBCGohBSAEQQJqIgQgEUcNAAsLIBANACAKIANBAnQiA2ogAyAHav0AAAAgFv3lASAV/eYBIAMgBmr9AAAA/eYB/QsAACAFIQMLIAMgCU4NACAJIANrIQQgBkUEQCAIBEACQCAEQQRJBEAgAyEGDAELIAMgBCALayIMaiEGIBn9EyEVIBj9EyEWQQAhBQNAIAogAyAFakECdCIEaiAEIAdq/QACACAW/eUBIBX95gEgBCAIav0AAgD95AH9CwIAIAVBBGoiBSAMRw0ACyALRQ0DCwNAIAogBkECdCIDaiADIAdqKgIAIBiTIBmUIAMgCGoqAgCSOAIAIAZBAWoiBiAJRw0ACwwCCwJAIARBBEkEQCADIQQMAQsgAyAEIAtrIgVqIQQgGf0TIRUgGP0TIRZBACEGA0AgCiADIAZqQQJ0IghqIAcgCGr9AAIAIBb95QEgFf3mAf0LAgAgBkEEaiIGIAVHDQALIAtFDQILA0AgCiAEQQJ0IgNqIAMgB2oqAgAgGJMgGZQ4AgAgBEEBaiIEIAlHDQALDAELIAgEQAJAIARBBEkEQCADIQUMAQsgAyAEIAtrIhRqIQUgGf0TIRUgGP0TIRZBACEEA0AgCiADIARqQQJ0IgxqIAcgDGr9AAIAIBb95QEgFf3mASAGIAxq/QACAP3mASAIIAxq/QACAP3kAf0LAgAgBEEEaiIEIBRHDQALIAtFDQILA0AgCiAFQQJ0IgNqIAMgB2oqAgAgGJMgGZQgAyAGaioCAJQgAyAIaioCAJI4AgAgBUEBaiIFIAlHDQALDAELAkAgBEEESQRAIAMhBAwBCyADIAQgC2siDGohBCAZ/RMhFSAY/RMhFkEAIQUDQCAKIAMgBWpBAnQiCGogByAIav0AAgAgFv3lASAV/eYBIAYgCGr9AAIA/eYB/QsCACAFQQRqIgUgDEcNAAsgC0UNAQsDQCAKIARBAnQiA2ogAyAHaioCACAYkyAZlCADIAZqKgIAlDgCACAEQQFqIgQgCUcNAAsLIAFBAWoiASACRw0ACwsLYQEBfyMAQSBrIgckAAJAIARBAEwNACAFQQBMDQAgB0EANgIcIAcgBjgCGCAHIAU2AhQgByADNgIQIAcgAjYCDCAHIAE2AgggByAANgIEQRogB0EEaiAEEBALIAdBIGokAAuWDQMJfwN7BX0gASACSARAQwAAAAAQOSESA0AgACgCACIKIAAoAggiBSABbEECdCIJaiEHAn0gBUEESARAQQEhBCAHKgIADAELQQghAyAH/QAAACEMQQQhBCAFQQhPBEADQCAMIAcgBEECdGr9AAAA/ekBIQwgAyIEQQRqIgMgBUwNAAsgBUH8////B3EhBAsgDP0fACAM/R8BlyAM/R8CIAz9HwOXlwshDwJAIAQgBU4NACAFIARrIgZBBE8EQCAHIARBAnRqIQsgBCAGQXxxIghqIQQgD/0TIQxBACEDA0AgCyADQQJ0av0AAgAgDP3rASEMIANBBGoiAyAIRw0ACyAMIAwgDP0NCAkKCwwNDg8AAQIDAAECA/3rASIMIAwgDP0NBAUGBwABAgMAAQIDAAECA/3rAf0fACEPIAYgCEYNAQsDQCAHIARBAnRqKgIAIhAgDyAPIBBdGyEPIARBAWoiBCAFRw0ACwsgCSAAKAIEIglqIQgCQAJAAkACQAJAAkACQCAAKAIMBEAgBUEATA0BIAVBBEkiBARAQQAhA0MAAAAAIRAMBQsgBUH8////B3EhAyAP/RMhDv0MAAAAAAAAAAAAAAAAAAAAACEMQQAhBgNAIAcgBkECdGr9AAIAIA795QEiDf0fABAN/RMgDf0fARAN/SABIA39HwIQDf0gAiAN/R8DEA39IAMgDP3kASEMIAZBBGoiBiADRw0ACyAMIAwgDf0NBAUGBwABAgMMDQ4PAAECA/3kASIMIAwgDP0NCAkKCwABAgMAAQIDAAECA/3kAf0fACEQIAMgBUcNBAwFCyAFQQBMDQZBACEDQwAAAAAhEAJAIAVBBEkiCw0AIAkgCmtBEEkNACAFQfz///8HcSEDIA/9EyEO/QwAAAAAAAAAAAAAAAAAAAAAIQxBACEGA0AgCCAGQQJ0IgRqIAQgB2r9AAIAIA795QEiDf0fABAN/RMgDf0fARAN/SABIA39HwIQDf0gAiAN/R8DEA39IAMiDf0LAgAgDSAM/eQBIQwgBkEEaiIGIANHDQALIAwgDCAN/Q0EBQYHAAECAwwNDg8AAQID/eQBIgwgDCAM/Q0ICQoLAAECAwABAgMAAQID/eQB/R8AIRAgAyAFRg0DCyADQQFyIQQgBUEBcQRAIAggA0ECdCIDaiADIAdqKgIAIA+TEA0iETgCACARIBCSIRAgBCEDCyAEIAVHDQEMAgsgDyASkiEPQQAhAwwECwNAIAggA0ECdCIEaiAEIAdqKgIAIA+TEA0iETgCACAIIARBBGoiBGogBCAHaioCACAPkxANIhM4AgAgEyARIBCSkiEQIANBAmoiAyAFRw0ACwsgEEMAAAAAXkUNA0MAAIA/IBCVIQ9BACEEIAtFBEAgD/0TIQxBACEDQQQhBgNAIAggA0ECdGoiAyAD/QAAACAM/eYB/QsAACAGIgQiA0EEaiIGIAVMDQALCyAEIAVPDQMgBSAEayIDQQRPBEAgCCAEQQJ0aiEGIAQgAyAFQQNxIgdrIgpqIQQgD/0TIQxBACEDA0AgBiADQQJ0aiIJIAn9AAIAIAz95gH9CwIAIANBBGoiAyAKRw0ACyAHRQ0ECwNAIAggBEECdGoiAyADKgIAIA+UOAIAIARBAWoiBCAFRw0ACwwDCwNAIAcgA0ECdGoqAgAgD5MQDSAQkiEQIANBAWoiAyAFRw0ACwsgEBA5IA+SIQ9BACEDIAQNACAP/RMhDEEAIQZBBCEEA0AgCCAGQQJ0IgNqIAMgB2r9AAAAIAz95QH9CwAAIAQiAyEGIANBBGoiBCAFTA0ACwsgAyAFTg0AAkAgBSADayIEQQRJBEAgAyEEDAELIAkgCmtBEEkEQCADIQQMAQsgAyAEIAVBA3EiCmsiCWohBCAP/RMhDEEAIQYDQCAIIAMgBmpBAnQiC2ogByALav0AAgAgDP3lAf0LAgAgBkEEaiIGIAlHDQALIApFDQELQQAhBiAFIAQiA2tBA3EiCgRAA0AgCCADQQJ0IglqIAcgCWoqAgAgD5M4AgAgA0EBaiEDIAZBAWoiBiAKRw0ACwsgBCAFa0F8Sw0AA0AgCCADQQJ0IgRqIAQgB2oqAgAgD5M4AgAgCCAEQQRqIgZqIAYgB2oqAgAgD5M4AgAgCCAEQQhqIgZqIAYgB2oqAgAgD5M4AgAgCCAEQQxqIgRqIAQgB2oqAgAgD5M4AgAgA0EEaiIDIAVHDQALCyABQQFqIgEgAkcNAAsLC0kBAX8jAEEQayIFJAACQCACQQBMDQAgA0EATA0AIAUgBDYCDCAFIAM2AgggBSABNgIEIAUgADYCAEEZIAUgAhAQCyAFQRBqJAAL7QkDH38BewF9AkAgASACTg0AIAAoAgwhBiAAKAIIIgtBAEoEQEMAAIA/IAuzlSIj/RMhIiAGQQNxIQwgBkECdCENIAZBBGsiD0EEcSEUIAYgC2wiEkECdCEVIAEgBmxBAnQhFiABIBJsQQJ0IRcgBiABQQJ0QQRqbCEYIA9BAnZBAWoiA0H8////B3EhGSADQQNxIRMgA0H+////B3EhGiAGIAEgC2xBAnRBBGpsIRsDQCABIBJsQQJ0IQUgACgCACEDIAAoAgQiBCABIAZsQQJ0aiEHAkAgBkEATA0AIA1FDQAgB0EAIA38CwALIAMgBWohHCAOIBVsIgUgAyAbamohHSADIBdqIAVqIR4gDSAObCIDIAQgGGpqIR8gBCAWaiADaiEgQQAhCgNAIBwgBiAKbEECdGohCEEAIQMCQCAGQQRIIiENAEEEIQRBACEFIA9BBE8EQANAIAcgA0ECdCIDaiIJIAMgCGr9AAAAIAn9AAAA/eQB/QsAACAHIARBAnQiA2oiCSADIAhq/QAAACAJ/QAAAP3kAf0LAAAgBEEEaiEDIARBCGohBCAFQQJqIgUgGkcNAAsLIBQNACAHIANBAnQiA2oiBSADIAhq/QAAACAF/QAAAP3kAf0LAAAgBCEDCwJAIAMgBk4NAAJAIAYgA2siBEEESQRAIAMhBQwBCwJAICAgA0ECdCIFaiAdIAogDWwiCWpPDQAgCSAeaiAFaiAfTw0AIAMhBQwBCyADIAQgDGsiCWohBUEAIQQDQCAHIAMgBGpBAnQiEGoiESAR/QACACAIIBBq/QACAP3kAf0LAgAgBEEEaiIEIAlHDQALIAxFDQELQQAhAyAGIAUiBGtBA3EiCQRAA0AgByAEQQJ0IhBqIhEgESoCACAIIBBqKgIAkjgCACAEQQFqIQQgA0EBaiIDIAlHDQALCyAFIAZrQXxLDQADQCAHIARBAnQiA2oiBSAFKgIAIAMgCGoqAgCSOAIAIAcgA0EEaiIFaiIJIAkqAgAgBSAIaioCAJI4AgAgByADQQhqIgVqIgkgCSoCACAFIAhqKgIAkjgCACAHIANBDGoiA2oiBSAFKgIAIAMgCGoqAgCSOAIAIARBBGoiBCAGRw0ACwsgCkEBaiIKIAtHDQALAkAgACgCEEUNAEEAIQMCQCAhDQBBACEFQQQhBEEAIQggD0EMTwRAA0AgByADQQJ0aiIDIAP9AAAAICL95gH9CwAAIAcgBEECdGoiAyAD/QAAACAi/eYB/QsAACADIAP9AAAQICL95gH9CwAQIAMgA/0AACAgIv3mAf0LACAgBEEMaiEDIARBEGohBCAIQQRqIgggGUcNAAsLIBNFDQADQCAHIANBAnRqIgMgA/0AAAAgIv3mAf0LAAAgBCIDQQRqIQQgBUEBaiIFIBNHDQALCyADIAZODQAgBiADayIEQQRPBEAgByADQQJ0aiEFIAMgBCAMayIIaiEDQQAhBANAIAUgBEECdGoiCiAK/QACACAi/eYB/QsCACAEQQRqIgQgCEcNAAsgDEUNAQsDQCAHIANBAnRqIgQgBCoCACAjlDgCACADQQFqIgMgBkcNAAsLIA5BAWohDiABQQFqIgEgAkcNAAsMAQsgBkEATA0AIAYgAiABa2xBAnQiAkUNACAAKAIEIAEgBmxBAnRqQQAgAvwLAAsLfAEBfyMAQSBrIgYkAAJAIAFBAEwNACADQQBMDQAgAkEATARAIAEgA2wiAEEATA0BIABBAnQiAEUNASAFQQAgAPwLAAwBCyAGIAQ2AhwgBiADNgIYIAYgAjYCFCAGIAU2AhAgBiAANgIMQRggBkEMaiABEBALIAZBIGokAAuLAwMEewV/AX0gAUEATARAIANDAAAAADgCAA8LQRAhCSABQRBPBEADQCAAIAhBAnRqIgj9AAAwIAT95AEhBCAI/QAAICAF/eQBIQUgCP0AABAgBv3kASEGIAj9AAAAIAf95AEhByAJIghBEGoiCSABTA0ACyAGIAf95AEgBf3kASAE/eQBIQQgAUHw////B3EhCAsgBP0fACAE/R8BkiAE/R8CkiAE/R8DkiENAkAgASAITQ0AIAEgCGsiCUEETwRA/QwAAAAAAAAAAAAAAAAAAAAAIA39IAAhBCAAIAhBAnRqIQogCCAJIAFBA3EiC2siDGohCEEAIQkDQCAKIAlBAnRq/QACACAE/eQBIQQgCUEEaiIJIAxHDQALIAQgBCAE/Q0EBQYHAAECAwwNDg8AAQID/eQBIgQgBCAE/Q0ICQoLAAECAwABAgMAAQID/eQB/R8AIQ0gC0UNAQsDQCAAIAhBAnRqKgIAIA2SIQ0gCEEBaiIIIAFHDQALCyADIA0gAbOVIA0gAhs4AgAL7hAEBn8BewF9AXwgAkEIdCICIAAoAggiAyACIANIGyEDIAFBCHQhASAAKAIEIQQgACgCACEFAkACQAJAAkACQAJAAkACQAJAAkAgACgCDA4GBQQDAgEABgsgASADTg0IA0AgBCABQQJ0IgBqIAAgBWoqAgC7EDu2OAIAIAFBAWoiASADRw0ACwwICyABIANODQcDQCAEIAFBAnQiAGogACAFaioCALsiC0QAAAAAAADgP6IgC0TNO39mnqDmP6IQO0QAAAAAAADwP6CitjgCACABQQFqIgEgA0cNAAsMBwsgASADTg0GAkAgAyABayIAQQRJBEAgASECDAELIAQgBWtBEEkEQCABIQIMAQsgASAAIANBA3EiBmsiB2ohAkEAIQADQCAEIAAgAWpBAnQiCGogBSAIav0AAgAiCf0MAAAAPwAAAD8AAAA/AAAAP/3mASAJIAn95gH9DBMnNz0TJzc9Eyc3PRMnNz395gEgCf3mASAJ/eQB/QwqQkw/KkJMPypCTD8qQkw//eYBIgn9HwAQEv0TIAn9HwEQEv0gASAJ/R8CEBL9IAIgCf0fAxAS/SAD/QwAAIA/AACAPwAAgD8AAIA//eQB/eYB/QsCACAAQQRqIgAgB0cNAAsgBkUNBwsgAkEBaiEAIAMgAmtBAXEEQCAEIAJBAnQiAWogASAFaioCACIKQwAAAD+UIAogCpRDEyc3PZQgCpQgCpJDKkJMP5QQEkMAAIA/kpQ4AgAgACECCyAAIANGDQYDQCAEIAJBAnQiAGogACAFaioCACIKQwAAAD+UIAogCpRDEyc3PZQgCpQgCpJDKkJMP5QQEkMAAIA/kpQ4AgAgBCAAQQRqIgBqIAAgBWoqAgAiCkMAAAA/lCAKIAqUQxMnNz2UIAqUIAqSQypCTD+UEBJDAACAP5KUOAIAIAJBAmoiAiADRw0ACwwGCyABIANODQUCQCADIAFrIgBBBEkEQCABIQIMAQsgBCAFa0EQSQRAIAEhAgwBCyABIAAgA0EDcSIGayIHaiECQQAhAANAIAUgACABakECdCIIav0AAgAhCSAEIAhqIAkgCf3hASIJ/R8AEA39EyAJ/R8BEA39IAEgCf0fAhAN/SACIAn9HwMQDf0gA/0MAACAPwAAgD8AAIA/AACAP/3kAf3nAf0LAgAgAEEEaiIAIAdHDQALIAZFDQYLIAJBAWohACADIAJrQQFxBEAgBSACQQJ0IgFqKgIAIQogASAEaiAKIAqMEA1DAACAP5KVOAIAIAAhAgsgACADRg0FA0AgBSACQQJ0IgBqKgIAIQogACAEaiAKIAqMEA1DAACAP5KVOAIAIAUgAEEEaiIAaioCACEKIAAgBGogCiAKjBANQwAAgD+SlTgCACACQQJqIgIgA0cNAAsMBQsgASADTg0EAkAgAyABayIAQQRJBEAgASECDAELIAQgBWtBEEkEQCABIQIMAQsgASAAIANBA3EiBmsiB2ohAkEAIQADQCAEIAAgAWpBAnQiCGr9DAAAgD8AAIA/AACAPwAAgD/9DAAAgD8AAIA/AACAPwAAgD8gBSAIav0AAgD94QEiCf0fABAN/RMgCf0fARAN/SABIAn9HwIQDf0gAiAJ/R8DEA39IAP95AH95wH9CwIAIABBBGoiACAHRw0ACyAGRQ0FCyACQQFqIQAgAyACa0EBcQRAIAQgAkECdCIBakMAAIA/IAEgBWoqAgCMEA1DAACAP5KVOAIAIAAhAgsgACADRg0EA0AgBCACQQJ0IgBqQwAAgD8gACAFaioCAIwQDUMAAIA/kpU4AgAgBCAAQQRqIgBqQwAAgD8gACAFaioCAIwQDUMAAIA/kpU4AgAgAkECaiICIANHDQALDAQLIAFBBHIiAiADTA0BIAEhAAwCCyABIANODQICQCADIAFrIgBBBEkEQCABIQIMAQsgBCAFa0EQSQRAIAEhAgwBCyABIAAgA0EDcSIGayIHaiECQQAhAANAIAQgACABakECdCIIaiAFIAhq/QACACIJ/R8AEBL9EyAJ/R8BEBL9IAEgCf0fAhAS/SACIAn9HwMQEv0gA/0LAgAgAEEEaiIAIAdHDQALIAZFDQMLIAMgAiIBa0EDcSIGBEBBACEAA0AgBCABQQJ0IgdqIAUgB2oqAgAQEjgCACABQQFqIQEgAEEBaiIAIAZHDQALCyACIANrQXxLDQIDQCAEIAFBAnQiAGogACAFaioCABASOAIAIAQgAEEEaiICaiACIAVqKgIAEBI4AgAgBCAAQQhqIgJqIAIgBWoqAgAQEjgCACAEIABBDGoiAGogACAFaioCABASOAIAIAFBBGoiASADRw0ACwwCCwNAIAQgAUECdCIAaiAAIAVq/QAAAP0MAAAAAAAAAAAAAAAAAAAAAP3pAf0LAAAgAiIAIgFBBGoiAiADTA0ACwsgACADTg0AAkAgAyAAayIGQQRJBEAgACECDAELIAQgBWtBEEkEQCAAIQIMAQsgACAGQXxxIgdqIQJBACEBA0AgBCAAIAFqQQJ0IghqIAUgCGr9AAIA/QwAAAAAAAAAAAAAAAAAAAAA/ekB/QsCACABQQRqIgEgB0cNAAsgBiAHRg0BCyADIAIiAWtBA3EiBgRAQQAhAANAIAQgAUECdCIHaiAFIAdqKgIAQwAAAACXOAIAIAFBAWohASAAQQFqIgAgBkcNAAsLIAIgA2tBfEsNAANAIAQgAUECdCIAaiAAIAVqKgIAQwAAAACXOAIAIAQgAEEEaiICaiACIAVqKgIAQwAAAACXOAIAIAQgAEEIaiICaiACIAVqKgIAQwAAAACXOAIAIAQgAEEMaiIAaiAAIAVqKgIAQwAAAACXOAIAIAFBBGoiASADRw0ACwsLRwEBfyMAQRBrIgQkACADQQBKBEAgBCAANgIMIAQgAjYCBCAEIAE2AgAgBCADNgIIQRcgBCADQf8BakEIdhAQCyAEQRBqJAALiiIDC38DfQR7IAJBCnQiAiAAKAIMIgMgAiADSBshCCABQQp0IQICQAJAAkACQAJAAkACQAJAAkACQCAAKAIYIgkOAwABAwQLAkACQCAAKAIcDgMAAQMFCyAAKAIIIQMgACgCBCEJIAAoAgAhBCAAKAIUIQUgCCACQQRyIgBIBEAgAiEBDAkLAkACQAJAIAUOAwABAggLA0AgAyACQQJ0IgFqIAEgCWr9AAAAIAEgBGr9AAAA/eQB/QsAACAAIgEhAiABQQRqIgAgCEwNAAsMCgsDQCADIAJBAnQiAWogASAEav0AAAAgASAJav0AAAD95QH9CwAAIAAiASECIAFBBGoiACAITA0ACwwJCwNAIAMgAkECdCIBaiABIAlq/QAAACABIARq/QAAAP3mAf0LAAAgACIBIQIgAUEEaiIAIAhMDQALDAgLIAAoAgghAyAAKAIAIQkgACgCFCEEIAAoAgQqAgAhDgJAIAggAkEEciIASARAIAIhAQwBC0MAAIA/IA6V/RMhEyAO/RMhEgNAIAAhASADIAJBAnQiAGogACAJav0AAAAhEQJ7AkACQAJAAkAgBA4DAgEAAwsgESAS/eYBDAMLIBEgEv3lAQwCCyARIBL95AEMAQsgESAT/eYBC/0LAAAgASECIAFBBGoiACAITA0ACwsgASAITg0IAkAgCCABayIFQQRJBEAgASECDAELIAEgBUF8cSIGaiECQwAAgD8gDpX9EyEUIA79EyERIAT9ESESQQAhAANAIAMgACABakECdCIHaiAHIAlq/QACACITIBH95gEgEyAR/eUBIBMgEf3kASATIBT95gEgEv0MAAAAAAAAAAAAAAAAAAAAAP03/VIgEv0MAQAAAAEAAAABAAAAAQAAAP03/VIgEv0MAgAAAAIAAAACAAAAAgAAAP03/VL9CwIAIABBBGoiACAGRw0ACyAFIAZGDQkLQwAAgD8gDpUhEANAIAkgAkECdCIAaioCACEPIAAgA2oCfQJAAkACQAJAIAQOAwIBAAMLIA8gDpQMAwsgDyAOkwwCCyAPIA6SDAELIA8gEJQLOAIAIAJBAWoiAiAIRw0ACwwICyAAKAIcDQIgACgCCCEDIAAoAgQhCSAAKAIUIQQgACgCACoCACEOIAggAkEEciIASARAIAIhAQwGCyAO/RMhEQJAAkACQCAEDgMAAQIHCwNAIAMgAkECdCIBaiABIAlq/QAAACAR/eQB/QsAACAAIgEhAiABQQRqIgAgCEwNAAsMBwsDQCADIAJBAnQiAWogESABIAlq/QAAAP3lAf0LAAAgACIBIQIgAUEEaiIAIAhMDQALDAYLA0AgAyACQQJ0IgFqIAEgCWr9AAAAIBH95gH9CwAAIAAiASECIAFBBGoiACAITA0ACwwFCyACIAhODQYDQCACQQJ0IgEgACgCCGohBSAAKAIAIAFqIQYgACgCBCACIAAoAhAiAW8iBEECdGohByAAKAIUIQlBACEDAkAgASAEayIBIAggAmsiBCABIARIGyILQQRIDQACQAJAAkACQCAJDgMCAQADC0EEIQEgC0EEayIKQQRPBEAgCkECdkEBakH+////B3EhDEEAIQQDQCAFIANBAnQiA2ogAyAHav0AAAAgAyAGav0AAAD95gH9CwAAIAUgAUECdCIDaiADIAdq/QAAACADIAZq/QAAAP3mAf0LAAAgAUEEaiEDIAFBCGohASAEQQJqIgQgDEcNAAsLIApBBHENAyAFIANBAnQiA2ogAyAHav0AAAAgAyAGav0AAAD95gH9CwAAIAEhAwwDC0EEIQEgC0EEayIKQQRPBEAgCkECdkEBakH+////B3EhDEEAIQQDQCAFIANBAnQiA2ogAyAGav0AAAAgAyAHav0AAAD95QH9CwAAIAUgAUECdCIDaiADIAZq/QAAACADIAdq/QAAAP3lAf0LAAAgAUEEaiEDIAFBCGohASAEQQJqIgQgDEcNAAsLIApBBHENAiAFIANBAnQiA2ogAyAGav0AAAAgAyAHav0AAAD95QH9CwAAIAEhAwwCC0EEIQEgCSEDIAtBBGsiCkEETwRAIApBAnZBAWpB/v///wdxIQxBACEEA0AgBSADQQJ0IgNqIAMgB2r9AAAAIAMgBmr9AAAA/eQB/QsAACAFIAFBAnQiA2ogAyAHav0AAAAgAyAGav0AAAD95AH9CwAAIAFBBGohAyABQQhqIQEgBEECaiIEIAxHDQALCyAKQQRxDQEgBSADQQJ0IgNqIAMgB2r9AAAAIAMgBmr9AAAA/eQB/QsAACABIQMMAQtBBCEBIAtBBGsiCkEETwRAIApBAnZBAWpB/v///wdxIQxBACEEA0AgBSADQQJ0IgNqIAMgBmr9AAAAIAMgB2r9AAAA/ecB/QsAACAFIAFBAnQiA2ogAyAGav0AAAAgAyAHav0AAAD95wH9CwAAIAFBBGohAyABQQhqIQEgBEECaiIEIAxHDQALCyAKQQRxDQAgBSADQQJ0IgNqIAMgBmr9AAAAIAMgB2r9AAAA/ecB/QsAACABIQMLAkAgAyALTg0AAkAgCyADayIBQQRJBEAgAyEBDAELIAMgASALQQNxIgxrIg1qIQEgCf0RIRFBACEEA0AgBSADIARqQQJ0IgpqIAcgCmr9AAIAIhIgBiAKav0AAgAiE/3kASATIBL95QEgEiAT/eYBIBMgEv3nASAR/QwCAAAAAgAAAAIAAAACAAAA/Tf9UiAR/QwBAAAAAQAAAAEAAAABAAAA/Tf9UiAR/QwAAAAAAAAAAAAAAAAAAAAA/Tf9Uv0LAgAgBEEEaiIEIA1HDQALIAxFDQELA0AgByABQQJ0IgNqKgIAIQ8gAyAGaioCACEOIAMgBWoCfQJAAkACQAJAIAkOAwABAgMLIA8gDpIMAwsgDiAPkwwCCyAPIA6UDAELIA4gD5ULOAIAIAFBAWoiASALRw0ACwsgAiALaiICIAhIDQALDAYLIAAoAhwNACACIAhODQUDQCACQQJ0IgEgACgCCGohBSAAKAIEIAFqIQYgACgCACACIAAoAhAiAW8iBEECdGohByAAKAIUIQlBACEDAkAgASAEayIBIAggAmsiBCABIARIGyILQQRIDQACQAJAAkACQCAJDgMCAQADC0EEIQEgC0EEayIKQQRPBEAgCkECdkEBakH+////B3EhDEEAIQQDQCAFIANBAnQiA2ogAyAGav0AAAAgAyAHav0AAAD95gH9CwAAIAUgAUECdCIDaiADIAZq/QAAACADIAdq/QAAAP3mAf0LAAAgAUEEaiEDIAFBCGohASAEQQJqIgQgDEcNAAsLIApBBHENAyAFIANBAnQiA2ogAyAGav0AAAAgAyAHav0AAAD95gH9CwAAIAEhAwwDC0EEIQEgC0EEayIKQQRPBEAgCkECdkEBakH+////B3EhDEEAIQQDQCAFIANBAnQiA2ogAyAHav0AAAAgAyAGav0AAAD95QH9CwAAIAUgAUECdCIDaiADIAdq/QAAACADIAZq/QAAAP3lAf0LAAAgAUEEaiEDIAFBCGohASAEQQJqIgQgDEcNAAsLIApBBHENAiAFIANBAnQiA2ogAyAHav0AAAAgAyAGav0AAAD95QH9CwAAIAEhAwwCC0EEIQEgCSEDIAtBBGsiCkEETwRAIApBAnZBAWpB/v///wdxIQxBACEEA0AgBSADQQJ0IgNqIAMgBmr9AAAAIAMgB2r9AAAA/eQB/QsAACAFIAFBAnQiA2ogAyAGav0AAAAgAyAHav0AAAD95AH9CwAAIAFBBGohAyABQQhqIQEgBEECaiIEIAxHDQALCyAKQQRxDQEgBSADQQJ0IgNqIAMgBmr9AAAAIAMgB2r9AAAA/eQB/QsAACABIQMMAQtBBCEBIAtBBGsiCkEETwRAIApBAnZBAWpB/v///wdxIQxBACEEA0AgBSADQQJ0IgNqIAMgB2r9AAAAIAMgBmr9AAAA/ecB/QsAACAFIAFBAnQiA2ogAyAHav0AAAAgAyAGav0AAAD95wH9CwAAIAFBBGohAyABQQhqIQEgBEECaiIEIAxHDQALCyAKQQRxDQAgBSADQQJ0IgNqIAMgB2r9AAAAIAMgBmr9AAAA/ecB/QsAACABIQMLAkAgAyALTg0AAkAgCyADayIBQQRJBEAgAyEBDAELIAMgASALQQNxIgxrIg1qIQEgCf0RIRFBACEEA0AgBSADIARqQQJ0IgpqIAYgCmr9AAIAIhIgByAKav0AAgAiE/3kASATIBL95QEgEiAT/eYBIBMgEv3nASAR/QwCAAAAAgAAAAIAAAACAAAA/Tf9UiAR/QwBAAAAAQAAAAEAAAABAAAA/Tf9UiAR/QwAAAAAAAAAAAAAAAAAAAAA/Tf9Uv0LAgAgBEEEaiIEIA1HDQALIAxFDQELA0AgBiABQQJ0IgNqKgIAIQ8gAyAHaioCACEOIAMgBWoCfQJAAkACQAJAIAkOAwABAgMLIA8gDpIMAwsgDiAPkwwCCyAPIA6UDAELIA4gD5ULOAIAIAFBAWoiASALRw0ACwsgAiALaiICIAhIDQALDAULIAIgCE4NBCAAKAIIIQQgACgCBCEFIAAoAgAhBiAAKAIUIQcgACgCHCELA0AgAiEBAkACQAJAIAkOAgIBAAsgAiAAKAIQbyEBDAELQQAhAQsgAiEDAkACQAJAIAsOAgIBAAsgAiAAKAIQbyEDDAELQQAhAwsgBSADQQJ0aioCACEPIAYgAUECdGoqAgAhDiAEIAJBAnRqAn0CQAJAAkACQCAHDgMAAQIDCyAPIA6SDAMLIA4gD5MMAgsgDyAOlAwBCyAOIA+VCzgCACACQQFqIgIgCEcNAAsMBAsDQCADIAJBAnQiAWogASAEav0AAAAgASAJav0AAAD95wH9CwAAIAAiASECIAFBBGoiACAITA0ACwwCCwNAIAMgAkECdCIBaiARIAEgCWr9AAAA/ecB/QsAACAAIgEhAiABQQRqIgAgCEwNAAsLIAEgCE4NAQJAIAggAWsiBUEESQRAIAEhAgwBCyABIAVBfHEiBmohAiAO/RMhESAE/REhEkEAIQADQCADIAAgAWpBAnQiB2ogByAJav0AAgAiEyAR/eQBIBEgE/3lASATIBH95gEgESAT/ecBIBL9DAIAAAACAAAAAgAAAAIAAAD9N/1SIBL9DAEAAAABAAAAAQAAAAEAAAD9N/1SIBL9DAAAAAAAAAAAAAAAAAAAAAD9N/1S/QsCACAAQQRqIgAgBkcNAAsgBSAGRg0CCwNAIAkgAkECdCIAaioCACEPIAAgA2oCfQJAAkACQAJAIAQOAwABAgMLIA8gDpIMAwsgDiAPkwwCCyAPIA6UDAELIA4gD5ULOAIAIAJBAWoiAiAIRw0ACwwBCyABIAhODQACQCAIIAFrIgZBBEkEQCABIQIMAQsgASAGQXxxIgdqIQIgBf0RIRFBACEAA0AgAyAAIAFqQQJ0IgtqIAkgC2r9AAIAIhIgBCALav0AAgAiE/3kASATIBL95QEgEiAT/eYBIBMgEv3nASAR/QwCAAAAAgAAAAIAAAACAAAA/Tf9UiAR/QwBAAAAAQAAAAEAAAABAAAA/Tf9UiAR/QwAAAAAAAAAAAAAAAAAAAAA/Tf9Uv0LAgAgAEEEaiIAIAdHDQALIAYgB0YNAQsDQCAJIAJBAnQiAGoqAgAhDyAAIARqKgIAIQ4gACADagJ9AkACQAJAAkAgBQ4DAAECAwsgDyAOkgwDCyAOIA+TDAILIA8gDpQMAQsgDiAPlQs4AgAgAkEBaiICIAhHDQALCwtrAQF/IwBBIGsiCCQAIAZBAEoEQCAIIAQ2AhwgCCACNgIYIAggADYCFCAIIAU2AgggCCADNgIEIAggATYCACAIQQEgByAHQQFMGzYCECAIIAY2AgxBFiAIIAZB/wdqQQp2EBALIAhBIGokAAvFCgMOfwV7An0CQCABIAJODQAgACgCDCELIAAoAhgiDEEATARAIAIgAWtBAnQiAEUNASALIAFBAnRqQQAgAPwLAA8LIAAoAhQhDSAAKAIAIQ4gACgCBCEPA0AgASAMbCEQQQAhCkMAAAAAIRYDQCANIApBCHQiCGshByAPIAogEGpBwgBsaiIJLgBAIgRB/wdxIQYgBEGAgICAeHEhAyAEQf//A3EhBQJAAkAgBEEKdkEfcSIEBEAgBEEfRw0BIAMgBUENdHJBgICA/AdyIQMMAgsgBkUNASADIAVBCSAGZyIDQR9za3RBDnRBgID/A3EgA0EXdGtqQYCAgJgEaiEDDAELIARBF3QgBkENdHIgA2pBgICAwANqIQMLIAggDmohBiADviEXAkAgB0H/AUwEQEEAIQRBACEDA0ACQCADIAdODQAgAyAJai0AAEEDcSIFQQFNBEAgBQ0BIAQgAyAGaiwAAGshBAwBCyAEIAMgBmosAABqIQQLIANBAWoiA0EgRw0AC0EAIQMDQAJAIANBIGoiBSAHTg0AIAMgCWotAABBAnZBA3EiCEEBTQRAIAgNASAEIAUgBmosAABrIQQMAQsgBCAFIAZqLAAAaiEECyADQQFqIgNBIEcNAAtBACEDA0ACQCADQUBrIgUgB04NACADIAlqLQAAQQR2QQNxIghBAU0EQCAIDQEgBCAFIAZqLAAAayEEDAELIAQgBSAGaiwAAGohBAsgA0EBaiIDQSBHDQALQQAhAwNAAkAgA0HgAGoiBSAHTg0AIAMgCWosAAAiCEEATgRAIAhBP0sNASAEIAUgBmosAABrIQQMAQsgBCAFIAZqLAAAaiEECyADQQFqIgNBIEcNAAsgCUEgaiEJQQAhAwNAAkAgA0GAAWoiBSAHTg0AIAMgCWotAABBA3EiCEEBTQRAIAgNASAEIAUgBmosAABrIQQMAQsgBCAFIAZqLAAAaiEECyADQQFqIgNBIEcNAAtBACEDA0ACQCADQaABaiIFIAdODQAgAyAJai0AAEECdkEDcSIIQQFNBEAgCA0BIAQgBSAGaiwAAGshBAwBCyAEIAUgBmosAABqIQQLIANBAWoiA0EgRw0AC0EAIQMDQAJAIANBwAFqIgUgB04NACADIAlqLQAAQQR2QQNxIghBAU0EQCAIDQEgBCAFIAZqLAAAayEEDAELIAQgBSAGaiwAAGohBAsgA0EBaiIDQSBHDQALQQAhAwNAAkAgA0HgAWoiBSAHTg0AIAMgCWosAAAiCEEATgRAIAhBP0sNASAEIAUgBmosAABrIQQMAQsgBCAFIAZqLAAAaiEECyADQQFqIgNBIEcNAAsMAQtBACEHQQEhA/0MAAAAAAAAAAAAAAAAAAAAACERA0AgBiAHQQd0aiEEIAkgB0EFdGoiB/0AABAhEyAH/QAAACEUQQAhBwNAIBQgB0EBdCIF/W39DAMDAwMDAwMDAwMDAwMDAwP9TiIS/YoBIAQgB0EFdGoiCP0AAAAiFf2IAf26ASAR/a4BIBL9iQEgFf2HAf26Af2uASATIAX9bf0MAwMDAwMDAwMDAwMDAwMDA/1OIhH9igEgCP0AABAiEv2IAf26Af2uASAR/YkBIBL9hwH9ugH9rgEhESAHQQFqIgdBBEcNAAtBASEHIANBACEDDQALIBH9GwAgEf0bAWogEf0bAmogEf0bA2ogACgCCCAKQQJ0aigCAGshBAsgBLIgF5QgFpIhFiAKQQFqIgogDEcNAAsgCyABQQJ0aiAAKgIQIBaUOAIAIAFBAWoiASACRw0ACwsLrgMCCH8BeyMAQaAEayIJJAACQCAFQQBMDQAgBkEATA0AIAVB/wFqQQh2IQsgCUEgaiEMIAVBgYACTwRAIAtBAnQQFCINIQwgDUUNAQsgAyAEaiEQIAAgAWohDiAFIQADQEEAIQMCQCAFIApBCHQiBEwNAEGAAkEBIAAgAEEBTBsiASABQYACThshASAEIA5qIQ9BACEEIABBBE4EQCABQfwDcSEE/QwAAAAAAAAAAAAAAAAAAAAAIREDQCARIAMgD2r9XAAA/YcB/acB/a4BIREgA0EEaiIDIARHDQALIBEgESAR/Q0ICQoLDA0ODwABAgMAAQID/a4BIhEgESAR/Q0EBQYHAAECAwABAgMAAQID/a4B/RsAIQMgASAERg0BCwNAIAMgBCAPaiwAAGohAyAEQQFqIgQgAUcNAAsLIAwgCkECdGogAzYCACAAQYACayEAIApBAWoiCiALRw0ACyAJIAs2AhwgCSAFNgIYIAkgAjgCFCAJIAw2AgwgCSAQNgIIIAkgDjYCBCAJIAcgCEECdGo2AhBBFSAJQQRqIAYQECANEBMLIAlBoARqJAAL8wUCAnsQfyABIAJIBEAgACgCBCEKIAAoAhwiBUF+cSEMIAAoAgAhDSAAKAIYIQ4gACgCFCEPIAAoAhAhECAAKAIMIREgACgCCCESIAVBAUohEwNAIA8gASAFbCIAaiEGIAAgEGohByAAIBFqIQggACASaiEJQQAhAP0MAAAAAAAAAAAAAAAAAAAAACIEIQNBAiELIBMEQANAIAAgCGotAABBBHT9AASQCP0Mq6qqPquqqj6rqqo+q6qqPv3mASAAIAlqLQAAQQR0/QAEkAj95AEgACAHai0AAEEEdP0ABJAI/Qw5juM9OY7jPTmO4z05juM9/eYB/eQBIAAgBmotAABBBHT9AASQCP0MJrQXPSa0Fz0mtBc9JrQXPf3mAf3kASAKIABBBHRqIhT9AAAA/eYBIAP95AEhAyAIIABBAXIiAGotAABBBHT9AASQCP0Mq6qqPquqqj6rqqo+q6qqPv3mASAAIAlqLQAAQQR0/QAEkAj95AEgACAHai0AAEEEdP0ABJAI/Qw5juM9OY7jPTmO4z05juM9/eYB/eQBIAAgBmotAABBBHT9AASQCP0MJrQXPSa0Fz0mtBc9JrQXPf3mAf3kASAU/QAAEP3mASAE/eQBIQQgCyIAQQJqIgsgBUwNAAsgDCEACyAAIAVIBEADQCAAIAhqLQAAQQR0/QAEkAj9DKuqqj6rqqo+q6qqPquqqj795gEgACAJai0AAEEEdP0ABJAI/eQBIAAgB2otAABBBHT9AASQCP0MOY7jPTmO4z05juM9OY7jPf3mAf3kASAAIAZqLQAAQQR0/QAEkAj9DCa0Fz0mtBc9JrQXPSa0Fz395gH95AEgCiAAQQR0av0AAAD95gEgA/3kASEDIABBAWoiACAFRw0ACwsgDSABQQJ0aiADIAT95AEiA/0fACAD/R8BkiAD/R8CkiAD/R8DkiAOIAFBAXRqLwEAQQ10QYDA//8AcUGAgIDAA2q+lDgCACABQQFqIgEgAkcNAAsLC4UBAQF/IwBBIGsiCyQAIAhBAEoEQCALIAdBBG02AhwgCyAFIAZBAXRqNgIYIAsgACABQQJ0ajYCBCALIAkgCkECdGo2AgAgCyACIANqIgA2AgggCyAAIARBA2xqNgIUIAsgACAEQQF0ajYCECALIAAgBGo2AgxBFCALIAgQEAsgC0EgaiQAC9oFAgR7DH8CQCABIAJODQAgACgCBCEIIAAoAgAhDCAAKAIIIQ4gACgCDCIHQQROBEAgB0EBcSEPIAggB0H8////B3EiCUEEdGohECAHIAlBAXIiEUYhEgNAIA4gASAHbGohCkEAIQBBBCEL/QwAAAAAAAAAAAAAAAAAAAAAIgQhBf0MAAAAAAAAAAAAAAAAAAAAACEG/QwAAAAAAAAAAAAAAAAAAAAAIQMDQCAIIABBBHRqIg39AAAwIAAgCmoiAC0AA0EEdP0ABJAI/eYBIAT95AEhBCAN/QAAICAALQACQQR0/QAEkAj95gEgBf3kASEFIA39AAAQIAAtAAFBBHT9AASQCP3mASAG/eQBIQYgDf0AAAAgAC0AAEEEdP0ABJAI/eYBIAP95AEhAyALIgBBBGoiCyAHTA0ACwJAIAcgCUwNACAPBH8gEP0AAAAgCSAKai0AAEEEdP0ABJAI/eYBIAP95AEhAyARBSAJCyEAIBINAANAIAggAEEBaiILQQR0av0AAAAgCiALai0AAEEEdP0ABJAI/eYBIAggAEEEdGr9AAAAIAAgCmotAABBBHT9AASQCP3mASAD/eQB/eQBIQMgAEECaiIAIAdHDQALCyAMIAFBAnRqIAUgBv3kASAE/eQBIAP95AEiA/0fACAD/R8BkiAD/R8CkiAD/R8DkjgCACABQQFqIgEgAkcNAAsMAQsgB0EASgRAIAdBAUYhCQNAIAwgAUECdGoCeyAI/QAAACAOIAEgB2xqIgAtAABBBHT9AASQCP3mASIDIAkNABogCP0AABAgAC0AAUEEdP0ABJAI/eYBIAP95AEiAyAHQQJGDQAaIAj9AAAgIAAtAAJBBHT9AASQCP3mASAD/eQBCyID/R8AIAP9HwGSIAP9HwKSIAP9HwOSOAIAIAFBAWoiASACRw0ACwwBCyACIAFrQQJ0IgBFDQAgDCABQQJ0akEAIAD8CwALC1IBAX8jAEEQayIIJAAgBUEASgRAIAggBEEEbTYCDCAIIAIgA2o2AgggCCAAIAFBAnRqNgIEIAggBiAHQQJ0ajYCAEETIAggBRAQCyAIQRBqJAAL9g0EA3sOfwN9AX4jAEGAAmsiBiQAAkAgASACTg0AIAAoAgwhDSAAKAIQIg5BAEwEQCACIAFrQQJ0IgBFDQEgDSABQQJ0akEAIAD8CwAMAQsgACgCACABIA5sQdIBbGohByAAKAIIIQ8gACgCBCEQA0BDAAAAACEUQQAhCgNAIActANEBIghBB3YhCSAIQQh0IgsgBy0A0AFyQf8HcSEAIA8gCkECdGoCfQJAIAhBAnZBH3EiCARAIAhBH0cNASAJQR90QYCAgAJBACAAG3JBgICA/AdyvgwCCyAABEAgALNDAACAM5QiFYwgFSALwUEASBsMAgsgCUEfdL4MAQsgCEEXdCAJQR90ciAAQQ10ckGAgIDAA2q+CyEVIBAgCkEIdGohCyoCACEWIAf9AAAgIQQgBiAH/QAAgAEiA/0MMDAwMDAwMDAwMDAwMDAwMP1OIAf9AAAAIgVBBP1t/VD9DODg4ODg4ODg4ODg4ODg4OD9bv0LBEAgBiADQQL9bf0MMDAwMDAwMDAwMDAwMDAwMP1OIARBBP1t/VD9DODg4ODg4ODg4ODg4ODg4OD9bv0LBGAgBiADQQL9a/0MMDAwMDAwMDAwMDAwMDAwMP1OIAT9DA8PDw8PDw8PDw8PDw8PDw/9Tv1Q/Qzg4ODg4ODg4ODg4ODg4ODg/W79CwQgIAYgA0EE/Wv9DDAwMDAwMDAwMDAwMDAwMDD9TiAF/QwPDw8PDw8PDw8PDw8PDw8P/U79UP0M4ODg4ODg4ODg4ODg4ODg4P1u/QsEACAH/QAAMCEEIAYgB/0AAJABIgP9DDAwMDAwMDAwMDAwMDAwMDD9TiAH/QAAECIFQQT9bf1Q/Qzg4ODg4ODg4ODg4ODg4ODg/W79CwRQIAYgA0EC/W39DDAwMDAwMDAwMDAwMDAwMDD9TiAEQQT9bf1Q/Qzg4ODg4ODg4ODg4ODg4ODg/W79CwRwIAYgA0EC/Wv9DDAwMDAwMDAwMDAwMDAwMDD9TiAE/QwPDw8PDw8PDw8PDw8PDw8P/U79UP0M4ODg4ODg4ODg4ODg4ODg4P1u/QsEMCAGIANBBP1r/QwwMDAwMDAwMDAwMDAwMDAw/U4gBf0MDw8PDw8PDw8PDw8PDw8PD/1O/VD9DODg4ODg4ODg4ODg4ODg4OD9bv0LBBAgB/0AAGAhBCAGIAf9AACgASID/QwwMDAwMDAwMDAwMDAwMDAw/U4gB/0AAEAiBUEE/W39UP0M4ODg4ODg4ODg4ODg4ODg4P1u/QsEwAEgBiADQQL9bf0MMDAwMDAwMDAwMDAwMDAwMP1OIARBBP1t/VD9DODg4ODg4ODg4ODg4ODg4OD9bv0LBOABIAYgA0EC/Wv9DDAwMDAwMDAwMDAwMDAwMDD9TiAE/QwPDw8PDw8PDw8PDw8PDw8P/U79UP0M4ODg4ODg4ODg4ODg4ODg4P1u/QsEoAEgBiADQQT9a/0MMDAwMDAwMDAwMDAwMDAwMP1OIAX9DA8PDw8PDw8PDw8PDw8PDw/9Tv1Q/Qzg4ODg4ODg4ODg4ODg4ODg/W79CwSAASAH/QAAcCEEIAYgB/0AALABIgP9DDAwMDAwMDAwMDAwMDAwMDD9TiAH/QAAUCIFQQT9bf1Q/Qzg4ODg4ODg4ODg4ODg4ODg/W79CwTQASAGIANBAv1t/QwwMDAwMDAwMDAwMDAwMDAw/U4gBEEE/W39UP0M4ODg4ODg4ODg4ODg4ODg4P1u/QsE8AEgBiADQQL9a/0MMDAwMDAwMDAwMDAwMDAwMP1OIAT9DA8PDw8PDw8PDw8PDw8PDw/9Tv1Q/Qzg4ODg4ODg4ODg4ODg4ODg/W79CwSwASAGIANBBP1r/QwwMDAwMDAwMDAwMDAwMDAw/U4gBf0MDw8PDw8PDw8PDw8PDw8PD/1O/VD9DODg4ODg4ODg4ODg4ODg4OD9bv0LBJABIAdBwAFqIQxBACEAQQEhCUIAIRcDQCAAQQd0IREgDCAAQQN0aiESQQAhAANAIBIgAEEBdGoiEzAAACAGIABBBXQgEWoiCGr9AAQAIgP9hwEgCCALav0AAAAiBP2HAf26ASAD/YgBIAT9iAH9ugH9rgEiA/0bACAD/RsBaiAD/RsCaiAD/RsDaqx+IBd8IBMwAAEgBiAIQRByIghq/QAEACID/YcBIAggC2r9AAAAIgT9hwH9ugEgA/2IASAE/YgB/boB/a4BIgP9GwAgA/0bAWogA/0bAmogA/0bA2qsfnwhFyAAQQFqIgBBBEcNAAtBASEAIAlBACEJDQALIAdB0gFqIQcgFSAWlCAXtJQgFJIhFCAKQQFqIgogDkcNAAsgDSABQQJ0aiAUOAIAIAFBAWoiASACRw0ACwsgBkGAAmokAAsWACAAIAEgAiADIAQgBSAGIAdBEhAdC/4NBAN7EX8CfQF+IwBBgAJrIgYkAAJAIAEgAk4NACACIAFrQQJ0IgoEQCAAKAIMIAFBAnRqQQAgCvwLAAsgACgCECIOQQBMDQAgACgCDCEPIAAoAhQhECAAKAIAIREgACgCCCESIAAoAgQhEwNAIBMgC0EIdGohDSARIAsgEGwgAWpB0gFsaiEHIBIgC0ECdGoqAgAhGCABIQoDQCAHLQDRASIIQQd2IQkgCEEIdCIMIActANABckH/B3EhAAJ9AkAgCEECdkEfcSIIBEAgCEEfRw0BIAlBH3RBgICAAkEAIAAbckGAgID8B3K+DAILIAAEQCAAs0MAAIAzlCIXjCAXIAzBQQBIGwwCCyAJQR90vgwBCyAIQRd0IAlBH3RyIABBDXRyQYCAgMADar4LIRcgB/0AACAhBCAGIAf9AACAASID/QwwMDAwMDAwMDAwMDAwMDAw/U4gB/0AAAAiBUEE/W39UP0M4ODg4ODg4ODg4ODg4ODg4P1u/QsEQCAGIANBAv1t/QwwMDAwMDAwMDAwMDAwMDAw/U4gBEEE/W39UP0M4ODg4ODg4ODg4ODg4ODg4P1u/QsEYCAGIANBAv1r/QwwMDAwMDAwMDAwMDAwMDAw/U4gBP0MDw8PDw8PDw8PDw8PDw8PD/1O/VD9DODg4ODg4ODg4ODg4ODg4OD9bv0LBCAgBiADQQT9a/0MMDAwMDAwMDAwMDAwMDAwMP1OIAX9DA8PDw8PDw8PDw8PDw8PDw/9Tv1Q/Qzg4ODg4ODg4ODg4ODg4ODg/W79CwQAIAf9AAAwIQQgBiAH/QAAkAEiA/0MMDAwMDAwMDAwMDAwMDAwMP1OIAf9AAAQIgVBBP1t/VD9DODg4ODg4ODg4ODg4ODg4OD9bv0LBFAgBiADQQL9bf0MMDAwMDAwMDAwMDAwMDAwMP1OIARBBP1t/VD9DODg4ODg4ODg4ODg4ODg4OD9bv0LBHAgBiADQQL9a/0MMDAwMDAwMDAwMDAwMDAwMP1OIAT9DA8PDw8PDw8PDw8PDw8PDw/9Tv1Q/Qzg4ODg4ODg4ODg4ODg4ODg/W79CwQwIAYgA0EE/Wv9DDAwMDAwMDAwMDAwMDAwMDD9TiAF/QwPDw8PDw8PDw8PDw8PDw8P/U79UP0M4ODg4ODg4ODg4ODg4ODg4P1u/QsEECAH/QAAYCEEIAYgB/0AAKABIgP9DDAwMDAwMDAwMDAwMDAwMDD9TiAH/QAAQCIFQQT9bf1Q/Qzg4ODg4ODg4ODg4ODg4ODg/W79CwTAASAGIANBAv1t/QwwMDAwMDAwMDAwMDAwMDAw/U4gBEEE/W39UP0M4ODg4ODg4ODg4ODg4ODg4P1u/QsE4AEgBiADQQL9a/0MMDAwMDAwMDAwMDAwMDAwMP1OIAT9DA8PDw8PDw8PDw8PDw8PDw/9Tv1Q/Qzg4ODg4ODg4ODg4ODg4ODg/W79CwSgASAGIANBBP1r/QwwMDAwMDAwMDAwMDAwMDAw/U4gBf0MDw8PDw8PDw8PDw8PDw8PD/1O/VD9DODg4ODg4ODg4ODg4ODg4OD9bv0LBIABIAf9AABwIQQgBiAH/QAAsAEiA/0MMDAwMDAwMDAwMDAwMDAwMP1OIAf9AABQIgVBBP1t/VD9DODg4ODg4ODg4ODg4ODg4OD9bv0LBNABIAYgA0EC/W39DDAwMDAwMDAwMDAwMDAwMDD9TiAEQQT9bf1Q/Qzg4ODg4ODg4ODg4ODg4ODg/W79CwTwASAGIANBAv1r/QwwMDAwMDAwMDAwMDAwMDAw/U4gBP0MDw8PDw8PDw8PDw8PDw8PD/1O/VD9DODg4ODg4ODg4ODg4ODg4OD9bv0LBLABIAYgA0EE/Wv9DDAwMDAwMDAwMDAwMDAwMDD9TiAF/QwPDw8PDw8PDw8PDw8PDw8P/U79UP0M4ODg4ODg4ODg4ODg4ODg4P1u/QsEkAEgB0HAAWohDEEAIQBBASEJQgAhGQNAIABBB3QhFCAMIABBA3RqIRVBACEAA0AgFSAAQQF0aiIWMAAAIAYgAEEFdCAUaiIIav0ABAAiA/2HASAIIA1q/QAAACIE/YcB/boBIAP9iAEgBP2IAf26Af2uASID/RsAIAP9GwFqIAP9GwJqIAP9GwNqrH4gGXwgFjAAASAGIAhBEHIiCGr9AAQAIgP9hwEgCCANav0AAAAiBP2HAf26ASAD/YgBIAT9iAH9ugH9rgEiA/0bACAD/RsBaiAD/RsCaiAD/RsDaqx+fCEZIABBAWoiAEEERw0AC0EBIQAgCUEAIQkNAAsgDyAKQQJ0aiIAIAAqAgAgFyAYlCAZtJSSOAIAIAdB0gFqIQcgCkEBaiIKIAJHDQALIAtBAWoiCyAORw0ACwsgBkGAAmokAAsWACAAIAEgAiADIAQgBSAGIAdBERAdC6gBAgR/AX0CQCABIAJODQAgACgCDCIFQQBKBEAgACgCACABIAVsQbABbGohAyAAKAIEIQYDQEMAAAAAIQdBACEEA0AgAyAGIARBCnRqED0gB5IhByADQbABaiEDIARBAWoiBCAFRw0ACyAAKAIIIAFBAnRqIAc4AgAgAUEBaiIBIAJHDQALDAELIAIgAWtBAnQiAkUNACAAKAIIIAFBAnRqQQAgAvwLAAsLZQEBfyMAQSBrIggkAAJAIARBAEwNACAFQQBMDQAgCCAFNgIcIAggBEEIdjYCGCAIIAIgA2o2AgwgCCAGIAdBAnRqNgIUIAggACABQQJ0ajYCEEEQIAhBDGogBRAQCyAIQSBqJAALrwEBCX8CQCABIAJODQAgAiABa0ECdCIDBEAgACgCCCABQQJ0akEAIAP8CwALIAAoAgwiBkEATA0AIAAoAhAhByAAKAIAIQggACgCBCEJA0AgCSAEQQp0aiEKIAggBCAHbCABakGwAWxqIQUgASEDA0AgACgCCCADQQJ0aiILIAsqAgAgBSAKED2SOAIAIAVBsAFqIQUgA0EBaiIDIAJHDQALIARBAWoiBCAGRw0ACwsLZQEBfyMAQSBrIggkAAJAIARBAEwNACAFQQBMDQAgCCAFNgIcIAggBEEIdjYCGCAIIAIgA2o2AgwgCCAGIAdBAnRqNgIUIAggACABQQJ0ajYCEEEPIAhBDGogBRAQCyAIQSBqJAALugECBX8BfQJAIAEgAk4NACAAKAIQIgVBAEoEQCAAKAIAIAEgBWxBkAFsaiEEIAAoAgghBiAAKAIEIQcDQEMAAAAAIQhBACEDA0AgBCAHIANBCHRqIAYgA0ECdGoqAgAQPiAIkiEIIARBkAFqIQQgA0EBaiIDIAVHDQALIAAoAgwgAUECdGogCDgCACABQQFqIgEgAkcNAAsMAQsgAiABa0ECdCICRQ0AIAAoAgwgAUECdGpBACAC/AsACwsWACAAIAEgAiADIAQgBSAGIAdBDhAdC8sBAgp/An0CQCABIAJODQAgAiABa0ECdCIDBEAgACgCDCABQQJ0akEAIAP8CwALIAAoAhAiBkEATA0AIAAoAhQhByAAKAIAIQggACgCCCEJIAAoAgQhCgNAIAogBEEIdGohCyAIIAQgB2wgAWpBkAFsaiEFIAkgBEECdGoqAgAhDSABIQMDQCAFIAsgDRA+IQ4gACgCDCADQQJ0aiIMIA4gDCoCAJI4AgAgBUGQAWohBSADQQFqIgMgAkcNAAsgBEEBaiIEIAZHDQALCwspACAAKAIkBEAgACABIAJBACAAKAIcEDwPCyAAQQAgACgCGCABIAIQPAsWACAAIAEgAiADIAQgBSAGIAdBDRAdC7MEAwJ7C38BfQJAIAEgAk4NACAAKAIIIQ0gACgCDCIJQQBMBEAgAiABa0ECdCIARQ0BIA0gAUECdGpBACAA/AsADwsgACgCACABIAlsQSJsaiEKIAlBImwhDiAAKAIEIQ8DQEMAAAAAIRBBACELIAohAANAIA8gC0EHdGohBiAALQABIgdBGHRBgICAgHhxIQUgAC0AACAHQQh0ciIIQf8HcSEMIAhB//8DcSEIAkACQCAHQQJ2QR9xIgcEQCAHQR9HDQEgCEENdCAFckGAgID8B3IhBQwCCyAMRQ0BIAVBgICAmARyIAxnIgVBF3RrIAhBCSAFQR9za3RBDnRBgID/A3FyIQUMAQsgB0EXdCAMQQ10ciAFakGAgIDAA2ohBQsgBv0AAAAgAP0AAAIiA/2HASIE/acB/foB/eYBIAb9AAAQIAT9qAH9+gH95gH95AEgBv0AACAgA/2IASID/acB/foB/eYB/eQBIAb9AAAwIAP9qAH9+gH95gH95AEgBv0AAEAgAP0AABIiA/2HASIE/acB/foB/eYB/eQBIAb9AABQIAT9qAH9+gH95gH95AEgBv0AAGAgA/2IASID/acB/foB/eYB/eQBIAb9AABwIAP9qAH9+gH95gH95AEiA/0fACAD/R8BkiAD/R8CkiAD/R8DkiAFvpQgEJIhECAAQSJqIQAgC0EBaiILIAlHDQALIA0gAUECdGogEDgCACAKIA5qIQogAUEBaiIBIAJHDQALCwuBAQEBfyMAQSBrIggkAAJAIAVBAEwNACAEQQBMBEAgBUECdCIARQ0BIAYgB0ECdGpBACAA/AsADAELIAggBTYCHCAIIARBBXY2AhggCCACIANqNgIMIAggBiAHQQJ0ajYCFCAIIAAgAUECdGo2AhBBDCAIQQxqIAUQEAsgCEEgaiQAC6gEAgJ7DH8CQCABIAJODQAgAiABa0ECdCIFBEAgACgCCCABQQJ0akEAIAX8CwALIAAoAgwiDEEATA0AIAAoAgghDSAAKAIQIQ4gACgCACEPIAAoAgQhEANAIBAgCkEHdGohBSAPIAogDmwgAWpBImxqIQcgASEAA0AgBy0AASIIQRh0QYCAgIB4cSEGIActAAAgCEEIdHIiC0H/B3EhCSALQf//A3EhCwJAAkAgCEECdkEfcSIIBEAgCEEfRw0BIAtBDXQgBnJBgICA/AdyIQYMAgsgCUUNASAGQYCAgJgEciAJZyIGQRd0ayALQQkgBkEfc2t0QQ50QYCA/wNxciEGDAELIAhBF3QgCUENdHIgBmpBgICAwANqIQYLIA0gAEECdGoiCSAF/QAAACAH/QAAAiID/YcBIgT9pwH9+gH95gEgBf0AABAgBP2oAf36Af3mAf3kASAF/QAAICAD/YgBIgP9pwH9+gH95gH95AEgBf0AADAgA/2oAf36Af3mAf3kASAF/QAAQCAH/QAAEiID/YcBIgT9pwH9+gH95gH95AEgBf0AAFAgBP2oAf36Af3mAf3kASAF/QAAYCAD/YgBIgP9pwH9+gH95gH95AEgBf0AAHAgA/2oAf36Af3mAf3kASID/R8AIAP9HwGSIAP9HwKSIAP9HwOSIAa+lCAJKgIAkjgCACAHQSJqIQcgAEEBaiIAIAJHDQALIApBAWoiCiAMRw0ACwsLgQEBAX8jAEEgayIIJAACQCAFQQBMDQAgBEEATARAIAVBAnQiAEUNASAGIAdBAnRqQQAgAPwLAAwBCyAIIAU2AhwgCCAEQQV2NgIYIAggAiADajYCDCAIIAYgB0ECdGo2AhQgCCAAIAFBAnRqNgIQQQsgCEEMaiAFEBALIAhBIGokAAuqBwMPfwR7An0CQCABIAJODQAgACgCCCENIAAoAhAiCkEATARAIAIgAWtBAnQiAEUNASANIAFBAnRqQQAgAPwLAA8LIAAoAgAgASAKbEEYbGohCyAKQRhsIQ8gACgCDCEQIAAoAgQhEQNAQwAAAAAhFkEAIQkgCyEGA0AgBi0AASIOQRh0QYCAgIB4cSEHIAYtAANBCHQiBSAGLQACciEAIAYtAAAgDkEIdHIiA0H/B3EhDCADQf//A3EhBCAQIAlBAnRqAkACQCAOQQJ2QR9xIgMEQCADQR9HDQEgBEENdCAHckGAgID8B3IhBwwCCyAMRQ0BIAdBgICAmARyIAxnIgNBF3RrIARBCSADQR9za3RBDnRBgID/A3FyIQcMAQsgA0EXdCAMQQ10ciAHakGAgIDAA2ohBwsgESAJQQd0aiEEKgIAIABB/wdxIQggBUEQdEGAgICAeHEhBQJAAkAgAEEKdkEfcSIDBEAgA0EfRw0BIABBDXQgBXJBgICA/AdyIQUMAgsgCEUNASAFQYCAgJgEciAIZyIDQRd0ayAAQQkgA0Efc2t0QQ50QYCA/wNxciEFDAELIANBF3QgCEENdHIgBWpBgICAwANqIQULIAW+lCAWkiAE/QAAUCAG/QkABCIU/QwCAgICAgICAgMDAwMDAwMD/Q79DAECBAgQIECAAQIECBAgQID9Tv0MAAAAAAAAAAAAAAAAAAAAAP0k/QwQEBAQEBAQEBAQEBAQEBAQ/U4gBv0AAAgiFUEE/W39UCIT/YkBIhL9qgH9+gH95gEgBP0AAEAgEv2pAf36Af3mAf3kASAE/QAAYCAT/YoBIhL9qQH9+gH95gH95AEgBP0AAHAgEv2qAf36Af3mAf3kASAE/QAAECAV/QwPDw8PDw8PDw8PDw8PDw8P/U4gFP0MAAAAAAAAAAABAQEBAQEBAf0O/QwBAgQIECBAgAECBAgQIECA/U79DAAAAAAAAAAAAAAAAAAAAAD9JP0MEBAQEBAQEBAQEBAQEBAQEP1O/VAiE/2JASIS/aoB/foB/eYB/eQBIAT9AAAAIBL9qQH9+gH95gH95AEgBP0AACAgE/2KASIS/akB/foB/eYB/eQBIAT9AAAwIBL9qgH9+gH95gH95AEiEv0fACAS/R8BkiAS/R8CkiAS/R8DkiAHvpSSIRYgBkEYaiEGIAlBAWoiCSAKRw0ACyANIAFBAnRqIBY4AgAgCyAPaiELIAFBAWoiASACRw0ACwsLFgAgACABIAIgAyAEIAUgBiAHQQoQHgujBwIQfwR7AkAgASACTg0AIAIgAWtBAnQiCQRAIAAoAgggAUECdGpBACAJ/AsACyAAKAIQIgtBAEwNACAAKAIIIQwgACgCDCENIAAoAhQhDiAAKAIAIQ8gACgCBCEQA0AgDSAKQQJ0aiERIBAgCkEHdGohBiAPIAogDmwgAWpBGGxqIQcgASEJA0AgBy0AASIEQRh0QYCAgIB4cSEFIActAANBCHQiEiAHLQACciEAIActAAAgBEEIdHIiCEH/B3EhAyAIQf//A3EhCAJAAkAgBEECdkEfcSIEBEAgBEEfRw0BIAhBDXQgBXJBgICA/AdyIQUMAgsgA0UNASAFQYCAgJgEciADZyIFQRd0ayAIQQkgBUEfc2t0QQ50QYCA/wNxciEFDAELIARBF3QgA0ENdHIgBWpBgICAwANqIQULIABB/wdxIQQgEkEQdEGAgICAeHEhAwJAAkAgAEEKdkEfcSIIBEAgCEEfRw0BIABBDXQgA3JBgICA/AdyIQMMAgsgBEUNASADQYCAgJgEciAEZyIDQRd0ayAAQQkgA0Efc2t0QQ50QYCA/wNxciEDDAELIAhBF3QgBEENdHIgA2pBgICAwANqIQMLIAwgCUECdGoiACAAKgIAIBEqAgAgA76UkiAG/QAAUCAH/QkABCIT/QwCAgICAgICAgMDAwMDAwMD/Q79DAECBAgQIECAAQIECBAgQID9Tv0MAAAAAAAAAAAAAAAAAAAAAP0k/QwQEBAQEBAQEBAQEBAQEBAQ/U4gB/0AAAgiFEEE/W39UCIV/YkBIhb9qgH9+gH95gEgBv0AAEAgFv2pAf36Af3mAf3kASAG/QAAYCAV/YoBIhX9qQH9+gH95gH95AEgBv0AAHAgFf2qAf36Af3mAf3kASAG/QAAECAU/QwPDw8PDw8PDw8PDw8PDw8P/U4gE/0MAAAAAAAAAAABAQEBAQEBAf0O/QwBAgQIECBAgAECBAgQIECA/U79DAAAAAAAAAAAAAAAAAAAAAD9JP0MEBAQEBAQEBAQEBAQEBAQEP1O/VAiE/2JASIU/aoB/foB/eYB/eQBIAb9AAAAIBT9qQH9+gH95gH95AEgBv0AACAgE/2KASIT/akB/foB/eYB/eQBIAb9AAAwIBP9qgH9+gH95gH95AEiE/0fACAT/R8BkiAT/R8CkiAT/R8DkiAFvpSSOAIAIAdBGGohByAJQQFqIgkgAkcNAAsgCkEBaiIKIAtHDQALCwsWACAAIAEgAiADIAQgBSAGIAdBCRAeC5QGAwx/BHsCfQJAIAEgAk4NACAAKAIIIQsgACgCECIIQQBMBEAgAiABa0ECdCIARQ0BIAsgAUECdGpBACAA/AsADwsgACgCACABIAhsQRZsaiEJIAhBFmwhDCAAKAIMIQ0gACgCBCEOA0BDAAAAACETQQAhBiAJIQADQCAOIAZBB3RqIQQgAC0AASIFQRh0QYCAgIB4cSEDIAAtAAAgBUEIdHIiB0H/B3EhCiAHQf//A3EhByANIAZBAnRqKgIAIRQCQAJAIAVBAnZBH3EiBQRAIAVBH0cNASAHQQ10IANyQYCAgPwHciEDDAILIApFDQEgA0GAgICYBHIgCmciA0EXdGsgB0EJIANBH3NrdEEOdEGAgP8DcXIhAwwBCyAFQRd0IApBDXRyIANqQYCAgMADaiEDCyAE/QAAUCAA/QkAAiIP/QwCAgICAgICAgMDAwMDAwMD/Q79DAECBAgQIECAAQIECBAgQID9Tv0MAAAAAAAAAAAAAAAAAAAAAP0k/QwQEBAQEBAQEBAQEBAQEBAQ/U4gAP0AAAYiEEEE/W39UCIR/YkBIhL9qgH9+gH95gEgBP0AAEAgEv2pAf36Af3mAf3kASAE/QAAYCAR/YoBIhH9qQH9+gH95gH95AEgBP0AAHAgEf2qAf36Af3mAf3kASAE/QAAECAQ/QwPDw8PDw8PDw8PDw8PDw8P/U4gD/0MAAAAAAAAAAABAQEBAQEBAf0O/QwBAgQIECBAgAECBAgQIECA/U79DAAAAAAAAAAAAAAAAAAAAAD9JP0MEBAQEBAQEBAQEBAQEBAQEP1O/VAiD/2JASIQ/aoB/foB/eYB/eQBIAT9AAAAIBD9qQH9+gH95gH95AEgBP0AACAgD/2KASIP/akB/foB/eYB/eQBIAT9AAAwIA/9qgH9+gH95gH95AEiD/0fASAUQwAAgMGUkiAP/R8AkiAP/R8CkiAP/R8DkiADvpQgE5IhEyAAQRZqIQAgBkEBaiIGIAhHDQALIAsgAUECdGogEzgCACAJIAxqIQkgAUEBaiIBIAJHDQALCwubAQEBfyMAQTBrIgwkAAJAIAlBAEwNACAKQQBMDQAgDCAINgIcIAwgBTYCGCAMIAI2AhQgDCAJNgIgIAwgCjYCJCAMIAkgCk82AiwgDCALQQAgC0EAShs2AiggDCAGIAdBAnRqNgIQIAwgAyAEQQJ0ajYCDCAMIAAgAUECdGo2AghBASAMQQhqIAkgCiAJIApLGxAQCyAMQTBqJAALFgAgACABIAIgAyAEIAUgBiAHQQgQHguJBgIOfwR7AkAgASACTg0AIAIgAWtBAnQiAwRAIAAoAgggAUECdGpBACAD/AsACyAAKAIQIgpBAEwNACAAKAIIIQsgACgCDCEMIAAoAhQhDSAAKAIAIQ4gACgCBCEPA0AgDCAGQQJ0aiEQIA8gBkEHdGohAyAOIAYgDWwgAWpBFmxqIQUgASEAA0AgBS0AASIHQRh0QYCAgIB4cSEEIAUtAAAgB0EIdHIiCUH/B3EhCCAJQf//A3EhCQJAAkAgB0ECdkEfcSIHBEAgB0EfRw0BIAlBDXQgBHJBgICA/AdyIQQMAgsgCEUNASAEQYCAgJgEciAIZyIEQRd0ayAJQQkgBEEfc2t0QQ50QYCA/wNxciEEDAELIAdBF3QgCEENdHIgBGpBgICAwANqIQQLIAsgAEECdGoiCCAD/QAAUCAF/QkAAiIR/QwCAgICAgICAgMDAwMDAwMD/Q79DAECBAgQIECAAQIECBAgQID9Tv0MAAAAAAAAAAAAAAAAAAAAAP0k/QwQEBAQEBAQEBAQEBAQEBAQ/U4gBf0AAAYiEkEE/W39UCIT/YkBIhT9qgH9+gH95gEgA/0AAEAgFP2pAf36Af3mAf3kASAD/QAAYCAT/YoBIhP9qQH9+gH95gH95AEgA/0AAHAgE/2qAf36Af3mAf3kASAD/QAAECAS/QwPDw8PDw8PDw8PDw8PDw8P/U4gEf0MAAAAAAAAAAABAQEBAQEBAf0O/QwBAgQIECBAgAECBAgQIECA/U79DAAAAAAAAAAAAAAAAAAAAAD9JP0MEBAQEBAQEBAQEBAQEBAQEP1O/VAiEf2JASIS/aoB/foB/eYB/eQBIAP9AAAAIBL9qQH9+gH95gH95AEgA/0AACAgEf2KASIR/akB/foB/eYB/eQBIAP9AAAwIBH9qgH9+gH95gH95AEiEf0fASAQKgIAQwAAgMGUkiAR/R8AkiAR/R8CkiAR/R8DkiAEvpQgCCoCAJI4AgAgBUEWaiEFIABBAWoiACACRw0ACyAGQQFqIgYgCkcNAAsLCxYAIAAgASACIAMgBCAFIAYgB0EHEB4L8QQDC38DewF9AkAgASACTg0AIAAoAgghCyAAKAIMIgdBAEwEQCACIAFrQQJ0IgBFDQEgCyABQQJ0akEAIAD8CwAPCyAAKAIAIAEgB2xBEmxqIQggB0ESbCEMIAAoAgQhDQNAQwAAAAAhEUEAIQkgCCEAA0AgDSAJQQd0aiEEIAAtAAEiBUEYdEGAgICAeHEhAyAALQAAIAVBCHRyIgZB/wdxIQogBkH//wNxIQYCQAJAIAVBAnZBH3EiBQRAIAVBH0cNASAGQQ10IANyQYCAgPwHciEDDAILIApFDQEgA0GAgICYBHIgCmciA0EXdGsgBkEJIANBH3NrdEEOdEGAgP8DcXIhAwwBCyAFQRd0IApBDXRyIANqQYCAgMADaiEDCyAE/QAAACAA/QAAAiIO/QwPDw8PDw8PDw8PDw8PDw8P/U79DPj4+Pj4+Pj4+Pj4+Pj4+Pj9biIP/YcBIhD9pwH9+gH95gEgBP0AABAgEP2oAf36Af3mAf3kASAE/QAAICAP/YgBIg/9pwH9+gH95gH95AEgBP0AADAgD/2oAf36Af3mAf3kASAE/QAAQCAOQQT9bf0M+Pj4+Pj4+Pj4+Pj4+Pj4+P1uIg79hwEiD/2nAf36Af3mAf3kASAE/QAAUCAP/agB/foB/eYB/eQBIAT9AABgIA79iAEiDv2nAf36Af3mAf3kASAE/QAAcCAO/agB/foB/eYB/eQBIg79HwAgDv0fAZIgDv0fApIgDv0fA5IgA76UIBGSIREgAEESaiEAIAlBAWoiCSAHRw0ACyALIAFBAnRqIBE4AgAgCCAMaiEIIAFBAWoiASACRw0ACwsLBAAjAAsQACMAIABrQXBxIgAkACAACwYAIAAkAAsnACAAJAMgASQEIAIkBSADJAYCQCACRQ0AIARFDQBBpD4gBDYCAAsLCgAgACQIIAEkBwsEAEIAC4EBAQF/IwBBIGsiCCQAAkAgBUEATA0AIARBAEwEQCAFQQJ0IgBFDQEgBiAHQQJ0akEAIAD8CwAMAQsgCCAFNgIcIAggBEEFdjYCGCAIIAIgA2o2AgwgCCAGIAdBAnRqNgIUIAggACABQQJ0ajYCEEEGIAhBDGogBRAQCyAIQSBqJAAL8AIBB38jAEEgayIDJAAgAyAAKAIcIgQ2AhAgACgCFCEFIAMgAjYCHCADIAE2AhggAyAFIARrIgE2AhQgASACaiEFQQIhBwJ/AkACQAJAIAAoAjwgA0EQaiIBQQIgA0EMahACIgQEfyMBIAQ2AgBBfwVBAAsEQCABIQQMAQsDQCAFIAMoAgwiBkYNAiAGQQBIBEAgASEEDAQLIAFBCEEAIAYgASgCBCIISyIJG2oiBCAGIAhBACAJG2siCCAEKAIAajYCACABQQxBBCAJG2oiASABKAIAIAhrNgIAIAUgBmshBSAAKAI8IAQiASAHIAlrIgcgA0EMahACIgYEfyMBIAY2AgBBfwVBAAtFDQALCyAFQX9HDQELIAAgACgCLCIBNgIcIAAgATYCFCAAIAEgACgCMGo2AhAgAgwBCyAAQQA2AhwgAEIANwMQIAAgACgCAEEgcjYCAEEAIAdBAkYNABogAiAEKAIEawsgA0EgaiQACwQAQQALIAEBfyMDKAJ4IgBBAf4XAgAgABAkIABBAUEA/kgCABoL5gQCDH8DewJAIAEgAk4NACACIAFrQQJ0IgMEQCAAKAIIIAFBAnRqQQAgA/wLAAsgACgCDCIKQQBMDQAgACgCCCELIAAoAhAhDCAAKAIAIQ0gACgCBCEOA0AgDiAIQQd0aiEDIA0gCCAMbCABakESbGohBSABIQADQCAFLQABIgZBGHRBgICAgHhxIQQgBS0AACAGQQh0ciIJQf8HcSEHIAlB//8DcSEJAkACQCAGQQJ2QR9xIgYEQCAGQR9HDQEgCUENdCAEckGAgID8B3IhBAwCCyAHRQ0BIARBgICAmARyIAdnIgRBF3RrIAlBCSAEQR9za3RBDnRBgID/A3FyIQQMAQsgBkEXdCAHQQ10ciAEakGAgIDAA2ohBAsgCyAAQQJ0aiIHIAP9AAAAIAX9AAACIg/9DA8PDw8PDw8PDw8PDw8PDw/9Tv0M+Pj4+Pj4+Pj4+Pj4+Pj4+P1uIhD9hwEiEf2nAf36Af3mASAD/QAAECAR/agB/foB/eYB/eQBIAP9AAAgIBD9iAEiEP2nAf36Af3mAf3kASAD/QAAMCAQ/agB/foB/eYB/eQBIAP9AABAIA9BBP1t/Qz4+Pj4+Pj4+Pj4+Pj4+Pj4/W4iD/2HASIQ/acB/foB/eYB/eQBIAP9AABQIBD9qAH9+gH95gH95AEgA/0AAGAgD/2IASIP/acB/foB/eYB/eQBIAP9AABwIA/9qAH9+gH95gH95AEiD/0fACAP/R8BkiAP/R8CkiAP/R8DkiAEvpQgByoCAJI4AgAgBUESaiEFIABBAWoiACACRw0ACyAIQQFqIgggCkcNAAsLCwQAIwMLGgAgAEEA/kECAEEDRgRAIABB/////wcQEQsLgQEBAX8jAEEgayIIJAACQCAFQQBMDQAgBEEATARAIAVBAnQiAEUNASAGIAdBAnRqQQAgAPwLAAwBCyAIIAU2AhwgCCAEQQV2NgIYIAggAiADajYCDCAIIAYgB0ECdGo2AhQgCCAAIAFBAnRqNgIQQQUgCEEMaiAFEBALIAhBIGokAAsCAAvrBAEGfyMDIgFBAToAKCABIAA2AkAgAUEAOgApIAFBAf4lAnxBAWsiAARAIAFB/ABqIQIDQCACIABEAAAAAAAA8H8QGhogAv4QAgAiAA0ACwsgASgCeBA1AkAgASgCeCIA/hACAEUEQCAAEDcMAQtB7D0QDxogAEHoPTYCOCAAQZw+KAIANgI0QZw+IAA2AgAgACgCNCAANgI4Qew9EA4LIwMhAgNAIAIoAkQiAARAIAAoAgQgACgCACACIAAoAgg2AkQRAAAMAQsLAkAjAyIALQAqQQFxRQ0AA0AQQiAAIAAtACpB/gFxOgAqQQAhAgNAIAJBAnQiAygCwEchBCAAKAJIIANqIgYoAgAhAyAGQQA2AgACQCADRQ0AIARFDQAgBEEqRg0AECcgAyAEEQAAEEILIAJBAWoiAkGAAUcNAAsQJyAALQAqQQFxRQ0BIAVBA0kgBUEBaiEFDQALCxAsQQBBAf4eAvRCGgJAIAEoAkwiAEUNACAAIAFBzABqIgJGDQADQCAAQQhrKAIAIAEgADYCVCABIAAoAgA2AkwgAEEMayIAQYCAgIAE/kECACEDIAFBADYCVEUgA0EATnFFBEAgAEEBEBELIAIoAgAiAEUNASAAIAJHDQALCxBFQazBAEGswQAoAgBBAWsiADYCACAARQRAQavBAEEAOgAACyABKAIMIgAgASgCCCICNgIIIAIgADYCDCABIAE2AgggASABNgIMECsjBUUEQEEAJANBACQEQQAkBUEBJAYgAUEgaiIAQQJBAf5IAgBBA0YEQCABEAEPCyAAQQD+FwIAIABBARARDwtBABAEAAsSACAAKAIsIgBBhAEQRiAAEBML+hUDEn8EewF9IwBBgBBrIgwkAAJAIAAoAhgiCUEATA0AIAEgAk4NACAAKAIIIAFBAnRqIQUgAiABa0ECdCEEIAAoAhRBAnQhByAJQQRPBEAgCUH8////B3EhCwNAIARFIgpFBEAgBSADIAdsakEAIAT8CwALIApFBEAgBSAHIANBAXJsakEAIAT8CwALIApFBEAgBSAHIANBAnJsakEAIAT8CwALIApFBEAgBSAHIANBA3JsakEAIAT8CwALIANBBGohAyAIQQRqIgggC0cNAAsLIAlBA3EiCkUNAANAIAQEQCAFIAMgB2xqQQAgBPwLAAsgA0EBaiEDIAZBAWoiBiAKRw0ACwsCQCAAKAIgIgtBAEwNACAJQQFGBEAgAUEBdCENIAAoAgggAUECdGohCSAAKAIAIQ4gAiABayIKQQhIIRBBACEHA0AgACgCBCAAKAIQIAdsaiANaiEIIA4gB0ECdGoqAgAhGUEAIQUgEEUEQCAZ/RMhFkEAIQRBCCEGA0AgCSAEQQJ0aiIDIBYgCCAEQQF0av0AAAAiF/2pAUEN/asBIhX9DADg/w8A4P8PAOD/DwDg/w/9TiAV/QwAAIAPAACADwAAgA8AAIAP/U4iFf0MAACADwAAgA8AAIAPAACAD/03/QwAAAA4AAAAOAAAADgAAAA4/U79rgEiGP0MAACAOAAAgDgAAIA4AACAOP2uAf0MAACAuAAAgLgAAIC4AACAuP3kASAY/QwAAAA4AAAAOAAAADgAAAA4/a4BIBX9DAAAAAAAAAAAAAAAAAAAAAD9N/1SIBf9pwH9DAAAAIAAAACAAAAAgAAAAID9Tv1Q/eYBIAP9AAAA/eQB/QsAACADIBYgF/2qAUEN/asBIhX9DADg/w8A4P8PAOD/DwDg/w/9TiAV/QwAAIAPAACADwAAgA8AAIAP/U4iFf0MAACADwAAgA8AAIAPAACAD/03/QwAAAA4AAAAOAAAADgAAAA4/U79rgEiGP0MAACAOAAAgDgAAIA4AACAOP2uAf0MAACAuAAAgLgAAIC4AACAuP3kASAY/QwAAAA4AAAAOAAAADgAAAA4/a4BIBX9DAAAAAAAAAAAAAAAAAAAAAD9N/1SIBf9qAH9DAAAAIAAAACAAAAAgAAAAID9Tv1Q/eYBIAP9AAAQ/eQB/QsAECAGIgUiBEEIaiIGIApMDQALCwJAIAUgCk4NAAJAIAIgASAFamsiBkEESQRAIAUhAwwBCyAFIAZBfHEiC2ohAyAZ/RMhF0EAIQQDQCAJIAQgBWoiEUECdGoiEiAXIAggEUEBdGr9XQAAIhb9DP9//3//f/9//3//f/9//3/9Tv2pAUEN/asBIhUgFf0MAACADwAAgA8AAIAPAACAD/1OIhX9DAAAgA8AAIAPAACADwAAgA/9N/0MAAAAOAAAADgAAAA4AAAAOP1O/a4BIhj9DAAAgDgAAIA4AACAOAAAgDj9rgH9DAAAgLgAAIC4AACAuAAAgLj95AEgGP0MAAAAOAAAADgAAAA4AAAAOP2uASAV/QwAAAAAAAAAAAAAAAAAAAAA/Tf9UiAW/acB/QwAAACAAAAAgAAAAIAAAACA/U79UP3mASAS/QACAP3kAf0LAgAgBEEEaiIEIAtHDQALIAYgC0YNAQsDQCAJIANBAnRqIgQgGSAIIANBAXRqLwAAIgVB//8BcUENdCIGQYCAgMADQQAgBkGAgID8AHEiBkGAgID8AEYbaiILQYCAgMADaiALQYCAgMQDar5DAACAuJK8IAYbIAVBEHRBgICAgHhxcr6UIAQqAgCSOAIAIANBAWoiAyAKRw0ACwsgB0EBaiIHIAAoAiBIDQALDAELIAEgAk4NACACIAFrIREDQCALQQBKBEBBgAQgAiABayIOIA5BgAROGyEGQYAEIBEgDUEJdGsiAyADQYAEThsiEEEDcSEKIAFBAXQhEkEAIQcgDkEHSiEUA0AgACgCBCAAKAIQIAdsaiASaiEIQQghBEEAIQMgFARAA0AgDCADQQJ0aiIFIAggA0EBdGr9AAAAIhf9qgFBDf2rASIW/QwA4P8PAOD/DwDg/w8A4P8P/U4gFv0MAACADwAAgA8AAIAPAACAD/1OIhb9DAAAgA8AAIAPAACADwAAgA/9N/0MAAAAOAAAADgAAAA4AAAAOP1O/a4BIhX9DAAAgDgAAIA4AACAOAAAgDj9rgH9DAAAgLgAAIC4AACAuAAAgLj95AEgFf0MAAAAOAAAADgAAAA4AAAAOP2uASAW/QwAAAAAAAAAAAAAAAAAAAAA/Tf9UiAX/agB/QwAAACAAAAAgAAAAIAAAACA/U79UP0LBBAgBSAX/akBQQ39qwEiFv0MAOD/DwDg/w8A4P8PAOD/D/1OIBb9DAAAgA8AAIAPAACADwAAgA/9TiIW/QwAAIAPAACADwAAgA8AAIAP/Tf9DAAAADgAAAA4AAAAOAAAADj9Tv2uASIV/QwAAIA4AACAOAAAgDgAAIA4/a4B/QwAAIC4AACAuAAAgLgAAIC4/eQBIBX9DAAAADgAAAA4AAAAOAAAADj9rgEgFv0MAAAAAAAAAAAAAAAAAAAAAP03/VIgF/2nAf0MAAAAgAAAAIAAAACAAAAAgP1O/VD9CwQAIAQiA0EIaiIEIAZMDQALCwJAIAMgBk4NAAJAIBAgA2siBEEESQRAIAMhBAwBCyADIAQgCmsiD2ohBEEAIQUDQCAMIAMgBWoiE0ECdGogCCATQQF0av1dAAAiF/0M/3//f/9//3//f/9//3//f/1O/akBQQ39qwEiFiAW/QwAAIAPAACADwAAgA8AAIAP/U4iFv0MAACADwAAgA8AAIAPAACAD/03/QwAAAA4AAAAOAAAADgAAAA4/U79rgEiFf0MAACAOAAAgDgAAIA4AACAOP2uAf0MAACAuAAAgLgAAIC4AACAuP3kASAV/QwAAAA4AAAAOAAAADgAAAA4/a4BIBb9DAAAAAAAAAAAAAAAAAAAAAD9N/1SIBf9pwH9DAAAAIAAAACAAAAAgAAAAID9Tv1Q/QsCACAFQQRqIgUgD0cNAAsgCkUNAQsDQCAMIARBAnRqIAggBEEBdGovAAAiA0H//wFxQQ10IgVBgICAwANBACAFQYCAgPwAcSIFQYCAgPwARhtqIg9BgICAwANqIA9BgICAxANqvkMAAIC4krwgBRsgA0EQdEGAgICAeHFyNgIAIARBAWoiBCAGRw0ACwtBACEIIAlBAEoEQANAIAAoAgggACgCFCAIbEECdGogAUECdGohCSAAKAIAIAAoAgwgCGxBAnRqIAdBAnRqKgIAIRlBACEDIA5BBE4EQCAZ/RMhF0EAIQRBBCEFA0AgCSAEQQJ0IgNqIgQgAyAMav0ABAAgF/3mASAE/QAAAP3kAf0LAAAgBSIDIQQgA0EEaiIFIAZMDQALCwJAIAMgBk4NAAJAIBAgA2siBEEESQRAIAMhBAwBCyADIAQgCmsiC2ohBCAZ/RMhF0EAIQUDQCAJIAMgBWpBAnQiD2oiEyAT/QACACAMIA9q/QACACAX/eYB/eQB/QsCACAFQQRqIgUgC0cNAAsgCkUNAQsDQCAJIARBAnQiA2oiBSAFKgIAIAMgDGoqAgAgGZSSOAIAIARBAWoiBCAGRw0ACwsgCEEBaiIIIAAoAhgiCUgNAAsgACgCICELCyAHQQFqIgcgC0gNAAsLIA1BAWohDSABQYAEaiIBIAJIDQALCyAMQYAQaiQACwwAQeTCACMD/hcCAAsmAQJ/QaDBACgCAEEAEB8gACEBA0AgASgCWCABEC8iASAARw0ACwuGAQEBfyMAQTBrIgwkAAJAIAlBAEwNACAKQQBMDQAgDCAKNgIoIAwgCTYCJCAMIAg2AiAgDCAFNgIcIAwgAjYCGCAMIAMgBGo2AhAgDCALQQAgC0EAShs2AiwgDCAGIAdBAnRqNgIUIAwgACABQQJ0ajYCDEEEIAxBDGogChAQCyAMQTBqJAALvQEAAkACQAJAQejXAEEAQQH+SAIADgIAAQILQYAIJAFBgAhBAEEF/AsAQZAIQQBBxAr8CAAAQdQSQQBBHvwLAEHyEkEAQfYq/AgBACMJBEAAC0HoPUEAQTT8CwBBnD5BAEHlAPwIAgBBgT9BAEE//AsAQcA/QQBBCPwIAwBB0D9BAEGYGPwLAEHo1wBBAv4XAgBB6NcAQX/+AAIAGgwBC0Ho1wBBAUJ//gECABoL/AkA/AkBQQEkCfwJAvwJAwuJAQBB8NcEJAhB8NcAJAdBgMIAQQI2AgBB4MEAQeDBADYCAEGswgBBrMIANgIAQcDCAEHIwQA2AgBB+MEAQSo2AgBBqMIAQZDDADYCAEHswQBB4MEANgIAQejBAEHgwQA2AgBB4MEAEAlBlMIAIwg2AgBBmMIAIwgjB2s2AgBB4MEAEEFB4MEAEAgLC7I2BAHECgAAgL8AAIC/AACAvwAAgL8AAAAAAACAvwAAgL8AAIC/AACAPwAAgL8AAIC/AACAvwAAAEAAAIC/AACAvwAAgL8AAIC/AAAAAAAAgL8AAIC/AAAAAAAAAAAAAIC/AACAvwAAgD8AAAAAAACAvwAAgL8AAABAAAAAAAAAgL8AAIC/AACAvwAAgD8AAIC/AACAvwAAAAAAAIA/AACAvwAAgL8AAIA/AACAPwAAgL8AAIC/AAAAQAAAgD8AAIC/AACAvwAAgL8AAABAAACAvwAAgL8AAAAAAAAAQAAAgL8AAIC/AACAPwAAAEAAAIC/AACAvwAAAEAAAABAAACAvwAAgL8AAIC/AACAvwAAAAAAAIC/AAAAAAAAgL8AAAAAAACAvwAAgD8AAIC/AAAAAAAAgL8AAABAAACAvwAAAAAAAIC/AACAvwAAAAAAAAAAAACAvwAAAAAAAAAAAAAAAAAAgL8AAIA/AAAAAAAAAAAAAIC/AAAAQAAAAAAAAAAAAACAvwAAgL8AAIA/AAAAAAAAgL8AAAAAAACAPwAAAAAAAIC/AACAPwAAgD8AAAAAAACAvwAAAEAAAIA/AAAAAAAAgL8AAIC/AAAAQAAAAAAAAIC/AAAAAAAAAEAAAAAAAACAvwAAgD8AAABAAAAAAAAAgL8AAABAAAAAQAAAAAAAAIC/AACAvwAAgL8AAIA/AACAvwAAAAAAAIC/AACAPwAAgL8AAIA/AACAvwAAgD8AAIC/AAAAQAAAgL8AAIA/AACAvwAAgL8AAAAAAACAPwAAgL8AAAAAAAAAAAAAgD8AAIC/AACAPwAAAAAAAIA/AACAvwAAAEAAAAAAAACAPwAAgL8AAIC/AACAPwAAgD8AAIC/AAAAAAAAgD8AAIA/AACAvwAAgD8AAIA/AACAPwAAgL8AAABAAACAPwAAgD8AAIC/AACAvwAAAEAAAIA/AACAvwAAAAAAAABAAACAPwAAgL8AAIA/AAAAQAAAgD8AAIC/AAAAQAAAAEAAAIA/AACAvwAAgL8AAIC/AAAAQAAAgL8AAAAAAACAvwAAAEAAAIC/AACAPwAAgL8AAABAAACAvwAAAEAAAIC/AAAAQAAAgL8AAIC/AAAAAAAAAEAAAIC/AAAAAAAAAAAAAABAAACAvwAAgD8AAAAAAAAAQAAAgL8AAABAAAAAAAAAAEAAAIC/AACAvwAAgD8AAABAAACAvwAAAAAAAIA/AAAAQAAAgL8AAIA/AACAPwAAAEAAAIC/AAAAQAAAgD8AAABAAACAvwAAgL8AAABAAAAAQAAAgL8AAAAAAAAAQAAAAEAAAIC/AACAPwAAAEAAAABAAACAvwAAAEAAAABAAAAAQAAAgL8AAIC/AACAvwAAgL8AAAAAAAAAAAAAgL8AAIC/AAAAAAAAgD8AAIC/AACAvwAAAAAAAABAAACAvwAAgL8AAAAAAACAvwAAAAAAAIC/AAAAAAAAAAAAAAAAAACAvwAAAAAAAIA/AAAAAAAAgL8AAAAAAAAAQAAAAAAAAIC/AAAAAAAAgL8AAIA/AACAvwAAAAAAAAAAAACAPwAAgL8AAAAAAACAPwAAgD8AAIC/AAAAAAAAAEAAAIA/AACAvwAAAAAAAIC/AAAAQAAAgL8AAAAAAAAAAAAAAEAAAIC/AAAAAAAAgD8AAABAAACAvwAAAAAAAABAAAAAQAAAgL8AAAAAAACAvwAAgL8AAAAAAAAAAAAAAAAAAIC/AAAAAAAAAAAAAIA/AACAvwAAAAAAAAAAAAAAQAAAgL8AAAAAAAAAAAAAgL8B9iqAPwAAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAIC/AACAPwAAAAAAAAAAAAAAAAAAgD8AAAAAAAAAAAAAgD8AAIA/AAAAAAAAAAAAAABAAACAPwAAAAAAAAAAAACAvwAAAEAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAIA/AAAAQAAAAAAAAAAAAAAAQAAAAEAAAAAAAAAAAAAAgL8AAIC/AACAPwAAAAAAAAAAAACAvwAAgD8AAAAAAACAPwAAgL8AAIA/AAAAAAAAAEAAAIC/AACAPwAAAAAAAIC/AAAAAAAAgD8AAAAAAAAAAAAAAAAAAIA/AAAAAAAAgD8AAAAAAACAPwAAAAAAAABAAAAAAAAAgD8AAAAAAACAvwAAgD8AAIA/AAAAAAAAAAAAAIA/AACAPwAAAAAAAIA/AACAPwAAgD8AAAAAAAAAQAAAgD8AAIA/AAAAAAAAgL8AAABAAACAPwAAAAAAAAAAAAAAQAAAgD8AAAAAAACAPwAAAEAAAIA/AAAAAAAAAEAAAABAAACAPwAAAAAAAIC/AACAvwAAAEAAAAAAAAAAAAAAgL8AAABAAAAAAAAAgD8AAIC/AAAAQAAAAAAAAABAAACAvwAAAEAAAAAAAACAvwAAAAAAAABAAAAAAAAAAAAAAAAAAAAAQAAAAAAAAIA/AAAAAAAAAEAAAAAAAAAAQAAAAAAAAABAAAAAAAAAgL8AAIA/AAAAQAAAAAAAAAAAAACAPwAAAEAAAAAAAACAPwAAgD8AAABAAAAAAAAAAEAAAIA/AAAAQAAAAAAAAIC/AAAAQAAAAEAAAAAAAAAAAAAAAEAAAABAAAAAAAAAgD8AAABAAAAAQAAAAAAAAABAAAAAQAAAAEAAAAAAAACAvwAAgL8AAIC/AACAPwAAAAAAAIC/AACAvwAAgD8AAIA/AACAvwAAgL8AAIA/AAAAQAAAgL8AAIC/AACAPwAAgL8AAAAAAACAvwAAgD8AAAAAAAAAAAAAgL8AAIA/AACAPwAAAAAAAIC/AACAPwAAAEAAAAAAAACAvwAAgD8AAIC/AACAPwAAgL8AAIA/AAAAAAAAgD8AAIC/AACAPwAAgD8AAIA/AACAvwAAgD8AAABAAACAPwAAgL8AAIA/AACAvwAAAEAAAIC/AACAPwAAAAAAAABAAACAvwAAgD8AAIA/AAAAQAAAgL8AAIA/AAAAQAAAAEAAAIC/AACAPwAAgL8AAIC/AAAAAAAAgD8AAAAAAACAvwAAAAAAAIA/AACAPwAAgL8AAAAAAACAPwAAAEAAAIC/AAAAAAAAgD8AAIC/AAAAAAAAAAAAAIA/AAAAAAAAAAAAAAAAAACAPwAAgD8AAAAAAAAAAAAAgD8AAABAAAAAAAAAAAAAAIA/AACAvwAAgD8AAAAAAACAPwAAAAAAAIA/AAAAAAAAgD8AAIA/AACAPwAAAAAAAIA/AAAAQAAAgD8AAAAAAACAPwAAgL8AAABAAAAAAAAAgD8AAAAAAAAAQAAAAAAAAIA/AACAPwAAAEAAAAAAAACAPwAAAEAAAABAAAAAAAAAgD8AAIC/AACAvwAAgD8AAIA/AAAAAAAAgL8AAIA/AACAPwAAgD8AAIC/AACAPwAAgD8AAABAAACAvwAAgD8AAIA/AACAvwAAAAAAAIA/AACAPwAAAAAAAAAAAACAPwAAgD8AAIA/AAAAAAAAgD8AAIA/AAAAQAAAAAAAAIA/AACAPwAAgL8AAIA/AACAPwAAgD8AAAAAAACAPwAAgD8AAIA/AACAPwAAgD8AAIA/AACAPwAAAEAAAIA/AACAPwAAgD8AAIC/AAAAQAAAgD8AAIA/AAAAAAAAAEAAAIA/AACAPwAAgD8AAABAAACAPwAAgD8AAABAAAAAQAAAgD8AAIA/AACAvwAAgL8AAABAAACAPwAAAAAAAIC/AAAAQAAAgD8AAIA/AACAvwAAAEAAAIA/AAAAQAAAgL8AAABAAACAPwAAgL8AAAAAAAAAQAAAgD8AAAAAAAAAAAAAAEAAAIA/AACAPwAAAAAAAABAAACAPwAAAEAAAAAAAAAAQAAAgD8AAIC/AACAPwAAAEAAAIA/AAAAAAAAgD8AAABAAACAPwAAgD8AAIA/AAAAQAAAgD8AAABAAACAPwAAAEAAAIA/AACAvwAAAEAAAABAAACAPwAAAAAAAABAAAAAQAAAgD8AAIA/AAAAQAAAAEAAAIA/AAAAQAAAAEAAAABAAACAPwAAgL8AAIC/AACAvwAAAEAAAAAAAACAvwAAgL8AAABAAACAPwAAgL8AAIC/AAAAQAAAAEAAAIC/AACAvwAAAEAAAIC/AAAAAAAAgL8AAABAAAAAAAAAAAAAAIC/AAAAQAAAgD8AAAAAAACAvwAAAEAAAABAAAAAAAAAgL8AAABAAACAvwAAgD8AAIC/AAAAQAAAAAAAAIA/AACAvwAAAEAAAIA/AACAPwAAgL8AAABAAAAAQAAAgD8AAIC/AAAAQAAAgL8AAABAAACAvwAAAEAAAAAAAAAAQAAAgL8AAABAAACAPwAAAEAAAIC/AAAAQAAAAEAAAABAAACAvwAAAEAAAIC/AACAvwAAAAAAAABAAAAAAAAAgL8AAAAAAAAAQAAAgD8AAIC/AAAAAAAAAEAAAABAAACAvwAAAAAAAABAAACAvwAAAAAAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAEAAAIA/AAAAAAAAAAAAAABAAAAAQAAAAAAAAAAAAAAAQAAAgL8AAIA/AAAAAAAAAEAAAAAAAACAPwAAAAAAAABAAACAPwAAgD8AAAAAAAAAQAAAAEAAAIA/AAAAAAAAAEAAAIC/AAAAQAAAAAAAAABAAAAAAAAAAEAAAAAAAAAAQAAAgD8AAABAAAAAAAAAAEAAAABAAAAAQAAAAAAAAABAAACAvwAAgL8AAIA/AAAAQAAAAAAAAIC/AACAPwAAAEAAAIA/AACAvwAAgD8AAABAAAAAQAAAgL8AAIA/AAAAQAAAgL8AAAAAAACAPwAAAEAAAAAAAAAAAAAAgD8AAABAAACAPwAAAAAAAIA/AAAAQAAAAEAAAAAAAACAPwAAAEAAAIC/AACAPwAAgD8AAABAAAAAAAAAgD8AAIA/AAAAQAAAgD8AAIA/AACAPwAAAEAAAABAAACAPwAAgD8AAABAAACAvwAAAEAAAIA/AAAAQAAAAAAAAABAAACAPwAAAEAAAIA/AAAAQAAAgD8AAABAAAAAQAAAAEAAAIA/AAAAQAAAgL8AAIC/AAAAQAAAAEAAAAAAAACAvwAAAEAAAABAAACAPwAAgL8AAABAAAAAQAAAAEAAAIC/AAAAQAAAAEAAAIC/AAAAAAAAAEAAAABAAAAAAAAAAAAAAABAAAAAQAAAgD8AAAAAAAAAQAAAAEAAAABAAAAAAAAAAEAAAABAAACAvwAAgD8AAABAAAAAQAAAAAAAAIA/AAAAQAAAAEAAAIA/AACAPwAAAEAAAABAAAAAQAAAgD8AAABAAAAAQAAAgL8AAABAAAAAQAAAAEAAAAAAAAAAQAAAAEAAAABAAACAPwAAAEAAAABAAAAAQAAAAEAAAABAAAAAQAAAAED+gitlRxVnQAAAAAAAADhDAAD6/kIudr86O568mvcMvb39/////98/PFRVVVVVxT+RKxfPVVWlPxfQpGcREYE/AAAAAAAAyELvOfr+Qi7mPyTEgv+9v84/tfQM1whrrD/MUEbSq7KDP4Q6Tpvg11U/AAAAAAAAAAAAAAAAAADwP26/iBpPO5s8NTP7qT327z9d3NicE2BxvGGAdz6a7O8/0WaHEHpekLyFf27oFePvPxP2ZzVS0ow8dIUV07DZ7z/6jvkjgM6LvN723Slr0O8/YcjmYU73YDzIm3UYRcfvP5nTM1vko5A8g/PGyj6+7z9te4NdppqXPA+J+WxYte8//O/9khq1jjz3R3IrkqzvP9GcL3A9vj48otHTMuyj7z8LbpCJNANqvBvT/q9mm+8/Dr0vKlJWlbxRWxLQAZPvP1XqTozvgFC8zDFswL2K7z8W9NW5I8mRvOAtqa6agu8/r1Vc6ePTgDxRjqXImHrvP0iTpeoVG4C8e1F9PLhy7z89Mt5V8B+PvOqNjDj5au8/v1MTP4yJizx1y2/rW2PvPybrEXac2Za81FwEhOBb7z9gLzo+9+yaPKq5aDGHVO8/nTiGy4Lnj7wd2fwiUE3vP43DpkRBb4o81oxiiDtG7z99BOSwBXqAPJbcfZFJP+8/lKio4/2Oljw4YnVuejjvP31IdPIYXoc8P6ayT84x7z/y5x+YK0eAPN184mVFK+8/XghxP3u4lryBY/Xh3yTvPzGrCW3h94I84d4f9Z0e7z/6v28amyE9vJDZ2tB/GO8/tAoMcoI3izwLA+SmhRLvP4/LzomSFG48Vi8+qa8M7z+2q7BNdU2DPBW3MQr+Bu8/THSs4gFChjwx2Ez8cAHvP0r401053Y88/xZksgj87j8EW447gKOGvPGfkl/F9u4/aFBLzO1KkrzLqTo3p/HuP44tURv4B5m8ZtgFba7s7j/SNpQ+6NFxvPef5TTb5+4/FRvOsxkZmbzlqBPDLePuP21MKqdIn4U8IjQSTKbe7j+KaSh6YBKTvByArARF2u4/W4kXSI+nWLwqLvchCtbuPxuaSWebLHy8l6hQ2fXR7j8RrMJg7WNDPC2JYWAIzu4/72QGOwlmljxXAB3tQcruP3kDodrhzG480DzBtaLG7j8wEg8/jv+TPN7T1/Aqw+4/sK96u86QdjwnKjbV2r/uP3fgVOu9HZM8Dd39mbK87j+Oo3EANJSPvKcsnXayue4/SaOT3Mzeh7xCZs+i2rbuP184D73G3ni8gk+dViu07j/2XHvsRhKGvA+SXcqkse4/jtf9GAU1kzzaJ7U2R6/uPwWbii+3mHs8/ceX1BKt7j8JVBzi4WOQPClUSN0Hq+4/6sYZUIXHNDy3RlmKJqnuPzXAZCvmMpQ8SCGtFW+n7j+fdplhSuSMvAncdrnhpe4/qE3vO8UzjLyFVTqwfqTuP67pK4l4U4S8IMPMNEaj7j9YWFZ43c6TvCUiVYI4ou4/ZBl+gKoQVzxzqUzUVaHuPygiXr/vs5O8zTt/Zp6g7j+CuTSHrRJqvL/aC3USoO4/7qltuO9nY7wvGmU8sp/uP1GI4FQ93IC8hJRR+X2f7j/PPlp+ZB94vHRf7Oh1n+4/sH2LwEruhrx0gaVImp/uP4rmVR4yGYa8yWdCVuuf7j/T1Aley5yQPD9d3k9poO4/HaVNudwye7yHAetzFKHuP2vAZ1T97JQ8MsEwAe2h7j9VbNar4etlPGJOzzbzou4/Qs+zL8WhiLwSGj5UJ6TuPzQ3O/G2aZO8E85MmYml7j8e/xk6hF6AvK3HI0Yap+4/bldy2FDUlLztkkSb2ajuPwCKDltnrZA8mWaK2ceq7j+06vDBL7eNPNugKkLlrO4//+fFnGC2ZbyMRLUWMq/uP0Rf81mD9ns8NncVma6x7j+DPR6nHwmTvMb/kQtbtO4/KR5si7ipXbzlxc2wN7fuP1m5kHz5I2y8D1LIy0S67j+q+fQiQ0OSvFBO3p+Cve4/S45m12zKhby6B8pw8cDuPyfOkSv8r3E8kPCjgpHE7j+7cwrhNdJtPCMj4xljyO4/YyJiIgTFh7xl5V17ZszuP9Ux4uOGHIs8My1K7JvQ7j8Vu7zT0buRvF0lPrID1e4/0jHunDHMkDxYszATntnuP7Nac26EaYQ8v/15VWve7j+0nY6Xzd+CvHrz079r4+4/hzPLkncajDyt01qZn+juP/rZ0UqPe5C8ZraNKQfu7j+6rtxW2cNVvPsVT7ii8+4/QPamPQ6kkLw6WeWNcvnuPzSTrTj01mi8R1778nb/7j81ilhr4u6RvEoGoTCwBe8/zd1fCtf/dDzSwUuQHgzvP6yYkvr7vZG8CR7XW8IS7z+zDK8wrm5zPJxShd2bGe8/lP2fXDLjjjx60P9fqyDvP6xZCdGP4IQ8S9FXLvEn7z9nGk44r81jPLXnBpRtL+8/aBmSbCxrZzxpkO/cIDfvP9K1zIMYioC8+sNdVQs/7z9v+v8/Xa2PvHyJB0otR+8/Sal1OK4NkLzyiQ0Ih0/vP6cHPaaFo3Q8h6T73BhY7z8PIkAgnpGCvJiDyRbjYO8/rJLB1VBajjyFMtsD5mnvP0trAaxZOoQ8YLQB8yFz7z8fPrQHIdWCvF+bezOXfO8/yQ1HO7kqibwpofUURobvP9OIOmAEtnQ89j+L5y6Q7z9xcp1R7MWDPINMx/tRmu8/8JHTjxL3j7zakKSir6TvP310I+KYro288WeOLUiv7z8IIKpBvMOOPCdaYe4buu8/Muupw5QrhDyXums3K8XvP+6F0TGpZIo8QEVuW3bQ7z/t4zvkujeOvBS+nK392+8/nc2RTTuJdzzYkJ6BwefvP4nMYEHBBVM88XGPK8Lz7z8AAAAAAADwP3SFFdOw2e8/D4n5bFi17z9RWxLQAZPvP3tRfTy4cu8/qrloMYdU7z84YnVuejjvP+HeH/WdHu8/FbcxCv4G7z/LqTo3p/HuPyI0Ekym3u4/LYlhYAjO7j8nKjbV2r/uP4JPnVYrtO4/KVRI3Qer7j+FVTqwfqTuP807f2aeoO4/dF/s6HWf7j+HAetzFKHuPxPOTJmJpe4/26AqQuWs7j/lxc2wN7fuP5Dwo4KRxO4/XSU+sgPV7j+t01qZn+juP0de+/J2/+4/nFKF3ZsZ7z9pkO/cIDfvP4ek+9wYWO8/X5t7M5d87z/akKSir6TvP0BFblt20O8/AAAAAAAA6EKUI5FL+GqsP/PE+lDOv84/1lIM/0Iu5j8AAAAAAAA4Q/6CK2VHFUdAlCORS/hqvD7zxPpQzr8uP9ZSDP9CLpY/vvP4eexh9j/eqoyA93vVvz2Ir0rtcfU/223Ap/C+0r+wEPDwOZX0P2c6UX+uHtC/hQO4sJXJ8z/pJIKm2DHLv6VkiAwZDfM/WHfACk9Xxr+gjgt7Il7yPwCBnMcrqsG/PzQaSkq78T9eDozOdk66v7rlivBYI/E/zBxhWjyXsb+nAJlBP5XwPx4M4Tj0UqK/AAAAAAAA8D8AAAAAAAAAAKxHmv2MYO4/hFnyXaqlqj+gagIfs6TsP7QuNqpTXrw/5vxqVzYg6z8I2yB35SbFPy2qoWPRwuk/cEciDYbCyz/tQXgD5oboP+F+oMiLBdE/YkhT9dxn5z8J7rZXMATUP+85+v5CLuY/NIO4SKMO0L9qC+ALW1fVPyNBCvL+/9+/AWXoHgAA6B4AAAAAAQAAAgAAAAAAAAUAAAAAAAAAAAAAACwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAC0AAAAuAAAA6CcAAAAEAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAD/////CgEIMB8AAPArAQA=" } diff --git a/skainet-backends/skainet-backend-wasm-simd/src/wasmJsMain/kotlin/sk/ainet/exec/wasm/WasmCpuOps.kt b/skainet-backends/skainet-backend-wasm-simd/src/wasmJsMain/kotlin/sk/ainet/exec/wasm/WasmCpuOps.kt index eb1df89f0..2973e26b4 100644 --- a/skainet-backends/skainet-backend-wasm-simd/src/wasmJsMain/kotlin/sk/ainet/exec/wasm/WasmCpuOps.kt +++ b/skainet-backends/skainet-backend-wasm-simd/src/wasmJsMain/kotlin/sk/ainet/exec/wasm/WasmCpuOps.kt @@ -2,25 +2,54 @@ package sk.ainet.exec.wasm import sk.ainet.context.schedule.Schedule import sk.ainet.exec.tensor.ops.DefaultCpuOpsBase +import sk.ainet.lang.memory.Scope +import sk.ainet.lang.memory.ScopeKind import sk.ainet.lang.tensor.Shape import sk.ainet.lang.tensor.Tensor import sk.ainet.lang.tensor.data.FloatArrayTensorData +import sk.ainet.lang.tensor.data.StorageFloatTensorData import sk.ainet.lang.tensor.data.TensorData import sk.ainet.lang.tensor.data.TensorDataFactory +import sk.ainet.lang.tensor.ops.FusedNormOps import sk.ainet.lang.tensor.ops.TensorOps +import sk.ainet.lang.tensor.storage.MemoryDomain import sk.ainet.lang.types.DType import sk.ainet.lang.types.FP32 +import kotlin.reflect.KClass /** - * The browser tier's `TensorOps` (SKEEP-007 §T1): [DefaultCpuOpsBase] with the dense FP32 matmuls - * (`x · Wᵀ` and `A · B`) routed through the SIMD128 sidecar. Packed weights need no override — they reach the pack through - * `KernelDispatch` like on every other tier. Everything else is the common scalar implementation, - * and so is everything here while the sidecar is not ready. + * The browser tier's `TensorOps` (SKEEP-007 §T1 and the resident-activation step): [DefaultCpuOpsBase] + * with the dense matmuls, the elementwise arithmetic, the activations and the reductions routed + * through the SIMD128 sidecar, and their outputs kept **resident** in the sidecar's linear memory + * ([WasmLinearTensorData]) so a chain of them never copies an activation back to the Kotlin heap + * between kernels. Packed weights need no override — they reach the pack through `KernelDispatch` + * like on every other tier. Everything else is the common scalar implementation, which reads a + * resident tensor through its heap mirror (one bulk download, the first time), and so is + * everything here while the sidecar is not ready. + * + * What goes to the sidecar, and when: + * + * - `matmul`, `matmulWeightTransposed` (dense FP32): always; the output is resident. + * - `add`/`subtract`/`multiply`/`divide`: when at least one operand is already resident. Two heap + * operands stay on the common path — uploading both costs more than the scalar loop saves, and + * a chain starts resident at its first matmul anyway. + * - `sigmoid`/`silu`/`gelu`/`erf`/`tanh`: always — the transcendental loop is several times the + * upload. `relu`, `sum`, `mean`: when the operand is resident. + * - `softmax`/`logSoftmax` over the last dimension, `scaledDotProductAttention` and the fused + * normalisations of [FusedNormOps] (what `LayerNormalization` calls): always — one sidecar pass + * against several allocating scalar passes, whatever the operands' home. + * - `conv1d`/`conv2d`/`conv3d` (dense FP32): always — im2col and the GEMM in the sidecar, the + * weight resident after its first call, the output resident. + * + * An output is resident only while [WasmSimd.residentBudgetBytes] allows; above it the kernel still + * runs in the sidecar and the result is read back to the heap. Scoped contexts (`ForwardScope`) + * allocate the resident output from the scope, so `reset()` frees the sidecar bytes; ambient outputs + * are freed when their tensor is collected (see [WasmLinearStorage]). */ public class WasmCpuOps( dataFactory: TensorDataFactory, schedule: Schedule, -) : DefaultCpuOpsBase(dataFactory, schedule) { +) : DefaultCpuOpsBase(dataFactory, schedule), FusedNormOps { override fun withSchedule(schedule: Schedule): TensorOps = if (schedule === this.schedule) this else WasmCpuOps(dataFactory, schedule) @@ -28,65 +57,534 @@ public class WasmCpuOps( /** The sidecar computes `x · Wᵀ` from the weight's rows itself; no Kotlin-side row-dot and no transposed copy. */ override fun preferDirectWeightTransposed(): Boolean = false + // ── operands and outputs ────────────────────────────────────────────────────────────────────── + + /** A sidecar address for an operand: a resident tensor's own, or a staged copy freed after the call. */ + private class Operand(val ptr: Int, val temp: Boolean) + + private fun isResident(t: Tensor<*, *>): Boolean = (t.data as? WasmLinearTensorData<*>)?.isResident == true + + /** Dense FP32 the sidecar can take: resident, a heap array, or a slab window. */ + private fun isSidecarFloat(t: Tensor<*, *>): Boolean = + t.dtype == FP32::class && (t.data is WasmLinearTensorData<*> || t.data is FloatArrayTensorData<*> || t.data is StorageFloatTensorData<*>) + + private fun operand(t: Tensor<*, *>): Operand? = when (val d = t.data) { + is WasmLinearTensorData<*> -> if (d.isResident) Operand(d.pointer(), false) else null + is FloatArrayTensorData<*> -> stage(d.buffer, 0, t.shape.volume) + is StorageFloatTensorData<*> -> { + val s = d.storage + s.checkAlive() + stage(s.floats!!, s.arrayOffset, t.shape.volume) + } + else -> null + } + + private fun stage(src: FloatArray, offset: Int, count: Int): Operand { + val ptr = WasmKernels.alloc(maxOf(count, 1) * 4) + if (count > 0) WasmKernels.writeFloats(ptr, src, offset, count) + return Operand(ptr, true) + } + + private fun release(o: Operand?) { + if (o != null && o.temp) WasmKernels.free(o.ptr) + } + + /** A resident output allocation from the active scope (ambient: GC-managed), or `null` when the scope cannot place it in the sidecar. */ + private fun residentStorage(bytes: Long): WasmLinearStorage? { + val scope = dispatchScope() + return if (scope === Scope.Ambient) WasmLinearStorage.allocate(bytes, ScopeKind.AMBIENT, zero = false) + else scope.allocate(bytes, MemoryDomain.HOST_OFFHEAP) as? WasmLinearStorage + } + /** - * Dense `A(m,k) · B(k,n)` on heap FP32 operands runs the sidecar GEMM. The common implementation - * serves this shape with its own `[k, n]` loop before it consults the registry, so without this - * override the SIMD128 kernel was reachable only through `KernelDispatch` (batched or view - * operands) and a plain 2-D matmul stayed at scalar speed. Both operands are staged per call: - * `B` here is not necessarily a parameter (attention scores, projections of activations), so - * nothing is kept resident. A packed weight is not `FloatArrayTensorData` and keeps its path. + * Run [fill] into an output buffer of [shape]: resident when the budget allows, otherwise a + * temporary that is read back to the heap and adopted through the factory. */ @Suppress("UNCHECKED_CAST") + private inline fun output( + shape: Shape, + dtype: KClass, + inputs: Array>, + fill: (outPtr: Int) -> Unit, + ): Tensor { + val n = shape.volume + val bytes = n.toLong() * 4 + if (WasmSimd.residentBudgetBytes - WasmKernels.residentActivationBytes() >= bytes) { + residentStorage(bytes)?.let { storage -> + fill(storage.pointer()) + return newTensor(WasmLinearTensorData(shape, storage) as TensorData, dtype, *inputs) + } + } + val tmp = WasmKernels.alloc(maxOf(n, 1) * 4) + try { + fill(tmp) + val out = FloatArray(n) + if (n > 0) WasmKernels.readFloats(tmp, out, 0, n) + return floatResult(shape, dtype, out, *inputs) + } finally { + WasmKernels.free(tmp) + } + } + + /** + * A resident copy of a dense FP32 [tensor] — the explicit "upload once" for an input that many + * sidecar ops will read (a bench's operands, a prompt's embeddings), so none of them stages it + * per call. The tensor itself if it is already resident; the common data unchanged when the + * sidecar is not ready or the budget is spent. + */ + public fun toResident(tensor: Tensor): Tensor { + if (!WasmKernels.isReady || !isSidecarFloat(tensor) || isResident(tensor)) return tensor + val n = tensor.shape.volume + val o = operand(tensor) ?: return tensor + try { + val src = o.ptr + val bytes = n.toLong() * 4 + if (WasmSimd.residentBudgetBytes - WasmKernels.residentActivationBytes() < bytes) return tensor + val storage = residentStorage(bytes) ?: return tensor + WasmKernels.copy(src, storage.pointer(), n * 4) + @Suppress("UNCHECKED_CAST") + return newTensor(WasmLinearTensorData(tensor.shape, storage) as TensorData, tensor.dtype, tensor) + } finally { + release(o) + } + } + + // ── matmuls ─────────────────────────────────────────────────────────────────────────────────── + + /** + * Dense `A(m,k) · B(k,n)` on FP32 operands runs the sidecar GEMM. The common implementation + * serves this shape with its own `[k, n]` loop before it consults the registry, so without this + * override the SIMD128 kernel was reachable only through `KernelDispatch`. Resident operands are + * read in place; heap operands are staged per call (`B` is not necessarily a parameter). A packed + * weight is not dense FP32 data and keeps its path. + */ override fun matmul(a: Tensor, b: Tensor): Tensor { - if (WasmKernels.isReady && a.rank == 2 && b.rank == 2 && a.dtype == FP32::class && b.dtype == FP32::class) { - val ad = a.data as? FloatArrayTensorData<*> - val bd = b.data as? FloatArrayTensorData<*> - if (ad != null && bd != null && untransposedWeight(b) == null) { - val m = a.shape[0] - val k = a.shape[1] - val n = b.shape[1] - require(k == b.shape[0]) { "Matrix multiplication shape mismatch: ${a.shape} vs ${b.shape}" } - val out = FloatArray(m * n) - if (m > 0 && n > 0) WasmFp32MatmulKernel.matmul(ad.buffer, 0, k, bd.buffer, 0, n, out, 0, n, m, n, k) - val outData = dataFactory.adoptFloatArray(Shape(m, n), a.dtype, out) as TensorData - return newTensor(outData, a.dtype, a, b) + if (WasmKernels.isReady && a.rank == 2 && b.rank == 2 && isSidecarFloat(a) && isSidecarFloat(b) && untransposedWeight(b) == null) { + val m = a.shape[0] + val k = a.shape[1] + val n = b.shape[1] + require(k == b.shape[0]) { "Matrix multiplication shape mismatch: ${a.shape} vs ${b.shape}" } + if (m > 0 && n > 0) { + var ao: Operand? = null + var bo: Operand? = null + try { + ao = operand(a) ?: return super.matmul(a, b) + bo = operand(b) ?: return super.matmul(a, b) + val aPtr = ao.ptr + val bPtr = bo.ptr + return output(Shape(m, n), a.dtype, arrayOf(a, b)) { out -> WasmKernels.fp32Matmul(aPtr, k, bPtr, n, out, n, m, n, k) } + } finally { + release(bo) + release(ao) + } } } return super.matmul(a, b) } - @Suppress("UNCHECKED_CAST") override fun matmulWeightTransposed(x: Tensor, weight: Tensor): Tensor { - if (WasmKernels.isReady && weight.shape.rank == 2 && x.rank >= 1 && x.dtype == FP32::class && weight.dtype == FP32::class) { - val w = weight.data as? FloatArrayTensorData<*> - val a = x.data as? FloatArrayTensorData<*> - if (w != null && a != null) { - val n = weight.shape[0] - val k = weight.shape[1] - if (x.shape.dimensions.last() == k) { - val m = if (k == 0) 0 else x.shape.volume / k - val out = FloatArray(m * n) - if (m > 0 && n > 0) { - val inPtr = WasmKernels.alloc(m * k * 4) - val outPtr = WasmKernels.alloc(m * n * 4) - try { - WasmKernels.writeFloats(inPtr, a.buffer, 0, m * k) - // Resident like a packed weight: a parameter is immutable once loaded, and - // re-uploading it per call made single-token decode run at copy speed. - val wPtr = WasmKernels.denseWeightHandle(w.buffer) - WasmKernels.fp32MatmulWt(inPtr, wPtr, outPtr, m, n, k) - WasmKernels.readFloats(outPtr, out, 0, m * n) - } finally { - WasmKernels.free(outPtr) - WasmKernels.free(inPtr) + if (WasmKernels.isReady && weight.shape.rank == 2 && x.rank >= 1 && isSidecarFloat(x) && isSidecarFloat(weight)) { + val n = weight.shape[0] + val k = weight.shape[1] + if (x.shape.dimensions.last() == k) { + val m = if (k == 0) 0 else x.shape.volume / k + if (m > 0 && n > 0) { + var xo: Operand? = null + var wo: Operand? = null + try { + xo = operand(x) ?: return super.matmulWeightTransposed(x, weight) + // A heap weight is a parameter: resident after its first call (re-uploading it + // per call made single-token decode run at copy speed). Resident and slab + // weights are read where they are. + wo = when (val wd = weight.data) { + is FloatArrayTensorData<*> -> Operand(WasmKernels.denseWeightHandle(wd.buffer), false) + else -> operand(weight) ?: return super.matmulWeightTransposed(x, weight) } + val xPtr = xo.ptr + val wPtr = wo.ptr + val outShape = if (x.rank == 1) Shape(n) else Shape(*(x.shape.dimensions.copyOf(x.rank - 1) + n)) + return output(outShape, x.dtype, arrayOf(x, weight)) { out -> WasmKernels.fp32MatmulWt(xPtr, wPtr, out, m, n, k) } + } finally { + release(wo) + release(xo) } - val outShape = if (x.rank == 1) Shape(n) else Shape(*(x.shape.dimensions.copyOf(x.rank - 1) + n)) - val outData = dataFactory.adoptFloatArray(outShape, x.dtype, out) as TensorData - return newTensor(outData, x.dtype, x, weight) } } } return super.matmulWeightTransposed(x, weight) } + + // ── elementwise ─────────────────────────────────────────────────────────────────────────────── + + /** The operand's index mode against the broadcast output shape, or `null` for a broadcast the kernel does not take. */ + private fun mode(s: Shape, out: Shape): Int? = when { + s == out -> WasmKernels.F32_FULL + s.volume == 1 -> WasmKernels.F32_SCALAR + s.rank >= 1 && out.rank >= 1 && s[s.rank - 1] == out[out.rank - 1] && s.dimensions.dropLast(1).all { it == 1 } -> WasmKernels.F32_BIAS + else -> null + } + + private fun binary(op: Int, a: Tensor, b: Tensor): Tensor? { + if (!WasmKernels.isReady || a.dtype != b.dtype || !isSidecarFloat(a) || !isSidecarFloat(b)) return null + if (!isResident(a) && !isResident(b)) return null + val outShape = try { + broadcastShapes(a.shape, b.shape) + } catch (e: IllegalArgumentException) { + return null + } + val n = outShape.volume + if (n == 0) return null + val aMode = mode(a.shape, outShape) ?: return null + val bMode = mode(b.shape, outShape) ?: return null + val inner = outShape[outShape.rank - 1] + var ao: Operand? = null + var bo: Operand? = null + try { + ao = operand(a) ?: return null + bo = operand(b) ?: return null + val aPtr = ao.ptr + val bPtr = bo.ptr + return output(outShape, a.dtype, arrayOf(a, b)) { out -> WasmKernels.f32Binary(op, aPtr, aMode, bPtr, bMode, out, n, inner) } + } finally { + release(bo) + release(ao) + } + } + + override fun add(a: Tensor, b: Tensor): Tensor = + binary(WasmKernels.F32_ADD, a, b) ?: super.add(a, b) + + override fun subtract(a: Tensor, b: Tensor): Tensor = + binary(WasmKernels.F32_SUB, a, b) ?: super.subtract(a, b) + + override fun multiply(a: Tensor, b: Tensor): Tensor = + binary(WasmKernels.F32_MUL, a, b) ?: super.multiply(a, b) + + override fun divide(a: Tensor, b: Tensor): Tensor = + binary(WasmKernels.F32_DIV, a, b) ?: super.divide(a, b) + + // ── activations ─────────────────────────────────────────────────────────────────────────────── + + private fun unary(op: Int, t: Tensor, heapToo: Boolean): Tensor? { + if (!WasmKernels.isReady || !isSidecarFloat(t)) return null + if (!heapToo && !isResident(t)) return null + val n = t.shape.volume + if (n == 0) return null + val o = operand(t) ?: return null + try { + val inPtr = o.ptr + return output(t.shape, t.dtype, arrayOf(t)) { out -> WasmKernels.f32Unary(op, inPtr, out, n) } + } finally { + release(o) + } + } + + override fun relu(tensor: Tensor): Tensor = + unary(WasmKernels.F32_RELU, tensor, heapToo = false) ?: super.relu(tensor) + + override fun sigmoid(tensor: Tensor): Tensor = + unary(WasmKernels.F32_SIGMOID, tensor, heapToo = true) ?: super.sigmoid(tensor) + + override fun silu(tensor: Tensor): Tensor = + unary(WasmKernels.F32_SILU, tensor, heapToo = true) ?: super.silu(tensor) + + override fun gelu(tensor: Tensor, approximate: Boolean): Tensor = + unary(if (approximate) WasmKernels.F32_GELU_TANH else WasmKernels.F32_GELU_EXACT, tensor, heapToo = true) + ?: super.gelu(tensor, approximate) + + override fun erf(tensor: Tensor): Tensor = + unary(WasmKernels.F32_ERF, tensor, heapToo = true) ?: super.erf(tensor) + + override fun tanh(tensor: Tensor): Tensor = + unary(WasmKernels.F32_TANH, tensor, heapToo = true) ?: super.tanh(tensor) + + // ── reductions ──────────────────────────────────────────────────────────────────────────────── + + private fun reduce(t: Tensor, dim: Int?, mean: Boolean): Tensor? { + if (!WasmKernels.isReady || !isSidecarFloat(t) || !isResident(t)) return null + val rank = t.rank + val volume = t.shape.volume + if (volume == 0) return null + val o = operand(t) ?: return null + try { + val inPtr = o.ptr + if (dim == null) { + return output(Shape(), t.dtype, arrayOf(t)) { out -> WasmKernels.f32ReduceAll(inPtr, volume, mean, out) } + } + val nd = if (dim < 0) dim + rank else dim + if (nd !in 0 until rank) return null + val dims = t.shape.dimensions + var outer = 1 + for (i in 0 until nd) outer *= dims[i] + val red = dims[nd] + var inner = 1 + for (i in nd + 1 until rank) inner *= dims[i] + val outDims = IntArray(rank - 1) + var oi = 0 + for (i in 0 until rank) { if (i == nd) continue; outDims[oi++] = dims[i] } + return output(Shape(outDims), t.dtype, arrayOf(t)) { out -> WasmKernels.f32ReduceAxis(inPtr, outer, red, inner, mean, out) } + } finally { + release(o) + } + } + + override fun sum(tensor: Tensor, dim: Int?): Tensor = + reduce(tensor, dim, mean = false) ?: super.sum(tensor, dim) + + override fun mean(tensor: Tensor, dim: Int?): Tensor = + reduce(tensor, dim, mean = true) ?: super.mean(tensor, dim) + + // ── fused row kernels: softmax, the normalisations, attention ───────────────────────────────── + + private fun softmaxRows(t: Tensor, dim: Int, log: Boolean): Tensor? { + if (!WasmKernels.isReady || !isSidecarFloat(t)) return null + val rank = t.rank + if (rank == 0) return null + val nd = if (dim < 0) dim + rank else dim + if (nd != rank - 1) return null + val cols = t.shape[nd] + val n = t.shape.volume + if (cols <= 0 || n == 0) return null + val o = operand(t) ?: return null + try { + val inPtr = o.ptr + return output(t.shape, t.dtype, arrayOf(t)) { out -> WasmKernels.f32SoftmaxRows(inPtr, out, n / cols, cols, log) } + } finally { + release(o) + } + } + + override fun softmax(tensor: Tensor, dim: Int): Tensor = + softmaxRows(tensor, dim, log = false) ?: super.softmax(tensor, dim) + + override fun logSoftmax(tensor: Tensor, dim: Int): Tensor = + softmaxRows(tensor, dim, log = true) ?: super.logSoftmax(tensor, dim) + + /** A norm parameter of [cols] values as a sidecar operand: in place or staged when dense FP32, copied out otherwise (a placeholder). */ + private fun normParam(t: Tensor<*, *>, cols: Int): Operand? { + if (t.shape.volume != cols) return null + if (t.dtype == FP32::class) operand(t)?.let { return it } + return stage(t.data.copyToFloatArray(), 0, cols) + } + + private fun norm( + input: Tensor, gamma: Tensor?, beta: Tensor?, eps: Float, normalizedRank: Int, rms: Boolean, + ): Tensor? { + if (!WasmKernels.isReady || !isSidecarFloat(input)) return null + val rank = input.rank + if (normalizedRank !in 1..rank) return null + var cols = 1 + for (i in rank - normalizedRank until rank) cols *= input.shape[i] + val n = input.shape.volume + if (cols == 0 || n == 0) return null + val rows = n / cols + var io: Operand? = null + var go: Operand? = null + var bo: Operand? = null + try { + io = operand(input) ?: return null + go = gamma?.let { normParam(it, cols) ?: return null } + bo = beta?.let { normParam(it, cols) ?: return null } + val inPtr = io.ptr + val gPtr = go?.ptr ?: 0 + val bPtr = bo?.ptr ?: 0 + val inputs = listOfNotNull(input, gamma, beta).toTypedArray() + return output(input.shape, input.dtype, inputs) { out -> + if (rms) WasmKernels.f32RmsNorm(inPtr, gPtr, out, rows, cols, eps) + else WasmKernels.f32LayerNorm(inPtr, gPtr, bPtr, out, rows, cols, eps) + } + } finally { + release(bo) + release(go) + release(io) + } + } + + override fun layerNorm(input: Tensor, gamma: Tensor?, beta: Tensor?, eps: Float, normalizedRank: Int): Tensor? = + norm(input, gamma, beta, eps, normalizedRank, rms = false) + + override fun rmsNorm(input: Tensor, gamma: Tensor?, eps: Float, normalizedRank: Int): Tensor? = + norm(input, gamma, null, eps, normalizedRank, rms = true) + + /** + * Attention in one sidecar pass (`skainet_f32_sdpa`): the same operand contract and mask rules + * as the common implementation, the scores never materialised on the Kotlin heap, query rows + * threaded in the sidecar's pool. Resident operands are read in place — a chain + * `q/k/v = x · Wᵀ → sdpa → · Wₒᵀ` never leaves linear memory. + */ + override fun scaledDotProductAttention( + query: Tensor, + key: Tensor, + value: Tensor, + mask: Tensor?, + scale: Float, + causal: Boolean, + keyLengths: Tensor?, + window: Int, + rightContext: Int, + ): Tensor { + if (!WasmKernels.isReady || !isSidecarFloat(query) || !isSidecarFloat(key) || !isSidecarFloat(value) || (mask != null && !isSidecarFloat(mask))) { + return super.scaledDotProductAttention(query, key, value, mask, scale, causal, keyLengths, window, rightContext) + } + requireSdpaOperands(query, key, value, window, rightContext) + val batch = query.shape[0] + val heads = query.shape[1] + val seqQ = query.shape[2] + val headDim = query.shape[3] + val kvHeads = key.shape[1] + val seqKV = key.shape[2] + val lengths = sdpaKeyLengths(keyLengths, batch, seqKV) + val maskDims = sdpaMaskDims(mask, batch, heads, kvHeads, seqQ, seqKV) ?: intArrayOf(1, 1, 1, 1) + val effectiveScale = if (scale == 0f) (1.0 / kotlin.math.sqrt(headDim.toDouble())).toFloat() else scale + var qo: Operand? = null + var ko: Operand? = null + var vo: Operand? = null + var mo: Operand? = null + var lengthsPtr = 0 + try { + qo = operand(query) ?: return super.scaledDotProductAttention(query, key, value, mask, scale, causal, keyLengths, window, rightContext) + ko = operand(key) ?: return super.scaledDotProductAttention(query, key, value, mask, scale, causal, keyLengths, window, rightContext) + vo = operand(value) ?: return super.scaledDotProductAttention(query, key, value, mask, scale, causal, keyLengths, window, rightContext) + mo = mask?.let { operand(it) ?: return super.scaledDotProductAttention(query, key, value, mask, scale, causal, keyLengths, window, rightContext) } + if (lengths != null && batch > 0) { + lengthsPtr = WasmKernels.alloc(batch * 4) + WasmKernels.writeInts(lengthsPtr, lengths, 0, batch) + } + val qPtr = qo.ptr + val kPtr = ko.ptr + val vPtr = vo.ptr + val mPtr = mo?.ptr ?: 0 + val lPtr = lengthsPtr + return output(Shape(batch, heads, seqQ, headDim), query.dtype, arrayOf(query, key, value)) { out -> + WasmKernels.f32Sdpa( + qPtr, kPtr, vPtr, out, batch, heads, kvHeads, seqQ, seqKV, headDim, + effectiveScale, causal, window, rightContext, lPtr, mPtr, maskDims[0], maskDims[1], maskDims[2], maskDims[3], + ) + } + } finally { + if (lengthsPtr != 0) WasmKernels.free(lengthsPtr) + release(mo) + release(vo) + release(ko) + release(qo) + } + } + + // ── convolutions: im2col + GEMM in the sidecar ──────────────────────────────────────────────── + + /** + * `conv1d` / `conv2d` / `conv3d` on dense FP32 operands run `skainet_f32_conv`: the receptive + * fields are gathered patch-major into a sidecar scratch and multiplied against the weight rows + * by the SIMD128 `x · Wᵀ` kernel, both threaded over patches — the JVM's im2col + Vector API + * path, in the sidecar. The weight is a parameter and stays resident after its first call; a heap + * input is staged per call and a resident one read in place; the output is resident, so a + * `conv → relu → …` chain never leaves linear memory. Groups (depthwise included), stride, + * padding and dilation go to the kernel as they are, a bias of `[c_out]` or `[1, c_out, 1, …]` + * is added there. Whatever the common validation would reject stays on the common path, which + * raises its message; `null` here means "not this path". + */ + private fun conv( + input: Tensor, weight: Tensor, bias: Tensor?, + spatial: Int, stride: IntArray, padding: IntArray, dilation: IntArray, groups: Int, + ): Tensor? { + if (!WasmKernels.isReady || !isSidecarFloat(input) || !isSidecarFloat(weight)) return null + if (bias != null && !isSidecarFloat(bias)) return null + val rank = spatial + 2 + if (input.rank != rank || weight.rank != rank || groups < 1) return null + val n = input.shape[0] + val cIn = input.shape[1] + val cOut = weight.shape[0] + if (n <= 0 || cOut <= 0 || cIn % groups != 0 || cOut % groups != 0 || weight.shape[1] != cIn / groups) return null + if (bias != null) { + val ok = (bias.rank == 1 && bias.shape[0] == cOut) || (bias.rank == rank && bias.shape[1] == cOut && bias.shape.volume == cOut) + if (!ok) return null + } + // Three spatial dims for the kernel; a lower rank fills the leading ones with the identity geometry. + val inDims = IntArray(3) { 1 } + val k = IntArray(3) { 1 } + val s = IntArray(3) { 1 } + val p = IntArray(3) + val d = IntArray(3) { 1 } + val outDims = IntArray(3) { 1 } + for (i in 0 until spatial) { + val j = 3 - spatial + i + inDims[j] = input.shape[2 + i] + k[j] = weight.shape[2 + i] + s[j] = stride[i] + p[j] = padding[i] + d[j] = dilation[i] + if (s[j] <= 0 || d[j] <= 0 || k[j] <= 0 || p[j] < 0) return null + outDims[j] = (inDims[j] + 2 * p[j] - d[j] * (k[j] - 1) - 1) / s[j] + 1 + if (outDims[j] <= 0) return null + } + val patchCount = outDims[0].toLong() * outDims[1] * outDims[2] + val colSize = (cIn / groups).toLong() * k[0] * k[1] * k[2] + val colBytes = patchCount * colSize * 4 + if (colSize <= 0 || colBytes > Int.MAX_VALUE) return null + val outShape = Shape(*(intArrayOf(n, cOut) + outDims.copyOfRange(3 - spatial, 3))) + val geo = intArrayOf( + n, cIn, cOut, groups, + inDims[0], inDims[1], inDims[2], k[0], k[1], k[2], s[0], s[1], s[2], + p[0], p[1], p[2], d[0], d[1], d[2], outDims[0], outDims[1], outDims[2], + ) + check(geo.size == WasmKernels.F32_CONV_GEO_INTS) + var io: Operand? = null + var wo: Operand? = null + var bo: Operand? = null + var colPtr = 0 + var geoPtr = 0 + try { + io = operand(input) ?: return null + wo = when (val wd = weight.data) { + is WasmLinearTensorData<*> -> operand(weight) ?: return null + is FloatArrayTensorData<*> -> Operand(WasmKernels.denseWeightHandle(wd.buffer), false) + else -> operand(weight) ?: return null + } + bo = bias?.let { operand(it) ?: return null } + colPtr = WasmKernels.alloc(colBytes.toInt()) + geoPtr = WasmKernels.alloc(geo.size * 4) + WasmKernels.writeInts(geoPtr, geo, 0, geo.size) + val inPtr = io.ptr + val wPtr = wo.ptr + val bPtr = bo?.ptr ?: 0 + val cPtr = colPtr + val gPtr = geoPtr + val inputs = listOfNotNull(input, weight, bias).toTypedArray() + return output(outShape, input.dtype, inputs) { out -> WasmKernels.f32Conv(inPtr, wPtr, bPtr, out, cPtr, gPtr) } + } finally { + if (geoPtr != 0) WasmKernels.free(geoPtr) + if (colPtr != 0) WasmKernels.free(colPtr) + release(bo) + release(wo) + release(io) + } + } + + override fun conv1d( + input: Tensor, weight: Tensor, bias: Tensor?, + stride: Int, padding: Int, dilation: Int, groups: Int, + ): Tensor = + conv(input, weight, bias, 1, intArrayOf(stride), intArrayOf(padding), intArrayOf(dilation), groups) + ?: super.conv1d(input, weight, bias, stride, padding, dilation, groups) + + override fun conv2d( + input: Tensor, weight: Tensor, bias: Tensor?, + stride: Pair, padding: Pair, dilation: Pair, groups: Int, + ): Tensor = + conv( + input, weight, bias, 2, + intArrayOf(stride.first, stride.second), intArrayOf(padding.first, padding.second), intArrayOf(dilation.first, dilation.second), groups, + ) ?: super.conv2d(input, weight, bias, stride, padding, dilation, groups) + + override fun conv3d( + input: Tensor, weight: Tensor, bias: Tensor?, + stride: Triple, padding: Triple, dilation: Triple, groups: Int, + ): Tensor = + conv( + input, weight, bias, 3, + intArrayOf(stride.first, stride.second, stride.third), + intArrayOf(padding.first, padding.second, padding.third), + intArrayOf(dilation.first, dilation.second, dilation.third), + groups, + ) ?: super.conv3d(input, weight, bias, stride, padding, dilation, groups) } diff --git a/skainet-backends/skainet-backend-wasm-simd/src/wasmJsMain/kotlin/sk/ainet/exec/wasm/WasmFusedKernels.kt b/skainet-backends/skainet-backend-wasm-simd/src/wasmJsMain/kotlin/sk/ainet/exec/wasm/WasmFusedKernels.kt new file mode 100644 index 000000000..aed183a2a --- /dev/null +++ b/skainet-backends/skainet-backend-wasm-simd/src/wasmJsMain/kotlin/sk/ainet/exec/wasm/WasmFusedKernels.kt @@ -0,0 +1,128 @@ +package sk.ainet.exec.wasm + +import sk.ainet.backend.api.kernel.Fp32AttentionKernel +import sk.ainet.backend.api.kernel.Fp32NormKernel +import sk.ainet.backend.api.kernel.Fp32SoftmaxKernel + +/* + * The fused row kernels of the sidecar (`f32_fused.c`) behind the SPI faces of `KernelProvider` + * (SKEEP-007 fused ops): the same accessors the JVM FFM provider answers, for callers that hold + * heap arrays. Every face stages its operands into the sidecar per call and reads the output + * back; [WasmCpuOps] bypasses them and hands the kernels resident operands directly. + */ + +/** `skainet_f32_softmax_rows` over heap arrays. */ +public object WasmFp32SoftmaxKernel : Fp32SoftmaxKernel { + override fun softmaxRows(input: FloatArray, inputOffset: Int, out: FloatArray, outOffset: Int, rows: Int, cols: Int, log: Boolean) { + require(rows >= 0 && cols >= 0) { "rows and cols must be non-negative; got rows=$rows cols=$cols" } + if (rows == 0 || cols == 0) return + val n = rows * cols + val inPtr = WasmKernels.alloc(n * 4) + val outPtr = WasmKernels.alloc(n * 4) + try { + WasmKernels.writeFloats(inPtr, input, inputOffset, n) + WasmKernels.f32SoftmaxRows(inPtr, outPtr, rows, cols, log) + WasmKernels.readFloats(outPtr, out, outOffset, n) + } finally { + WasmKernels.free(outPtr) + WasmKernels.free(inPtr) + } + } +} + +/** `skainet_f32_layer_norm` / `skainet_f32_rms_norm` over heap arrays. */ +public object WasmFp32NormKernel : Fp32NormKernel { + override fun layerNorm( + input: FloatArray, inputOffset: Int, + gamma: FloatArray?, beta: FloatArray?, + out: FloatArray, outOffset: Int, + rows: Int, cols: Int, eps: Float, + ): Unit = run(input, inputOffset, gamma, beta, out, outOffset, rows, cols, eps, rms = false) + + override fun rmsNorm( + input: FloatArray, inputOffset: Int, + gamma: FloatArray?, + out: FloatArray, outOffset: Int, + rows: Int, cols: Int, eps: Float, + ): Unit = run(input, inputOffset, gamma, null, out, outOffset, rows, cols, eps, rms = true) + + private fun run( + input: FloatArray, inputOffset: Int, + gamma: FloatArray?, beta: FloatArray?, + out: FloatArray, outOffset: Int, + rows: Int, cols: Int, eps: Float, rms: Boolean, + ) { + require(rows >= 0 && cols >= 0) { "rows and cols must be non-negative; got rows=$rows cols=$cols" } + if (rows == 0 || cols == 0) return + require(gamma == null || gamma.size >= cols) { "gamma has ${gamma?.size} values, the row has $cols" } + require(beta == null || beta.size >= cols) { "beta has ${beta?.size} values, the row has $cols" } + val n = rows * cols + val inPtr = WasmKernels.alloc(n * 4) + val outPtr = WasmKernels.alloc(n * 4) + val gPtr = if (gamma != null) WasmKernels.alloc(cols * 4) else 0 + val bPtr = if (beta != null) WasmKernels.alloc(cols * 4) else 0 + try { + WasmKernels.writeFloats(inPtr, input, inputOffset, n) + if (gamma != null) WasmKernels.writeFloats(gPtr, gamma, 0, cols) + if (beta != null) WasmKernels.writeFloats(bPtr, beta, 0, cols) + if (rms) WasmKernels.f32RmsNorm(inPtr, gPtr, outPtr, rows, cols, eps) + else WasmKernels.f32LayerNorm(inPtr, gPtr, bPtr, outPtr, rows, cols, eps) + WasmKernels.readFloats(outPtr, out, outOffset, n) + } finally { + if (bPtr != 0) WasmKernels.free(bPtr) + if (gPtr != 0) WasmKernels.free(gPtr) + WasmKernels.free(outPtr) + WasmKernels.free(inPtr) + } + } +} + +/** `skainet_f32_sdpa` over heap arrays. */ +public object WasmFp32AttentionKernel : Fp32AttentionKernel { + override fun sdpa( + query: FloatArray, key: FloatArray, value: FloatArray, out: FloatArray, + batch: Int, heads: Int, kvHeads: Int, + seqQ: Int, seqKV: Int, headDim: Int, + scale: Float, causal: Boolean, window: Int, rightContext: Int, + keyLengths: IntArray?, + mask: FloatArray?, maskDims: IntArray?, + ) { + require(batch >= 0 && heads >= 0 && kvHeads >= 0 && seqQ >= 0 && seqKV >= 0 && headDim >= 0) { "dimensions must be non-negative" } + require(kvHeads > 0 && heads % kvHeads == 0) { "heads ($heads) must be a multiple of kvHeads ($kvHeads)" } + require(keyLengths == null || keyLengths.size == batch) { "keyLengths has ${keyLengths?.size} rows, batch is $batch" } + require((mask == null) == (maskDims == null) && (maskDims == null || maskDims.size == 4)) { "mask and maskDims (4 dims) go together" } + val outCount = batch * heads * seqQ * headDim + require(outCount <= out.size) { "output needs $outCount floats, array has ${out.size}" } + if (outCount == 0) return + val kvCount = batch * kvHeads * seqKV * headDim + val md = maskDims ?: intArrayOf(1, 1, 1, 1) + val maskCount = md[0] * md[1] * md[2] * md[3] + val qPtr = WasmKernels.alloc(outCount * 4) + val kPtr = WasmKernels.alloc(maxOf(kvCount, 1) * 4) + val vPtr = WasmKernels.alloc(maxOf(kvCount, 1) * 4) + val outPtr = WasmKernels.alloc(outCount * 4) + val lPtr = if (keyLengths != null) WasmKernels.alloc(batch * 4) else 0 + val mPtr = if (mask != null) WasmKernels.alloc(maxOf(maskCount, 1) * 4) else 0 + try { + WasmKernels.writeFloats(qPtr, query, 0, outCount) + if (kvCount > 0) { + WasmKernels.writeFloats(kPtr, key, 0, kvCount) + WasmKernels.writeFloats(vPtr, value, 0, kvCount) + } + if (keyLengths != null) WasmKernels.writeInts(lPtr, keyLengths, 0, batch) + if (mask != null && maskCount > 0) WasmKernels.writeFloats(mPtr, mask, 0, maskCount) + WasmKernels.f32Sdpa( + qPtr, kPtr, vPtr, outPtr, batch, heads, kvHeads, seqQ, seqKV, headDim, + scale, causal, window, rightContext, lPtr, mPtr, md[0], md[1], md[2], md[3], + ) + WasmKernels.readFloats(outPtr, out, 0, outCount) + } finally { + if (mPtr != 0) WasmKernels.free(mPtr) + if (lPtr != 0) WasmKernels.free(lPtr) + WasmKernels.free(outPtr) + WasmKernels.free(vPtr) + WasmKernels.free(kPtr) + WasmKernels.free(qPtr) + } + } +} diff --git a/skainet-backends/skainet-backend-wasm-simd/src/wasmJsMain/kotlin/sk/ainet/exec/wasm/WasmKernelProvider.kt b/skainet-backends/skainet-backend-wasm-simd/src/wasmJsMain/kotlin/sk/ainet/exec/wasm/WasmKernelProvider.kt index 77848742e..23b1d505f 100644 --- a/skainet-backends/skainet-backend-wasm-simd/src/wasmJsMain/kotlin/sk/ainet/exec/wasm/WasmKernelProvider.kt +++ b/skainet-backends/skainet-backend-wasm-simd/src/wasmJsMain/kotlin/sk/ainet/exec/wasm/WasmKernelProvider.kt @@ -3,6 +3,10 @@ package sk.ainet.exec.wasm import sk.ainet.backend.api.kernel.Bf16MatmulKernel import sk.ainet.backend.api.kernel.Fp16MatmulKernel import sk.ainet.backend.api.kernel.Fp32MatmulKernel +import sk.ainet.backend.api.kernel.Fp32AttentionKernel +import sk.ainet.backend.api.kernel.Fp32MatmulWtKernel +import sk.ainet.backend.api.kernel.Fp32NormKernel +import sk.ainet.backend.api.kernel.Fp32SoftmaxKernel import sk.ainet.backend.api.kernel.KernelProvider import sk.ainet.backend.api.kernel.Q4KMatmulKernel import sk.ainet.backend.api.kernel.Q4_0MatmulKernel @@ -14,8 +18,12 @@ import sk.ainet.backend.api.kernel.Q8_0MatmulKernel /** * The sidecar as a [KernelProvider] (SKEEP-007 §T1): the dense FP32 GEMM and the feed-order packed - * kernels, so `KernelPacks.install()` and `chooseQuantizedMatmulHeap` serve from SIMD128 as well. - * Priority 50 — above the scalar provider (0), the same tier the JVM's Panama provider sits at. + * kernels, so `KernelPacks.install()` and `chooseQuantizedMatmulHeap` serve from SIMD128 as well, + * plus the fused row kernels (softmax, the normalisations, attention — [WasmFp32SoftmaxKernel], + * [WasmFp32NormKernel], [WasmFp32AttentionKernel]) the JVM FFM provider exposes under the same + * accessors. Priority 50 — above the scalar provider (0), the same tier the JVM's Panama provider + * sits at. [WasmCpuOps] does not go through these heap faces: it hands the sidecar resident + * operands directly. */ public object WasmKernelProvider : KernelProvider { override val name: String = "wasm-simd128" @@ -23,8 +31,12 @@ public object WasmKernelProvider : KernelProvider { override fun isAvailable(): Boolean = WasmKernels.isReady override fun matmulFp32(): Fp32MatmulKernel = WasmFp32MatmulKernel + override fun matmulFp32Wt(): Fp32MatmulWtKernel = WasmFp32MatmulWtKernel override fun matmulBf16(): Bf16MatmulKernel = WasmBf16MatmulKernel override fun matmulFp16(): Fp16MatmulKernel = WasmFp16MatmulKernel + override fun softmaxFp32(): Fp32SoftmaxKernel = WasmFp32SoftmaxKernel + override fun normFp32(): Fp32NormKernel = WasmFp32NormKernel + override fun attentionFp32(): Fp32AttentionKernel = WasmFp32AttentionKernel override fun matmulQ8_0(): Q8_0MatmulKernel = object : Q8_0MatmulKernel { override fun matmul(input: FloatArray, inputOffset: Int, weight: ByteArray, weightByteOffset: Int, inputDim: Int, outputDim: Int, output: FloatArray, outputOffset: Int) = @@ -165,3 +177,36 @@ public object WasmFp32MatmulKernel : Fp32MatmulKernel { } } } + +/** + * `out(m,n) = x(m,k) · w(n,k)ᵀ` in the sidecar (`skainet_fp32_matmul_wt`). A whole-array weight is a + * parameter and stays resident after its first call ([WasmKernels.denseWeightHandle]); a windowed + * weight is staged per call, as the activation always is. + */ +public object WasmFp32MatmulWtKernel : Fp32MatmulWtKernel { + override fun matmulWt( + x: FloatArray, xOffset: Int, + w: FloatArray, wOffset: Int, + out: FloatArray, outOffset: Int, + m: Int, n: Int, k: Int, + ) { + require(m >= 0 && n >= 0 && k >= 0) { "m, n, k must be non-negative; got m=$m n=$n k=$k" } + if (m == 0 || n == 0) return + val resident = wOffset == 0 && w.size == n * k + val inPtr = WasmKernels.alloc(maxOf(m * k, 1) * 4) + val wPtr = if (resident) WasmKernels.denseWeightHandle(w) else WasmKernels.alloc(maxOf(n * k, 1) * 4) + val outPtr = WasmKernels.alloc(m * n * 4) + try { + if (k > 0) { + WasmKernels.writeFloats(inPtr, x, xOffset, m * k) + if (!resident) WasmKernels.writeFloats(wPtr, w, wOffset, n * k) + } + WasmKernels.fp32MatmulWt(inPtr, wPtr, outPtr, m, n, k) + WasmKernels.readFloats(outPtr, out, outOffset, m * n) + } finally { + WasmKernels.free(outPtr) + if (!resident) WasmKernels.free(wPtr) + WasmKernels.free(inPtr) + } + } +} diff --git a/skainet-backends/skainet-backend-wasm-simd/src/wasmJsMain/kotlin/sk/ainet/exec/wasm/WasmKernels.kt b/skainet-backends/skainet-backend-wasm-simd/src/wasmJsMain/kotlin/sk/ainet/exec/wasm/WasmKernels.kt index 19b00a64c..9ce6d14ea 100644 --- a/skainet-backends/skainet-backend-wasm-simd/src/wasmJsMain/kotlin/sk/ainet/exec/wasm/WasmKernels.kt +++ b/skainet-backends/skainet-backend-wasm-simd/src/wasmJsMain/kotlin/sk/ainet/exec/wasm/WasmKernels.kt @@ -51,6 +51,7 @@ public object WasmKernels { loadEmbeddedSidecar(mjs, wasm).await() } modeName = if (mt) "simd128-mt" else "simd128-st" + jsResidentState(mod()) } /** Drop the sidecar and every cached weight; the next [initialize] starts over. Tests only. */ @@ -138,6 +139,70 @@ public object WasmKernels { } } + /** Copy [count] bytes from sidecar memory at [srcPtr] into [dst]. */ + public fun readBytes(srcPtr: Int, dst: ByteArray, dstOffset: Int, count: Int) { + val m = mod() + var done = 0 + while (done < count) { + val n = minOf(count - done, CHUNK_BYTES) + withScopedMemoryAllocator { allocator -> + val base = allocator.allocate(n) + jsCopyOut(m, wasmMemory, srcPtr + done, base.address.toInt(), n) + var p: Pointer = base + var i = 0 + while (i + 4 <= n) { + val v = p.loadInt() + val o = dstOffset + done + i + dst[o] = (v and 0xFF).toByte() + dst[o + 1] = ((v ushr 8) and 0xFF).toByte() + dst[o + 2] = ((v ushr 16) and 0xFF).toByte() + dst[o + 3] = ((v ushr 24) and 0xFF).toByte() + p = p + 4 + i += 4 + } + while (i < n) { + dst[dstOffset + done + i] = p.loadByte() + p = p + 1 + i++ + } + } + done += n + } + } + + /** Copy [bytes] bytes inside sidecar memory from [srcPtr] to [dstPtr]. */ + public fun copy(srcPtr: Int, dstPtr: Int, bytes: Int) { + if (bytes > 0) jsCopyWithin(mod(), srcPtr, dstPtr, bytes) + } + + /** Zero [bytes] bytes of sidecar memory at [ptr]. */ + public fun zero(ptr: Int, bytes: Int) { + if (bytes > 0) jsZero(mod(), ptr, bytes) + } + + // ── resident activations (Storage.WasmLinear) ───────────────────────────────────────────────── + + /** + * Sidecar bytes currently held by resident activations ([WasmLinearStorage]), scoped and + * ambient — the figure [WasmSimd.residentBudgetBytes] is checked against. + */ + public fun residentActivationBytes(): Long = if (isReady) jsResidentBytes(mod()).toLong() else 0L + + /** Allocate [bytes] for a resident activation; counted in [residentActivationBytes]. */ + internal fun allocResident(bytes: Int): Int = jsResidentAlloc(mod(), bytes) + + /** Free a resident activation's bytes and stop counting them. */ + internal fun freeResident(ptr: Int, bytes: Int): Unit = jsResidentFree(mod(), ptr, bytes) + + /** + * Register an ambient (GC-managed) resident allocation with the sidecar's `FinalizationRegistry`: + * the returned token is held by the Kotlin storage object, and when that object is collected the + * registry frees the sidecar bytes. Explicit `close()` must [untrackResident] first. + */ + internal fun trackResident(ptr: Int, bytes: Int): JsAny = jsResidentTrack(mod(), ptr, bytes) + + internal fun untrackResident(token: JsAny): Unit = jsResidentUntrack(mod(), token) + // ── resident weights ────────────────────────────────────────────────────────────────────────── private class Resident(val ptr: Int, val bytes: Int) @@ -213,6 +278,111 @@ public object WasmKernels { public fun narrowFloatMatmul(fnName: String, aPtr: Int, aStride: Int, bPtr: Int, bByteStride: Int, cPtr: Int, cStride: Int, m: Int, n: Int, k: Int): Unit = jsNarrowFloatMatmul(mod(), fnName, aPtr, aStride, bPtr, bByteStride, cPtr, cStride, m, n, k) + /** + * `skainet_ternary_f32_gemv`: one FP32 activation row against a sequentially packed BitNet b1.58 + * weight (`inputDim / 4` bytes per output row), per-tensor scale not applied. + */ + public fun ternaryF32Gemv(inPtr: Int, weightPtr: Int, inputDim: Int, outputDim: Int, outPtr: Int): Unit = + jsTernaryF32Gemv(mod(), inPtr, weightPtr, inputDim, outputDim, outPtr) + + /** + * `skainet_ternary_lmhead_stage1`: four planes at [planesPtr] (each [planeStrideBytes] apart) with + * the fused weights `{1, ⅓, ⅑, 1/27}`, times the FP16 row scales at [rowScalePtr]. + */ + public fun ternaryLmheadStage1(inPtr: Int, planesPtr: Int, planeStrideBytes: Int, rowScalePtr: Int, inputDim: Int, outputDim: Int, outPtr: Int): Unit = + jsTernaryLmheadStage1(mod(), inPtr, planesPtr, planeStrideBytes, rowScalePtr, inputDim, outputDim, outPtr) + + /** `skainet_bitnet_gemv_tq2_0`: int8 activation codes with one absmax scale against TQ2_0 blocks. */ + public fun bitnetGemvTq2_0(activationPtr: Int, activationScale: Float, weightPtr: Int, inputDim: Int, outputDim: Int, outPtr: Int): Unit = + jsBitnetGemvTq2_0(mod(), activationPtr, activationScale, weightPtr, inputDim, outputDim, outPtr) + + /** Operand index modes of [f32Binary] (`skainet_f32_binary`). */ + public const val F32_FULL: Int = 0 + public const val F32_SCALAR: Int = 1 + public const val F32_BIAS: Int = 2 + + /** Operators of [f32Binary]. */ + public const val F32_ADD: Int = 0 + public const val F32_SUB: Int = 1 + public const val F32_MUL: Int = 2 + public const val F32_DIV: Int = 3 + + /** Functions of [f32Unary] (`skainet_f32_unary`). */ + public const val F32_RELU: Int = 0 + public const val F32_SIGMOID: Int = 1 + public const val F32_SILU: Int = 2 + public const val F32_GELU_TANH: Int = 3 + public const val F32_GELU_EXACT: Int = 4 + public const val F32_ERF: Int = 5 + public const val F32_TANH: Int = 6 + + /** `skainet_f32_binary`: `out[i] = a[ia] op b[ib]` over [n] outputs, each side indexed by its mode (full, scalar, last-dim bias of [inner]). */ + public fun f32Binary(op: Int, aPtr: Int, aMode: Int, bPtr: Int, bMode: Int, outPtr: Int, n: Int, inner: Int): Unit = + jsF32Binary(mod(), op, aPtr, aMode, bPtr, bMode, outPtr, n, inner) + + /** `skainet_f32_unary`: `out[i] = f(in[i])`. */ + public fun f32Unary(op: Int, inPtr: Int, outPtr: Int, n: Int): Unit = jsF32Unary(mod(), op, inPtr, outPtr, n) + + /** `skainet_f32_reduce_all`: the sum (or mean) of [n] floats into the single float at [outPtr]. */ + public fun f32ReduceAll(inPtr: Int, n: Int, mean: Boolean, outPtr: Int): Unit = + jsF32ReduceAll(mod(), inPtr, n, if (mean) 1 else 0, outPtr) + + /** `skainet_f32_reduce_axis`: sum (or mean) over the middle of an `(outer, red, inner)` decomposition. */ + public fun f32ReduceAxis(inPtr: Int, outer: Int, red: Int, inner: Int, mean: Boolean, outPtr: Int): Unit = + jsF32ReduceAxis(mod(), inPtr, outer, red, inner, if (mean) 1 else 0, outPtr) + + /** `skainet_f32_softmax_rows`: row-wise softmax (or log-softmax when [log]) over [rows] rows of [cols]. */ + public fun f32SoftmaxRows(inPtr: Int, outPtr: Int, rows: Int, cols: Int, log: Boolean): Unit = + jsF32SoftmaxRows(mod(), inPtr, outPtr, rows, cols, if (log) 1 else 0) + + /** `skainet_f32_layer_norm` over [rows] rows of [cols]; [gammaPtr] / [betaPtr] `0` skip the affine. */ + public fun f32LayerNorm(inPtr: Int, gammaPtr: Int, betaPtr: Int, outPtr: Int, rows: Int, cols: Int, eps: Float): Unit = + jsF32LayerNorm(mod(), inPtr, gammaPtr, betaPtr, outPtr, rows, cols, eps) + + /** `skainet_f32_rms_norm` over [rows] rows of [cols]; [gammaPtr] `0` skips the scale. */ + public fun f32RmsNorm(inPtr: Int, gammaPtr: Int, outPtr: Int, rows: Int, cols: Int, eps: Float): Unit = + jsF32RmsNorm(mod(), inPtr, gammaPtr, outPtr, rows, cols, eps) + + /** + * `skainet_f32_sdpa`: fused scaled dot-product attention with the ops layer's structural masks + * (causal, sliding window with right context, per-batch key lengths at [lengthsPtr] or `0`) and + * an optional dense additive mask at [maskPtr] (`0` for none) broadcast over the four mask dims. + */ + public fun f32Sdpa( + qPtr: Int, kPtr: Int, vPtr: Int, outPtr: Int, + batch: Int, heads: Int, kvHeads: Int, seqQ: Int, seqKV: Int, headDim: Int, + scale: Float, causal: Boolean, window: Int, rightContext: Int, + lengthsPtr: Int, maskPtr: Int, maskB: Int, maskH: Int, maskQ: Int, maskK: Int, + ): Unit = jsF32Sdpa( + mod(), qPtr, kPtr, vPtr, outPtr, batch, heads, kvHeads, seqQ, seqKV, headDim, + scale, if (causal) 1 else 0, window, rightContext, lengthsPtr, maskPtr, maskB, maskH, maskQ, maskK, + ) + + /** Int32 count of the geometry block `skainet_f32_conv` reads (`f32_conv.c` names the order). */ + public const val F32_CONV_GEO_INTS: Int = 22 + + /** + * `skainet_f32_conv`: dense FP32 conv1d/2d/3d as im2col + GEMM. [geoPtr] holds the + * [F32_CONV_GEO_INTS] int32 geometry (batch, channels, groups, three spatial dims each of input, + * kernel, stride, padding, dilation, output), [colPtr] is scratch of patch_count × col_size + * floats, [biasPtr] `0` for none. + */ + public fun f32Conv(inPtr: Int, wPtr: Int, biasPtr: Int, outPtr: Int, colPtr: Int, geoPtr: Int): Unit = + jsF32Conv(mod(), inPtr, wPtr, biasPtr, outPtr, colPtr, geoPtr) + + /** Write [count] ints of [src] from [srcOffset] at [dstPtr], little-endian (the sidecar's int32). */ + public fun writeInts(dstPtr: Int, src: IntArray, srcOffset: Int, count: Int) { + val bytes = ByteArray(count * 4) + for (i in 0 until count) { + val v = src[srcOffset + i] + bytes[i * 4] = v.toByte() + bytes[i * 4 + 1] = (v shr 8).toByte() + bytes[i * 4 + 2] = (v shr 16).toByte() + bytes[i * 4 + 3] = (v shr 24).toByte() + } + writeBytes(dstPtr, bytes, 0, count * 4) + } + // ── diagnostics ─────────────────────────────────────────────────────────────────────────────── /** @@ -227,10 +397,11 @@ public object WasmKernels { val crossOriginIsolated: Boolean, val hardwareConcurrency: Int, val residentWeightBytes: Long, + val residentActivationBytes: Long = 0L, ) { override fun toString(): String = "mode=$mode simd128=$simd128 threads=$threads crossOriginIsolated=$crossOriginIsolated " + - "cores=$hardwareConcurrency residentWeightBytes=$residentWeightBytes" + "cores=$hardwareConcurrency residentWeightBytes=$residentWeightBytes residentActivationBytes=$residentActivationBytes" } /** Current sidecar capabilities; `mode == "none"` before [initialize]. */ @@ -241,6 +412,7 @@ public object WasmKernels { crossOriginIsolated = jsCrossOriginIsolated(), hardwareConcurrency = jsHardwareConcurrency(), residentWeightBytes = if (isReady) residentWeightBytes() else 0L, + residentActivationBytes = residentActivationBytes(), ) } @@ -301,6 +473,115 @@ private fun jsFp32MatmulWt(mod: JsAny, inPtr: Int, wPtr: Int, outPtr: Int, m: In private fun jsNarrowFloatMatmul(mod: JsAny, fnName: String, aPtr: Int, aStride: Int, bPtr: Int, bByteStride: Int, cPtr: Int, cStride: Int, m: Int, n: Int, k: Int): Unit = js("mod[fnName](aPtr, 0, aStride, bPtr, 0, bByteStride, cPtr, 0, cStride, m, n, k)") +private fun jsTernaryF32Gemv(mod: JsAny, inPtr: Int, weightPtr: Int, inputDim: Int, outputDim: Int, outPtr: Int): Unit = + js("mod._skainet_ternary_f32_gemv(inPtr, 0, weightPtr, 0, inputDim, outputDim, outPtr, 0)") + +private fun jsTernaryLmheadStage1(mod: JsAny, inPtr: Int, planesPtr: Int, planeStrideBytes: Int, rowScalePtr: Int, inputDim: Int, outputDim: Int, outPtr: Int): Unit = + js("mod._skainet_ternary_lmhead_stage1(inPtr, 0, planesPtr, 0, planeStrideBytes, rowScalePtr, 0, inputDim, outputDim, outPtr, 0)") + +private fun jsBitnetGemvTq2_0(mod: JsAny, activationPtr: Int, activationScale: Float, weightPtr: Int, inputDim: Int, outputDim: Int, outPtr: Int): Unit = + js("mod._skainet_bitnet_gemv_tq2_0(activationPtr, 0, activationScale, weightPtr, 0, inputDim, outputDim, outPtr, 0)") + +private fun jsCopyWithin(mod: JsAny, srcPtr: Int, dstPtr: Int, bytes: Int): Unit = + js("new Uint8Array(mod.wasmMemory.buffer).copyWithin(dstPtr, srcPtr, srcPtr + bytes)") + +private fun jsZero(mod: JsAny, ptr: Int, bytes: Int): Unit = + js("new Uint8Array(mod.wasmMemory.buffer, ptr, bytes).fill(0)") + +// Resident-activation accounting lives on the module object: the byte count and, where the runtime +// has one, a FinalizationRegistry that frees an ambient allocation when its Kotlin storage object has +// been collected (Kotlin/Wasm objects are traced by the same GC as the token they hold). +private fun jsResidentState(mod: JsAny): JsAny = js( + """ + (function() { + if (!mod.__skainetResident) { + var st = { bytes: 0, registry: null }; + if (typeof FinalizationRegistry !== 'undefined') { + st.registry = new FinalizationRegistry(function (h) { mod._free(h.ptr); st.bytes -= h.bytes; }); + } + mod.__skainetResident = st; + } + return mod.__skainetResident; + })() + """, +) + +private fun jsResidentBytes(mod: JsAny): Double = js("(mod.__skainetResident ? mod.__skainetResident.bytes : 0)") + +private fun jsResidentAlloc(mod: JsAny, bytes: Int): Int = js( + """ + (function() { + var st = mod.__skainetResident; + var p = mod._malloc(bytes); + st.bytes += bytes; + return p; + })() + """, +) + +private fun jsResidentFree(mod: JsAny, ptr: Int, bytes: Int): Unit = js( + """ + (function() { + var st = mod.__skainetResident; + mod._free(ptr); + st.bytes -= bytes; + })() + """, +) + +private fun jsResidentTrack(mod: JsAny, ptr: Int, bytes: Int): JsAny = js( + """ + (function() { + var st = mod.__skainetResident; + var token = {}; + if (st.registry) st.registry.register(token, { ptr: ptr, bytes: bytes }, token); + return token; + })() + """, +) + +private fun jsResidentUntrack(mod: JsAny, token: JsAny): Unit = js( + """ + (function() { + var st = mod.__skainetResident; + if (st && st.registry) st.registry.unregister(token); + })() + """, +) + +private fun jsF32Binary(mod: JsAny, op: Int, aPtr: Int, aMode: Int, bPtr: Int, bMode: Int, outPtr: Int, n: Int, inner: Int): Unit = + js("mod._skainet_f32_binary(op, aPtr, aMode, bPtr, bMode, outPtr, n, inner)") + +private fun jsF32Unary(mod: JsAny, op: Int, inPtr: Int, outPtr: Int, n: Int): Unit = + js("mod._skainet_f32_unary(op, inPtr, outPtr, n)") + +private fun jsF32ReduceAll(mod: JsAny, inPtr: Int, n: Int, mean: Int, outPtr: Int): Unit = + js("mod._skainet_f32_reduce_all(inPtr, n, mean, outPtr)") + +private fun jsF32ReduceAxis(mod: JsAny, inPtr: Int, outer: Int, red: Int, inner: Int, mean: Int, outPtr: Int): Unit = + js("mod._skainet_f32_reduce_axis(inPtr, outer, red, inner, mean, outPtr)") + +private fun jsF32SoftmaxRows(mod: JsAny, inPtr: Int, outPtr: Int, rows: Int, cols: Int, log: Int): Unit = + js("mod._skainet_f32_softmax_rows(inPtr, outPtr, rows, cols, log)") + +private fun jsF32LayerNorm(mod: JsAny, inPtr: Int, gammaPtr: Int, betaPtr: Int, outPtr: Int, rows: Int, cols: Int, eps: Float): Unit = + js("mod._skainet_f32_layer_norm(inPtr, gammaPtr, betaPtr, outPtr, rows, cols, eps)") + +private fun jsF32RmsNorm(mod: JsAny, inPtr: Int, gammaPtr: Int, outPtr: Int, rows: Int, cols: Int, eps: Float): Unit = + js("mod._skainet_f32_rms_norm(inPtr, gammaPtr, outPtr, rows, cols, eps)") + +private fun jsF32Sdpa( + mod: JsAny, qPtr: Int, kPtr: Int, vPtr: Int, outPtr: Int, + batch: Int, heads: Int, kvHeads: Int, seqQ: Int, seqKV: Int, headDim: Int, + scale: Float, causal: Int, window: Int, rightContext: Int, + lengthsPtr: Int, maskPtr: Int, maskB: Int, maskH: Int, maskQ: Int, maskK: Int, +): Unit = js( + "mod._skainet_f32_sdpa(qPtr, kPtr, vPtr, outPtr, batch, heads, kvHeads, seqQ, seqKV, headDim, scale, causal, window, rightContext, lengthsPtr, maskPtr, maskB, maskH, maskQ, maskK)" +) + +private fun jsF32Conv(mod: JsAny, inPtr: Int, wPtr: Int, biasPtr: Int, outPtr: Int, colPtr: Int, geoPtr: Int): Unit = + js("mod._skainet_f32_conv(inPtr, wPtr, biasPtr, outPtr, colPtr, geoPtr)") + private fun jsCrossOriginIsolated(): Boolean = js("typeof crossOriginIsolated !== 'undefined' && crossOriginIsolated === true") private fun jsHardwareConcurrency(): Int = js( diff --git a/skainet-backends/skainet-backend-wasm-simd/src/wasmJsMain/kotlin/sk/ainet/exec/wasm/WasmLinearStorage.kt b/skainet-backends/skainet-backend-wasm-simd/src/wasmJsMain/kotlin/sk/ainet/exec/wasm/WasmLinearStorage.kt new file mode 100644 index 000000000..055a2a46a --- /dev/null +++ b/skainet-backends/skainet-backend-wasm-simd/src/wasmJsMain/kotlin/sk/ainet/exec/wasm/WasmLinearStorage.kt @@ -0,0 +1,195 @@ +@file:OptIn(kotlin.js.ExperimentalWasmJsInterop::class) + +package sk.ainet.exec.wasm + +import sk.ainet.lang.memory.OffHeapAllocator +import sk.ainet.lang.memory.Owner +import sk.ainet.lang.memory.ScopeKind +import sk.ainet.lang.memory.Storage +import sk.ainet.lang.memory.StorageId +import sk.ainet.lang.memory.trace.NoopTraceSink +import sk.ainet.lang.memory.trace.TraceEvent +import sk.ainet.lang.memory.trace.TraceSink +import sk.ainet.lang.tensor.Shape +import sk.ainet.lang.tensor.TensorId +import sk.ainet.lang.tensor.data.FloatArrayTensorData +import sk.ainet.lang.types.DType + +/** + * The wasm tier's [Storage.OffHeap]: a region of the SIMD128 sidecar's linear memory + * (SKEEP-007 resident activations). Bytes here are read and written by the sidecar's kernels in + * place; the Kotlin side sees them through [copyInto]/[copyFrom] (bulk copies across the module + * boundary), never element by element. This is the binding `PlatformStorage.offHeapAllocator` + * receives from [WasmSimd.install], so a `ForwardScope` or `ModelScope` asking for + * `MemoryDomain.HOST_OFFHEAP` gets sidecar memory that its `reset()`/`close()` frees. + * + * Lifetime. A scoped storage ([ScopeKind.FORWARD] / [ScopeKind.MODEL]) is freed exactly once, by its + * scope through [close]. An ambient storage ([ScopeKind.AMBIENT], the default for op outputs in a + * notebook-style forward pass) has no scope to free it, so it is registered with a + * `FinalizationRegistry` in the sidecar glue: when this Kotlin object is collected the registry + * frees the sidecar bytes — the same GC-managed lifetime a heap `FloatArray` has, one hop later. + * Finalization is not prompt, which is why the ops layer also keeps resident activations under + * [WasmSimd.residentBudgetBytes] and falls back to heap outputs above it. + */ +public class WasmLinearStorage private constructor( + override val id: StorageId, + private val base: Int, + override val sizeBytes: Long, + override val owner: Owner, + override val debugOrigin: TensorId?, + override val sink: TraceSink, + private val token: JsAny?, + private val mutable: Boolean, +) : Storage.OffHeap() { + + override val isMutable: Boolean get() = (owner as? Owner.Alias)?.parent?.isMutable ?: mutable + + /** The byte address in sidecar memory — kernels take it once per call. Throws when closed. */ + public fun pointer(): Int { + checkAlive() + return base + } + + override fun slice(offsetBytes: Long, lengthBytes: Long): WasmLinearStorage { + checkAlive() + require(offsetBytes >= 0 && lengthBytes >= 0 && offsetBytes + lengthBytes <= sizeBytes) { + "slice [$offsetBytes, ${offsetBytes + lengthBytes}) outside $sizeBytes bytes" + } + return WasmLinearStorage(StorageId.next(), base + offsetBytes.toInt(), lengthBytes, Owner.Alias(this), debugOrigin, sink, null, mutable) + } + + override fun copyInto(dest: ByteArray, destOffset: Int, offset: Long, length: Int) { + checkAlive() + require(offset >= 0 && length >= 0 && offset + length <= sizeBytes) { "copyInto [$offset, ${offset + length}) outside $sizeBytes bytes" } + WasmKernels.readBytes(base + offset.toInt(), dest, destOffset, length) + } + + override fun copyFrom(src: ByteArray, srcOffset: Int, offset: Long, length: Int) { + checkAlive() + require(isMutable) { "storage $id is not mutable" } + require(offset >= 0 && length >= 0 && offset + length <= sizeBytes) { "copyFrom [$offset, ${offset + length}) outside $sizeBytes bytes" } + WasmKernels.writeBytes(base + offset.toInt(), src, srcOffset, length) + } + + override fun onClose() { + if (owner !is Owner.Owned || !WasmKernels.isReady) return + token?.let { WasmKernels.untrackResident(it) } + WasmKernels.freeResident(base, sizeBytes.toInt()) + } + + override fun toString(): String = + "WasmLinear($id, $sizeBytes B @$base, $owner${debugOrigin?.let { ", $it" } ?: ""}${if (isAlive) "" else ", closed"})" + + /** The hook [WasmSimd.install] hands to `PlatformStorage.offHeapAllocator`. */ + public object Allocator : OffHeapAllocator { + override fun allocate(bytes: Long, scope: ScopeKind, origin: TensorId?, sink: TraceSink): Storage = + WasmLinearStorage.allocate(bytes, scope, origin, sink) + + /** Resident weights plus resident activations in the sidecar — the bytes `MemoryProbe` cannot see otherwise. */ + override fun heldBytes(): Long? = + if (WasmKernels.isReady) WasmKernels.residentWeightBytes() + WasmKernels.residentActivationBytes() else 0L + } + + public companion object { + /** + * Allocate [bytes] bytes of sidecar memory, zeroed unless [zero] is `false` (an op output that + * is fully written next), owned by a scope of kind [scope]. Requires the sidecar to be up. + */ + public fun allocate( + bytes: Long, + scope: ScopeKind = ScopeKind.AMBIENT, + origin: TensorId? = null, + sink: TraceSink = NoopTraceSink, + zero: Boolean = true, + ): WasmLinearStorage { + require(bytes in 0..Int.MAX_VALUE.toLong()) { "sidecar storage is limited to 2 GB per allocation, requested $bytes bytes" } + check(WasmKernels.isReady) { "WasmSimd.install() has not completed: no sidecar memory to allocate from" } + val size = bytes.toInt() + val ptr = WasmKernels.allocResident(size) + if (zero) WasmKernels.zero(ptr, size) + val token = if (scope == ScopeKind.AMBIENT) WasmKernels.trackResident(ptr, size) else null + val s = WasmLinearStorage(StorageId.next(), ptr, bytes, Owner.Owned(scope), origin, sink, token, true) + if (sink.isEnabled) sink.emit(TraceEvent.Allocation(s.id.value, scope, bytes, origin)) + return s + } + } +} + +/** + * Dense FP32 tensor data whose bytes live in the sidecar ([WasmLinearStorage]) — what the ops of + * [WasmCpuOps] hand out, so a chain `matmul → gelu → multiply → add` never copies an activation back + * to the Kotlin heap between kernels. + * + * It is a [FloatArrayTensorData] so every common fast path keeps working on it: the first access to + * [buffer] (or to an element, or [copyToFloatArray], or [view]) downloads the whole tensor once into + * a heap mirror that is kept for the tensor's life. A sidecar kernel never triggers that — it reads + * [pointer] — and the download is the same single bulk copy the matmul output paid before resident + * activations existed. Ops do not mutate their inputs, so the mirror and the sidecar copy agree; + * [set] writes both. + */ +public class WasmLinearTensorData( + initialShape: Shape, + public val storage: WasmLinearStorage, +) : FloatArrayTensorData { + + override val shape: Shape = Shape(initialShape.dimensions.copyOf()) + private val strides: IntArray = IntArray(shape.rank).also { st -> + var stride = 1 + for (i in shape.rank - 1 downTo 0) { + st[i] = stride + stride *= shape.dimensions[i] + } + } + private var mirror: FloatArray? = null + + init { + require(storage.sizeBytes >= shape.volume.toLong() * 4) { + "storage holds ${storage.sizeBytes} bytes, shape $shape needs ${shape.volume.toLong() * 4}" + } + } + + /** `true` while the sidecar bytes are alive: kernels use [pointer] and nothing is downloaded. */ + public val isResident: Boolean get() = storage.isAlive + + /** The sidecar address of element 0. Throws when the storage is closed. */ + public fun pointer(): Int = storage.pointer() + + /** `true` once a heap mirror has been downloaded (diagnostics and tests). */ + public val isMirrored: Boolean get() = mirror != null + + override val buffer: FloatArray + get() { + storage.checkAlive() + mirror?.let { return it } + val n = shape.volume + val arr = FloatArray(n) + if (n > 0) WasmKernels.readFloats(storage.pointer(), arr, 0, n) + mirror = arr + return arr + } + + private fun flatIndex(indices: IntArray): Int { + require(indices.size == shape.dimensions.size) { + "Number of indices (${indices.size}) must match tensor dimensions (${shape.dimensions.size})" + } + var flat = 0 + for (i in indices.indices) { + val idx = indices[i] + require(idx >= 0 && idx < shape.dimensions[i]) { "Index $idx out of bounds for dimension $i with size ${shape.dimensions[i]}" } + flat += idx * strides[i] + } + return flat + } + + override fun get(vararg indices: Int): Float = buffer[flatIndex(indices)] + + override fun set(vararg indices: Int, value: Float) { + storage.checkAlive() + require(storage.isMutable) { "storage ${storage.id} is not mutable" } + val flat = flatIndex(indices) + mirror?.let { it[flat] = value } + WasmKernels.writeFloats(storage.pointer() + flat * 4, floatArrayOf(value), 0, 1) + } + + override fun copyToFloatArray(): FloatArray = buffer.copyOf() +} diff --git a/skainet-backends/skainet-backend-wasm-simd/src/wasmJsMain/kotlin/sk/ainet/exec/wasm/WasmSimd.kt b/skainet-backends/skainet-backend-wasm-simd/src/wasmJsMain/kotlin/sk/ainet/exec/wasm/WasmSimd.kt index 6038f5afd..01f7cdee4 100644 --- a/skainet-backends/skainet-backend-wasm-simd/src/wasmJsMain/kotlin/sk/ainet/exec/wasm/WasmSimd.kt +++ b/skainet-backends/skainet-backend-wasm-simd/src/wasmJsMain/kotlin/sk/ainet/exec/wasm/WasmSimd.kt @@ -3,6 +3,7 @@ package sk.ainet.exec.wasm import sk.ainet.backend.api.kernel.KernelDispatch import sk.ainet.backend.api.kernel.KernelRegistry import sk.ainet.exec.tensor.ops.WasmCpuOpsOverride +import sk.ainet.lang.memory.PlatformStorage /** * Entry point of the browser kernel pack (SKEEP-007 §T1). @@ -13,13 +14,24 @@ import sk.ainet.exec.tensor.ops.WasmCpuOpsOverride * ``` * * [install] loads the SIMD128 sidecar (pthreads build when the page is cross-origin isolated), - * registers [WasmKernelProvider] and [WasmSimdKernelPack], and makes every context built afterwards - * use [WasmCpuOps]. Without it — or before it completes — the engine runs the common scalar lane, - * so an application may start rendering before the sidecar is up and simply get faster later. + * registers [WasmKernelProvider], [WasmSimdKernelPack] and [WasmTernaryKernelPack], binds + * `MemoryDomain.HOST_OFFHEAP` to the sidecar's linear memory ([WasmLinearStorage], so activations stay + * resident between kernels) and makes every context built afterwards use [WasmCpuOps]. Without it — + * or before it completes — the engine runs the common scalar lane, so an application may start + * rendering before the sidecar is up and simply get faster later. */ public object WasmSimd { public val isInstalled: Boolean get() = WasmKernels.isReady && WasmCpuOpsOverride.factory != null + /** + * How many sidecar bytes resident activations may hold before [WasmCpuOps] returns heap outputs + * instead (the kernels still run in the sidecar; only the result is copied back). Ambient + * activations are freed when their tensors are collected, which is not prompt, so this is the + * bound on what a long notebook-style session can pin in linear memory. 768 MB by default; the + * wasm32 sidecar has 4 GB in total, shared with the resident weights. + */ + public var residentBudgetBytes: Long = 768L * 1024 * 1024 + public suspend fun install(url: String? = null, threads: Boolean? = null) { WasmKernels.initialize(url, threads) KernelRegistry.register(WasmKernelProvider) @@ -27,6 +39,8 @@ public object WasmSimd { // the auto-install on the next dispatch picks the sidecar provider and this pack's kernels. KernelDispatch.clearForTesting() WasmSimdKernelPack.install() + WasmTernaryKernelPack.install() + PlatformStorage.offHeapAllocator = WasmLinearStorage.Allocator WasmCpuOpsOverride.factory = { factory, schedule -> WasmCpuOps(factory, schedule) } // The same one-line banner the JVM factory prints, so a console shows which tier runs. println("[SKaiNET] Using SIMD128 sidecar kernels (${WasmKernels.features()})") @@ -38,6 +52,7 @@ public object WasmSimd { /** Back to the scalar lane; frees every resident weight. Tests and model switching. */ public fun uninstall() { WasmCpuOpsOverride.factory = null + PlatformStorage.offHeapAllocator = null KernelDispatch.clearForTesting() WasmKernels.clearForTesting() } diff --git a/skainet-backends/skainet-backend-wasm-simd/src/wasmJsMain/kotlin/sk/ainet/exec/wasm/WasmTernaryKernelPack.kt b/skainet-backends/skainet-backend-wasm-simd/src/wasmJsMain/kotlin/sk/ainet/exec/wasm/WasmTernaryKernelPack.kt new file mode 100644 index 000000000..2e348c795 --- /dev/null +++ b/skainet-backends/skainet-backend-wasm-simd/src/wasmJsMain/kotlin/sk/ainet/exec/wasm/WasmTernaryKernelPack.kt @@ -0,0 +1,124 @@ +package sk.ainet.exec.wasm + +import sk.ainet.backend.api.kernel.BitNetGemvNative +import sk.ainet.backend.api.kernel.TernaryF32GemvNative +import sk.ainet.backend.api.kernel.TernaryF32KernelPack +import sk.ainet.backend.api.kernel.TernaryKernelPacks +import sk.ainet.backend.api.kernel.TernaryLmheadNative +import sk.ainet.backend.api.kernel.TernaryPlanesKernelPack +import sk.ainet.backend.api.kernel.ViewKernelPack + +/** + * The ternary kernels of the browser pack — the wasm sibling of the JVM `FfmTernaryKernelPackFactory` + * and the Android JNI ternary pack. Three native faces over the sidecar's exports: + * + * - [WasmTernaryF32Gemv] — `skainet_ternary_f32_gemv`, exact FP32 activations × `BITNET_B1_58`; + * - [WasmTernaryLmhead] — `skainet_ternary_lmhead_stage1`, the fused four-plane `BITNET_PLANES` lm_head; + * - [WasmBitNetGemv] — `skainet_bitnet_gemv_tq2_0`, int8 activations × `TQ2_0`. + * + * Each hands its weight to the sidecar **once** ([WasmKernels.weightHandle]) and stages the + * activation and output per call, like every other kernel in this module. Without this pack a + * BitNet weight in a browser went to the decoding reference kernel — correct, and hours per + * projection at model size. Installed by [WasmSimd.install] after the sidecar is up; there is no + * `ServiceLoader` on wasm. + */ +public object WasmTernaryKernelPack : ViewKernelPack { + override val name: String = "wasm-simd128-ternary" + + override fun install() { + if (!WasmKernels.isReady) return + TernaryF32KernelPack.install(WasmTernaryF32Gemv) + TernaryPlanesKernelPack.install(WasmTernaryLmhead) + TernaryKernelPacks.install(WasmBitNetGemv) + } +} + +/** `skainet_ternary_f32_gemv` in the sidecar: the sequential b1.58 payload, scale applied by the pack. */ +public object WasmTernaryF32Gemv : TernaryF32GemvNative { + override val name: String get() = "wasm-simd128" + + override fun gemvPacked( + activation: FloatArray, activationOffset: Int, + weight: ByteArray, weightByteOffset: Int, + inputDim: Int, outputDim: Int, + out: FloatArray, outOffset: Int, + ) { + require(inputDim % 4 == 0) { "WasmTernaryF32Gemv: inputDim must be a multiple of 4; got $inputDim" } + if (outputDim == 0) return + if (inputDim == 0) { + out.fill(0f, outOffset, outOffset + outputDim) + return + } + val weightPtr = WasmKernels.weightHandle(weight, 0, weight.size) + weightByteOffset + val inPtr = WasmKernels.alloc(inputDim * 4) + val outPtr = WasmKernels.alloc(outputDim * 4) + try { + WasmKernels.writeFloats(inPtr, activation, activationOffset, inputDim) + WasmKernels.ternaryF32Gemv(inPtr, weightPtr, inputDim, outputDim, outPtr) + WasmKernels.readFloats(outPtr, out, outOffset, outputDim) + } finally { + WasmKernels.free(outPtr) + WasmKernels.free(inPtr) + } + } +} + +/** `skainet_ternary_lmhead_stage1` in the sidecar: planes and row scales read from the resident weight. */ +public object WasmTernaryLmhead : TernaryLmheadNative { + override val name: String get() = "wasm-simd128" + + override fun lmheadStage1( + activation: FloatArray, activationOffset: Int, + weight: ByteArray, planesByteOffset: Int, + planeStrideBytes: Int, rowScaleByteOffset: Int, + inputDim: Int, outputDim: Int, + out: FloatArray, outOffset: Int, + ) { + require(inputDim % 4 == 0) { "WasmTernaryLmhead: inputDim must be a multiple of 4; got $inputDim" } + require(rowScaleByteOffset % 2 == 0) { "WasmTernaryLmhead: rowScaleByteOffset must be 2-byte aligned; got $rowScaleByteOffset" } + if (outputDim == 0) return + val weightPtr = WasmKernels.weightHandle(weight, 0, weight.size) + val inPtr = WasmKernels.alloc(maxOf(inputDim, 1) * 4) + val outPtr = WasmKernels.alloc(outputDim * 4) + try { + if (inputDim > 0) WasmKernels.writeFloats(inPtr, activation, activationOffset, inputDim) + WasmKernels.ternaryLmheadStage1( + inPtr, weightPtr + planesByteOffset, planeStrideBytes, weightPtr + rowScaleByteOffset, + inputDim, outputDim, outPtr, + ) + WasmKernels.readFloats(outPtr, out, outOffset, outputDim) + } finally { + WasmKernels.free(outPtr) + WasmKernels.free(inPtr) + } + } +} + +/** `skainet_bitnet_gemv_tq2_0` in the sidecar: one token's int8 codes against resident TQ2_0 blocks. */ +public object WasmBitNetGemv : BitNetGemvNative { + override val name: String get() = "wasm-simd128" + + override fun gemvTq2_0( + activation: ByteArray, activationOffset: Int, activationScale: Float, + weight: ByteArray, weightByteOffset: Int, + inputDim: Int, outputDim: Int, + out: FloatArray, outOffset: Int, + ) { + if (outputDim == 0) return + if (inputDim == 0) { + out.fill(0f, outOffset, outOffset + outputDim) + return + } + val weightPtr = WasmKernels.weightHandle(weight, 0, weight.size) + weightByteOffset + val actPtr = WasmKernels.alloc(inputDim) + val outPtr = WasmKernels.alloc(outputDim * 4) + try { + WasmKernels.writeBytes(actPtr, activation, activationOffset, inputDim) + WasmKernels.bitnetGemvTq2_0(actPtr, activationScale, weightPtr, inputDim, outputDim, outPtr) + WasmKernels.readFloats(outPtr, out, outOffset, outputDim) + } finally { + WasmKernels.free(outPtr) + WasmKernels.free(actPtr) + } + } +} diff --git a/skainet-backends/skainet-backend-wasm-simd/src/wasmJsTest/kotlin/sk/ainet/exec/wasm/WasmSimdParityTest.kt b/skainet-backends/skainet-backend-wasm-simd/src/wasmJsTest/kotlin/sk/ainet/exec/wasm/WasmSimdParityTest.kt index 31c96f6eb..db3a78e2a 100644 --- a/skainet-backends/skainet-backend-wasm-simd/src/wasmJsTest/kotlin/sk/ainet/exec/wasm/WasmSimdParityTest.kt +++ b/skainet-backends/skainet-backend-wasm-simd/src/wasmJsTest/kotlin/sk/ainet/exec/wasm/WasmSimdParityTest.kt @@ -1,11 +1,32 @@ package sk.ainet.exec.wasm import kotlinx.coroutines.test.runTest +import sk.ainet.backend.api.kernel.BitNetGemvKernel import sk.ainet.backend.api.kernel.KernelDispatch import sk.ainet.backend.api.kernel.KernelRegistry import sk.ainet.backend.api.kernel.KernelStrictness import sk.ainet.backend.api.kernel.NoSuchKernelException +import sk.ainet.backend.api.kernel.TernaryF32GemvKernel +import sk.ainet.backend.api.kernel.TernaryPlanesMatmulKernel import sk.ainet.context.DirectCpuExecutionContext +import sk.ainet.context.ScopedExecutionContext +import sk.ainet.lang.memory.ForwardScope +import sk.ainet.lang.memory.PlatformStorage +import sk.ainet.lang.memory.StorageClosedException +import sk.ainet.lang.memory.plan.StorageCapabilities +import sk.ainet.lang.tensor.data.DenseFloatArrayTensorData +import sk.ainet.lang.tensor.storage.MemoryDomain +import sk.ainet.lang.memory.I8Absmax +import sk.ainet.lang.memory.Scope +import sk.ainet.lang.memory.Storage +import sk.ainet.lang.memory.TensorView +import sk.ainet.lang.memory.TernaryBlockDecoder +import sk.ainet.lang.memory.TernaryCodec +import sk.ainet.lang.memory.trace.RecordingTraceSink +import sk.ainet.lang.memory.trace.TraceEvent +import sk.ainet.lang.tensor.data.BitNetB158TensorData +import sk.ainet.lang.tensor.data.BitNetPlanesTensorData +import sk.ainet.lang.tensor.storage.TensorEncoding import sk.ainet.exec.kernel.ScalarBf16MatmulKernel import sk.ainet.exec.kernel.ScalarFp16MatmulKernel import sk.ainet.exec.kernel.ScalarQ4_KMatmulKernel @@ -23,7 +44,12 @@ import sk.ainet.lang.tensor.data.Q8_0BlockTensorData import sk.ainet.lang.tensor.data.Q8_0Quantizer import sk.ainet.lang.tensor.data.TensorData import sk.ainet.lang.tensor.matmulWeightTransposed +import sk.ainet.lang.types.DType import sk.ainet.lang.types.FP32 +import sk.ainet.lang.types.Int32 +import sk.ainet.lang.nn.normalization.LayerNormalization +import sk.ainet.lang.tensor.ops.AttentionMask +import sk.ainet.lang.tensor.ops.FusedNormOps import kotlin.math.abs import kotlin.math.sqrt import kotlin.random.Random @@ -36,8 +62,8 @@ import kotlin.test.assertTrue /** * SKEEP-007 §T1: the SIMD128 sidecar agrees with the common scalar lane on every path it takes - * over — row-major packed matmuls (Q8_0, Q4_0, Q4_K via the pack), dense `x · Wᵀ`, and the dense - * GEMM — and the wiring makes a freshly built context use [WasmCpuOps]. + * over — row-major packed matmuls (Q8_0, Q4_0, Q4_K via the pack), dense `x · Wᵀ`, the dense GEMM + * and the three ternary kernels — and the wiring makes a freshly built context use [WasmCpuOps]. * * Runs on Node (`wasmJsNodeTest`): the sidecar is loaded from the module's own resources. Lane * sums differ from the sequential scalar accumulation in the last bits, so comparisons are @@ -70,6 +96,8 @@ class WasmSimdParityTest { WasmSimd.uninstall() } + private fun heapFloats(view: TensorView): FloatArray = checkNotNull((view.storage as Storage.Heap).floats) + @Suppress("UNCHECKED_CAST") private fun q8(ctx: DirectCpuExecutionContext, n: Int, k: Int, seed: Int): Tensor = ctx.fromData(Q8_0Quantizer.quantize(pseudo(n * k, seed, 2f), Shape(n, k)) as TensorData, FP32::class) @@ -282,5 +310,388 @@ class WasmSimdParityTest { val direct = FloatArray(m * n) WasmFp32MatmulKernel.matmul(x, 0, k, b, 0, n, direct, 0, n, m, n, k) assertClose(refMm, direct, "provider gemm") + // The direct `x · Wᵀ` provider kernel: whole-array weight (resident) and a windowed one (staged). + val provider = checkNotNull(KernelRegistry.bestAvailable()) + val wt = checkNotNull(provider.matmulFp32Wt()) + assertTrue(provider.supports("matmul_wt", listOf("Float32", "Float32"))) + val directWt = FloatArray(m * n) + wt.matmulWt(x, 0, w, 0, directWt, 0, m, n, k) + assertClose(refWt, directWt, "provider x·Wᵀ") + val window = FloatArray(1 + n * k + 3).also { w.copyInto(it, 1) } + val windowed = FloatArray(2 + m * n) + wt.matmulWt(x, 0, window, 1, windowed, 2, m, n, k) + assertClose(refWt, windowed.copyOfRange(2, 2 + m * n), "windowed x·Wᵀ") + } + + /** + * The ternary kernels reach the sidecar through the same `KernelDispatch` keys the JVM FFM and + * Android JNI packs serve, and agree with the Kotlin reference kernels. Row counts above the row + * pool's threshold (512) so the mt sidecar splits the rows over its workers. + */ + @Test + fun `ternary weights run on the sidecar and match the reference kernels`() = runTest { + WasmSimd.install() + val rng = Random(31) + + // BITNET_B1_58, exact FP32 activations, two rows so the pack's row loop runs. + run { + val k = 256; val n = 600; val rows = 2 + val values = FloatArray(n * k) { (rng.nextInt(3) - 1) * 0.5f } + val w = BitNetB158TensorData.fromFloats(Shape(n, k), values).packedView + val a = TensorView.dense(Storage.Heap.wrap(FloatArray(rows * k) { rng.nextFloat() - 0.5f }), Shape(rows, k), FP32) + val out = TensorView.dense(Storage.Heap.floats(rows * n), Shape(rows, n), FP32) + val sink = RecordingTraceSink() + KernelDispatch.matmul(a, w, out, Scope.Ambient, sink) + assertEquals("ternary_f32_gemv/wasm-simd128", sink.eventsOf().single().kernel) + val ref = TensorView.dense(Storage.Heap.floats(rows * n), Shape(rows, n), FP32) + TernaryF32GemvKernel(TernaryF32GemvKernel.keyFor()).run(listOf(a, w), ref) + assertClose(heapFloats(ref), heapFloats(out), "ternary f32 gemv", rel = 1e-4f) + } + + // BITNET_PLANES: the fused four-plane lm_head, two calls per row, against the 8-plane reference. + run { + val k = 64; val n = 520 + val values = FloatArray(n * k) { (rng.nextFloat() - 0.5f) * 2f } + val w = BitNetPlanesTensorData.fromFloats(Shape(n, k), values).packedView + val a = TensorView.dense(Storage.Heap.wrap(FloatArray(k) { rng.nextFloat() - 0.5f }), Shape(1, k), FP32) + val out = TensorView.dense(Storage.Heap.floats(n), Shape(1, n), FP32) + val sink = RecordingTraceSink() + KernelDispatch.matmul(a, w, out, Scope.Ambient, sink) + assertEquals("ternary_planes_matmul/wasm-simd128", sink.eventsOf().single().kernel) + val ref = TensorView.dense(Storage.Heap.floats(n), Shape(1, n), FP32) + TernaryPlanesMatmulKernel(TernaryPlanesMatmulKernel.keyFor()).run(listOf(a, w), ref) + assertClose(heapFloats(ref), heapFloats(out), "ternary planes lm_head", rel = 1e-4f) + } + + // TQ2_0 with int8 activations: integer block sums, so the match is exact up to the scale products. + run { + val k = 768; val n = 600 + val values = FloatArray(n * k) { (rng.nextInt(3) - 1) * 0.5f } + val w = TensorView.packed( + Storage.Heap.wrap(TernaryCodec.encode(TensorEncoding.TQ2_0, values)), Shape(n, k), + TensorEncoding.TQ2_0, TernaryBlockDecoder(TensorEncoding.TQ2_0), + ) + val a = I8Absmax.requantize( + TensorView.dense(Storage.Heap.wrap(FloatArray(k) { rng.nextFloat() - 0.5f }), Shape(1, k), FP32), Scope.Ambient, + ) + val out = TensorView.dense(Storage.Heap.floats(n), Shape(1, n), FP32) + val sink = RecordingTraceSink() + KernelDispatch.matmul(a, w, out, Scope.Ambient, sink) + assertEquals("bitnet_gemv/wasm-simd128", sink.eventsOf().single().kernel) + val ref = TensorView.dense(Storage.Heap.floats(n), Shape(1, n), FP32) + BitNetGemvKernel(BitNetGemvKernel.keyFor(w.format)).run(listOf(a, w), ref) + assertClose(heapFloats(ref), heapFloats(out), "bitnet gemv tq2_0") + } + assertTrue(WasmKernels.residentWeightBytes() > 0, "the ternary weights stay resident") + } + + /** + * Resident activations: a matmul's output stays in the sidecar, the elementwise ops, activations + * and reductions that follow read it in place and leave their result there too, and every value + * agrees with the common scalar lane. Only the final read materialises a heap mirror. + */ + @Test + fun `an activation chain stays resident in the sidecar and matches the scalar lane`() = runTest { + val scalar = DirectCpuExecutionContext() + val m = 7; val n = 40; val k = 24 + val x = pseudo(m * k, 41) + val w = pseudo(n * k, 43) + val bias = pseudo(n, 47) + val xs = scalar.fromFloatArray(Shape(m, k), FP32::class, x) + val ws = scalar.fromFloatArray(Shape(n, k), FP32::class, w) + val bs = scalar.fromFloatArray(Shape(n), FP32::class, bias) + val two = scalar.fromFloatArray(Shape(1), FP32::class, floatArrayOf(2f)) + fun chain(ctx: DirectCpuExecutionContext, x: Tensor, w: Tensor, b: Tensor, c: Tensor, probe: (String, Tensor) -> Unit): Map { + val ops = ctx.ops + val out = LinkedHashMap() + fun keep(name: String, t: Tensor) { probe(name, t); out[name] = t.data.copyToFloatArray() } + val h = ops.matmulWeightTransposed(x, w); probe("h", h) + val g = ops.gelu(h); probe("gelu", g) + val ge = ops.gelu(h, approximate = false); keep("geluExact", ge) + val s = ops.multiply(g, h); probe("mul", s) + val o = ops.add(s, b); probe("addBias", o) + keep("biasAdd", ops.add(b, s)) + keep("subScalar", ops.subtract(o, c)) + keep("scalarSub", ops.subtract(c, o)) + keep("div", ops.divide(o, ops.add(ops.relu(o), c))) + keep("relu", ops.relu(o)) + keep("sigmoid", ops.sigmoid(o)) + keep("silu", ops.silu(o)) + keep("tanh", ops.tanh(o)) + keep("erf", ops.erf(o)) + keep("sumAll", ops.sum(o)) + keep("meanAll", ops.mean(o)) + keep("sumLast", ops.sum(o, -1)) + keep("meanFirst", ops.mean(o, 0)) + keep("out", o) + return out + } + val ref = chain(scalar, xs, ws, bs, two) { _, _ -> } + + WasmSimd.install() + val fast = DirectCpuExecutionContext() + val xf = fast.fromFloatArray(Shape(m, k), FP32::class, x) + val wf = fast.fromFloatArray(Shape(n, k), FP32::class, w) + val bf = fast.fromFloatArray(Shape(n), FP32::class, bias) + val cf = fast.fromFloatArray(Shape(1), FP32::class, floatArrayOf(2f)) + val before = WasmKernels.residentActivationBytes() + val got = chain(fast, xf, wf, bf, cf) { name, t -> + val d = t.data + assertIs>(d, "$name should be resident") + assertTrue(d.isResident, "$name alive") + if (name == "h" || name == "gelu" || name == "mul" || name == "addBias") assertTrue(!d.isMirrored, "$name was not downloaded") + } + assertTrue(WasmKernels.residentActivationBytes() > before, "resident bytes grew") + for ((name, values) in ref) { + assertClose(values, checkNotNull(got[name]) { name }, name) + } + // Shapes of the reductions follow the common lane. + assertEquals(m, got.getValue("sumLast").size) + assertEquals(n, got.getValue("meanFirst").size) + assertEquals(1, got.getValue("sumAll").size) + } + + @Suppress("UNCHECKED_CAST") + private fun lengthsTensor(ctx: DirectCpuExecutionContext, lengths: IntArray): Tensor = + ctx.fromIntArray(Shape(lengths.size), Int32::class, lengths) as Tensor + + @Test + fun `softmax, the normalisations and attention run fused in the sidecar and match the scalar lane`() = runTest { + val scalar = DirectCpuExecutionContext() + val rows = 16; val cols = 96 + val xArr = pseudo(rows * cols, 51, 3f) + val gArr = FloatArray(cols) { 0.5f + (it % 7) * 0.1f } + val bArr = FloatArray(cols) { ((it % 5) - 2) * 0.05f } + val batch = 2; val heads = 4; val kvHeads = 2; val seqQ = 5; val seqKV = 9; val headDim = 16 + val qArr = pseudo(batch * heads * seqQ * headDim, 53) + val kArr = pseudo(batch * kvHeads * seqKV * headDim, 55) + val vArr = pseudo(batch * kvHeads * seqKV * headDim, 57) + val lengths = intArrayOf(seqKV, 4) + val maskArr = pseudo(seqQ * seqKV, 59) + val wqArr = pseudo(cols * cols, 61) + + fun run(ctx: DirectCpuExecutionContext, probe: (String, Tensor) -> Unit): Map { + val ops = ctx.ops + val out = LinkedHashMap() + fun keep(name: String, t: Tensor) { probe(name, t); out[name] = t.data.copyToFloatArray() } + val x = ctx.fromFloatArray(Shape(rows, cols), FP32::class, xArr) + keep("softmax", ops.softmax(x, -1)) + keep("logSoftmax", ops.logSoftmax(x, -1)) + keep("softmaxDim0", ops.softmax(x, 0)) + val gamma = ctx.fromFloatArray(Shape(cols), FP32::class, gArr) + val beta = ctx.fromFloatArray(Shape(cols), FP32::class, bArr) + val ln = LayerNormalization(intArrayOf(cols), eps = 1e-5, elementwiseAffine = true, name = "ln", initGamma = gamma, initBeta = beta) + keep("layerNorm", ln.forward(x, ctx)) + val lnPlain = LayerNormalization(intArrayOf(cols), eps = 1e-5, elementwiseAffine = false, name = "lnPlain") + keep("layerNormPlain", lnPlain.forward(x, ctx)) + (ops as? FusedNormOps)?.rmsNorm(x, gamma, 1e-6f, 1)?.let { keep("rmsNorm", it) } + // On the sidecar the operands are made resident first (as a projection's output is), so + // attention reads them in place: the probe checks they were never downloaded. + val wasm = ops as? WasmCpuOps + fun resident(t: Tensor) = wasm?.toResident(t) ?: t + val q = resident(ctx.fromFloatArray(Shape(batch, heads, seqQ, headDim), FP32::class, qArr)) + val k = resident(ctx.fromFloatArray(Shape(batch, kvHeads, seqKV, headDim), FP32::class, kArr)) + val v = resident(ctx.fromFloatArray(Shape(batch, kvHeads, seqKV, headDim), FP32::class, vArr)) + keep("sdpa", ops.scaledDotProductAttention(q, k, v)) + probe("q", q); probe("k", k); probe("v", v) + keep("sdpaCausal", ops.scaledDotProductAttention(q, k, v, causal = true)) + keep("sdpaLengths", ops.scaledDotProductAttention(q, k, v, causal = true, keyLengths = lengthsTensor(ctx, lengths))) + keep("sdpaWindow", ops.scaledDotProductAttention(q, k, v, keyLengths = lengthsTensor(ctx, lengths), window = 3, rightContext = 1)) + val mask = ctx.fromFloatArray(Shape(seqQ, seqKV), FP32::class, maskArr) + keep("sdpaMask", ops.scaledDotProductAttention(q, k, v, mask = mask, scale = 0.3f)) + val dense = ctx.fromFloatArray(Shape(batch, 1, seqQ, seqKV), FP32::class, AttentionMask.additive(batch, seqQ, seqKV, true, 3, 1, lengths)) + keep("sdpaDense", ops.scaledDotProductAttention(q, k, v, mask = dense)) + keep("sdpaStructured", ops.scaledDotProductAttention(q, k, v, causal = true, keyLengths = lengthsTensor(ctx, lengths), window = 3, rightContext = 1)) + keep("sdpaPadded", ops.scaledDotProductAttention(q, k, v, keyLengths = lengthsTensor(ctx, intArrayOf(0, seqKV)))) + // A projection's output goes straight into the normalisation without a download. + val wq = ctx.fromFloatArray(Shape(cols, cols), FP32::class, wqArr) + val proj = ops.matmulWeightTransposed(x, wq) + keep("projNorm", lnPlain.forward(proj, ctx)) + probe("proj", proj) + return out + } + val ref = run(scalar) { _, _ -> } + assertTrue(ref.containsKey("rmsNorm").not() || scalar.ops !is FusedNormOps, "the scalar lane has no fused-norm hook") + + WasmSimd.install() + val fast = DirectCpuExecutionContext() + assertIs(fast.ops) + val got = run(fast) { name, t -> + if (name == "softmaxDim0") return@run + val d = t.data + assertIs>(d, "$name should be resident") + if (name == "proj" || name == "q" || name == "k" || name == "v") assertTrue(!d.isMirrored, "$name was not downloaded") + } + for ((name, values) in ref) { + val (rel, abs) = if (name.startsWith("sdpa") || name.contains("Norm")) 1e-4f to 1e-5f else 1e-5f to 1e-6f + val actual = checkNotNull(got[name]) { name } + assertEquals(values.size, actual.size, "$name size") + var maxAbs = 0f + for (v in values) maxAbs = maxOf(maxAbs, kotlin.math.abs(v)) + val tol = maxAbs * rel + abs + for (i in values.indices) assertTrue(kotlin.math.abs(values[i] - actual[i]) <= tol, "$name[$i]: ${actual[i]} vs ${values[i]} (tol $tol)") + } + assertTrue(got.containsKey("rmsNorm"), "rmsNorm served by the sidecar") + // The structured rules equal the dense -1e30 mask on the sidecar kernel. + val a = got.getValue("sdpaStructured"); val b = got.getValue("sdpaDense") + for (i in a.indices) assertTrue(abs(a[i] - b[i]) <= 1e-6f, "structured vs dense [$i]") + assertTrue(got.getValue("sdpaPadded").all { it.isFinite() }, "fully padded row is finite") + } + + @Test + fun `convolutions run as im2col and GEMM in the sidecar and match the scalar lane`() = runTest { + val scalar = DirectCpuExecutionContext() + // conv2d over a [2, 4, 9, 7] input: 3×3 same padding with and without bias, a 2×3 kernel with + // stride, dilation and asymmetric padding, depthwise, two groups with stride, 1×1, the 4-D bias + // form; conv1d over [2, 3, 17] with stride, dilation and groups; conv3d over [1, 2, 5, 6, 7]. + val x2Arr = pseudo(2 * 4 * 9 * 7, 71); val w2Arr = pseudo(6 * 4 * 3 * 3, 73); val b2Arr = pseudo(6, 75) + val w2sArr = pseudo(5 * 4 * 2 * 3, 77) + val wDwArr = pseudo(4 * 1 * 3 * 3, 79) + val wGArr = pseudo(6 * 2 * 3 * 3, 81) + val w11Arr = pseudo(8 * 4, 83) + val x1Arr = pseudo(2 * 3 * 17, 85); val w1Arr = pseudo(4 * 3 * 5, 87); val b1Arr = pseudo(4, 89) + val w1gArr = pseudo(6 * 1 * 3, 91) + val x3Arr = pseudo(1 * 2 * 5 * 6 * 7, 93); val w3Arr = pseudo(3 * 2 * 2 * 3 * 3, 95); val b3Arr = pseudo(3, 97) + + fun run(ctx: DirectCpuExecutionContext, probe: (String, Tensor) -> Unit): Map { + val ops = ctx.ops + val out = LinkedHashMap() + fun keep(name: String, t: Tensor) { probe(name, t); out[name] = t.data.copyToFloatArray() } + val x2 = ctx.fromFloatArray(Shape(2, 4, 9, 7), FP32::class, x2Arr) + val w2 = ctx.fromFloatArray(Shape(6, 4, 3, 3), FP32::class, w2Arr) + val b2 = ctx.fromFloatArray(Shape(6), FP32::class, b2Arr) + keep("conv2d", ops.conv2d(x2, w2, b2, 1 to 1, 1 to 1, 1 to 1, 1)) + keep("conv2dNoBias", ops.conv2d(x2, w2, null, 1 to 1, 1 to 1, 1 to 1, 1)) + keep("conv2dBias4d", ops.conv2d(x2, w2, ctx.fromFloatArray(Shape(1, 6, 1, 1), FP32::class, b2Arr), 1 to 1, 1 to 1, 1 to 1, 1)) + val w2s = ctx.fromFloatArray(Shape(5, 4, 2, 3), FP32::class, w2sArr) + keep("conv2dStrideDilation", ops.conv2d(x2, w2s, null, 2 to 1, 0 to 2, 2 to 1, 1)) + val wDw = ctx.fromFloatArray(Shape(4, 1, 3, 3), FP32::class, wDwArr) + keep("conv2dDepthwise", ops.conv2d(x2, wDw, null, 1 to 1, 1 to 1, 1 to 1, 4)) + val wG = ctx.fromFloatArray(Shape(6, 2, 3, 3), FP32::class, wGArr) + keep("conv2dGrouped", ops.conv2d(x2, wG, b2, 1 to 2, 1 to 1, 1 to 1, 2)) + val w11 = ctx.fromFloatArray(Shape(8, 4, 1, 1), FP32::class, w11Arr) + keep("conv2d1x1", ops.conv2d(x2, w11, null, 1 to 1, 0 to 0, 1 to 1, 1)) + val x1 = ctx.fromFloatArray(Shape(2, 3, 17), FP32::class, x1Arr) + val w1 = ctx.fromFloatArray(Shape(4, 3, 5), FP32::class, w1Arr) + val b1 = ctx.fromFloatArray(Shape(4), FP32::class, b1Arr) + keep("conv1d", ops.conv1d(x1, w1, b1, 2, 2, 1, 1)) + keep("conv1dDilation", ops.conv1d(x1, w1, null, 1, 4, 2, 1)) + val w1g = ctx.fromFloatArray(Shape(6, 1, 3), FP32::class, w1gArr) + keep("conv1dGrouped", ops.conv1d(x1, w1g, null, 1, 1, 1, 3)) + val x3 = ctx.fromFloatArray(Shape(1, 2, 5, 6, 7), FP32::class, x3Arr) + val w3 = ctx.fromFloatArray(Shape(3, 2, 2, 3, 3), FP32::class, w3Arr) + val b3 = ctx.fromFloatArray(Shape(3), FP32::class, b3Arr) + keep("conv3d", ops.conv3d(x3, w3, b3, Triple(1, 2, 1), Triple(1, 1, 0), Triple(1, 1, 1), 1)) + // A layer's chain: the convolution's output feeds the activation without a download, and + // a resident input (the previous layer's output) is read in place. + keep("convRelu", ops.relu(ops.conv2d(x2, w2, b2, 1 to 1, 1 to 1, 1 to 1, 1))) + val wasm = ops as? WasmCpuOps + val xResident = wasm?.toResident(x2) ?: x2 + keep("conv2dResidentInput", ops.conv2d(xResident, w2, b2, 1 to 1, 1 to 1, 1 to 1, 1)) + probe("xResident", xResident) + return out + } + val ref = run(scalar) { _, _ -> } + + WasmSimd.install() + val fast = DirectCpuExecutionContext() + assertIs(fast.ops) + val got = run(fast) { name, t -> + val d = t.data + assertIs>(d, "$name should be resident") + if (name == "xResident") assertTrue(!d.isMirrored, "the resident input was not downloaded") + } + for ((name, values) in ref) assertClose(values, checkNotNull(got[name]) { name }, name) + + // A weight is a parameter: resident after its first call, not re-uploaded by the next one. + val x2 = fast.fromFloatArray(Shape(2, 4, 9, 7), FP32::class, x2Arr) + val w2 = fast.fromFloatArray(Shape(6, 4, 3, 3), FP32::class, w2Arr) + val before = WasmKernels.residentWeightBytes() + fast.ops.conv2d(x2, w2, null, 1 to 1, 1 to 1, 1 to 1, 1) + val held = WasmKernels.residentWeightBytes() + assertEquals(before + w2Arr.size * 4L, held, "the weight went resident on its first call") + fast.ops.conv2d(x2, w2, null, 1 to 1, 1 to 1, 1 to 1, 1) + assertEquals(held, WasmKernels.residentWeightBytes(), "a second call with the same weight uploads nothing") + } + + @Test + fun `heap operands keep the common path while transcendental activations and matmul outputs go resident`() = runTest { + WasmSimd.install() + val ctx = DirectCpuExecutionContext() + assertIs(ctx.ops) + val a = ctx.fromFloatArray(Shape(3, 5), FP32::class, pseudo(15, 1)) + val b = ctx.fromFloatArray(Shape(3, 5), FP32::class, pseudo(15, 2)) + // Two heap operands: no upload, the common scalar lane answers on the heap. + assertIs>(ctx.ops.add(a, b).data) + assertIs>(ctx.ops.relu(a).data) + assertIs>(ctx.ops.sum(a).data) + // A transcendental on a heap operand is worth the upload; its output is resident. + val g = ctx.ops.gelu(a) + assertIs>(g.data) + // ... and from there the chain is resident, including a heap partner. + assertIs>(ctx.ops.add(g, b).data) + assertIs>(ctx.ops.relu(g).data) + assertIs>(ctx.ops.sum(g).data) + // Explicit upload once. + val r = (ctx.ops as WasmCpuOps).toResident(a) + assertIs>(r.data) + assertClose(a.data.copyToFloatArray(), r.data.copyToFloatArray(), "toResident") + assertIs>(ctx.ops.add(r, b).data) + // A resident tensor reads like a heap one: elements, the view, and a common op that has no override. + val gd = g.data as WasmLinearTensorData<*> + val expected = DirectCpuExecutionContext().let { s -> s.ops.gelu(s.fromFloatArray(Shape(3, 5), FP32::class, pseudo(15, 1))).data.copyToFloatArray() } + assertClose(expected, g.data.copyToFloatArray(), "gelu via mirror") + assertEquals(expected[7], gd.get(1, 2)) + assertTrue(gd.isMirrored) + assertClose(expected, ctx.ops.transpose(ctx.ops.transpose(g)).data.copyToFloatArray(), "common op on a resident tensor") + } + + @Test + fun `scoped resident outputs are freed at reset and the budget falls back to the heap`() = runTest { + WasmSimd.install() + assertTrue(StorageCapabilities.current().supportsOffHeap, "the hook binds HOST_OFFHEAP while installed") + val scope = ForwardScope(slabFloats = 0) + val ctx = ScopedExecutionContext(DirectCpuExecutionContext(), scope) + val x = ctx.fromFloatArray(Shape(4, 8), FP32::class, pseudo(32, 9)) + val before = WasmKernels.residentActivationBytes() + val y = ctx.ops.gelu(x) + val yd = assertIs>(y.data) + assertEquals(sk.ainet.lang.memory.ScopeKind.FORWARD, yd.storage.scope) + assertEquals(before + 32 * 4, WasmKernels.residentActivationBytes()) + val values = y.data.copyToFloatArray() + scope.reset() + assertEquals(before, WasmKernels.residentActivationBytes(), "reset freed the sidecar bytes") + assertTrue(!yd.isResident) + assertFailsWith { y.data.copyToFloatArray() } + assertEquals(32, values.size) + + // Budget: the kernel still runs in the sidecar, the result comes back to the heap. + val saved = WasmSimd.residentBudgetBytes + try { + WasmSimd.residentBudgetBytes = 0L + val plain = DirectCpuExecutionContext() + val h = plain.fromFloatArray(Shape(4, 8), FP32::class, pseudo(32, 9)) + val z = plain.ops.gelu(h) + assertIs>(z.data) + assertClose(values, z.data.copyToFloatArray(), "budget fallback") + } finally { + WasmSimd.residentBudgetBytes = saved + } + + // The storage itself: zeroed allocation, byte round trip, slice, close. + val s = assertIs(PlatformStorage.allocate(64, MemoryDomain.HOST_OFFHEAP)) + val zeros = ByteArray(64) { 1 } + s.copyInto(zeros) + assertTrue(zeros.all { it == 0.toByte() }, "allocated zeroed") + val bytes = ByteArray(64) { it.toByte() } + s.copyFrom(bytes) + val back = ByteArray(16) + s.slice(8, 16).copyInto(back) + assertTrue(back.contentEquals(bytes.copyOfRange(8, 24)), "slice reads the sub-range") + val held = WasmKernels.residentActivationBytes() + s.close() + assertEquals(held - 64, WasmKernels.residentActivationBytes()) + WasmSimd.uninstall() + assertTrue(!StorageCapabilities.current().supportsOffHeap, "heap only once uninstalled") } } diff --git a/skainet-io/skainet-io-core/src/jsMain/kotlin/sk/ainet/io/JsBlobRandomAccessSource.kt b/skainet-io/skainet-io-core/src/jsMain/kotlin/sk/ainet/io/JsBlobRandomAccessSource.kt index 35523a61c..830bd84d1 100644 --- a/skainet-io/skainet-io-core/src/jsMain/kotlin/sk/ainet/io/JsBlobRandomAccessSource.kt +++ b/skainet-io/skainet-io-core/src/jsMain/kotlin/sk/ainet/io/JsBlobRandomAccessSource.kt @@ -3,7 +3,6 @@ package sk.ainet.io import kotlinx.coroutines.await import org.khronos.webgl.ArrayBuffer import org.khronos.webgl.Int8Array -import org.khronos.webgl.get import org.w3c.files.Blob import kotlin.js.Promise @@ -53,12 +52,11 @@ public class JsBlobRandomAccessSource private constructor( if (length == 0) return ByteArray(0) - val pos = position.toInt() - val requestedEnd = pos + length - - // If within preloaded buffer, return from cache - if (requestedEnd <= preloadedBuffer.size) { - return preloadedBuffer.copyOfRange(pos, requestedEnd) + // Within the preloaded window: served from the heap. Compared as Long — a position past + // 2 GB must not be truncated into the window. + if (position + length <= preloadedBuffer.size) { + val pos = position.toInt() + return preloadedBuffer.copyOfRange(pos, pos + length) } // Beyond preloaded data - throw helpful error @@ -120,16 +118,15 @@ public class JsBlobRandomAccessSource private constructor( if (length == 0) return ByteArray(0) - val pos = position.toInt() - val requestedEnd = pos + length - - // If within preloaded buffer, return from cache (fast path) - if (requestedEnd <= preloadedBuffer.size) { - return preloadedBuffer.copyOfRange(pos, requestedEnd) + // Within the preloaded window: served from the heap (fast path), compared as Long. + if (position + length <= preloadedBuffer.size) { + val pos = position.toInt() + return preloadedBuffer.copyOfRange(pos, pos + length) } - // Read from blob using slice + arrayBuffer - val slice = blob.slice(pos, requestedEnd) + // Beyond it: slice + arrayBuffer. `Blob.slice` takes JavaScript numbers (53 bits), so a Blob + // over 2 GB is addressed as a Double, not through the stdlib's Int overload. + val slice = blob.asDynamic().slice(position.toDouble(), (position + length).toDouble()) as Blob val arrayBuffer = (slice.asDynamic().arrayBuffer() as Promise).await() return arrayBufferToByteArray(arrayBuffer) } @@ -152,19 +149,15 @@ public class JsBlobRandomAccessSource private constructor( * @return JsBlobRandomAccessSource ready for use */ public suspend fun open(blob: Blob, preloadSize: Int = DEFAULT_PRELOAD_SIZE): JsBlobRandomAccessSource { - val actualPreloadSize = minOf(preloadSize, blob.size.toInt()) + val actualPreloadSize = minOf(preloadSize.toLong(), blob.size.toLong()).toInt() val slice = blob.slice(0, actualPreloadSize) val arrayBuffer = (slice.asDynamic().arrayBuffer() as Promise).await() val bytes = arrayBufferToByteArray(arrayBuffer) return JsBlobRandomAccessSource(blob, bytes) } - /** - * Convert JavaScript ArrayBuffer to Kotlin ByteArray. - */ - private fun arrayBufferToByteArray(buffer: ArrayBuffer): ByteArray { - val int8Array = Int8Array(buffer) - return ByteArray(int8Array.length) { int8Array[it] } - } + /** A Kotlin/JS `ByteArray` *is* an `Int8Array`: no copy, no per-element loop. */ + private fun arrayBufferToByteArray(buffer: ArrayBuffer): ByteArray = + Int8Array(buffer).unsafeCast() } } diff --git a/skainet-io/skainet-io-core/src/wasmJsMain/kotlin/sk/ainet/io/JsBlobRandomAccessSource.kt b/skainet-io/skainet-io-core/src/wasmJsMain/kotlin/sk/ainet/io/JsBlobRandomAccessSource.kt index 72b7860b5..3bf376803 100644 --- a/skainet-io/skainet-io-core/src/wasmJsMain/kotlin/sk/ainet/io/JsBlobRandomAccessSource.kt +++ b/skainet-io/skainet-io-core/src/wasmJsMain/kotlin/sk/ainet/io/JsBlobRandomAccessSource.kt @@ -1,164 +1,225 @@ -@file:OptIn(kotlin.js.ExperimentalWasmJsInterop::class) +@file:OptIn(kotlin.js.ExperimentalWasmJsInterop::class, kotlin.wasm.unsafe.UnsafeWasmMemoryApi::class) package sk.ainet.io import kotlinx.coroutines.await import org.khronos.webgl.ArrayBuffer -import org.khronos.webgl.Int8Array -import org.khronos.webgl.get import org.w3c.files.Blob import kotlin.js.Promise +import kotlin.wasm.unsafe.Pointer +import kotlin.wasm.unsafe.wasmMemory +import kotlin.wasm.unsafe.withScopedMemoryAllocator /** - * WASM JS implementation of [RandomAccessSource] using Blob API. + * [RandomAccessSource] over a browser `Blob` — a `File` from an ``, a fetched body, a Cache + * API entry — for Kotlin/Wasm. * - * Uses Blob.slice() for efficient random access to file data without loading - * the entire file into memory. + * Positions and [size] are `Long`: a Blob may be larger than 2 GB (one shard of a sharded checkpoint + * routinely is), and `Blob.slice` takes JavaScript numbers, which carry 53 bits. Only a single read + * is bounded by the 2 GB `ByteArray` it lands in. * - * **Important**: Browser Blob operations are inherently asynchronous. - * This implementation pre-loads metadata section during [open] for synchronous - * access via [readAt], and provides [readAtAsync] for accessing data beyond - * the preloaded buffer. + * Bytes cross into Kotlin through wasm linear memory — one `Uint8Array.set` per 4 MB stage, then a + * wasm loop into the `ByteArray` — never one JavaScript call per byte. (Kotlin/Wasm has no direct + * `Int8Array` → `ByteArray` adapter; the per-byte loop this replaces was fifty million interop calls + * for the default 50 MB preload.) + * + * A Blob is asynchronous on the main thread, so there are two ways to read it: + * + * - [read] (suspending, [SuspendingRandomAccessSource]) works on any thread and for any range. + * - [readAt] (blocking, [RandomAccessSource]) is served from the preloaded window — the first + * [preloadedSize] bytes, enough for a header — and, **inside a Web Worker**, from the Blob itself + * through `FileReaderSync`, the browser's only synchronous file read, which exists on worker + * threads alone. [supportsSyncReads] says which case this source is in; a blocking read beyond the + * window on the main thread throws with that explanation rather than blocking the page. + * + * The blocking readers of the format modules (`StreamingSafeTensorsReader`, `StreamingGGUFReader`) + * therefore load a whole model from a Blob when the load runs in a worker — the same code path the + * JVM takes over a file — and parse headers from the preload when it runs on the main thread. */ public class JsBlobRandomAccessSource private constructor( private val blob: Blob, private val preloadedBuffer: ByteArray, - private val blobSize: Long -) : RandomAccessSource, SuspendingRandomAccessSource { - - override val size: Long = blobSize - + override val size: Long, /** - * The size of the preloaded buffer in bytes. - * Reads within this range are synchronous. + * Whether blocking reads reach the whole Blob: `true` on a worker thread (`FileReaderSync` is + * defined), `false` on the main thread, where only the preloaded window is readable blockingly. */ + public val supportsSyncReads: Boolean, +) : RandomAccessSource, SuspendingRandomAccessSource { + + /** The size of the preloaded window in bytes; blocking reads inside it never touch the Blob. */ public val preloadedSize: Int = preloadedBuffer.size override fun readAt(position: Long, length: Int): ByteArray { - require(position >= 0) { "Position must be non-negative: $position" } - require(length >= 0) { "Length must be non-negative: $length" } - require(position + length <= size) { - "Read beyond end of file: position=$position, length=$length, size=$size" - } - - if (length == 0) return ByteArray(0) - - val pos = position.toInt() - val requestedEnd = pos + length - - // If within preloaded buffer, return from cache - if (requestedEnd <= preloadedBuffer.size) { - return preloadedBuffer.copyOfRange(pos, requestedEnd) - } - - // Beyond preloaded data - throw helpful error - throw UnsupportedOperationException( - "Synchronous read beyond preloaded buffer (${preloadedBuffer.size} bytes). " + - "Position: $position, Length: $length. " + - "Use readAtAsync() for tensor data loading, or increase preloadSize when opening." - ) + checkRange(position, length) + val out = ByteArray(length) + readAt(position, out, 0, length) + return out } override fun readAt(position: Long, buffer: ByteArray, offset: Int, length: Int): Int { - require(position >= 0) { "Position must be non-negative: $position" } - require(offset >= 0) { "Offset must be non-negative: $offset" } - require(length >= 0) { "Length must be non-negative: $length" } - require(offset + length <= buffer.size) { - "Buffer overflow: offset=$offset, length=$length, buffer.size=${buffer.size}" - } - + checkRange(position, length) + checkBuffer(buffer, offset, length) if (length == 0) return 0 - - val data = readAt(position, length) - data.copyInto(buffer, offset, 0, data.size) - return data.size + if (position + length <= preloadedSize) { + preloadedBuffer.copyInto(buffer, offset, position.toInt(), position.toInt() + length) + return length + } + if (!supportsSyncReads) { + throw UnsupportedOperationException( + "Blocking read beyond the preloaded $preloadedSize bytes (position=$position, length=$length) on a " + + "thread without FileReaderSync — the browser's main thread. Use the suspending read(), raise " + + "preloadSize when opening, or run the load in a Web Worker, where Blob reads are synchronous.", + ) + } + readSync(position, buffer, offset, length) + return length } override fun close() { - // Blob doesn't need explicit closing + // A Blob has no handle to release. } - /** - * Asynchronously read bytes from any position. - * - * Use this for loading tensor data that may be beyond the preloaded buffer. - */ - /** - * The suspending read (#1037): unlike [readAt], this is not limited to the preloaded window — - * a range outside it is fetched from the blob instead of failing. - */ - override suspend fun read(position: Long, length: Int): ByteArray = readAtAsync(position, length) + /** The suspending read: any range, any thread (#1037). */ + override suspend fun read(position: Long, length: Int): ByteArray { + checkRange(position, length) + val out = ByteArray(length) + read(position, out, 0, length) + return out + } /** Suspending read into [buffer]; see [readAt]. */ override suspend fun read(position: Long, buffer: ByteArray, offset: Int, length: Int): Int { - val bytes = readAtAsync(position, length) - bytes.copyInto(buffer, offset) - return bytes.size + checkRange(position, length) + checkBuffer(buffer, offset, length) + if (length == 0) return 0 + if (position + length <= preloadedSize) { + preloadedBuffer.copyInto(buffer, offset, position.toInt(), position.toInt() + length) + return length + } + val slice = sliceBlob(blob, position.toDouble(), (position + length).toDouble()) + val arrayBuffer = blobArrayBuffer(slice).await() + copyArrayBuffer(arrayBuffer, buffer, offset, length) + return length } - public suspend fun readAtAsync(position: Long, length: Int): ByteArray { + /** [read] under its original name. */ + public suspend fun readAtAsync(position: Long, length: Int): ByteArray = read(position, length) + + private fun checkRange(position: Long, length: Int) { require(position >= 0) { "Position must be non-negative: $position" } require(length >= 0) { "Length must be non-negative: $length" } - require(position + length <= size) { - "Read beyond end of file: position=$position, length=$length, size=$size" - } - - if (length == 0) return ByteArray(0) + require(position + length <= size) { "Read beyond end of file: position=$position, length=$length, size=$size" } + } - val pos = position.toInt() - val requestedEnd = pos + length + private fun checkBuffer(buffer: ByteArray, offset: Int, length: Int) { + require(offset >= 0) { "Offset must be non-negative: $offset" } + require(offset + length <= buffer.size) { "Buffer overflow: offset=$offset, length=$length, buffer.size=${buffer.size}" } + } - // If within preloaded buffer, return from cache (fast path) - if (requestedEnd <= preloadedBuffer.size) { - return preloadedBuffer.copyOfRange(pos, requestedEnd) + /** `FileReaderSync` over the Blob, staged through linear memory. Worker threads only. */ + private fun readSync(position: Long, dest: ByteArray, destOffset: Int, length: Int) { + var done = 0 + while (done < length) { + val n = minOf(length - done, STAGE_BYTES) + withScopedMemoryAllocator { allocator -> + val base = allocator.allocate(n) + jsReadBlobSyncInto(blob, (position + done).toDouble(), n, wasmMemory, base.address.toInt()) + unstage(base, dest, destOffset + done, n) + } + done += n } - - // Read from blob using slice + arrayBuffer via JS interop - val slice = sliceBlob(blob, pos, requestedEnd) - val arrayBuffer = blobArrayBuffer(slice).await() - return arrayBufferToByteArray(arrayBuffer) } public companion object { - /** - * Default preload size: 50MB should cover metadata for most models. - */ + /** Default preload: 50 MB covers the header of every model format this project reads. */ public const val DEFAULT_PRELOAD_SIZE: Int = 50 * 1024 * 1024 + private const val STAGE_BYTES = 4 * 1024 * 1024 + /** - * Open a Blob for streaming random access. - * - * @param blob A browser Blob or File object - * @param preloadSize Number of bytes to preload for synchronous access + * Open [blob] for random access, preloading its first [preloadSize] bytes (or all of it when + * smaller) for blocking reads. Suspends for the preload; works on any thread. */ public suspend fun open(blob: Blob, preloadSize: Int = DEFAULT_PRELOAD_SIZE): JsBlobRandomAccessSource { - val blobSize = getBlobSize(blob) - val actualPreloadSize = minOf(preloadSize, blobSize) - val slice = sliceBlob(blob, 0, actualPreloadSize) - val sliceSize = getBlobSize(slice) - val arrayBuffer = blobArrayBuffer(slice).await() - val bufferByteLength = getArrayBufferByteLength(arrayBuffer) - val bytes = arrayBufferToByteArray(arrayBuffer) - return JsBlobRandomAccessSource(blob, bytes, blobSize.toLong()) + require(preloadSize >= 0) { "preloadSize must be non-negative: $preloadSize" } + val blobSize = getBlobSize(blob).toLong() + val preload = minOf(preloadSize.toLong(), blobSize).toInt() + val window = ByteArray(preload) + if (preload > 0) { + val slice = sliceBlob(blob, 0.0, preload.toDouble()) + val arrayBuffer = blobArrayBuffer(slice).await() + copyArrayBuffer(arrayBuffer, window, 0, preload) + } + return JsBlobRandomAccessSource(blob, window, blobSize, hasFileReaderSync()) } /** - * Convert JavaScript ArrayBuffer to Kotlin ByteArray using Int8Array. + * Open [blob] without suspending — for code that runs inside a Web Worker and is not in a + * coroutine. Every read is blocking there, so nothing is preloaded unless [preloadSize] says + * so. Throws on a thread without `FileReaderSync` (the main thread); use [open] there. */ - private fun arrayBufferToByteArray(buffer: ArrayBuffer): ByteArray { - // One interop call per byte: Kotlin/Wasm 2.4 ships no `Int8Array` → `ByteArray` adapter - // and arrays cannot cross the boundary directly. For a 50 MB preload that is 50 million - // calls before the first header byte is parsed; SKEEP-007 Tier 1 replaces this with the - // sidecar's linear-memory bridge. Until then the loop stays, without the per-read logging. - val int8Array = Int8Array(buffer) - return ByteArray(int8Array.length) { i -> int8Array[i] } + public fun openSync(blob: Blob, preloadSize: Int = 0): JsBlobRandomAccessSource { + require(preloadSize >= 0) { "preloadSize must be non-negative: $preloadSize" } + if (!hasFileReaderSync()) { + throw UnsupportedOperationException( + "openSync needs FileReaderSync, which exists only on Web Worker threads; on the main thread use the suspending open()", + ) + } + val blobSize = getBlobSize(blob).toLong() + val preload = minOf(preloadSize.toLong(), blobSize).toInt() + val source = JsBlobRandomAccessSource(blob, ByteArray(0), blobSize, true) + if (preload == 0) return source + val window = ByteArray(preload) + source.readSync(0L, window, 0, preload) + return JsBlobRandomAccessSource(blob, window, blobSize, true) + } + + /** Whether this thread can read a Blob synchronously (a Web Worker). */ + public fun hasFileReaderSync(): Boolean = jsHasFileReaderSync() + + /** Copy [length] bytes of [buffer] into [dest] at [destOffset], 4 MB stages through linear memory. */ + private fun copyArrayBuffer(buffer: ArrayBuffer, dest: ByteArray, destOffset: Int, length: Int) { + require(getArrayBufferByteLength(buffer) >= length) { + "the Blob delivered ${getArrayBufferByteLength(buffer)} bytes, $length were requested" + } + var done = 0 + while (done < length) { + val n = minOf(length - done, STAGE_BYTES) + withScopedMemoryAllocator { allocator -> + val base = allocator.allocate(n) + jsCopyBufferInto(buffer, done, n, wasmMemory, base.address.toInt()) + unstage(base, dest, destOffset + done, n) + } + done += n + } + } + + /** Linear memory at [base] → [dest]: four bytes per load, the tail byte by byte. */ + private fun unstage(base: Pointer, dest: ByteArray, destOffset: Int, n: Int) { + var p: Pointer = base + var i = 0 + while (i + 4 <= n) { + val v = p.loadInt() + val o = destOffset + i + dest[o] = (v and 0xFF).toByte() + dest[o + 1] = ((v ushr 8) and 0xFF).toByte() + dest[o + 2] = ((v ushr 16) and 0xFF).toByte() + dest[o + 3] = ((v ushr 24) and 0xFF).toByte() + p += 4 + i += 4 + } + while (i < n) { + dest[destOffset + i] = p.loadByte() + p += 1 + i++ + } } } } -// Use external JS function declarations for proper WASM interop @JsFun("(blob) => blob.size") -private external fun getBlobSize(blob: Blob): Int +private external fun getBlobSize(blob: Blob): Double @JsFun("(blob) => blob.arrayBuffer()") private external fun blobArrayBuffer(blob: Blob): Promise @@ -167,4 +228,15 @@ private external fun blobArrayBuffer(blob: Blob): Promise private external fun getArrayBufferByteLength(buf: ArrayBuffer): Int @JsFun("(blob, start, end) => blob.slice(start, end)") -private external fun sliceBlob(blob: Blob, start: Int, end: Int): Blob +private external fun sliceBlob(blob: Blob, start: Double, end: Double): Blob + +@JsFun("() => typeof FileReaderSync !== 'undefined'") +private external fun jsHasFileReaderSync(): Boolean + +/** `buf[srcOffset, srcOffset + length)` → Kotlin linear memory at [ptr]. */ +@JsFun("(buf, srcOffset, length, kmem, ptr) => new Uint8Array(kmem.buffer).set(new Uint8Array(buf, srcOffset, length), ptr)") +private external fun jsCopyBufferInto(buf: ArrayBuffer, srcOffset: Int, length: Int, kmem: JsAny, ptr: Int) + +/** `blob[start, start + length)` through `FileReaderSync` → Kotlin linear memory at [ptr]. Worker threads only. */ +@JsFun("(blob, start, length, kmem, ptr) => new Uint8Array(kmem.buffer).set(new Uint8Array(new FileReaderSync().readAsArrayBuffer(blob.slice(start, start + length))), ptr)") +private external fun jsReadBlobSyncInto(blob: Blob, start: Double, length: Int, kmem: JsAny, ptr: Int) diff --git a/skainet-io/skainet-io-core/src/wasmJsTest/kotlin/sk/ainet/io/JsBlobRandomAccessSourceTest.kt b/skainet-io/skainet-io-core/src/wasmJsTest/kotlin/sk/ainet/io/JsBlobRandomAccessSourceTest.kt new file mode 100644 index 000000000..26f417b4f --- /dev/null +++ b/skainet-io/skainet-io-core/src/wasmJsTest/kotlin/sk/ainet/io/JsBlobRandomAccessSourceTest.kt @@ -0,0 +1,190 @@ +@file:OptIn(kotlin.js.ExperimentalWasmJsInterop::class) + +package sk.ainet.io + +import kotlinx.coroutines.test.runTest +import org.w3c.files.Blob +import kotlin.test.Test +import kotlin.test.assertEquals +import kotlin.test.assertFailsWith +import kotlin.test.assertFalse +import kotlin.test.assertTrue + +/** A Blob of [n] bytes from a 32-bit LCG seeded with [seed]; [expectedByte] is the same sequence in Kotlin. */ +@JsFun( + """(n, seed) => { + const a = new Uint8Array(n); + let s = seed | 0; + for (let i = 0; i < n; i++) { s = (Math.imul(s, 1103515245) + 12345) | 0; a[i] = (s >>> 16) & 0xff; } + return new Blob([a]); + }""", +) +private external fun patternBlob(n: Int, seed: Int): Blob + +/** + * A `FileReaderSync` for the main thread, for the test alone: a synchronous `XMLHttpRequest` on a + * Blob URL with the `x-user-defined` charset, the pre-`FileReaderSync` way to read bytes + * synchronously. Lets the worker-only path of the source run under Karma, which has no worker. + */ +@JsFun( + """() => { + globalThis.FileReaderSync = class { + readAsArrayBuffer(blob) { + const url = URL.createObjectURL(blob); + try { + const x = new XMLHttpRequest(); + x.open('GET', url, false); + x.overrideMimeType('text/plain; charset=x-user-defined'); + x.send(null); + const s = x.responseText; + const out = new Uint8Array(s.length); + for (let i = 0; i < s.length; i++) out[i] = s.charCodeAt(i) & 0xff; + return out.buffer; + } finally { + URL.revokeObjectURL(url); + } + } + }; + }""", +) +private external fun installFileReaderSyncPolyfill() + +@JsFun("() => { delete globalThis.FileReaderSync; }") +private external fun removeFileReaderSyncPolyfill() + +@JsFun("() => performance.now()") +private external fun now(): Double + +/** + * The Kotlin/Wasm Blob source (JVM ↔ wasm IO parity): `Long` positions, bulk copies through linear + * memory, blocking reads from the preload on the main thread and from the Blob inside a worker. + */ +class JsBlobRandomAccessSourceTest { + + private val size = 9 * 1024 * 1024 + 123 + private val preload = 1024 * 1024 + private val seed = 0x5EED + + private fun expectedByte(i: Int): Byte { + var s = seed + // The LCG is sequential; the tests only ask for a few positions, so a rerun from the seed is fine. + for (k in 0..i) s = s * 1103515245 + 12345 + return ((s ushr 16) and 0xFF).toByte() + } + + private fun assertPattern(bytes: ByteArray, from: Int, position: Long, length: Int, label: String) { + var s = seed + var i = 0 + while (i < position) { s = s * 1103515245 + 12345; i++ } + for (k in 0 until length) { + s = s * 1103515245 + 12345 + val expected = ((s ushr 16) and 0xFF).toByte() + if (bytes[from + k] != expected) { + throw AssertionError("$label: byte at blob position ${position + k} is ${bytes[from + k]}, expected $expected") + } + } + } + + @Test + fun theSourceReportsTheBlobsSizeAndItsPreload() = runTest { + val source = JsBlobRandomAccessSource.open(patternBlob(size, seed), preloadSize = preload) + assertEquals(size.toLong(), source.size) + assertEquals(preload, source.preloadedSize) + assertFalse(source.supportsSyncReads, "Karma runs the test on the main thread, which has no FileReaderSync") + + val whole = JsBlobRandomAccessSource.open(patternBlob(4096, seed), preloadSize = preload) + assertEquals(4096, whole.preloadedSize, "a Blob smaller than the preload is preloaded whole") + assertEquals(0, JsBlobRandomAccessSource.open(patternBlob(4096, seed), preloadSize = 0).preloadedSize) + } + + @Test + fun blockingReadsServeThePreloadAndRefuseTheRestOnTheMainThread() = runTest { + val source = JsBlobRandomAccessSource.open(patternBlob(size, seed), preloadSize = preload) + + val head = source.readAt(0L, 16) + assertPattern(head, 0, 0L, 16, "head") + assertEquals(expectedByte(0), head[0]) + + val tail = source.readAt((preload - 8).toLong(), 8) + assertPattern(tail, 0, (preload - 8).toLong(), 8, "end of the preload") + + val into = ByteArray(40) + assertEquals(24, source.readAt(1000L, into, 8, 24)) + assertPattern(into, 8, 1000L, 24, "read into a buffer at an offset") + assertEquals(0, into[0].toInt()) + assertEquals(0, into[39].toInt()) + + assertEquals(0, source.readAt(5L, ByteArray(0), 0, 0)) + assertEquals(expectedByte(77), source.readByteAt(77L)) + + val error = assertFailsWith { source.readAt((preload - 4).toLong(), 8) } + assertTrue(error.message!!.contains("Web Worker"), error.message) + assertTrue(error.message!!.contains("read()"), error.message) + assertFailsWith { source.readAt(size.toLong() - 4, 8) } + assertFailsWith { source.readAt(-1L, 8) } + assertFailsWith { source.readAt(0L, ByteArray(4), 2, 4) } + } + + @Test + fun suspendingReadsReachAnyRangeInBulk() = runTest { + val source = JsBlobRandomAccessSource.open(patternBlob(size, seed), preloadSize = preload) + + // Crosses the preload boundary: served from the Blob as one range. + val across = source.read((preload - 100).toLong(), 300) + assertPattern(across, 0, (preload - 100).toLong(), 300, "across the preload boundary") + + // Deep in the Blob, odd length, odd position. + val deep = source.read(5_000_001L, 777) + assertPattern(deep, 0, 5_000_001L, 777, "deep read") + + // The last bytes. + val last = source.read(size.toLong() - 5, 5) + assertPattern(last, 0, size.toLong() - 5, 5, "last bytes") + assertEquals(0, source.read(size.toLong(), 0).size) + + // Into a buffer at an offset. + val into = ByteArray(64) + assertEquals(32, source.read(3_333_333L, into, 16, 32)) + assertPattern(into, 16, 3_333_333L, 32, "suspending read into a buffer") + + // A read spanning two 4 MB stage boundaries, checked byte for byte. + val big = 8 * 1024 * 1024 + 500 + val t0 = now() + val bulk = source.read(500_000L, big) + val ms = now() - t0 + assertEquals(big, bulk.size) + assertPattern(bulk, 0, 500_000L, big, "bulk read") + println("[JsBlobRandomAccessSourceTest] suspending read of ${big / (1024 * 1024)} MB: ${ms.toInt()} ms (${(big / 1048576.0 / (ms / 1000.0)).toInt()} MB/s)") + + assertFailsWith { source.read(size.toLong() - 4, 8) } + } + + @Test + fun insideAWorkerBlockingReadsComeFromTheBlob() = runTest { + installFileReaderSyncPolyfill() + try { + assertTrue(JsBlobRandomAccessSource.hasFileReaderSync()) + val source = JsBlobRandomAccessSource.open(patternBlob(size, seed), preloadSize = 0) + assertTrue(source.supportsSyncReads) + assertEquals(0, source.preloadedSize) + + val deep = source.readAt(5_000_000L, 100) + assertPattern(deep, 0, 5_000_000L, 100, "blocking read from the Blob") + + // Across a 4 MB stage boundary, with the buffer form. + val into = ByteArray(200) + assertEquals(120, source.readAt((4 * 1024 * 1024 - 50).toLong(), into, 40, 120)) + assertPattern(into, 40, (4 * 1024 * 1024 - 50).toLong(), 120, "blocking read across a stage") + + val sync = JsBlobRandomAccessSource.openSync(patternBlob(size, seed), preloadSize = 4096) + assertEquals(size.toLong(), sync.size) + assertEquals(4096, sync.preloadedSize) + assertPattern(sync.readAt(0L, 4096), 0, 0L, 4096, "openSync preload") + assertPattern(sync.readAt(size.toLong() - 10, 10), 0, size.toLong() - 10, 10, "openSync tail") + } finally { + removeFileReaderSyncPolyfill() + } + assertFalse(JsBlobRandomAccessSource.hasFileReaderSync()) + assertFailsWith { JsBlobRandomAccessSource.openSync(patternBlob(16, seed)) } + } +} diff --git a/skainet-io/skainet-io-safetensors/build.gradle.kts b/skainet-io/skainet-io-safetensors/build.gradle.kts index 2213a8afd..85c976da0 100644 --- a/skainet-io/skainet-io-safetensors/build.gradle.kts +++ b/skainet-io/skainet-io-safetensors/build.gradle.kts @@ -66,6 +66,15 @@ kotlin { val commonTest by getting { dependencies { implementation(libs.kotlin.test) + implementation(libs.kotlinx.coroutines.test) + } + } + // The browser-side sharded load end to end: Blobs through io-core's JsBlobRandomAccessSource + // into the loader, materialised on the common CPU context. + val wasmJsTest by getting { + dependencies { + implementation(libs.kotlinx.browser) + implementation(project(":skainet-backends:skainet-backend-cpu")) } } val jvmTest by getting { diff --git a/skainet-io/skainet-io-safetensors/src/commonMain/kotlin/sk/ainet/io/safetensors/ShardedSafeTensorsParametersLoader.kt b/skainet-io/skainet-io-safetensors/src/commonMain/kotlin/sk/ainet/io/safetensors/ShardedSafeTensorsParametersLoader.kt index 186376510..3c96b7f7f 100644 --- a/skainet-io/skainet-io-safetensors/src/commonMain/kotlin/sk/ainet/io/safetensors/ShardedSafeTensorsParametersLoader.kt +++ b/skainet-io/skainet-io-safetensors/src/commonMain/kotlin/sk/ainet/io/safetensors/ShardedSafeTensorsParametersLoader.kt @@ -2,6 +2,7 @@ package sk.ainet.io.safetensors import sk.ainet.context.ExecutionContext import sk.ainet.io.ParametersLoader +import sk.ainet.io.RandomAccessSource import sk.ainet.lang.memory.plan.WeightForm import sk.ainet.lang.tensor.Shape import sk.ainet.lang.tensor.Tensor @@ -31,13 +32,15 @@ import kotlin.reflect.KClass * them all — a 40-shard load should not die on tensor 900 (mirrors the * GGUF loader's fail-fast contract, #919). * - * Platform note: this loader resolves shard files by path - * (`openRandomAccessSource`), which yields no sources on js/wasm/ - * androidNativeArm32. A provider-per-shard factory - * (`(shardFilename) -> RandomAccessSource`) for those platforms is a - * planned follow-up and needs an - * `openFromParsedIndex(index, shardProvider)` companion on - * [StreamingShardedSafeTensorsReader]. + * Two ways to name the checkpoint: the [indexPath] constructor resolves + * the shards beside the index file through `openRandomAccessSource`, + * which is how a JVM, Android or Kotlin/Native host loads; the + * `(indexJson, shardSource)` constructor takes the index as text and a + * provider that opens each shard by file name — the form for a browser + * or Wasm host, where there is no file system and a shard is a `Blob` + * (`JsBlobRandomAccessSource`) or fetched bytes + * (`ByteArrayRandomAccessSource`). Both deliver identical tensors for + * identical bytes. * * @param indexPath Path to `model.safetensors.index.json`. * @param onProgress Optional progress callback (current, total, tensorName); @@ -54,17 +57,66 @@ import kotlin.reflect.KClass * family-side skip policy (size guards, name allowlists) — dtype/policy * handling stays engine-side. `null` loads everything. */ -public class ShardedSafeTensorsParametersLoader( - private val indexPath: String, - private val onProgress: (current: Long, total: Long, message: String?) -> Unit = { _, _, _ -> }, - private val bf16Policy: Bf16LoadPolicy = NarrowFloatLoadPolicy.DEQUANT_TO_FP32, - private val fp16Policy: NarrowFloatLoadPolicy = NarrowFloatLoadPolicy.DEQUANT_TO_FP32, - private val allowPartial: Boolean = false, - private val tensorFilter: ((ShardedTensorInfo) -> Boolean)? = null, - private val weightForm: WeightForm? = null, - private val weightFormFor: ((String) -> WeightForm?)? = null, +public class ShardedSafeTensorsParametersLoader private constructor( + private val openReader: suspend (allowPartial: Boolean) -> StreamingShardedSafeTensorsReader, + private val onProgress: (current: Long, total: Long, message: String?) -> Unit, + private val bf16Policy: Bf16LoadPolicy, + private val fp16Policy: NarrowFloatLoadPolicy, + private val allowPartial: Boolean, + private val tensorFilter: ((ShardedTensorInfo) -> Boolean)?, + private val weightForm: WeightForm?, + private val weightFormFor: ((String) -> WeightForm?)?, ) : ParametersLoader { + /** The checkpoint by the path of its index file; shards are resolved beside it. */ + public constructor( + indexPath: String, + onProgress: (current: Long, total: Long, message: String?) -> Unit = { _, _, _ -> }, + bf16Policy: Bf16LoadPolicy = NarrowFloatLoadPolicy.DEQUANT_TO_FP32, + fp16Policy: NarrowFloatLoadPolicy = NarrowFloatLoadPolicy.DEQUANT_TO_FP32, + allowPartial: Boolean = false, + tensorFilter: ((ShardedTensorInfo) -> Boolean)? = null, + weightForm: WeightForm? = null, + weightFormFor: ((String) -> WeightForm?)? = null, + ) : this( + openReader = { partial -> StreamingShardedSafeTensorsReader.openFromIndex(indexPath, partial) }, + onProgress = onProgress, + bf16Policy = bf16Policy, + fp16Policy = fp16Policy, + allowPartial = allowPartial, + tensorFilter = tensorFilter, + weightForm = weightForm, + weightFormFor = weightFormFor, + ) + + /** + * The checkpoint by the content of its index file and a provider that opens each shard by + * file name (`model-00001-of-00003.safetensors`, …), answering `null` for a missing one. The + * provider may suspend — a browser opens a `Blob` per shard inside it — and is asked once per + * shard; the sources it returns are closed when `load` returns. See + * [StreamingShardedSafeTensorsReader.openFromIndexContent]. + */ + public constructor( + indexJson: String, + shardSource: suspend (shardFilename: String) -> RandomAccessSource?, + onProgress: (current: Long, total: Long, message: String?) -> Unit = { _, _, _ -> }, + bf16Policy: Bf16LoadPolicy = NarrowFloatLoadPolicy.DEQUANT_TO_FP32, + fp16Policy: NarrowFloatLoadPolicy = NarrowFloatLoadPolicy.DEQUANT_TO_FP32, + allowPartial: Boolean = false, + tensorFilter: ((ShardedTensorInfo) -> Boolean)? = null, + weightForm: WeightForm? = null, + weightFormFor: ((String) -> WeightForm?)? = null, + ) : this( + openReader = { partial -> StreamingShardedSafeTensorsReader.openFromIndexContent(indexJson, shardSource, partial) }, + onProgress = onProgress, + bf16Policy = bf16Policy, + fp16Policy = fp16Policy, + allowPartial = allowPartial, + tensorFilter = tensorFilter, + weightForm = weightForm, + weightFormFor = weightFormFor, + ) + init { weightForm?.let { SafeTensorsWeightForms.validate(it, "weightForm") } } @@ -81,7 +133,7 @@ public class ShardedSafeTensorsParametersLoader( dtype: KClass, onTensorLoaded: (String, Tensor) -> Unit ) { - StreamingShardedSafeTensorsReader.openFromIndex(indexPath, allowPartial).use { reader -> + openReader(allowPartial).use { reader -> val tensors = tensorFilter?.let { filter -> reader.tensors.filter(filter) } ?: reader.tensors @@ -137,6 +189,24 @@ public class ShardedSafeTensorsParametersLoader( tensorFilter = tensorFilter, ) + /** [withPolicy] for the `(indexJson, shardSource)` form. */ + public fun withPolicy( + indexJson: String, + shardSource: suspend (shardFilename: String) -> RandomAccessSource?, + policy: DTypePolicy, + onProgress: (current: Long, total: Long, message: String?) -> Unit = { _, _, _ -> }, + allowPartial: Boolean = false, + tensorFilter: ((ShardedTensorInfo) -> Boolean)? = null, + ): ShardedSafeTensorsParametersLoader = ShardedSafeTensorsParametersLoader( + indexJson = indexJson, + shardSource = shardSource, + onProgress = onProgress, + bf16Policy = SafeTensorsMaterializer.mapPolicyToBf16(policy), + fp16Policy = SafeTensorsMaterializer.mapPolicyToFp16(policy), + allowPartial = allowPartial, + tensorFilter = tensorFilter, + ) + /** * Pre-scan [tensors] and throw one aggregated error naming every * tensor whose dtype cannot materialize into [dtype], before any diff --git a/skainet-io/skainet-io-safetensors/src/commonMain/kotlin/sk/ainet/io/safetensors/StreamingShardedSafeTensorsReader.kt b/skainet-io/skainet-io-safetensors/src/commonMain/kotlin/sk/ainet/io/safetensors/StreamingShardedSafeTensorsReader.kt index c639d4e95..85b61e344 100644 --- a/skainet-io/skainet-io-safetensors/src/commonMain/kotlin/sk/ainet/io/safetensors/StreamingShardedSafeTensorsReader.kt +++ b/skainet-io/skainet-io-safetensors/src/commonMain/kotlin/sk/ainet/io/safetensors/StreamingShardedSafeTensorsReader.kt @@ -2,6 +2,7 @@ package sk.ainet.io.safetensors import kotlinx.coroutines.flow.Flow import kotlinx.coroutines.flow.MutableSharedFlow +import sk.ainet.io.RandomAccessSource import sk.ainet.io.model.LoadingProgress import sk.ainet.io.model.LoadingStage import sk.ainet.io.model.ProgressReportingLoader @@ -35,11 +36,19 @@ import sk.ainet.lang.tensor.storage.TensorStorage * * progressJob.cancel() * ``` + * + * Where there is no file system — a browser, a Wasm host — the shards come from wherever the caller + * keeps them, through [openFromIndexContent]: the index as text and one [RandomAccessSource] per + * shard name (a `JsBlobRandomAccessSource` over a `File` input, a `ByteArrayRandomAccessSource` over + * fetched bytes). Both entry points open every shard exactly once and keep it open until [close]. */ public class StreamingShardedSafeTensorsReader private constructor( /** The parsed index file */ public val index: SafeTensorsIndex, - private val basePath: String + /** Opens the shard with this file name, or returns `null` when it is missing. */ + private val openShard: suspend (shardFilename: String) -> RandomAccessSource?, + /** The file path a shard name resolves to, when the shards are files; `null` otherwise. */ + private val pathOf: (shardFilename: String) -> String?, ) : ProgressReportingLoader, AutoCloseable { private val _progress = MutableSharedFlow(replay = 0, extraBufferCapacity = 64) @@ -64,6 +73,9 @@ public class StreamingShardedSafeTensorsReader private constructor( /** Map of shard filename to its reader (for tensor data access) */ private val shardReaders: MutableMap = mutableMapOf() + /** The file behind each loaded shard, when it has one ([pathOf], else the source's own `filePath`). */ + private val shardPaths: MutableMap = mutableMapOf() + /** Whether all shards were loaded successfully */ public val isComplete: Boolean get() = _missingShards.isEmpty() @@ -120,6 +132,8 @@ public class StreamingShardedSafeTensorsReader private constructor( * @throws IllegalStateException if the containing shard was not * loaded, or if the per-shard reader does not surface the * tensor (consistency check). + * @throws UnsupportedOperationException if the shard did not come + * from a file (a Blob or in-memory source has nothing to map). */ public fun loadTensorStorageMapped(tensor: ShardedTensorInfo): TensorStorage { val reader = shardReaders[tensor.shardFilename] @@ -128,7 +142,10 @@ public class StreamingShardedSafeTensorsReader private constructor( ?: throw IllegalStateException( "Tensor '${tensor.name}' not found in shard '${tensor.shardFilename}'", ) - val path = resolveShardPath(tensor.shardFilename) + val path = shardPaths[tensor.shardFilename] + ?: throw UnsupportedOperationException( + "Shard '${tensor.shardFilename}' is not a file (its source has no filePath): mapped loading needs file-backed shards", + ) return reader.loadTensorStorageMapped(streamingTensor, path) } @@ -164,15 +181,24 @@ public class StreamingShardedSafeTensorsReader private constructor( _progress.emit(LoadingProgress.StageStarted(discoverStage, "Checking shard files")) val discoverStart = currentTimeMillis() - val availableShards = mutableListOf() + // Each shard is opened once, here, and the source is handed to its reader below; a shard + // whose opener answers `null` (or throws, under allowPartial) is missing. + val openedShards = LinkedHashMap() val missingShardFiles = mutableListOf() for (shardFilename in shardFiles) { - val shardPath = resolveShardPath(shardFilename) - val source = createRandomAccessSource(shardPath) + val source = try { + openShard(shardFilename) + } catch (e: Exception) { + if (!allowPartial) { + openedShards.values.forEach { it.close() } + _progress.emit(LoadingProgress.Failed(discoverStage, e, "Failed to open shard: $shardFilename")) + throw e + } + null + } if (source != null) { - source.close() - availableShards.add(shardFilename) + openedShards[shardFilename] = source } else { missingShardFiles.add(shardFilename) } @@ -181,25 +207,20 @@ public class StreamingShardedSafeTensorsReader private constructor( _missingShards.addAll(missingShardFiles) if (missingShardFiles.isNotEmpty() && !allowPartial) { - _progress.emit(LoadingProgress.Failed( - discoverStage, - SafeTensorsShardException.IncompleteShard( - availableShards.size, - shardFiles.size, - missingShardFiles - ), - "Missing shards: ${missingShardFiles.joinToString()}" - )) - throw SafeTensorsShardException.IncompleteShard( - availableShards.size, + openedShards.values.forEach { it.close() } + val incomplete = SafeTensorsShardException.IncompleteShard( + openedShards.size, shardFiles.size, missingShardFiles ) + _progress.emit(LoadingProgress.Failed(discoverStage, incomplete, "Missing shards: ${missingShardFiles.joinToString()}")) + throw incomplete } _progress.emit(LoadingProgress.StageCompleted(discoverStage, currentTimeMillis() - discoverStart)) // Load each shard + val availableShards = openedShards.keys.toList() for ((shardIndex, shardFilename) in availableShards.withIndex()) { val stageId = "load_shard_${shardIndex + 1}" val stageName = "Loading shard ${shardIndex + 1}/${availableShards.size}" @@ -208,13 +229,11 @@ public class StreamingShardedSafeTensorsReader private constructor( _progress.emit(LoadingProgress.StageStarted(stage, shardFilename)) val shardStart = currentTimeMillis() + val source = openedShards.getValue(shardFilename) try { - val shardPath = resolveShardPath(shardFilename) - val source = createRandomAccessSource(shardPath) - ?: throw SafeTensorsShardException.ShardNotFound(shardFilename, shardPath) - val reader = StreamingSafeTensorsReader.open(source) shardReaders[shardFilename] = reader + shardPaths[shardFilename] = pathOf(shardFilename) ?: source.filePath // Add tensors with shard info val shardInfo = SafeTensorsIndexParser.parseShardFilename(shardFilename) @@ -244,7 +263,11 @@ public class StreamingShardedSafeTensorsReader private constructor( _progress.emit(LoadingProgress.StageCompleted(stage, currentTimeMillis() - shardStart)) } catch (e: Exception) { + source.close() if (!allowPartial) { + // Release what was opened: this reader is never returned to the caller. + close() + availableShards.drop(shardIndex + 1).forEach { openedShards.getValue(it).close() } _progress.emit(LoadingProgress.Failed(stage, e, "Failed to load shard: $shardFilename")) throw e } @@ -268,15 +291,6 @@ public class StreamingShardedSafeTensorsReader private constructor( _progress.emit(LoadingProgress.Completed(totalDuration, summary)) } - private fun resolveShardPath(shardFilename: String): String { - // basePath is the directory containing the index file - return if (basePath.endsWith("/")) { - basePath + shardFilename - } else { - "$basePath/$shardFilename" - } - } - // ========== Companion ========== public companion object { @@ -299,14 +313,56 @@ public class StreamingShardedSafeTensorsReader private constructor( val index = SafeTensorsIndexParser.parse(indexContent) - // Get base directory + // Shards live beside the index file val basePath = indexPath.substringBeforeLast("/") + val resolve = { shardFilename: String -> resolveShardPath(basePath, shardFilename) } + + return open(index, indexPath, allowPartial, openShard = { createRandomAccessSource(resolve(it)) }, pathOf = resolve) + } - val reader = StreamingShardedSafeTensorsReader(index, basePath) + /** + * Open a sharded SafeTensors model whose index is already in hand as text and whose shards + * the caller provides — the entry point for hosts without a file system (#1246 follow-up). + * + * [shardSource] is asked once per shard name in the index (`model-00001-of-00003.safetensors`, + * …) and answers with an open [RandomAccessSource] the reader owns from then on, or `null` for + * a shard that is not there. It may suspend, so a browser can open a `Blob` per shard inside + * it. Mapped loading ([loadTensorStorageMapped]) is available only for sources that carry a + * [RandomAccessSource.filePath]. + * + * @param indexJson The content of `model.safetensors.index.json` + * @param shardSource Opens a shard by file name; `null` when it is missing + * @param allowPartial If true, continue loading even if some shards are missing + * @throws SafeTensorsIndexParseException if the index is invalid + * @throws SafeTensorsShardException if shards cannot be read + */ + public suspend fun openFromIndexContent( + indexJson: String, + shardSource: suspend (shardFilename: String) -> RandomAccessSource?, + allowPartial: Boolean = false, + ): StreamingShardedSafeTensorsReader = + openFromParsedIndex(SafeTensorsIndexParser.parse(indexJson), shardSource, allowPartial) + + /** [openFromIndexContent] for an index the caller has parsed already. */ + public suspend fun openFromParsedIndex( + index: SafeTensorsIndex, + shardSource: suspend (shardFilename: String) -> RandomAccessSource?, + allowPartial: Boolean = false, + ): StreamingShardedSafeTensorsReader = + open(index, "", allowPartial, openShard = shardSource, pathOf = { null }) + + private suspend fun open( + index: SafeTensorsIndex, + indexLabel: String, + allowPartial: Boolean, + openShard: suspend (String) -> RandomAccessSource?, + pathOf: (String) -> String?, + ): StreamingShardedSafeTensorsReader { + val reader = StreamingShardedSafeTensorsReader(index, openShard, pathOf) // Emit parse_index stage (already done, but emit for consistency) val parseStage = LoadingStage("parse_index", "Parsing index", 0, 2 + index.shardCount + 1) - reader._progress.emit(LoadingProgress.StageStarted(parseStage, indexPath)) + reader._progress.emit(LoadingProgress.StageStarted(parseStage, indexLabel)) reader._progress.emit(LoadingProgress.StageCompleted(parseStage, 0)) // Load shards @@ -315,6 +371,9 @@ public class StreamingShardedSafeTensorsReader private constructor( return reader } + private fun resolveShardPath(basePath: String, shardFilename: String): String = + if (basePath.endsWith("/")) basePath + shardFilename else "$basePath/$shardFilename" + /** * Open a sharded SafeTensors model from any shard file. * diff --git a/skainet-io/skainet-io-safetensors/src/commonTest/kotlin/sk/ainet/io/safetensors/ShardFixtures.kt b/skainet-io/skainet-io-safetensors/src/commonTest/kotlin/sk/ainet/io/safetensors/ShardFixtures.kt new file mode 100644 index 000000000..99ce81f44 --- /dev/null +++ b/skainet-io/skainet-io-safetensors/src/commonTest/kotlin/sk/ainet/io/safetensors/ShardFixtures.kt @@ -0,0 +1,95 @@ +package sk.ainet.io.safetensors + +/** + * In-memory sharded SafeTensors fixtures for the provider-based entry points: shard bytes and the + * index text, built in common Kotlin so the JVM, wasmJs and native tests share one checkpoint. + * + * Layout (the JVM file fixture's, without the F16 shard entry): + * - `model-00001-of-00002.safetensors`: `alpha.weight` (F32 2×2), `bravo.weight` (BF16 4) + * - `model-00002-of-00002.safetensors`: `zulu.weight` (F32 3) + */ +internal object ShardFixtures { + + class Entry(val name: String, val dtype: String, val shape: List, val bytes: ByteArray) + + const val SHARD_1 = "model-00001-of-00002.safetensors" + const val SHARD_2 = "model-00002-of-00002.safetensors" + + val alphaValues = floatArrayOf(1f, 2f, 3f, 4f) + val bravoValues = floatArrayOf(0.5f, -1.0f, 2.5f, -64.0f) // exactly representable in BF16 + val zuluValues = floatArrayOf(-7f, 0f, 42f) + + /** Little-endian FP32. */ + fun fp32Bytes(values: FloatArray): ByteArray { + val out = ByteArray(values.size * 4) + for ((i, v) in values.withIndex()) { + val bits = v.toRawBits() + out[i * 4] = (bits and 0xFF).toByte() + out[i * 4 + 1] = ((bits ushr 8) and 0xFF).toByte() + out[i * 4 + 2] = ((bits ushr 16) and 0xFF).toByte() + out[i * 4 + 3] = ((bits ushr 24) and 0xFF).toByte() + } + return out + } + + /** Little-endian BF16: the upper half of the FP32 bits, rounded to nearest even. */ + fun bf16Bytes(values: FloatArray): ByteArray { + val out = ByteArray(values.size * 2) + for ((i, v) in values.withIndex()) { + val bits = v.toRawBits() + val rounding = 0x7FFF + ((bits ushr 16) and 1) + val bf16 = (bits + rounding) ushr 16 + out[i * 2] = (bf16 and 0xFF).toByte() + out[i * 2 + 1] = ((bf16 ushr 8) and 0xFF).toByte() + } + return out + } + + /** One SafeTensors file: `u64 header size | JSON header | tensor bytes` in entry order. */ + fun shard(entries: List, metadata: Map = emptyMap()): ByteArray { + var offset = 0L + val headerEntries = ArrayList() + if (metadata.isNotEmpty()) { + headerEntries += "\"__metadata__\": {" + metadata.entries.joinToString(", ") { (k, v) -> "\"$k\": \"$v\"" } + "}" + } + for (e in entries) { + val start = offset + offset += e.bytes.size + headerEntries += "\"${e.name}\": {\"dtype\": \"${e.dtype}\", \"shape\": [${e.shape.joinToString(", ")}], \"data_offsets\": [$start, $offset]}" + } + val header = headerEntries.joinToString(prefix = "{", postfix = "}", separator = ", ").encodeToByteArray() + val out = ByteArray(8 + header.size + offset.toInt()) + for (i in 0 until 8) out[i] = ((header.size.toLong() shr (i * 8)) and 0xFF).toByte() + header.copyInto(out, 8) + var p = 8 + header.size + for (e in entries) { + e.bytes.copyInto(out, p) + p += e.bytes.size + } + return out + } + + /** The two-shard checkpoint: shard name → bytes. */ + fun twoShards(): Map = mapOf( + SHARD_1 to shard( + listOf( + Entry("alpha.weight", "F32", listOf(2L, 2L), fp32Bytes(alphaValues)), + Entry("bravo.weight", "BF16", listOf(4L), bf16Bytes(bravoValues)), + ), + metadata = mapOf("format" to "pt"), + ), + SHARD_2 to shard(listOf(Entry("zulu.weight", "F32", listOf(3L), fp32Bytes(zuluValues)))), + ) + + /** `model.safetensors.index.json` for [twoShards]. */ + fun twoShardIndex(shards: Map = twoShards()): String = """ + { + "metadata": {"total_size": ${shards.values.sumOf { it.size.toLong() }}}, + "weight_map": { + "alpha.weight": "$SHARD_1", + "bravo.weight": "$SHARD_1", + "zulu.weight": "$SHARD_2" + } + } + """.trimIndent() +} diff --git a/skainet-io/skainet-io-safetensors/src/commonTest/kotlin/sk/ainet/io/safetensors/ShardedSafeTensorsReaderSourceTest.kt b/skainet-io/skainet-io-safetensors/src/commonTest/kotlin/sk/ainet/io/safetensors/ShardedSafeTensorsReaderSourceTest.kt new file mode 100644 index 000000000..8fa600f1d --- /dev/null +++ b/skainet-io/skainet-io-safetensors/src/commonTest/kotlin/sk/ainet/io/safetensors/ShardedSafeTensorsReaderSourceTest.kt @@ -0,0 +1,140 @@ +package sk.ainet.io.safetensors + +import kotlinx.coroutines.test.runTest +import sk.ainet.io.ByteArrayRandomAccessSource +import sk.ainet.io.RandomAccessSource +import sk.ainet.lang.tensor.storage.BufferHandle +import kotlin.test.Test +import kotlin.test.assertContentEquals +import kotlin.test.assertEquals +import kotlin.test.assertFailsWith +import kotlin.test.assertFalse +import kotlin.test.assertTrue + +/** + * [StreamingShardedSafeTensorsReader.openFromIndexContent]: a sharded checkpoint whose shards the + * caller provides — the entry point for a browser or Wasm host, where there is no path to resolve + * (JVM ↔ wasm IO parity). Runs on every target; the sources are in-memory. + */ +class ShardedSafeTensorsReaderSourceTest { + + /** A source that records whether it was closed and may claim to be a file. */ + private class Probe(bytes: ByteArray, override val filePath: String? = null) : RandomAccessSource { + private val inner = ByteArrayRandomAccessSource(bytes) + var closed = 0 + override val size: Long get() = inner.size + override fun readAt(position: Long, length: Int): ByteArray = inner.readAt(position, length) + override fun readAt(position: Long, buffer: ByteArray, offset: Int, length: Int): Int = inner.readAt(position, buffer, offset, length) + override fun close() { closed++ } + } + + private val shards = ShardFixtures.twoShards() + private val index = ShardFixtures.twoShardIndex(shards) + + @Test + fun theProviderIsAskedOncePerShardAndTensorsArriveNameSorted() = runTest { + val asked = ArrayList() + val opened = HashMap() + val reader = StreamingShardedSafeTensorsReader.openFromIndexContent(index, { name -> + asked += name + shards[name]?.let { Probe(it).also { p -> opened[name] = p } } + }) + + assertEquals(listOf(ShardFixtures.SHARD_1, ShardFixtures.SHARD_2), asked, "each shard is opened exactly once") + assertEquals(listOf("alpha.weight", "bravo.weight", "zulu.weight"), reader.tensors.map { it.name }) + assertTrue(reader.isComplete) + assertEquals(listOf(ShardFixtures.SHARD_1, ShardFixtures.SHARD_2), reader.loadedShards) + assertEquals(2, reader.index.shardCount) + assertEquals(shards.values.sumOf { it.size.toLong() }, reader.totalSize) + assertEquals("pt", reader.metadata["format"], "shard metadata is merged") + + val zulu = reader.tensors.single { it.name == "zulu.weight" } + assertEquals("2/2", zulu.shardLocation) + assertContentEquals(ShardFixtures.fp32Bytes(ShardFixtures.zuluValues), reader.loadTensorData(zulu)) + assertContentEquals(ShardFixtures.fp32Bytes(ShardFixtures.alphaValues), reader.loadTensorData("alpha.weight")) + assertContentEquals(ShardFixtures.bf16Bytes(ShardFixtures.bravoValues), reader.loadTensorData("bravo.weight")) + + val mapped = assertFailsWith { reader.loadTensorStorageMapped(zulu) } + assertTrue(mapped.message!!.contains("filePath"), mapped.message) + + assertTrue(opened.values.all { it.closed == 0 }, "sources stay open while the reader is") + reader.close() + assertTrue(opened.values.all { it.closed == 1 }, "close() closes every shard source once: ${opened.mapValues { it.value.closed }}") + } + + @Test + fun aSourceThatNamesItsFileKeepsMappedLoadingAvailable() = runTest { + val reader = StreamingShardedSafeTensorsReader.openFromIndexContent(index, { name -> + shards[name]?.let { Probe(it, filePath = "/models/$name") } + }) + val storage = reader.loadTensorStorageMapped("alpha.weight") + val handle = storage.buffer as BufferHandle.FileBacked + assertEquals("/models/${ShardFixtures.SHARD_1}", handle.path) + assertEquals(ShardFixtures.alphaValues.size * 4L, handle.sizeInBytes) + reader.close() + } + + @Test + fun aMissingShardIsIncompleteUnlessPartialLoadsAreAllowed() = runTest { + val opened = ArrayList() + val onlyFirst: suspend (String) -> RandomAccessSource? = { name -> + if (name == ShardFixtures.SHARD_1) Probe(shards.getValue(name)).also { opened += it } else null + } + + val error = assertFailsWith { + StreamingShardedSafeTensorsReader.openFromIndexContent(index, onlyFirst) + } + assertEquals(listOf(ShardFixtures.SHARD_2), error.missingShards) + assertEquals(1, error.loadedCount) + assertTrue(opened.all { it.closed == 1 }, "the shard that was opened is closed again before the failure propagates") + + val partial = StreamingShardedSafeTensorsReader.openFromIndexContent(index, onlyFirst, allowPartial = true) + assertFalse(partial.isComplete) + assertEquals(listOf(ShardFixtures.SHARD_2), partial.missingShards) + assertEquals(listOf("alpha.weight", "bravo.weight"), partial.tensors.map { it.name }) + assertFailsWith { partial.loadTensorData("zulu.weight") } + partial.close() + } + + @Test + fun aShardThatFailsToParseClosesEverythingUnlessPartialLoadsAreAllowed() = runTest { + val opened = HashMap() + val corruptSecond: suspend (String) -> RandomAccessSource? = { name -> + val bytes = if (name == ShardFixtures.SHARD_2) ByteArray(64) { 0x7F } else shards.getValue(name) + Probe(bytes).also { opened[name] = it } + } + + assertFailsWith { StreamingShardedSafeTensorsReader.openFromIndexContent(index, corruptSecond) } + assertEquals(2, opened.size) + assertTrue(opened.values.all { it.closed >= 1 }, "every opened source is closed on failure: ${opened.mapValues { it.value.closed }}") + + opened.clear() + val partial = StreamingShardedSafeTensorsReader.openFromIndexContent(index, corruptSecond, allowPartial = true) + assertEquals(listOf(ShardFixtures.SHARD_2), partial.missingShards) + assertEquals(listOf("alpha.weight", "bravo.weight"), partial.tensors.map { it.name }) + assertEquals(1, opened.getValue(ShardFixtures.SHARD_2).closed, "the corrupt shard's source is released") + partial.close() + } + + @Test + fun aProviderThatThrowsPropagatesUnlessPartialLoadsAreAllowed() = runTest { + val throwing: suspend (String) -> RandomAccessSource? = { name -> + if (name == ShardFixtures.SHARD_2) throw IllegalStateException("network down") else Probe(shards.getValue(name)) + } + val error = assertFailsWith { StreamingShardedSafeTensorsReader.openFromIndexContent(index, throwing) } + assertEquals("network down", error.message) + + val partial = StreamingShardedSafeTensorsReader.openFromIndexContent(index, throwing, allowPartial = true) + assertEquals(listOf(ShardFixtures.SHARD_2), partial.missingShards) + partial.close() + } + + @Test + fun theParsedIndexEntryPointTakesAnIndexObject() = runTest { + val parsed = SafeTensorsIndexParser.parse(index) + val reader = StreamingShardedSafeTensorsReader.openFromParsedIndex(parsed, { shards[it]?.let(::ByteArrayRandomAccessSource) }) + assertEquals(3, reader.tensors.size) + assertEquals(parsed, reader.index) + reader.close() + } +} diff --git a/skainet-io/skainet-io-safetensors/src/jvmTest/kotlin/sk/ainet/io/safetensors/ShardedSafeTensorsParametersLoaderJvmTest.kt b/skainet-io/skainet-io-safetensors/src/jvmTest/kotlin/sk/ainet/io/safetensors/ShardedSafeTensorsParametersLoaderJvmTest.kt index 8dc18ad20..ccd68ece8 100644 --- a/skainet-io/skainet-io-safetensors/src/jvmTest/kotlin/sk/ainet/io/safetensors/ShardedSafeTensorsParametersLoaderJvmTest.kt +++ b/skainet-io/skainet-io-safetensors/src/jvmTest/kotlin/sk/ainet/io/safetensors/ShardedSafeTensorsParametersLoaderJvmTest.kt @@ -13,6 +13,7 @@ import kotlin.test.assertFailsWith import kotlin.test.assertTrue import kotlinx.coroutines.runBlocking import sk.ainet.context.DirectCpuExecutionContext +import sk.ainet.io.ByteArrayRandomAccessSource import sk.ainet.io.JvmRandomAccessSource import sk.ainet.lang.tensor.Tensor import sk.ainet.lang.tensor.data.Bf16DenseTensorData @@ -308,6 +309,48 @@ class ShardedSafeTensorsParametersLoaderJvmTest { } } + @Test + fun `the provider constructor delivers what the path constructor delivers`() { + val dir = Files.createTempDirectory("sharded-loader-") + try { + val indexPath = writeTwoShardFixture(dir) + val (pathOrder, byPath) = loadAll(ShardedSafeTensorsParametersLoader(indexPath)) + + val indexJson = File(indexPath).readText() + val asked = mutableListOf() + val (providerOrder, byProvider) = loadAll( + ShardedSafeTensorsParametersLoader(indexJson, { name -> + asked += name + JvmRandomAccessSource.open(dir.resolve(name).toString()) + }), + ) + + assertEquals(pathOrder, providerOrder) + assertEquals(listOf("model-00001-of-00002.safetensors", "model-00002-of-00002.safetensors"), asked) + for (name in pathOrder) { + assertContentEquals( + byPath[name]!!.data.copyToFloatArray(), + byProvider[name]!!.data.copyToFloatArray(), + "$name through the provider", + ) + } + + // In-memory shards (what a fetch delivers) through the same constructor, with a policy. + val bytes = asked.associateWith { dir.resolve(it).toFile().readBytes() } + val (_, byBytes) = loadAll( + ShardedSafeTensorsParametersLoader.withPolicy( + indexJson, + { name -> bytes[name]?.let(::ByteArrayRandomAccessSource) }, + DTypePolicy.Prefer(FP32), + ), + ) + assertContentEquals(alphaValues, byBytes["alpha.weight"]!!.data.copyToFloatArray()) + assertContentEquals(zuluValues, byBytes["zulu.weight"]!!.data.copyToFloatArray()) + } finally { + dir.toFile().deleteRecursively() + } + } + @Test fun `one-shard index parity with single-file loader`() { // Guards the SafeTensorsMaterializer extraction: identical bytes through diff --git a/skainet-io/skainet-io-safetensors/src/wasmJsTest/kotlin/sk/ainet/io/safetensors/ShardedSafeTensorsBlobTest.kt b/skainet-io/skainet-io-safetensors/src/wasmJsTest/kotlin/sk/ainet/io/safetensors/ShardedSafeTensorsBlobTest.kt new file mode 100644 index 000000000..f453f75d8 --- /dev/null +++ b/skainet-io/skainet-io-safetensors/src/wasmJsTest/kotlin/sk/ainet/io/safetensors/ShardedSafeTensorsBlobTest.kt @@ -0,0 +1,135 @@ +@file:OptIn(kotlin.js.ExperimentalWasmJsInterop::class) + +package sk.ainet.io.safetensors + +import kotlinx.coroutines.test.runTest +import org.khronos.webgl.Uint8Array +import org.w3c.files.Blob +import sk.ainet.context.DirectCpuExecutionContext +import sk.ainet.io.JsBlobRandomAccessSource +import sk.ainet.lang.tensor.Tensor +import sk.ainet.lang.types.FP32 +import kotlin.test.Test +import kotlin.test.assertContentEquals +import kotlin.test.assertEquals +import kotlin.test.assertTrue + +@JsFun("(n) => new Uint8Array(n)") +private external fun newUint8Array(n: Int): Uint8Array + +@JsFun("(a, i, v) => { a[i] = v; }") +private external fun setByte(a: Uint8Array, i: Int, v: Int) + +@JsFun("(a) => new Blob([a])") +private external fun toBlob(a: Uint8Array): Blob + +/** The fixtures are a few hundred bytes: one interop call per byte is fine here. */ +private fun ByteArray.toBlob(): Blob { + val a = newUint8Array(size) + for (i in indices) setByte(a, i, this[i].toInt() and 0xFF) + return toBlob(a) +} + +/** A main-thread `FileReaderSync` for the test (see io-core's blob source test): sync XHR on a Blob URL. */ +@JsFun( + """() => { + globalThis.FileReaderSync = class { + readAsArrayBuffer(blob) { + const url = URL.createObjectURL(blob); + try { + const x = new XMLHttpRequest(); + x.open('GET', url, false); + x.overrideMimeType('text/plain; charset=x-user-defined'); + x.send(null); + const s = x.responseText; + const out = new Uint8Array(s.length); + for (let i = 0; i < s.length; i++) out[i] = s.charCodeAt(i) & 0xff; + return out.buffer; + } finally { + URL.revokeObjectURL(url); + } + } + }; + }""", +) +private external fun installFileReaderSyncPolyfill() + +@JsFun("() => { delete globalThis.FileReaderSync; }") +private external fun removeFileReaderSyncPolyfill() + +/** + * A sharded SafeTensors checkpoint from browser Blobs, end to end: the index as text, one + * `JsBlobRandomAccessSource` per shard through the provider constructor, tensors materialised on + * the CPU context — the browser's counterpart of the JVM's path-based load (IO parity, WP-D1). + */ +class ShardedSafeTensorsBlobTest { + + private val shards = ShardFixtures.twoShards() + private val index = ShardFixtures.twoShardIndex(shards) + + private suspend fun loadAll(loader: ShardedSafeTensorsParametersLoader): Pair, Map>> { + val ctx = DirectCpuExecutionContext.create() + val order = ArrayList() + val out = HashMap>() + loader.load(ctx, FP32::class) { name, tensor -> + order += name + out[name] = tensor + } + return order to out + } + + private fun assertFixture(order: List, tensors: Map>) { + assertEquals(listOf("alpha.weight", "bravo.weight", "zulu.weight"), order) + assertContentEquals(ShardFixtures.alphaValues, tensors.getValue("alpha.weight").data.copyToFloatArray()) + assertContentEquals(ShardFixtures.bravoValues, tensors.getValue("bravo.weight").data.copyToFloatArray(), "BF16 dequantised exactly") + assertContentEquals(ShardFixtures.zuluValues, tensors.getValue("zulu.weight").data.copyToFloatArray()) + assertEquals(listOf(2, 2), tensors.getValue("alpha.weight").shape.dimensions.toList()) + } + + @Test + fun onTheMainThreadThePreloadedWindowServesTheLoad() = runTest { + val blobs = shards.mapValues { it.value.toBlob() } + val asked = ArrayList() + val loader = ShardedSafeTensorsParametersLoader(index, { name -> + asked += name + blobs[name]?.let { JsBlobRandomAccessSource.open(it) } // default preload covers a small shard whole + }) + val (order, tensors) = loadAll(loader) + assertEquals(listOf(ShardFixtures.SHARD_1, ShardFixtures.SHARD_2), asked) + assertFixture(order, tensors) + } + + @Test + fun insideAWorkerNothingIsPreloadedAndTheBlobIsReadBlockingly() = runTest { + installFileReaderSyncPolyfill() + try { + val blobs = shards.mapValues { it.value.toBlob() } + val sources = ArrayList() + val loader = ShardedSafeTensorsParametersLoader(index, { name -> + blobs[name]?.let { JsBlobRandomAccessSource.openSync(it).also(sources::add) } + }) + val (order, tensors) = loadAll(loader) + assertTrue(sources.all { it.preloadedSize == 0 && it.supportsSyncReads }, sources.toString()) + assertFixture(order, tensors) + } finally { + removeFileReaderSyncPolyfill() + } + } + + @Test + fun aMissingShardBlobIsReportedLikeAMissingFile() = runTest { + val blobs = shards.mapValues { it.value.toBlob() } + val loader = ShardedSafeTensorsParametersLoader(index, { name -> + if (name == ShardFixtures.SHARD_2) null else JsBlobRandomAccessSource.open(blobs.getValue(name)) + }) + var failure: Throwable? = null + try { loadAll(loader) } catch (e: SafeTensorsShardException.IncompleteShard) { failure = e } + assertEquals(listOf(ShardFixtures.SHARD_2), (failure as SafeTensorsShardException.IncompleteShard).missingShards) + + val partial = ShardedSafeTensorsParametersLoader(index, { name -> + if (name == ShardFixtures.SHARD_2) null else JsBlobRandomAccessSource.open(blobs.getValue(name)) + }, allowPartial = true) + val (order, _) = loadAll(partial) + assertEquals(listOf("alpha.weight", "bravo.weight"), order) + } +} diff --git a/skainet-lang/skainet-lang-core/api/jvm/skainet-lang-core.api b/skainet-lang/skainet-lang-core/api/jvm/skainet-lang-core.api index c2564589a..e55c6a44d 100644 --- a/skainet-lang/skainet-lang-core/api/jvm/skainet-lang-core.api +++ b/skainet-lang/skainet-lang-core/api/jvm/skainet-lang-core.api @@ -1551,6 +1551,7 @@ public final class sk/ainet/lang/memory/plan/AllocationResolver { public final class sk/ainet/lang/memory/plan/Budget { public static final field Companion Lsk/ainet/lang/memory/plan/Budget$Companion; public static final field RESERVE_ANDROID_JVM J + public static final field RESERVE_BROWSER J public static final field RESERVE_NATIVE J public fun (JLjava/lang/String;)V public final fun component1 ()J @@ -1913,6 +1914,7 @@ public final class sk/ainet/lang/memory/plan/PlanVsActualLine { } public final class sk/ainet/lang/memory/plan/PlannerProfile { + public static final field BROWSER_AVAILABLE_BYTES J public static final field Companion Lsk/ainet/lang/memory/plan/PlannerProfile$Companion; public static final field MOBILE_RAM_CEILING J public static final field OFF_HEAP_THRESHOLD J @@ -1952,6 +1954,7 @@ public final class sk/ainet/lang/memory/plan/PlannerProfile { public final class sk/ainet/lang/memory/plan/PlannerProfile$Companion { public final fun forDevice (Lsk/ainet/lang/memory/plan/DeviceMemory;)Lsk/ainet/lang/memory/plan/PlannerProfile; + public final fun getBROWSER ()Lsk/ainet/lang/memory/plan/PlannerProfile; public final fun getDESKTOP ()Lsk/ainet/lang/memory/plan/PlannerProfile; public final fun getEDGE ()Lsk/ainet/lang/memory/plan/PlannerProfile; public final fun getMOBILE_2GB ()Lsk/ainet/lang/memory/plan/PlannerProfile; @@ -6430,6 +6433,11 @@ public final class sk/ainet/lang/tensor/ops/FlattenOperation : sk/ainet/lang/ten public fun validateInputs (Ljava/util/List;)Lsk/ainet/lang/tensor/ops/ValidationResult; } +public abstract interface class sk/ainet/lang/tensor/ops/FusedNormOps { + public abstract fun layerNorm (Lsk/ainet/lang/tensor/Tensor;Lsk/ainet/lang/tensor/Tensor;Lsk/ainet/lang/tensor/Tensor;FI)Lsk/ainet/lang/tensor/Tensor; + public abstract fun rmsNorm (Lsk/ainet/lang/tensor/Tensor;Lsk/ainet/lang/tensor/Tensor;FI)Lsk/ainet/lang/tensor/Tensor; +} + public final class sk/ainet/lang/tensor/ops/GeluOperation : sk/ainet/lang/tensor/ops/BaseOperation { public static final field APPROXIMATE Ljava/lang/String; public static final field Companion Lsk/ainet/lang/tensor/ops/GeluOperation$Companion; diff --git a/skainet-lang/skainet-lang-core/src/commonMain/kotlin/sk/ainet/lang/memory/plan/MemoryPlan.kt b/skainet-lang/skainet-lang-core/src/commonMain/kotlin/sk/ainet/lang/memory/plan/MemoryPlan.kt index 7d50cd043..fb1309f44 100644 --- a/skainet-lang/skainet-lang-core/src/commonMain/kotlin/sk/ainet/lang/memory/plan/MemoryPlan.kt +++ b/skainet-lang/skainet-lang-core/src/commonMain/kotlin/sk/ainet/lang/memory/plan/MemoryPlan.kt @@ -128,6 +128,8 @@ public data class Budget(val bytes: Long, val description: String) { public const val RESERVE_ANDROID_JVM: Long = 700L * MiB /** Reserve on Kotlin/Native targets (decision #11). */ public const val RESERVE_NATIVE: Long = 300L * MiB + /** Reserve in a browser tab: the page, the JavaScript heap and the Kotlin/Wasm runtime. */ + public const val RESERVE_BROWSER: Long = 256L * MiB public fun of(bytes: Long): Budget = Budget(bytes, "explicit") public fun available(availableBytes: Long, reserve: Long = RESERVE_ANDROID_JVM): Budget = diff --git a/skainet-lang/skainet-lang-core/src/commonMain/kotlin/sk/ainet/lang/memory/plan/PlannerProfile.kt b/skainet-lang/skainet-lang-core/src/commonMain/kotlin/sk/ainet/lang/memory/plan/PlannerProfile.kt index bc9dec69c..193f49dfa 100644 --- a/skainet-lang/skainet-lang-core/src/commonMain/kotlin/sk/ainet/lang/memory/plan/PlannerProfile.kt +++ b/skainet-lang/skainet-lang-core/src/commonMain/kotlin/sk/ainet/lang/memory/plan/PlannerProfile.kt @@ -159,9 +159,38 @@ public data class PlannerProfile( reserveBytes = Budget.RESERVE_NATIVE, ) - /** The profile a device's own numbers call for: mobile below [MOBILE_RAM_CEILING] of RAM. */ - public fun forDevice(device: DeviceMemory): PlannerProfile = - if (device.totalRamBytes <= MOBILE_RAM_CEILING) MOBILE_2GB else DESKTOP + /** + * A browser tab (Kotlin/Wasm or Kotlin/JS): what the page, the JavaScript heap and the + * Kotlin runtime need is reserved ([Budget.RESERVE_BROWSER]), dense checkpoints are + * quantized to `Q8_0` at load — a Hugging Face FP32 checkpoint is held in a quarter of its + * bytes and served by the packed SIMD128 kernels — and the KV cache auto-quantizes past 80 % + * of the budget. Weights are never mapped: there is no file to map, a model arrives through + * a `Blob` or a fetch. Off-heap allocations ([domainFor]) go to the kernel sidecar's linear + * memory when `WasmSimd.install()` has bound it, else to the heap. + * + * The budget comes from [BROWSER_AVAILABLE_BYTES] unless the caller knows better + * (`navigator.deviceMemory` is coarse and rounds down): wasm32 addresses 4 GB per module + * and a single array holds 2 GB, so 2 GB is what a tab can count on. + */ + public val BROWSER: PlannerProfile = PlannerProfile( + name = "browser", + reserveBytes = Budget.RESERVE_BROWSER, + kvAutoQuantizeAbove = 0.80, + quantizeDenseTo = sk.ainet.lang.tensor.storage.TensorEncoding.Q8_0, + ) + + /** What [BROWSER] plans against when the host reports nothing: 2 GB. */ + public const val BROWSER_AVAILABLE_BYTES: Long = 2L * 1024 * MiB + + /** + * The profile a device's own numbers call for: [BROWSER] on a web host (Kotlin/JS, + * Kotlin/Wasm), otherwise mobile below [MOBILE_RAM_CEILING] of RAM. + */ + public fun forDevice(device: DeviceMemory): PlannerProfile = when { + isWebHost -> BROWSER + device.totalRamBytes <= MOBILE_RAM_CEILING -> MOBILE_2GB + else -> DESKTOP + } /** Devices with at most this much RAM get [MOBILE_2GB]. */ public const val MOBILE_RAM_CEILING: Long = 3L * 1024 * MiB diff --git a/skainet-lang/skainet-lang-core/src/commonMain/kotlin/sk/ainet/lang/memory/plan/WebHost.kt b/skainet-lang/skainet-lang-core/src/commonMain/kotlin/sk/ainet/lang/memory/plan/WebHost.kt new file mode 100644 index 000000000..1eb5b811d --- /dev/null +++ b/skainet-lang/skainet-lang-core/src/commonMain/kotlin/sk/ainet/lang/memory/plan/WebHost.kt @@ -0,0 +1,9 @@ +package sk.ainet.lang.memory.plan + +/** + * Whether this build runs on a web host — Kotlin/JS or Kotlin/Wasm in a browser, a worker or + * Node — where there is no file system to map weights from, a single array holds 2 GB and the + * memory a tab can count on is what [PlannerProfile.BROWSER] assumes. Decides + * [PlannerProfile.forDevice]; a compile-time fact per target, not a runtime probe. + */ +internal expect val isWebHost: Boolean diff --git a/skainet-lang/skainet-lang-core/src/commonMain/kotlin/sk/ainet/lang/nn/normalization/LayerNormalization.kt b/skainet-lang/skainet-lang-core/src/commonMain/kotlin/sk/ainet/lang/nn/normalization/LayerNormalization.kt index 41c79be5d..4cf92e544 100644 --- a/skainet-lang/skainet-lang-core/src/commonMain/kotlin/sk/ainet/lang/nn/normalization/LayerNormalization.kt +++ b/skainet-lang/skainet-lang-core/src/commonMain/kotlin/sk/ainet/lang/nn/normalization/LayerNormalization.kt @@ -86,7 +86,20 @@ public class LayerNormalization( require(normDims.contentEquals(normalizedShape)) { "Input shape ${normDims.contentToString()} doesn't match normalized shape ${normalizedShape.contentToString()}" } - + + // A backend with a one-pass kernel (SKEEP-007 fused ops: the wasm sidecar, the JVM FFM + // provider) serves the FP32 layer in one call through the optional FusedNormOps hook. + // Any other op set — including recording and differentiating ones — never implements + // it and runs the decomposition below unchanged. + if (input.dtype == FP32::class) { + val fused = input.ops as? sk.ainet.lang.tensor.ops.FusedNormOps + if (fused != null) { + val gamma = if (elementwiseAffine) params[0].value else null + val beta = if (elementwiseAffine) params[1].value else null + fused.layerNorm(input, gamma, beta, eps.toFloat(), normalizedShape.size)?.let { return@withForwardHooks it } + } + } + // Compute the numerically-sensitive normalization (mean / variance / // std / divide) in f32, regardless of the model dtype. This is standard // LayerNorm practice — PyTorch and JAX upcast fp16/bf16 LayerNorm to f32 diff --git a/skainet-lang/skainet-lang-core/src/commonMain/kotlin/sk/ainet/lang/tensor/ops/FusedNormOps.kt b/skainet-lang/skainet-lang-core/src/commonMain/kotlin/sk/ainet/lang/tensor/ops/FusedNormOps.kt new file mode 100644 index 000000000..6e8c7add5 --- /dev/null +++ b/skainet-lang/skainet-lang-core/src/commonMain/kotlin/sk/ainet/lang/tensor/ops/FusedNormOps.kt @@ -0,0 +1,40 @@ +package sk.ainet.lang.tensor.ops + +import sk.ainet.lang.tensor.Tensor +import sk.ainet.lang.types.DType + +/** + * Optional fused normalisations a [TensorOps] implementation may offer beside the interface — + * the hook `LayerNormalization` (and an RMS-norm module) consults before running the decomposed + * `mean` / `subtract` / `multiply` / `sqrt` / `divide` chain, so a backend with a one-pass kernel + * (the SIMD128 sidecar on wasmJs, the native FFM provider on the JVM) serves the layer in one + * call and the others keep the chain unchanged. Not part of [TensorOps]: an op set that does not + * implement this interface — a recording or differentiating one — never sees the fused call, and + * the recorded graph and the gradients keep their decomposed form. + * + * Both functions return `null` when the operands are not something the fused path takes (a dtype + * other than FP32, parameters that are not dense, a kernel that is not loaded), and the caller + * continues with the decomposition. The normalised dimensions are the trailing [normalizedRank] + * dimensions of [input], flattened into one row; `gamma` and `beta` have exactly that shape. + */ +public interface FusedNormOps { + /** + * `(input − mean) / sqrt(var + eps) · gamma + beta` over each row of the trailing + * [normalizedRank] dimensions, with the biased variance; [gamma] / [beta] `null` skip the affine. + */ + public fun layerNorm( + input: Tensor, + gamma: Tensor?, + beta: Tensor?, + eps: Float, + normalizedRank: Int, + ): Tensor? + + /** `input / sqrt(mean(input²) + eps) · gamma` over each row of the trailing [normalizedRank] dimensions. */ + public fun rmsNorm( + input: Tensor, + gamma: Tensor?, + eps: Float, + normalizedRank: Int, + ): Tensor? +} diff --git a/skainet-lang/skainet-lang-core/src/commonTest/kotlin/sk/ainet/lang/memory/plan/PlannerProfileTest.kt b/skainet-lang/skainet-lang-core/src/commonTest/kotlin/sk/ainet/lang/memory/plan/PlannerProfileTest.kt index 219636e2e..1793a1ef0 100644 --- a/skainet-lang/skainet-lang-core/src/commonTest/kotlin/sk/ainet/lang/memory/plan/PlannerProfileTest.kt +++ b/skainet-lang/skainet-lang-core/src/commonTest/kotlin/sk/ainet/lang/memory/plan/PlannerProfileTest.kt @@ -179,11 +179,41 @@ class PlannerProfileTest { totalRamBytes = ramMb * mb, availableRamBytes = ramMb * mb / 2, heapMaxBytes = 512 * mb, ) + if (isWebHost) { + // A tab's RAM figure says nothing about what the tab may use: the web profile applies regardless. + assertEquals(PlannerProfile.BROWSER, PlannerProfile.forDevice(device(2048))) + assertEquals(PlannerProfile.BROWSER, PlannerProfile.forDevice(device(16384))) + return + } assertEquals(PlannerProfile.MOBILE_2GB, PlannerProfile.forDevice(device(2048))) assertEquals(PlannerProfile.MOBILE_2GB, PlannerProfile.forDevice(device(3072)), "3 GB phones are still phones") assertEquals(PlannerProfile.DESKTOP, PlannerProfile.forDevice(device(16384))) } + // --- browser ----------------------------------------------------------------------------- + + @Test + fun theBrowserProfileQuantizesDenseWeightsAndNeverMapsThem() { + val p = PlannerProfile.BROWSER + assertEquals("browser", p.name) + assertEquals(Budget.RESERVE_BROWSER, p.reserveBytes) + assertEquals(TensorEncoding.Q8_0, p.quantizeDenseTo, "a Hugging Face FP32 checkpoint is served packed in a tab") + assertFalse(p.weightsMapped, "there is no file to map in a browser") + assertFalse(p.strict, "a tab would rather load slowly than not at all") + assertEquals(0.80, p.kvAutoQuantizeAbove) + assertEquals(PlannerProfile.OFF_HEAP_THRESHOLD, p.offHeapThresholdBytes, "off-heap means the kernel sidecar's memory when installed") + assertEquals(MemoryDomain.HOST_OFFHEAP, p.domainFor(1024L * 1024)) + assertEquals(2048 * mb, PlannerProfile.BROWSER_AVAILABLE_BYTES) + } + + @Test + fun theBrowserProfilePlansAgainstTheTwoGigabyteDefault() { + val profiled = PlannerProfile.BROWSER.plan(input(), PlannerProfile.BROWSER_AVAILABLE_BYTES) + assertEquals((2048 - 256) * mb, profiled.plan.budget!!.bytes) + assertTrue(profiled.render().contains("profile browser"), profiled.render()) + assertFalse(profiled.notes.any { it.contains("mapped") }, profiled.notes.toString()) + } + @Test fun theRenderedPlanNamesTheProfileAndItsDecisions() { val text = PlannerProfile.MOBILE_2GB.plan(input(ctx = 8192), availableBytes = 2048 * mb).render() diff --git a/skainet-lang/skainet-lang-core/src/jsMain/kotlin/sk/ainet/lang/memory/MemoryProbe.js.kt b/skainet-lang/skainet-lang-core/src/jsMain/kotlin/sk/ainet/lang/memory/MemoryProbe.js.kt index 33bf85920..04e7bdb71 100644 --- a/skainet-lang/skainet-lang-core/src/jsMain/kotlin/sk/ainet/lang/memory/MemoryProbe.js.kt +++ b/skainet-lang/skainet-lang-core/src/jsMain/kotlin/sk/ainet/lang/memory/MemoryProbe.js.kt @@ -1,8 +1,18 @@ package sk.ainet.lang.memory -/** No process-level view here: a browser or Wasm host does not expose one. */ +/** + * Kotlin/JS: no process to ask, but Chromium reports the JavaScript heap — where every Kotlin/JS + * object lives — through `performance.memory.usedJSHeapSize`; other engines do not, and then the + * probe says `null`. Page faults are not observable in a browser. + */ public actual object MemoryProbe { - public actual fun rssBytes(): Long? = null + public actual fun rssBytes(): Long? = jsHeapUsedBytes() public actual fun majorFaults(): Long? = null public actual fun minorFaults(): Long? = null + + /** `performance.memory.usedJSHeapSize`, or `null` where the engine does not expose it. */ + public fun jsHeapUsedBytes(): Long? { + val used: Double = js("(typeof performance !== 'undefined' && performance.memory) ? performance.memory.usedJSHeapSize : -1") + return if (used >= 0) used.toLong() else null + } } diff --git a/skainet-lang/skainet-lang-core/src/jsMain/kotlin/sk/ainet/lang/memory/plan/WebHost.web.kt b/skainet-lang/skainet-lang-core/src/jsMain/kotlin/sk/ainet/lang/memory/plan/WebHost.web.kt new file mode 100644 index 000000000..179d7a866 --- /dev/null +++ b/skainet-lang/skainet-lang-core/src/jsMain/kotlin/sk/ainet/lang/memory/plan/WebHost.web.kt @@ -0,0 +1,4 @@ +package sk.ainet.lang.memory.plan + +/** A browser, a worker or Node: the web profile applies. */ +internal actual val isWebHost: Boolean = true diff --git a/skainet-lang/skainet-lang-core/src/jvmAndroidMain/kotlin/sk/ainet/lang/memory/plan/WebHost.other.kt b/skainet-lang/skainet-lang-core/src/jvmAndroidMain/kotlin/sk/ainet/lang/memory/plan/WebHost.other.kt new file mode 100644 index 000000000..4e27ec99b --- /dev/null +++ b/skainet-lang/skainet-lang-core/src/jvmAndroidMain/kotlin/sk/ainet/lang/memory/plan/WebHost.other.kt @@ -0,0 +1,4 @@ +package sk.ainet.lang.memory.plan + +/** A process with its own memory and (where the OS has one) a file system: not a web host. */ +internal actual val isWebHost: Boolean = false diff --git a/skainet-lang/skainet-lang-core/src/nativeMain/kotlin/sk/ainet/lang/memory/plan/WebHost.other.kt b/skainet-lang/skainet-lang-core/src/nativeMain/kotlin/sk/ainet/lang/memory/plan/WebHost.other.kt new file mode 100644 index 000000000..4e27ec99b --- /dev/null +++ b/skainet-lang/skainet-lang-core/src/nativeMain/kotlin/sk/ainet/lang/memory/plan/WebHost.other.kt @@ -0,0 +1,4 @@ +package sk.ainet.lang.memory.plan + +/** A process with its own memory and (where the OS has one) a file system: not a web host. */ +internal actual val isWebHost: Boolean = false diff --git a/skainet-lang/skainet-lang-core/src/wasmJsMain/kotlin/sk/ainet/lang/memory/MemoryProbe.wasmJs.kt b/skainet-lang/skainet-lang-core/src/wasmJsMain/kotlin/sk/ainet/lang/memory/MemoryProbe.wasmJs.kt index 33bf85920..027dc6644 100644 --- a/skainet-lang/skainet-lang-core/src/wasmJsMain/kotlin/sk/ainet/lang/memory/MemoryProbe.wasmJs.kt +++ b/skainet-lang/skainet-lang-core/src/wasmJsMain/kotlin/sk/ainet/lang/memory/MemoryProbe.wasmJs.kt @@ -1,8 +1,50 @@ +@file:OptIn(kotlin.js.ExperimentalWasmJsInterop::class, kotlin.wasm.unsafe.UnsafeWasmMemoryApi::class) + package sk.ainet.lang.memory -/** No process-level view here: a browser or Wasm host does not expose one. */ +import kotlin.wasm.unsafe.wasmMemory + +/** + * Kotlin/Wasm: what the page holds for this module. There is no process to ask — a tab has no + * `/proc` — so the resident set is assembled from the three places SKaiNET's bytes live: + * + * - the JavaScript heap, where Kotlin/Wasm allocates every object (a `FloatArray` is a GC array, + * not linear memory): `performance.memory.usedJSHeapSize`, which Chromium exposes and other + * engines do not — [jsHeapUsedBytes] is `null` there; + * - this module's own linear memory, the staging area for interop copies ([linearMemoryBytes]); + * - the memory an installed [OffHeapAllocator] holds — the kernel sidecar's linear memory, where + * resident weights and activations live — through [OffHeapAllocator.heldBytes] ([offHeapBytes]). + * + * The sum is what a tab is charged for this model. Page faults are not observable in a browser, + * so the fault counters stay `null`, as the common contract allows. + */ public actual object MemoryProbe { - public actual fun rssBytes(): Long? = null + + /** JS heap in use (when the engine reports it) + this module's linear memory + off-heap bytes held. */ + public actual fun rssBytes(): Long? = + linearMemoryBytes() + (jsHeapUsedBytes() ?: 0L) + (offHeapBytes() ?: 0L) + public actual fun majorFaults(): Long? = null public actual fun minorFaults(): Long? = null + + /** Bytes of this Kotlin/Wasm module's linear memory (interop staging; grows, never shrinks). */ + public fun linearMemoryBytes(): Long = jsByteLength(wasmMemory).toLong() + + /** `performance.memory.usedJSHeapSize`, or `null` where the engine does not expose it. */ + public fun jsHeapUsedBytes(): Long? = jsHeapUsed().takeIf { it >= 0 }?.toLong() + + /** `performance.memory.jsHeapSizeLimit` — what the engine will let the heap grow to — or `null`. */ + public fun jsHeapLimitBytes(): Long? = jsHeapLimit().takeIf { it >= 0 }?.toLong() + + /** Bytes held by the installed [PlatformStorage.offHeapAllocator], or `null` without one (or one that cannot say). */ + public fun offHeapBytes(): Long? = PlatformStorage.offHeapAllocator?.heldBytes() } + +@JsFun("(kmem) => kmem.buffer.byteLength") +private external fun jsByteLength(kmem: JsAny): Double + +@JsFun("() => (typeof performance !== 'undefined' && performance.memory) ? performance.memory.usedJSHeapSize : -1") +private external fun jsHeapUsed(): Double + +@JsFun("() => (typeof performance !== 'undefined' && performance.memory) ? performance.memory.jsHeapSizeLimit : -1") +private external fun jsHeapLimit(): Double diff --git a/skainet-lang/skainet-lang-core/src/wasmJsMain/kotlin/sk/ainet/lang/memory/PlatformStorage.web.kt b/skainet-lang/skainet-lang-core/src/wasmJsMain/kotlin/sk/ainet/lang/memory/PlatformStorage.web.kt index 59d28c3d2..efaaf424b 100644 --- a/skainet-lang/skainet-lang-core/src/wasmJsMain/kotlin/sk/ainet/lang/memory/PlatformStorage.web.kt +++ b/skainet-lang/skainet-lang-core/src/wasmJsMain/kotlin/sk/ainet/lang/memory/PlatformStorage.web.kt @@ -5,24 +5,58 @@ import sk.ainet.lang.tensor.TensorId import sk.ainet.lang.tensor.storage.MemoryDomain /** - * JS / Wasm: linear memory only — no off-heap, no mmap (SKEEP-003 §4.8.4). Off-heap requests resolve - * to the heap (the planner records the fallback); files arrive through fetch/range requests into - * heap slabs, so [mapFile] is unsupported here. + * A binding of [MemoryDomain.HOST_OFFHEAP] for Kotlin/Wasm, installed by a backend that owns + * memory outside the Kotlin heap — the SIMD128 kernel sidecar's linear memory + * (`skainet-backend-wasm-simd`, `WasmSimd.install()`), where activations can stay resident between + * kernels instead of crossing the boundary per op. This module cannot depend on that backend, so + * the binding is a hook, exactly like the ops override in `skainet-backend-cpu`. + */ +public fun interface OffHeapAllocator { + /** Allocate [bytes] zeroed bytes off the Kotlin heap, owned by a scope of kind [scope]. */ + public fun allocate(bytes: Long, scope: ScopeKind, origin: TensorId?, sink: TraceSink): Storage + + /** + * Bytes this allocator's memory currently holds — what [MemoryProbe.rssBytes] adds for the + * off-heap domain, since a browser has no process counter that would see them. `null` when the + * backend cannot say. + */ + public fun heldBytes(): Long? = null +} + +/** + * JS / Wasm: the Kotlin heap, plus whatever [offHeapAllocator] a backend installs for + * [MemoryDomain.HOST_OFFHEAP] (SKEEP-003 §4.8.4, SKEEP-007). With no allocator installed off-heap + * requests resolve to the heap (the planner records the fallback); with one they go off the Kotlin + * heap into the backend's memory and [supports] says so, so the planner can place activations + * there. Files arrive through fetch/range requests into heap slabs, so [mapFile] is unsupported. */ public actual object PlatformStorage { - public actual fun supports(domain: MemoryDomain): Boolean = domain == MemoryDomain.HOST_HEAP + /** + * The off-heap binding, or `null` for the heap fallback. Set by `WasmSimd.install()`, cleared + * by its `uninstall()`; a consumer with its own off-heap memory may set it as well. + */ + public var offHeapAllocator: OffHeapAllocator? = null + + public actual fun supports(domain: MemoryDomain): Boolean = + domain == MemoryDomain.HOST_HEAP || (domain == MemoryDomain.HOST_OFFHEAP && offHeapAllocator != null) public actual val supportsMappedFiles: Boolean get() = false public actual fun allocate(bytes: Long, domain: MemoryDomain, scope: ScopeKind, origin: TensorId?, sink: TraceSink): Storage { - require(bytes in 0..Int.MAX_VALUE.toLong()) { "heap storage is limited to 2 GB per array, requested $bytes bytes" } - return when (domain) { + require(bytes >= 0) { "bytes must be >= 0" } + when (domain) { MemoryDomain.MMAP_FILE, MemoryDomain.DEVICE_LOCAL -> throw IllegalArgumentException("$domain is not allocatable on this target") - else -> Storage.Heap.bytes(bytes.toInt(), scope, origin, sink) // HOST_OFFHEAP / PINNED / UNIFIED fall back to the heap + MemoryDomain.HOST_OFFHEAP, MemoryDomain.HOST_PINNED, MemoryDomain.UNIFIED -> { + offHeapAllocator?.let { return it.allocate(bytes, scope, origin, sink) } + } + MemoryDomain.HOST_HEAP -> Unit } + require(bytes <= Int.MAX_VALUE.toLong()) { "heap storage is limited to 2 GB per array, requested $bytes bytes" } + return Storage.Heap.bytes(bytes.toInt(), scope, origin, sink) } public actual fun mapFile(path: String, fileOffset: Long, length: Long, scope: ScopeKind, origin: TensorId?, sink: TraceSink): Storage = throw UnsupportedOperationException("memory-mapped files are not available on this target (use the suspend RandomAccessSource into heap slabs)") - public val info: PlatformStorageInfo = PlatformStorageInfo("heap (fallback)", "none") + public val info: PlatformStorageInfo + get() = PlatformStorageInfo(if (offHeapAllocator != null) "backend linear memory (installed allocator)" else "heap (fallback)", "none") } diff --git a/skainet-lang/skainet-lang-core/src/wasmJsMain/kotlin/sk/ainet/lang/memory/plan/WebHost.web.kt b/skainet-lang/skainet-lang-core/src/wasmJsMain/kotlin/sk/ainet/lang/memory/plan/WebHost.web.kt new file mode 100644 index 000000000..179d7a866 --- /dev/null +++ b/skainet-lang/skainet-lang-core/src/wasmJsMain/kotlin/sk/ainet/lang/memory/plan/WebHost.web.kt @@ -0,0 +1,4 @@ +package sk.ainet.lang.memory.plan + +/** A browser, a worker or Node: the web profile applies. */ +internal actual val isWebHost: Boolean = true diff --git a/skainet-lang/skainet-lang-core/src/wasmWasiMain/kotlin/sk/ainet/lang/memory/plan/WebHost.other.kt b/skainet-lang/skainet-lang-core/src/wasmWasiMain/kotlin/sk/ainet/lang/memory/plan/WebHost.other.kt new file mode 100644 index 000000000..4e27ec99b --- /dev/null +++ b/skainet-lang/skainet-lang-core/src/wasmWasiMain/kotlin/sk/ainet/lang/memory/plan/WebHost.other.kt @@ -0,0 +1,4 @@ +package sk.ainet.lang.memory.plan + +/** A process with its own memory and (where the OS has one) a file system: not a web host. */ +internal actual val isWebHost: Boolean = false