diff --git a/.github/workflows/validate.yml b/.github/workflows/validate.yml index ffbbc8a..8e11df4 100644 --- a/.github/workflows/validate.yml +++ b/.github/workflows/validate.yml @@ -21,3 +21,7 @@ jobs: - name: Run test suite run: python3 -m unittest discover -s tests -v + + - name: Internal register gate (bilingual) + working-directory: scripts + run: python3 test_internal_register_scan.py diff --git a/README.md b/README.md index da9d195..ab3f219 100644 --- a/README.md +++ b/README.md @@ -95,8 +95,12 @@ bằng chứng, `TERM` thuật ngữ, `STANCE` lập trường khoa học, `LOGI Quyết định `pass` yêu cầu mọi chiều đạt ít nhất 4/5 và không có lỗi chặn. Điểm trung bình cao không bù được lỗi đảo nghĩa, mất phủ định, nâng tương quan thành -nhân quả, sai số liệu, hỏng trích dẫn, đổi phạm vi hoặc bịa bằng chứng. Hai -quyết định còn lại là `revise` và `human_review`. Thang điểm và điều kiện chặn: +nhân quả, sai số liệu, hỏng trích dẫn, đổi phạm vi, bịa bằng chứng, hay văn +phong nội bộ/tự thuật. Văn phong đó bị cấm ở cả tiếng Việt và tiếng Anh: câu nói +về bản thảo, dự án, log kiểm tra, đường dẫn máy, placeholder hoặc cuộc hội thoại +soạn thảo không được xuất hiện trong văn bản công bố. Cơ chế kiểm định: +[`references/internal-register-gate.md`](references/internal-register-gate.md). +Hai quyết định còn lại là `revise` và `human_review`. Thang điểm và điều kiện chặn: [`references/quality-rubric.md`](references/quality-rubric.md). ## Kiến trúc kho mã @@ -105,10 +109,10 @@ quyết định còn lại là `revise` và `human_review`. Thang điểm và đ academic-prose/ ├── SKILL.md # Hợp đồng và bộ định tuyến chính ├── agents/openai.yaml # Bộ điều hợp giao diện tùy chọn -├── references/ # 15 tài liệu chuẩn ngôn ngữ, quy trình và phân loại lỗi +├── references/ # Tài liệu chuẩn ngôn ngữ, quy trình, thuật ngữ và phân loại lỗi ├── schemas/ # Lược đồ JSON cho hiện vật trung gian ├── evals/ # Tình huống tổng hợp cho viết, dịch, humanize và usage -├── scripts/ # Trình kiểm định và hai trình mô phỏng +├── scripts/ # Validator, scanner register nội bộ và mô phỏng └── tests/ # Kiểm thử kho mã và lược đồ ``` @@ -117,11 +121,12 @@ academic-prose/ ```bash python3 scripts/validate_skill.py # cổng kiểm định đầy đủ python3 -m unittest discover -s tests -v # kiểm thử kho mã +python3 scripts/test_internal_register_scan.py # cổng văn phong nội bộ, song ngữ python3 scripts/run_usage_simulations.py # riêng mô phỏng usage python3 scripts/run_capability_examples.py # riêng ví dụ năng lực ``` -Trình kiểm định gọi lại cả hai trình mô phỏng, nên một ví dụ hỏng sẽ làm cổng đỏ. +Trình kiểm định gọi lại cả hai trình mô phỏng và bộ test scanner register nội bộ, nên một ví dụ hỏng hoặc một lớp tiếng Anh/tiếng Việt bị sót sẽ làm cổng đỏ. Mỗi phép kiểm tra được đột biến có chủ đích để chứng minh nó biết thất bại; một phép kiểm tra không bao giờ đỏ thì không phải là bằng chứng. diff --git a/SKILL.md b/SKILL.md index 40ed49b..622d8f0 100644 --- a/SKILL.md +++ b/SKILL.md @@ -3,7 +3,7 @@ name: academic-prose description: Think through, structure, write, translate, revise, humanize, and audit academic discourse in Vietnamese and English. Automatically use whenever content serves an academic, scientific, research, higher-education, or scholarly purpose, including prose, manuscripts, reports, slides, teaching content, course materials, speaker notes, assessment items, English-to-Vietnamese and Vietnamese-to-English translation, and removal of AI writing patterns from scholarly text. Do not use to invent evidence, search literature, validate methods, discover citations, or manipulate document layout. license: MIT metadata: - version: "3.0.0" + version: "3.3.0" --- # Academic Prose @@ -54,6 +54,83 @@ For a bilingual document, keep one glossary with paired renderings, one claim ledger, and one stance calibration. A hedge present in one version must be present in the other. Divergence between versions is a `CONS` failure. +## Terminology Localization + +Whether a foreign term is translated, kept, or glossed is a **terminology decision +made once per concept**, recorded in the glossary, and enforced everywhere. It is +not a stylistic preference and not a per-sentence judgment. Author-drafted text +needs this as much as translation does: a writer leaves source-language terms in +place because those are the words the work was done in, so an untranslated term is +usually a decision nobody made. + +Decide by referent and reader, never by how familiar the word looks. Familiarity in +a laboratory, a codebase, or an English-language literature is not evidence that a +word is a proper name. Apply four tests in order — is the term a rigid designator; +does the field already own a rendering; would translating collapse a distinction the +argument needs; can the intended reader index the result — and record the verdict as +`keep_source`, `translate`, `translate_with_gloss`, `keep_with_gloss`, or +`needs_review`. + +Three consequences carry most of the weight: + +1. **The rendering depends on the discipline, not the word.** `baseline` is `mốc cơ + sở` in an evaluation, `kỳ gốc` in econometrics, and `giá trị ban đầu` in a + clinical trial. A single global word list is therefore the wrong instrument; each + glossary entry carries its domain. +2. **A rendering needs an authority, and inference is the lowest tier.** Legal + instrument or national standard, then discipline textbook or approved + dictionary, then attested journal usage, then an official international version, + then the writer's own morphology. Presenting a tier-5 coinage as the field's + settled term is `invented_vietnamese_term`. Where a calque is the settled term + (`rủi ro đạo đức`, `án lệ`, `học sâu`), the settled term wins. +3. **Translating a rigid designator is blocking, and so is collapsing a + distinction.** Model, product, standard, statute, gene, taxon, dataset, unit, and + identifier strings carry retrieval; a Vietnamese frame may surround them + (`hệ số alpha của Cronbach`) but the name survives. In the other direction, + validity/reliability, efficacy/effectiveness, hazard/risk, and + verification/validation each collapse into one Vietnamese word unless the + renderings are deliberately kept apart. + +Both directions are in scope: Vietnamese institutional, legal, and academic-title +terms are system-specific, and mapping them onto a near-equivalent without a gloss +is `institutional_false_friend`. + +Read [Terminology localization policy](references/terminology-localization.md) for +the tests, the discipline table, the load-bearing distinctions, the designator +inventory, the protected zones, and the audit procedure. + +**Do not re-narrow this section to the current paper.** A term that appeared in +one manuscript is locked in that document's glossary. Add a row to the polysemy +table only when the same source word has been observed to split across fields. +Persist a new lesson only as a test, an authority, a distinction, or a +genre/audience rule — never as another NLP (or any single-field) word list. + +## Internal Register (prohibited) + +Publication-facing prose is about the world. Prose about the manuscript, the +project, the drafting conversation, or a local artifact is **prohibited**, not +discouraged. Fluency, completeness of numbers, and a clean compile do not license +it. The control is a gate with four modes — write, check, read, sweep — not a +watched-word list. + +Three tests decide, in order: the semantic subject must be admissible; the +sentence must survive both a work-continues probe and a re-typesetting probe; a +reader holding only the published artifact must be able to verify it. Four +verdicts are licensed (`delete`, `recast`, `relocate`, `license`); **soften is +forbidden**. Recast preserves the scope the internal sentence was carrying, +or the repair becomes an overclaim. + +Run the gate on every `draft`, `develop`, `revise`, `humanize`, `audit`, and +`translate` delivery. One confirmed hit requires a whole-document sweep of both +language versions before reply. See [Internal register gate](references/internal-register-gate.md). + +**A gate is not implemented until its fixture executes.** Documentation, patterns, +and a scanner file establish only a proposed control. Before reporting the gate as +available, run one deliberately dirty fixture and one clean fixture; verify that +the former yields a finding/non-zero exit and the latter yields zero. Treat a +scanner hit as a candidate for the recorded four-verdict review, never as an +automatic rewrite or final scholarly judgment. + ## Writing Workflow For any writing capability, do not begin with polished sentences. Use this order: @@ -61,10 +138,10 @@ For any writing capability, do not begin with polished sentences. Use this order 1. **Rhetorical brief**: define target language, discipline, genre, audience, section, communicative purpose, central question, length, and constraints. 2. **Claim-evidence ledger**: separate supplied facts, author positions, supported inferences, and claims that still need sources. Never render `needs_source` as established fact. 3. **Discourse architecture**: arrange the main claim, supporting claims, evidence, warrants, qualifications, counterpositions, and implications according to the section's function. -4. **Paragraph design**: assign each paragraph one dominant rhetorical job and a controlled sequence of moves. -5. **Draft**: realize the architecture in contemporary academic prose in the target language, with stable terminology and calibrated stance. -6. **Adversarial review**: test whether every empirical statement has support, every connective is licensed, each paragraph advances the argument, and no fluent sentence hides a logical gap. -7. **Revision and gate**: repair evidence, architecture, stance, and coherence before surface polish. +4. **Paragraph design**: assign each paragraph one dominant rhetorical job, a controlled sequence of moves, and an **admissible subject** (object of study, data, method, result, cited claim, or licensed inference). A paragraph whose planned subject is the document, the project, a file, or the drafting conversation is rejected at the plan stage. +5. **Draft**: realize the architecture in contemporary academic prose in the target language, with stable terminology and calibrated stance. Draft from the claim ledger, not from a build log, audit sheet, chat, or task tracker. +6. **Adversarial review**: test whether every empirical statement has support, every connective is licensed, each paragraph advances the argument, and no fluent sentence hides a logical gap. Run the internal-register tests (referent, permanence, outsider verifiability) on Methods, Limitations, Future Work, and every integrity subsection. See [Internal register gate](references/internal-register-gate.md). +7. **Revision and gate**: repair evidence, architecture, stance, register, and coherence before surface polish. An internal-register hit is not polishable: recast the subject or relocate the content. Softening is forbidden. Read [Composition workflow](references/composition-workflow.md), [Argument and evidence](references/argument-and-evidence.md), and [Genre playbooks](references/genre-playbooks.md) for substantial writing. @@ -133,15 +210,16 @@ contract. 1. **Profile**: identify target language, discipline, genre, audience, section function, communicative purpose, and terminology policy. Infer only when evidence is sufficient; otherwise state the assumption. 2. **Map claims**: identify claim ownership, evidence status, actors, actions, negation, modality, causal status, scope, comparisons, quantities, and citation anchors. 3. **Architect**: establish claim dependencies, warrants, qualifications, section moves, and paragraph functions before producing substantial prose. -4. **Lock terminology**: maintain one document-level glossary with preferred, alternative, and prohibited renderings plus context rules. For bilingual work, pair the renderings across languages. +4. **Lock terminology**: maintain one document-level glossary with preferred, alternative, and prohibited renderings plus context rules. Record a localization policy per concept — `keep_source`, `translate`, `translate_with_gloss`, `keep_with_gloss`, or `needs_review` — with its domain and authority tier; see [Terminology localization policy](references/terminology-localization.md). For bilingual work, pair the renderings across languages. 5. **Realize**: write natural prose in the target language from the approved architecture. Preserve formulas, identifiers, citations, quotations, numbers, units, and structured placeholders. -6. **Audit independently**: for new writing, trace consequential statements to the claim ledger and paragraph plan; for translation, paraphrase, or humanizing, additionally compare source and output clause by clause. Fluency never excuses an evidence or logic gap. -7. **Revise and gate**: repair fabrication, claim-evidence mismatch, architecture, stance, and scope before sentence polish. Deliver only when no blocking failure remains; report unresolved evidence and terminology. +6. **Audit independently**: for new writing, trace consequential statements to the claim ledger and paragraph plan; for translation, paraphrase, or humanizing, additionally compare source and output clause by clause. Fluency never excuses an evidence, logic, or register gap. Run the internal-register scan and the manual pass; a scan-only clean report is a partial verification. +7. **Revise and gate**: repair fabrication, claim-evidence mismatch, architecture, stance, scope, and internal register before sentence polish. Deliver only when no blocking failure remains; report unresolved evidence, terminology, and register findings. One confirmed register hit requires a whole-document sweep of both language versions. Read these references as needed: - [Academic Vietnamese standard](references/academic-vietnamese-standard.md) - [Academic English standard](references/academic-english-standard.md) +- [Terminology localization policy](references/terminology-localization.md) - [Composition workflow](references/composition-workflow.md) - [Capability matrix](references/capability-matrix.md) - [Argument and evidence](references/argument-and-evidence.md) @@ -149,6 +227,9 @@ Read these references as needed: - [Deliverable playbooks](references/deliverable-playbooks.md) - [Rhetorical move registry](references/rhetorical-moves.md) - [Writing failure taxonomy](references/writing-failure-taxonomy.md) +- [Internal register gate](references/internal-register-gate.md) +- [Self-narration and config dump](references/self-narration-and-config-dump.md) +- [Artifact register to scientific register](references/artifact-register-to-scientific-register.md) - [AI pattern taxonomy](references/ai-pattern-taxonomy.md) - [Vietnamese AI pattern registry](references/ai-pattern-vietnamese.md) - [Cross-language transfer taxonomy](references/cross-language-transfer-taxonomy.md) @@ -199,6 +280,207 @@ For a short request, give the revised text first and a compact rationale. Do not For substantial new writing, return the clean text first unless the user requests planning only. Then report assumptions, claims still marked `needs_source`, terminology decisions, and material reasoning risks. Do not expose private chain-of-thought; provide concise, inspectable rationale through the rhetorical brief, claim ledger, and section/paragraph plan. +## Evidence-Bound Revision of Empirical Manuscripts + +For revisions that change an abstract, results interpretation, or quantitative framing: + +1. Build a claim--evidence ledger before prose edits. Distinguish canonical release counts, recovered/replayed counts, derived benchmark counts, and calibration estimates; do not merge them rhetorically. +2. Every new number in the abstract must already occur in the body or in a cited, versioned artifact. If a derived number is necessary, add and explain it first in Results, including its uncertainty and whether it identifies individual records or only a sample-level expectation. Never introduce a computed estimate solely in the abstract. +3. Treat labels, manifests, and checksums as part of the evidence boundary. If a manifest label conflicts with its count, repair the label and update every covering digest before reporting the artifact as valid. +4. Preserve distinctions between processing failure, unresolved judgment, semantic rejection, replay recovery, and expert validation. An outage or missing retrieval result must not be narrated as legal/content incorrectness. +5. After edits, rebuild with the document's declared engine rather than assuming `pdflatex`; record the actual page count with `pdfinfo` or PyMuPDF. Compare against a baseline build of the pre-edit source, not only against stale status notes. +6. Run text-level checks for citation/reference resolution, overfull boxes, notation and dash integrity, abstract word limits, and presence of every new quantitative claim. If automated visual review is unavailable, report that limitation and inspect rendered pages directly; do not call the visual gate fully passed. + +## From Project Artifacts to Scientific Prose + +Manuscripts drafted from a working repository tend to inherit the register of +their sources. Build logs, QA checklists, audit spreadsheets, and task trackers +answer *what state is the project in*, whereas a paper answers *what is now +known and under what conditions*. Prose that silently keeps the first register +reads as a progress report and invites desk rejection even when every number is +correct. + +Apply the **progress-report test** to each sentence in Limitations, Future Work, +Methods, and integrity subsections: + +> If the team did more work next week and this sentence changed, but the +> scientific finding did not, the sentence is bookkeeping. + +Recast bookkeeping into the claim it bounds: + +| Artifact register | Scientific register | +| --- | --- | +| the annotator files have empty rating columns | label reliability is unquantified, so no inter-rater agreement is reported | +| the domain field is a placeholder for all records | the dataset is not stratified by domain, so per-domain performance cannot be analyzed | +| automatic scores hit the maximum in several reports | the automatic scorer saturates and loses discriminative power | +| N candidates full-match, M partial, K semantic | the recovery mechanism operates at three anchor-precision levels, and recovered context is content-equivalent rather than verbatim | +| next steps are X, Y, Z | the open problems are X, Y, Z, each stated as a question with a testable design | + +Four further rules: + +1. **Internal QA is a premise, not a result.** Checksum matches, ID-collision + counts, and file-integrity checks belong in a subsection that states *why* + consistency is a precondition for the measurement, not as findings. +2. **Report reconciliation notes as design consequences.** A mismatch between + two counts is worth one sentence explaining the mechanism that produces it, + not a paragraph of arithmetic. +3. **Limitations name the affected inference.** Every limitation must say which + conclusion weakens and by how much scope. A limitation without a consequence + is `ceremonial_limitation`. +4. **Future work states problems, not tasks.** Each item needs the open + question, why current evidence cannot settle it, and a feasible design. + "Obtain expert review" is a task; "establish a human reference set to + estimate convergence between automatic and expert cognitive labels" is a + research problem. + +Keep every operational number that a reader needs to audit the work. The target +is a change of register, not a loss of verifiability. + +### When a figure and the prose disagree, the implementation adjudicates + +A mismatch between a figure label and the body text has two possible causes, and the +plausible one is often wrong. Observed: a pipeline figure named a vision-language model that +the corresponding Methods subsection never mentioned. The natural reading — "the figure +overstates, correct the figure" — was backwards. Reading the pipeline source showed the model +genuinely runs in that stage, so the *prose* was incomplete and the figure was right. + +Before editing either artifact, check the implementation, config, or run log that both are +supposed to describe. State which one you read. Deleting a correct figure label to match an +incomplete paragraph removes a real method component from the record, and it is +unrecoverable once the figure is regenerated. + +Corollary for revision passes: a summary section inherits errors from the body silently. When +a body paragraph explains that two counts have different denominators but the abstract states +them adjacently without that qualification, the abstract is asserting something the body +denies. Re-read every summary claim against the section it compresses, in both language +versions. + +A register leak is never confined to one section. In the measured case it appeared at **eight +sites** — Limitations, two Methods subsections, an integrity subsection, and the Conclusion — +because every place the draft touched an artifact inherited that artifact's unit of analysis. +When the user flags one such sentence, sweep the whole manuscript before replying. + +Two structural consequences that came with the same sweep: **separate ethics/intended-use from +Limitations** (rights scope is a normative statement, not a validity bound), and **order the +tail as** Hạn chế → Đạo đức và phạm vi sử dụng → Hướng nghiên cứu → Kết luận, since the +conclusion cannot precede the open problems it does not resolve. + +Expect a side effect: rewriting an artifact-register Methods paragraph forces a read of the +actual implementation, which is where undocumented stages surface. In the source session it +revealed a VLM stage present in the code and in Figure 1 but absent from the prose — the +earlier draft had planned to "fix the figure" instead. Register cleanup and evidence +verification are one pass, because an artifact-register sentence is usually one nobody traced +back to the system. + +See `references/artifact-register-to-scientific-register.md` for the eight-site defect table, +the future-work rewrite pattern (question → why current evidence cannot settle it → feasible +design), and what must be preserved rather than deleted. + +## Self-Narration and Config Dump + +Two register leaks sit one level below `operational_log_prose`. That failure +imports the register of a build log; these import the register of the author's own +**drafting notes** and of the **source code**. A manuscript can be numerically +correct, inside its page cap, and clean on every automated gate while still +reading as notes-to-self. + +Taxonomy codes: `self_reminder_prose`, `defensive_disclaimer_stack`, +`config_dump_prose`, `generated_artifact_drift`. + +**Self-reminder prose** is the draft instructing itself: `cần nêu rõ`, `cần nêu +thẳng`, `cần được diễn giải thận trọng`, `được nêu dưới đây`, *it should be noted +that*. The outline's imperative survived into the prose. Delete the instruction, +keep its result — but keep the scope it carried, or the repair becomes an overclaim +in the opposite direction. + +Its twin appears wherever the author fears overclaiming: a stack of `chỉ …`, +`không chứng minh …`, `không bảo đảm …` in adjacent sentences. Each is licensed +alone; stacked, the finding vanishes under self-defence. Convert *deny the strong +claim* into *state the bounded claim*, and let one boundary sentence carry the +scope. + +**Config dump prose** copies parameters from a config file with their identifier +form intact: `seed`, `backend`, `qa_id`, bare `vLLM`, `top-$p$ 0,9`, `512 token`. +Reproducibility needs the values, never the variable names. Keep every number, +name the role before the identifier (`thư viện suy luận vLLM`, `giá trị khởi tạo +ngẫu nhiên`), and group parameters by pipeline stage so each stage's sentence says +what it does before how it is configured. Whether the surrounding term is itself +translated is a terminology decision, not a local one; the rendering depends on the +discipline. + +A config dump usually leaves untranslated source-language terms behind it, because +both defects come from drafting in the language the work was done in. Repair them in +one pass, under [Terminology Localization](#terminology-localization) rather than +ad hoc. + +Two pipeline traps travel with this cleanup. An element that survives in the +rendered artifact after removal from source is hardcoded in the exporter — grep +the converter before editing the source a second time. And when the user returns +a hand-edited artifact, diff it against your last build and back-port every +change before regenerating, or the next build erases their work silently. +Structural moves are the ones a naive reading loses: a block relocated from the +end of the paper to the front reads as unrelated churn. + +See [Self-narration and config dump](references/self-narration-and-config-dump.md) +for the full substitution tables and the artifact-regeneration protocol, and +[Internal register gate](references/internal-register-gate.md) for the four-mode +control, the quantitative thresholds, and the scan. + +## Journal-Template Authenticity Gate + +When a user requires an official journal template and forbids creating or substituting one: + +1. Identify the exact target journal and section. A university-wide, legacy, or sibling-section template is not equivalent. +2. Retrieve the template only from a journal-controlled source. Record the official URL, filename, retrieval date, and SHA-256. +3. Compare it with a recent published article from the same target section: page size, columns, margins, body font, front matter, abstracts, and reference presentation. +4. If the target section publishes formatting rules but no downloadable template, do **not** fill a template from another section or recreate one. Deliver the verified manuscript source plus a concrete blocker; ask for the exact official template or authorized access. +5. Once the exact template is available, modify it in place and visually verify the rendered output before calling it submission-ready. +6. **A derived notes file is not template authority.** Before honouring any numeric limit — page cap, figure dimensions, word cap, font size — quote it from the template text or from a measured published article in the same section. A limit that appears only in your own summary file is a `fabricated_constraint` until re-derived. Never delete, shrink, or cut manuscript content to satisfy an unverified limit. + +This gate prevents a plausible-looking but noncompliant submission artifact, and it prevents the opposite failure: mutilating a compliant manuscript to obey a rule that was never imposed. + +## Abstract and Framing Contract + +An abstract is not a results digest, and Limitations are not its closing move. When +an abstract reads as a list of counts that terminates in caveats, the reviewer +cannot state what the paper set out to do — and that absence propagates: the +introduction, the research questions, and the contribution list inherit the same +directionlessness. Diagnose this at the paper level, not the sentence level. + +Required abstract progression, in this order: + +1. **Gap** — one sentence on what the field lacks, scoped to the target venue's readership. +2. **Aim** — one sentence naming what this paper builds, measures, or establishes. +3. **Contributions** — enumerated, each carrying the mechanism that makes it a contribution rather than an activity. "A four-stage pipeline" is an activity; "a four-stage pipeline whose reference-anchoring step makes every item traceable to its source provision" is a contribution. +4. **Principal finding** — including the result with reach beyond this dataset, if there is one. +5. **Validity boundary** — one sentence. Not two, not a closing paragraph. + +Structural checks that travel with it: + +- **Research questions map one-to-one onto contributions.** An RQ that yields no + contribution is a framing defect: it signals the paper is describing its own + process rather than answering something. An RQ of the form "what limitations + must be stated?" is the usual offender — it produces a Limitations section, not + a finding. Recast it as an empirical question whose answer *is* the contribution. +- **Counts in the abstract carry their denominator.** Two figures from different + denominators placed side by side read as a subtraction the author did not intend. + Name the population for each, or drop one. +- **Rank findings by reach.** When one result is method-general and another is + dataset-specific, the abstract and conclusion should say which is which. +- Enforce the venue's word cap on both language versions independently and report + the measured count, never the estimated one. +- **Compress to the cap in text space, not in build space.** Recount with a word + counter after every edit and rebuild only once the count is inside the cap. + Rewriting an abstract to sharpen its aim reliably *adds* words even when the + intent is to cut: one measured sequence ran 302 → 283 → 270 → 264 → 258 → + **266** → 256 → 254 → 244, so two of eight passes moved the wrong way and each + cost a full multi-pass LaTeX + BibTeX cycle for nothing. +- When the source draft is far over cap (478 words against a 250 cap), tell the + user that more than half their text was removed and that the abstract is the + section most needing their review. A silent 50% cut of an author's own words is + not a formatting change. + + ## Learning From Feedback Treat user-approved edits as candidates, not universal rules. Persist a lesson only with its language, discipline, genre, context, reason, positive example, counterexample, and scope. A rejected correction must not be learned. Never learn from unreviewed machine translations as if they were authoritative in either language. diff --git a/references/academic-english-standard.md b/references/academic-english-standard.md index 0ec7925..754c7cf 100644 --- a/references/academic-english-standard.md +++ b/references/academic-english-standard.md @@ -101,4 +101,10 @@ The upstream watched-word lists are English and apply directly. Three additions ## Cross-Language Symmetry +Vietnamese institutional, legal, administrative, and academic-title terms are +system-specific and rarely have an exact English equivalent. Preserve instrument +numbering, and gloss the institutional difference rather than substituting a +near-equivalent; see the reverse-direction section of +[Terminology localization policy](terminology-localization.md). + When a document exists in both languages, keep one glossary with paired renderings, one claim ledger, and one stance calibration. A hedge present in one language must be present in the other. A claim that is `needs_source` in Vietnamese is `needs_source` in English. Divergence between versions is a `CONS` failure, not a stylistic choice. diff --git a/references/academic-vietnamese-standard.md b/references/academic-vietnamese-standard.md index 78b4ab4..193e85f 100644 --- a/references/academic-vietnamese-standard.md +++ b/references/academic-vietnamese-standard.md @@ -55,6 +55,12 @@ that store those values are normally unnecessary. ## Protected Elements +Whether a required foreign term is translated, kept, or glossed is decided by +[Terminology localization policy](terminology-localization.md), recorded once per +concept in the glossary, and enforced across the document. The abstraction rule +above removes implementation tokens with no publication-facing function; it does not +license translating a name that carries the identity of an external object. + When a protected element is substantively required, copy formulas, symbols, values, units, citations, DOI, URLs, code, dataset names, model names, quoted text, and structured placeholders exactly unless the task explicitly changes diff --git a/references/ai-pattern-vietnamese.md b/references/ai-pattern-vietnamese.md index 09c965d..8fec93e 100644 --- a/references/ai-pattern-vietnamese.md +++ b/references/ai-pattern-vietnamese.md @@ -18,8 +18,8 @@ Vietnamese equivalent of §1 and §4. These are the strongest single-word signal **Repair.** Replace the evaluation with the measurement, or delete it. - Before: `Phương pháp đề xuất mang tính đột phá và cho kết quả vượt trội đáng kể so với các phương pháp hiện có.` -- After: `Phương pháp đề xuất đạt F1 0,847, cao hơn 2,3 điểm so với baseline BiLSTM-CRF.` -- When no number exists: `Phương pháp đề xuất cho kết quả cao hơn baseline BiLSTM-CRF.` and mark the magnitude `needs_source`. +- After: `Phương pháp đề xuất đạt F1 0,847, cao hơn 2,3 điểm so với mô hình cơ sở BiLSTM-CRF.` +- When no number exists: `Phương pháp đề xuất cho kết quả cao hơn mô hình cơ sở BiLSTM-CRF.` and mark the magnitude `needs_source`. `đáng kể` deserves separate attention. In statistics it renders *significant* and is licensed only when a test supports it. Used as a vague intensifier it is padding; used for an unsupported statistical claim it is a stance upgrade. Distinguish `khác biệt có ý nghĩa thống kê (p < 0,05)` from `cải thiện đáng kể`. @@ -85,7 +85,7 @@ Also watch: `sự` and `việc` stacked across a clause (`việc thực hiện s Vietnamese equivalent of §11, and a genuine hazard because Vietnamese offers a Sino-Vietnamese and a native word for most concepts. A model alternates them to avoid repetition; a reader infers two constructs. - Before: `Mô hình học sâu đạt độ chính xác cao. Mạng nơ-ron sâu này cho kết quả tốt. Kiến trúc đề xuất vượt các baseline.` -- After: `Mô hình học sâu đề xuất đạt độ chính xác 94%, cao hơn cả ba baseline.` +- After: `Mô hình học sâu đề xuất đạt độ chính xác 94%, cao hơn cả ba mô hình cơ sở.` Repetition of a locked term is correct. Never introduce a synonym to satisfy a repetition rule; that is `terminology_drift`, a major failure. diff --git a/references/artifact-register-to-scientific-register.md b/references/artifact-register-to-scientific-register.md new file mode 100644 index 0000000..c131663 --- /dev/null +++ b/references/artifact-register-to-scientific-register.md @@ -0,0 +1,76 @@ +# From Project Artifacts to Scientific Register + +Measured case: a Vietnamese dataset/benchmark paper (HOEIT-LegalQA → HUJOS-TT) drafted from a +working repository. Every number was correct and every citation resolved; the user still +rejected a whole section with: *"toàn nội dung kiểu này sao lại đưa vào em? bản thảo chủ yếu +trình bày mình đã làm gì thôi chứ? Còn hạn chế và hướng tương lai thì phải về mặt khoa học."* + +The draft had inherited the register of its sources — `quality_gates.json`, an audit summary, +two empty annotator CSVs, a claim ledger. Those answer *what state is the project in*. A paper +answers *what is now known, and under what conditions*. + +## The eight sites found in one sweep + +Not one section. A register leak shows up wherever the draft touched an artifact: + +| # | Section | Artifact-register text | Scientific-register replacement | +| --- | --- | --- | --- | +| 1 | Limitations | "Hai tệp annotator có 240 dòng nhưng các cột đánh giá đều trống" | labels have no human reference annotation, so reliability is unquantified and generator error cannot be separated from source-context error | +| 2 | Limitations | "nhãn lĩnh vực hiện là giá trị placeholder giống nhau cho toàn bộ 14.210 bản ghi" | the dataset is not stratified by domain, so per-domain performance cannot be analysed — a material gap in a domain where difficulty varies sharply | +| 3 | Limitations | "các điểm tự động đạt mức tối đa trong một số báo cáo" | the automatic scorer saturates near the ceiling and loses discriminative power, so it functions as a coarse error filter only | +| 4 | Limitations | "Audit trước đó cho thấy 84 ảnh và 78 ứng viên bảng chỉ là kho ứng viên, chưa có tài sản nào được người xác nhận" | the multimodal subset holds 29 items after adjudication, too few for a statistically meaningful visual evaluation; the paper therefore makes no multimodal-benchmark claim | +| 5 | Methods (context stage) | four per-bucket candidate tallies (1.528 / 469 / 891 / 263) plus "75 chuyển duyệt thủ công và 165 bị loại trong báo cáo đủ điều kiện" | the OCR-noise rate (31,15%) and the three anchor-precision levels of the recovery mechanism, plus the constraint that recovered context is content-equivalent, not verbatim | +| 6 | Methods (release stage) | "Mẫu số này thuộc tệp dữ liệu xử lý 16.158 bản ghi, không phải toàn bộ 14.210" and a 155/154-record reconciliation note | one sentence on the mechanism: both branches derive from the same filtered pool but place the language filter at different points, so the evaluation set is not a subset of the release and the two totals share no denominator | +| 7 | Results ("Kiểm tra tính toàn vẹn") | a QA checklist — 48 Markdown files match 48 PDFs, zero `qa_id` collisions | opens by stating *why* traceability and non-duplication are preconditions for measuring model performance, reports the checks as premises, then states explicitly what they do **not** establish (legal correctness) | +| 8 | Conclusion | "Các bước tiếp theo cần ưu tiên là …" (a task list) | a `Hướng nghiên cứu` section: four open problems, each with the question, why current evidence cannot settle it, and a feasible design | + +## The test that catches all eight + +> If the team did more work next week and this sentence changed, but the scientific finding did +> not, the sentence is bookkeeping. + +Rows 1–4 all fail it: fill the columns, add the tags, calibrate the scorer, qualify the assets +— every sentence evaporates while the paper's contribution is untouched. Rows 5–7 fail a +variant: the numbers are stable but their *unit of analysis* is the team's spreadsheet +(candidates, files, rows) rather than the phenomenon. + +## What NOT to do with the leak + +**Do not delete the numbers.** The target is a change of register, not a loss of +verifiability. Rows 5 and 6 keep their load-bearing quantities (31,15% noise rate; the fact +that two totals have different denominators) and drop only the reconstruction of the audit +trail. A reviewer must still be able to audit the work. + +**Do not let a limitation stand without its consequence.** Each rewritten limitation names the +inference it weakens: which claim is unavailable, over what scope. A limitation with no +consequence is `ceremonial_limitation`, a different failure with the same smell. + +**Separate ethics from limitations.** The original section was titled `Hạn chế và vấn đề đạo +đức` and mixed distribution rights into a list of measurement limits. Rights and intended-use +scope are a normative statement, not a validity bound; they earned their own section. + +**Reorder the tail.** Correct order after the sweep: Hạn chế → Đạo đức và phạm vi sử dụng → +Hướng nghiên cứu → Kết luận. The conclusion closes the paper; it cannot sit before the open +problems it does not resolve. + +## A side effect worth expecting + +Rewriting the Methods paragraph (row 5) forced a read of the actual pipeline code, which +revealed that a VLM stage present in the implementation and in Figure 1 was **missing from the +prose entirely** — the earlier draft had "resolved" the figure/text mismatch by planning to +edit the figure. Register cleanup and evidence verification are the same pass: an artifact- +register sentence is usually a sentence nobody traced back to the system. + +## Future-work rewrite pattern + +Each item needs three parts, in this order: + +1. the open question, phrased as a question about the world rather than about the project; +2. why the present evidence cannot settle it; +3. a design that could, concrete enough to argue with. + +Worked example from the same paper: *"Obtain expert review"* (task) → +*"Establish a human reference set annotated by legal and educational-measurement experts, +measure inter-rater reliability, and estimate convergence between automatic and expert +cognitive labels; this is a precondition before any claim about the dataset's cognitive +stratification counts as verified."* diff --git a/references/capability-matrix.md b/references/capability-matrix.md index 9bf31e8..675befc 100644 --- a/references/capability-matrix.md +++ b/references/capability-matrix.md @@ -35,6 +35,10 @@ primary capability. Declare the target language; it selects the target-language standard and the `LANG` scoring reference. Pair translation with a PDF tool only when extraction or reconstruction is also required. +- Terminology localization is not confined to `translate`. Authored text carries + source-language terms for the same reason translated text does, so `draft`, + `develop`, `revise`, and `audit` all run the localization decision and gate; + see [Terminology localization policy](terminology-localization.md). - The translation adapter explicitly covers both English-to-Vietnamese and Vietnamese-to-English academic transfer. Keep one claim ledger and glossary across the pair; direction changes the target-language realization, not the diff --git a/references/composition-workflow.md b/references/composition-workflow.md index be25419..3775641 100644 --- a/references/composition-workflow.md +++ b/references/composition-workflow.md @@ -4,7 +4,7 @@ Writing is not sentence expansion. It is the controlled conversion of a research ## 1. Rhetorical brief -Record: discipline and subfield; genre; section; readers; communicative purpose; research question; central claim; supplied evidence; required length; terminology policy; citation convention; and unresolved inputs. If the requested text depends on missing evidence, mark the gap instead of filling it plausibly. +Record: discipline and subfield; genre; section; readers; communicative purpose; research question; central claim; supplied evidence; required length; terminology policy; citation convention; and unresolved inputs. The terminology policy includes the localization stance — which source-language terms are kept, translated, or glossed — decided per concept rather than per sentence; see [Terminology localization policy](terminology-localization.md). If the requested text depends on missing evidence, mark the gap instead of filling it plausibly. ## 2. Claim-evidence ledger @@ -27,7 +27,14 @@ Reject an outline if sections merely name subjects, evidence has no claim, or a ## 4. Paragraph design -Assign one dominant function to each paragraph. Select only moves needed for that function: +Assign one dominant function to each paragraph, plus an **admissible subject**: +the object of study, the data, the method, a result, a cited work's claim, or an +inference available to the reader. A paragraph whose planned subject is the +manuscript, its sections, the project, a file, or the drafting conversation is +rejected here rather than repaired later; see +[Internal register gate](internal-register-gate.md). + +Select only moves needed for that function: - establish context or a precise problem; - state a claim; @@ -53,6 +60,7 @@ Review the draft independently: 3. Does every paragraph perform its assigned role and advance the section purpose? 4. Are causal, contrastive, and consequential connectives warranted? 5. Could a reader identify the central claim and its limits without reconstructing the author's intention? +6. Does every source-language term in the prose have a recorded reason to be there, and does every rendering hold one concept throughout? ## 7. Revision order diff --git a/references/cross-language-transfer-taxonomy.md b/references/cross-language-transfer-taxonomy.md index 02e726a..9fdac64 100644 --- a/references/cross-language-transfer-taxonomy.md +++ b/references/cross-language-transfer-taxonomy.md @@ -9,6 +9,15 @@ even in a text written by a human translator; a ceremonial adjective is an AI tell even in a text that was never translated. When both apply to the same span, repair the transfer error first, because it changes meaning. +They are also distinct from a terminology decision. Whether a term should be +translated at all, kept as a designator, or glossed is not an error in transfer; +it is a policy fixed once per concept in the glossary. `terminology_error` and +`terminology_drift` below presuppose that decision has been made. When a source +term appears untranslated in target-language prose, or a designator appears +translated, classify it under +[Terminology localization policy](terminology-localization.md) rather than as a +calque. + ## Direction-neutral errors These occur in both directions. diff --git a/references/domain-profiles.md b/references/domain-profiles.md index a2680b2..e7f3f52 100644 --- a/references/domain-profiles.md +++ b/references/domain-profiles.md @@ -6,6 +6,14 @@ Profiles tune terminology and discourse; they never override the fidelity contra Record discipline, subfield, genre, section, intended readers, locale/institutional convention, and whether English terms should appear at first mention. +The profile selects the terminology rendering, not only the register. The same +source word takes different Vietnamese renderings across the profiles below — +`baseline` is `mốc cơ sở` in an evaluation, `kỳ gốc` in econometrics, and `giá trị +ban đầu` in a clinical trial — so a rendering copied from a neighbouring discipline +is a terminology error even when the Vietnamese is idiomatic. Each glossary entry +carries its domain; see +[Terminology localization policy](terminology-localization.md). + ## Natural Sciences and Engineering - Prefer operational verbs and explicit variables, conditions, mechanisms, and uncertainty. diff --git a/references/internal-register-gate.md b/references/internal-register-gate.md new file mode 100644 index 0000000..168f9f1 --- /dev/null +++ b/references/internal-register-gate.md @@ -0,0 +1,280 @@ +# Internal Register Gate + +A manuscript says what is known about the world. An internal register says what +happened to the manuscript, to the project, or to the author while producing it. +The second is prohibited in publication-facing prose, and it is prohibited +absolutely: it is not a style preference that a good reason can override. + +The prohibition needs a mechanism rather than a warning, because every one of +these defects passes every automated check a project normally runs. The text +compiles, resolves its citations, stays inside its page and word caps, contains +no fabricated number, and reads fluently. A reviewer sees it on the first page. + +This gate operates in four modes, and all four are mandatory: + +| Mode | When | Output | +| --- | --- | --- | +| `write` | before and during drafting | an admissible subject declared per paragraph | +| `check` | before every delivery | scan report + gate verdict | +| `read` | auditing a text you did not write | findings, no silent rewriting | +| `sweep` | after any hit is found anywhere | whole-document, all sections, both language versions | + +## 1. What counts as internal register + +Six classes. They have one cause — the draft was written next to the work +instead of next to the reader — and they need different repairs. + +| Class | Code | The sentence is about | Example tell | +| --- | --- | --- | --- | +| self-address | `self_reminder_prose` | what the author must remember to write | `cần nêu rõ`, `cần thận trọng`, *it should be noted that* | +| document metadiscourse | `document_as_subject` | the paper, its sections, its own order | `phần này sẽ trình bày`, `như đã đề cập ở trên` | +| project bookkeeping | `progress_state_limitation` | the state of the team's work | `chưa kịp`, `sẽ bổ sung sau`, `đang trong quá trình` | +| operational log | `operational_log_prose`, `verification_log_prose` | files, runs, checks, tallies | `đã kiểm tra`, `checksum khớp`, `0 lỗi`, `đã biên dịch lại` | +| internal artifact | `internal_artifact_reference` | a path on the author's machine, a working filename, a ticket, a commit hash, a sheet column | `/Users/…`, `main.tex`, `v9_final`, `commit b4f0a2d`, `#123` | +| repository artifact | `repo_artifact_reference` | a script or data filename inside the project repository | `scripts/anchored_judge.py`, `all_judged_post_repair_v3.jsonl` | +| production residue | `assistant_residue`, `placeholder_residue`, `revision_response_leak` | the conversation or the revision that produced the text | `theo yêu cầu của anh`, `TODO`, `như phản biện đã chỉ ra` | + +`defensive_disclaimer_stack` is the seventh member and the only one with no +lexical signature: each of its sentences is licensed alone, and the defect is +their density. + +## 2. Three tests, applied to every candidate sentence + +The tests are ordered. The first that returns a verdict decides. They are +deliberately mechanical: a criterion an author can argue with is a criterion that +never fires. + +**Test 1 — Referent.** Identify the semantic subject. Admissible subjects are: +the object of study; the data; the method; a result; a cited work's claim; an +inference available to the reader. Inadmissible subjects are: this manuscript, +its sections, the act of writing, the project, the team, a file, a tool run, or +the conversation that produced the text. An inadmissible subject fails unless a +genre slot in §4 licenses it. + +**Test 2 — Permanence.** Two probes: + +- *Work probe.* Suppose the team does more work next week and the scientific + finding is unchanged. Does the sentence change? If yes, it is bookkeeping. +- *Format probe.* Suppose the text is re-typeset, renumbered, or absorbed into a + chapter. Does the sentence break? If yes, it is layout-dependent metadiscourse. + A resolved cross-reference (`Bảng 2 cho thấy`) does not break, because the + number is resolved for the reader. `ở đoạn trên` breaks. + +**Test 3 — Outsider verifiability.** Can a reader holding only the published +artifact parse and check the sentence? If verification requires the repository, +the chat log, the spreadsheet, or the reviewer letter, the sentence is internal +even when every word of it is true. + +## 3. Four verdicts, and the one that is forbidden + +| Verdict | Use when | Operation | +| --- | --- | --- | +| `delete` | the sentence carries no claim once the self-address is removed | remove it; confirm no scope was lost | +| `recast` | it carries a claim in the wrong register | change the **subject** to an admissible one and state the bound it licenses | +| `relocate` | the content is needed for audit but not for argument | move to data card, appendix, repository record, or response letter; keep every number a reader needs | +| `license` | a genre slot in §4 permits it and the quota is unused | keep unchanged, count against the quota | + +The forbidden operation is **soften**. Adding a hedge, a passive, or a +qualification to an internal sentence leaves it internal and now also vague. +`Có lẽ cần nêu rõ rằng…` is worse than the original. The repair changes what the +sentence is about, never how strongly it says it. + +Recast preserves scope. Deleting `cần được diễn giải thận trọng` and stopping +there produces an overclaim in the opposite direction; the caution it carried +must reappear as a stated boundary. This is the same trap as `hedge_loss`. + +Worked repairs: + +| Internal | Recast | +| --- | --- | +| Một giới hạn cần nêu thẳng: nhãn chưa được thẩm định. | Nhãn chưa có đối chiếu của chuyên gia, nên độ tin cậy của nhãn không định lượng được. | +| Phần này sẽ trình bày quy trình xây dựng dữ liệu. | *(delete; the section heading already says it)* | +| Nhóm chưa kịp bổ sung phân tầng theo lĩnh vực. | Dữ liệu không phân tầng theo lĩnh vực, nên không thể phân tích hiệu năng theo từng lĩnh vực. | +| Chúng tôi đã kiểm tra và không phát hiện trùng lặp định danh. | Tập bản ghi có định danh duy nhất, điều kiện cần để các tỷ lệ ở Mục 4 không bị đếm trùng; kiểm tra này không bảo đảm nội dung không trùng lặp. | +| Kết quả trong tệp `ket_qua_v9.csv` cho thấy… | Kết quả cho thấy… *(số liệu giữ nguyên, tên tệp chuyển vào tuyên bố dữ liệu)* | +| Theo góp ý của phản biện, chúng tôi đã bổ sung Bảng 3. | *(delete from the manuscript; the point belongs in the response letter)* | + +## 4. Licensed slots and false positives + +A detector that fires on these destroys correct academic writing. Each is +permitted, and most carry a quota. + +- **One roadmap passage per manuscript**, in the Introduction, naming what each + section establishes. A second roadmap anywhere else is `document_as_subject`. +- **Resolved cross-references** to numbered objects: `Bảng 2`, `Hình 4`, + `Mục 3.1`, `Phụ lục A`. Unlimited. +- **`Trong nghiên cứu này`** as a framing move, when content follows immediately + rather than a promise of content. +- **`Chúng tôi` in Methods**, including for a single author where the venue + permits it. Standard Vietnamese academic usage. +- **Genre-mandated sections**: Limitations, Future Work, Ethics and intended use, + Data availability, Reproducibility, Acknowledgements. The section is required; + its sentences still take an admissible subject. A limitation whose subject is + the inference is correct; one whose subject is the project is not. +- **Vietnamese thesis and grant slots**: `Tính cấp thiết của đề tài`, + `Ý nghĩa khoa học và thực tiễn`, `Kết luận và kiến nghị`. Repair weak content; + never delete the slot. +- **Named artifacts in a data or reproducibility statement**: repository URL, DOI, + dataset name, tool with version. These are `keep_source` designators under + [Terminology localization policy](terminology-localization.md), not internal + leakage. A path on the author's machine is leakage; a public identifier is not. +- **Response-to-reviewers letters and revision notes**: the genre *is* licensed + self-narration. `revision_response_leak` fires only when that register appears + in the manuscript. +- **Teaching material and speaker notes**: `Ở phần sau chúng ta sẽ…` is spoken + discourse structure, not manuscript metadiscourse. Declare the genre. + +## 5. Quantitative thresholds + +A gate needs numbers, otherwise it is advice. + +| Check | Threshold | +| --- | --- | +| self-address occurrences | 0 | +| assistant/chat residue | 0, blocking | +| placeholder residue (`TODO`, `TBD`, `[…]`, `???`) | 0, blocking | +| machine-local path, working filename, ticket, commit hash, or column name | 0, blocking | +| repository script or data filename in body prose | 0 outside a data/reproducibility statement; revision-level, not blocking | +| document-as-subject sentences | ≤ 1 per section, and ≤ 1 roadmap passage per document | +| verification-log sentences in Methods/Results body | 0; permitted in an integrity subsection only when paired with the property the check does **not** establish | +| project-state sentences in Limitations or Future Work | 0 | +| negation-of-claim markers (`chỉ`, `không phải`, `không chứng minh`, `không bảo đảm`, `không hàm ý`) | ≤ 2 within any 3 consecutive sentences | +| divergence between language versions after repair | 0 | + +`SEM` cannot score 4 while a blocking class is present. `VOICE` cannot score 4 +while any self-address remains. `CONS` cannot score 4 while one language version +was cleaned and the other was not. + +## 6. Write mode: prevent instead of detect + +The defect enters at drafting, so the cheapest control is upstream. + +1. In the paragraph plan, record an **admissible subject** for each paragraph + alongside its rhetorical function. A paragraph whose planned subject is the + document, the project, or a file is rejected at the plan stage. +2. Draft from the claim ledger, not from the build log, the audit sheet, or the + task tracker. When an artifact must be consulted, extract the number and close + the artifact before writing the sentence. +3. Never carry an outline's imperative into prose. Outline entries are written as + instructions (`nêu giới hạn về nhãn`); the prose realizes the instruction and + discards it. +4. Write Limitations from the inference backwards: name the conclusion that + weakens, then the mechanism that weakens it. Never from the task list forward. +5. Write Future Work as open questions with a feasible design, never as work the + team owes. + +## 7. Check mode: the scan + +Run [`scripts/internal_register_scan.py`](../scripts/internal_register_scan.py) +on the manuscript source before every delivery. + +```bash +python3 scripts/internal_register_scan.py main.tex --genre manuscript --report report.md +python3 scripts/internal_register_scan.py draft.docx --genre thesis --json findings.json +python3 scripts/internal_register_scan.py notes.md --genre response_letter +``` + +The scanner strips protected zones (math, verbatim, comments, citation and label +arguments, bibliography), segments sentences, applies the class patterns, and +evaluates the density thresholds. Exit code `1` means a blocking class is +present; `2` means revision-level findings only; `0` means clean. + +Two properties of the scanner matter more than its pattern list: + +- **A lexical hit is a candidate, not a verdict.** Every hit is resolved by the + three tests in §2. The scanner reports; the author decides and records the + verdict. +- **Absence of hits is not a pass.** Class `document_as_subject` and + `progress_state_limitation` have paraphrases no pattern catches. The scan + raises the floor; the manual pass over Methods, Limitations, Future Work, and + every integrity subsection is still required. + +Report the two numbers separately: hits the scanner found, and sites the manual +pass found. A scan-only clean report is a partial verification and must be +labelled as one. + +The verification suite is [`scripts/test_internal_register_scan.py`](../scripts/test_internal_register_scan.py) +with fixtures under `scripts/fixtures/`. Each test names the criterion it checks, +so a failure points at a rule here rather than at a regex. Run it after any +pattern change; a detector nobody tests silently decays into either noise or +blindness. + +### Calibration: three false positives that cost real edits + +Every one of these was found by running the scan on four real manuscripts, not on +the fixtures. A detector tuned only on its own examples measures nothing. + +1. **`không chỉ … mà còn` is an intensifier, not a denial.** It produced five + spurious `defensive_disclaimer_stack` hits in one paper's Discussion. A + negation marker followed by a correlative continuation is excluded. +2. **A LaTeX preamble is not prose.** `\newif`, `\InputIfFileExists{…}`, and + package options tripped the artifact patterns. Scan only between + `\begin{document}` and `\end{document}` when the marker is present. +3. **A repository filename is not a machine-local path.** `scripts/foo.py` in a + reproducibility statement is a legitimate public identifier; `/Users/van/…` + never is. Splitting these into two classes moved one manuscript from a false + `block` to a correct revision-level finding, and left the genuine blocker (a + bare commit hash in body prose) intact. + +The general rule these share: **severity follows what the reader can reach.** A +public repository path is reachable and belongs in a data statement; a path on +one machine, a chat turn, and a placeholder are unreachable and block. + +## 8. Read mode: auditing a text you did not write + +Report; do not rewrite. For each finding give the span, the class, the failing +test, and the proposed verdict — and where the verdict is `recast`, propose the +admissible subject rather than a finished sentence, since the author owns the +claim. Distinguish a finding from a preference: if a sentence passes all three +tests and you merely dislike it, it is not a finding. + +When the text is a translation or a bilingual pair, audit both versions. An +internal sentence removed from one version and left in the other is a `CONS` +failure, and the usual direction of the error is that the language the author +drafted in got cleaned and the other did not. + +## 9. Sweep mode: one hit means many + +A register leak is never confined to one sentence. It appears wherever the draft +touched the same source, because each of those places inherited that source's +unit of analysis. One measured sweep found **eight sites** from a single +user-reported sentence: Limitations, two Methods subsections, an integrity +subsection, and the Conclusion. + +When any hit is confirmed, sweep before replying: + +1. Every section, including the abstract, keywords, captions, table notes, + acknowledgements, and data statements. +2. Both language versions. +3. Every summary section against the section it compresses — a summary inherits + the body's register silently. +4. The exporter or template, if the artifact is generated. An internal string + that survives removal from the source is hardcoded downstream; see + `generated_artifact_drift`. + +Expect the sweep to surface substantive findings, not only register ones. +Rewriting an operational Methods paragraph forces a reading of the actual +implementation, which is where undocumented stages and mislabelled figures +appear. Register cleanup and evidence verification are one pass. + +## 10. Codes and severity + +Blocking: `assistant_residue`, `placeholder_residue`, +`internal_artifact_reference`. + +Revision on every occurrence: `self_reminder_prose`, `document_as_subject`, +`progress_state_limitation`, `operational_log_prose`, `verification_log_prose`, +`revision_response_leak`, `defensive_disclaimer_stack`. + +The three blocking classes are blocking because they are unarguable: no reader of +a published article can act on a path from the author's machine, a placeholder, or +a sentence addressed to whoever helped write the draft. The rest are judgments +that a competent author can defend in a specific case, which is why they require +a recorded verdict rather than an automatic edit. + +See [Writing failure taxonomy](writing-failure-taxonomy.md) for the full table, +[Self-narration and config dump](self-narration-and-config-dump.md) for the +self-address and config-dump substitution tables, +[Artifact register to scientific register](artifact-register-to-scientific-register.md) +for the eight-site defect table and the future-work rewrite pattern, and the +internal register gate in [Quality rubric](quality-rubric.md). diff --git a/references/quality-rubric.md b/references/quality-rubric.md index dd7c2f8..d33c04e 100644 --- a/references/quality-rubric.md +++ b/references/quality-rubric.md @@ -37,10 +37,22 @@ Any occurrence forces `revise` or `human_review`, regardless of average score: - `claim_without_status`: a consequential new-writing claim is presented as established although its evidence status is unknown or `needs_source`. - `range_notation_corruption`: an en dash, minus sign, or numeric separator that carries range, eponym, or value meaning is altered, including a hyphen substituted for an en dash. - `required_move_deletion`: a limitation, evidence boundary, alternative explanation, attribution, or template-mandated section is deleted rather than repaired. +- `overtranslation_of_designator`: a rigid designator is translated, so the named object is no longer reachable. +- `distinction_collapse_by_translation`: two source concepts separated in the argument receive one target rendering. +- `assistant_residue`: the drafting conversation is visible in the published text. +- `placeholder_residue`: a drafting token (`TODO`, `TBD`, `[…]`, `???`) survived delivery. +- `internal_artifact_reference`: a local path, working filename, ticket, commit, or sheet column appears in publication-facing prose. -The last two are specific to surface-level rewriting. They exist because a -style pass can destroy content without changing a single proposition: dashes -carry quantitative meaning, and genre-mandated moves carry scope. +`range_notation_corruption` and `required_move_deletion` are surface-rewriting +traps: a style pass can destroy content without changing a proposition, because +dashes carry quantitative meaning and genre-mandated moves carry scope. +`overtranslation_of_designator` and `distinction_collapse_by_translation` are +terminology traps: a localization pass can destroy retrieval or erase a contrast +the argument depends on. +`assistant_residue`, `placeholder_residue`, and `internal_artifact_reference` +are register traps: the text is true and still unusable, because a reader of the +published artifact cannot act on a conversation, a placeholder, or a path on the +author's machine. ## Decision @@ -64,3 +76,50 @@ maps to a blocking failure above. 7. Number format follows the target-language convention in prose and is untouched inside protected tokens. A rewrite that reads better and fails any of these is a regression. + +## Terminology localization gate + +Run when the output is Vietnamese, when either language version contains +source-language terms in body prose, or when a bilingual pair must stay aligned. +Each check maps to a code in +[Terminology localization policy](terminology-localization.md). + +1. Every source-language run in body prose has a recorded policy: `keep_source`, + `translate`, `translate_with_gloss`, `keep_with_gloss`, or `needs_review`. +2. No rigid designator was translated, and no preserved designator lacks a + Vietnamese category noun where a reader outside the toolchain needs one. +3. Every load-bearing distinction the argument uses has two distinct renderings. +4. Each rendering has an authority at tier 1--4, or is reported as `needs_review`; + no tier-5 coinage is asserted as the field's term. +5. One rendering per concept per language version, glossed at first use in the + abstract and at first use in the body, and paired across the two versions. + +`TERM` cannot score 4 while any check fails, and `CONS` cannot score 4 while a +concept alternates renderings. + +## Internal register gate + +Run on every delivery of publication-facing prose, in any language, whether the +text was authored or translated. Each check maps to a class in +[Internal register gate](internal-register-gate.md). + +1. Every candidate sentence flagged by the scan has a recorded verdict: + `delete`, `recast`, `relocate`, or `license`. **`soften` is not a verdict** — + hedging an internal sentence leaves it internal and now also vague. +2. Zero occurrences of the three blocking classes: `assistant_residue`, + `placeholder_residue`, `internal_artifact_reference`. +3. Zero self-address constructions; at most one roadmap passage per document and + at most one document-as-subject sentence per section. +4. No sentence in Limitations or Future Work has the project as its subject; each + names the inference it bounds or the question it opens. +5. No verification confirmation stands as a result: each states the property the + check establishes and the property it does not. +6. At most two claim-negation markers within any three consecutive sentences. +7. Both language versions were cleaned, and the manual pass over Methods, + Limitations, Future Work, and integrity subsections is reported separately + from the scan. + +`SEM` cannot score 4 while a blocking class is present. `VOICE` cannot score 4 +while any self-address remains. `CONS` cannot score 4 while one language version +was cleaned and the other was not. A scan-only clean report is a partial +verification and must be labelled as one. diff --git a/references/self-narration-and-config-dump.md b/references/self-narration-and-config-dump.md new file mode 100644 index 0000000..705f1e9 --- /dev/null +++ b/references/self-narration-and-config-dump.md @@ -0,0 +1,130 @@ +# Self-Narration and Config Dump + +Two register leaks distinct from `operational_log_prose`. That failure imports the +register of a **build log**. These two import the register of the author's own +**drafting notes** and of the **source code**. + +Measured case: the same Vietnamese dataset paper (HOEIT-LegalQA → HUJOS-TT). +Every number was correct, the manuscript hit its page cap, both language versions +passed the abstract word cap, and the visual gate returned zero findings. The user +still rejected the prose with: *"toàn là mấy câu tự thuật, tự nhắc nhở..."* + +Correctness is not the axis these fail on. They fail because the sentence is about +**writing the paper** rather than about the world. + +## 1. `self_reminder_prose` + +The draft tells itself what to do instead of doing it. The tell is a modal of +obligation aimed at the text: `cần nêu rõ`, `cần nêu thẳng`, `cần được diễn giải +thận trọng`, `cần thận trọng vì`, `được nêu dưới đây`, `cần nêu cùng với`, +`nên được đọc là`. In English: *it should be noted that*, *care must be taken +when interpreting*, *the following must be stated*. + +The instruction survived from the outline into the prose. A reader does not need +the instruction; they need its result. + +| Self-reminder | Direct statement | +| --- | --- | +| Cơ chế phục hồi này tạo ra một ràng buộc **cần nêu rõ** về bản chất dữ liệu | Cơ chế phục hồi **đặt ra** một ràng buộc về bản chất dữ liệu | +| Một giới hạn về tái lập **cần nêu thẳng**: … | Khả năng tái lập **có một giới hạn ở bước sinh câu hỏi**: … | +| Các lựa chọn cố định của quy trình **được nêu dưới đây** để nghiên cứu khác lặp lại được | **Toàn bộ quy trình sử dụng các tham số cố định sau.** | +| Ba quyết định phương pháp luận **cần nêu cùng với** các tham số trên | **Các tham số trên xuất phát từ** ba quyết định phương pháp luận | +| Qwen2.5-7B và Gemma-2-9B **cần được diễn giải thận trọng** vì cùng họ với … | Qwen2.5-7B và Gemma-2-9B **cùng họ với** các mô hình xây dựng dữ liệu … Hai kết quả này **vì vậy mang giá trị chẩn đoán** | +| Đặt kết quả cạnh các bộ đối sánh ở mục 2 **cần thận trọng** vì khác ngôn ngữ … | So sánh với các bộ đối sánh ở mục 2 **chỉ có giá trị định tính, do** khác biệt về ngôn ngữ … | +| **Mọi phát biểu** về "mức độ nhận thức" **nên được đọc là** nhãn thao tác | **Các phát biểu** về "mức độ nhận thức" **vì vậy là** nhãn thao tác | + +### Its defensive twin + +A related tell clusters where the author fears overclaiming: a chain of +`chỉ …`, `không phải là …`, `không chứng minh …`, `không nên được …`, `không bảo +đảm …` in adjacent sentences. Each is individually licensed; stacked, they read as +self-defence and the finding disappears under the disclaimers. + +Convert *deny the strong claim* into *state the bounded claim*: + +- `Các kiểm tra này **chỉ** thiết lập X. **Chúng không chứng minh** Y.` → + `Các kiểm tra này thiết lập X, **nhưng không thiết lập** Y, vì Y đòi hỏi …` +- `Kết quả cho thấy Z. **Chúng không chứng minh** nhân quả, **cũng không chứng + minh** đã thẩm định.` → `Kết quả cho thấy Z. **Hiệu ứng này giới hạn trong + thiết lập đã mô tả và không hàm ý** đã thẩm định.` + +One boundary sentence carrying the scope beats three sentences denying claims +nobody made. This is the sentence-level analogue of the abstract rule in +*Abstract and Framing Contract*: one validity boundary, not a closing paragraph. + +**Do not delete the hedge.** The scope must survive; only the self-address goes. +Deleting `không chứng minh nhân quả` outright is a `causal_overclaim` in the +opposite direction. + +## 2. `config_dump_prose` + +Parameters copied from a config file or CLI invocation, carrying their identifier +form into prose. Reproducibility requires the *values*; it never required the +variable names. + +| Config form | Prose form | +| --- | --- | +| `\texttt{marker-pdf}` | công cụ marker-pdf | +| `Qwen2.5-7B-Instruct-AWQ` | Qwen2.5-7B-Instruct **ở dạng lượng tử hóa** AWQ | +| `qua vLLM` | **triển khai bằng thư viện suy luận** vLLM | +| `backend sinh không cho cố định seed` | **công cụ suy luận không cho phép cố định giá trị khởi tạo ngẫu nhiên** | +| `không thay đổi tập \texttt{qa\_id}` | không thay đổi **tập định danh bản ghi** | +| `top-$p$ 0,9` | **ngưỡng lấy mẫu tích lũy** top-$p$ 0,9 | +| `hình phạt lặp 1,15` | **hệ số phạt lặp** 1,15 | +| `giới hạn 512 token` | **độ dài đầu ra tối đa 512 đơn vị từ (token)** | +| `lượng tử hóa 4 bit, nhiệt độ 0,3` | **ở chế độ lượng tử hóa 4 bit, nhiệt độ lấy mẫu 0,3** | + +Rules: + +1. **Keep every number.** Registers change; values do not. The rewritten paragraph + in the measured case retained all eleven parameters. +2. **Name the role before the identifier.** `thư viện suy luận vLLM` tells a reader + outside the toolchain what vLLM is; bare `vLLM` does not. +3. **A bare parameter list is not a Methods paragraph.** Group by pipeline stage + and let each stage's sentence say what the stage does, then how it is + configured. +4. **Keep the English technical term in parentheses** on first Vietnamese use when + the term is the field's standard (`đơn vị từ (token)`). Do not calque it away. +5. **A config dump leaves untranslated source-language terms next to it.** Both + defects have one cause — the draft was written in the language the work was done + in — so repair them in the same pass. Which terms are translated, kept, or + glossed is not decided case by case here: + [Terminology localization policy](terminology-localization.md) owns the four + tests, the per-discipline renderings, the load-bearing distinctions, and the + designator inventory. The config-dump repair is that policy's `keep_source` + half: a Vietnamese category noun in front of a preserved name + (`thư viện suy luận vLLM`). + +## 3. The generated-artifact trap + +Two pipeline failures surfaced while fixing the above. Both waste a full rebuild +cycle if unknown. + +**Hardcoded strings in the exporter.** A line was removed from the LaTeX source; +the DOCX still printed it. Cause: the LaTeX→DOCX converter emitted the string +literally, so the source was never its authority. When an element persists in the +rendered artifact after removal from the source, grep the exporter before editing +the source again. + +**Regeneration silently reverts hand edits.** The user hand-edited the delivered +DOCX: moved the English block from the end of the paper to the front, switched the +English author names to Western order, dropped the ngày nhận/chấp nhận line and +the whole author-information sheet, and changed `mô hình ngôn ngữ mở` to `mô hình +ngôn ngữ lớn`. None of that existed in the source. The next regeneration would +have erased all of it. + +Protocol when the user returns an edited artifact: + +1. Convert both their version and your last generated version to a diffable text + form (`pandoc --track-changes=all` for DOCX). +2. Diff and enumerate every change, including structural moves and reorderings — + not just word substitutions. +3. Back-port each change into the source, then rebuild and confirm the count of + changes survived. +4. Treat their edits as authoritative on structure and terminology, and re-verify + only the constraints their edits could break (page count, word caps, float + placement). + +The structural moves are the ones a naive diff loses: a block relocated from 90% +to 5% of the document shows up as one large deletion plus one large insertion, and +is easy to read as unrelated churn. diff --git a/references/terminology-localization.md b/references/terminology-localization.md new file mode 100644 index 0000000..0216eb8 --- /dev/null +++ b/references/terminology-localization.md @@ -0,0 +1,398 @@ +# Terminology Localization Policy + +Whether a foreign-language term is translated, kept, or glossed. This applies to +originally-authored text as much as to translation: an author drafting in +Vietnamese leaves English terms in for the same reason a translator does — they +are the words the work was done in. A term left untranslated is therefore not a +transfer error; it is a terminology decision that was never made. + +The decision is about the **referent and the reader**, never about how familiar +the word looks. Familiarity in a laboratory, a codebase, or an English-language +literature is not evidence that a word is a proper name. + +## 1. Four tests, in order + +Apply in sequence; the first test that returns a verdict decides. + +1. **Designator test.** Does the term name a unique external object whose identity + is carried by that string — a model, product, library, standard number, statute + short title, dataset, gene or taxon symbol, eponym, unit, code identifier, or a + quoted interface label? If yes: `keep_source`. Altering the string breaks + retrieval and citation. +2. **Established-rendering test.** Does the field already own a Vietnamese + rendering in an authoritative register (§3)? If yes: `translate`, and use that + rendering even if colleagues speak the English form. +3. **Distinction test.** Would translating collapse two source concepts that the + argument separates (§5)? If yes: `keep_with_gloss` or `translate_with_gloss`, + with the two renderings kept lexically distinct. +4. **Reader test.** Would a specialist fail to index the Vietnamese form, or a + non-specialist fail to read the English one? Use `translate_with_gloss` when the + term is the paper's own object of study or the field's retrieval key; otherwise + `translate`. + +If no test resolves, the entry is `needs_review`: write the source term with a +descriptive gloss and flag it. Do not coin silently. + +## 2. Four policies + +| Policy | Form in running text | Use when | +| --- | --- | --- | +| `keep_source` | the exact source string, optionally preceded by a Vietnamese category noun (`thư viện suy luận vLLM`, `chuẩn ISO 9001`) | rigid designators | +| `translate` | one Vietnamese rendering, no parenthesis | the field owns the concept | +| `translate_with_gloss` | Vietnamese + `(source form)` at first use | established rendering, but specialists index on the source term | +| `keep_with_gloss` | source term + Vietnamese descriptive gloss at first use | no stable Vietnamese rendering exists, or translation would collapse a needed distinction | + +A category noun in front of a preserved designator is the usual repair for prose +that reads as a config dump: it tells a reader outside the toolchain what kind of +object the name denotes without translating the name. + +## 3. Authority order for a Vietnamese rendering + +1. Legal instrument or national standard where the term is legally defined (luật, + nghị định, TCVN). +2. Discipline textbook, council-approved specialist dictionary, or ministry + curriculum. +3. Attested usage across Vietnamese journal articles in the same subfield. +4. An official Vietnamese version issued by an international body (WHO, ISO, UN). +5. The writer's own morphological inference. + +Tier 5 alone never licenses an asserted standard term. This skill does not search +literature; when no authority is at hand, record `needs_review` and keep the source +term with a gloss. A coined rendering presented as the field's established term is +`invented_vietnamese_term`. + +**Usage precedes etymology.** Where a calque is the field's settled term — `rủi ro +đạo đức` for *moral hazard*, `án lệ` for *precedent*, `học sâu` for *deep +learning* — the settled term wins over a more literal or more elegant coinage. +Terminological correctness here is a fact about the discourse community, not about +translation quality. + +## 4. The same source word takes different renderings by discipline + +This is why a global word list is the wrong instrument. + +| Source term | Discipline | Rendering | +| --- | --- | --- | +| `baseline` | evaluation, experiment | mốc cơ sở; thiết lập cơ sở; mô hình cơ sở | +| `baseline` | econometrics, finance | kỳ gốc | +| `baseline` | clinical research | giá trị ban đầu, trước can thiệp | +| `control` | experiment | đối chứng; nhóm chứng | +| `control` | management, policy | kiểm soát | +| `control` | engineering | điều khiển | +| `treatment` | clinical research | điều trị; can thiệp | +| `treatment` | statistics | mức xử lý | +| `treatment` | materials, chemistry | xử lý mẫu | +| `trial` | clinical research | thử nghiệm | +| `trial` | law | phiên tòa | +| `case` | medicine | ca bệnh | +| `case` | law | vụ việc; án | +| `case` | research methods | trường hợp | +| `subject` | empirical research | đối tượng nghiên cứu | +| `subject` | education, curriculum | môn học | +| `subject` | grammar, logic | chủ ngữ; chủ thể | +| `culture` | microbiology | nuôi cấy | +| `culture` | humanities, social science | văn hóa | +| `stress` | mechanics | ứng suất | +| `stress` | psychology, health | căng thẳng | +| `stress` | linguistics | trọng âm | +| `charge` | physics, chemistry | điện tích | +| `charge` | law | cáo buộc | +| `charge` | economics | phí | +| `resistance` | physics | điện trở; lực cản | +| `resistance` | medicine | kháng thuốc; sức đề kháng | +| `resistance` | social science | kháng cự | +| `cell` | biology | tế bào | +| `cell` | energy | pin | +| `solution` | chemistry | dung dịch | +| `solution` | policy, engineering | giải pháp | +| `volume` | physics, chemistry | thể tích | +| `volume` | publishing | tập | +| `mass` | physics | khối lượng | +| `mass` | social science | quần chúng | +| `weight` | physics | trọng lượng | +| `weight` | statistics, psychometrics | trọng số | +| `field` | physics | trường | +| `field` | academic writing | lĩnh vực | +| `order` | mathematics | bậc | +| `order` | law, administration | lệnh; quyết định | +| `order` | biological taxonomy | bộ | +| `degree` | mathematics, thermodynamics | bậc; độ | +| `degree` | education | bằng cấp | +| `mean` | statistics | trung bình | +| `population` | statistics | tổng thể | +| `population` | demography, ecology | dân số; quần thể | +| `incidence` | epidemiology | tỷ lệ mới mắc | +| `incidence` | optics, geometry | góc tới | +| `exposure` | epidemiology, toxicology | phơi nhiễm | +| `exposure` | radiology, photography | phơi sáng | +| `exposure` | finance | mức rủi ro đang gánh | +| `survey` | social science | khảo sát | +| `survey` | geodesy, civil engineering | trắc địa | +| `instrument` | measurement | công cụ đo | +| `instrument` | law | văn bản pháp lý | +| `protocol` | computing | giao thức | +| `protocol` | laboratory, clinical | quy trình | +| `protocol` | diplomacy | nghị định thư | +| `code` | computing | mã | +| `code` | law | bộ luật | +| `act` | law | đạo luật | +| `act` | sociology, ethics | hành vi | +| `sentence` | law | bản án | +| `sentence` | linguistics | câu | +| `party` | law | bên | +| `party` | political science | đảng | +| `evidence` | science | bằng chứng | +| `evidence` | law | chứng cứ | +| `argument` | academic prose | lập luận | +| `argument` | mathematics, computing | đối số | +| `function` | mathematics | hàm | +| `function` | biology, sociology | chức năng | +| `class` | computing, statistics | lớp | +| `class` | sociology | giai cấp; tầng lớp | +| `interest` | finance | lãi suất | +| `interest` | political science, ethics | lợi ích | +| `capital` | economics | vốn | +| `capital` | geography | thủ đô | +| `demand` | economics | cầu | +| `demand` | law | yêu cầu | +| `equity` | finance | vốn chủ sở hữu | +| `equity` | ethics, education, policy | công bằng | +| `inflation` | economics | lạm phát | +| `inflation` | psychometrics | phóng đại hệ số | +| `efficient` | economics | hiệu quả phân bổ | +| `efficient` | engineering | hiệu suất | +| `consistent` | statistics, of an estimator | vững | +| `consistent` | argument | nhất quán | +| `compliance` | policy, medicine | tuân thủ | +| `compliance` | mechanics | độ biến dạng | +| `regulation` | law, policy | quy định; điều tiết | +| `regulation` | physiology | điều hòa | +| `implementation` | computing, policy | triển khai | +| `implementation` | law | thực thi | +| `review` | research writing | tổng quan | +| `review` | peer evaluation | phản biện | +| `assessment` | education | đánh giá người học | +| `evaluation` | programme, policy | đánh giá chương trình | +| `performance` | computing, engineering | hiệu năng | +| `performance` | education, sport | thành tích | +| `performance` | arts | biểu diễn | +| `agency` | social theory | năng lực hành động | +| `agency` | administration | cơ quan | +| `construct` | psychometrics, social science | cấu niệm | +| `factor` | statistics | nhân tố; hệ số | +| `factor` | ordinary analysis | yếu tố | +| `index` | statistics, economics | chỉ số | +| `index` | publishing | mục lục | +| `rate` | epidemiology, demography | tỷ lệ | +| `rate` | physics | tốc độ | +| `rate` | finance | lãi suất | +| `panel` | econometrics, survey | dữ liệu bảng | +| `panel` | governance | hội đồng | +| `wave` | survey research | đợt | +| `wave` | physics | sóng | +| `pipeline` | computing | quy trình | +| `pipeline` | petroleum, civil engineering | đường ống | +| `outcome` | clinical trial | kết cục | +| `outcome` | software, engineering | đầu ra | +| `outcome` | education, policy | kết quả đầu ra | +| `bias` | statistics | sai lệch | +| `bias` | social science, ethics | thiên lệch; định kiến | +| `power` | statistics | lực kiểm định | +| `power` | electrical engineering | công suất | +| `power` | political science | quyền lực | +| `validation` | measurement, psychometrics | thẩm định giá trị | +| `validation` | software, simulation | kiểm định tính đúng đắn | +| `context` | linguistics, NLP | ngữ cảnh | +| `context` | policy, qualitative research | bối cảnh | +| `benchmark` | evaluation | bộ dữ liệu đối sánh; bài đánh giá đối sánh | +| `benchmark` | surveying, finance | mốc chuẩn; chỉ số tham chiếu | +| `significant` | statistics | có ý nghĩa thống kê | +| `significant` | ordinary academic prose | quan trọng, never as a silent stand-in for a test | + +Each glossary entry therefore carries its `domain`. One document may legitimately +use two renderings of the same source word when two disciplines are in play, +provided each rendering is locked to one concept. Copying a rendering across +disciplines is a terminology error even when the Vietnamese is idiomatic. + +The table is a **polysemy diagnostic**, not a dictionary and not an allowlist. A +term absent from it still runs the four tests, and a term present in it still +takes the rendering its own discipline uses. Add a row only when a source word has +been observed to split across fields; do not grow it into a bilingual glossary of +one field. + +## 5. Load-bearing distinctions + +The strongest argument for keeping or glossing a source term is that the target +language tends to absorb two concepts into one word. + +| Source pair | Must stay distinct | Collapse risk | +| --- | --- | --- | +| validity / reliability | giá trị (validity) / độ tin cậy (reliability) | both drift to `độ tin cậy` | +| accuracy / precision | two distinct renderings, each glossed at first use | contested in Vietnamese; do not assume one | +| efficacy / effectiveness | hiệu lực (efficacy) / hiệu quả thực hành (effectiveness) | both drift to `hiệu quả` | +| hazard / risk | nguy cơ tức thời (hazard) / nguy cơ (risk) | both drift to `nguy cơ` | +| assessment / evaluation | đánh giá người học / đánh giá chương trình | both drift to `đánh giá` | +| verification / validation | kiểm chứng / thẩm định | both drift to `kiểm định` | +| equity / equality | công bằng / bình đẳng | both drift to `công bằng` | +| statistical significance / practical importance | có ý nghĩa thống kê / quan trọng về thực hành | both drift to `đáng kể` | +| token / word | đơn vị từ (token) / từ | both drift to `từ` | + +When two source terms are separated in the argument, their renderings must be +separated too, and each carries its source form at first use. Collapsing them is +`distinction_collapse_by_translation`, a blocking failure, because it changes what +the text claims rather than how it reads. + +## 6. Rigid designators + +Never translate: model, architecture, product, and library names; standard and +instrument numbers (`ISO 9001`, `TCVN 5687`, `IEEE 802.11`, `ICD-10`); statute +short titles and provision numbers; gene, protein, and taxon symbols and binomials +(`TP53`, `SARS-CoV-2`, `Escherichia coli`); eponyms; dataset and index proper names +(`SQuAD`, `PISA`, `TIMSS`, `VN-Index`); quoted interface labels; code identifiers; +units and symbols. + +Two half-cases recur: + +- **An eponym takes a Vietnamese frame around a preserved name**: `hệ số alpha của + Cronbach`, `hiệu chỉnh Bonferroni`, `thang đo Likert`. The name survives; the + category noun is Vietnamese. +- **A word that is both a common noun and a designator splits by referent**: + `transformer` the electrical device is `máy biến áp`; `Transformer` the + architecture keeps its name. Decide by what the sentence refers to, not by + spelling. + +Translating a designator is `overtranslation_of_designator`, blocking for the same +reason as a corrupted citation: the reader can no longer reach the object. + +## 7. Where this policy does not reach + +- verbatim quotations, reference titles, and bibliographic metadata; +- figure and table content reproduced from a source; +- code, configuration keys, schema fields, CLI flags, and file names; +- interface labels that are themselves the object of study; +- the source-language version of a bilingual document; +- a venue instruction that mandates a particular form. + +Abstraction is not concealment: a methodological choice that a reader needs in +order to judge the work stays in the text, in disciplinary language. See the +publication-facing abstraction rule in +[Academic Vietnamese standard](academic-vietnamese-standard.md). + +## 8. Consistency and bilingual pairing + +- One rendering per concept per language version. Alternating between the source + term and its translation is `mixed_rendering`, a `CONS` failure. +- Gloss at first use in the abstract and at first use in the body. Not in every + section, not in table headers, not in figure captions where space is the + constraint. +- In a bilingual document, every translated term maps to the exact source term used + in the other version. A concept glossed in one version must be recoverable in the + other. +- Abbreviations: expand once per language version, then use the abbreviation. Do + not translate an abbreviation's letters (`GDP`, not a re-lettered Vietnamese + acronym) unless the field has an established Vietnamese abbreviation. + +## 9. Reverse direction: Vietnamese to English + +Vietnamese institutional, legal, administrative, and academic-title terms are +system-specific and rarely have an exact foreign equivalent. + +- Instruments: `Nghị định 15/2018/NĐ-CP` → *Decree 15/2018/ND-CP*; `Thông tư` → + *Circular*; `Bộ luật` → *Code* against `Luật` → *Law*. Keep the numbering intact. +- Titles: `phó giáo sư` → *Associate Professor*, with a note that it is a + state-conferred title rather than an appointment; `nghiên cứu sinh` → + *doctoral candidate*, not *research student*. +- Administrative units: `xã`, `phường`, `huyện`, `thị xã` → commune, ward, + district, town, with the Vietnamese in parentheses at first use when the argument + depends on the administrative level. + +Mapping any of these onto a near-equivalent without a gloss is +`institutional_false_friend`. Inflating a Sino-Vietnamese term into Latinate +English is `sino_vietnamese_overtranslation` in +[Cross-language transfer taxonomy](cross-language-transfer-taxonomy.md). + +## 10. Genre and audience + +The four tests decide *whether* a term is translated. Genre decides *how much +support the reader needs*. + +| Genre | Localization stance | +| --- | --- | +| journal article for specialists | translate generic terms; keep designators; gloss once at first use | +| Vietnamese thesis or ministry report | prefer established Vietnamese; gloss designators; follow the template's own terminology where it names one | +| conference paper in computing or physics | more designators in running text are expected; still translate generic nouns | +| teaching material, slides, assessment items | Vietnamese first with the source form in parentheses, because the learner must acquire both | +| clinical guideline or public-health text | official Vietnamese of the issuing body outranks journal usage | +| legal commentary | never paraphrase an operative term; keep the statutory wording and gloss the analysis | +| grant proposal | institutional Vietnamese for agencies, programmes, and degrees; do not invent English calques of them | +| abstract | one gloss per key term, since this is usually the first occurrence; do not spend the word cap on a second gloss | +| figure and table headers | the locked short form; no new parenthesis if the body already glossed it | + +A teaching text that keeps source-language generics in order to look technical +fails the reader test. A specialist article that translates a retrieval name fails +the designator test. One glossary can serve both genres with different +`gloss_at_first_use` values; it cannot serve them with different `preferred` +renderings. + +## 11. Loanword, calque, and Sino-Vietnamese + +Three historically distinct routes produce a Vietnamese scientific term. They are +not ranked by elegance. + +1. **Settled calque or Sino-Vietnamese term.** `học sâu`, `rủi ro đạo đức`, + `án lệ`, `hồi quy`, `phân tầng`, `nội suy`, `định lượng`. Use it. Replacing it + with a purer native paraphrase, or reverting to the source term, are both + errors. +2. **Settled loan.** `internet`, `vitamin`, `laser`, `virus`, `protein`, `radar`. + Do not force these into Vietnamese morphology to appear more formal. +3. **Unsettled term.** No tier-1--4 authority exists. Use `keep_with_gloss` or + `needs_review`; do not promote a one-off coinage. + +A source-language insertion in otherwise Vietnamese prose (`các baseline`, `một +pipeline hoàn chỉnh`) is not a loan in this sense; it is an undecided generic. In +the other direction, a Sino-Vietnamese term is not ceremonial vocabulary merely +because it is formal — see the false positives in +[Vietnamese AI pattern registry](ai-pattern-vietnamese.md). + +Multi-word terms are classified as a unit. `random forest` the algorithm is a +designator; `random sampling` is a generic method name and translates +(`lấy mẫu ngẫu nhiên`). Translating a named method word by word is +`overtranslation_of_designator`; retaining a generic collocation word by word is +`untranslated_generic_technicalism`. + +## 12. Venue, stylesheet, and author glossary + +A declared stylesheet outranks this policy's defaults for *form*, never for +identity. + +- If the venue forbids source-language terms in body prose, `translate_with_gloss` + tightens to `translate`, except for rigid designators, which still `keep_source` + behind a category noun. +- If the venue's keyword list or indexing language is English, those strings stay + English even when the body is Vietnamese. +- If the author supplies a glossary, it governs renderings within its scope. It + does not license translating a designator or collapsing a load-bearing + distinction. +- If two authorities at the same tier disagree, record both, choose one, and mark + `needs_review` rather than oscillating silently, which is `mixed_rendering`. + +Capitalization, italics, or scare quotes do not create a designator: `Baseline` as +a heading is still the generic noun. A trademark that has become a generic term in +the field follows current disciplinary usage rather than the legal status of the +mark. + +## 13. Audit procedure + +1. Extract every Latin-script run in body prose, excluding the protected zones in §7. +2. Classify each into `keep_source`, `translate`, `translate_with_gloss`, + `keep_with_gloss`, or `needs_review`. +3. Check the reverse direction: every Vietnamese rendering that should have been a + designator, and every institutional term mapped without a gloss. +4. Report counts per policy and the list of `needs_review` items. Do not narrate + the sweep. + +Codes: `untranslated_generic_technicalism`, `overtranslation_of_designator`, +`distinction_collapse_by_translation`, `invented_vietnamese_term`, +`mixed_rendering`, `institutional_false_friend`. The second and third are blocking. +See [Writing failure taxonomy](writing-failure-taxonomy.md) and the terminology +localization gate in [Quality rubric](quality-rubric.md). diff --git a/references/writing-failure-taxonomy.md b/references/writing-failure-taxonomy.md index 87f5bef..1769e93 100644 --- a/references/writing-failure-taxonomy.md +++ b/references/writing-failure-taxonomy.md @@ -20,7 +20,93 @@ transfer taxonomy. | `promotional_contribution` | contribution language exceeds the supplied comparison and evidence | delimit the baseline, addition, and scope | | `empty_transition` | a connective creates apparent flow without a logical relation | use conceptual continuity or remove it | | `claim_without_warrant` | evidence and conclusion are adjacent but their inferential link is absent | supply a licensed warrant or weaken the claim | +| `operational_log_prose` | internal build, audit, or QA bookkeeping is written as manuscript content: intermediate candidate tallies, file/column states, checklist confirmations, artifact names, reconciliation notes | keep the finding the check licenses; move the bookkeeping to an appendix, data statement, or repository record | +| `progress_state_limitation` | a limitation reports the current state of the project's work rather than a bound on inference: unfilled fields, unfinished steps, artifacts not yet produced | restate as the validity, reliability, generalizability, or measurement limit that the missing work leaves open | +| `todo_future_work` | future work lists tasks the team owes rather than open research questions | restate each item as a question, the design that would answer it, and what it would license | +| `results_digest_abstract` | the abstract enumerates counts and closes on caveats, so the aim and contributions are never stated | rebuild as gap → aim → contributions with mechanisms → principal finding → one boundary sentence | +| `unanswerable_rq` | a research question yields no contribution, typically by asking what limitations must be declared | recast as an empirical question whose answer is the contribution, or delete it and renumber | +| `activity_as_contribution` | a contribution names work performed rather than the mechanism that makes it novel or usable | add the mechanism: what the step enables that its absence would not | +| `fabricated_constraint` | a numeric limit is enforced from the author's own derived notes rather than the authoritative source, and content is cut to satisfy it | re-derive the limit from the template or a measured published article; restore any content removed for it | +| `self_reminder_prose` | the draft instructs itself instead of stating the result: a modal of obligation aimed at the text (`cần nêu rõ`, `cần nêu thẳng`, `cần được diễn giải thận trọng`, `được nêu dưới đây`, *it should be noted that*) | delete the instruction and keep its result as a direct assertion; the scope it carried must survive | +| `defensive_disclaimer_stack` | three or more adjacent sentences deny strong claims (`chỉ …`, `không phải là …`, `không chứng minh …`, `không bảo đảm …`) until the finding disappears under its own disclaimers | collapse to one boundary sentence stating the bounded claim, not denying the unmade one | +| `config_dump_prose` | parameters are copied from a config file or CLI call and keep their identifier form in prose (`seed`, `backend`, `qa_id`, bare `vLLM`, `top-$p$`, `512 token`) | keep every value; name the role before the identifier and group parameters by pipeline stage | +| `generated_artifact_drift` | an element persists in the rendered artifact after removal from source (exporter hardcode), or hand edits to a delivered artifact are lost on regeneration | grep the exporter before re-editing source; diff the user's returned artifact against your last build and back-port every change, including structural moves | +| `untranslated_generic_technicalism` | a source-language term is retained in target-language prose because it is familiar to the author's working environment, not because it names a unique external object | apply the designator and established-rendering tests; translate and lock one rendering | +| `overtranslation_of_designator` | a rigid designator is translated: model, product, library, standard number, statute short title, gene/taxon symbol, eponym, dataset name, unit, or identifier | restore the exact string; put the Vietnamese category noun in front of it instead | +| `distinction_collapse_by_translation` | two source concepts the argument separates receive one target rendering (validity/reliability, efficacy/effectiveness, hazard/risk, verification/validation) | restore two lexically distinct renderings and gloss each at first use | +| `invented_vietnamese_term` | a rendering inferred by the writer is presented as the field's established term | demote to a source term plus descriptive gloss and mark `needs_review`, or cite the authority | +| `mixed_rendering` | the same concept alternates between its source form and its translation, or between two translations, within one language version | lock one rendering per concept per version | +| `institutional_false_friend` | a system-specific Vietnamese institutional, legal, or academic-title term is mapped onto a foreign near-equivalent without a gloss | keep the numbering/identity and gloss the institutional difference | +| `document_as_subject` | the sentence is about the paper, a section, or the act of writing (`phần này sẽ trình bày`, `như đã đề cập ở trên`, *this section will*) | delete a second roadmap; recast remaining cases so the subject is the object of study | +| `verification_log_prose` | a checklist confirmation is reported as a result (`đã kiểm tra`, `checksum khớp`, `0 lỗi`) | state the property the check establishes and, explicitly, the property it does not; relocate the log | +| `internal_artifact_reference` | a path on the author's machine, a working filename, a ticket, a bare commit hash, or a sheet column appears in publication-facing prose | relocate the identifier to a data card or repository record; keep public DOI/URL/dataset names | +| `repo_artifact_reference` | a repository script or data filename appears in body prose outside a data or reproducibility statement | state what the step does; move the filename into the reproducibility statement, where it is a legitimate public identifier | +| `assistant_residue` | the drafting conversation is visible (`theo yêu cầu của anh`, *as requested*, *the assistant*) | delete; the published text has no addressee inside the production process | +| `placeholder_residue` | a drafting token survived (`TODO`, `TBD`, `[…]`, `???`) | resolve or delete before delivery | +| `revision_response_leak` | reviewer-letter register appears in the manuscript (`theo góp ý của phản biện`) | move it to the response letter; the manuscript states the resulting claim | `invented_sample_size`, `causal_overclaim`, `causal_claim`, -`population_generalization`, and `invented_citation` are blocking. Other -failures require revision when they materially affect the argument. +`population_generalization`, `invented_citation`, `fabricated_constraint`, +`assistant_residue`, `placeholder_residue`, and `internal_artifact_reference` +are blocking. Other failures require revision when they materially affect the +argument. + +`fabricated_constraint` is blocking because it is the only failure here that +destroys correct work. The others degrade a claim; this one deletes content that +was compliant, and the deletion is invisible afterwards — nothing in the +manuscript records that a section was shortened to satisfy a rule nobody +imposed. + +`self_reminder_prose`, `document_as_subject`, `defensive_disclaimer_stack`, +`config_dump_prose`, `progress_state_limitation`, `operational_log_prose`, +`verification_log_prose`, `todo_future_work`, and `revision_response_leak` +require revision on every occurrence rather than only when material. All are +visible to a reviewer on a first reading. A lexical scanner raises the floor; +paraphrases still require a manual pass over Methods, Limitations, Future Work, +and every integrity subsection. See +[Internal register gate](internal-register-gate.md). + +`overtranslation_of_designator` and `distinction_collapse_by_translation` are +blocking. Both change what the text refers to rather than how it reads: the first +severs the reader from an external object, the second erases a contrast the argument +depends on. The remaining terminology codes require revision, and +`invented_vietnamese_term` additionally requires that the uncertainty be reported +rather than absorbed. See +[Terminology localization policy](terminology-localization.md). + +`generated_artifact_drift` is blocking in one direction. Regenerating an +artifact over a user's hand edits destroys correct work exactly as +`fabricated_constraint` does, and just as invisibly — the rebuilt file carries +no record that the edits existed. + +## Operational record versus scientific claim + +`operational_log_prose`, `progress_state_limitation`, and `todo_future_work` +share one cause: a build log, audit spreadsheet, or task tracker is used as the +draft's source, so its unit of analysis (files, rows, candidates, completed +steps) survives into the manuscript. The text stays true and still fails, +because a reader cannot act on it. + +Apply this test to every sentence in a Methods, Limitations, or Future Work +section: **if the sentence would change when the team finishes more work but the +scientific finding would not, it is a progress report.** Two rewrites follow. + +- Progress form: *"The two annotator files contain 240 rows but the rating + columns are empty, so no kappa is reported."* The reader learns the state of + two files. +- Inference form: *"Labels have no human reference annotation, so label + reliability is unquantified and error from the generator cannot be separated + from error in the source context."* The reader learns which conclusions are + unavailable and why. + +Intermediate pipeline tallies follow the same rule. Report the quantity that +carries a mechanism (an error rate, a retention rate, a measured skew) and the +procedure it triggered; drop the per-bucket candidate counts that only +reconstruct the team's spreadsheet. Reconciliation notes between two artifacts +are content only when the discrepancy changes how a number may be used — then +state the consequence ("the two figures share no denominator and cannot be +compared"), not the audit trail. + +Verification checklists invert the usual direction of evidence: passing them +licenses nothing on its own, so a bare confirmation is not a result. Write the +property the check establishes and, explicitly, the property it does not. diff --git a/schemas/audit-record.schema.json b/schemas/audit-record.schema.json index 6a9d2d2..7451275 100644 --- a/schemas/audit-record.schema.json +++ b/schemas/audit-record.schema.json @@ -16,6 +16,7 @@ "rationale": {"type": "string", "minLength": 1}, "semantic_change": {"type": "boolean"}, "confidence": {"type": "number", "minimum": 0, "maximum": 1}, + "register_verdict": {"description": "Resolution of an internal-register finding. 'soften' is not available: hedging an internal sentence leaves it internal. See references/internal-register-gate.md", "enum": ["delete", "recast", "relocate", "license", "not_applicable"]}, "decision": {"enum": ["accept", "revise", "human_review"]} }, "additionalProperties": false diff --git a/schemas/glossary-entry.schema.json b/schemas/glossary-entry.schema.json index 024597d..9d1ee49 100644 --- a/schemas/glossary-entry.schema.json +++ b/schemas/glossary-entry.schema.json @@ -3,17 +3,98 @@ "$id": "https://github.com/mxuanvan02/academic-prose/schemas/glossary-entry.schema.json", "title": "Bilingual academic glossary entry", "type": "object", - "required": ["source", "preferred", "domain", "definition", "status"], + "required": [ + "source", + "preferred", + "domain", + "definition", + "status", + "policy" + ], "properties": { - "source": {"type": "string", "minLength": 1}, - "preferred": {"type": "string", "minLength": 1}, - "alternatives": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, - "avoid": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, - "domain": {"type": "string", "minLength": 1}, - "definition": {"type": "string", "minLength": 1}, - "context_rules": {"type": "array", "items": {"type": "string"}}, - "confidence": {"type": "number", "minimum": 0, "maximum": 1}, - "status": {"enum": ["proposed", "approved", "deprecated", "needs_review"]} + "source": { + "type": "string", + "minLength": 1 + }, + "preferred": { + "type": "string", + "minLength": 1 + }, + "alternatives": { + "type": "array", + "items": { + "type": "string" + }, + "uniqueItems": true + }, + "avoid": { + "type": "array", + "items": { + "type": "string" + }, + "uniqueItems": true + }, + "domain": { + "type": "string", + "minLength": 1 + }, + "definition": { + "type": "string", + "minLength": 1 + }, + "context_rules": { + "type": "array", + "items": { + "type": "string" + } + }, + "confidence": { + "type": "number", + "minimum": 0, + "maximum": 1 + }, + "status": { + "enum": [ + "proposed", + "approved", + "deprecated", + "needs_review" + ] + }, + "policy": { + "description": "Localization decision for this concept; see references/terminology-localization.md", + "enum": [ + "keep_source", + "translate", + "translate_with_gloss", + "keep_with_gloss", + "needs_review" + ] + }, + "authority_tier": { + "description": "1 legal/national standard, 2 textbook/approved dictionary, 3 attested journal usage, 4 official international version, 5 writer's morphological inference", + "type": "integer", + "minimum": 1, + "maximum": 5 + }, + "authority_note": { + "type": "string" + }, + "gloss_at_first_use": { + "type": "boolean" + }, + "paired_term": { + "description": "Exact term used for this concept in the other language version", + "type": "string" + }, + "distinguished_from": { + "description": "Concepts that must not share this rendering, e.g. reliability for an entry on validity", + "type": "array", + "items": { + "type": "string" + }, + "uniqueItems": true + } }, "additionalProperties": false } diff --git a/schemas/paragraph-plan.schema.json b/schemas/paragraph-plan.schema.json index dd6a0ee..ded22a9 100644 --- a/schemas/paragraph-plan.schema.json +++ b/schemas/paragraph-plan.schema.json @@ -3,10 +3,11 @@ "$id": "https://github.com/mxuanvan02/academic-prose/schemas/paragraph-plan.schema.json", "title": "Bilingual academic paragraph plan", "type": "object", - "required": ["paragraph_id", "dominant_function", "claim_ids", "moves", "transition_basis"], + "required": ["paragraph_id", "dominant_function", "admissible_subject", "claim_ids", "moves", "transition_basis"], "properties": { "paragraph_id": {"type": "string", "minLength": 1}, "dominant_function": {"type": "string", "minLength": 1}, + "admissible_subject": {"description": "Semantic subject the paragraph is about; the manuscript, project, file, or drafting conversation are inadmissible. See references/internal-register-gate.md", "enum": ["object_of_study", "data", "method", "result", "cited_claim", "licensed_inference"]}, "claim_ids": {"type": "array", "items": {"type": "string", "minLength": 1}, "minItems": 1, "uniqueItems": true}, "moves": {"type": "array", "items": {"enum": ["territory", "problem", "synthesized_gap", "purpose", "method", "result", "answer", "interpretation", "comparison_dimension", "convergence_or_tension", "alternative_explanation", "limitation", "evidence_boundary", "bounded_contribution", "bounded_implication", "research_need"]}, "minItems": 1}, "evidence_refs": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, diff --git a/scripts/fixtures/internal_register_clean.md b/scripts/fixtures/internal_register_clean.md new file mode 100644 index 0000000..1887447 --- /dev/null +++ b/scripts/fixtures/internal_register_clean.md @@ -0,0 +1,13 @@ +# Kết quả + +Nhãn không có đối chiếu của chuyên gia, nên độ tin cậy của nhãn chưa được định lượng. +Bảng 2 cho thấy độ chính xác 55,43% trên tập kiểm tra. +Dữ liệu không phân tầng theo lĩnh vực, nên hiệu năng theo từng lĩnh vực không thể phân tích. +Tập bản ghi có định danh duy nhất, điều kiện cần để các tỷ lệ ở Mục 4 không bị đếm trùng; kiểm tra này không thiết lập tính không trùng lặp về nội dung. +Dữ liệu công bố tại DOI 10.1234/example. + +# Thảo luận + +Mô hình có sai lệch hệ thống theo độ dài ngữ cảnh, giới hạn phạm vi suy rộng sang văn bản dài hơn. +Hiệu ứng quan sát được giới hạn trong thiết lập đã mô tả và không hàm ý quan hệ nhân quả. +Câu hỏi còn mở là liệu nhãn nhận thức có hội tụ với đánh giá của chuyên gia; thiết kế khả thi là xây tập tham chiếu do hai chuyên gia gán độc lập. diff --git a/scripts/fixtures/internal_register_clean_en.md b/scripts/fixtures/internal_register_clean_en.md new file mode 100644 index 0000000..1f822fb --- /dev/null +++ b/scripts/fixtures/internal_register_clean_en.md @@ -0,0 +1,13 @@ +Labels have no expert reference annotation, so label reliability is unquantified. + +The construction procedure uses fixed parameters throughout. + +The dataset is not stratified by domain, so per-domain performance cannot be analysed. + +Record identifiers are unique, a precondition for the rates in Section 4 not being double-counted; uniqueness of identifiers does not establish uniqueness of content. + +Results show X. + +Table 2 reports the accuracy of 55.43 percent. + +Data are released at https://doi.org/10.1234/example. diff --git a/scripts/fixtures/internal_register_dirty.md b/scripts/fixtures/internal_register_dirty.md new file mode 100644 index 0000000..d8c93ac --- /dev/null +++ b/scripts/fixtures/internal_register_dirty.md @@ -0,0 +1,18 @@ +# Kết quả + +Một giới hạn cần nêu rõ: nhãn chưa được thẩm định. +Phần này sẽ trình bày quy trình xây dựng dữ liệu. +Nhóm chưa kịp bổ sung phân tầng theo lĩnh vực. +Chúng tôi đã kiểm tra và checksum khớp, 0 lỗi. +Kết quả trong tệp /Users/van/ket_qua_v9.csv và main.tex cho thấy độ chính xác tăng. +Theo yêu cầu của anh, TODO sẽ được hoàn tất trước khi gửi. +Theo góp ý của phản biện, chúng tôi đã bổ sung Bảng 3. +Kết quả chỉ thiết lập tính nhất quán nội tại. +Chúng không chứng minh quan hệ nhân quả. +Chúng cũng không bảo đảm nhãn đã được chuyên gia thẩm định. + +# Thảo luận + +Như đã đề cập ở trên, mô hình có sai lệch hệ thống. +Mục này sẽ mô tả các hạn chế còn lại. +Bản ghi được lưu ở cột "ghi_chu_noi_bo" của sheet đối soát. diff --git a/scripts/fixtures/internal_register_dirty_en.md b/scripts/fixtures/internal_register_dirty_en.md new file mode 100644 index 0000000..650afac --- /dev/null +++ b/scripts/fixtures/internal_register_dirty_en.md @@ -0,0 +1,15 @@ +It should be noted that a limitation must be stated: labels have not yet been validated. + +This section will present the construction procedure. + +The team remains to be done with domain stratification; annotation is still in progress. + +We checked the identifiers and the checksum matched, with 0 errors. + +Results in `/Users/van/ket_qua_v9.csv` and `main.tex` show X. + +As requested, TODO will be completed. + +As the reviewer requested, we added Table 3. + +The results only establish X. They do not prove Y. They cannot guarantee Z. diff --git a/scripts/fixtures/internal_register_licensed.md b/scripts/fixtures/internal_register_licensed.md new file mode 100644 index 0000000..a64a0e8 --- /dev/null +++ b/scripts/fixtures/internal_register_licensed.md @@ -0,0 +1,23 @@ +# Mở đầu + +Trong nghiên cứu này, chúng tôi xây dựng tập dữ liệu hỏi đáp pháp luật tiếng Việt gồm 5.980 ngữ cảnh. +Mục 2 tổng quan các tài nguyên liên quan, Mục 3 mô tả quy trình xây dựng dữ liệu, Mục 4 báo cáo kết quả đánh giá và Mục 5 nêu các hạn chế. + +# Phương pháp + +Chúng tôi lấy mẫu ngữ cảnh trong khoảng 100–3.000 ký tự. +Bảng 1 trình bày phân bố độ dài, và Hình 2 cho thấy tỷ lệ trùng lặp theo giáo trình. +Quy trình dùng thư viện suy luận vLLM ở chế độ lượng tử hóa 4 bit, nhiệt độ lấy mẫu 0,7. + +# Tuyên bố dữ liệu + +Bộ dữ liệu công bố tại DOI 10.5281/zenodo.1234567 và kho lưu trữ https://github.com/example/vi-legalqa. +Phụ lục A liệt kê toàn bộ nguồn giáo trình. + +# Tính cấp thiết của đề tài + +Tài nguyên đánh giá cho miền pháp luật tiếng Việt còn thiếu, nên năng lực của các mô hình hiện có chưa được đo trên văn bản quy phạm trong nước. + +# Hạn chế + +Nhãn nhận thức là nhãn thao tác, nên các so sánh giữa mô hình chỉ có giá trị trong phạm vi định nghĩa nhãn này. diff --git a/scripts/fixtures/internal_register_protected.tex b/scripts/fixtures/internal_register_protected.tex new file mode 100644 index 0000000..a3c9a3f --- /dev/null +++ b/scripts/fixtures/internal_register_protected.tex @@ -0,0 +1,15 @@ +% TODO: nhớ kiểm tra lại bảng này trước khi gửi — comment nội bộ, không phải văn xuôi. +% Theo yêu cầu của anh, đổi thứ tự hai mục. +\section{Kết quả} + +Độ chính xác trên tập kiểm tra đạt 55,43\%, cao hơn thiết lập không ngữ cảnh 4,2 điểm phần trăm. +Ngưỡng phân loại được đặt tại $\text{TODO} = 0$ trong công thức $y = \sigma(Wx + b)$. + +\begin{verbatim} +# Theo yêu cầu của anh: chạy lại pipeline +python3 /Users/van/run_eval.py --config config.py --out v9_final.csv +# TODO: thêm seed +\end{verbatim} + +Bộ dữ liệu được công bố kèm mã nguồn \cite{main.tex2024} tại \url{https://example.org/TODO/dataset}. +Bảng~\ref{tab:main} và Mục~\ref{sec:method} trình bày chi tiết quy trình xây dựng dữ liệu. diff --git a/scripts/fixtures/internal_register_response_letter.md b/scripts/fixtures/internal_register_response_letter.md new file mode 100644 index 0000000..b7b1e8c --- /dev/null +++ b/scripts/fixtures/internal_register_response_letter.md @@ -0,0 +1,5 @@ +# Phản hồi nhận xét của phản biện + +Theo góp ý của phản biện 1, chúng tôi đã bổ sung Bảng 3 và viết lại Mục 3.2. +Như phản biện 2 đề nghị, chúng tôi đã kiểm tra lại toàn bộ số liệu và xác nhận tổng hàng khớp với tệp kết quả. +Phần này trình bày từng nhận xét kèm thay đổi tương ứng trong bản thảo đã hiệu đính. diff --git a/scripts/internal_register_scan.py b/scripts/internal_register_scan.py new file mode 100644 index 0000000..bbe0d0f --- /dev/null +++ b/scripts/internal_register_scan.py @@ -0,0 +1,363 @@ +#!/usr/bin/env python3 +"""Detect publication-internal register in text, LaTeX, Markdown, or DOCX. + +A lexical hit is a candidate, never a verdict: the gate document requires each +finding to be resolved by the referent, permanence, and outsider-verifiability +tests, and the resolution recorded as delete / recast / relocate / license. +Absence of hits is not a pass either -- paraphrases of the paraphrasable classes +have no lexical signature. + +Exit: 0 clean, 1 blocking class present, 2 revision-level findings, 3 input error. +""" +from __future__ import annotations + +import argparse +import json +import re +import sys +import zipfile +from pathlib import Path +from xml.etree import ElementTree as ET + +# --- classes ----------------------------------------------------------------- + +BLOCKING = {"assistant_residue", "placeholder_residue", "internal_artifact_reference"} + +PATTERNS: dict[str, list[str]] = { + "self_reminder_prose": [ + r"cần\s+(?:nêu|lưu\s+ý|thận\s+trọng|được\s+nêu|phải\s+nêu|được\s+diễn\s+giải)", + r"được\s+nêu\s+(?:dưới\s+đây|ở\s+dưới)", + r"(?:xin|cần)\s+lưu\s+ý\s+rằng", + r"it\s+should\s+be\s+noted", + r"care\s+must\s+be\s+taken", + r"the\s+following\s+must\s+be\s+stated", + ], + "document_as_subject": [ + r"(?:phần|mục|chương)\s+này\s+(?:sẽ|trình\s+bày|mô\s+tả|giới\s+thiệu)", + r"như\s+đã\s+(?:đề\s+cập|trình\s+bày|nói)\s+(?:ở\s+)?(?:trên|dưới|phần\s+trước)", + r"ở\s+(?:đoạn|phần)\s+(?:trên|dưới|trước|sau)\b", + r"this\s+(?:section|paper|article|chapter)\s+(?:will|presents\s+below)", + r"as\s+(?:mentioned|discussed)\s+above", + ], + "progress_state_limitation": [ + r"chưa\s+(?:kịp|hoàn\s+tất|bổ\s+sung|thực\s+hiện|được\s+thẩm\s+định)", + r"sẽ\s+(?:bổ\s+sung|hoàn\s+thiện|cập\s+nhật)\s+(?:sau|trong\s+thời\s+gian\s+tới)", + r"đang\s+trong\s+quá\s+trình", + r"nhóm\s+(?:nghiên\s+cứu\s+)?chưa\b", + r"remains?\s+to\s+be\s+done", + r"(?:has\s+not|have\s+not|hasn't|haven't)\s+(?:yet\s+)?(?:been\s+)?(?:completed|finished|added|validated|annotated)", + r"(?:will|is\s+planned\s+to)\s+(?:be\s+)?(?:added|completed|updated)\s+(?:later|in\s+future\s+work|in\s+a\s+future\s+release)", + r"(?:is|are)\s+still\s+(?:in\s+progress|underway|pending)", + ], + "verification_log_prose": [ + r"(?:chúng\s+tôi\s+)?đã\s+(?:kiểm\s+tra|biên\s+dịch|chạy|xác\s+nhận|rà\s+soát)", + r"checksum\s+(?:khớp|match)", + r"\b0\s*lỗi\b", + r"\bkhông\s+phát\s+hiện\s+lỗi\b", + r"đã\s+được\s+kiểm\s+tra\s+lại", + r"(?:we\s+)?(?:checked|verified|confirmed|reran|re-ran|compiled)\b", + r"(?:the\s+)?checksum\s+(?:matched|matches)", + r"\b(?:zero|0)\s+(?:errors|failures)\b", + r"(?:no\s+(?:errors|failures)\s+(?:were\s+)?found|passed\s+all\s+checks)", + ], + # Blocking: no reader of the published artifact can reach these. + "internal_artifact_reference": [ + r"(?:/Users|/home|/tmp|/var/folders|[A-Z]:\\\\)[^\s,;)`'\"}]*", + r"\b(?:main|draft|manuscript|ban_thao)\.(?:tex|docx?)\b", + r"\bv\d+(?:[._-]?final|[._-]?rev\d*)\b", + r"\b(?:commit|sha)\s+[0-9a-f]{7,40}\b", + r"(?:^|\s)#\d{2,}\b", + r"\b(?:cột|sheet|tab)\s+[\"'`][^\"'`]+[\"'`]", + ], + # Revision-level: a repo-relative artifact name is licensed inside a data or + # reproducibility statement and a leak anywhere else. + "repo_artifact_reference": [ + r"\b[\w./-]*\w+\.(?:py|ipynb|csv|tsv|jsonl?|xlsx|log|sh|yaml|yml)\b", + r"\bscripts?/[\w./-]+", + ], + "assistant_residue": [ + r"theo\s+(?:yêu\s+cầu|đề\s+nghị)\s+của\s+(?:anh|chị|bạn|ông|bà)", + r"như\s+(?:anh|chị|bạn)\s+(?:đã\s+)?(?:yêu\s+cầu|nói)", + r"as\s+(?:you\s+)?requested", + r"\bthe\s+assistant\b", + r"chúng\s+ta\s+đã\s+(?:sửa|viết|thống\s+nhất)", + ], + "placeholder_residue": [ + r"\b(?:TODO|TBD|FIXME|XXX)\b", + r"\[\s*\.\.\.\s*\]", + r"\[(?:điền|cần\s+bổ\s+sung|chưa\s+có)[^\]]*\]", + r"\?{3,}", + r"\bLorem\s+ipsum\b", + ], + "revision_response_leak": [ + r"theo\s+(?:góp\s+ý|yêu\s+cầu|nhận\s+xét)\s+của\s+(?:phản\s+biện|người\s+phản\s+biện|reviewer)", + r"as\s+the\s+reviewer\s+(?:requested|suggested)", + r"trong\s+bản\s+(?:sửa|hiệu\s+đính)\s+này,?\s+chúng\s+tôi\s+đã", + ], +} + +# Claim-denial markers only. A bare `chỉ` or `không` is ordinary Vietnamese; the +# defect is a stack of sentences that deny a claim nobody made. +CLAIM_DENIAL = re.compile( + r"(?:^|[\s,;(])(?:" + r"không\s+(?:chứng\s+minh|bảo\s+đảm|hàm\s+ý|thiết\s+lập|khẳng\s+định|" + r"cho\s+phép\s+(?:kết\s+luận|suy\s+rộng)|đủ\s+để\s+kết\s+luận)" + r"|không\s+phải\s+là" + r"|chỉ\s+(?:thiết\s+lập|cho\s+thấy|mang\s+tính|có\s+giá\s+trị|phản\s+ánh|áp\s+dụng)" + r"|do(?:es)?\s+not\s+(?:prove|guarantee|imply|establish|validate|support|demonstrate)" + r"|cannot\s+(?:prove|guarantee|imply|establish|validate|support|demonstrate)" + r"|only\s+(?:establish(?:es)?|show(?:s)?|reflect(?:s)?|appl(?:y|ies)\s+to|has\s+diagnostic\s+value)" + r")", + re.I, +) +# `không chỉ ... mà còn` is emphatic, the opposite of a disclaimer. +DENIAL_EXCLUDE = re.compile(r"không\s+chỉ|not\s+only", re.I) + +# Sections where a repo-relative artifact name is the licensed register. +ARTIFACT_LICENSED_SECTION = re.compile( + r"(?:tái\s+lập|khả\s+năng\s+tái\s+lập|tuyên\s+bố\s+dữ\s+liệu|dữ\s+liệu\s+và\s+mã|" + r"phụ\s+lục|reproducib|data\s+(?:and\s+code\s+)?availab|code\s+availab|" + r"artifact|appendix|supplementary)", + re.I, +) + +GENRE_LICENSED: dict[str, set[str]] = { + "manuscript": set(), + "thesis": set(), + "report": set(), + "abstract": set(), + "response_letter": {"revision_response_leak", "assistant_residue", "document_as_subject"}, + "revision_notes": {"revision_response_leak", "assistant_residue", "document_as_subject"}, + "teaching": {"document_as_subject"}, + "slides": {"document_as_subject"}, + "speaker_notes": {"document_as_subject"}, +} + +SENTENCE = re.compile(r"(?<=[.!?;])\s+|\n{2,}") +HEADING = re.compile(r"^(?:#{1,6}\s+\S|\\(?:sub)*section\*?\{)", re.M) +HEADING_TITLE = re.compile(r"^(?:#{1,6}\s+(?P.+)|\\(?:sub)*section\*?\{(?P[^}]*)\})") + +# Thresholds mirror references/internal-register-gate.md section 5. +THRESHOLDS = { + "self_reminder_prose": 0, + "assistant_residue": 0, + "placeholder_residue": 0, + "internal_artifact_reference": 0, + "repo_artifact_reference": 0, + "progress_state_limitation": 0, + "verification_log_prose": 0, + "revision_response_leak": 0, + "document_as_subject": 1, # one roadmap per document + "defensive_disclaimer_stack": 0, +} + + +def read_input(path: Path) -> str: + if path.suffix.lower() == ".docx": + with zipfile.ZipFile(path) as zf: + root = ET.fromstring(zf.read("word/document.xml")) + parts: list[str] = [] + for node in root.iter(): + if node.tag.endswith("}p"): + parts.append("\n\n") + elif node.tag.endswith("}t"): + parts.append(node.text or "") + return "".join(parts) + return path.read_text(encoding="utf-8") + + +def strip_protected(text: str) -> str: + """Blank out the zones the policy exempts (gate document section 7).""" + is_tex = "\\begin{" in text or "\\section" in text or "\\documentclass" in text + if is_tex: + # The preamble is configuration, not publication-facing prose. + body = re.split(r"\\begin\{document\}", text, maxsplit=1) + text = body[1] if len(body) > 1 else text + text = re.split(r"\\end\{document\}", text, maxsplit=1)[0] + text = re.sub(r"(? str: + """Drop LaTeX command names but keep their textual argument.""" + protected_heads = re.sub(r"\\((?:sub)*section)\*?\{", r"\n\n@@HEAD:\1@@ ", text) + stripped = re.sub(r"\\[a-zA-Z@]+\*?(?:\[[^\]]*\])?", " ", protected_heads) + stripped = stripped.replace("{", " ").replace("}", " ") + return re.sub(r"@@HEAD:((?:sub)*section)@@", r"\\\1{", stripped) + + +def _heading_title(chunk: str) -> str: + match = HEADING_TITLE.match(chunk.strip()) + if not match: + return "" + return (match.group("md") or match.group("tex") or "").strip() + + +def split_sections(text: str) -> list[tuple[int, str, str]]: + """Return (index, heading_title, section_text). Index 0 is front matter.""" + bounds = [m.start() for m in HEADING.finditer(text)] + if not bounds: + return [(0, "", text)] + chunks: list[tuple[int, str, str]] = [] + if bounds[0] > 0: + chunks.append((0, "", text[: bounds[0]])) + for i, start in enumerate(bounds, start=1): + end = bounds[i] if i < len(bounds) else len(text) + chunk = text[start:end] + chunks.append((i, _heading_title(chunk), chunk)) + return chunks + + +def scan(text: str, genre: str = "manuscript") -> dict: + licensed = GENRE_LICENSED.get(genre, set()) + clean = flatten_markup(strip_protected(text)) + findings: list[dict] = [] + total_sentences = 0 + + for section_idx, heading, section_text in split_sections(clean): + sentences = [s.strip() for s in SENTENCE.split(section_text) if s.strip()] + total_sentences += len(sentences) + artifact_slot = bool(ARTIFACT_LICENSED_SECTION.search(heading)) + per_section_doc_subject = 0 + + for local_idx, sentence in enumerate(sentences, 1): + for code, patterns in PATTERNS.items(): + if code in licensed: + continue + matched = [p for p in patterns if re.search(p, sentence, re.I)] + if not matched: + continue + finding = { + "section": section_idx, + "heading": heading, + "sentence": local_idx, + "class": code, + "span": sentence[:300], + "matched": matched, + "blocking": code in BLOCKING, + "licensable": False, + } + if code == "document_as_subject": + per_section_doc_subject += 1 + # The first such sentence in the opening section may be the + # licensed roadmap; the author records `license` as verdict. + finding["licensable"] = section_idx <= 1 and per_section_doc_subject == 1 + elif code == "repo_artifact_reference": + finding["licensable"] = artifact_slot + findings.append(finding) + + for i in range(max(0, len(sentences) - 2)): + window = sentences[i : i + 3] + count = sum(len(CLAIM_DENIAL.findall(DENIAL_EXCLUDE.sub(" ", s))) for s in window) + if count >= 3: + findings.append( + { + "section": section_idx, + "heading": heading, + "sentence": i + 1, + "class": "defensive_disclaimer_stack", + "span": " ".join(window)[:500], + "matched": [f"{count} claim-denial markers in 3 sentences"], + "blocking": False, + "licensable": False, + } + ) + + counts: dict[str, int] = {} + for f in findings: + counts[f["class"]] = counts.get(f["class"], 0) + 1 + + checks = [] + for code, limit in THRESHOLDS.items(): + if code in licensed: + continue + observed = sum(1 for f in findings if f["class"] == code and not f["licensable"]) + checks.append( + {"check": code, "observed": observed, "threshold": limit, "pass": observed <= limit} + ) + + actionable = [f for f in findings if not f["licensable"]] + blocking = sorted({f["class"] for f in actionable if f["blocking"]}) + exit_code = 1 if blocking else (2 if actionable else 0) + + return { + "genre": genre, + "sentences": total_sentences, + "counts": counts, + "findings": findings, + "thresholds": checks, + "blocking": blocking, + "gate": "block" if blocking else ("revise" if actionable else "scan_clean"), + "manual_pass_required": True, + "note": "A lexical hit is a candidate; absence of hits is a partial verification only.", + "exit_code": exit_code, + } + + +def render_report(result: dict) -> str: + lines = [ + "# Internal register scan", + "", + f"- genre: `{result['genre']}`", + f"- sentences: {result['sentences']}", + f"- gate: **{result['gate']}**", + "", + ] + if result["blocking"]: + lines += ["## Blocking classes", ""] + [f"- `{c}`" for c in result["blocking"]] + [""] + lines += ["## Thresholds", "", "| check | observed | limit | pass |", "| --- | --- | --- | --- |"] + for c in result["thresholds"]: + lines.append( + f"| `{c['check']}` | {c['observed']} | {c['threshold']} | {'yes' if c['pass'] else 'NO'} |" + ) + lines += ["", "## Findings", ""] + if not result["findings"]: + lines.append("No lexical candidates. The manual pass is still required.") + for f in result["findings"]: + tag = " (licensable)" if f["licensable"] else "" + lines.append(f"- `{f['class']}`{tag} — §{f['section']} s{f['sentence']}: {f['span']}") + lines += ["", f"> {result['note']}"] + return "\n".join(lines) + "\n" + + +def main(argv: list[str] | None = None) -> int: + ap = argparse.ArgumentParser(description=__doc__) + ap.add_argument("input", type=Path) + ap.add_argument("--genre", default="manuscript", choices=sorted(GENRE_LICENSED)) + ap.add_argument("--report", type=Path, help="write a Markdown report") + ap.add_argument("--json", dest="json_path", type=Path, help="write the raw findings") + ap.add_argument("--quiet", action="store_true") + args = ap.parse_args(argv) + try: + result = scan(read_input(args.input), args.genre) + except (OSError, KeyError, zipfile.BadZipFile, ET.ParseError) as exc: + print(f"input error: {exc}", file=sys.stderr) + return 3 + if args.json_path: + args.json_path.write_text( + json.dumps(result, ensure_ascii=False, indent=2) + "\n", encoding="utf-8" + ) + if args.report: + args.report.write_text(render_report(result), encoding="utf-8") + if not args.quiet: + print(render_report(result)) + return result["exit_code"] + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/scripts/test_internal_register_scan.py b/scripts/test_internal_register_scan.py new file mode 100644 index 0000000..a22692a --- /dev/null +++ b/scripts/test_internal_register_scan.py @@ -0,0 +1,269 @@ +#!/usr/bin/env python3 +"""Verification suite for the internal register gate. + +Each test names the gate criterion it verifies, so a failure points at a rule in +references/internal-register-gate.md rather than at an opaque regex. + +Run: python3 scripts/test_internal_register_scan.py +""" +from __future__ import annotations + +import unittest +from pathlib import Path + +from internal_register_scan import BLOCKING, THRESHOLDS, read_input, scan, split_sections, strip_protected + +FIXTURES = Path(__file__).parent / "fixtures" + + +def scan_fixture(name: str, genre: str = "manuscript") -> dict: + return scan(read_input(FIXTURES / name), genre) + + +def classes(result: dict) -> set[str]: + return {f["class"] for f in result["findings"]} + + +def actionable_classes(result: dict) -> set[str]: + return {f["class"] for f in result["findings"] if not f["licensable"]} + + +class TestDetection(unittest.TestCase): + """Criterion 1-2: every prohibited class is detected and graded.""" + + @classmethod + def setUpClass(cls) -> None: + cls.dirty = scan_fixture("internal_register_dirty.md") + cls.dirty_en = scan_fixture("internal_register_dirty_en.md") + + def test_all_prohibited_classes_detected(self) -> None: + expected = { + "self_reminder_prose", + "document_as_subject", + "progress_state_limitation", + "verification_log_prose", + "internal_artifact_reference", + "assistant_residue", + "placeholder_residue", + "revision_response_leak", + "defensive_disclaimer_stack", + } + self.assertTrue(expected.issubset(classes(self.dirty)), classes(self.dirty)) + + def test_english_dirty_fixture_detects_the_same_classes(self) -> None: + expected = { + "self_reminder_prose", + "document_as_subject", + "progress_state_limitation", + "verification_log_prose", + "internal_artifact_reference", + "assistant_residue", + "placeholder_residue", + "revision_response_leak", + "defensive_disclaimer_stack", + } + self.assertTrue(expected.issubset(classes(self.dirty_en)), classes(self.dirty_en)) + self.assertEqual(self.dirty_en["exit_code"], 1) + self.assertEqual(self.dirty_en["gate"], "block") + + def test_blocking_classes_force_exit_1(self) -> None: + self.assertEqual(self.dirty["exit_code"], 1) + self.assertEqual(self.dirty["gate"], "block") + self.assertEqual(set(self.dirty["blocking"]), BLOCKING) + + def test_every_threshold_reported(self) -> None: + reported = {c["check"] for c in self.dirty["thresholds"]} + self.assertEqual(reported, set(THRESHOLDS)) + + def test_disclaimer_stack_needs_three_markers(self) -> None: + two = scan("Kết quả chỉ thiết lập X. Chúng không chứng minh Y. Dữ liệu gồm 500 bản ghi.") + self.assertNotIn("defensive_disclaimer_stack", classes(two)) + three = scan( + "Kết quả chỉ thiết lập X. Chúng không chứng minh Y. Chúng không bảo đảm Z." + ) + self.assertIn("defensive_disclaimer_stack", classes(three)) + three_en = scan( + "The results only establish X. They do not prove Y. They cannot guarantee Z." + ) + self.assertIn("defensive_disclaimer_stack", classes(three_en)) + + +class TestCleanText(unittest.TestCase): + """Criterion 3: recast prose passes; the scan does not fire on repairs.""" + + @classmethod + def setUpClass(cls) -> None: + cls.clean = scan_fixture("internal_register_clean.md") + cls.clean_en = scan_fixture("internal_register_clean_en.md") + + def test_clean_fixture_scans_clean(self) -> None: + self.assertEqual(self.clean["findings"], [], self.clean["findings"]) + self.assertEqual(self.clean["exit_code"], 0) + self.assertEqual(self.clean["gate"], "scan_clean") + + def test_english_clean_fixture_scans_clean(self) -> None: + self.assertEqual(self.clean_en["findings"], [], self.clean_en["findings"]) + self.assertEqual(self.clean_en["exit_code"], 0) + + def test_clean_report_still_demands_manual_pass(self) -> None: + self.assertTrue(self.clean["manual_pass_required"]) + self.assertIn("partial verification", self.clean["note"]) + + +class TestFalsePositives(unittest.TestCase): + """Criterion 4: licensed slots must not be reported as defects.""" + + @classmethod + def setUpClass(cls) -> None: + cls.fp = scan_fixture("internal_register_licensed.md") + + def test_licensed_slots_produce_no_actionable_finding(self) -> None: + self.assertEqual(actionable_classes(self.fp), set(), self.fp["findings"]) + self.assertEqual(self.fp["exit_code"], 0) + + def test_resolved_cross_reference_is_not_a_hit(self) -> None: + r = scan("Bảng 2 cho thấy độ chính xác 55,43%. Hình 4 trình bày ma trận nhầm lẫn.") + self.assertEqual(r["findings"], []) + + def test_public_identifiers_are_not_internal_artifacts(self) -> None: + r = scan( + "Dữ liệu công bố tại https://doi.org/10.1234/example và bộ SQuAD được dùng để đối chiếu." + ) + self.assertNotIn("internal_artifact_reference", classes(r)) + + def test_vietnamese_first_person_in_methods_is_allowed(self) -> None: + r = scan("Chúng tôi lấy mẫu 500 bản ghi theo phương pháp phân tầng.") + self.assertEqual(r["findings"], []) + + def test_correlative_not_only_is_not_a_disclaimer_stack(self) -> None: + # "không chỉ … mà còn" is an intensifier, not a claim denial. Measured on + # a real manuscript, this construction produced five false stack hits. + text = ( + "Ưu thế của mô hình không chỉ đến từ biểu diễn mạnh mà còn từ tầng chọn lọc. " + "Ý nghĩa không chỉ nằm ở chỉ số mà còn ở thiết kế. " + "Kết quả không chỉ đúng trên một tập mà còn trên tập thứ hai." + ) + self.assertNotIn("defensive_disclaimer_stack", classes(scan(text))) + text_en = ( + "The advantage comes not only from representation but also from selection. " + "The implication lies not only in the scores but also in the design. " + "The result holds not only on one set but also on the second." + ) + self.assertNotIn("defensive_disclaimer_stack", classes(scan(text_en))) + + def test_latex_preamble_is_not_prose(self) -> None: + text = ( + "\\documentclass{article}\n" + "\\newif\\ifFinal\n\\InputIfFileExists{publication_metadata.validated.tex}{}{}\n" + "\\begin{document}\nDữ liệu gồm 500 bản ghi.\n\\end{document}\n" + ) + self.assertEqual(actionable_classes(scan(text)), set(), scan(text)["findings"]) + + +class TestArtifactSeverity(unittest.TestCase): + """Criterion 9: a machine-local path blocks; a repo script name is revision-level.""" + + def test_local_path_is_blocking(self) -> None: + r = scan("Kết quả trong tệp /Users/van/ket_qua.csv cho thấy X.") + self.assertIn("internal_artifact_reference", r["blocking"]) + self.assertEqual(r["exit_code"], 1) + + def test_repo_script_in_reproducibility_statement_is_revision_not_block(self) -> None: + r = scan("Prompt construction is versioned in scripts/anchored_judge.py.") + self.assertIn("repo_artifact_reference", classes(r)) + self.assertEqual(r["blocking"], []) + self.assertEqual(r["exit_code"], 2) + + +class TestRoadmapQuota(unittest.TestCase): + """Criterion 5: one roadmap is licensable; a second is a finding.""" + + def test_first_roadmap_is_licensable(self) -> None: + r = scan("Phần này sẽ trình bày quy trình xây dựng dữ liệu.") + docsub = [f for f in r["findings"] if f["class"] == "document_as_subject"] + self.assertEqual(len(docsub), 1) + self.assertTrue(docsub[0]["licensable"]) + self.assertEqual(r["exit_code"], 0) + + def test_second_roadmap_in_later_section_is_actionable(self) -> None: + text = ( + "# Mở đầu\n\nPhần này sẽ trình bày quy trình xây dựng dữ liệu.\n\n" + "# Phương pháp\n\nPhần này sẽ mô tả các tham số.\n\n" + "# Kết quả\n\nMục này sẽ giới thiệu số liệu.\n" + ) + r = scan(text) + docsub = [f for f in r["findings"] if f["class"] == "document_as_subject"] + self.assertGreaterEqual(len(docsub), 3) + self.assertEqual(sum(1 for f in docsub if f["licensable"]), 1) + self.assertEqual(r["exit_code"], 2) + check = next(c for c in r["thresholds"] if c["check"] == "document_as_subject") + self.assertFalse(check["pass"]) + + +class TestGenreProfiles(unittest.TestCase): + """Criterion 6: the genre decides which register is licensed.""" + + def test_response_letter_licenses_its_own_register(self) -> None: + r = scan_fixture("internal_register_response_letter.md", genre="response_letter") + self.assertNotIn("revision_response_leak", classes(r)) + self.assertNotIn("assistant_residue", classes(r)) + + def test_same_text_as_manuscript_is_a_finding(self) -> None: + r = scan_fixture("internal_register_response_letter.md", genre="manuscript") + self.assertIn("revision_response_leak", classes(r)) + + def test_teaching_genre_licenses_forward_reference(self) -> None: + text = "Ở phần sau chúng ta sẽ xét ví dụ thứ hai." + self.assertIn("document_as_subject", classes(scan(text, "manuscript"))) + self.assertNotIn("document_as_subject", classes(scan(text, "teaching"))) + + +class TestProtectedZones(unittest.TestCase): + """Criterion 7: protected zones are exempt (policy section 7).""" + + def test_latex_protected_zones_are_stripped(self) -> None: + r = scan_fixture("internal_register_protected.tex") + self.assertEqual(actionable_classes(r), set(), r["findings"]) + + def test_verbatim_block_is_exempt(self) -> None: + text = ( + "\\documentclass{article}\n" + "\\begin{verbatim}\nTODO: chạy lại /Users/van/main.tex\n\\end{verbatim}\n" + "Dữ liệu gồm 500 bản ghi.\n" + ) + self.assertEqual(scan(text)["findings"], []) + + def test_comment_is_exempt_but_body_is_not(self) -> None: + text = "\\section{Kết quả}\n% TODO: sửa lại đoạn này\nDữ liệu gồm 500 bản ghi.\n" + self.assertEqual(scan(text)["findings"], []) + leaked = "\\section{Kết quả}\nTODO: sửa lại đoạn này.\n" + self.assertIn("placeholder_residue", classes(scan(leaked))) + + def test_math_and_citation_arguments_are_exempt(self) -> None: + text = "\\section{A}\nTa có $v_1 = 0$ và \\cite{main.tex2020} cho kết quả tương tự.\n" + self.assertNotIn("internal_artifact_reference", classes(scan(text))) + + +class TestInvariants(unittest.TestCase): + """Criterion 8: the scan's own contract.""" + + def test_lexical_hit_is_never_a_verdict(self) -> None: + for name in ("internal_register_dirty.md", "internal_register_clean.md"): + self.assertTrue(scan_fixture(name)["manual_pass_required"]) + + def test_empty_input_is_clean_not_crashing(self) -> None: + r = scan("") + self.assertEqual(r["exit_code"], 0) + self.assertEqual(r["sentences"], 0) + + def test_sections_are_split_on_headings(self) -> None: + chunks = split_sections("# A\n\nx.\n\n# B\n\ny.\n") + self.assertEqual(len(chunks), 2) + + def test_strip_protected_preserves_plain_prose(self) -> None: + text = "Dữ liệu gồm 500 bản ghi." + self.assertIn("500 bản ghi", strip_protected(text)) + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/scripts/validate_skill.py b/scripts/validate_skill.py index bb4dc0b..ffe2880 100644 --- a/scripts/validate_skill.py +++ b/scripts/validate_skill.py @@ -29,6 +29,10 @@ "references/deliverable-playbooks.md", "references/rhetorical-moves.md", "references/writing-failure-taxonomy.md", + "references/terminology-localization.md", + "references/internal-register-gate.md", + "references/self-narration-and-config-dump.md", + "references/artifact-register-to-scientific-register.md", "schemas/audit-record.schema.json", "schemas/glossary-entry.schema.json", "schemas/claim-ledger.schema.json", @@ -41,6 +45,12 @@ "evals/capability-examples.json", "scripts/run_usage_simulations.py", "scripts/run_capability_examples.py", + "scripts/internal_register_scan.py", + "scripts/test_internal_register_scan.py", + "scripts/fixtures/internal_register_dirty.md", + "scripts/fixtures/internal_register_dirty_en.md", + "scripts/fixtures/internal_register_clean.md", + "scripts/fixtures/internal_register_clean_en.md", ) CAPABILITIES = { @@ -287,12 +297,31 @@ def main() -> int: if example_mutations < 60: raise SystemExit("capability examples do not exercise enough atomic mutations") + register_scan = subprocess.run( + [sys.executable, str(ROOT / "scripts/test_internal_register_scan.py")], + cwd=ROOT / "scripts", + text=True, + capture_output=True, + check=False, + ) + if register_scan.returncode != 0: + raise SystemExit( + "internal register scan tests failed:\n" + + register_scan.stdout + + register_scan.stderr + ) + if "FAILED" in register_scan.stdout or "ERROR" in register_scan.stdout: + raise SystemExit( + "internal register scan tests reported failures:\n" + register_scan.stdout + ) + print( "academic-prose validation passed " f"({len(cases)} translation/revision evals, {len(writing_cases)} writing evals, " f"{humanize_count} humanize evals, {len(usage_cases)} usage simulations, " f"{len(capability_examples)} capability examples, " - f"{rejected_mutations + example_mutations} rejected mutations)" + f"{rejected_mutations + example_mutations} rejected mutations, " + "internal-register bilingual scan)" ) return 0 diff --git a/tests/test_repository.py b/tests/test_repository.py index cfc09d2..8cb0f8e 100644 --- a/tests/test_repository.py +++ b/tests/test_repository.py @@ -33,6 +33,10 @@ def test_required_files_exist(self) -> None: "references/domain-profiles.md", "references/quality-rubric.md", "references/pdf-translate-integration.md", + "references/terminology-localization.md", + "references/internal-register-gate.md", + "references/self-narration-and-config-dump.md", + "references/artifact-register-to-scientific-register.md", "schemas/audit-record.schema.json", "schemas/glossary-entry.schema.json", "schemas/claim-ledger.schema.json", @@ -43,6 +47,12 @@ def test_required_files_exist(self) -> None: "evals/humanize-cases.jsonl", "evals/usage-claim-cases.json", "scripts/run_usage_simulations.py", + "scripts/internal_register_scan.py", + "scripts/test_internal_register_scan.py", + "scripts/fixtures/internal_register_dirty.md", + "scripts/fixtures/internal_register_dirty_en.md", + "scripts/fixtures/internal_register_clean.md", + "scripts/fixtures/internal_register_clean_en.md", ) for relative_path in required: with self.subTest(path=relative_path): @@ -74,8 +84,12 @@ def test_rubric_declares_seven_dimensions_and_blocking_failures(self) -> None: "stance_upgrade", "range_notation_corruption", "required_move_deletion", + "assistant_residue", + "placeholder_residue", + "internal_artifact_reference", ): self.assertIn(blocking_error, rubric) + self.assertIn("Internal register gate", rubric) def test_writing_is_the_primary_workflow(self) -> None: skill = (ROOT / "SKILL.md").read_text(encoding="utf-8")