queue: reconcile LaTeX with rendered maths, and repair damaged lines

reading-queue.md

@@ -126,19 +126,19 @@ Accretion onto supermassive black hole binaries — 2405.14843
 Astrometric Jitter as a Detection Diagnostic for Recoiling and Slingshot Supermassive Black Hole Candidates — 2405.11026
 Detection and Prediction of Future Massive Black Hole Mergers with Machine Learning and Truncated Waveforms — 2405.11340
 How do Primordial Black Holes change the Halo Mass Function and Structure? — 2405.11381
-Tip of the iceberg: overmassive black holes at 4<z<7 found by JWST are not inconsistent with the local MBH-M⋆ relationProj  C
+Tip of the iceberg: overmassive black holes at 4<z<7 found by JWST are not inconsistent with the local MBH-M⋆ relation
 
 
 https://arxiv.org/pdf/2103.16254.pdf
 https://arxiv.org/pdf/2112.03576.pdf
 
-as several triplet SMBHs have
-been observed in the local Universe (Deane et al. 2014;
-Pfeifle et al. 2019; Liu et al. 2019; Kollatschny et al.
-2020).
+NOTE as several triplet SMBHs have
+NOTE been observed in the local Universe (Deane et al. 2014;
+NOTE Pfeifle et al. 2019; Liu et al. 2019; Kollatschny et al.
+NOTE 2020).
 
-, shrink the orbits further (see
-e.g. Amaro-Seoane et al. 2022, for a review).
+NOTE , shrink the orbits further (see
+NOTE e.g. Amaro-Seoane et al. 2022, for a review).
 
 The Redshift Difference in Gravitational Lensed Systems: A Novel Probe of Cosmology — 2308.07529
 Evidence for a milli-parsec separation Supermassive Black Hole Binary with quasar microlensing
@@ -204,8 +204,7 @@ Review of solutions to the Cusp-core problem of the ΛCDM Model
 Matching the mass function of Milky Way satellites in competing dark matter model
 The shape of dark matter haloes: results from weak lensing in the Ultraviolet Near-Infrared Optical Northern Survey (UNIONS) — 2209.09088
 A lensing multi-messenger channel: Combining LIGO-Virgo-Kagra lensed gravitational-wave measurements with Euclid observations
-Chandra Observations of Spikey: A Possible Self-lensing Supermassive Black Hole
-Binary System
+Chandra Observations of Spikey: A Possible Self-lensing Supermassive Black Hole Binary System
 The massive relic galaxy NGC 1277 is dark matter deficient. From dynamical models of integral-field stellar kinematics out to five effective radii — 2303.11360
 The impact of baryonic potentials on the gravothermal evolution of self-interacting dark matter haloes — 2306.08028
 
@@ -241,25 +240,25 @@ https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fjournals.plos.
 
 
 
-postprocessing (Kelley et al. 2017a;
-Katz et al. 2020; Volonteri et al. 2020) find that many
-binaries stall and fail to merge within a Hubble time.
-For example, the fiducial model in Kelley et al. (2017a)
-sees a peak of MBHB lifetimes at a median value is 29
-Gyr, with only ∼ 7 % binaries have lifetime less than 1
-Gyr.
+NOTE postprocessing (Kelley et al. 2017a;
+NOTE Katz et al. 2020; Volonteri et al. 2020) find that many
+NOTE binaries stall and fail to merge within a Hubble time.
+NOTE For example, the fiducial model in Kelley et al. (2017a)
+NOTE sees a peak of MBHB lifetimes at a median value is 29
+NOTE Gyr, with only ∼ 7 % binaries have lifetime less than 1
+NOTE Gyr.
 
 __SMBH__
 
 Massive black holes in galactic nuclei: Theory and Simulations
 
     Cant remember
 
-Massive Black Holes summary: https://arxiv.org/pdf/2304.10233.pdf
+Massive Black Holes summary — https://arxiv.org/pdf/2304.10233.pdf
 
     Thorough summary of black hole binarys
 
-__SMBH Binaries_
+__SMBH Binaries__
 
 Hierarchical Bayesian inference on an analytical model of the LISA massive black hole binary population
 

scripts/arxiv_refs.py

@@ -109,20 +109,45 @@ def search_url(title: str) -> str:
 
 # --- title matching -----------------------------------------------------------
 _PUNCT_RE = re.compile(r"[^a-z0-9]+")
+#: A LaTeX control sequence: ``\leq``, ``\Lambda``, ``\rm``, ``\,``.
+_LATEX_CMD_RE = re.compile(r"\\[A-Za-z]+|\\.")
 
 
 def normalise_title(title: str) -> str:
-    """A title reduced to its comparable core: lowercase alphanumerics only.
+    """A title reduced to its comparable core: lowercase ASCII alphanumerics.
 
     arXiv, the digest and a hand-pasted queue line disagree constantly about
     LaTeX (``$z=2.48$``), dashes (``-``/``–``/``—``), accents and double spaces,
     and none of that disagreement is about *which paper it is*. Stripping it all
     lets the backfill demand an exact match on what remains, rather than a fuzzy
     match on what does not — a wrong id is worse than no id, because it links
     the reader confidently to somebody else's paper.
+
+    The one disagreement that needs more than stripping is **maths**, because
+    the two sides spell it differently rather than merely punctuate it
+    differently: arXiv's metadata carries the LaTeX source (``$0.5\\leq
+    z<1.0$``, ``$\\Lambda$CDM``, ``$M_\\star$``) while a title pasted off the
+    abstract page carries the rendered glyphs (``0.5≤z<1.0``, ``ΛCDM``,
+    ``M⋆``). Neither survives a plain punctuation strip into the same string.
+
+    So both are erased rather than reconciled: LaTeX control sequences go, and
+    so does every non-ASCII character left after accent folding. ``\\leq`` and
+    ``≤`` both become nothing, and the two spellings meet in the middle —
+    ``05z10`` from either side. Erasing is deliberately preferred to a
+    ``\\leq``→``≤`` translation table, which would need every symbol
+    astronomers use and would silently mismatch the day it missed one. The
+    reduction is lossier, and the uniqueness rule in :func:`match_entries` is
+    what keeps that safe: a title that reduces onto two arXiv papers matches
+    neither.
+
+    Accented Latin is *kept* (``Ekström`` → ``ekstrom``): NFKD splits the accent
+    off as a combining mark and the base letter is ASCII, so it survives the
+    non-ASCII drop that removes Greek and maths.
     """
     folded = unicodedata.normalize("NFKD", title)
     folded = "".join(c for c in folded if not unicodedata.combining(c))
+    folded = _LATEX_CMD_RE.sub(" ", folded)
+    folded = folded.encode("ascii", "ignore").decode("ascii")
     return _PUNCT_RE.sub("", folded.lower())
 
 

tests/test_arxiv_refs.py

@@ -73,6 +73,39 @@ def test_normalise_title_ignores_the_disagreements_that_are_not_about_identity()
     assert len({arxiv_refs.normalise_title(t) for t in same}) == 1
 
 
+def test_latex_source_and_rendered_glyphs_reduce_to_the_same_title():
+    """arXiv carries the LaTeX; a pasted line carries the rendered glyphs.
+
+    Every pair below is one paper written two ways, and every pair failed to
+    match before maths was erased rather than punctuated away.
+    """
+    for latex, rendered in [
+            (r"$0.5\leq z<1.0$",                    "0.5≤z<1.0"),
+            (r"$\Lambda$CDM Model",                 "ΛCDM Model"),
+            (r"the local $M_{\rm BH}-M_\star$ relation",
+             "the local MBH-M⋆ relation"),
+            (r"$2.2\times10^{-21}$eV",              "2.2×10−21eV"),
+            (r"galaxies at $z\sim3$",               "galaxies at z∼3"),
+            (r"SLICE -- Combining X-ray in AC\,114", "SLICE - Combining X-ray in AC 114")]:
+        assert arxiv_refs.normalise_title(latex) == \
+            arxiv_refs.normalise_title(rendered), latex
+
+
+def test_accented_latin_survives_but_greek_does_not():
+    """NFKD splits the accent off a Latin letter, so the ASCII base remains."""
+    assert arxiv_refs.normalise_title("Ekström") == "ekstrom"
+    assert arxiv_refs.normalise_title("Muñoz-Darias") == "munozdarias"
+    # Greek has no ASCII base and is erased on both sides alike
+    assert arxiv_refs.normalise_title("ΛCDM") == arxiv_refs.normalise_title(r"$\Lambda$CDM")
+
+
+def test_erasing_maths_still_keeps_different_papers_apart():
+    """The reduction is lossier — it must not collapse unrelated titles."""
+    a = arxiv_refs.normalise_title(r"Dark matter at $z\sim3$ in dwarf galaxies")
+    b = arxiv_refs.normalise_title(r"Dark energy at $z\sim3$ in dwarf galaxies")
+    assert a != b
+
+
 def test_a_match_needs_exactly_one_arxiv_paper():
     entries = arxiv_refs.parse_api_entries(
         _atom(("2608.00001", "The Koi Pond"), ("2608.00002", "Another Paper")))