Adding robustness to IUPAC degenerate nucleotides in reads by andrewdavidsmith · Pull Request #142 · smithlabcode/falco

andrewdavidsmith · 2026-10-28T01:42:31Z

Possible future modification to issue a warning if non-ACGTN are seen

…r is not found (e.g., binary was moved relative to original install dir)

… non ACGTN nucs in modified input files

…hat might have iupac symbols

…hing to iupac robustness mode

… degen nucs

… for robustness to IUPAC degen nucs

…, and modifying the read sequence part to fix iupac degenerate nucleotides to N

…r choice to copy the file's data buffer vs. modifying it directly. Direct modification can't work with memory mapped files using multiple cores because it invalidates all caches and just adding the write mode (so no segfault on mmap data modification) causes many-fold slowdown

…or but importantly having an overload to take fq_gz_task_t so that the fq_task_t can make a locally allocated buffer that can be modified

…etailed help about iupac degen nucs

…d to assumptions on functions that take the sequences

…ze function implementation) and adding a condition for holding the new type fq_gz_task_t

…type from fq_task_t to fq_gz_task_t which has the same behavior, as fq_task_t was just changed to make a copy of the buffer so it can be modified local to the thread's core

… longer has --iupac arg

…falco no longer has --iupac arg

…sk_t so it accepts a buffer to copy mmap data into, so reallocates only as needed

…r mmap plain fastq file input and avoid reallocating; the process_reads for fq_task_t accepts and uses this buffer

…r reading fastq using fread with a preprocessor guard to distinguish

…distinguish between builds using mmapped files and not

…to distinguish between builds using mmapped files and not

…ing for plain FASTQ files

…FOR_PLAIN_FASTQ

… swapping filesystem file size for st_size. And fixing headers for build conditions

…ine, so including it generally

andrewdavidsmith added 30 commits October 14, 2026 15:42

src/get_binary_dir.cpp: added a warning message in case the config di…

689769e

…r is not found (e.g., binary was moved relative to original install dir)

src/original_duplicates.cpp: removing unneeded maybe_unused

83b7c7c

src/falco.cpp: adding a note to the detailed help warning users about…

6e2478e

… non ACGTN nucs in modified input files

src/run_mode.hpp: adding option for robustness to iupac degen symbols

522c9d0

src/results_summary.hpp: forgot to turn clang-format back on

059b46b

src/falco_utils.hpp: adding basic functions for anlayzing sequences t…

9776897

…hat might have iupac symbols

src/falco.cpp: adding args for robustness to iupac symbols

fe21db3

src/results_collector.hpp: adding iupac mode via template

8c54686

src/falco_analyzer.cpp: wrapping the analyze implementation for switc…

1548098

…hing to iupac robustness mode

cmake/tests.cmake: adding a test for iupac robustness

d0404f4

data/test_data/md5sum.txt: adding hash for iupac test

896db77

data/test_scripts/fastq_iupac.sh: adding test for robustness to iupac…

5967c1d

… degen nucs

src/run_mode.cpp: added more info related to the do_iupac run mode

cdfde1a

src/falco.cpp: some cleanup and better info

096303a

src/falco_analyzer.cpp: static analysis fixes

6075942

src/bamrec.cpp: modifying how nucleotides are assigned from BAM codes…

104a8d5

… for robustness to IUPAC degen nucs

src/samrec.cpp: modifying how nucleotides are assigned from BAM codes…

d8d28ee

… for robustness to IUPAC degen nucs

src/run_mode.hcpp: undoing the changes related to iupac mode

2de92b5

src/fqrec.hpp: (1) assuming memory pointed to by 'cursor' is writable…

b6a5dae

…, and modifying the read sequence part to fix iupac degenerate nucleotides to N

src/results_collector.hpp: mostly reverting templated results_collect…

14f60f0

…or but importantly having an overload to take fq_gz_task_t so that the fq_task_t can make a locally allocated buffer that can be modified

src/falco.cpp: reverting iupac options and modifying the comment in d…

536e85f

…etailed help about iupac degen nucs

src/falco_utils.hpp: reverting iupac code and adding a comment relate…

630c930

…d to assumptions on functions that take the sequences

src/falco_analyzer.cpp: reverting iupac-specific (a template on analy…

6bcb1e3

…ze function implementation) and adding a condition for holding the new type fq_gz_task_t

src/fastq_gz_file.cpp and src/fastq_bgzf_file.cpp: changing the task …

b8eef27

…type from fq_task_t to fq_gz_task_t which has the same behavior, as fq_task_t was just changed to make a copy of the buffer so it can be modified local to the thread's core

data/test_data/md5sum.txt: updating test hash for iupac test falco no…

3a444a7

… longer has --iupac arg

data/test_scripts/fastq_iupac.sh: updating test script iupac test as …

826de0a

…falco no longer has --iupac arg

src/fqrec.hpp: adding a size function to fqrec

d06cb36

src/results_collector.hpp: modifying process_reads overload for fq_ta…

adabede

…sk_t so it accepts a buffer to copy mmap data into, so reallocates only as needed

src/falco_analyzer.cpp: adding fq_buffer to hold thread local data fo…

4d0381f

…r mmap plain fastq file input and avoid reallocating; the process_reads for fq_task_t accepts and uses this buffer

andrewdavidsmith added 10 commits October 27, 2026 16:04

src/fqrec.hpp: simplifying loop

97f927e

src/fastq_file.hcpp: merging the code for mmap fastq with the code fo…

734a197

…r reading fastq using fread with a preprocessor guard to distinguish

src/falco_analyzer.cpp: using a preprocessor guard to distinguish to …

a73911d

…distinguish between builds using mmapped files and not

src/results_collector.hpp: using a preprocessor guard to distinguish …

dee9042

…to distinguish between builds using mmapped files and not

README.md: adding explanation about building falco to use memory mapp…

9f013e5

…ing for plain FASTQ files

CMakeLists.txt: adding configure argument that will turn on USE_MMAP_…

6c78116

…FOR_PLAIN_FASTQ

src/fastq_file.hcpp: adding posix_advise sequential for non-macos and…

9e1e895

… swapping filesystem file size for st_size. And fixing headers for build conditions

src/fastq_file.cpp: fixing prepro guard around fcntl.h on apple

3c11c97

src/falco.cpp: attempting to make the message about ACGTN more clear

6e6e398

src/fastq_file.cpp: apparently fcntl.h is needed on systems besides m…

0836bf1

…ine, so including it generally

andrewdavidsmith merged commit 2caf316 into main Oct 28, 2026
3 checks passed

andrewdavidsmith deleted the robustness-to-iupac-degen branch October 28, 2026 01:49

Navigation Menu

Repository navigation

Adding robustness to IUPAC degenerate nucleotides in reads - #142

Adding robustness to IUPAC degenerate nucleotides in reads#142
andrewdavidsmith merged 40 commits into
mainfrom
robustness-to-iupac-degen

andrewdavidsmith commented Oct 28, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

andrewdavidsmith commented Oct 28, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant