Knobs for a build the caller feeds with FASTA/FASTQ paths. More...

Collaboration diagram for cuddl::path_build_options:

Public Attributes
reference_build_statistics *	statistics = nullptr

unsigned	parser_workers = default_parser_workers()
	Loaders to run.

std::optional< size_t >	staging_bytes {}
	Arena bytes.

transfer_mode	transfer = transfer_mode::automatic
	How the build moves bytes.

decompression_backend	decompression = decompression_backend::automatic

bool	direct_reads = false
	Read the files with `O_DIRECT`, past the page cache.

std::optional< std::reference_wrapper< device_blacklist const > >	blacklist = std::nullopt
	Reusable lookup; keep alive through this synchronous build.

Detailed Description

Knobs for a build the caller feeds with FASTA/FASTQ paths.

Definition at line 172 of file reference_database_file.cuh.

Member Data Documentation

◆ blacklist

std::optional<std::reference_wrapper<device_blacklist const> > cuddl::path_build_options::blacklist = std::nullopt

Reusable lookup; keep alive through this synchronous build.

Definition at line 193 of file reference_database_file.cuh.

◆ decompression

decompression_backend cuddl::path_build_options::decompression = decompression_backend::automatic

Definition at line 186 of file reference_database_file.cuh.

◆ direct_reads

bool cuddl::path_build_options::direct_reads = false

Read the files with O_DIRECT, past the page cache.

On Lustre that costs a quarter of the kernel time: on an A100 node with 32 loaders, 100,000 uncached genomes built in 33.4-36.2 s instead of 36.3-40.4 s. The files stay uncached, so running again over them reads them from storage again: 43.9 s instead of 29.7 s with page-cache reads.

Definition at line 191 of file reference_database_file.cuh.

◆ parser_workers

unsigned cuddl::path_build_options::parser_workers = default_parser_workers()

Loaders to run.

The build runs at most one per input, at most the hardware, and never none, so a larger count is only a ceiling. One loader loads one genome at a time.

Definition at line 176 of file reference_database_file.cuh.

◆ staging_bytes

std::optional<size_t> cuddl::path_build_options::staging_bytes {}

Arena bytes.

Unset sizes the arena from free device memory and what the inputs can fill, capped at 128 MiB so sketching overlaps loading.

Definition at line 179 of file reference_database_file.cuh.

179{};

◆ statistics

reference_build_statistics* cuddl::path_build_options::statistics = nullptr

Definition at line 173 of file reference_database_file.cuh.

◆ transfer

transfer_mode cuddl::path_build_options::transfer = transfer_mode::automatic

How the build moves bytes.

automatic asks the device: a device that reads pageable host memory, which is what makes Grace Hopper and Grace Blackwell coherent, stages the loader's buffers in place and copies nothing; anything else opts into or out of page-locked transfers. The direct_bytes, staged_bytes and in_place fields say which path a build actually took, and device_files how many files the gpu backend loaded on the device.

Definition at line 185 of file reference_database_file.cuh.

The documentation for this struct was generated from the following file:

reference_database_file.cuh

Public Attributes