Добавил:
Sekretar
kiopkiopkiop18@yandex.ru
t.me/Prokururor I Вовсе не секретарь, но почту проверяю
Опубликованный материал нарушает ваши авторские права? Сообщите нам.
Вуз:
Предмет:
Файл:Ординатура / Хирургия / Библиотека им академика М.И. Перельмана / Книга_5529_Библиотеки_им_академика_М_И_Перельмана
.pdf
304 Bioinformatics of Autoimmune Diseases
"--m-metadata-file meta/metadata.tsv "
"--o-visualization results/core-metrics-results/unweighted-unifrac-
emperor.qzv"
)
]
for cmd in cmds:
run_command(cmd, "Visualizing alpha/beta diversity")
The visualize _ diversity function is responsible for producing a set of visualizations
and statistical analyses that help interpret the results of diversity analyses in a microbiome study. It
focuses on both alpha and beta diversity measures, and it uses metadata to evaluate whether there are
statistically signicant differences between groups of samples. This is achieved by running a series
of QIIME 2 commands stored in a list and executed one by one. The rst two commands perform
alpha diversity group signicance tests. The rst one uses the Faith’s Phylogenetic Diversity vector,
which incorporates phylogenetic relationships into the diversity calculation, while the second uses
Pielou’s Evenness vector, which measures how evenly taxa are distributed in each sample. In each
case, the diversity vector is matched against the metadata le to test whether diversity signicantly
differs across sample groups. The outputs are interactive visualizations in .qzv format that can be
explored to assess statistical group differences and to generate boxplots colored by metadata categories. The third command tests for group differences in beta diversity using the unweighted UniFrac
distance matrix, which reects differences in community composition between samples while considering phylogenetic relationships. The test evaluates whether these differences are associated with
the “condition” column from the metadata le, and the --p-pairwise ag ensures that pairwise
comparisons between groups are included. The resulting visualization enables researchers to identify whether microbial communities differ signicantly between experimental groups. The nal
command generates a PCoA plot using Emperor, based on the unweighted UniFrac PCoA results.
This plot visually represents the clustering of samples in a reduced-dimensional space, allowing for
intuitive comparisons of microbial community structure between groups dened in the metadata.
The output is a rich, interactive plot that can be rotated, ltered, and colored based on metadata attributes. Altogether, the visualize _ diversity function adds interpretive power to the pipeline
by generating outputs that assess the signicance and structure of microbial diversity across different conditions. These visualizations play a central role in determining whether observed biological
differences correspond to measurable shifts in microbial composition or diversity. These plots can
be viewed using QIIME 2 View at https://view.qiime2.org/.
def differential_abundance():
"""Perform differential abundance analysis using ANCOM."""
cmds = [
(
"qiime composition add-pseudocount "
"--i-table results/table.qza "
"--o-composition-table results/composition-table.qza"
),
(
"qiime composition ancom "
"--i-table results/composition-table.qza "
"--m-metadata-file meta/metadata.tsv "
"--m-metadata-column condition "
"--o-visualization results/ancom-condition.qzv"
)
]
for cmd in cmds:
run_command(cmd, "Running differential abundance with ANCOM")
305 Roles of Bacteria in Autoimmune Diseases
The differential _ abundance function is responsible for identifying features, such as
microbial taxa, that differ signicantly in abundance between sample groups. It uses ANCOM
(Analysis of Composition of Microbiomes), a statistical method designed specically for compositional microbiome data, where only relative abundances are known. Since raw count data in
microbiome studies often violate the assumptions of standard differential analysis due to their compositional nature, ANCOM uses log-ratio transformations and accounts for the fact that increasing
one taxon’s abundance implies a decrease in others. The function rst executes a command to add
a pseudocount to the feature table. This step is necessary because the ANCOM algorithm requires
all values to be strictly positive. The input results/table.qza, which contains the feature table
generated from the DADA2 denoising step, is transformed into a compositional format by adding
one to each count. The result is a new artifact results/composition-table.qza, suitable for
log-ratio analysis. Next, the function runs ANCOM using the pseudocount-adjusted table. It references the sample metadata provided in meta/metadata.tsv and uses a specic metadata column
labeled condition to dene the groups for comparison. ANCOM tests each feature for differen-
tial abundance between these groups and generates an output visualization le named results/
ancom-condition.qzv. This .qz v le provides an interactive report showing which features
are signicantly different, the strength of the evidence, and the number of pairwise log-ratio comparisons in which each feature was found to differ. By executing these steps sequentially using the
run _ command utility, the function ensures that compositional correction and statistical testing
are applied in a reproducible and interpretable way. The result is a clear and statistically grounded
view of which taxa may be biologically relevant in distinguishing between experimental conditions.
The overall script orchestrates a full amplicon-based microbiome analysis pipeline in QIIME
2. Starting from raw paired-end FASTQ sequences and metadata, it processes the data through
quality control, denoising, taxonomic classication, diversity analysis, and differential abundance
testing. It produces a rich set of outputs in .qza and .qzv formats that allow researchers to explore
microbial community composition, diversity patterns, and associations with experimental conditions. The modular structure of the script makes it easy to run, troubleshoot, and extend for various
experimental designs in microbiome studies.
9.3.2 SHOTGUN METAGENOMICS WORKFLOW
Shotgun metagenomics is a comprehensive and high-resolution approach to analyzing the composition and functional potential of microbial communities in a given environment. Unlike ampliconbased methods that target specic marker genes such as 16S rRNA, shotgun metagenomics involves
the untargeted sequencing of all genomic content within a sample. This approach provides not only
taxonomic proles but also insight into metabolic pathways, resistance genes, virulence factors, and
evolutionary relationships among organisms. The workow begins with careful sample collection,
which must minimize contamination and preserve nucleic acid integrity. Environmental or hostassociated samples, such as stool, saliva, and skin swabs, are typically stored using DNA-stabilizing
agents or frozen promptly to ensure sample delity. This is especially critical in studies investigating autoimmune diseases, where microbial dysbiosis may be subtle yet biologically meaningful.
The next step is total DNA extraction, which must be efcient and unbiased to capture the
diverse microbial taxa present, including bacteria, archaea, viruses, fungi, and even small eukaryotes. Mechanical lysis methods like bead beating are often combined with enzymatic digestion to
ensure thorough disruption of tough microbial cell walls while preventing excessive DNA shearing.
Extraction kits are selected based on the sample type and expected microbial content. Following
extraction, the DNA is quantied and assessed for purity using uorometric assays like Qubit and
spectrophotometric tools such as Nanodrop, as well as integrity analysis by agarose gel electrophoresis or capillary electrophoresis. High-quality DNA is critical for library preparation, where
fragmentation of genomic DNA into smaller pieces, typically 200–600 bp, is performed using
mechanical shearing methods such as sonication or enzymatic digestion. These fragments are then
306 Bioinformatics of Autoimmune Diseases
end-repaired, adenylated, and ligated to platform-specic sequencing adapters. Library quality is
evaluated by Bioanalyzer or TapeStation to ensure correct fragment size distribution and to detect
adapter dimers or undesired artifacts.
The prepared libraries are subjected to high-throughput sequencing using platforms such as
Illumina NovaSeq, which offers short-read sequences with deep coverage, or Oxford Nanopore and
PacBio platforms for long-read sequencing, which offer advantages in resolving complex genomes,
repeat regions, and horizontal gene transfer events. Illumina remains the most commonly used
platform due to its high accuracy, throughput, and well-developed computational pipelines. The
resulting raw sequencing reads undergo rigorous quality control using tools like FastQC to identify issues such as low base quality scores, adapter contamination, and GC content anomalies.
Trimming and ltering are then performed using software such as Trimmomatic or Cutadapt
to remove low-quality bases and adapter sequences, ensuring that downstream analyses are not
compromised by artifacts.
After quality ltering, host DNA contamination is often removed, especially in human microbiome studies, using alignment tools like Bowtie2 or BWA against the human reference genome. This
step is essential in studies involving autoimmune diseases, where microbial signals must be distinguished from host-derived sequences. The remaining reads are then analyzed for taxonomic composition and functional potential. Taxonomic proling may be performed using classication-based
approaches or assembly-based approaches. Classifcation-based approaches, such as Kraken2,
MetaPhlAn, or Centrifuge, assign reads or assembled contigs to known microbial taxa by
comparing them against curated databases of microbial genomes. These tools vary in sensitivity
and specicity, and researchers often select them based on the nature of the sample and the granularity of resolution desired. Functional proling aims to identify genes, gene families, and metabolic pathways present in the microbiome. This can be done either by direct read-based annotation
using tools like HUMAnN or MG-RAST, or by rst assembling reads into contigs using metagenomic assemblers such as MEGAHIT or metaSPAdes, followed by gene prediction using tools
like Prodigal. Functional annotations are assigned based on homology searches against databases
such as KEGG, eggNOG, or UniProt using tools like DIAMOND or BLAST. Assembly-based
approaches enable genome-resolved metagenomics, allowing the reconstruction of metagenomeassembled genomes (MAGs) using binning tools like MetaBAT, MaxBin, or CONCOCT. MAGs
can then be rened, dereplicated, and taxonomically classied using GTDB-Tk, providing insights
into previously uncharacterized microbes.
Once both taxonomic and functional proles are generated, statistical and multivariate analyses
are used to identify differences between sample groups. In studies focused on autoimmune diseases,
comparisons are made between affected and healthy cohorts to identify microbial species or genes
associated with disease onset, progression, or severity. Techniques such as differential abundance
testing (e.g., using DESeq2, ANCOM, or ALDEx2), ordination methods like PCoA, and machine
learning algorithms are employed to uncover signicant microbial signatures. These may include
enrichment of pathobionts, loss of benecial commensals, or presence of microbial gene functions
associated with inammation, immune modulation, or toxin production. For example, alterations in
microbial metabolic pathways related to tryptophan metabolism, SCFA synthesis, or LPS biosynthesis have been implicated in conditions like RA, SLE, and inammatory bowel diseases.
Importantly, shotgun metagenomics enables the exploration of the resistome (the collection of
antimicrobial resistance genes within a community), which may have implications for treatment
strategies, particularly when immunosuppressive therapies are used in autoimmune disease management. Furthermore, virulome analysis identies virulence factors and microbial genes that may
trigger or exacerbate immune dysregulation. Integration of these data with host metadata, including
clinical parameters, cytokine levels, HLA typing, and genetic susceptibility loci, allows for systemslevel insights into host–microbe interactions. Increasingly, multi-omics approaches that combine
shotgun metagenomics with metatranscriptomics, proteomics, and metabolomics are employed to
capture the dynamic interplay between microbial communities and host immune responses.
307 Roles of Bacteria in Autoimmune Diseases
The results of shotgun metagenomic analyses can be visualized through interactive dashboards,
heatmaps, network graphs, and functional pathway maps, allowing researchers and clinicians to
identify key patterns and biomarkers. Interpretation must account for technical variability, batch
effects, and biological confounders such as diet, age, medication, and geography. Therefore, robust
experimental design, inclusion of technical replicates, and appropriate normalization strategies
are crucial for reproducibility and biological interpretation. As computational tools and databases
continue to evolve, shotgun metagenomics is becoming increasingly accessible and informative,
driving discoveries in microbiome research and its implications for autoimmune pathogenesis, diagnosis, and potentially, personalized microbial therapeutics.
9.3.2.1 Classication-Based Metagenomics Approach
Classication-based metagenomics data analysis involves identifying and quantifying the taxonomic composition of microbial communities directly from raw sequencing data, without the need
to assemble genomes or gene fragments. This approach starts with raw reads obtained through
high-throughput sequencing of environmental or biological samples. These reads are subjected to
preprocessing steps, such as quality ltering and adapter trimming, to ensure that the data used in
the analysis is reliable and free of technical artifacts. Once the reads are cleaned, they are compared
against reference databases that contain known sequences from various microbial taxa, enabling the
assignment of taxonomic labels based on sequence similarity or composition-based metrics.
One of the key features of classication-based metagenomics is its reliance on established taxonomic classication tools such as Kaiju, Kraken2, Centrifuge, or MetaPhlAn. These tools utilize
different algorithms to classify reads at various taxonomic levels, including species, genus, family,
or higher ranks. Some methods perform exact k-mer matching against large reference databases,
while others translate reads into protein space to improve sensitivity and identify sequences that
may be poorly represented in nucleotide databases. The result of this classication step is a taxonomic prole that outlines the relative abundance of organisms present in the sample, usually summarized in the form of count tables or percentage abundances.
This method is particularly advantageous for large-scale or exploratory studies where researchers aim to capture a broad overview of microbial community structure, diversity, and shifts across
conditions or environments. Because it bypasses the need for computationally expensive assembly
and gene prediction steps, classication-based analysis is also relatively fast and scalable. However,
it depends heavily on the completeness and accuracy of reference databases; novel or poorly characterized organisms may be misclassied or remain unclassied. Nevertheless, this approach provides an efcient and informative strategy for examining microbial ecology, tracking community
changes, and comparing microbiomes across experimental groups, disease states, or environmental
gradients.
9.3.2.1.1 Installation of Software Packages
To set up a dedicated environment for shotgun metagenomics data analysis, the rst step is to create
a new Conda environment named shotgun-env with Python version 3.12, along with a specic
version of SQLite (3.41.2), and the scikit-bio library, which is essential for microbial diversity and
statistical analysis.
conda create -n shotgun-env python=3.12 sqlite=3.41.2 scikit-bio
conda activate shotgun-env
This environment encapsulates all necessary dependencies, helping to avoid version conicts
and ensuring reproducibility. After the environment is created, it is activated so that all subsequent
package installations occur within this isolated setup.
Following the activation of the environment, the next step involves installing essential bioinformatics tools.
308 Bioinformatics of Autoimmune Diseases
conda install -c bioconda sra-tools -y
conda install -c conda-forge biopython -y
conda install -c conda-forge mamba
conda install anaconda::scikit-learn
mamba install -c bioconda kaiju
pip install pandas matplotlib seaborn scikit-bio jinja2 plotly
The sra-tools package is installed from the Bioconda channel. This toolkit is required to
access and download sequencing data from the NCBI SRA repository, which is a common source
for raw metagenomic data. Next, biopython is installed from the c ond a-forge channel.
Biopython provides robust functionality for parsing biological le formats and interacting with
biological databases, making it vital for pre-processing and sequence handling tasks.
To further streamline package management, mamba is installed. Mamba is a high-performance
alternative to Conda for installing packages, signicantly improving installation speed and dependency resolution. With mamba in place, the k aiju taxonomic classier is installed using the
Bioconda channel. Kaiju is a key component of the analysis pipeline, allowing for the classication
of metagenomic reads against a reference database based on translated protein sequences, which is
particularly useful for complex microbial communities.
Finally, several Python packages are installed using pip to support data analysis and visualization. These include pandas for data manipulation, matplotlib and seaborn for plotting,
jinja2 for templating reports, plotly for interactive visualizations, and scikit-bio again to
ensure compatibility with the pip ecosystem. With this comprehensive set of tools, the environment
is fully prepared to handle the various stages of shotgun metagenomics data processing, from raw
data retrieval to statistical analysis and visualization.
It is important to note that the software packages used in this setup may undergo updates or
changes over time. New versions may introduce enhancements, changes in functionality, or updated
dependencies that could affect compatibility or performance. Therefore, users are strongly encouraged to consult the ofcial documentation of each package (whether available through Conda,
Bioconda, pip, or the developers’ repositories) to ensure they are using the most suitable versions
and following current best practices. Staying informed about updates and reviewing changelogs can
help prevent unexpected issues and ensure a smooth analysis workow.
9.3.2.1.2 Kaiju Taxonomic Classier
Kaiju is a fast and memory-efcient taxonomic classier designed specically for metagenomic
sequence data. Unlike nucleotide-based classiers, Kaiju operates at the protein level by translating
input DNA reads into amino acid sequences and then matching them against a database of annotated
protein sequences. This approach enhances its ability to identify microbial species, particularly
those that are distantly related or lack well-characterized nucleotide sequences. Kaiju is especially
effective in analyzing complex microbial communities from environmental or clinical samples,
offering improved sensitivity in taxonomic classication due to its protein-level search strategy.
The computational requirements of Kaiju depend largely on the size of the database being used.
While Kaiju itself is relatively lightweight and optimized for performance, its memory usage can
be signicant during classication, particularly when using the full NCBI non-redundant protein
database (nr). Running Kaiju with the full database can require upwards of 60–100 GB of RAM.
However, Kaiju also provides smaller, more manageable database options such as the “refseq” or
“minimal” databases, which are reduced in size and allow Kaiju to run on systems with more modest memory resources, though with a corresponding trade-off in classication breadth.
Kaiju databases must be downloaded and indexed separately before running the classier. Users
can obtain these databases from the ofcial Kaiju website or GitHub repository. The databases
are typically distributed as compressed archives that, once downloaded, should be extracted into
a designated directory. After extraction, they must be indexed using Kaiju’s built-in indexing tools
to ensure they are in the proper format for classication. It is important to choose a database that
309 Roles of Bacteria in Autoimmune Diseases
TABLE 9.2
Selected Samples from the NCBI BioProject PRJEB28543
# SRA Run ID Age Condition Gender
1 ERR2784696 23 MS Female
2 ERR2784697 23 MS Female
3 ERR2784698 23 MS Female
4 ERR2784716 25 MS Male
5 ERR2784717 25 MS Male
6 ERR2784718 25 MS Male
7 ERR2784728 50 Control Female
8 ERR2784729 50 Control Female
9 ERR2784730 50 Control Female
10 ERR2784736 29 Control Male
11 ERR2784737 29 Control Male
12 ERR2784738 29 Control Male
matches the scope and resolution needed for the analysis, and to store the database on a high-speed
storage device to minimize I/O bottlenecks during classication. Clear instructions for downloading and setting up the databases are provided in Kaiju’s documentation, which should be consulted
to ensure compatibility with the installed version of the software.
9.3.2.1.3 Acquiring Raw Data for Shotgun Metagenomics Analysis
In this section, we use the NCBI BioProject PRJEB28543, which includes sequencing datasets from
a study investigating the gut microbiome of treatment-naïve MS patients across different ethnic
groups. These datasets offer comprehensive insights into the microbial composition and functional potential of the gut microbiome in early-stage MS patients compared to healthy individuals.
For demonstration purposes, we use only a subset of the available data. Specically, we selected
12samples, including both healthy individuals and MS patients, with representation from both male
and female subjects. Table 9.2 presents the metadata for these samples.
Follow the steps discussed in the previous section, along with the bash script “data/raw/
download _ fastq.sh”, to download the FASTQ les from the NCBI SRA and compress them.
9.3.2.1.4 Shotgun Metagenomics Classication Pipeline
The shotgun metagenomics pipeline “shotgun _ class _ pipeline.py” is a comprehensive
Python-based workow that automates the entire analysis process, starting from raw paired-end
FASTQ les and a metadata le. It performs both taxonomic and functional proling of metagenomic samples, followed by statistical analyses and visualization, and nally produces an HTML
report that summarizes all results. This pipeline is particularly suitable for microbiome studies
where differences in microbial composition and function are being investigated across experimental
groups, such as control versus disease conditions. The pipeline assumes that the raw sequencing
data is stored in a structured directory, and that sample metadata includes information like condition and gender. It orchestrates a series of bioinformatics tools such as Kaiju for taxonomic classication and HUMAnN3 for functional proling, and integrates their outputs using Python packages
to visualize diversity, perform principal component analysis (PCA), generate heatmaps, and identify
statistically signicant differences between sample groups.
def get_sample_pairs(raw_dir):
files = glob(os.path.join(raw_dir, "*.fastq.gz"))
samples = {}
310 Bioinformatics of Autoimmune Diseases
for f in files:
base = os.path.basename(f)
sid = base.split("_")[0].split(".")[0]
if sid not in samples:
samples[sid] = [None, None]
if "_1" in base or "_R1" in base:
samples[sid][0] = f
elif "_2" in base or "_R2" in base:
samples[sid][1] = f
return samples
The get _ sample _ pairs function is responsible for identifying and organizing paired-
end FASTQ les stored in a given directory. When shotgun metagenomic sequencing is performed
using paired-end technology, each sample typically generates two sequencing les: one containing
forward reads (read 1) and the other containing reverse reads (read 2). These les are usually named
with a consistent pattern that includes IDs such as _1, _2, _R1, or _R2 to distinguish between the
two read directions. The function starts by searching the specied directory, raw _ dir, for all
les that end with the .fa s t q.gz extension. It uses the glob function to return a list of matching
le paths, which are then iterated over one by one. For each le, the base name (i.e., the lename
without its directory path) is extracted. From this base name, the sample ID is determined by splitting the name at underscores or periods and taking the rst part. This assumes that the sample ID
appears at the beginning of the lename and is consistent between the paired les. The function
maintains a dictionary called samples, where each key is a sample ID and the corresponding value
is a two-element list. The rst element will hold the path to the forward read le (read 1), and the
second element will hold the path to the reverse read le (read 2). As it processes each le, the function checks whether the lename includes a pattern indicating it is a forward read (_1 or _R1) or a
reverse read (_2 or _R2). It assigns each le to the appropriate position in the list for that sample
ID. After processing all les, the function returns the samples dictionary, which now maps each
sample ID to a tuple of two le paths: the forward and reverse FASTQ les. This output structure is
essential for downstream functions that require paired-end input, such as taxonomic classication
and functional proling, allowing the pipeline to systematically process each sample in a consistent
and automated way.
def load_metadata(meta_file):
df = pd.read_csv(meta_file)
df.columns = df.columns.str.strip()
df['runID'] = df['runID'].astype(str)
return df
The load _ metadata function is designed to read and process a metadata le that contains
sample information used throughout the shotgun metagenomics pipeline. It takes as input the path
to the metadata le (meta/metadata.csv), typically a comma-separated values (CSV) le with
rows corresponding to individual samples and columns containing attributes such as runID, condition, and gender as follows.
runID,control,gender
ERR2784696,MS,female
ERR2784697,MS,female
ERR2784698,MS,female
ERR2784716,MS,male
ERR2784717,MS,male
ERR2784718,MS,male
ERR2784728,Control,female
ERR2784729,Control,female
311 Roles of Bacteria in Autoimmune Diseases
ERR2784730,Control,female
ERR2784736,Control,male
ERR2784737,Control,male
ERR2784738,Control,male
The function begins by reading the le using pandas.read _ csv(), which loads the data
into a pandas DataFrame. This provides a structured, tabular representation of the metadata, making it easy to access and manipulate each column. To ensure consistency, the function applies d f.
columns.str.strip(), which removes any leading or trailing whitespace characters from
the column names. This step is important because metadata les created or edited manually, for
example, in Excel or Google Sheets, may contain accidental spaces that would otherwise cause
mismatches during downstream merging or ltering operations. Finally, the function converts the
runID column to a string type explicitly using astype(str). This ensures that all sample IDs
are treated as string objects, even if they are numeric. This is a critical step for matching sample
IDs with lenames and other data entries that may be handled as strings elsewhere in the pipeline.
The processed DataFrame is then returned for use in subsequent analyses, including merging with
taxonomic and functional results, grouping samples by experimental condition, and plotting. This
function ensures that the metadata is clean, standardized, and ready for reliable integration into the
larger workow.
def run_kaiju(read1, read2, output_prefix, db_path):
for i, read in enumerate([read1, read2], 1):
out_file = f"{output_prefix}_R{i}.out"
cmd = [
"kaiju",
"-t", NODES_DMP,
"-f", db_path,
"-i", read,
"-o", out_file,
"-z", THREADS
]
print(f"Running Kaiju on {read}))
result = subprocess.run(cmd, capture_output=True, text=True)
if result.returncode != 0:
print("Kaiju failed:", result.stderr)
raise RuntimeError("Kaiju failed")
The r u n _ k aiju function is responsible for executing Kaiju taxonomic classication sepa-
rately on each of the paired-end read les, one for the forward reads and one for the reverse reads. It
takes as input four arguments: read1 and read2, which are the le paths to the forward and reverse
FASTQ les for a given sample; output _ prefix, which serves as the base name for the output
les; and db _ path, which is the path to the Kaiju database index le (.f m i) to be used for clas-
sication. Inside the function, it loops over the two reads using enumerate, assigning them the sufx
_ R1.o ut and _ R2.o ut, respectively, to distinguish the outputs from forward and reverse reads.
For each read, it constructs a kaiju command using appropriate ags: -t species the taxonomy tree
le (nodes.dmp), -f points to the Kaiju database index, -i provides the input FASTQ le, -o sets
the output le name, and -z sets the number of threads for parallel processing. Before executing
the command, it prints a message indicating which read le is being processed, making it easier to
track progress. The s u bp r o ces s.r u n() function then executes the command, capturing both
the standard output and error output. If the Kaiju command returns a non-zero exit code, which
indicates failure, the function prints the error message from Kaiju and raises a RuntimeError
to stop further execution. This mechanism ensures that users are alerted if Kaiju fails on any of the
reads and can check the error details immediately. The nal output of this function is two Kaiju
312 Bioinformatics of Autoimmune Diseases
output les per sample, each containing taxonomic classications for the reads in either the forward
or reverse le.
def merge_kaiju_outputs(files):
dfs = []
for f in files:
df = pd.read_csv(f, sep="\t", header=None)
df.columns = ["read", "status", "taxon",
"tax_id", "score", "name", "add"]
sid = os.path.basename(f).split("_kaiju")[0]
df["sample"] = sid
dfs.append(df)
full_df = pd.concat(dfs)
merged = full_df.groupby(["sample", "name"]).size().
reset_index(name="count")
return merged.pivot(index="name", columns="sample", values="count").
fillna(0)
The merge _ kaiju _ outputs function is responsible for aggregating and organizing taxo-
nomic classication results from multiple Kaiju output les into a structured matrix format that can
be used for downstream analysis. Each Kaiju output le represents the classication results for one
sample, and this function takes all of those les together and produces a unied table where microbial
taxa are rows, samples are columns, and the values represent the number of reads assigned to each
taxon in each sample. The function begins by initializing an empty list named dfs to hold individual
DataFrames. It then iterates over the list of input le paths provided in the les argument. For each
le, it reads the contents using pandas.read _ csv() with tab separation and no header, since
Kaiju outputs are tab-delimited and do not include column names. After loading the data, it assigns
column names to the DataFrame: “read”, “status”, “taxon”, “tax _ id”, “score”, “name”,
and “add”, corresponding to the elds in a Kaiju output line. These elds include the read ID, classication status, the matched taxon name, taxon ID, alignment score, and some additional alignment
info. Next, the function extracts the sample ID from the le name by taking the base name of the le
and removing the sufx after _ kaiju . This sample ID is added as a new column called “sample”
to label the DataFrame with the corresponding sample. Each resulting DataFrame is then appended
to the dfs list. Once all Kaiju outputs are processed and labeled with their sample IDs, the function
concatenates the individual DataFrames into a single DataFrame called full _ df. This combined
DataFrame contains taxonomic classication results from all samples. It then uses the gr o upb y()
function to group the data by sample and name (the taxon name), and counts how many times each
taxon appears in each sample using .size(). The grouped data is reset into a at structure with
three columns: sample, name, and count. Finally, the function pivots this grouped data using the
piv o t() method, so that the taxon names become the row indices, sample IDs become the column
headers, and the cell values represent the counts of reads assigned to each taxon. Any missing values
are lled with zeros using .fil ln a(0), indicating that no reads were assigned to that taxon in that
sample. The nal output is a taxon-by-sample abundance matrix, which is crucial for diversity analysis, statistical testing, and visualization in downstream parts of the metagenomics pipeline.
def run_humann(input_fastq, sample_id, output_dir):
os.makedirs(output_dir, exist_ok=True)
cmd = [
"humann", "--input", input_fastq,
"--output", output_dir,
"--output-basename", sample_id,
"--threads", THREADS
]
subprocess.run(cmd, check=True)
313 Roles of Bacteria in Autoimmune Diseases
The run _ humann function is responsible for executing HUMAnN3, a tool used for func-
tional proling of metagenomic sequencing data. It takes three arguments: input _ fastq, which
is the path to the input FASTQ le containing the sequencing reads (usually the forward reads);
sample _ id, which serves as a unique ID for naming the output les; and output _ dir, which
species the directory where the HUMAnN3 output les will be saved. Inside the function, the
rst operation is to create the output directory if it doesn’t already exist, using os.makedirs()
with exist _ ok=True to avoid raising an error if the directory already exists. This ensures that
the results of the analysis have a designated location to be stored. Next, the function constructs a
command as a list called cmd. This command calls the humann executable with several options:
--i n p u t species the input FASTQ le to be analyzed; --output designates the directory where
the output will be stored; --output-basename sets the prex for all output lenames so they
are clearly associated with the specic sample; and --th re ads denes how many CPU threads
HUMAnN3 should use, allowing for parallel execution and faster runtime. The value of THREADS
is assumed to be dened globally in the script, usually as a string indicating the number of available
threads (e.g., “4” or “8”). Finally, the function runs this command using su b pr o c e ss.r u n(c m d,
ch e ck=T r u e). This executes the HUMAnN3 pipeline in a subprocess. If HUMAnN3 encounters an error and exits with a non-zero status, the check=True argument causes Python to raise
a CalledProcessError, which helps catch failures during execution and ensures that downstream
steps don’t proceed with incomplete or missing outputs. The output of this function is not returned
directly in Python, but it includes several les written by HUMAnN3 to disk, such as:
* a pathway abundance table,
* a gene family abundance table,
* and a stratied le that shows which microbial species contributed to which pathways or
genes.
These outputs are later used for comparative functional analysis across samples.
def merge_humann_tables(output_dir):
cmd = [
"humann_join_tables",
"--input", output_dir,
"--output", os.path.join(output_dir, "merged_pathabundance.tsv"),
"--file_name", "pathabundance"
]
subprocess.run(cmd, check=True)
The function merge _ humann _ tables(output _ dir) is designed to merge multiple
pathway abundance tables generated by HUMAnN3 into a single comprehensive le for downstream
analysis and visualization. HUMAnN3 produces separate output les for each sample when analyzing multiple metagenomic datasets, and each le contains pathway abundance information specic
to that sample. This function simplies the process of combining all those individual les into a
unied matrix where each row corresponds to a metabolic pathway and each column corresponds to
a sample. The function constructs a command to run humann _ join _ tables, which is a util-
ity provided by HUMAnN3 for joining per-sample tables into a multi-sample table. The --i n p u t
option species the directory where the individual sample les are located. The --output option
denes the name of the merged output le, in this case merged _ pathabundance.tsv, which
is stored in the same output directory. The --file _ na me parameter tells the tool to look for les
containing pathway abundance data by specifying the keyword “pathabundance” as the identi-
fying sufx. When the command is executed using subprocess.run(cmd, check=True), it
runs the humann _ join _ tables tool with the specied arguments. If the command fails for
any reason, such as missing input les or incorrect formatting, the c hec k=Tr ue ag ensures that
Соседние файлы в папке Библиотека им академика М.И. Перельмана