Добавил:
Sekretar
kiopkiopkiop18@yandex.ru
t.me/Prokururor I Вовсе не секретарь, но почту проверяю
Опубликованный материал нарушает ваши авторские права? Сообщите нам.
Вуз:
Предмет:
Файл:Ординатура / Хирургия / Библиотека им академика М.И. Перельмана / Книга_5529_Библиотеки_им_академика_М_И_Перельмана
.pdf
294 Bioinformatics of Autoimmune Diseases
FIGURE 9.2 The tree structure of input les and scripts.
Both artifacts and visualizations are tightly integrated into QIIME 2‘s provenance system,
which meticulously tracks every step of the analytical workow. This integration means that when
a user examines a .qz v visualization, they can trace back the commands and parameters used to
generate it. Similarly, when sharing .q z a or .qz v les, collaborators not only receive the data or
results but also the full analytical context. This design ensures transparency, reproducibility, and
collaboration, making QIIME 2 a powerful platform for microbiome and amplicon-based metagenomic analysis.
Each visualization is self-contained and requires no installation of QIIME 2 on the v iew er ’s
machine if using the online viewer available at https://view.qiime2.org. This makes it easy
to share .qz v les with collaborators, reviewers, or instructors who simply need to explore the
results without rerunning the analysis. Alternatively, visualizations can also be opened in a QIIME
2 environment using the following command, which launches the visualization locally in the web
browser:
qiime tools view filename.qzv
These visualizations provide essential summaries and quality checks, helping users make decisions about trimming parameters, ltering strategies, or interpreting taxonomic composition. The
interactive elements, such as dropdowns, tabs, and embedded plots, allow deeper inspection of the
data than a static gure would provide. This capacity for detailed and interactive exploration makes
.qzv les not just a record of results, but an active part of the scientic reasoning process in microbiome research.
9.3.1.5 QIIME 2 Plugins
QIIME 2 plugins are modular components that extend the functionality of the QIIME 2 framework,
allowing users to perform diverse tasks in microbiome and metagenomics analysis. Each plugin is
295 Roles of Bacteria in Autoimmune Diseases
responsible for a specic type of operation, such as sequence quality control, taxonomic classication, phylogenetic tree construction, or diversity analysis. These plugins are written in Python and
are built on top of the QIIME 2 framework, which ensures consistency in how inputs and outputs are
handled through artifacts and visualizations. Plugins communicate with the QIIME 2 core using a
type-safe system that ensures that only appropriate data is passed into each command, reducing the
chance of user error and maintaining the integrity of the analysis.
Each plugin typically wraps a powerful third-party tool or algorithm. For example, the dada2
plugin provides an interface to the DADA2 algorithm, implemented in R, for denoising sequences
and removing chimeras. Similarly, the feature-classifier plugin includes methods for taxonomic classication using machine learning, such as the Naive Bayes classier implemented through
scikit-learn. Other plugins use widely accepted tools like MAFFT for multiple sequence alignment,
FastTree for phylogenetic inference, and VSEARCH for clustering and chimera detection. This integration allows users to leverage best-in-class methods through a unied and reproducible platform.
What makes QIIME 2 plugins particularly powerful is the provenance tracking system that
records every step taken in an analysis, including the plugin, method, inputs, parameters, and outputs. This ensures that all results are fully reproducible and traceable. Users can inspect provenance
graphs to understand exactly how a result was generated, even months or years later. Furthermore,
the plugin architecture allows developers to create custom plugins that suit their specic research
needs, expanding the system’s capabilities without modifying the core framework. Through its plugin system, QIIME 2 remains exible, scalable, and extensible, enabling researchers to build workows tailored to their data and scientic questions.
9.3.1.6 QIIME 2 Denoising
Denoising in QIIME 2 is a critical step in amplicon-based metagenomic analysis that aims to correct
sequencing errors, remove chimeric sequences, and accurately infer the true biological sequences
present in a microbial community. This process is necessary because raw reads generated by highthroughput sequencing technologies, particularly those targeting 16S rRNA genes, often contain
noise in the form of PCR errors, sequencing mistakes, and artifacts. These inaccuracies can signicantly distort estimates of microbial diversity and abundance if not properly addressed. The denoising step transforms the raw sequence data into a high-quality, representative set of sequences that
can be condently used in downstream analyses.
QIIME 2 supports multiple denoising algorithms, the most widely used being DADA2 and
Deblur. Both of these methods work by identifying and correcting sequencing errors to reconstruct
ASVs, which represent exact sequences inferred to exist in the original biological sample. The qiime
dada2 denoise-paired and qiime dada2 denoise-single commands are used for
denoising paired-end and single-end reads, respectively, using the DADA2 plugin. This method
models the error rates in the sequencing data and uses these models to correct reads, detect and
remove chimeras, and infer ASVs with single-nucleotide resolution. Deblur, on the other hand, uses
a different approach that relies on a static error model and works best with single-end reads. The
command qiime deblur denoise-16S is used for Deblur-based denoising.
The outputs of these denoising processes are three essential QIIME 2 artifacts. The rst is a
feature table in .q z a format that contains the number of times each ASV was observed in each
sample. The second is a representative sequences le, which stores the actual nucleotide sequences
of the inferred ASVs. The third is a denoising statistics le, which reports information such as the
number of reads input, reads ltered, chimeras removed, and nal reads retained per sample. These
outputs form the basis for most of the downstream analyses in microbiome research, including taxonomy assignment, phylogenetic analysis, and diversity metrics.
To understand the importance of denoising, it is essential to distinguish between OTUs and
ASVs. OTUs were traditionally used to group similar sequences together, often at a 97% similarity
threshold, in an attempt to approximate species-level classication. This method, however, is sensitive to the arbitrary choice of similarity threshold and can cluster together sequences that differ
296 Bioinformatics of Autoimmune Diseases
by only a few nucleotides, potentially hiding true biological variation. In contrast, ASVs represent
exact sequence variants inferred from the data, differing by even a single nucleotide, and thus offer
much higher resolution. ASVs are also reproducible across studies because they are not dependent
on a reference database or clustering threshold, unlike OTUs. This shift from OTU-based to ASVbased analysis has become a major advancement in microbiome research, promoting more accurate,
precise, and reproducible characterization of microbial communities. Denoising, therefore, is not
merely a quality control step but a transformative part of the pipeline that denes the granularity
and reliability of the entire analysis.
9.3.1.7 The Amplicon-Based Metagenomic Pipeline
The complete amplicon-based metagenomic pipeline is saved in the qiime2 _ pipeline.py
le. In the following sections, we describe the functions included in this pipeline:
def run_command(command, description=""):
"""Run a shell command and print status."""
print(f"\n Running: {description}\n{command}")
process = subprocess.run(command, shell=True)
if process.returncode != 0:
raise RuntimeError(f"Command failed: {description}")
The run _ command function serves as a utility to execute shell commands within the Python
script. It prints a message indicating the operation being performed and the actual command, then
runs it using Python’s subprocess module. If the command fails (i.e., returns a non-zero exit code), it
raises a runtime error with a message indicating which step failed. This function ensures consistent
command execution and error reporting across the pipeline.
def import_sequences():
"""Import sequences using manifest format."""
cmd = (
"qiime tools import "
"--type 'SampleData[PairedEndSequencesWithQuality]' "
"--input-path meta/manifest.tsv "
"--output-path data/paired-end-demux.qza "
"--input-format PairedEndFastqManifestPhred33V2"
)
run_command(cmd, "Importing paired-end sequences via manifest")
The import _ sequences function initiates the QIIME 2 analysis by importing paired-end
sequence data. It uses a manifest le in TSV format, which maps sample IDs to their respective
forward and reverse FASTQ les. The command converts the raw FASTQ data into a .qza artifact
named paired-end-demux.qza stored in the data/ directory. This QIIME 2 artifact format
allows further downstream analysis using the QIIME 2 framework.
def summarize_sequences():
"""Summarize demux data to assess quality."""
cmd = (
"qiime demux summarize "
"--i-data data/paired-end-demux.qza "
"--o-visualization results/demux-summary.qzv"
)
run_command(cmd, "Summarizing demultiplexed sequences")
The summarize _ sequences function takes the demultiplexed QIIME 2 artifact pro-
duced in the previous step and summarizes the quality of the sequence data. The output is a .q z v
297 Roles of Bacteria in Autoimmune Diseases
visualization le, de mu x-s u mm ar y.q zv, that allows users to inspect quality scores across reads
and samples using QIIME 2 View. This step is crucial for assessing sequence quality and determining trimming or truncation parameters for denoising.
def denoise_dada2():
cmd = (
"qiime dada2 denoise-paired "
"--i-demultiplexed-seqs data/paired-end-demux.qza "
"--p-trim-left-f 0 "
"--p-trim-left-r 0 "
"--p-trunc-len-f 145 "
"--p-trunc-len-r 145 "
"--p-max-ee-f 3 "
"--p-max-ee-r 3 "
"--p-n-reads-learn 100000 "
"--o-table results/table.qza "
"--o-representative-sequences results/rep-seqs.qza "
"--o-denoising-stats results/denoising-stats.qza"
)
run_command(cmd, "Denoising with DADA2 for 151 bp paired-end reads")
The denoise _ dada2 function performs sequence quality control, denoising, and chimera
removal using the DADA2 algorithm tailored for 151 bp paired-end reads. It takes the demultiplexed QIIME 2 artifact paired-end-demux.qza as input and applies specic ltering parameters: no trimming from the 5′ end (--p-trim-left-f 0, --p-trim-left-r 0), and truncation
of forward and reverse reads at 145 bases (--p -t r u n c -le n-f 14 5 , --p -t r u nc -l e n -r 14 5)
to remove low-quality tails. The function also sets maximum expected errors to 3 for both forward
and reverse reads (--p-max-ee-f 3, --p-max-ee-r 3) to reduce the inuence of reads with
high error probability. A subset of 100,000 reads (--p-n-reads-learn 100000) is used to train
the error model. The outputs include a feature table (ta ble.qza) with counts of high-resolution
ASVs per sample, a representative sequence le (rep-seqs.qza) for downstream taxonomic and
phylogenetic analysis, and denoising statistics (denoising-stats.qza) for quality control and
performa nce review.
def visualize_feature_table():
cmds = [
(
"qiime feature-table summarize "
"--i-table results/table.qza "
f"--o-visualization results/table.qzv "
"--m-sample-metadata-file meta/metadata.tsv"
),
(
"qiime feature-table tabulate-seqs "
"--i-data results/rep-seqs.qza "
"--o-visualization results/rep-seqs.qzv"
),
(
"qiime metadata tabulate "
"--m-input-file results/denoising-stats.qza "
"--o-visualization results/denoising-stats.qzv"
),
]
for cmd in cmds:
run_command(cmd, "Visualizing feature table and stats")
298 Bioinformatics of Autoimmune Diseases
The visualize _ feature _ table function is responsible for generating a set of interac-
tive visualizations that provide insights into the core outputs of the denoising stage in a QIIME2
amplicon analysis pipeline. It processes the results of the DADA2 denoising step, particularly the
feature table, representative sequences, and denoising statistics, and converts them into .qzv visu-
alization les that can be interactively explored using QIIME 2 View (https://view.qiime2.org/).
The rst command in the function uses qiime feature-table summarize to create a
summary of the feature table stored in results/table.qza. This summary includes informa-
tion such as the number of features observed in each sample, total frequency per sample, and statistics about sample depth. It also uses the metadata le meta/metadata.tsv to correlate sample
statistics with metadata categories, helping users detect outliers or under-sequenced samples. The
output, results/table.qzv, is a key le for evaluating sequencing depth and data complete-
ness across samples. Figure 9.3 displays ta ble.q z v, which provides a summary of the feature
table, including the frequency per sample, frequency per feature, and histograms visualizing these
frequencies.
The second command invokes qiime feature-table tabulate-seqs on the results/
rep-seqs.qza le to generate results/rep-seqs.qzv, which is a detailed visualization of
the representative sequences. This allows users to inspect the sequence content of each unique feature, check sequence lengths, and search for specic sequences if needed. This visualization can
be used for sanity-checking that sequences are of expected lengths and format, and is also useful
for tracing specic ASVs in downstream taxonomic classication. Figure 9.4 displays re p-se qs.
qzv, which provides information about sequences.
The third command uses qiime metadata tabulate to visualize the le results/
denoising-stats.qza, which contains detailed statistics from DADA2 about read ltering,
denoising, merging, and chimera removal. The resulting le, results/denoising-stats.qzv,
allows users to explore the performance of the denoising algorithm across all samples. This visualization helps in identifying samples that may have failed during denoising or experienced substantial read loss. Figure 9.5 displays the denoising statistics table.
Altogether, this function plays a vital role in quality control and data validation. Visualizing these
three key outputs (feature abundance, sequence identity, and denoising performance) enables users
to make informed decisions before proceeding with taxonomy assignment and diversity analyses.
FIGURE 9.3 Statistics of the feature table (table.qzv).
299 Roles of Bacteria in Autoimmune Diseases
FIGURE 9.4 The sequence table (rep-seqs.qzv).
Each command is executed sequentially through the run _ command function, ensuring a
uniform process and immediate feedback in case of errors.
def assign_taxonomy():
classifier_path = "classifier/2024.09.backbone.full-length.
nb.sklearn-1.4.2.qza"
if not os.path.exists(classifier_path):
raise FileNotFoundError(
f"{classifier_path} not found. "
"Download a compatible classifier."
)
cmd = (
"qiime feature-classifier classify-sklearn "
f"--i-classifier {classifier_path} "
"--i-reads results/rep-seqs.qza "
"--o-classification results/taxonomy.qza"
)
run_command(cmd, "Assigning taxonomy")
The assig n _ ta xonomy function is responsible for classifying representative sequences by
assigning them to known taxonomic groups based on a reference database. This is a key step in
FIGURE 9.5 The denoising statistics table (denoising-stats.qzv).
300 Bioinformatics of Autoimmune Diseases
amplicon-based microbiome analysis, as it translates the anonymous sequence variants into biologically meaningful taxa, such as bacterial genera or species. The function begins by specifying the
path to a pre-trained Naïve Bayes classier in QIIME 2 Artifact format (.qza). The classier used
here, 2024.09.b ac k b o ne.f u l l-le ng t h.n b.s k le a r n-1.4.2.q z a , is trained on a comprehensive reference SILVA database and adapted to match the specic QIIME version and scikit-learn
version in use. Before proceeding, the function checks whether the classier le actually exists at
the specied path. If the le is not found, it raises a FileNotFoundError, halting the script and
prompting the user to download a compatible classier. This validation step prevents downstream
failures due to missing or incompatible classier les and ensures the analysis is reproducible with
the correct reference dataset. Once the classier le is veried, the function constructs a QIIME
2 command using qiime feature-classifier classify-sklearn, which applies the
Naïve Bayes model to the input representative sequences stored in results/rep-seqs.qza. The
output is written to results/taxonomy.qza, which is a .qza artifact containing taxonomic
labels and their associated condence scores for each ASVs. This result serves as a foundation for
later steps in the pipeline, such as taxonomy visualization and interpretation of microbial composition across different sample groups. By wrapping the command in the run _ command function,
the script ensures that this classication step is executed with error tracking and clear feedback to
the user. The successful execution of this function enables the linkage between raw sequencing data
and ecological or clinical insights by providing taxonomic identities that can be compared, visualized, and statistically analyzed.
def visualize_taxonomy():
"""Visualize taxonomy assignments."""
cmds = [
(
"qiime metadata tabulate "
"--m-input-file results/taxonomy.qza "
"--o-visualization results/taxonomy.qzv"
),
(
"qiime taxa barplot "
"--i-table results/table.qza "
"--i-taxonomy results/taxonomy.qza "
"--m-metadata-file meta/metadata.tsv "
"--o-visualization results/taxa-bar-plots.qzv"
),
]
for cmd in cmds:
run_command(cmd, "Generating taxonomy visualizations")
The visualize _ taxonomy function is responsible for creating visual representations of the
taxonomic composition of the microbial communities identied in the samples. It operates in two
main steps, each involving a QIIME 2 command that generates a .qz v visualization le. These
visualizations can be explored interactively through QIIME 2 View and are critical for interpreting
the biological relevance of the sequencing data.
The rst command uses qiime metadata tabulate to convert the t a x o n o m y.q z a le,
which contains the taxonomic classications assigned to each representative sequence, into a .qz v
le called t a x o no m y.q z v . This le displays the taxonomic assignments in a tabular, humanreadable format. Each row in the resulting table corresponds to an ASVs and lists its predicted
taxonomic lineage, condence score, and ID. This view allows users to browse and lter taxonomic
information conveniently. Figure 9.6 displays the feature ID, taxon, and condence for each ASV.
The second command generates a bar plot using qiime taxa barplot (Figure 9.7), which
visualizes the distribution of taxa across all samples. It takes the feature table (t a ble.q za), the
FIGURE 9.6 The taxonomic assignment table (taxonomy.qzv).
301 Roles of Bacteria in Autoimmune Diseases
FIGURE 9.7 Bar plot showing the distribution of taxa across all samples.
taxonomy classications (t a x o n o m y.q z a), and the sample metadata (metadata.tsv) as input.
The output, ta xa-b ar-plots.qz v, is an interactive bar chart that shows the relative abundance
of microbial taxa at different taxonomic levels (such as phylum, genus, or species) for each sample.
This plot helps researchers quickly identify dominant taxa, explore patterns of community composition, and compare taxonomic proles across experimental groups.
By iterating over the list of commands, the function ensures that both the tabular taxonomy summary and the bar plot visualization are generated and stored in the results directory. These outputs
are essential tools for summarizing and interpreting the biological identities of microbial communities inferred from 16S rRNA gene amplicon sequencing data.
def generate_phylogeny():
"""Generates phylogenetic tree required for beta diversity."""
cmds = [
(
"qiime phylogeny align-to-tree-mafft-fasttree "
"--i-sequences results/rep-seqs.qza "
302 Bioinformatics of Autoimmune Diseases
"--o-alignment results/aligned-rep-seqs.qza "
"--o-masked-alignment results/masked-alignment.qza "
"--o-tree results/unrooted-tree.qza "
"--o-rooted-tree results/rooted-tree.qza"
)
]
for cmd in cmds:
run_command(cmd, "Generating phylogenetic tree")
The generate _ phylogeny function is responsible for constructing a phylogenetic tree
from the representative sequences derived from denoised amplicon data. This tree is essential for
conducting phylogenetic diversity analyses, such as UniFrac, which require knowledge of the evolutionary relationships among microbial taxa. The function begins by preparing a single QIIME2
command that wraps a complete workow using the align-to-tree-mafft-fasttree
pipeline. This pipeline automates several steps: it rst aligns the representative sequences using
MAFFT, a multiple sequence alignment tool well-suited for microbial data due to its accuracy and
speed. Once the sequences are aligned, a masking step is performed to lter out highly variable
regions that might introduce noise or bias in the phylogenetic inference. This step ensures that only
informative parts of the alignment are used in tree building. After masking, the FastTree algorithm
is applied to build an unrooted tree based on the masked alignment. FastTree is efcient for large
datasets and produces trees that approximate maximum-likelihood phylogenies. Finally, the script
produces a rooted version of the tree, which is often required for downstream diversity metrics
that rely on a dened evolutionary direction. The outputs of this function include four QIIME2
artifacts: aligned-rep-seqs.qza, masked-alignment.qza, unrooted-tree.qza, and
rooted-tree.qza. These les represent, respectively, the multiple sequence alignment, the
alignment after masking, the inferred phylogenetic tree without a root, and the rooted phylogenetic
tree. The rooted tree, in particular, is critical for calculating phylogenetic-based diversity metrics
such as Faith’s Phylogenetic Diversity and UniFrac distances. This function, by consolidating
alignment and tree construction into one step, simplies an otherwise complex process and ensures
consistency across samples in the dataset.
To visualize the phylogenetic tree generated in QIIME 2, you can use the qiime phylogeny
view-tree plugin (for simple inspection in some QIIME 2 versions) or, more commonly, export
the tree and view it with specialized tools like iTOL, FigTree, or EMPress. QIIME 2 itself doesn’t
directly produce .q zv visualizations for trees like it does for other artifacts, but here’s how you can
do it in practice. First, export the rooted tree from the QIIME 2 artifact:
qiime tools export \
--input-path results/rooted-tree.qza \
--output-path results/trees
This will produce a le named tree.nwk (Newick format) in the results/trees di rectory.
You can then visualize it using one of the phylogenetic tree visualization programs.
def compute_diversity_metrics(sampling_depth=10000):
"""Run core diversity metrics for alpha and beta diversity."""
cmd = (
"qiime diversity core-metrics-phylogenetic "
"--i-phylogeny results/rooted-tree.qza "
"--i-table results/table.qza "
f"--p-sampling-depth {sampling_depth} "
"--m-metadata-file meta/metadata.tsv "
"--output-dir results/core-metrics-results"
)
run_command(cmd, "Computing diversity metrics")
303 Roles of Bacteria in Autoimmune Diseases
The compute _ diversity _ metrics function is designed to calculate core diversity met-
rics that characterize the microbial composition within and between samples. These metrics fall under
two broad categories: alpha diversity, which measures the richness and evenness of species within
individual samples, and beta diversity, which compares microbial community composition across samples. The function takes an optional parameter called sampling _ depth, which by default is set
to 10,000. This value represents the rarefaction depth, meaning the number of sequences randomly
sampled from each sample to standardize sequencing effort across all samples. This standardization is
necessary to make fair comparisons, as different samples often have varying numbers of reads due to
sequencing depth variability. The function constructs a QIIME 2 command that calls core-metrics-
phylogenetic, a comprehensive method that computes a variety of diversity metrics in one go. It
uses two key input les: the rooted-tree.qza, which is the phylogenetic tree built earlier using the
representative sequences, and ta ble.qza , which is the feature table that contains the frequency of
each ASVs across all samples. The presence of the phylogenetic tree allows the function to compute
phylogeny-based metrics such as Faith’s Phylogenetic Diversity and UniFrac distances, which take into
account the evolutionary relationships between microbial taxa. The command also includes the meta/
metadata.tsv le, which provides contextual information about each sample, such as treatment
group or environmental condition. This metadata is essential for later statistical testing and visualization
of group differences. The computed results are saved in a dedicated output directory called results/
core-metrics-results, where QIIME 2 stores a variety of outputs, including alpha diversity vec-
tors, beta diversity distance matrices, and PCoA ordinations. By encapsulating all core diversity com-
putations into a single step, this function streamlines a critical part of the microbiome analysis pipeline,
producing standardized and reproducible metrics that are foundational for biological interpretation.
def visualize_diversity():
"""Group significance tests and visualizations for diversity."""
cmds = [
(
"qiime diversity alpha-group-significance "
"--i-alpha-diversity results/core-metrics-results/faith_pd_vector.qza "
"--m-metadata-file meta/metadata.tsv "
"--o-visualization results/core-metrics-results/faith-pd-group-
significance.qzv"
),
(
"qiime diversity alpha-group-significance "
"--i-alpha-diversity results/core-metrics-results/evenness_vector.qza "
"--m-metadata-file meta/metadata.tsv "
"--o-visualization results/core-metrics-results/evenness-group-
significance.qzv"
),
(
"qiime diversity beta-group-significance "
"--i-distance-matrix results/core-metrics-results/unweighted_unifrac_
distance_matrix.qza "
"--m-metadata-file meta/metadata.tsv "
"--m-metadata-column condition "
"--o-visualization results/core-metrics-results/unweighted-unifrac-
condition.qzv "
"--p-pairwise"
),
(
"qiime emperor plot "
"--i-pcoa results/core-metrics-results/unweighted_unifrac_pcoa_
results.qza "
Соседние файлы в папке Библиотека им академика М.И. Перельмана