Добавил:
Sekretar
kiopkiopkiop18@yandex.ru
t.me/Prokururor I Вовсе не секретарь, но почту проверяю
Опубликованный материал нарушает ваши авторские права? Сообщите нам.
Вуз:
Предмет:
Файл:Ординатура / Хирургия / Библиотека им академика М.И. Перельмана / Книга_5529_Библиотеки_им_академика_М_И_Перельмана
.pdf
314 Bioinformatics of Autoimmune Diseases
an exception is raised and the pipeline is halted to prevent further processing on incomplete data.
The result of this function is a merged tab-separated le where pathways are listed as rows and
samples as columns, with values representing normalized abundance levels. This le is essential for
comparative functional analysis, heatmap visualization, and statistical interpretation of metabolic
pathway activity across the metagenomic dataset.
def plot_taxonomic_abundance(count_df, metadata, output_dir):
df = count_df.T.reset_index().melt(id_vars="index")
df.columns = ["sample", "taxon", "count"]
df = df.merge(metadata, left_on="sample", right_on="runID")
plt.figure(figsize=(14, 6))
sns.barplot(data=df, x="taxon", y="count",
hue="condition", errorbar=None)
plt.xticks(rotation=90)
plt.tight_layout()
plt.savefig(os.path.join(output_dir, "taxonomic_abundance.png"))
The plot _ taxonomic _ abundance function is responsible for generating a visual sum-
mary of microbial taxonomic composition across all samples using a grouped bar plot. It takes
three inputs: count _ df, which is a DataFrame containing taxonomic counts per sample (with
taxa as rows and sample IDs as columns); metadata, which is a DataFrame containing descriptive
information about each sample, including at least the sample ID and the experimental condition;
and output _ dir, which species the directory where the output image should be saved. The
function begins by transposing the count _ df DataFrame so that samples become rows and taxa
become columns. It then resets the index so that the sample IDs, which are originally column labels,
become a column named “index”. This reshaped DataFrame is converted into long-form using the
m el t() function, so that each row represents a single observation of a taxon’s count in a particular
sample. The id _ vars="index" argument ensures that the sample IDs are preserved while the
taxon columns are transformed into two new columns: one for the taxon names and one for the
associated count values. The column names are then renamed to [“sample”, “taxon”, “count”] for
clarity and consistency. Next, the long-form data is merged with the metadata based on the sample
IDs. This step adds contextual information such as experimental condition (e.g., MS or Control) to
each row in the melted DataFrame, enabling color-based grouping in the nal plot. The function
then initializes a new gure using Matplotlib and creates a bar plot using Seaborn, where the x-axis
represents taxonomic groups, the y-axis represents the count of reads assigned to each taxon, and
the bars are colored by condition. Setting errorbar=None ensures that no condence intervals
are shown, which can be useful when dealing with count data that isn’t necessarily normally distributed. The x-axis labels (taxon names) are rotated by 90° to make them readable, especially when
there are many taxa with long names. The t ig ht _ l ay o ut() function is called to automatically
adjust spacing and prevent clipping of labels or legends. Finally, the plot is saved as a PNG (Portable
Network Graphics) le named taxonomic _ abundance.png in the specied output directory.
This visualization provides a clear, comparative overview of the most abundant taxa across different
sample groups and is especially useful for identifying dominant microbial members and visually
assessing community shifts between experimental conditions.
def plot_shannon_diversity(count_df, metadata, output_dir):
counts = count_df.T
counts.index.name = "sample"
shannon = alpha_diversity("shannon", counts.values, ids=counts.index)
df = pd.DataFrame({"sample": shannon.index, "shannon": shannon.
values})
df = df.merge(metadata, left_on="sample", right_on="runID")
plt.figure(figsize=(6, 4))
315 Roles of Bacteria in Autoimmune Diseases
sns.boxplot(data=df, x="condition", y="shannon", hue="gender")
plt.title("Shannon Diversity by Condition and Gender")
plt.tight_layout()
plt.savefig(os.path.join(output_dir, "shannon_diversity.png"))
The plot _ shannon _ diversity function is designed to calculate and visualize the alpha
diversity of microbial communities using the Shannon diversity index. Alpha diversity reects the
richness and evenness of species (or taxa) within each individual sample. This particular function
takes three inputs: count _ df, which is a DataFrame of taxonomic counts with taxa as rows and
samples as columns; metadata, which is a DataFrame containing sample-level metadata such
as experimental condition and gender; and output _ dir, the directory where the resulting plot
will be saved. The function begins by transposing the count matrix so that samples become rows
and taxa become columns. This reorientation is necessary because the alpha _ diversity
function from the scikit-bio library expects input in this format, where each row represents a
sample and each column represents a taxonomic feature. The index of the transposed DataFrame
is explicitly labeled “sample” to clarify its role in downstream merging with the metadata. Next,
the alpha _ diversity function is called with the method set to “shannon”, which computes
the Shannon index for each sample based on its taxon abundances. This index accounts for both
the number of taxa present and their relative proportions, offering a measure of community complexity. The resulting object is a series of diversity values indexed by sample ID. The function then
constructs a new DataFrame with two columns: one for the sample ID and one for its corresponding Shannon index. This DataFrame is merged with the original metadata table using the runID
column as a key, allowing the function to annotate each sample’s diversity score with its condition
and gender. With the merged data prepared, the function creates a boxplot using Seaborn, displaying Shannon diversity on the y-axis and experimental condition on the x-axis. The hue parameter
is set to “gender”, so that samples are further differentiated within each condition group by gender,
using color coding. The gure is sized for clarity, and the layout is tightened to avoid overlapping
elements. Finally, the gure is saved as a PNG image in the specied output directory under the lename “shannon _ diversity.png”. This visualization enables researchers to easily compare
the microbial diversity across conditions and genders, and to observe any patterns or differences
that might be biologically signicant.
def plot_pca(count_df, metadata, output_dir):
df = count_df.T
df = df.loc[metadata["runID"]]
pca = PCA(n_components=2)
components = pca.fit_transform(df)
pc_df = pd.DataFrame(components, columns=["PC1", "PC2"])
pc_df["runID"] = df.index
pc_df = pc_df.merge(metadata, on="runID")
fig = px.scatter(
pc_df, x="PC1", y="PC2", color="condition", symbol="gender",
hover_data=["runID"], title="PCA of Taxonomic Profiles"
)
fig.write_html(os.path.join(output_dir, "pca_taxa.html"))
The plot _ pca function performs PCA on the taxonomic abundance data from metagenomic
samples and generates an interactive scatter plot to visualize the similarities or differences between
microbial community compositions across samples. It takes three inputs: count _ df, which is
a DataFrame containing taxon counts where rows represent taxa and columns represent samples;
metadata, which is a DataFrame with sample metadata including runID, condition, and gender;
and output _ dir, the directory where the PCA plot will be saved. At the beginning of the func-
tion, the taxonomic count DataFrame is transposed so that rows correspond to samples and columns
316 Bioinformatics of Autoimmune Diseases
correspond to taxa, which is the required format for PCA. The samples are then reordered to match
the order in the metadata le using the list of runID values. This ensures that metadata such as
condition and gender can later be correctly aligned with each sample’s PCA coordinates. PCA is
then performed using the PCA class from scikit-learn, reducing the high-dimensional microbial
abundance data to two principal components, PC1 and PC2, which capture the most variance in
the data. The resulting components are stored in a new DataFrame with two columns, “PC1” and
“PC2”, and the sample IDs are added as a new column named “runID”. This DataFrame is then
merged with the metadata so that each point in the PCA plot has its associated condition and gender information. The merged data is passed to Plotly Express to generate an interactive scatter plot
where each point represents a sample, and its position reects the overall structure of its microbial community. The color of each point indicates the experimental condition (such as “MS” or
“Control”), and the shape of the marker reects the sample’s gender. Additional information, such
as the sample ID, is available on hover. Finally, the interactive PCA plot is saved as an HTML le
named pca _ tax a.html in the specied output directory. This le can be opened in any web
browser, allowing users to explore patterns in the data, identify clusters of similar microbial communities, and assess whether microbial proles separate clearly between experimental groups. This
visual output is particularly valuable for hypothesis generation and understanding the structure of
microbiome datasets.
def plot_top_pathway_heatmap(humann_path, metadata, output_dir, top_n=20):
df = pd.read_csv(humann_path, sep="\t", index_col=0)
df = df.loc[df.index.str.startswith("UNMAPPED") == False]
df = df.drop(columns=["UNMAPPED"], errors='ignore')
top_pathways = df.sum(axis=1).nlargest(top_n).index
top_df = df.loc[top_pathways]
top_df.columns.name = "runID"
top_df =
top_df.T.reset_index().merge(metadata, on="runID").set_index("runID")
heatmap_data = top_df[top_pathways]
plt.figure(figsize=(12, 8))
sns.heatmap(heatmap_data.T, cmap="viridis",
cbar_kws={"label": "Abundance"})
plt.title("Top Functional Pathways (HUMAnN)")
plt.tight_layout()
plt.savefig(os.path.join(output_dir, "top_pathways_heatmap.png"))
The plot _ top _ pathway _ heatmap function is designed to visualize the most abundant
functional pathways across a set of metagenomic samples using output from HUMAnN3. It creates
a heatmap showing the relative abundance of the top metabolic pathways identied across samples
and uses metadata to ensure sample labels and groupings are preserved and meaningful. The function begins by reading the merged HUMAnN3 pathway abundance le, which is typically in a
tab-delimited format with pathways as rows and sample IDs as columns. The le is loaded into a
DataFrame, with the rst column used as the index. The function then removes any rows that start
with the string “UNMAPPED”, which represent reads that could not be assigned to any known
pathway. It also attempts to drop the “UNMAPPED” column, if it exists, to ensure only biologically relevant pathways are retained in the analysis. Next, it calculates the total abundance of each
pathway across all samples by summing along each row. It selects the top n pathways (by default
20) with the highest total abundance. These top pathways are used to subset the original DataFrame,
resulting in a reduced matrix containing only the most relevant functional features. The column
names are relabeled as “runID” to match the sample IDs used in the metadata, and the DataFrame is
transposed so that samples are rows and pathways are columns. The transposed data is merged with
the metadata on the sample ID column runID. This merge ensures that sample labels are consistent
317 Roles of Bacteria in Autoimmune Diseases
and that downstream groupings and analyses can include additional metadata elds like condition
or gender, if needed. After merging, the DataFrame is re-indexed based on runID, and the top pathway columns are extracted for visualization. The function then creates a heatmap using the seaborn
library, which shows the abundance of each of the top pathways across all samples. The heatmap
uses the “viridis” color map and includes a color bar to represent abundance values. The nal gure
is titled “Top Functional Pathways (HUMAnN)”, and layout adjustments are made to prevent label
overlap. The resulting image is saved as a PNG le in the specied output directory. This heatmap
allows researchers to quickly spot pathway enrichment patterns across samples, helping to link
microbial functional activity with experimental groups or phenotypes.
def analyze_differential_abundance(count_df, metadata, output_dir):
group_map = metadata.set_index("runID")["condition"].to_dict()
control = [c for c in count_df.columns if group_map.get(c, '').
lower() == "control"]
treated = [c for c in count_df.columns if group_map.get(c, '').
lower() == "ms"]
result = []
for taxon in count_df.index:
ctrl_vals = count_df.loc[taxon, control]
treat_vals = count_df.loc[taxon, treated]
stat, pval = ttest_ind(ctrl_vals, treat_vals, equal_var=False)
result.append((taxon, pval))
df_result = pd.DataFrame(result,
columns=["taxon", "p_value"]).sort_values("p_value")
df_result.to_csv(os.path.join(output_dir, "differential_abundance.csv"),
index=False)
The function analyze _ differential _ abundance is designed to statistically compare
the abundance of microbial taxa between two experimental groups (specically, a control group
and a treatment group labeled “ms” (for MS or a similar condition)) based on the information provided in the metadata. This type of analysis is central to metagenomic studies that aim to identify
microbial taxa whose presence or abundance signicantly differs between conditions, which may
suggest a biological association with the disease or treatment under study. The function begins
by extracting the mapping between sample IDs and their corresponding experimental condition
from the metadata DataFrame. It creates a dictionary called group _ map where each sample’s
runID is mapped to its associated condition. This mapping allows the function to determine which
columns in the taxonomic count matrix belong to the control group and which to the treated (MS)
group. It uses list comprehensions to extract lists of sample IDs corresponding to each group, checking the condition eld in a case-insensitive manner. Next, the function iterates over each taxon in
the count DataFrame’s index. For every taxon, it retrieves the abundance values across all control
samples and all treated samples by selecting the appropriate columns. These values are stored in
two separate arrays: ctrl _ vals and treat _ vals. The function then performs Welch’s t-test
using the ttest _ ind function from the s c i p y.s t at s library. Welch’s t-test is used instead of
Student’s t-test because it does not assume equal variance between the two groups, making it more
appropriate for real biological data where variance can differ widely. For each taxon tested, the
function stores the taxon name and the resulting p-value in a list named result. Once all taxa have
been tested, it converts the results list into a new pandas DataFrame with two columns: taxon and
p _ value. This DataFrame is then sorted in ascending order of p-value, so the most statistically
signicant taxa appear at the top. Finally, the function writes this sorted DataFrame to a CSV le
named differential _ abundance.csv in the specied output directory. This output serves
as a key result in the pipeline, allowing researchers to identify and prioritize taxa that differ most
signicantly between the experimental conditions.
318 Bioinformatics of Autoimmune Diseases
def analyze_gender_stratified_differential(count_df, metadata, output_dir):
results = []
for gender in metadata["gender"].unique():
meta_sub = metadata[metadata["gender"] == gender]
group_map = meta_sub.set_index("runID")["condition"].to_dict()
ctrl = [c for c in count_df.columns if group_map.get(c, "").
lower() == "control"]
treat = [c for c in count_df.columns if group_map.get(c, "").
lower() == "ms"]
for taxon in count_df.index:
ctrl_vals = count_df.loc[taxon, ctrl]
treat_vals = count_df.loc[taxon, treat]
if len(ctrl_vals) >= 2 and len(treat_vals) >= 2:
stat, pval = ttest_ind(ctrl_vals, treat_vals,
equal_var=False)
results.append((taxon, gender, pval))
df = pd.DataFrame(results, columns=["taxon", "gender", "p_value"])
df.to_csv(os.path.join(output_dir, "gender_stratified_diff.csv"),
index=False)
The analyze _ gender _ stratified _ differential function is designed to per-
form taxon-level differential abundance analysis stratied by gender. This means that it analyzes
microbial taxonomic differences between two conditions (such as disease versus control) within
each gender group separately, allowing researchers to identify taxa that are differentially abundant
specically in males or females. The function takes as input a taxonomic count matrix count _
df, where rows represent taxa and columns represent sample IDs, along with a metadata DataFrame
containing sample annotations such as runID, condition, and gender. It also requires an output _
dir to save the results. The function begins by initializing an empty list called results, which
will store the outcome of statistical tests. It then loops over each unique gender in the metadata,
effectively splitting the dataset by gender category. For each gender group, it lters the metadata to
include only those samples belonging to that gender, producing a sub-metadata DataFrame called
meta _ sub. From this, it creates a dictionary group _ map mapping sample IDs to their respective conditions (such as “Control” or “MS”). Using this mapping, it builds two lists: one for control
samples (ctrl) and another for treated or disease samples (treat). These lists are composed of sample
IDs in the count matrix that match the gender-specic control or treatment labels, accounting for
case-insensitive text. Next, the function iterates through each taxon in the count matrix. For each
taxon, it retrieves the read count values associated with the control and treatment samples within
the current gender. If there are at least two samples in both groups, which is a basic requirement for
a meaningful statistical comparison, it performs an independent two-sample t-test (ttest _ ind)
using Welch’s correction for unequal variance. This test evaluates whether the mean abundance of
that taxon differs signicantly between the control and treatment groups for the given gender. The
result of each t-test, which includes the taxon name, the gender group, and the calculated p-value,
is appended to the results list. After all gender groups and taxa have been processed, the accumulated results are converted into a pandas DataFrame with columns taxon, gender, and p _ value.
Finally, this DataFrame is written to a CSV le named gender _ stratified _ diff.csv in
the specied output directory. This output enables researchers to investigate gender-specic microbial patterns and determine whether certain taxa are differentially enriched in males or females,
potentially revealing sex-based biological differences in microbiome-associated conditions.
def generate_html_report(output_dir):
env = Environment(loader=FileSystemLoader("."))
template_str = """
<html>
319 Roles of Bacteria in Autoimmune Diseases
<head><title>Shotgun Metagenomics Report</title></head>
<body>
<h1>Shotgun Metagenomics Analysis Report</h1>
<ul>
<li><a href="taxonomic_abundance.png">Taxonomic Abundance</a></li>
<li><a href="shannon_diversity.png">Shannon Diversity Plot</a></li>
<li><a href="pca_taxa.html">PCA of Taxonomic Profiles</a></li>
<li><a href="top_pathways_heatmap.png">Top Pathway Heatmap</a></li>
<li><a href="differential_abundance.csv">Differential Abundance
Table</a></li>
<li><a href="humann/merged_pathabundance.tsv">Merged HUMAnN
Pathways</a></li>
</ul>
</body>
</html>
"""
with open(os.path.join(output_dir, "report.html"), "w") as f:
f.write(template_str)
The generate _ html _ report(output _ dir) function is responsible for automatically
generating a simple, static HTML summary page that compiles and links together the key outputs
produced by the shotgun metagenomics pipeline. This function is designed to provide users with
a convenient way to view and access the results through a web browser, without needing to dig
through the le system manually. Internally, the function uses the jinja2 templating engine to prepare the HTML content. However, in this particular implementation, it constructs the HTML as a
raw string instead of using an external template le. The template_str variable holds the full HTML
markup for the report. This includes a <head> section for the page title and a <body> section that
introduces the report with a heading and a list of links. Each list item contains a hyperlink ( tag)
pointing to a result le generated earlier in the pipeline, such as the taxonomic abundance plot,
Shannon diversity plot, PCA visualization, heatmap of functional pathways, the differential abundance CSV table, and the merged HUMAnN pathway abundance le. The HTML is written to a le
named report.html located in the specied output_dir, which is typically the results directory used
throughout the pipeline. This le can then be opened in any standard web browser to conveniently
explore the outcome of the analysis in a centralized, user-friendly format. The HTML structure is
intentionally lightweight and clean to ensure compatibility across systems and fast loading without
any dependencies on JavaScript or CSS libraries. By summarizing the visualizations and data tables
into one navigable page, this function improves the interpretability and accessibility of the shotgun
metagenomics pipeline output.
def main():
os.makedirs(OUTPUT_DIR, exist_ok=True)
metadata = load_metadata(META_FILE)
samples = get_sample_pairs(RAW_DIR)
kaiju_outputs = []
for sample_id, (r1, r2) in samples.items():
if not r1 or not r2:
continue
kaiju_out = os.path.join(OUTPUT_DIR, f"{sample_id}_kaiju.out")
run_kaiju(r1, r2, kaiju_out, KAIJU_DB)
kaiju_outputs.append(kaiju_out)
kaiju_table = merge_kaiju_outputs(kaiju_outputs)
plot_taxonomic_abundance(kaiju_table, metadata, OUTPUT_DIR)
plot_shannon_diversity(kaiju_table, metadata, OUTPUT_DIR)
plot_pca(kaiju_table, metadata, OUTPUT_DIR)
320 Bioinformatics of Autoimmune Diseases
analyze_differential_abundance(kaiju_table, metadata, OUTPUT_DIR)
analyze_gender_stratified_differential(kaiju_table, metadata, OUTPUT_DIR)
for sample_id, (r1, r2) in samples.items():
if not r1:
continue
run_humann(r1, sample_id, HUMANN_OUTPUT)
merge_humann_tables(HUMANN_OUTPUT)
merged_path = os.path.join(HUMANN_OUTPUT, "merged_pathabundance.tsv")
plot_top_pathway_heatmap(merged_path, metadata, OUTPUT_DIR)
generate_html_report(OUTPUT_DIR)
The m ain() function serves as the central controller of the entire shotgun metagenomics data
analysis pipeline. It organizes the execution of all major tasks, from loading input data to producing nal reports. Its primary role is to ensure that all parts of the pipeline are executed in a logical
sequence and that the intermediate and nal outputs are generated and stored correctly. The function begins by ensuring that the output directory exists using os.makedirs(), which prepares a place
to store all results, such as gures, tables, and log les. It then loads the sample metadata from a
CSV le using the load_metadata() function. This metadata contains information such as sample
IDs, experimental conditions (e.g., MS versus Control), and gender, which are later used to group
and analyze the samples. It also retrieves paired-end sample le paths using get_sample_pairs(),
which looks in the raw data directory and matches forward and reverse FASTQ les for each sample. After initializing a list to collect Kaiju output le paths, the function enters a loop over all
sample pairs. For each sample, if both forward and reverse read les are found, it constructs a le
path for the Kaiju output and then calls run_kaiju() to perform taxonomic classication. The result
of each Kaiju run is appended to the kaiju_outputs list, which accumulates all classication outputs
for later analysis. Once all samples are processed, the Kaiju output les are merged into a single
abundance table using merge_kaiju_outputs(). This table contains read counts assigned to microbial
taxa across all samples, with rows representing taxa and columns representing samples. The next
set of commands performs exploratory data analysis and statistical evaluations on the taxonomic
data. plot_taxonomic_abundance() generates a barplot showing the distribution of microbial taxa
grouped by condition, while plot_shannon_diversity() computes and visualizes Shannon diversity
indices to evaluate within-sample diversity, grouped by both condition and gender. Then, plot_pca()
reduces the high-dimensional taxonomic abundance data to two principal components and plots the
samples in an interactive scatter plot, allowing for the visualization of clustering patterns among
the samples. Following this, analyze_differential_abundance() performs statistical tests to nd taxa
that are differentially abundant between conditions. Additionally, analyze_gender_stratied_differential() carries out similar tests separately for male and female subgroups, helping to identify
sex-specic microbial signatures. The function then shifts to functional proling using HUMAnN3.
It loops through the same sample list again, this time running HUMAnN3 on each forward read le
using the run_humann() function. This step identies metabolic pathways and gene families in each
sample. Once all samples have been processed by HUMAnN3, the merge_humann_tables() function consolidates individual pathway abundance tables into a single merged table. From this merged
table, plot_top_pathway_heatmap() visualizes the most abundant metabolic pathways across all
samples using a heatmap, which helps in understanding the functional differences between microbial communities. Finally, the generate_html_report() function is called to create a single HTML
report that links all visualizations and results. This report serves as a user-friendly summary of the
entire analysis, allowing researchers to review outputs and navigate through the gures and tables
with ease. Altogether, the main() function coordinates a complete, end-to-end workow for shotgun
metagenomics analysis, enabling users to go from raw sequence data and metadata to a detailed
set of visual and statistical outputs that reveal both taxonomic and functional characteristics of the
microbial communities under study.
321 Roles of Bacteria in Autoimmune Diseases
9.3.2.2 Assembly-Based Metagenomics Approach
Assembly-based metagenomics data analysis involves reconstructing longer contiguous DNA
sequences, known as contigs or scaffolds, from raw sequencing reads obtained directly from environmental samples. Unlike read-based approaches, which rely on mapping short reads to reference databases, assembly-based methods aim to piece together entire microbial genomes or large
genomic fragments. This process begins with quality control and preprocessing of the raw pairedend FASTQ les to remove low-quality bases, adapters, and contaminants. Once the reads are
cleaned, assemblers such as MEGAHIT or metaSPAdes are used to stitch overlapping sequences
into contiguous regions. These assemblers are designed specically to handle the complexity and
diversity of metagenomic datasets, which often include sequences from hundreds or thousands of
different microbial species.
The resulting contigs are then subjected to binning, a process that clusters sequences into groups
believed to originate from the same organism. Binning can be achieved through unsupervised methods based on sequence composition and coverage, or through supervised approaches using reference
genomes. Tools like MetaBAT2, MaxBin, or CONCOCT are commonly used for this step. The
binned genomes, referred to as MAGs, provide insights into the structure and diversity of microbial
communities at the genome level, enabling the study of uncultured and novel organisms.
Following binning, taxonomic and functional annotation is performed to identify the organisms
present and their potential roles in the community. This involves aligning assembled sequences
against databases such as GTDB, NCBI RefSeq, or UniProt for taxonomy, and KEGG, eggNOG, or
Pfam for functional annotation. These annotations allow researchers to infer metabolic pathways,
resistance genes, and ecological functions encoded within the metagenome. The assembled and
annotated data can be used for downstream analyses, including comparative genomics, ecological
modeling, and the study of host–microbe interactions.
Assembly-based metagenomics is computationally intensive, requiring large memory and processing capabilities due to the complexity of the assembly and binning processes. However, it
offers signicant advantages in resolving strain-level diversity and detecting novel genes and pathways that are missed by read-based methods. It enables a more comprehensive understanding of
microbial ecosystems, particularly in complex environments such as soil, marine systems, and the
human gut.
9.3.2.2.1 Software Requirements for Assembly-Based Metagenomics
The assembly-based shotgun metagenomics pipeline described here relies on a combination of bioinformatics software packages and Python libraries to perform quality control, assembly, binning,
gene prediction, functional annotation, pathway reconstruction, and statistical analysis. These tools
are predominantly available through the Bioconda and Conda Forge channels, which provide precompiled bioinformatics software in a reproducible manner. To ensure compatibility and manage
dependencies effectively, it is strongly recommended to create a dedicated Conda environment for
running this pipeline. The required software tools span several categories, each performing a distinct function within the pipeline.
For quality control of raw sequencing reads, FastQC is used to assess sequence quality metrics such as per-base quality, GC content, and overrepresented sequences. This tool is essential
for identifying any major problems with the raw data before proceeding to downstream analysis.
MEGAHIT is used for metagenomic assembly of paired-end reads into contigs. It is a memoryefcient assembler optimized for large-scale metagenomic datasets. Following assembly, Burrows–
Wheeler aligner (BWA) is employed to map reads back to contigs, which is a necessary step for
generating coverage information used by the binning software. SAMtools is used in conjunction
with BWA to convert, sort, and index the alignment les.
For binning, MetaBAT2 is employed to cluster contigs into MAGs based on coverage and sequence
composition. This step enables the reconstruction of microbial genomes from the metagenomic
322 Bioinformatics of Autoimmune Diseases
data. Prodigal is used for gene prediction on assembled contigs. It identies open reading frames
(ORFs) and produces amino acid sequences of predicted genes, which are then used in downstream
functional annotation. DIAMOND is used to rapidly align protein sequences against a reference
protein database such as UniProt or KEGG. This tool provides a fast alternative to BLAST, making
it suitable for high-throughput metagenomic annotation. For pathway reconstruction, MinPath is
used to infer metabolic pathways from lists of KEGG Orthology IDs obtained through DIAMOND
annotations. It applies a parsimony-based algorithm to minimize the number of pathways required
to explain the gene content.
In addition to the core bioinformatics tools, several Python libraries are required to perform data
handling, visualization, and statistical analysis. These include pandas for tabular data manipulation, matplotlib and seaborn for plotting, scikit-learn for dimensionality reduction such as PCA, and
scikit-bio for ecological diversity calculations. The standard Python libraries such as os, subprocess,
and glob are also used to manage le paths and execute shell commands.
To install the required software, users should rst install Miniconda or Anaconda. Once Conda
is available, a new environment can be created and activated using the command:
conda create -n metagenomics_env python=3.10
After activating the environment with conda activate metagenomics _ env, the required
packages can be installed from Bioconda and Conda Forge. The command to install the bioinformatics tools is:
conda install -c bioconda -c conda-forge \
fastqc megahit bwa samtools \
metabat2 prodigal diamond minpath
The Python plotting and analysis libraries can be installed with:
pip install pandas matplotlib seaborn scikit-learn scikit-bio
This setup ensures that all necessary software components are installed and congured in a consistent environment, enabling the successful execution of the complete assembly-based metagenomics
pipeline.
9.3.2.2.2 Annotation with DIAMOND
In the assembly-based metagenomics pipeline, we will use a uniprot.dmnd le in the functional
annotation of predicted protein sequences. After assembling the metagenomic contigs from raw
sequencing data and predicting ORFs using a gene-calling tool like Prodigal, the pipeline produces
a FASTA le containing amino acid sequences of all predicted proteins from each sample. At this
stage, these sequences are essentially anonymous; they lack any biological meaning unless they
can be compared against a database of known proteins. The uniprot.dmnd le is a DIAMOND-
formatted version of the UniProt database, which contains curated and computationally annotated
protein sequences along with their functional classications. By aligning the predicted proteins
against this database using the DIAMOND tool, the pipeline can determine which known proteins
(and their associated functions) the predicted sequences most closely resemble. This homologybased mapping provides each sequence with a functional label, typically in the form of a UniProt
ID, KEGG Orthology (KO) term, or protein name, depending on the content of the database used.
The result of this alignment step is a tabular output that lists matches between each predicted
protein and its closest known counterpart in the UniProt database. These matches are then counted
and used to build a sample-by-function abundance matrix, which quanties the functional potential
of each microbial community. This matrix is foundational for downstream analyses, such as pathway reconstruction, heatmap generation, diversity measurement, and statistical testing to identify
differentially abundant functions between experimental groups. In this context, uniprot.dmnd
provides the reference knowledge that enables a functional interpretation of the assembled and
annotated metagenomes. Without this step, the pipeline would yield sequences but no biological
insight into what those genes actually do.
323 Roles of Bacteria in Autoimmune Diseases
To download and prepare the UniProt database in DIAMOND format (uniprot.dmnd) for functional annotation, follow these steps:
9.3.2.2.2.1 Download the UniProt Protein FASTA For comprehensive annotation, download
the UniRef90 or Swiss-Prot database:
wget https://ftp.uniprot.org/pub/databases/uniprot/uniref/uniref90/
uniref90.fasta.gz
gunzip uniref90.fasta.gz
Alternatively, you can download the smaller, manually curated Swiss-Prot database:
wget https://ftp.uniprot.org/pub/databases/uniprot/current_release/
knowledgebase/complete/uniprot_sprot.fasta.gz
gunzip uniprot_sprot.fasta.gz
9.3.2.2.2.2 Create the DIAMOND Database For UniRef90:
diamond makedb --in uniref90.fasta -d uniport
For Swiss-Prot:
diamond makedb --in uniprot_sprot.fasta -d uniport
This will produce u n ip or t.dm nd that you can use in the pipeline for annotation.
9.3.2.2.3 The FASQ Files and Metadata
In this assembly-based shotgun metagenomics pipeline, we use the same raw paired-end FASTQ
les and metadata that were described earlier in the classication-based pipeline. The FASTQ les
are stored in the dat a/raw directory and represent metagenomic sequencing data from multiple
biological samples. Each sample is represented by a pair of forward and reverse reads, and the le
naming convention allows the pipeline to identify and process them as matched pairs. These raw
reads are the foundational input used throughout the entire workow, beginning with quality control and continuing through to contig assembly and functional analysis.
The metadata le used here is also the same as in the previous approach. It is stored in meta/
metadata.csv and contains three key columns: runID, condition, and gender. The runID column
includes sample IDs that match the FASTQ le names, while the condition column distinguishes
between samples from individuals with MS and those from control subjects. The gender column
indicates whether each sample is from a male or female subject. This metadata is used in the later
stages of the pipeline for stratied statistical analysis and visualizations, enabling us to compare
microbial functional proles between experimental groups and to explore gender-based differences
in microbial activity. By keeping the input data consistent between the classication-based and
assembly-based pipelines, we ensure comparability of results while leveraging the strengths of both
analytical approaches.
9.3.2.2.4 The Assembly-Based Metagenomics Pipeline
The assembly-based shotgun metagenomics pipeline “shotgun _ assem ble _ pipeline.
py” is a comprehensive Python-based program designed to process raw paired-end sequencing
data and produce taxonomic and functional proles through genome-resolved analysis. This pipeline orchestrates the complete workow, starting from raw FASTQ les, moving through assembly
and binning, gene prediction, functional annotation, and pathway reconstruction, and ending with
abundance matrix generation, statistical analysis, and data visualization. It is tailored for microbiome research involving complex communities, such as in studies of autoimmune diseases, where
understanding the metabolic and genomic composition of microbial ecosystems can reveal key
biological insights. The program integrates widely used bioinformatics tools, including MEGAHIT
Соседние файлы в папке Библиотека им академика М.И. Перельмана