Добавил:
Sekretar
kiopkiopkiop18@yandex.ru
t.me/Prokururor I Вовсе не секретарь, но почту проверяю
Опубликованный материал нарушает ваши авторские права? Сообщите нам.
Вуз:
Предмет:
Файл:Ординатура / Хирургия / Библиотека им академика М.И. Перельмана / Книга_2767_Библиотеки_им_академика_М_И_Перельмана
.pdf
7.3 What data tocollect
https://t.me/medicina_free
of the methodological quality of included studies and reduces the potential to conduct
informative heterogeneity investigations. In such circumstances, review authors are
strongly encouraged to contact the original investigators. Contact details of study
authors, when not available from the study reports, can be obtained from more recent
publications, from university or institutional staff listings, from membership directories
of professional societies or by a general search of the internet. If the contact author
named in the study report cannot be contacted or does not respond, it is worthwhile
attempting to contact other authors. Response rates from study authors vary (from 43%
(Cooper 2019) to 68% (Selph 2014) in published case studies). A number of strategies
have been investigated to try to increase the number of successful contacts, including
short emails with attachments compared to long emails without attachments
(Godolphin 2019), the use of reminder emails (Cooper 2019) or the use of phone calls to
supplement email contact (Danko 2019).
In the absence of a clear consensus regarding the optimal approach, review authors
should consider the nature of the information they require and make their request
accordingly. For descriptive information about the conduct of the study, it may be most
appropriate to ask openthreshold was used to define a positive test result?). If particular numerical data are
required, it may be more helpful to request them specifically, possibly providing a short
data collection form with a 2×2 contingency table indicating test and threshold (either
blank for the study author to complete or partially completed with blank fields for missing data), or to offer to run the analyses on the original data.
Similar strategies may be used to identify additional unpublished studies or to access
full reports of studies available only as conference abstracts.
It is good practice for review authors to keep a record of author contact, to acknowledge authors who have taken the time to respond and, importantly, for data included in
a review as a result of author contact to be clearly identified.
ended questions (e.g. how were participants recruited, or what
7.3 What data tocollect
7.3.1 What are data?
For the purposes of this chapter, we define ‘data’ to be any information about (or
derived from) a study, including details of methods, participants, setting, context, index
tests, target condition and reference standards, results, publications and investigators.
Review authors should plan in advance what data will be required for their systematic
review and develop a strategy for obtaining them. The data to be sought should be
described in the protocol with an explanation of the relevance of the data where
needed.
The data collected for a review should be displayed for each included study in a table
of ‘Characteristics of included studies’. Important study characteristics should be
collected that adequately describe the included studies and that support review
author judgements of methodological quality (see Chapter8). Data collection should
also support the construction of summary tables and figures, and enable syntheses
and meta- analyses, including the investigation of potential sources of heterogeneity. It
is advisable to develop outlines of tables and figures that will appear in the review prior
135
7 Collecting data
https://t.me/medicina_free
to commencing data extraction so that sufficient data items are extracted, and to avoid
extracting information that will not appear in the final review. Review authors should
familiarize themselves with reporting guidelines for systematic reviews (see PRISMA
extension for Diagnostic Test Accuracy studies (McInnes 2018)) to ensure that relevant
elements and sections are incorporated.
Chapter8 details the information needed to inform the assessment of risk of bias and
applicability of study results. The following sections review the types of information
that should be sought, and these are summarized in Table 7.3.a (Li 2015). Review
authors may need to request missing information from study authors.
Table7.3.a Checklist ofitems toconsider indata collection
Information about data extraction from reports
Name of data extractors, date of data extraction, and identification features of each report from
which data are being extracted
Eligibility criteria
Confirm eligibility of the study for the review
Reason for exclusion
Study methods (participant sampling)
Study design (see Chapter3)
Recruitment (how were eligible participants identified) and sampling procedures used
Single or multicentre study; if multicentre, number of recruiting centres
Enrolment start and end dates
Source(s) of funding or other material support for the study
Authors’ financial relationship and other potential conflicts of interest
Participant characteristics and setting
Setting
Region(s) and country/countries from which study participants were recruited
Study eligibility criteria, including any prior testing or treatment
Sample size (participants and unit of analysis, if different), distinguishing number recruited, if
reported, from number analysed
Number of participants with the target condition
Characteristics of participants (e.g. age, sex, comorbidity, socio- economic status)
Index test(s)
Describe the index test(s), ideally with sufficient detail for replication:
• Technical aspects of each index test, including full test name, test manufacturer, version numbers
and catalogue number if relevant
• Threshold(s) for test positivity (and any rationale for selection
• Details of who performed the test (if different from test interpreter)
• Factors relevant to test interpretation (e.g. staff qualifications, timing of interpretation, number of
observers and method of interpretation (single, consensus, mean etc.))
• Any attempts at blinding of test assessors to final diagnosis and to additional clinical information
or other test results
*
*)
*
136
7.3 What data tocollect
https://t.me/medicina_free
Table7.3.a (Continued)
Target condition and reference standard(s)
Describe target condition and reference standard, including:
• Severity of disease in participants with the target condition
• Differential diagnoses of participants without the target condition
Describe reference standard(s) used to establish presence of target condition, including details of:
• Tests (single test, combination of tests, expert panel etc.)
• Threshold for defining presence of the target condition, if relevant
• Observers
• Any attempts at blinding of assessors to index test results
Flow and timing
Time interval between index tests, if more than one test evaluated
Timing of reference standard in relation to application of index tests*
If more than one reference standard, report number of participants (with and without the target
condition) per reference standard
Report any study investigator exclusion of participants from analysis, with reasons (e.g. lost to
up, missing data, index test failures, etc.)
followReport any review team exclusion of participants from analysis, with reasons (e.g. if data for only a
subgroup of participants in the study are eligible for the review)
Results
Report 2×2 contingency table data for each relevant combination of index test, threshold and
reference standard or target condition (i.e.true positives (TP), false positives (FP), false negatives
(FN), true negatives (TN)) and indicate if data were backwere obtained from investigators
Report covariates to be coded for each 2×2 contingency table, including any categorization of data
If subgroup analysis is planned, the same information would need to be extracted for each
participant subgroup
Miscellaneous (optional)
Reference to other relevant studies
Correspondence required and information received following author contact
Miscellaneous comments from the study authors or by the review authors
*
*
calculated from published data or if data
*Full description required for assessments of risk of bias (see Chapter8).
7.3.2 Study methods (participant recruitment andsampling)
Poor research methods can influence study findings and introduce biases into results.
For detailed guidance on study designs for estimating test accuracy, refer to Chapter3.
Test accuracy can be estimated in any study where participants receive one or more
index test(s) and at least one reference standard. Designs can recruit participants and
obtain index test results in different ways. Some studies may start recruiting newly
identified participants. Others may obtain index test results in previously collected
samples or acquired images. Details of sampling methods (consecutive, random, ‘convenience’) and recruitment dates should be collected.
137
7 Collecting data
https://t.me/medicina_free
Additional information on study design characteristics that may affect the rigour of
the study’s conduct but may not lead directly to risk of bias may also be collected; for
example, the funding source of the study, potential conflicts of interest of the study
authors, whether ethical approval was obtained, and whether a sample size calculation
was performed a priori.
7.3.3 Participant characteristics andsetting
Details of participants are collected to enable assessment of the comparability of participants between included studies, and to allow assessment of how directly and completely the participants in the included studies reflect the original review question. Data
need to be identified that allow assessment of whether the test is being used in the
participants for whom it is intended.
Typically, aspects that should be collected are those that could (or are believed to)
affect test accuracy and those that could help review users assess applicability, including to populations beyond the direct review question. For example, if the test is considered for use in symptomatic and asymptomatic people, this information should be
collected and consideration should be given to separate analysis by symptom status.
Care should be taken to avoid extracting information that is not thought to affect test
accuracy, nor to help apply results.
Age and sex are standard characteristics to report, and summary information about
these should always be collected unless they are clearly obvious from the context.
These characteristics are likely to be presented in different formats (e.g. ages as means
or medians, with standard deviations or ranges; sex as percentages or counts for the
whole study or for participants with and without the target condition). Review authors
should seek consistent quantities where possible, and indicate whether summary characteristics apply to the study as a whole (before or after any exclusions) or to those with
and without the target condition separately. It may not be possible to select the most
consistent statistics until data collection is complete across all or most included
studies.
It is critical to collect information that characterizes why participants were selected
for testing. This may relate to the symptoms they present with, their clinical history or
results of previous tests. For some tests and target conditions, thepresence of certain
comorbid conditions may also be important to record. Clinical characteristics relevant
to the review question (e.g. glucose level for reviews on diabetes) are also important for
understanding the severity or stage of the disease. It may be important to obtain expert
clinical input to identify aspects that are regarded as critical to collect and report to
ensure that a review will be useful.
Criteria that were used to define eligible participants can be a particularly important
source of diversity across studies. For example, in a review of a test for skin cancer it is
important to know what type of skin lesions were eligible for inclusion (e.g. melanocytic
only, any pigmented lesion, or any pigmented or non- pigmented lesion).
If the setting of studies may influence test accuracy or the applicability of results, then
information on these should be collected. Typical settings include specialist diagnostic
centres, tertiary care hospitals, acute care hospitals, emergency facilities, general practice
or community settings. Sometimes studies are conducted in different geographical
regions with important differences that could affect the prevalence or severity of the
138
7.3 What data tocollect
https://t.me/medicina_free
target condition, for example endemicity of an infectious disease. Timing of the study may
be associated with important technology differences or trends in accuracy over time. If
such information is important for the interpretation of the review, it should be collected.
Important characteristics of the participants in each included study should be summarized for the reader in the table of ‘Characteristics of included studies’. It is recommended that a structure for this text is set up when planning the data extraction,
identifying which items will be reported and in what format. This will ensure that data
extraction and creation of this table entry can be performed efficiently.
7.3.4 Index test(s)
Details of index tests should be collected. Again, details are required for aspects that
could affect test accuracy or that could help review users assess applicability both for
the intended use of the test and for allowing judgement by review readers in relation to
other circumstances. Where feasible, information should be sought (and presented in
the review) that is sufficient for replication of the index test. This includes any additional testing procedures or the initiation of any therapeutic interventions.
A full description of index test characteristics could include some or all of the following:
●
Technical aspects of each index test and test manufacturer if applicable, e.g.:
∘
biomarker assay, method of analysis (ELISA, PCR, other), batch specifications,
sample collection and storage;
∘
imaging test, type of test, frequency or magnet strength, use of contrast, scan coverage; or
∘
instructions followed when using the test (e.g. manuals and ‘instructions for use’
documents, also called ‘product inserts’).
●
Classification system or algorithm used, e.g. for clinical assessment or image inter-
pretation, including details of the version used, if more than one available.
●
Threshold(s) for test positivity (and rationale for selection).
●
Who performed the test (if different from the test interpreter).
●
Factors relevant to test interpretation, e.g. staff qualifications or expertise, number of
assessors or observers, method of interpretation (single, consensus, mean or other),
and availability of additional clinical information.
●
Timing of test interpretation in relation to reference standard.
It is important to identify where a test is being used in a clinical pathway in each study.
Where the index test is likely to be an add on to other available tests (Chapter3), a
description of other tests that may be carried out concurrently or within the time frame
of testing should be provided. For evaluations of complex testing strategies that involve
the application of more than one test according to some predefined rule, the degree to
which specified procedures or components of the strategy were implemented as
planned may need to be reported.
Comparisons of variations on the delivery of a test, for example whether it was undertaken in accordance with the manufacturer’s instructions for use, may be important secondary objectives and sensitivity analyses to consider in a review. It is important that
data are extracted and coded in a way that enables such analyses to be completed.
Important characteristics of the index tests in each included study should be summarized for the reader in the table of ‘Characteristics of included studies’. Additional tables or
139
7 Collecting data
https://t.me/medicina_free
diagrams depicting the testing pathway can assist descriptions of multi- component testing
strategies so that review users can better assess review applicability to their context.
7.3.5 Target condition andreference standard
The definition of the target condition and the reference standard used to identify the
presence or absence of the target condition should be collected. It is important to report
both differences between studies as to how the target condition is defined and the use
of different reference standards for confirming the presence and/or absence of the target condition, and to investigate if they lead to variations in test accuracy or affect the
applicability of a study beyond the review question.
The target condition, proportion of participants with the target condition and severity
of disease in participants with the target condition should be provided. The differential
diagnoses of participants without the target condition can also affect test performance
and should be recorded.
The reference standard may be a single test (e.g. histology), may be composed of a
number of individual tests (
composite reference standard, e.g. an imaging test plus participant follow- up to identify false positive or false negative results) or could be based on
a consensus diagnosis by an expert group of clinicians (e.g. based on one- year follow- up
and all available information). Details of all tests and assessors and the definition of the
threshold for defining presence of the target condition should be recorded along with any
rules for combining individual test results. If a disease classification system is used, details
of the specific version or versions used should be specified. For example, the Liver Imaging
Reporting and Data System (LI- RADS) has a number of iterations (van der Pol 2019), as
does the American College of Rheumatology classification criteria for rheumatoid arthritis (Cader 2011). The availability of additional clinical information that might influence
the reference standard should also be recorded. Where feasible, information should be
sought (and presented in the review) that is sufficient for replication of the reference
standard. Review authors may need to request missing information from study authors.
There may be more than one eligible reference standard for the review question. If
possible, these should be ranked prior to data extraction, so that priority can be given
to extracting data according to the preferred reference standard in studies presenting
data according to more than one reference standard.
Important characteristics of the target condition and reference standards used in
each included study should be summarized for the reader in the table of ‘Characteristics
of included studies’.
7.3.6 Flow andtiming
Participants’ flow through a study and the timing of tests and reference standards
should be recorded in order to identify any missing data, the completeness and
approach to verification by the reference standard, and timings of tests to investigate
possible changes to disease status between application of the index test and the reference standard (dueto progression of disease or to any therapeutic intervention received
during the time interval between tests). Information on the timing of the reference
standard in relation to the application of index tests and the number of participants
receiving each reference standard should be collected. Missing data can result from
140
7.3 What data tocollect
https://t.me/medicina_free
participants not undergoing the index test(s) or reference standards as planned, or from
failure of the index test or reference standard.
Each review is likely to present its own unique issues, and review authors should be
alert to particular aspects of the way in which studies are undertaken and reported. For
example, some studies discuss ‘samples’ rather than ‘participants’ and it is not always
clear whether they are analysing multiple samples from individual patients.
Relevant details related to flow and timing in each included study should be summarized for the reader in the table of ‘Characteristics of included studies’.
7.3.7 Extracting study results andconverting tothe desired format
Results data arise from the cross- classification of the results of the index test for individual participants in a study against the reference standard result. For most reviews,
results are summarized at study level as paired data by the reference standard (sensitivity
and specificity) or by index test results (positive and negative predictive values).
Reports of studies can include several results for a single index test according to
different test thresholds, different scoring systems used or different reference standards.
For example, a number of different imaging characteristics can be observed for the
same image, different numerical thresholds can be applied to test results reported as
continuous data, or categorical variables can be grouped in different ways. Review protocols should be as specific as possible about relevant test thresholds and a framework
should be pre- specified in the protocol to facilitate making choices between multiple
eligible measures or results. For instance, a hierarchy of preferred thresholds or approaches
to scoring or interpretation of images might be created. Any additional decisions or
changes to this framework made once the data are collected should be reported in the
review as changes to the protocol.
The unit of analysis (e.g. participant, or body part, or site) should be recorded for each
result when it is not obvious (see Chapter4). For many tests simple 2×2 tables will be
available; for others with multiple categories for index and/or reference standard
results, 3×2, 3×3 or even larger contingency tables can potentially be extracted and
decisions will be required about how to extract these into a 2×2 format.
In most cases, it is desirable to collect contingency table data for each combination of
index test, threshold, reference standard and other variables of relevance to the review
question. Sometimes studies will report accuracy results in a 2×2 table format, but
more often the numbers required for meta- analysis are not reported in this way. Other
statistics can be collected and converted into the required format. For example, reports
of sample size, the proportion of participants with the target condition and sensitivity
and specificity can be used to reconstruct 2×2 tables. Details of recalculation are
provided below. When insufficient information is presented in a study report, author
contact should be considered (see Section7.2.2).
7.3.7.1 Obtaining 2×2data fromaccuracy measures
The data needed to populate a 2×2 table can be calculated from reported accuracy
measures if the sample size for the table is known (this is not always the same as the
total sample size for the study) and at least one additional piece of information is
provided. The calculations can be straightforward and easily calculated using a hand
141
7 Collecting data
https://t.me/medicina_free
calculator. The calculator in Cochrane’s primary review authoring tool, Review Manager
(RevMan; training.cochrane.org/online- learning/core- software- cochrane- reviews/
revman), is useful for less straightforward combinations of accuracy measures.
If the total number tested is known and the proportion with the target condition is
reported, the 2×2 data can be derived from the reported sensitivity and specificity in six
simple steps described in Table7.3.b. At Step1, the number of people with the target
condition can be calculated from the total tested multiplied by 29.7% (proportion with
the target condition). The number without the target condition is obtained by subtracting the number with the target condition from the total tested (Step2). At Step3, the
number of true positives is derived by calculating 40.8% (sensitivity) of the number with
the target condition, and the number of true negatives derived by calculating 97.8%
(specificity) of the number without the target condition (Step4). The number of false
negatives and false positives can then be calculated by subtracting the number of true
positives and true negatives from the column totals (Step5 and Step6).
A similar procedure can be followed for reported positive predictive value (PPV) and
negative predictive value (NPV) if the total number tested and the number of participants with a positive index test result are known, as described inTable7.3.c. At Step1
the number testing positive in the study is entered or derived from the reported percentage testing positive (in this case 13.6%), and the number testing negative calculated by subtraction of test positives from the total number tested (Step2). The number
Table7.3.b Estimating 2×2 contingency table data fromreported sensitivity andspecificity
Example of
reported data → Sensitivity 40.8% Specificity 97.8%
Estimation
of 2×2 data
Index test
positive
Index test
negative
Column totals (Step1)
Table7.3.c Estimating 2×2 contingency table data fromreported PPV andNPV
Example of reported
data → PPV 88.9% NPV 79.6%
Estimation of 2×2 data
Index test positive (Step3)
Index test negative (Step6)
Column totals TP+FN FP+TN 330
People with
target condition
(Step3)
TP = 98 × 0.408 = 40
(Step5)
FN = 98 − 40 = 58
330 × 0.297 = 98
People with
target condition
TP = 45 × 0.889 = 40
FN = 285 − 227 = 58
People without
target condition Row totals
(Step6)
FP = 232 − 227 = 5
(Step4)
TN = 232 × 0.978 = 227
(Step2)
330 − 98 = 232
People without
target condition Row totals
(Step5)
FP = 45 − 40 = 5
(Step4)
TN = 285 × 0.796 = 227
Sample size 330
Target condition present 29.7%
TP+FP
FN+TN
330
Sample size 330
Index test positive 13.6%
(Step1)
330 × 0.136 = 45
(Step2)
330 − 45 = 285
142
7.3 What data tocollect
https://t.me/medicina_free
Table7.3.d Estimating 2×2 contingency table data fromreported specificity andNPV
Example of reported
data → Specificity 97.8% NPV 79.6%
Estimation of 2×2
data
Index test positive (Step7)
Index test negative (Step6)
Column totals (Step1)
People with
target condition
TP = 98 − 58 = 40
FN = 285 − 227 = 58
330 × 0.297 = 98
People without
target condition Row totals
(Step4)
FP = 232 − 227 = 5
(Step3)
TN = 232 × 0.978 = 227
(Step2)
330 − 98 = 232
Sample size 330
Target condition present 29.7%
TP+FP
(Step5)
*
227/0.796 = 285
330
*Based on formula NPV × Index test negative = TN → Index test negative = TN/NPV
of true positives is 88.9% (PPV) of the number testing positive (Step3) and the number
of true negatives is 79.6% (NPV) of the number testing negative (Step4). The rest of the
2×2 table can be completed by subtraction of true positives and true negatives from the
row totals (Steps 5 and 6).
Contingency tables can also be calculated from studies that report the total number
tested with a combination of either sensitivity or specificity and either PPV or NPV, as
long as the number with the target condition or number testing positive on the index
test is also known. Table7.3.d shows that once the column totals are known (Step1 and
Step2), the number of true negatives and false positives can be calculated from reported
specificity in the same way as previously described for Table7.3.b (Step3 and Step4).
The number of true negatives and reported NPV can then be entered into a rearranged
formula for calculation of NPV to calculate the total number testing negative (row total
in Step5). The number of false negatives can be completed by subtraction of true negatives from the row total (Step6) and the number of true positives completed by subtraction of false negatives from the column total (Step7).
Although the formulae are more complicated, the 2×2 contingency table data can also
be hand calculated from reported positive and negative likelihood ratios or using sensitivity (or specificity) and the positive (or negative) likelihood ratio. Box7.3.a shows the
formulae for calculating sensitivity and specificity from reported positive likelihood
ratios (LRP) and negative likelihood ratios (LRN).
Once sensitivity and specificity are known, the steps laid out in Table 7.3.b can be
followed to calculate the 2×2data. Alternatively, the RevMan calculator can be used. It
is also possible to use sample size and reported sensitivity and specificity with 95%
confidence intervals to calculate 2×2 tables. However, due to the possibility for error,
this should only be carried out by those familiar with the underlying equations for estimation of confidence intervals and access to software for the appropriate methods.
Whenever 2×2 tables are calculated from reported results, it is helpful to cross- check
the results against any other reported accuracy measures. For example, if a 2×2 table is
derived from reported sensitivity and specificity, the PPV and NPV from the resulting
2×2 table can be checked against the PPV and NPV reported in the study, if available. If
there are any discrepancies between the derived results based on the 2×2 table and the
data in the study report, review authors should consider whether the difference could
143
7 Collecting data
LRPLRP LRN
1
LRP
sensitivity
1
sensitivity
specificity
https://t.me/medicina_free
Box 7.3.a Estimating sensitivity or specificity fromlikelihood ratios
To calculate sensitivity and specificity:
ensitivity
Recalculate LRP to double- check calculations:
LRN, negative likelihood ratio; LRP, positive likelihood ratio.
LRPLRN
LRP LRN
1
specificity
be due to a typographical or calculation error. If no error is suspected, then contacting
the study authors should be attempted. If the difference is large and the numbers cannot be resolved satisfactorily, then the study should be considered for exclusion on the
basis of being internally inconsistent.
Some reviews will include studies where estimation of test accuracy was not the
primary objective, but data are presented in such a way that 2×2 tables can still be
extracted. For example, reviews of tests used in standard clinical practice, such as
routine laboratory markers or imaging tests, could include studies where participant
characteristics, including the results of the index test of interest, are tabulated according
to the presence or absence of the target condition according to the reference standard,
providing the required numbers of true- positive and false positive results, respectively.
The information required to calculate 2×2 tables from accuracy measures is usually
provided in study tables; however, if one or more pieces of information needed is
missing it is important also to check the Abstract and any available (online) supplementary information. Sometimes the data needed can even be provided in the
Discussion section of an article, where study authors may discuss or present their
results in a different way from the main Results section.
Any 2×2 tables that have been derived from reported accuracy measures should be
clearly identified on the data extraction form.
7.3.7.2 Using global measures
In some studies, global measures of test accuracy such as the diagnostic odds ratio
(DOR) or, more commonly, the area under the receiver operating characteristic (ROC)
curve or c- statistic may be reported instead of paired sensitivities and specificities.
These global measures cannot be used to derive 2×2 contingency table data. Where
indicated, these global measures of test accuracy should always be extracted with their
respective 95% confidence intervals or standard errors.
144
Соседние файлы в папке Библиотека им академика М.И. Перельмана