Digital Healthcare Partners is a digital health accelerator in Korea that discovers, cultivates, invests in, and connects digital health startups. It provides mentoring, business development support, clinical validation, and investment to early-stage startups. Recent deals include a seed investment in 3billion, a company developing genetic diagnosis services for rare diseases using genome analysis. Global trends in digital health funding in Q1 2017 included large deals in areas like population health, EHR, and e-commerce. The largest deal was Grail's $900M series B for its liquid biopsy cancer diagnostic technology.
8. DHP는 디지털 헬스케어 전문 엑셀러레이터로서,
디지털 헬스케어/의료 스타트업을 발굴, 육성, 연결하고 투자합니다.
발굴 • 세상을 바꿀 수 있는 혁신적인 헬스케어 스타트업 및 예비 창업팀을 발굴합니다.
• 발굴을 위해 DHP Office Hour, 해커톤, 자체 행사 개최 등의 다방면의 채널을 활용합니다.
육성 • 의료/헬스케어 전문가들로 이루어진 파트너 및 자문가들이 초기 스타트업을 멘토링합니다.
• 사업 개발, 아이템 검증, 임상 연구, 인허가 관련 자문 등 전방위적으로 지원합니다.
투자 • 초기 스타트업 및 예비 창업팀에게 정해진 원칙에 따라 지분 투자를 집행합니다.
• 스타트업을 성장시켜 지분 가치의 상승에 따라서 재무적 수익을 추구합니다.
연결 • 초기 스타트업을 병원, 규제기관, 보험사, VC, 대학 등 다양한 이해관계자들과 연결합니다.
• 파트너와 자문가들의 네트워크를 적극 활용하여 스타트업을 의료계 이너서클로 끌어들입니다.
9. DHP는 최고의 의료 전문가들이 초기 헬스케어 스타트업에
의학 자문, 의료 기관 연계, 임상 검증, 투자 유치 등을 지원합니다.
최윤섭 대표파트너 정지훈 파트너 김치원 파트너
• 성균관대학교 디지털헬스학과 교수
• 최윤섭 디지털 헬스케어 연구소 소장
• VUNO, Zikto, 녹십자홀딩스 등 자문
• 저서: ‘헬스케어 이노베이션’
• 전) 서울대학교 의과대학 암연구소 교수
• 전) 서울대학교병원 의생명연구원 교수
• 포항공대 전산생물학 이학박사
• 포항공대 컴퓨터공학/생명과학 학사
• 경희사이버대학 미디어커뮤니케이션학과 교수
• 빅뱅엔젤스 파트너
• Lunit, 매직에코, 휴레이포지티브 등 자문
• 저서: ‘제 4의 불', ‘거의 모든 IT의 역사’ 등
• 전) 명지병원 IT융합연구소장
• 한양대학교 의과대학 의학사
• 서울대학교 보건정책관리학 석사
• USC 의공학박사
• 내과전문의, 서울와이즈요양병원 원장
• 성균관대학교 디지털 헬스학과 교수
• Noom, Zikto, Future Play 등 자문
• 저서: ‘의료, 미래를 만나다’
• 전) 맥킨지 서울사무소 경영컨설턴트
• 전) 삼성서울병원 의료관리학과 교수
• 서울대학교 의과대학 졸업
• 연세대학교 보건대학원 석사
11. DHP는 유전체 분석 기반의 희귀질환 진단 서비스를 개발하는
3billion에 시드 투자 및 엑셀러레이션을 시작하였습니다.
• 마크로젠의 유전체 분석 전문가들이 2016년 11월 스핀오프
• 대표 이사 금창원은 유전체 분석 전문가이자 연쇄 창업가
• 유전체 분석으로 4,000여개 희귀 유전 질환을 한 번에 진단
• 해외 시장 타겟, 2,000불의 비용으로 2-3주 내 진단
• 2017년 2월 시제품 출시
• http://3billion.io
12. Contents
• 2017 1Q 미국 VC 투자 동향
• ‘Liquid Biopsy’: Illumina and Grail
• 23andMe의 DTC 서비스 FDA 인허가 확대
• IBM Watson for Oncology 도입 광풍(?)
• 의사를 능가하는 Deep Learning 연구 결과들
• 의학적 효용을 증명한 헬스케어 스타트업의 증가
18. •펀딩을 가장 많이 받은 분야는 Genomics and Sequencing 분야
•Human Longevity ($220M), Color Genomics ($45M), Seven Bridges Genomics ($45M)
•Pathway Genomics ($40M), Emulate ($28M)
https://rockhealth.com/reports/2016-year-end-funding-report-a-reality-check-for-digital-health
20. •총 451개 VC 및 CVC가 투자를 집행
•3개 이상의 deal 을 한 곳은 40개 투자자
•총 투자자 중 1/3 정도는 ‘하나의’ deal 만 진행
•237개는 디지털 헬스케어 기업에 '처음' 투자한 곳 (화이자 포함)
https://rockhealth.com/reports/2016-year-end-funding-report-a-reality-check-for-digital-health
21. •최근 3년 동안 Merk, J&J, GSK 등의 제약사들의 디지털 헬스케어 분야 투자 급증
•2015-2016년 총 22건의 deal (=2010-2014년의 5년간 투자 건수와 동일)
•Merck 가 가장 활발: 2009년부터 Global Health Innovation Fund 를 통해 24건 투자 ($5-7M)
•GSK 의 경우 2014년부터 6건 (via VC arm, SR One): including Propeller Health
•Grail: cancer diagnostic spin-off from Illumina (Liquid biopsy)
•$900m Series B, in March 2017
•가장 많은 제약사가 참여한 투자: J&J, Merck, Bristol-Myers-Squibb
22. •2017 1Q에 총 71건의 deal; $1B funding 으로 strong start
•트럼프 정부의 의료 및 규제 정책의 불확실성이 리스크로 보였으나, 크게 영향을 미치지는 않은 것으로 보임
•Rock Health의 경우,
•Digital Healthcare 분야의 정의가 보수적 (ie. 진단회사인 Grail은 누락)
•미국 내의 $20m 이상의 deal 만을 조사
27. in the understanding of tumour heterogeneity; second,
the role of surgery as a therapeutic modality in the era of
targeted therapy; third, the use of personalized therapy
in the perioperative period and, finally, the possibilities
of personalization of surgical procedures according to
lung cancer subtypes.
VATS lobectomy showed that intraoperative blood loss
was significantly reduced in the VATS group compared
with open lobectomy in nine studies; however, no differ-
ence was observed in five studies and the values were not
reported in seven studies.12
Hospital stay was also signifi-
cantly shorter in VATS group in five studies. Park et al.,13
Heterogeneity in patients
with adenocarcinoma
of the lung according
to driver oncogenes
Heterogeneity within
patients with
EGFR mutation
Heterogeneity in
resistance mechanisms
in one patient
HER2
3%
EGFR
~40% in Asians
~15% in Caucasians
ALK
~5%
KRAS
~15% in Asians
~30% in Caucasians
RET
~1%
ROS1
~1%
BRAF
~1%
PIK3CA
~1%
NRAS
~1%
MET
<5%
Others?
Exon 19del
~50%
L858R
~40%
Sensitive
Inherent resistance
CRKL
~3%
BIM
20–40%
IκB
~30%
Inherent T790M
~2% by sequencing
~30% by sensitive
method
Further
heterogeneity
EGFR-TKI
Drug X
T790M
MET
a cb
T790M
Heterogeneity in patients
with adenocarcinoma
of the lung according
to driver oncogenes
Heterogeneity within
patients with
EGFR mutation
Heterogeneity
resistance mecha
in one patien
HER2
3%
EGFR
~40% in Asians
~15% in Caucasians
ALK
~5%
KRAS
~15% in Asians
~30% in Caucasians
RET
~1%
ROS1
~1%
BRAF
~1%
PIK3CA
~1%
NRAS
~1%
MET
<5%
Others?
Exon 19del
~50%
L858R
~40%
Sensitive
Inherent resistance
CRKL
~3%
BIM
20–40%
IκB
~30%
Inherent T790M
~2% by sequencing
~30% by sensitive
method
Further
heterogeneity
EGFR-TKI
Drug
T790M
ME
a cb
T790M
Figure 1 | Various classes of tumour heterogeneity in adenocarcinoma of the lung. a | Heterogeneity in patients with
adenocarcinoma of the lung according to driver oncogenes that are crucial for selecting targeted drugs for treatment.2,76
Number of people reflects approximate incidence.2,76
b | Heterogeneity in patients with EGFR mutations, resulting in
MitsudomiT, Suda K,YatabeY. Nat Rev Clin Oncol. 2013 Apr;10(4):235-44.
Heterogeneity in Lung Adenocarcinoma
30. Nat Genet. 2014 Feb 26;46(3):214-5.
Intratumoral heterogeneity in kidney cancer
31. Nat Genet. 2014 Mar;46(3):225-33.
E S
226 VOLUME 46 | NUMBER 3 | MARCH 2014 NATURE G
Figure 1 Regional distribution of nonsynonymous mutations in ten ccRCC tumors. Mutations that failed validation were not included. Heat map
indicate the presence of a mutation (yellow) or its absence (blue) in each region. Category 1 high-confidence driver mutations and category 2 p
driver mutations are highlighted in magenta. The table shows the number of nonsynonymous mutations and the ratio of heterogeneous mutation
tumor. An asterisk indicates where VHL methylation was included in the analysis.
226 VOLUME 46 | NUMBER 3 | MARCH 2014 NATURE G
Figure 1 Regional distribution of nonsynonymous mutations in ten ccRCC tumors. Mutations that failed validation were not included. Heat ma
indicate the presence of a mutation (yellow) or its absence (blue) in each region. Category 1 high-confidence driver mutations and category 2 p
driver mutations are highlighted in magenta. The table shows the number of nonsynonymous mutations and the ratio of heterogeneous mutatio
tumor. An asterisk indicates where VHL methylation was included in the analysis.
Figure 1 Regional distribution of nonsynonymous mutations in ten ccRCC tumors. Mutations that failed validation were not included. Heat map
indicate the presence of a mutation (yellow) or its absence (blue) in each region. Category 1 high-confidence driver mutations and category 2 p
driver mutations are highlighted in magenta. The table shows the number of nonsynonymous mutations and the ratio of heterogeneous mutation
tumor. An asterisk indicates where VHL methylation was included in the analysis.
226 VOLUME 46 | NUMBER 3 | MARCH 2014 NATURE G
Figure 1 Regional distribution of nonsynonymous mutations in ten ccRCC tumors. Mutations that failed validation were not included. Heat ma
indicate the presence of a mutation (yellow) or its absence (blue) in each region. Category 1 high-confidence driver mutations and category 2 p
driver mutations are highlighted in magenta. The table shows the number of nonsynonymous mutations and the ratio of heterogeneous mutatio
tumor. An asterisk indicates where VHL methylation was included in the analysis.
226 VOLUME 46 | NUMBER 3 | MARCH 2014 NATURE
Figure 1 Regional distribution of nonsynonymous mutations in ten ccRCC tumors. Mutations that failed validation were not included. Heat m
indicate the presence of a mutation (yellow) or its absence (blue) in each region. Category 1 high-confidence driver mutations and category 2
driver mutations are highlighted in magenta. The table shows the number of nonsynonymous mutations and the ratio of heterogeneous mutati
tumor. An asterisk indicates where VHL methylation was included in the analysis.
Regional distribution of nonsynonymous mutations
in ten ccRCC tumors
Heat maps indicate the presence of a mutation (yellow) or its absence (blue) in each region.
Category 1 high-confidence driver mutations and category 2 probable driver mutations are highlighted in magenta.
32. E S
226 VOLUME 46 | NUMBER 3 | MARCH 2014 NATURE G
Figure 1 Regional distribution of nonsynonymous mutations in ten ccRCC tumors. Mutations that failed validation were not included. Heat map
indicate the presence of a mutation (yellow) or its absence (blue) in each region. Category 1 high-confidence driver mutations and category 2 p
driver mutations are highlighted in magenta. The table shows the number of nonsynonymous mutations and the ratio of heterogeneous mutation
tumor. An asterisk indicates where VHL methylation was included in the analysis.
226 VOLUME 46 | NUMBER 3 | MARCH 2014 NATURE G
Figure 1 Regional distribution of nonsynonymous mutations in ten ccRCC tumors. Mutations that failed validation were not included. Heat ma
indicate the presence of a mutation (yellow) or its absence (blue) in each region. Category 1 high-confidence driver mutations and category 2 p
driver mutations are highlighted in magenta. The table shows the number of nonsynonymous mutations and the ratio of heterogeneous mutatio
tumor. An asterisk indicates where VHL methylation was included in the analysis.
Figure 1 Regional distribution of nonsynonymous mutations in ten ccRCC tumors. Mutations that failed validation were not included. Heat map
indicate the presence of a mutation (yellow) or its absence (blue) in each region. Category 1 high-confidence driver mutations and category 2 p
driver mutations are highlighted in magenta. The table shows the number of nonsynonymous mutations and the ratio of heterogeneous mutation
tumor. An asterisk indicates where VHL methylation was included in the analysis.
226 VOLUME 46 | NUMBER 3 | MARCH 2014 NATURE G
Figure 1 Regional distribution of nonsynonymous mutations in ten ccRCC tumors. Mutations that failed validation were not included. Heat ma
indicate the presence of a mutation (yellow) or its absence (blue) in each region. Category 1 high-confidence driver mutations and category 2 p
driver mutations are highlighted in magenta. The table shows the number of nonsynonymous mutations and the ratio of heterogeneous mutatio
tumor. An asterisk indicates where VHL methylation was included in the analysis.
226 VOLUME 46 | NUMBER 3 | MARCH 2014 NATURE
Figure 1 Regional distribution of nonsynonymous mutations in ten ccRCC tumors. Mutations that failed validation were not included. Heat m
indicate the presence of a mutation (yellow) or its absence (blue) in each region. Category 1 high-confidence driver mutations and category 2
driver mutations are highlighted in magenta. The table shows the number of nonsynonymous mutations and the ratio of heterogeneous mutati
tumor. An asterisk indicates where VHL methylation was included in the analysis.
Regional distribution of nonsynonymous mutations
in ten ccRCC tumors
Heat maps indicate the presence of a mutation (yellow) or its absence (blue) in each region.
Category 1 high-confidence driver mutations and category 2 probable driver mutations are highlighted in magenta.
Nat Genet. 2014 Mar;46(3):225-33.
33. A RT I C L E S
We determined the regional distribution
f nonsynonymous mutations on the basis of
ata from ultra-deep amplicon sequencing.
We called a mutation as being present in a
umor region if a nucleotide substitution was
etected in 0.5% of reads or an indel was
etected in 1% of reads. We chose these
hresholds on the basis of the error rate of
he sequencing platform13. The regional
istribution of 28 mutations for which
ltra-deep sequencing data were not avail-
ble was inferred from the exome sequenc-
ng data. Exome sequencing of EV001 and
EV002 has previously been reported2 and was
ncluded in this analysis. On average, 67%
range of 28–92%) of the nonsynonymous
omatic mutations were heterogeneous and
ot detectable across all sampled regions of
n individual tumor (Fig. 1). The presence
f somatic mutational heterogeneity in all
10/10) treatment-naive or pretreated cases indicates that ITH, char-
cterized by the spatial separation of subclones, is a common feature
n stage T2–T4 ccRCCs.
To identify the optimal number of biopsies that can reliably detect
he majority of nonsynonymous somatic mutations in a tumor, we
alculated the number of mutations that would have been detected
heterogeneity specifically in EV003 and EV006. No other clinical or
pathological characteristic seemed to correlate with mutational ITH,
and larger series will be required to determine the biological basis for
the diversity in ccRCC phylogenetic structures.
Identification of intraregional subclones
R4b
GL
VHL
SETD2
SETD2
KDM5C
MTOR
R8
KDM5C
R4a
R5
R3
R2
R1
R9
M1
M2a
M2b
SETD2
EV001 EV003
R6 R7
R1
R5
GL
R9
VHL
(methylation)
PBRM1
EV005
R6dom
R7
R1R5
R3
R4, R6min
R2
GL
VHL
PBRM1
PIK3CA
PIK3CA
SF3B1
EV006 EV007 RMH002
R6
R7
R1
R2
R3
PBRM1
BAP1
TP53
RMH004
R8
R10
R2
GL
VT
R4
VHL
PBRM1
ATM
PTEN
SMARCA4
R3
MSH6
PBRM1
ARID1A
RMH008
R4min
R5, R7
R6min
R8
GL
R1
R2
R3
VHL
BAP1 TSC2
BAP1 BAP1
R6dom
R4dom
RK26
PBRM1
TP53 BAP1
R3, R4
R11
R9
GL
R1
R2
VHL
R5min
R10
R7
R5dom
R8
R6
10 non
synonymous
mutations
Trunk
Internal branch
Terminal branch
EV002
R7
R1
R3
R6
GL
R9
VHL
PBRM1
SETD2
TP53
R4
M
PTEN
PTEN
SETD2
R3
GL
GL GL
R4 R7
VHL
VHL
VHL
LN1a, LN1b
R2R6
R1
R1
R15
R9min
R9dom
R3min
BAP1
SETD2
R5,R7
R2, R3dom
R6
PIK3CA
SETD2
TP53
R4
R3R4
R2
igure 3 Phylogenetic trees generated by
maximum parsimony from M-seq data for ten
cRCC tumors. Trees for EV001 and EV002
re adapted from Gerlinger et al.2. Branch
nd trunk lengths are proportional to the
umber of nonsynonymous mutations acquired
n the corresponding branch or trunk. Driver
mutations were acquired by the indicated
enes in the branches the arrows indicate.
river mutations defining parallel evolution
vents are highlighted by color. Trees are
ooted at the germline (GL) DNA sequence,
etermined by exome sequencing of DNA from
eripheral blood.
Phylogenetic trees generated for ten ccRCC tumors
Mutational processes change during tumor evolution
ccRCCs can traverse different evolutionary routes simultaneously
34. Br J Cancer. 2010 Oct 12;103(8):1139-43.
resistance develops. A further obstacle for the interpretation of
large-scale somatic mutation analyses is that fitness effects of the
vast majority of mutations are unknown. The RNA interference-
based functional genomic screening approaches can experimen-
tally test the phenotypic effect of silencing large numbers of genes
individually and may support the interpretation of mutation
data sets by identifying genes that influence cellular fitness or drug
sensitivity.
cells in vitro (Duesberg et al
recurrence after drug treatmen
2010). The clinically importan
geneity could accelerate evolu
enhance biological fitness to
pressures could in turn favour t
unstable cancer cells by can
advantages conferred by genom
must be balanced against the s
result from the generation o
deleterious mutations or tumou
chromosomal instability in anim
Importantly, evolutionary mod
instability can be positively selec
advantage in environments
(e.g. during chemotherapy) in
cycle arrest after DNA damage
cells that are negatively selected
cell cycle arrest and have a lower
Wodarz, 2003).
Thus, it is conceivable that the
instability required to accelerate
of cancers and that excessive
tumour. Results from animal tu
excessive chromosomal instabili
role leads to the tantalising prop
genome instability provides
intervention (Weaver et al, 2007
EVIDENCE FOR DRUG RE
EVOLUTION
The harsh clinical reality is th
almost invariably occurs in adva
leading to disease progression an
examples highlight how Darwi
tumoural genetic heterogeneity
pressure of systemic cancer t
resistance from a Darwinian
Genetic heterogeneity
Time
Bottleneck
Drug treatment
Cancercellpopulation
Figure 1 Schematic view of tumour heterogeneity during tumour
progression and treatment. Acquired mutations in daughter cells of a single
founder cell (left) promote diversion into subclones (different colours
reflect different clones). Some new mutations lead to accelerated growth
(for example yellow and orange clones). Fitness reducing mutations lead
to negative selection (cells with brown cytoplasm). Drug treatment leads to
selective survival of a drug resistant clone (pink) and generates an
evolutionary bottleneck that reduces genetic heterogeneity transiently.
Heterogeneity is re-established rapidly through acquisition of mutations
by daughter cells of the resistant clone.
Darwinian evolution of tumor elucidate clonal heterogeneity
• Acquired mutations in daughter cells of a single founder cell (left) promote diversion into subclones
• Drug treatment leads to selective survival of a drug resistant clone (pink) and generates an evolutionary bottleneck
that reduces genetic heterogeneity transiently.
• Heterogeneity is re-established rapidly through acquisition of mutations by daughter cells of the resistant clone.
35. P E R S P E C T I V E
Fig. 1. A trunk-branch model of intratumor heterogeneity. (A) The development of intratumor heterogeneity is analogous to a growing tree. The
trunk harbors the founding ubiquitous driver mutations of a cancer present in every tumor subclone and region. The sprouting branches represent
different geographically separated regions of the tumor or subclones present within single biopsies that carry heterogeneous mutations that are
not present in every tumor cell or tumor region. Such mutations may distinguish the biological behavior of subclones and harbor the potential to
become driver mutations under distinct selection pressures. Ubiquitous genetic events present in the trunk may provide more tractable biomarkers
and therapeutic targets than heterogeneous events in the branches. We describe three levels of complexity: level 1, the trunk carries driver events,
whereas the branches carry neutral mutations; level 2, the trunk carries driver events, whereas the branches carry neutral or additional driver events
that may harbor convergent phenotypes (for example, distinct mutations in SETD2 or PTEN occur in different regions of the same renal cancer and
converge on the same pathway resulting in its inactivation) (4); level 3, level 1, and level 2 events plus neutral mutations in the branches (or trunk) that
become driver events under selection pressures (11, 17–20). With level 1 complexity, one biomarker can be developed against one target; with level
2 and 3 complexity, a single biomarker is unlikely to be sufficient. The risk of drug resistance may increase with each level of complexity. (B) Clonal ar-
chitecture as a biomarker.The polygenic nature of drug resistance and intratumor heterogeneity may exacerbate difficulties in predicting therapeutic
outcome. Consideration of tumor growth within a Darwinian evolutionary tree framework may support the identification of new predictive biomark-
Level 1
complexity
Level 2
complexity
Level 3
complexity
Clonal architecture
as a biomarker
A BTrunk-branch
hypothesis
onApril4,2012stm.sciencemag.org
A trunk-branch model of intratumor heterogeneity
• The trunk harbors the founding ubiquitous driver mutations of a cancer present in every tumor subclone and region.
• The sprouting branches represent different geographically separated regions of the tumor or subclones present within
single biopsies that carry heterogeneous mutations that are not present in every tumor cell or tumor region.
Sci Transl Med. 2012 Mar 28;4(127):127
38. Release and extraction of cfDNA from the blood
•cfDNA 는 건강한 세포가 사멸할 때뿐만 아니라, 암 세포가 사멸할 때도 혈액 속으로 나온다.
•Liquid biopsy (액체 생검)
•혈액 속에서 cfDNA를 추출하여 암세포에서 나온 DNA를 detection 하고 분석
•암의 재발 유무 조기 발견, 항암제의 약효 파악, 암 세포의 유전 변이 파악 등에 활용
http://www.nature.com/nrclinonc/journal/v10/n8/full/nrclinonc.2013.110.html
40. Importantly, the data provided by these
tests indicate that these genotypes are
not common in the plasma of individuals
that are presumably cancer-free (Thress
et al., 2015). It is worth noting that tu-
mor-derived RNA and DNA methylation
patterns can also be detected in the
with highly conserved biology, a popula-
tion of cancer patients behaves as a het-
erogeneous collection of many diseases,
each of which carries additional heteroge-
neity in its own right. Therefore, identifying
a finite number of protein or nucleic acid
biomarkers that are highly sensitive and
ctDNA molecules to reliably measure
them in a background of mostly non-tu-
mor-derived cfDNA. We estimate that
such a broad and deep sequencing
approach could require orders of magni-
tude more sequence data than liquid bi-
opsy assays currently use (Table 1). To
Table 1. Comparison of ctDNA Liquid Biopsy Test to Potential Cancer Screening Test
Indication Tumor Liquid Biopsy (Genotyping, Monitoring) Early Cancer Detection
Target population Patients with known diagnosis of cancer Asymptomatic individuals
Tissue reference Can be informed by tissue analyses No prior knowledge of tissue
Key performance characteristics Sensitivity and specificity for specific
actionable genotypes
d Sensitivity and specificity for clinically
detectable cancer
d Premium on specificity in individuals
without detectable cancer
d Tissue of origin needed to guide workup
Clinical Endpoint for Utility Therapeutic benefit with specific therapies Net outcome improvement with early detection
and local treatment of cancer
Genes Covered 10-50 100-1000s
ctDNA Limit of Detection 0.1% <0.01%
Importance of Novel Variant Detection Low High
Amount of Sequencing 1x 100X
Study Size for Clinical Validity and Utility 100’s 10,000 - 100,000 s
Next-Generation Sequencing of Circulating Tumor DNA
for Early Cancer Detection
Cell 168, February 9, 2017
41. C A N C E R
Circulating tumor DNA analysis detects minimal
residual disease and predicts recurrence in patients
with stage II colon cancer
Jeanne Tie,1,2,3,4
*†
Yuxuan Wang,5†
Cristian Tomasetti,6,7
Lu Li,6
Simeon Springer,5
Isaac Kinde,8
Natalie Silliman,5
Mark Tacey,9
Hui-Li Wong,1,3,4
Michael Christie,1,3,10
Suzanne Kosmider,2
Iain Skinner,2
Rachel Wong,1,11,12
Malcolm Steel,11
Ben Tran,1,2,3,4
Jayesh Desai,1,3,4
Ian Jones,4,13
Andrew Haydon,14
Theresa Hayes,15
Tim J. Price,16
Robert L. Strausberg,17
Luis A. Diaz Jr.,5
Nickolas Papadopoulos,5
Kenneth W. Kinzler,5
Bert Vogelstein,5
*†
Peter Gibbs1,2,3,4,17
*†
Detection of circulating tumor DNA (ctDNA) after resection of stage II colon cancer may identify patients at the highest
risk of recurrence and help inform adjuvant treatment decisions. We used massively parallel sequencing–based
assays to evaluate the ability of ctDNA to detect minimal residual disease in 1046 plasma samples from a prospective
cohort of 230 patients with resected stage II colon cancer. In patients not treated with adjuvant chemotherapy, ctDNA
was detected postoperatively in 14 of 178 (7.9%) patients, 11 (79%) of whom had recurred at a median follow-up
of 27 months; recurrence occurred in only 16 (9.8 %) of 164 patients with negative ctDNA [hazard ratio (HR), 18;
95% confidence interval (CI), 7.9 to 40; P < 0.001]. In patients treated with chemotherapy, the presence of ctDNA
after completion of chemotherapy was also associated with an inferior recurrence-free survival (HR, 11; 95% CI,
1.8 to 68; P = 0.001). ctDNA detection after stage II colon cancer resection provides direct evidence of residual
disease and identifies patients at very high risk of recurrence.
INTRODUCTION
About 1.3 million cases of colorectal cancer are diagnosed annually
worldwide (1). In patients with stage II colon cancer (~25% of all
colorectal cancer), management after surgical resection remains a
clinical dilemma, with about 80% cured by surgery alone (2). The cur-
rent approach to defining recurrence risk for patients with early-
tus in the tumor defines a low-risk group in which adjuvant chemo-
therapy is not beneficial (6, 7). Most recently, multiple tissue-based
gene signatures have been shown to have prognostic significance,
but again with modest hazard ratios (HRs) of 1.4 to 3.7 (8–11).
In practice, adjuvant chemotherapy is more frequently offered
to high-risk stage II patients, with the justification that high-risk
R E S E A R C H A R T I C L E
http://stm.sciencemag.orgDownloadedfrom
Tie, J., Wang, Y., Tomasetti, C., Li, L., Springer, S. et al. (2016). Sci. Transl. Med. 8, 346ra92.
42. Circulating tumor DNA analysis detects minimal residual disease
and predicts recurrence in patients with stage II colon cancer
postoperative adjuvant chemotherapy 를 받지 않은 환자군에 대해서,
ctDNA 양성/음성 기반으로 RFS 을 효과적으로 구분할 수 있음
Tie, J., Wang, Y., Tomasetti, C., Li, L., Springer, S. et al. (2016). Sci. Transl. Med. 8, 346ra92.
43. Circulating tumor DNA analysis detects minimal residual disease
and predicts recurrence in patients with stage II colon cancer
RFS를 ctDNA 여부에 의해서 판단하는 것이 (A)
기존의 T stage, LN yield, LVI 등 기반의 (clinicopathogic) 위험군 분류(B)보다
더욱 효과적일 가능성이 있음
Tie, J., Wang, Y., Tomasetti, C., Li, L., Springer, S. et al. (2016). Sci. Transl. Med. 8, 346ra92.
44. Circulating tumor DNA analysis detects minimal residual disease
and predicts recurrence in patients with stage II colon cancer
기존의 위험군 분류 기준에 의해서 저위험군(C)과 고위험군(D)을 따로 나눠서 ctDNA의 검출 여부로 보게 되더라도,
그 중에서도 RFS 예후 예측을 효과적으로 할 수 있음
Tie, J., Wang, Y., Tomasetti, C., Li, L., Springer, S. et al. (2016). Sci. Transl. Med. 8, 346ra92.
45. Circulating tumor DNA analysis detects minimal residual disease
and predicts recurrence in patients with stage II colon cancer
postoperative adjuvant chemo therapy 를 받은 환자의
항암제 치료 도중과 이후의 ctDNA 변화와 이후 재발여부의 관계
A, B의 경우
•chemo 시작시에는 ctDNA가 positive였다가,
•chemo 받는 동안에는 negative가 되고,
•chemo 끝난 후에는 증가해서 결국 재발
•이 과정에서 기존의 표준 바이오마커인 CEA는 detection 에 실패
Tie, J., Wang, Y., Tomasetti, C., Li, L., Springer, S. et al. (2016). Sci. Transl. Med. 8, 346ra92.
46. Circulating tumor DNA analysis detects minimal residual disease
and predicts recurrence in patients with stage II colon cancer
C, D 환자는 chemo 받는 동안 ctDNA가 negative가 되고
이후에도 유지되어서, 이후 f/u 에서도 재발하지 않음
이 환자들의 경우에는 CEA도 결과는 동일
Tie, J., Wang, Y., Tomasetti, C., Li, L., Springer, S. et al. (2016). Sci. Transl. Med. 8, 346ra92.
47. Circulating tumor DNA analysis detects minimal residual disease
and predicts recurrence in patients with stage II colon cancer
E, F 환자의 경우에는 ctDNA가 각각 false negative, false polisive 결과
•E 환자: 수술 후 10개월 경에 재발하였으나, ctDNA 수치는 negative
•F 환자: ctDNA는 계속 들쭉날쭉 했는데 36개월까지 재발을 하지 않음
Tie, J., Wang, Y., Tomasetti, C., Li, L., Springer, S. et al. (2016). Sci. Transl. Med. 8, 346ra92.
48. Circulating tumor DNA analysis detects minimal residual disease
and predicts recurrence in patients with stage II colon cancer
Tie, J., Wang, Y., Tomasetti, C., Li, L., Springer, S. et al. (2016). Sci. Transl. Med. 8, 346ra92.
49. pointing to cancers th
by R (i.e., those with t
account for cancer in
seem particularly we
miologic investigation
appear unavoidable n
they will become avo
are at least four sourc
cells: quantum effects o
made by polymerase
tion of bases (32), and
produced reactive oxy
olites (33). The last o
be reduced by the
dant drugs (34). The
principle, be reduced
cient repair genes int
or through other crea
As a result of the
ulation, cancer is tod
of death in the world
the best way to reduc
of a third contributo
does not diminish t
prevention but emph
can be prevented by a
factors (Figs. 2 and 3
vention is not the on
exists or can be im
ondary prevention, i.e
vention, can also be
which all mutations a
Fig. 3. Etiology of driver gene mutations in women with cancer. For each of 18 representative
cancer types, the schematic depicts the proportion of mutations that are inherited, due to environmental
factors, or due to errors in DNA replication (i.e., not attributable to either heredity or environment).The sum
of these three proportions is 100%. The color codes for hereditary, replicative, and environmental factors
are identical and span white (0%) to brightest red (100%). The numerical values used to construct this
figure, as well as the values for 14 other cancer types not shown in the figure, are provided in table S6. B,
brain; Bl, bladder; Br, breast; C, cervical; CR, colorectal; E, esophagus; HN, head and neck; K, kidney; Li, liver;
Lk, leukemia; Lu, lung; M, melanoma; NHL, non-Hodgkin lymphoma; O, ovarian; P, pancreas; S, stomach;
RESEARCH | REPORTEtiology of driver gene mutations in women with cancer
Cristian Tomasetti , Science 2017
유전적 요인(Hereditary), 환경적 요인(Environmental)에 비해서,
DNA replication에 의한 driver mutation (Replicative)의 비율이 암종의 구분 없이 매우 높다.
따라서, 암의 조기 발견의 중요성이 더욱 높아지고 있음.
50.
51.
52. https://www.illumina.com/content/dam/illumina-marketing/documents/company/investor-relations/investor_presentations/illumina_investor_presentation.pdf
Product MiniSeq
™
MiSeq
®
NextSeq HiSeq
®
HiSeq
®
X
4000 Five Ten
Output per run 7.5 Gb 15 Gb 120 Gb 1.5 Tb 1.8 Tb 1.8 Tb
Instrument price $49.5K $99K $275K $900K $6M1 $10M1
Utilization2 $20K–$25K $40K–$45K $100K–$150K $300K–$350K $625K–$725K
Installed base3 370 ~5,300 ~1,800 ~1,900 ~400
Sequencing Power for Every Scale
The broadest portfolio offering available
1. Based on purchase of 5 and 10 units for HiSeq X Five and HiSeq X Ten, respectively
2. Company’s projected annual instrument utilization per installed instrument; HiSeq and HiSeq X utilization to be combined later in
• 2014년 1월 출시
• 기기 하나에 약 10억원
• 10개 번들 판매로 최소 구입 단위는 100억원
• 미국의 브로드 연구소, 호주의 가반의학연구소, 한국의 마크로젠
53. 6
Shipping
Q1 2017
$985K
Shipping
Early 2018
$850K
NovaSeq 6000NovaSeq 5000
NovaSeq 5000 Flow Cells
NovaSeq 6000 Flow Cells
1 Tb* 2 Tb 4 Tb* 6 Tb*Output/Run:
NovaSeq System
Scalable throughput to complete studies faster and more economically
*S1 and S4 flow cells expected to begin shipping in Q3 2017; S3 flow cell expected to
begin shipping in early 2018
https://www.illumina.com/content/dam/illumina-marketing/documents/company/investor-relations/investor_presentations/illumina_investor_presentation.pdf
54. • 2017년 1월 NovaSeq 5000, 6000 발표
• 몇년 내로 $100로 WES 를 실현하겠다고 공언
• 2일에 60명의 WES 가능 (한 명당 한 시간 이하)
57. Value Chain of Sequencing Industry
Sequencing Analysis
Diagnosis Treatment
Consumer Service
58. Illumina tries to eat everything in sequencing market
Sequencing Analysis
Diagnosis Treatment
Consumer Service
개인유전정보 앱스토어$100 m funding, co-founding (2015)
NIPT(비침습 태아 산전진단)
$350m 인수 (2013)
Analysis
Liquid Biopsy (액체 생검)
Spin-off (2016.1)/ $100m
빌게이츠, 제프 베조스 등 투자
59. • 일루미나는 NGS 기기를 만드는 하드웨어 기업으로 시작
• 시퀀싱 시장 점유를 기반으로 value chain 후반의 진단, 소비자 서비스 시장으로 진출 중
• (via 인수, 투자, 공동 설립)
• 과거 인터넷 산업에 비유하자면,
• 초기에는 Cisco 같은 네트워크 인프라를 구축하는 기업이 수익
• 나중에는 인프라를 활용한 서비스 제공 기업이 성장 (구글, 페이스북…)
• 일루미나는 그 둘을 모두 하겠다는 것
Illumina tries to eat everything in sequencing market
Sequencing Analysis
Diagnosis Treatment
Consumer Service
개인유전정보 앱스토어$100 m funding, co-founding (2015)
NIPT(비침습 태아 산전진단)
$350m 인수 (2013)
Analysis
Liquid Biopsy (액체 생검)
Spin-off (2016.1)/ $100m
빌게이츠, 제프 베조스 등 투자
60.
61. •Series A: $100m
•Series B: $900m
•Biotech funding round 사상 최고액으로 평가
•ARCH Venture Partners led the round with participation from J&J, Amazon, BMS, Celgene, Varian, and Merck.
•Liquid Biopsy의 임상 연구에 활용할 계획
62. • Grail 이 발표한 최초의 대규모 임상 연구 (2016년 12월): Mayo Clinic, MSKCC 등 50여개 병원 참여
• 10,000명의 환자의 혈액을 분석으로 시작 (추후 확대 예정)
• 7,000명의 암 환자
• 3,000명의 정상인
• 정상인 혈액과 암 환자의 cell free genome profile 을 파악하기 위한 연구
• 정상인의 cf genome의 heterogeneity 역시 연구: 정상인 - 암환자 구분에 도움
• ‘high intensitiy’ sequencing: ultra-deep sequencing & ultra-wide sequencing 을 사용하게 될 것
63. • 2017년 4월 대규모 유방암 환자 임상시험 STRIVE를 개시한다고 공표
• 유방암 조기 발견을 위한 blood test 의 개발 목적
• 120,000명 규모
• Mayo Clinic 과 Sutter Health 에서 유방암 정기검사 (mammography)를 받는 환자들 대상
• ultra-deep sequencing & ultra-wide sequencing 을 사용하게 될 것
• 이 임상 결과를 바탕으로 pan-cancer test 의 개발에도 사용하게 될 것
64. Leading Edge
Commentary
Next-Generation Sequencing
of Circulating Tumor DNA
for Early Cancer Detection
Alexander M. Aravanis,1,2 Mark Lee,1,2 and Richard D. Klausner1,*
1GRAIL, Menlo Park, CA 94402, USA
2Co-first author
*Correspondence: klausner.rick@gmail.com
http://dx.doi.org/10.1016/j.cell.2017.01.030
Curative therapies are most successful when cancer is diagnosed and treated at an early stage. We
advocate that technological advances in next-generation sequencing of circulating, tumor-derived
nucleic acids hold promise for addressing the challenge of developing safe and effective cancer
screening tests.
Cancer-specific mortality from most
types of solid tumors has barely
decreased in decades, despite an expo-
nential increase in our knowledge about
cancer pathogenesis and significant in-
vestments in the development of effective
treatments. The past few years have
witnessed a dramatic success of immu-
notherapies in treating a subgroup of
patients with a variety of tumor types,
including lung, bladder, and kidney, as
well as Hodgkin’s lymphoma and mela-
noma. While such breakthroughs offer
the hope of prolonged survival for some
patients with advanced cancers, finding
cancers earlier would still afford the great-
est chance for cure, given that the survival
rates for patients with early diagnoses are
five to ten times higher compared with late
stage disease (Cho et al., 2014). By
tion algorithms that either miss a large
number of invasive cancers or make the
costly trade-off of over-diagnosing and
consequently over treating. For instance,
high false-positive rates from mammog-
raphy in breast cancer screening, low-
dose CT in lung cancer screening, and
prostate-specific antigen (PSA) screening
(Nelson et al., 2016a; Aberle et al., 2011;
Chou et al., 2011) represent a significant
cost to the healthcare system, with result-
ing mental and physical morbidity, and
even mortality in some cases (Nelson
et al., 2016b).
Even where cancer screening has pro-
duced significant stage shifts, as with
breast and prostate cancer screening,
the impact on cancer-specific mortality
has not been a predictable outcome
(Berry, 2014). Multiple explanations may
cancers are in a pre-metastatic state and
thus still curable. This kinetic aspect of
cancer progression is poorly understood,
but it is essential to informing effective
screening intervals. It is worth noting
that mammography and PSA are only sur-
rogate measures of cancer, which have
poor specificity and provide little insight
into tumor biology. We would argue that
for successful screening, we need a
platform that provides direct, sensitive,
and specific measures of cancer and its
attributes, which have bearing on clinical
behavior.
Circulating Tumor DNA
Profiling of a tumor’s somatic alterations
has become routine, and many clinical
tests are now available that interrogate
anywhere from a few genes to the whole
65. •국내에서도 삼성유전체연구소를 비롯한 몇몇 그룹이 Liquid Biopsy 를 연구
•삼성유전체연구소에서 LiquidScan을 개발했다고 발표 (2017.4)
•(기사 제목처럼) 피 한 방울은 아니고, 20ml 정도 필요
•현재 췌장암 및 유방암 연구 중
•췌장암의 경우 LiquidScan을 통해 기존 방식보다 2-3개월 미리 재발 여부파악 가능
71. • 질병 위험도 검사: BRCA로 유방암 위험도 예측
• 보인자 검사: 부모가 블룸증후군 유전자 보인자인지 판별
• 약물 민감도 검사: 와파린 등에 대한 민감도 검사
• 일반적 특징 검사: 곱슬머리, 카페인 대사, 유당 분해 능력 등
• 조상 분석: 내 조상이 어느 대륙에서 왔는가
Tests of 23andMe
76. Inherited Conditions
혈색소증은 유전적 원인으로 철에 대한 체내 대사에 이상이 생겨 음식을 통
해 섭취한 철이 너무 많이 흡수되는 질환입니다. 너무 많이 흡수된 철은 우
리 몸의 여러 장기, 특히 간, 심장 및 췌장에 과다하게 축적되며 이들 장기
를 손상시킴으로써 간질환, 심장질환 및 악성종양을 유발합니다.
77. Traits
음주 후 얼굴이 붉어지는가
쓴 맛을 감지할 수 있나
귀지 유형
눈 색깔
곱슬머리 여부
유당 분해 능력
말라리아 저항성
대머리가 될 가능성
근육 퍼포먼스
혈액형
노로바이러스 저항성
HIV 저항성
흡연 중독 가능성
80. • 질병 위험도 검사: BRCA로 유방암 위험도 예측
• 보인자 검사: 부모가 블룸증후군 유전자 보인자인지 판별
• 약물 민감도 검사: 와파린 등에 대한 민감도 검사
• 일반적 특징 검사: 곱슬머리, 카페인 대사, 유당 분해 능력 등
• 조상 분석: 내 조상이 어느 대륙에서 왔는가
Tests of 23andMe
(until Nov 2013)
81. • 제한적 유전정보: 일부분의 유전정보 (SNP) 만을 분석
• 환경적 요인 고려 불가: 대부분의 질병은 환경+유전 요인 작용
• So What? : 유전적 위험도를 알아도 대비책이 없거나 불분명
Personal Genome Service 의 한계
84. • 의사를 통하지 않는 DTC 방식에 대한 우려
• 이러한 서비스의 정확성 및 안정성에 대한 우려
• 결과를 받은 사용자들이 제대로 이해할 수 있을지, 오남용에 대한 우려
• 특히, BRCA 유전자에 대한 검사
• Analytic & clinical validation data 제출 지연
85. • 질병 위험도 검사: BRCA로 유방암 위험도 예측
• 보인자 검사: 부모가 블룸증후군 유전자 보인자인지 판별
• 약물 민감도 검사: 와파린 등에 대한 민감도 검사
• 일반적 특징 검사: 곱슬머리, 카페인 대사, 유당 분해 능력 등
• 조상 분석: 내 조상이 어느 대륙에서 왔는가
Tests of 23andMe
(Nov 2013 - Oc 2015)
88. • 질병 위험도 검사: BRCA로 유방암 위험도 예측
• 보인자 검사: 부모가 블룸증후군 유전자 보인자인지 판별
• 약물 민감도 검사: 와파린 등에 대한 민감도 검사
• 일반적 특징 검사: 곱슬머리, 카페인 대사, 유당 분해 능력 등
• 조상 분석: 내 조상이 어느 대륙에서 왔는가
Tests of 23andMe
(Oct 2015 - April 2017)
89. 2017년 4월 6일 FDA가 23andMe의 질병 위험도 예측 서비스의
DTC (Direct-to-Consumer) 판매를 허가
90. FDA의 23andMe 질병 위험도 예측 DTC 서비스 허가
•아래와 같은 10가지 질병의 위험도 예측에 대해서 DTC 허가
•파킨슨병 (Parkinson’s disease)
•알츠하이머 (Late-onset Alzheimer’s disease)
•셀리악병(Celiac disease)
•알파-1 항트립신 결핍증 (Alpha-1 antitrypsin deficiency)
•조발성 1차성 근긴장이상증 (Early-onset primary dystonia)
•XI 혈액응고인자 결핍증 (혈우병C) (Factor XI deficiency, a blood clotting disorder)
•제 1형 고셔병 (Gaucher disease type 1)
•포도당-6-인산탈수소효소(G6PD) 결핍증 (Glucose-6-Phosphate Dehydrogenase deficiency)
•유전성 혈색소침착증(Hereditary hemochromatosis)
•유전적 혈전 기호증(Hereditary thrombophilia)
•FDA는 향후 다른 질병 위험도 예측 검사에 대해서 시장 출시 전 심사(premarket review)를 면제
91. FDA의 23andMe 질병 위험도 예측 DTC 서비스 허가
•임상 연구를 통하여 인허가를 위한 근거 자료 마련
•분석적 타당성(analytical validity)
•임상적 타당성(clinical validity)
•임상적 유용성(clinical utility)
•23andMe의 타액 키트를 통해서 정확하고 일관적으로 유전 변이를 발견할 수 있다는 것을 증명
•검사하는 유전적 변이가 개별 질병의 위험도에 영향을 준다는 명확한 연구 결과.
•환자의 DTC 결과 오남용에 대한 반박
•영국에서 25,000명에게 질병 위험도 예측 서비스를 DTC로 제공한 결과,
자해 등 위험한 결과가 한 건도 발생하지 않았음
•사용자들이 질병 위험도 예측의 결과 레포트의 90% 이상을 이해
92. • 질병 위험도 검사
• 보인자 검사: 부모가 블룸증후군 유전자 보인자인지 판별
• 약물 민감도 검사: 와파린 등에 대한 민감도 검사
• 일반적 특징 검사: 곱슬머리, 카페인 대사, 유당 분해 능력 등
• 조상 분석: 내 조상이 어느 대륙에서 왔는가
Tests of 23andMe
(April 2017- 현재)
93. $115m 펀딩
100만 명 돌파
2006
23andMe 창업
20162007 2012 2013 2014 2015
구글 벤처스
360만 달러 투자
2008
$99 로
가격 인하
FDA 판매 중지 명령
영국에서
DTC 서비스 시작
FDA 블룸증후군
DTC 서비스 허가
FDA에 블룸증후군
테스트 승인 요청
FDA에 510(k) 제출
FDA 510(k) 철회
보인자 등 DTC
서비스 재개 ($199)
캐나다에서
DTC 서비스 시작
Genetech, pFizer가
23andMe 데이터 구입
자체 신약 개발
계획 발표
120만 명 돌파
$399 로
가격 인하
23andMe Chronicle
Business
Regulation
애플 리서치키트와
데이터 수집 협력
50만 명 돌파30만 명 돌파
TV 광고 시작
2017
FDA의
질병위험도 검사
DTC 서비스 허가
+
관련 규제 면제
프로세스 확립
Digital Healthcare Institute
Director,Yoon Sup Choi, PhD
yoonsup.choi@gmail.com
95. 생명윤리법 개정안 및 DTC 허용 계획
•2015년 12월 9일, 국회에서 ‘생명윤리법 개정안’ 의결
•‘비의료기관은 보건복지부장관이 정하는 경우에만 의료기관의 의뢰 없이도
질병 예방 목적의 유전자 검사를 제한적으로 직접할 수 있도록 허용한다'
•보건복지부 2016년 업무보고: 유전자 검사 제도 개선
•질병 예방 목적의 일부 유전자/유전체 검사를 비의료기관에서 직접 실시 (2016년 6월)
•최적 치료법에 필요한 유전자/유전체 검사의 경우 건강보험 적용 (2016년 11월)
•표적치료제 선택 검사 확대
•약물반응예측검사 추가
97. Category 분류기준
I
건강보험 요양급여 등재 혹은 신의료기술평가를 통해 안정성 및 유효성이 인정된 유전자 검사
로 임상적 사용목적(Intended use)이 동일한 경우
II
아직까지 건강보험 요양급여의 등재 혹은 신의료기술평가를 통해 안정성 및 유효성이 인정되
지 않았지만, 임상적 유효성 근거가 있는 검사로 임상적 사용 목적이 동일한 경우
III
건강보험 요양급여 등재 혹은 신의료기술평가를 통해 안정성, 유효성이 인정 받은 검사 및 임
상적 유효성의 근거가 있는 유전자 검사를 건강인에게 시행하는 경우
IV
건강보험 요양급여 등재 혹은 신의료기술평가를 통해 안정성, 유효성이 인정 받은 검사 및 임
상적 유효성의 근거가 있는 유전자 검사를 적절한 임상적 사용 목적 외에 의학적 근거가 부족
한 용도로 사용하는 경우
V
건강보험 요양급여 미등재 혹은 안정성 유효성에 관한 신의료기술평가를 받지 않은 검사로 과
학적 타당성의 입증이 불확실하거나, 검사대상자를 오도할 우려가 있는 신체 외관, 성격 등의
형질에 관한 검사
VI
건강보험 요양급여 미등재 혹은 안정성, 유효성에 관한 신의료기술평가를 받지 않은 검사로
임상적 유효성에 대한 근거가 부족한 검사
유전자 검사평가원에서 제안한 유전자 검사 분류표
98. 한국 DTC 유전정보 분석 제한적 허용
(2016.6.30)
• 「비의료기관 직접 유전자검사 실시 허용 관련 고시 제정, 6.30일시행」
• 2015년 12월「생명윤리 및 안전에 관한 법률」개정(‘15.12.29개정, ’16.6.30시행)과
제9차 무역투자진흥회의(’16.2월) 시 발표한 규제 개선의 후속조치 일환으로 추진
• 민간 유전자검사 업체에서는 혈당, 혈압, 피부노화, 체질량지수 등 12개 검사항목과
관련된 46개 유전자를 직접 검사 가능
http://www.mohw.go.kr/m/noticeView.jsp?MENU_ID=0403&cont_seq=333112&page=1
검사항목 (유전자수) 유전자명
1 체질량지수(3) FTO, MC4R, BDNF
2 중성지방농도(8) GCKR, DOCK7, ANGPTL3, BAZ1B, TBL2, MLXIPL, LOC105375745, TRIB1
3 콜레스테롤(8) CELSR2, SORT1, HMGCR, ABO, ABCA1, MYL2, LIPG, CETP
4 혈 당(8) CDKN2A/B, G6PC2, GCK, GCKR, GLIS3, MTNR1B, DGKB-TMEM195, SLC30A8
5 혈 압(8) NPR3, ATP2B1, NT5C2, CSK, HECTD4, GUCY1A3, CYP17A1, FGF5
6 색소 침착(2) OCA2, MC1R
7 탈 모(3) chr20p11(rs1160312, rs2180439), IL2RA, HLA-DQB1
8 모발 굵기(1) EDAR
9 피부 노화(1) AGER
10 피부 탄력(1) MMP1
11 비타민C농도(1) SLC23A1(SVCT1)
12 카페인대사(2) AHR, CYP1A1-CYP1A2
99. DTC 유전정보 분석 서비스
미국 vs. 한국
Table 1
분석 항목 분석 항목 예시 DTC (미국) DTC (한국)
개인유전정보 분석
질병 위험도 유방암(안젤리나 졸리) O 불가
약물 민감도 와파린 민감도 X X
열성유전질환 보인자 블룸 증후군 O X
웰니스 카페인 분해, 대머리 O 12개만 가능
조상 분석 O 불명확
•미국에서 허용된 보인자 검사, 질병 위험도 예측 검사 DTC 서비스는 여전히 한국에서 불법
•더 큰 문제는 잣대 자체가 FDA 등 글로벌 규제 기조나 산업계에서 통용되는 기준과 다름.
질병/약물/보인자/웰니스/조상 분석 등의 업계에서 받아들여지는 분류를 무시하고 있음.
•글로벌 수준에 발맞추기는 커녕, 한국에서만 통용되는 자체적인 별도 규제 분류 체계를
갈수록 더 만들어가면서, 국내 산업의 갈라파고스화를 심화 시키고 있음
102. 600,000 pieces of medical evidence
2 million pages of text from 42 medical journals and clinical trials
69 guidelines, 61,540 clinical trials
IBM Watson on Medicine
Watson learned...
+
1,500 lung cancer cases
physician notes, lab results and clinical research
+
14,700 hours of hands-on training
103.
104.
105.
106. Empowering the Oncology Community for Cancer Care
Genomics
Oncology
Clinical
Trial
Matching
Watson Health’s oncology clients span more than 35 hospital systems
“Empowering the Oncology Community
for Cancer Care”
Andrew Norden, KOTRA Conference, March 2017, “The Future of Health is Cognitive”
108. Watson Genomics Overview
20
Watson Genomics Content
• 20+ Content Sources Including:
• Medical Articles (23Million)
• Drug Information
• Clinical Trial Information
• Genomic Information
Case Sequenced
VCF / MAF, Log2, Dge
Encryption
Molecular Profile
Analysis
Pathway Analysis
Drug Analysis
Service Analysis, Reports, & Visualizations
109. At HIMSS 2017, provided Hye Jin Kam of Asan Medical Center
110. At HIMSS 2017, provided Hye Jin Kam of Asan Medical Center
111. 식약처 인공지능
가이드라인 초안
Medtronic과
혈당관리 앱 시연
2011 2012 2013 2014 2015
Jeopardy! 우승
뉴욕 MSK암센터 협력
(Lung cancer)
MD앤더슨 협력
(Leukemia)
MD앤더슨
Pilot 결과 발표
@ASCO
Watson Fund,
WellTok 에 투자
($22m)
The NewYork
Genome Center 협력
(Glioblastoma 분석)
GeneMD,
Watson Mobile Developer
Challenge의 winner 선정
Watson Fund,
Pathway Genomics 투자
Cleveland Clinic 협력
(Cancer Genome Analysis)
한국 IBM
Watson 사업부 신설
Watson Health 출범
Phytel & Explorys 인수
J&J,Apple, Medtronic 협력
Epic & Mayo Clinic 제휴
(EHR data 분석)
동경대 도입
(oncology)
14 Cancer Center 제휴
(Cancer Genome Analysis)
Mayo Clinic 협력
(clinical trail matching)
Watson Fund,
Modernizing Medicine
투자
Academia
Business
Pathway Genomics OME
closed alpha 시작
TurvenHealth
인수
Apple ResearchKit
통한 수면 연구 시작
2017
가천대 길병원
Watson 도입
(oncology)
Medtronic
Sugar.IQ 출시
제약사
Teva와 제휴
인도 Manipal Hospital
Watson 도입
태국 Bumrungrad
International Hospital,
Watson 도입
최윤섭 디지털헬스케어 연구소, 소장
(주)디지털 헬스케어 파트너스, 대표파트너
최윤섭, Ph.D.
yoonsup.choi@gmail.com
IBM Watson in Healthcare
Merge
Healthcare
인수
2016
Under Amour
제휴
Broad 연구소 협력 발표
(유전체 분석-항암제 내성)
Manipal Hospital의
WFO 정확성 발표
대구가톨릭병원
대구동산병원
WFO 도입
건양대병원
Watson 도입
(oncology)
부산대학병원
Watson 도입
(oncology/
genomics)
112. 식약처 인공지능
가이드라인 초안
Medtronic과
혈당관리 앱 시연
2011 2012 2013 2014 2015
Jeopardy! 우승
뉴욕 MSK암센터 협력
(Lung cancer)
MD앤더슨 협력
(Leukemia)
MD앤더슨
Pilot 결과 발표
@ASCO
Watson Fund,
WellTok 에 투자
($22m)
The NewYork
Genome Center 협력
(Glioblastoma 분석)
GeneMD,
Watson Mobile Developer
Challenge의 winner 선정
Watson Fund,
Pathway Genomics 투자
Cleveland Clinic 협력
(Cancer Genome Analysis)
한국 IBM
Watson 사업부 신설
Watson Health 출범
Phytel & Explorys 인수
J&J,Apple, Medtronic 협력
Epic & Mayo Clinic 제휴
(EHR data 분석)
동경대 도입
(oncology)
14 Cancer Center 제휴
(Cancer Genome Analysis)
Mayo Clinic 협력
(clinical trail matching)
Watson Fund,
Modernizing Medicine
투자
Academia
Business
Pathway Genomics OME
closed alpha 시작
TurvenHealth
인수
Apple ResearchKit
통한 수면 연구 시작
2017
가천대 길병원
Watson 도입
(oncology)
Medtronic
Sugar.IQ 출시
제약사
Teva와 제휴
인도 Manipal Hospital
Watson 도입
태국 Bumrungrad
International Hospital,
Watson 도입
최윤섭 디지털헬스케어 연구소, 소장
(주)디지털 헬스케어 파트너스, 대표파트너
최윤섭, Ph.D.
yoonsup.choi@gmail.com
IBM Watson in Healthcare
Merge
Healthcare
인수
2016
Under Amour
제휴
Broad 연구소 협력 발표
(유전체 분석-항암제 내성)
Manipal Hospital의
WFO 정확성 발표
대구가톨릭병원
대구동산병원
WFO 도입
건양대병원
Watson 도입
(oncology)
부산대학병원
Watson 도입
(oncology/
genomics)
115. • 부산대학병원 (2017년 1월)
• Watson의 솔루션 두 가지를 도입
• Watson for Oncology
• Watson for Genomics
116. • 건양대학병원 Watson for Oncology 도입
• 2017년 3월
• “최원준 건양대병원장은 "지역 환자들은 수도권의
여러 병원을 찾아다닐 필요가 없어질 것"이라며 "병
원의 우수한 협진 팀과 인공지능 의료 시스템의 시
너지를 바탕으로 암 환자에게 최상의 의료 서비스를
제공하겠다"고 약속했다."
117.
118. IBM Watson Health
Organizations Leveraging Watson
Watson for Oncology
Best Doctors (second opinion)
Bumrungrad International Hospital
Confidential client (Bangladesh and Nepal)
Gachon University Gil Medical Center (Korea)
Hangzhou Cognitive Care – 50+ Chinese hospitals
Jupiter Medical Center
Manipal Hospitals – 16 Indian Hospitals
MD Anderson (**Oncology Expert Advisor)
Memorial Sloan Kettering Cancer Center
MRDM - Zorg (Netherlands)
Pusan National University Hospital
Clinical Trial Matching
Best Doctors (second opinion)
Confidential – Major Academic Center
Highlands Oncology Group
Froedtert & Medical College of Wisconsin
Mayo Clinic
Multiple Life Sciences pilots
24
Watson Genomic Analytics
Ann & Robert H Lurie Children’s Hospital of Chicago
BC Cancer Agency
City of Hope
Cleveland Clinic
Columbia University, Irwing Cancer Center
Duke Cancer Institute
Fred & Pamela Buffett Cancer Center
Fleury (Brazil)
Illumina 170 Gene Panel
NIH Japan
McDonnell Institute at Washington University in St. Louis
New York Genome Center
Pusan National University Hospital
Quest Diagnostics
Stanford Health
University of Kansas Cancer Center
University of North Carolina Lineberger Cancer Center
University of Southern California
University of Washington Medical Center
University of Tokyo
Yale Cancer Center
Andrew Norden, KOTRA Conference, March 2017, “The Future of Health is Cognitive”
Watson for Oncology 는 현재 전세계 70여개 병원에 도입
119. • 인공지능으로 인한 인간 의사의 권위 약화
• 환자의 자기 결정권 및 권익 증대
• 의사의 진료 방식 및 교육 방식의 변화 필요
http://news.donga.com/3/all/20170320/83400087/1
120. • 의사와 Watson의 판단이 다른 경우?
• NCCN 가이드라인과 다른 판단을 주기는 것으로 보임
• 100 여명 중에 5 case.
• 환자의 판단이 합리적이라고 볼 수 있는가?
• Watson의 정확도는 검증되지 않았음
• ‘제 4차 산업혁명’ 등의 buzz word의 영향으로 보임
• 임상 시험이 필요하지 않은가?
• 환자들의 선호는 인공지능의 adoption rate 에 영향
• 병원 도입에 영향을 미치는 요인들
• analytical validity
• clinical validity/utility
• 의사들의 인식/심리적 요인
• 환자들의 인식/심리적 요인
• 규제 환경 (인허가, 수가 등등)
• 결국 환자가 원하면 (그것이 의학적으로 타당한지를 떠나서)
병원 도입은 더욱 늘어날 수 밖에 없음
121. • Watson에 대한 환자 반응이 생각보다 매우 좋음
• 도입 2개월만에 85명 암 환자 진료
• 기존의 길병원 예측보다는 더 빠른 수치일 듯
• Big5 에서도 길병원으로 전원 문의 증가 한다는 후문
• 교수들이 더 열심히 상의하고 환자 본다고 함
122. • Trained by 400 cases of historical patients cases
• Assessed accuracy OEA treatment suggestions
using MD Anderson’s physicians’ decision as benchmark
• When 200 leukemia cases were tested,
• False positive rate=2.9%
• False negative rate=0.4%
• Overall accuracy of treatment recommendation=82.6%
• Conclusion: Suggested personalized treatment option
showed reasonably high accuracy
MDAnderson’s Oncology ExpertAdvisor Powered by IBM Watson
:AWeb-Based Cognitive Clinical Decision Support Tool
Koichi Takahashi, MD (ASCO 2014)
123. Annals of Oncology (2016) 27 (suppl_9): ix179-ix180. 10.1093/annonc/mdw601
Validation study to assess performance of IBM cognitive
computing system Watson for oncology with Manipal
multidisciplinary tumour board for 1000 consecutive cases:
An Indian experience
• MMDT(Manipal multidisciplinary tumour board) treatment recommendation and
data of 1000 cases of 4 different cancers breast (638), colon (126), rectum (124)
and lung (112) which were treated in last 3 years was collected.
• Of the treatment recommendations given by MMDT, WFO provided
50% in REC, 28% in FC, 17% in NREC
• Nearly 80% of the recommendations were in WFO REC and FC group
• 5% of the treatment provided by MMDT was not available with WFO
• The degree of concordance varied depending on the type of cancer
• WFO-REC was high in Rectum (85%) and least in Lung (17.8%)
• high with TNBC (67.9%); HER2 negative (35%)
• WFO took a median of 40 sec to capture, analyze and give the treatment.
(vs MMDT took the median time of 15 min)
124. Sung Won Park,APFCP, 2017
Assessing the performance of Watson for Oncology using colon
cancer cases treated with surgery and adjuvant chemotherapy
at Gachon University Gil Medical Center
• Stage II with high risk and stage III colon cancer patients (N=162)
• Retrospective study: From September 1, 2014 to August 31, 2016
• Gachon University Gil Medical Center (GMC)
• Generally accepted by GMC-recommendation in 83.3%
• Concordant with
• WFO-Rec: 53.1%
• WFO-FC: 30.2%
• WFO-NREC: 13.0%
• Not included: 3.7%
125. WHY?
• 국가별 가이드라인의 차이
• WFO는 기본적으로 MSKCC 기준
• 인종적 차이, 인허가 약물의 차이, 보험 제도의 차이
• NCCN 가이드라인의 업데이트
• 암종별 치료 가능한 옵션의 다양성 차이
• 폐암: 다양함 vs 직장암: 다양하지 않음
• TNBC: 다양하지 않음 vs HER2 (-): 다양함
126. 원칙이 필요하다
•어떤 환자의 경우, 왓슨에게 의견을 물을 것인가?
•왓슨을 (암종별로) 얼마나 신뢰할 것인가?
•왓슨의 의견을 환자에게 공개할 것인가?
•왓슨과 의료진의 판단이 다른 경우 어떻게 할 것인가?
•왓슨에게 보험 급여를 매길 수 있는가?
이러한 기준에 따라 의료의 질/치료효과가 달라질 수 있으나,
현재 개별 병원이 개별적인 기준으로 활용하게 됨
130. 당뇨성 망막병증
• 당뇨병의 대표적 합병증: 당뇨병력이 30년 이상 환자 90% 발병
• 안과 전문의들이 안저(안구의 안쪽)를 사진으로 찍어서 판독
• 망막 내 미세혈관 생성, 출혈, 삼출물 정도를 파악하여 진단
131. Copyright 2016 American Medical Association. All rights reserved.
Development and Validation of a Deep Learning Algorithm
for Detection of Diabetic Retinopathy
in Retinal Fundus Photographs
Varun Gulshan, PhD; Lily Peng, MD, PhD; Marc Coram, PhD; Martin C. Stumpe, PhD; Derek Wu, BS; Arunachalam Narayanaswamy, PhD;
Subhashini Venugopalan, MS; Kasumi Widner, MS; Tom Madams, MEng; Jorge Cuadros, OD, PhD; Ramasamy Kim, OD, DNB;
Rajiv Raman, MS, DNB; Philip C. Nelson, BS; Jessica L. Mega, MD, MPH; Dale R. Webster, PhD
IMPORTANCE Deep learning is a family of computational methods that allow an algorithm to
program itself by learning from a large set of examples that demonstrate the desired
behavior, removing the need to specify rules explicitly. Application of these methods to
medical imaging requires further assessment and validation.
OBJECTIVE To apply deep learning to create an algorithm for automated detection of diabetic
retinopathy and diabetic macular edema in retinal fundus photographs.
DESIGN AND SETTING A specific type of neural network optimized for image classification
called a deep convolutional neural network was trained using a retrospective development
data set of 128 175 retinal images, which were graded 3 to 7 times for diabetic retinopathy,
diabetic macular edema, and image gradability by a panel of 54 US licensed ophthalmologists
and ophthalmology senior residents between May and December 2015. The resultant
algorithm was validated in January and February 2016 using 2 separate data sets, both
graded by at least 7 US board-certified ophthalmologists with high intragrader consistency.
EXPOSURE Deep learning–trained algorithm.
MAIN OUTCOMES AND MEASURES The sensitivity and specificity of the algorithm for detecting
referable diabetic retinopathy (RDR), defined as moderate and worse diabetic retinopathy,
referable diabetic macular edema, or both, were generated based on the reference standard
of the majority decision of the ophthalmologist panel. The algorithm was evaluated at 2
operating points selected from the development set, one selected for high specificity and
another for high sensitivity.
RESULTS TheEyePACS-1datasetconsistedof9963imagesfrom4997patients(meanage,54.4
years;62.2%women;prevalenceofRDR,683/8878fullygradableimages[7.8%]);the
Messidor-2datasethad1748imagesfrom874patients(meanage,57.6years;42.6%women;
prevalenceofRDR,254/1745fullygradableimages[14.6%]).FordetectingRDR,thealgorithm
hadanareaunderthereceiveroperatingcurveof0.991(95%CI,0.988-0.993)forEyePACS-1and
0.990(95%CI,0.986-0.995)forMessidor-2.Usingthefirstoperatingcutpointwithhigh
specificity,forEyePACS-1,thesensitivitywas90.3%(95%CI,87.5%-92.7%)andthespecificity
was98.1%(95%CI,97.8%-98.5%).ForMessidor-2,thesensitivitywas87.0%(95%CI,81.1%-
91.0%)andthespecificitywas98.5%(95%CI,97.7%-99.1%).Usingasecondoperatingpoint
withhighsensitivityinthedevelopmentset,forEyePACS-1thesensitivitywas97.5%and
specificitywas93.4%andforMessidor-2thesensitivitywas96.1%andspecificitywas93.9%.
CONCLUSIONS AND RELEVANCE In this evaluation of retinal fundus photographs from adults
with diabetes, an algorithm based on deep machine learning had high sensitivity and
specificity for detecting referable diabetic retinopathy. Further research is necessary to
determine the feasibility of applying this algorithm in the clinical setting and to determine
whether use of the algorithm could lead to improved care and outcomes compared with
current ophthalmologic assessment.
JAMA. doi:10.1001/jama.2016.17216
Published online November 29, 2016.
Editorial
Supplemental content
Author Affiliations: Google Inc,
Mountain View, California (Gulshan,
Peng, Coram, Stumpe, Wu,
Narayanaswamy, Venugopalan,
Widner, Madams, Nelson, Webster);
Department of Computer Science,
University of Texas, Austin
(Venugopalan); EyePACS LLC,
San Jose, California (Cuadros); School
of Optometry, Vision Science
Graduate Group, University of
California, Berkeley (Cuadros);
Aravind Medical Research
Foundation, Aravind Eye Care
System, Madurai, India (Kim); Shri
Bhagwan Mahavir Vitreoretinal
Services, Sankara Nethralaya,
Chennai, Tamil Nadu, India (Raman);
Verily Life Sciences, Mountain View,
California (Mega); Cardiovascular
Division, Department of Medicine,
Brigham and Women’s Hospital and
Harvard Medical School, Boston,
Massachusetts (Mega).
Corresponding Author: Lily Peng,
MD, PhD, Google Research, 1600
Amphitheatre Way, Mountain View,
CA 94043 (lhpeng@google.com).
Research
JAMA | Original Investigation | INNOVATIONS IN HEALTH CARE DELIVERY
(Reprinted) E1
Copyright 2016 American Medical Association. All rights reserved.
132. Training Set / Test Set
• CNN으로 후향적으로 128,175개의 안저 이미지 학습
• 미국의 안과전문의 54명이 3-7회 판독한 데이터
• 우수한 안과전문의들 7-8명의 판독 결과와 인공지능의 판독 결과 비교
• EyePACS-1 (9,963 개), Messidor-2 (1,748 개)a) Fullscreen mode
b) Hit reset to reload this image. This will reset all of the grading.
c) Comment box for other pathologies you see
eFigure 2. Screenshot of the Second Screen of the Grading Tool, Which Asks Graders to Assess the
Image for DR, DME and Other Notable Conditions or Findings
134. • EyePACS-1 과 Messidor-2 의 AUC = 0.991, 0.990
• 7-8명의 안과 전문의와 sensitivity, specificity 가 동일한 수준
• F-score: 0.95 (vs. 인간 의사는 0.91)
Additional sensitivity analyses were conducted for sev-
eralsubcategories:(1)detectingmoderateorworsediabeticreti-
effects of data set size on algorithm performance were exam-
ined and shown to plateau at around 60 000 images (or ap-
Figure 2. Validation Set Performance for Referable Diabetic Retinopathy
100
80
60
40
20
0
0
70
80
85
95
90
75
0 5 10 15 20 25 30
100806040
Sensitivity,%
1 – Specificity, %
20
EyePACS-1: AUC, 99.1%; 95% CI, 98.8%-99.3%A
100
High-sensitivity operating point
High-specificity operating point
100
80
60
40
20
0
0
70
80
85
95
90
75
0 5 10 15 20 25 30
100806040
Sensitivity,%
1 – Specificity, %
20
Messidor-2: AUC, 99.0%; 95% CI, 98.6%-99.5%B
100
High-specificity operating point
High-sensitivity operating point
Performance of the algorithm (black curve) and ophthalmologists (colored
circles) for the presence of referable diabetic retinopathy (moderate or worse
diabetic retinopathy or referable diabetic macular edema) on A, EyePACS-1
(8788 fully gradable images) and B, Messidor-2 (1745 fully gradable images).
The black diamonds on the graph correspond to the sensitivity and specificity of
the algorithm at the high-sensitivity and high-specificity operating points.
In A, for the high-sensitivity operating point, specificity was 93.4% (95% CI,
92.8%-94.0%) and sensitivity was 97.5% (95% CI, 95.8%-98.7%); for the
high-specificity operating point, specificity was 98.1% (95% CI, 97.8%-98.5%)
and sensitivity was 90.3% (95% CI, 87.5%-92.7%). In B, for the high-sensitivity
operating point, specificity was 93.9% (95% CI, 92.4%-95.3%) and sensitivity
was 96.1% (95% CI, 92.4%-98.3%); for the high-specificity operating point,
specificity was 98.5% (95% CI, 97.7%-99.1%) and sensitivity was 87.0% (95%
CI, 81.1%-91.0%). There were 8 ophthalmologists who graded EyePACS-1 and 7
ophthalmologists who graded Messidor-2. AUC indicates area under the
receiver operating characteristic curve.
Research Original Investigation Accuracy of a Deep Learning Algorithm for Detection of Diabetic Retinopathy
Results
138. LETTERH
his task, the CNN achieves 72.1±0.9% (mean±s.d.) overall
he average of individual inference class accuracies) and two
gists attain 65.56% and 66.0% accuracy on a subset of the
set. Second, we validate the algorithm using a nine-class
rtition—the second-level nodes—so that the diseases of
have similar medical treatment plans. The CNN achieves
two trials, one using standard images and the other using
images, which reflect the two steps that a dermatologist m
to obtain a clinical impression. The same CNN is used for a
Figure 2b shows a few example images, demonstrating th
distinguishing between malignant and benign lesions, whic
visual features. Our comparison metrics are sensitivity an
Acral-lentiginous melanoma
Amelanotic melanoma
Lentigo melanoma
…
Blue nevus
Halo nevus
Mongolian spot
…
Training classes (757)Deep convolutional neural network (Inception v3) Inference classes (varies by task)
92% malignant melanocytic lesion
8% benign melanocytic lesion
Skin lesion image
Convolution
AvgPool
MaxPool
Concat
Dropout
Fully connected
Softmax
Deep CNN layout. Our classification technique is a
Data flow is from left to right: an image of a skin lesion
e, melanoma) is sequentially warped into a probability
over clinical classes of skin disease using Google Inception
hitecture pretrained on the ImageNet dataset (1.28 million
1,000 generic object classes) and fine-tuned on our own
29,450 skin lesions comprising 2,032 different diseases.
ning classes are defined using a novel taxonomy of skin disease
oning algorithm that maps diseases into training classes
(for example, acrolentiginous melanoma, amelanotic melano
melanoma). Inference classes are more general and are comp
or more training classes (for example, malignant melanocytic
class of melanomas). The probability of an inference class is c
summing the probabilities of the training classes according to
structure (see Methods). Inception v3 CNN architecture repr
from https://research.googleblog.com/2016/03/train-your-ow
classifier-with.html
GoogleNet Inception v3
• 129,450개의 피부과 병변 이미지 데이터를 자체 제작
• 미국의 피부과 전문의 18명이 데이터 curation
• CNN (Inception v3)으로 이미지를 학습
• 피부과 전문의들 21명과 인공지능의 판독 결과 비교
• 표피세포 암 (keratinocyte carcinoma)과 지루각화증(benign seborrheic keratosis)의 구분
• 악성 흑색종과 양성 병변 구분 (표준 이미지 데이터 기반)
• 악성 흑색종과 양성 병변 구분 (더마토스코프로 찍은 이미지 기반)
139. Skin cancer classification performance of
the CNN and dermatologists. LETT
a
b
0 1
Sensitivity
0
1
Specificity
Melanoma: 130 images
0 1
Sensitivity
0
1
Specificity
Melanoma: 225 images
Algorithm: AUC = 0.96
0 1
Sensitivity
0
1
Specificity
Melanoma: 111 dermoscopy images
0 1
Sensitivity
0
1
Specificity
Carcinoma: 707 images
Algorithm: AUC = 0.96
0 1
Sensitivity
0
1
Specificity
Melanoma: 1,010 dermoscopy images
Algorithm: AUC = 0.94
0 1
Sensitivity
0
1
Specificity
Carcinoma: 135 images
Algorithm: AUC = 0.96
Dermatologists (25)
Average dermatologist
Algorithm: AUC = 0.94
Dermatologists (22)
Average dermatologist
Algorithm: AUC = 0.91
Dermatologists (21)
Average dermatologist
cancer classification performance of the CNN and
21명 중에 인공지능보다 정확성이 떨어지는 피부과 전문의들이 상당수 있었음
피부과 전문의들의 평균 성적도 인공지능보다 좋지 않았음
140. Skin cancer classification performance of
the CNN and dermatologists. LETT
a
b
0 1
Sensitivity
0
1
Specificity
Melanoma: 130 images
0 1
Sensitivity
0
1
Specificity
Melanoma: 225 images
Algorithm: AUC = 0.96
0 1
Sensitivity
0
1
Specificity
Melanoma: 111 dermoscopy images
0 1
Sensitivity
0
1
Specificity
Carcinoma: 707 images
Algorithm: AUC = 0.96
0 1
Sensitivity
0
1
Specificity
Melanoma: 1,010 dermoscopy images
Algorithm: AUC = 0.94
0 1
Sensitivity
0
1
Specificity
Carcinoma: 135 images
Algorithm: AUC = 0.96
Dermatologists (25)
Average dermatologist
Algorithm: AUC = 0.94
Dermatologists (22)
Average dermatologist
Algorithm: AUC = 0.91
Dermatologists (21)
Average dermatologist
cancer classification performance of the CNN and
142. Figure 4. Participating Pathologists’ Interpretations of Each of the 240 Breast Biopsy Test Cases
0 25 50 75 100
Interpretations, %
2
4
6
8
10
12
14
16
18
20
22
24
26
28
30
32
34
36
38
40
42
44
46
48
50
52
54
56
58
60
62
64
66
68
70
72
Case
Benign without atypia
72 Cases
2070 Total interpretations
A
0 25 50 75 100
Interpretations, %
218
220
222
224
226
228
230
232
234
236
238
240
Case
Invasive carcinoma
23 Cases
663 Total interpretations
D
0 25 50 75 100
Interpretations, %
147
145
149
151
153
155
157
159
161
163
165
167
169
171
173
175
177
179
181
183
185
187
189
191
193
195
197
199
201
203
205
207
209
211
213
215
217
Case
DCIS
73 Cases
2097 Total interpretations
C
0 25 50 75 100
Interpretations, %
74
76
78
80
82
84
86
88
90
92
94
96
98
100
102
104
106
108
110
112
114
116
118
120
122
124
126
128
130
132
134
136
138
140
142
144
Case
Atypia
72 Cases
2070 Total interpretations
B
Benign without atypia
Atypia
DCIS
Invasive carcinoma
Pathologist interpretation
DCIS indicates ductal carcinoma in situ.
Diagnostic Concordance in Interpreting Breast Biopsies Original Investigation Research
Elmore etl al. JAMA 2015
Diagnostic Concordance Among Pathologists
Interpreting Breast Biopsy Specimens
The overall agreement between the individual pathologists’
interpretations and the expert consensus–derived reference
diagnoses was 75.3% (total 240 cases)
143. Elmore etl al. JAMA 2015
Diagnostic Concordance Among Pathologists
Interpreting Breast Biopsy Specimens
• Concordance noted in 5194 of 6900 case interpretations or 75.3%.
• Reference diagnosis was obtained from consensus of 3 experienced breast pathologists.
spentonthisactivitywas16(95%CI,15-17);43participantswere
awarded the maximum 20 hours.
Pathologists’ Diagnoses Compared With Consensus-Derived
Reference Diagnoses
The 115 participants each interpreted 60 cases, providing 6900
total individual interpretations for comparison with the con-
sensus-derived reference diagnoses (Figure 3). Participants
agreed with the consensus-derived reference diagnosis for
75.3% of the interpretations (95% CI, 73.4%-77.0%). Partici-
pants (n = 94) who completed the CME activity reported that
Patient and Pathologist Characteristics Associated With
Overinterpretation and Underinterpretation
The association of breast density with overall pathologists’
concordance (as well as both overinterpretation and under-
interpretation rates) was statistically significant, as shown
in Table 3 when comparing mammographic density grouped
into 2 categories (low density vs high density). The overall
concordance estimates also decreased consistently with
increasing breast density across all 4 Breast Imaging-
Reporting and Data System (BI-RADS) density categories:
BI-RADS A, 81% (95% CI, 75%-86%); BI-RADS B, 77% (95%
Figure 3. Comparison of 115 Participating Pathologists’ Interpretations vs the Consensus-Derived Reference
Diagnosis for 6900 Total Case Interpretationsa
Participating Pathologists’ Interpretation
ConsensusReference
Diagnosisb
Benign
without atypia Atypia DCIS
Invasive
carcinoma Total
Benign without atypia 1803 200 46 21 2070
Atypia 719 990 353 8 2070
DCIS 133 146 1764 54 2097
Invasive carcinoma 3 0 23 637 663
Total 2658 1336 2186 720 6900
DCIS indicates ductal carcinoma
in situ.
a
Concordance noted in 5194 of
6900 case interpretations or
75.3%.
b
Reference diagnosis was obtained
from consensus of 3 experienced
breast pathologists.
Diagnostic Concordance in Interpreting Breast Biopsies Original Investigation Research
Comparison of 115 Participating Pathologists’ Interpretations vs
the Consensus-Derived Reference Diagnosis for 6900 Total Case Interpretations
144. Constructing higher-level
contextual/relational features:
Relationships between epithelial
nuclear neighbors
Relationships between morphologically
regular and irregular nuclei
Relationships between epithelial
and stromal objects
Relationships between epithelial
nuclei and cytoplasm
Characteristics of
stromal nuclei
and stromal matrix
Characteristics of
epithelial nuclei and
epithelial cytoplasm
Building an epithelial/stromal classifier:
Epithelial vs.stroma
classifier
Epithelial vs.stroma
classifier
B
Basic image processing and feature construction:
H&E image Image broken into superpixels Nuclei identified within
each superpixel
A
Relationships of contiguous epithelial
regions with underlying nuclear objects
Learning an image-based model to predict survival
Processed images from patients Processed images from patients
C
D
onNovember17,2011stm.sciencemag.orgwnloadedfrom
TMAs contain 0.6-mm-diameter cores (median
of two cores per case) that represent only a small
sample of the full tumor. We acquired data from
two separate and independent cohorts: Nether-
lands Cancer Institute (NKI; 248 patients) and
Vancouver General Hospital (VGH; 328 patients).
Unlike previous work in cancer morphom-
etry (18–21), our image analysis pipeline was
not limited to a predefined set of morphometric
features selected by pathologists. Rather, C-Path
measures an extensive, quantitative feature set
from the breast cancer epithelium and the stro-
ma (Fig. 1). Our image processing system first
performed an automated, hierarchical scene seg-
mentation that generated thousands of measure-
ments, including both standard morphometric
descriptors of image objects and higher-level
contextual, relational, and global image features.
The pipeline consisted of three stages (Fig. 1, A
to C, and tables S8 and S9). First, we used a set of
processing steps to separate the tissue from the
background, partition the image into small regions
of coherent appearance known as superpixels,
find nuclei within the superpixels, and construct
Constructing higher-level
contextual/relational features:
Relationships between epithelial
nuclear neighbors
Relationships between morphologically
regular and irregular nuclei
Relationships between epithelial
and stromal objects
Relationships between epithelial
nuclei and cytoplasm
Characteristics of
stromal nuclei
and stromal matrix
Characteristics of
epithelial nuclei and
epithelial cytoplasm
Epithelial vs.stroma
classifier
Epithelial vs.stroma
classifier
Relationships of contiguous epithelial
regions with underlying nuclear objects
Learning an image-based model to predict survival
Processed images from patients
alive at 5 years
Processed images from patients
deceased at 5 years
L1-regularized
logisticregression
modelbuilding
5YS predictive model
Unlabeled images
Time
P(survival)
C
D
Identification of novel prognostically
important morphologic features
basic cellular morphologic properties (epithelial reg-
ular nuclei = red; epithelial atypical nuclei = pale blue;
epithelial cytoplasm = purple; stromal matrix = green;
stromal round nuclei = dark green; stromal spindled
nuclei = teal blue; unclassified regions = dark gray;
spindled nuclei in unclassified regions = yellow; round
nuclei in unclassified regions = gray; background =
white). (Left panel) After the classification of each
image object, a rich feature set is constructed. (D)
Learning an image-based model to predict survival.
Processed images from patients alive at 5 years after
surgery and from patients deceased at 5 years after
surgery were used to construct an image-based prog-
nostic model. After construction of the model, it was
applied to a test set of breast cancer images (not
used in model building) to classify patients as high
or low risk of death by 5 years.
www.ScienceTranslationalMedicine.org 9 November 2011 Vol 3 Issue 108 108ra113 2
onNovember17,2011stm.sciencemag.orgDownloadedfrom
Digital Pathologist
Sci Transl Med. 2011 Nov 9;3(108):108ra113