[Music]
Hi everyone. So next up we have
object-oriented oncology. How we make
sense of complex patient journeys
presented by Georgie and Gazla. Take it
away guys.
All right. Thank you so much.
[Applause]
Uh oh yeah. Um uh so thanks. Our
presentation today is made up of kind of
three stories that we're going to weave
together so that you can understand
where we are going um and why we're
trying to work on this. Um so the first
of those stories is about open science
communities um and the role of data
standards for generating real world
evidence. Um and the second is
specifically about oncology research.
And then the third is more about
research translation um or take the path
that you have to take from the insight
to making an impact in the clinic where
you embed findings from research into
care delivery to make a real difference
for patients. Um and I think on the
surface those stories might not look
like they're intentioned terribly much.
Um and in fact in a lot of ways uh they
are different sides of the same coin.
But when you look a bit closer um we
find that there's a b a bunch of subtle
ways that there actually are competing
priorities. Um and I think you know the
title of this presentation as well as
the fact that we're at PyCon um probably
gives a fair amount of our solution away
um to be honest. Uh so it's a spoiler
alert. Um it has a lot to do with Python
objects and OMS.
Um so Gazella and I are joining you here
today from um a group called Maradulu
Budari Gumar um or Sphere which is a
multiddisciplinary translational
research network um that is made up of
health system and academic partners and
it covers a lot of southern Sydney. So
these are the organizations that make up
the sphere network. Um, and I think the
most helpful thing to know about um, the
translational centers and how they work
is that they've been set up to deliver
research that is embedded into healthare
um, to improve health outcomes and
equity. Um, and it means that typically
researchers have appointments at
multiple institutions that facilitates a
flow of data uh, resources and expertise
um, across the system. So the work that
we're sharing today uh represents the
efforts of a team that straddles
southwestern Sydney local health
district, the University of New South
Wales and the Ingram Institute for
Applied Medical Research.
Um specifically we're a part of the uh
cancer clinical academic group and I
like to share this slide in
presentations because I think it
illustrates uh really the
multi-disiplinary nature of our work um
where we're we are embedded with
clinical leadership and clinician
researchers all across the cancer care
continuum. So every one of our uh
clinical colleagues actively practices
uh clinical cancer care delivery in
parallel with their research. Um and we
also collaborate with implementation
scientists who are there to help us work
with the people and the processes of the
health system.
Um and again with PyCon being the clue um
although overall we are a
multi-disiplinary gang um our home
specifically is in the reducing
variation in clinical practice area of
focus um which is the one that's the
most strongly aligned with data science
um and research data platform
development efforts.
Um so specifically our overarching
mission um is to use routinely collected
observational data uh to study what is
unwarranted variation in clinical cancer
care. Um, I'd say that warranted
variation uh comprises treatment that
deviates from what is considered the
evidence-based care, but it's due to uh
reasons specifically around patients
co-orbidities um or informed patient
decision-m
um whereas the residual variation that
can't be explained in those ways uh
represents treatment that's actually
substandard um and that's our target for
increasing understanding and
interventions.
Um so the goal is to use um routinely
collected data to generate evidence um
for best practice care across all cancer
domains. Um and we broke this into kind
of general families or shapes of
questions. Um our first step is
characterizing the type of variations
that exist. Um I think that most health
consumers uh probably would be surprised
at the gap that exists between what
their expectations are and the actual
level of oversight and business
intelligence that happens at a clinical
level. um simply describing the systemic
gaps and where they exist can already be
incredibly powerful. Um but once we've
actually measured that variation, we
face the more difficult and more
abstract task um understanding why that
variation exists. Um and that's
essential because without it uh we can't
know whether or not the particular
difference in care is warranted. So we
need to know who's experiencing the
variation, which populations are
affected, and are they affected in
disproportionate fashion? Um
and then so and and in what ways? And
then so what so which outcomes can we
actually um attribute to that variation?
What differences it makes to patients
and providers um or to the system as a
whole? And finally the holy grail that
we've got is what can we actually do
about it? Um what capacity do we have to
affect um to make a change there and
reduce the unwarranted variation and
what kinds of actions make an actual
difference in practice.
So if we start with the first story um
open science communities and data
standards for real world evidence in a
clinical trial setting everything is
very very well uh tightly controlled um
and that pro protocols are specified
very rigidly um which includes strong
restrictions as to which patients can
even take part in the trial and there
are reasons that this does make sense.
Um it's much much easier to work out um
what how well a treatment is working if
you remove the complexities of a mixed
population. Um this is on top of the
fact that trial participation risks vary
a lot um depending on different groups.
So it means that frail patients, those
with a lot of um multiple other
conditions at the same time and the
elderly for example um are often
excluded. But the catch is that for
cancer those specific excluded
populations are the ones who look like
the patients that are actually seen in
the clinic most of the time. Um and the
gap between the wi the world of
protocols um and the world of practice
is just one of the myriad arguments for
why real world evidence matters so much.
Um there are a lot of challenges with
this but a big one is scale. So to
generate compelling evidence in a noisy
real world context you need a huge
amount of data. Um to make sure that
those insights are actually
generalizable the data has to come from
disconnected sources um including
administrative records, registries,
different institutions and different
care settings. And of course scaling the
data um brings with it a very real risk
to patient privacy. Um and most of the
time the preferred solution is not to
pull all of this data in a central
location. Um instead we use the approach
of uh federated analytics where
basically the data stays as close as it
can to the generating institution. Um it
may be inside the its generating network
or it may be you know in an associated
group but certainly not sending it to a
central um pool. And then we use careful
data harmonization uh to allow us to run
study packages in a distributed fashion.
Um so I'm going to hand over to Gazilla
now to help you understand um some of
the ways that we work towards um that
properly federated network um the common
data model.
Thanks Georgie. Um yes to make federated
analytics and u network studies possible
we need a shared language for uh the
data set itself. That's where common
data models come in. The idea is simple.
Every institution can keep its own
system and uh structures, but data is
transformed through ETL pipelines into a
standard format that everyone agrees on.
Once that's done, um the same analysis
can run in many different places and uh
the results can be combined.
Uh one of the most widely used
approaches in health research uh is the
OMOP common data model. It gives us a
consistent way to present um um to
represent clinical data and it's um
supported by an open science community
ODC community uh that builds tools and
uh methods on top of it. But designing a
single model that works across every
institution, every clinical domain and
every use case is a hugely ambitious
task. Uh here's the catch. When uh you
try to build a model that does
everything, you have to make tradeoffs
to to um um cover all um these different
use cases. Common data models like the
OMOP common data models usually choose
uh flexibility over ease of use. Uh most
of the details comes from the
vocabularies
um the the vocabularies they linked to
rather than from complex relationships
between the tables. Uh that's why
clinical context like events,
measurements, treatments and outcomes is
stored in a fairly simple domain
specific tables.
Um of course these design uh tradeoffs
come with their own challenges. In this
table we can see the uh main design
choices behind OMOP. A lot of OMO's
flexibility comes from using linked
vocabularies to carry meaning rather
than building strict rules into the
database itself. That makes it flexible,
but it also means it's harder to enforce
rules uh capture complex com uh
relationship or answer detailed question
with simple queries. And let's be
honest, most people you using these data
models are clinicians first, researchers
second, and only reluctantly database
experts. So the result is often long
complex queries that build up over time.
This um shows this slows things down and
um standards usually end up in documents
instead of being built into the system
where they can be tested or validated.
Um this isn't a criticism. It's just the
reality of who's involved and in many
ways it shows the effort of the
community. people with uh deep um domain
experts turning complex messy clinical
data into something that can actually
truly generate evidence.
Now let's turn to oncology research
specifically
uh in the observational research
community. It's almost a cliche to start
by saying oncology data are complicated,
but the truth is they really are. And uh
the one sizefits all solutions uh we
just discussed um often clash with that
level of complexity.
Uh what makes uh the cancer patient
journey so challenging to describe? a
few key uh key factors. First, patients
have long timelines with many events and
multi-pety care happening over months or
years. Uh data covers many types of OMIX
and treatment from different specialties
and matching the right treatment to the
right induction uh indication is um
often complicated.
uh tumors can return or progress um at
unpredictable times and treatment
decisions shifts accordingly. Um and
capturing these changes over time
requires careful tracking and finally
treatments are demanding and in uh
entirely valid for some variation in
care to exist. For instance, a patient
might choose to uh pause or stop uh
their therapy for their own well-being.
All of these uh means that the standard
tricks that uh we use to simplify
observational data can quickly run into
uh limits when um applied to ancology.
Navigating the balance between the
general and uh oncology specific need is
a pattern that we see at many points in
the clinical research uh life cycle.
There are a bunch of different clinical
data models where you see the same
pattern the tension between general
purpose models and specific needs of
oncology. Many general purpose um
clinical and research data models
include a tailored set of conventions
designed to handle the complexities of
uh cancer care. Essentially an oncology
specific layer built on top of the uh
broader model. Uh for example uh CIDISK
uh has um cancer therapeutic area
specification for oncology clinical
trial. Fire has M code for uh
operational oncology data capture and
OMOP has the oncology extension for
observational studies and real world
evidence.
The general MO model provides the
framework for almost anything. While the
oncology specific uh subset gives us the
shortcuts and rules we actually need to
model this complexity effectively.
The version of ankology specific subset
keeps the strict structure of the
general purpose tables but adds a few
important changes. The most important is
the concept concept of modifiers. Uh
records from different tables can be
linked uh in predicted pre predefined uh
ways to produce a richer more clinically
meaningful description of the disease
state. For example, disease stage
morphology and great aren't just
captured. They're now properly tied to
uh the diagnosis record itself which is
critical for for our understanding of uh
the patient trajectory. Under uh surface
this might not sound gamechanging but
the impact is big. Um it turns a limited
general model into something truly
usable for oncology studies.
And the other main innovation comes from
the introduction of episodes
uh linking related events together to
describe the full course of the disease
from diagnosis, response, recurrence and
survival as well as grouping treatments
into their cycles, regimens and lines of
therapy.
This is greatly improves our ability to
appropriately characterize the t the
target populations of interest. Um this
is an extreme example but the point
remains it is uh impossible to uh single
out study populations without uh linking
together complex chains of event across
disease progression and lines of
therapy.
Okay, we have gotten through our first
two uh stories without a single mention
of Python. Georgie will uh share our
third and final uh story, the one on
research translation and uh what she has
developed and uh which I've been using
uh since I joined uh the group to work
closely with clinicians and answer their
questions.
All right, thank you. Um so at its
simplest research translation is the
process um of taking insights from the
research world and embedding them into
care. Um so they don't just sit in a
paper but make actual difference to
patients. Um and it could mean finding a
new clinical guideline um building a
decision support tool or even just
changing how the the care teams um
reflect on their own practice. At its
most ambitious, we do start to
approximate the elusive learning health
system um which was promised as just
around the corner in 2007. Um where
datadriven knowledge continually uh
drives practice improvement. Um, and I
think there's a there's a whole lot of
systemwide um and systemic reasons for
this failure to deliver, but at least
one of them is the disconnect between
wanting to um implement only
interventions that have actually been
proven to work, but then in the real
world of siloed data environments, um,
we struggle to replicate exactly uh in a
different context because the underlying
data can't easily be aligned. Um, and I
stole this code from a tutorial on
getting started with the oncology
extension because I think it gives a
handle on what is typical in practice.
Um it's a very slight exaggeration
because there are there are some
graphical interfaces for generating
static queries like this one. Um so I'm
somewhat like uh leaning on the idea of
literally having to look up the codes
for each and every condition. Uh but not
by very much and if you actually want to
explore the data in a hands-on way. Um
and you can see our delightful uh
polymorphic keys make an exper an an
appearance there as well. Um so how do
you actually use these kinds of static
queries to build a learning health
system? You don't. um we saw an
opportunity for enriching the fluency of
these investigations with Python. Um and
I wasn't sure how much uh background I
need to provide for this audience
because I'm very very comfortable now uh
giving technical talks to clinicians um
which is its own challenge. Um but this
is the first time that we've had like
the opposite uh story. So you'll have to
forgive me but just in case um you're
not familiar with OMS or object object
relational mappings I'll take just a
minute to kind of go through the
conceptual grounding. Um so starting
with the actual database hopefully it's
fair to assume everyone's got a good
mental model for what that looks like.
Um but then we have the application
layer uh where we describe the code in
code a representation of the underlying
data model. Um and it allows us to
interact with the database records
directly as instantiated Python objects.
So what does this actually look like? Um
if we look at the mapping of the
ubiquitous concept table for a second.
Um table definitions are expressed as
classes. Um a row is an instance of that
class. Columns are attributes. um and
the OM specific mappings of the columns
define the details of things like the
keys and the data types.
And then finally we have the abstraction
which is where the guts of the OM lie.
So this provides a translation between
the back and the front ends of the stack
doing things like converting the Python
code into dialect specific SQL um and
instantiating Python objects that
represent the returned results of
executed queries.
So how does this improve the situation
for clinician researchers and for
actually getting those results into the
clinic? So for one thing um Python
objects are far more expressive than the
database rows. The generic tables of
OMOP can actually be turned into objects
that have some uh that can capture the
their own purpose and their use in
practice which makes it a lot easier for
researchers to find the details that
they're looking for. So in practice, a
richly expressed relational algebra
turns many of these queries into
functions um which has the side benefit
of making the ETL itself the ETL process
itself inherently validatable at design
time not just post hawk um on the
transformed data. A concept object
that's returned as a unit concept um has
enough context to know that it needs to
behave differently to one that was
returned via value as concept um for a
basic example. And this can even extend
to that concept object having different
behavior depending on further relational
details like dates or age variables.
Um, and here are a couple of use cases
for why you might want to do this. Um,
so the first is what I just mentioned,
bringing those va data validation checks
further up the ETL pipeline. It's a
trivial example, but you can use the
validates decorator to intercept events
when an object is created or modified.
And then you raise an exception, so it's
impossible to write improperly formed
records to the database. or you can
choose to handle it and modify the
incorrect data before writing it out.
And this turns the typical OMO best
practice for post hoc data quality
checks on its head. Um, and if you need
to reference vocabulary tables to
perform the domain checks or similar,
you can employ class level methods
instead pulling in um the context when
you're uh building the class. Um,
so here are some other examples showing
how it's possible to provide to validate
one field relative to another within the
same table. Um, and furthermore, how you
can customize the event that the
validation scripts actually listen for.
Um, which is very useful when you want
to validate complex objects like
episodes which are built in like a
step-wise fashion.
Um, and this is a simplified uh version
of our visualization stack. Um, you
know, and there's a couple of reasons to
do it this way. Um, but I think the main
reason that uh this is an innovation for
for this user group um is that it's all
loosely coupled into layers. Um, so it
means that you can build and reuse the
core application logic in multiple
places. um just changing the way that we
interact with the published API. Um and
then here's a basic version of our API
layer. Um and I always like to stick on
that warning um there just in case
there's any way that people can think
we're mishandling our private uh patient
data. Um, but here's an example of where
the same underlying condition logic can
be applied either as a normal condition
via the condition schema.
Um, or you can render it as a cancer
specific schema um, by applying a
different serialization logic that
starts to pull in those modifier objects
that Gazilla was telling you about
before um, that just get nested below
the parent JSON.
And this is one of my proudest moments
um solving this through the OM approach
which is uh handling the proper handling
of the polymorphic relationships between
the measurements um observations and
episode tables um because each of these
can point to a number of different
targets. And so this intermediary class
abstracts away that complexity um so
it's hidden to the end users. They don't
really need to know that they need to
look in like seven different locations
to piece together the event trail.
It means you can conveniently grab all
of the episode events as a collection um
directly via the events property instead
of needing to handle them as flattened
joins against each individual target. Um
and the logical uh endgame here is this
query where you can return an individual
all of their nested episodes and all of
the episode events of any type uh with a
single call. and final one. Um, but
there's a few examples of ways to
implement extended properties to build
convention logic um, within the data
model definition and it takes care of
all of the complexity of handling for
instance modality of treatment um,
episodes. And again, it be it ends up
being impossible to misinterpret those
documents with long descriptive
conventions um, if you have an agreed
implementation of the most important
ones. And we do need to do more work on
this one in terms of the design patterns
for holding the more basic uh raw
objects and then their enriched subasses
because there is a fairly serious uh
performance implication with this one.
Um and I wanted to end just by rolling
back up to the bigger picture um as to
our kind of whole research program. Um,
so we're using these tools all across
the research pipeline um to identify the
unwarranted variation, understand what
drives it, um, and then pushing those
insights back into the clinic. Um, all
for our ultimate goal, um, of improving
patient experience and outcomes. And
just to leave you wanting a bit more for
next time we come, um, we're currently
using this uh, all of this is the basis
for a whole data visualization framework
um, to support the deployment of these
research visualizations directly into
the health district environments. Um so
we we leverage the highly regularized
structure of the common data model to to
create flexible arbitrary depth um
queries for reporting to
multi-disiplinary team meetings. Um we
have a beta version of the libraries. Um
if you want to have a play, please get
into it and let us know what you think.
Okay.
[Applause]
Thanks guys. That was fantastic. Um
we've got time for a few questions. Has
we got any questions from the audience?
And we got one at the back there.
Um, thank you for an excellent talk. Um,
given that you're trying to track
improvements to the situation, I'm
assuming, um, is that visualization that
you're talking about supposed to help,
um, see that those changes are having an
effect? Yeah, I mean so at the moment
the the multi-disiplinary teams some of
them have a some kind of a datadriven
practice where they will manually
abstract reports that they pull out like
um from all different systems on a
six-monthly cycle right and um I think
for teams who are like that we've seen
that that has actually improved practice
um but it's on such a long time scale
that the answer you get is like I can't
tell you who that was um I I'm sure
there was a reason that that happened oh
we've changed our practice by now So
bringing that um closer to real time um
has the capacity to do that. The other
thing as well is like being able to see
the trend because if something's below a
given benchmark but it's trending
upwards that's a very different story to
something that is just above the
benchmark but is you know trending
downwards. So these are the these are
the things we're working with the
implementation science groups um to you
know collaborate with the clinicians and
co-design their visualizations that they
think will actually make a difference in
the clinic.
Uh Paul Leardy. Hi. Um in your setting,
do you have instances where you need to
do performance optimization?
And if so, how do you go about doing it?
Are you talking about uh like data
performance or clinician performance?
Sorry, I assume data. No,
I'm talking about performance of the
code, not
Oh, yes. Oh, yeah.
And this is I think harking back to one
of the talks this morning where
sometimes sometimes the SQL is just the
right way to do things. But I think that
that probably is much more relevant in a
case where you have something that is
designed for these kinds of uses and
there are so many compromises with this
schema that the performance is uh it
doesn't matter what you do it is it's
going to be slow. Um and so we are um
working out the design patterns at the
moment and we've had uh yeah some some
luck kind of cing and materializing
those views as well. So yeah, we're
working on it. Um, at the moment it's a
six-monthly manual process. If we can do
a weekly one, that's already huge. So
yes and no.
Cool. Thanks very much. I did like the
comment in there um in the code saying
not really sure this should be uh
repeated in this way. And I guess I'm
interested to sort of hear what you've
learned about sort of good programming
in the process of doing this. What sort
of things would you like other people to
be doing?
Um yeah, again so we have a lot of um
trying to make small winds and
on-roading like trying to build on-ramps
for people to um become familiar with
things. And so there are definitely
sometimes where you you would like to do
something nifty and performant and smart
and all these kinds of things, but that
uh actually a big part of our role in
the job that we're in is to um yeah help
people who are keen to kind of uh
develop those practices. Um so yeah, it
it is definitely a um a balance for u
being able to communicate and and try
and help people kind of hit that level
of complexity uh that they need to get
to. um because there is there's
enthusiasm, a lot of informal kind of ad
hoc efforts, but there's not a lot of um
good cohesion across the practice. I
would say
I think we maybe have time for one last
question if anyone else has anything
they want to ask.
Thanks for the talk. Um you've kind of
got this unified semantic ontology to u
for clinicians to use. Yeah.
Um, in my context, I've found that it's
great having everyone speaking the same
language, but then uh or using the same
words, but then they use them in
different ways, and it's hard to
actually, you know, uh, communicate
between people. I was wondering if you
could comment on how that's been trying
to get clinicians to use that language
in the right way or in a consistent
manner.
Yeah. So, um, the mappings, we don't we
don't put any constraints on the data
entry side of things. that's that's a
hiding to nothing. Um you what will
happen if you do that in too rigid of
fashion it'll be they'll just put it in
the free text instead. So we we don't
ask the clinicians to do that. Um and
instead we try and kind of uh use sort
of pragmatic engineering decisions to
pull the pipeline. Um yeah and make sure
that we're doing it at the right stage
in the process.
Well, thank you very much guys. Here's a
pyon mug for each of you to say thank
you.
Thank you so much.
Thank you. Thank you very much.