foreign
welcome back everyone I hope everyone
had a good lunch
uh our first Speaker back after lunch is
Seoul kaganoff principal at Deloitte and
involved in the apis AIDS Australia
conference so so please take it away
okay thank you
okay well thanks for coming along today
I hope you're all enjoying the
conference I'm having a great time I got
my geekong with the uh ska earlier today
um right
um so I'm talking about not only apis
for the connected universe but also
events related to the connected universe
as well I guess a motivation for picking
this topic is that I work with a lot of
people on designing distributed
Solutions on designing apis and they've
been driven architectures Etc and I'm
increasingly hearing people so it's a
vibe that I observe
um a lot of people saying oh we want to
make everything now event driven and
asynchronous and that's going to be the
new goodness and I kind of worry when I
hear that because I think there's
benefits and drawbacks to events
event-driven architecture there's
benefits and drawbacks to apis and so I
wanted to talk about that today and talk
about how you can conceptualize the two
how they relate to each other and how
you can make judicious choice assets as
to um which way to go
so just in terms of um defining what
we're talking about I think you all know
about apis so I'll probably just skip
over that one but I'm talking about web
apis not so much modbus apis but with
web apis
um and when we talk about events we're
talking about the representation of
something that has already happened
typically in the world things like
somebody's clicked the button on the
user user interface
a trade has happened on the stock market
so we've got a new price tick on a
particular instrument
oh we had something like a plane has
landed or a train goes over a way Bridge
etc those are events that we are
interested in in our systems
um and when we talk about event-driven
architecture then typically these are
solutions or applications where we are
using events to uh to deliver an outcome
to communicate information between
Loosely coupled services and good
examples of events are Twitter where
we're getting this Pub sub I'm I'm
forever grateful to Twitter for making
everybody understand what Pub sub means
right publish subscribe
podcasts are event driven you might not
know the behind your podcast feed is a
thing called RSS which is a list of
events representing the publication of a
media stream
um and then you know areas like
Logistics and those sorts of things
utilize event-driven architecture
extensively
so
if I'm a solution architect and I'm
deciding how to design a solution I
might be looking at well should it be
API driven should it be event driven and
there's various trade-offs that I can
consider from that perspective so were
they if we look at apis how do I use
apis to deliver an outcome there's some
benefits around apis they're pretty easy
to use the tooling is very mature the
tools and methods are very mature we've
got lots of affordances built into our
libraries Etc so apis are great they're
awesome there's direct control over an
API when I make a call
when I get I will wait and get back a
response and that response might be what
I'm expecting or it might be an error
and at that point I can then respond to
that error or outcome and uh act
accordingly so
so they're very easy to work with
there's also strong consistency usually
in a well-designed system when I call an
API and I make an update and the
response comes back I can be guaranteed
that that piece of information has gone
into that system typically
um however there's some drawbacks to
apis
um one is latency because I have to wait
for the request and the response there's
a round trip time there okay there's
serialization overhead and there's also
runtime coupling so if I make a call and
the server isn't there what am I
supposed to do about that or if I make a
call and no response comes back at times
out what do I need to do so that runtime
coupling is often seen by people as a as
a drawback
on the other hand the vent driven
architecture has got some strengths
which address some of those issues with
apis
typically you can get higher volume and
scale through events because they're
Loosely coupled they're we're just
publishing
we do a fire and forget we don't have to
wait for a response we can get higher
volumes through our system
uh there's runtime loose coupling when I
publish an event
there might be one or more subscribers
to that that event I don't need to wait
around and make sure that they've
received the uh the information
and I can talk to multiple subscribers
at the time depending on what technology
I'm using
um however there's some also drawbacks
around the Venture of an architecture
because I'm typically doing this
asynchronous publish
um and fire and forget I don't
necessarily have the information about
failures at that point in time I often
have to use other mechanisms to detect
and respond to failures in my solution
and that can that can create complexity
uh so complex workflows in event driven
architectures can be difficult or
challenging
so if apis have got some benefits if
events have got some benefits uh how
should I choose and maybe the answer is
not to choose but why don't you use both
okay and the way that you can think
about using both apis and events
together in a solution is to think about
State models
foreign
so I'm going to introduce a state model
to you the concept of State modeling and
I'm going to use a Rideshare application
right Uber Uber ride share or DD or what
have you
um as an example and we start out in
order to get the state model for our
solution we start out with our user
stories and I won't read through all of
these but typically the the business
analysts have come to me and have said
here we go here's the user stories can
you go build this please right so I get
a bunch of user stories about a rider
wants to make a request for a ride a
driver will accept that request the
driver will show up at the pickup
location
the rider will jump into the car and
they'll head to the destination
after the ride has completed there will
be a charge in the bill right
um pretty standard stuff that I'm sure
everybody's pretty familiar with
so using these requirements these user
stories we can come up with a diagram
that looks kind of like this
and this is this represents the state
model for that ride share application
our state model is really it's just a
graph the nodes in the graph those
um those rectangles rounded rectangles
they are allowed states in the solution
and the arrows the lines between the
states represent State transitions right
that's um I I'm sure many of you
familiar with this
so basically the idea is it's kind of
like a workflow but actually it
represents the universe of all possible
states that my application can go
through and so for example if we start
out with that Circle up the top where
where the the rider has opened their app
and the requests for the ride that
creates a new ride request and that
immediately goes into the searching
state so the ride request is out there
and we're searching for a driver to
respond to that ride request so we're
now in that first state of searching and
in the ideal World a driver will pick up
that request they will respond on their
app and say cool I'll I'll pick that
person up
and we will transition from the
searching state to the accepted state so
now the the rider knows is notified on
their app that a driver has accepted the
request and uh and is on their way to
pick them up okay
uh after the accepted State we go
through to the next state which is where
the driver is on approach and we want to
send a notification to say that the car
is approaching it's one minute away
um Rider hops into the car
we then in the next state which is in
progress and then at the conclusion of
the ride when they get to the
destination we go into the final state
which is the completed State okay
um that that's the sunny day scenario
where the ride succeeds we've also got
other possibilities in the sense that
well first of all there may be no
drivers in the area so no one picks it
up okay and that means that the ride
request could time out and go into the
state up at the top left which is
there's no uh drivers available and
that's pretty much the end of that ride
uh tough luck to take a taxi
um or the either the rider May cancel
their request or the driver maybe after
accepting a request the driver may
cancel as well sometimes that happens
so you get the idea we've got these
State the state model and the arrows
represent transitions from one state to
another
so what has this got to do with apis
so basically
you can use the state model to design
your apis and if you think about an API
um an API gives you a way of Performing
two two real two types of actions
one is a command so a command might be
create a write request a command might
be accept a ride another command might
be change the destination location
so commands end up being
a mechanism an affordance which allows
you to move your state model from one
state to another state okay so commands
move you across those arrows they move
you from one state to another state
there's also another thing that an API
needs to do is tell you what the current
state is okay and that's a query so my
API a well-designed API should allow you
to get the current state of the ride has
it been accepted by a driver yet I don't
know let's get the state and check and
see okay
so you can think about
an API as being a way of driving
um a state through a sorry a model
through its different states and that's
all inherent in the the HTTP protocol
it's all built around that state
modeling and in fact if you're familiar
with a arrest API representational State
transfer that's the s in rest s stands
for state
um
you can also use the state model for
events and basically any one of those
arrows between states is an opportunity
for you to publish an event representing
the change in state right so a command
changes state and an event notifies you
that the state has changed okay so for
example when uh the rider creates a new
ride request that's a command the driver
receives a notification an event
notification to say hey there's a new
ride request in your area what do you
want to do okay
um so any
transition in our state model is an
opportunity to trigger an event
notification
so if we come back to our Rideshare
model
I've color-coded it now to show the
different perspectives from the
different users in in our solution so
the green
arrows represent commands
that a rider
can perform through their API
the pink arrows represent commands that
a driver can perform through their API
and then the um
the blue arrows represent kind of system
Transitions and there because the car is
on its way through its journey and the
GPS system is picking that up and and
notifying people about things like the
car is approaching okay so if we go from
the top we can see that the rider
uses the API uses their app to create a
ride request
um and that moves us into the searching
State now it's actually that command
that line
um also represents an event notification
out to all the drivers in the area
saying there's a new ride request okay
the driver can then respond through a
command to say I'll take that one and
that's moves us into the accepted State
uh then when the car is on its way and
the GPS system figures out uh in some
magical way that the car is one minute
away from pickup that's when the system
can trigger an event notification out to
typically the rider to say get ready the
cars around the corner okay so you can
see that these arrows are in each case
either a command or an event or in many
cases both right
and lo and behold if we
um okay so if we look at the
um apis that we might set up for that
um that application for the ride share
application
um these are what you might have with a
restful API call so create a ride
request as a post to a collection of
requests uh get a ride request I want to
check the state of that request is a get
request if I want to cancel the request
I might send a delete and then if I
wanted to change the pickup or the
destination along the way I might do a
put into that and then lo and behold if
we look at the Uber Rider API well who
would have thought their API looks
exactly like that okay so they're
thinking along the same lines which is
great to see so the Uber API represents
those State transitions as commands on
their restful API or rest their API
okay
um
so we can see how the state model has
helped us design
commands for our apis we can also see
now in blue what are the potential
events that a rider is interested in
um from their app so events like a
driver has accepted the ride the driver
is arriving
the ride is complete or the ride has
been canceled by a driver okay you kind
of want to know about that
and lo and behold also if we went to the
Uber API documentation we would see that
Uber has web hooks to notify
a subscriber about these changes in
state okay and this is what the web hook
looks like it's an event it's got a
unique ID it's got a time stamp it's got
information about the ride and the URL
to the particular ride that it
references okay now actually the Uber
web hooks have been deprecated now don't
know why maybe it wasn't getting much
use or maybe it was getting too much use
and it was too expensive don't know but
anyway they've thought about web hooks
when they were designing this which is a
good thing
so we've looked at the rider perspective
of the solution
um we can also look at the driver
perspective and the driver will also
have their API okay so the driver's API
will be about accept the ride request
get a ride request cancel the ride
request and also a driver is interested
in events like the new ride request is
the the main one or a change in the
pickup location the driver wants to know
about that
so we can see that our
simple ride share application has got a
combination of both apis and events to
it and we can use them to complement
each other so why are events important
and useful in this type of solution well
first of all we've got publish subscribe
so we've got loose loose coupling
publish subscribe we can publish those
events to multiple different people loss
or applications in the solution
um and uh the events are eventually
consistent which can be a good thing or
it can be a bad thing if you want
eventual consistency then that's great
if you want strong consistency maybe
it's not so good
foreign
I can illustrate the power of events
through looking at the broader context
so we've just considered Riders and
drivers so far but what about all of the
other parts of the system what about a
billing system or a reputation rating
system okay those are other things that
I need to consider in my solution and we
can see that if I create a ride request
the event notification goes to the
driver
if the driver accepts the ride the event
notification goes to the rider as we've
discussed the billing system is probably
also interested when a driver accepts
the request because we might have a
requirement where if the rider cancels
uh we might want to charge them a
cancellation fee okay so the billing
system is interested in those kinds of
events
uh once we're in progress we might also
let the Billing System know that the
ride is actually underway
and then when the ride is completed
obviously the Billing System wants to
know so that we can then charge the card
associated with that Rider and probably
the rating system might want to know so
we've got this published subscribe we've
got this multiple
um fan out of being able to notify
different parts of the system when those
events happen
okay
so
um hopefully I've persuaded you the apis
and events are really useful together
and try not to think about all apis or
all event driven I worry when people go
to the extremes like that
um not that it's wrong but you need to
know what you think what you're wanting
okay so aprs and events they are
harmonious they work in relation with
each other
they there's a duality between them
they're both different expressions of
the same underlying State model that
we're working with
um and a good balance like yin and yang
a good balance is a solution that uses
them both judiciously
so where do you start start simple news
apis for commands and queries so
commands and queries just use your good
old HTTP request reply apis event
streams use you know use something like
Kafka to publish event streams out and
also when you're choosing between
getting the state of a system whether
it's via a query or a notification
consider your consistency requirements
does it need to be strong consistency
use a query or if you're happy with
eventual consistency
um use an event
think carefully about
choreography events for complex
processes we've seen a simple process so
events work really well in our ride
share example because that's a
relatively simple end-to-end process but
if I'm doing complex processes like um
like like a mortgage application or
something like that that may not be so
good because you get into race
conditions you get into out of order
events and it can be really hairy for
those kinds of things so
[Music]
um
think carefully if you want to go all
event driven
thank you for your time and I'm not sure
if we got time for a question or
so we do okay
[Applause]
thank you very much sir if anyone has
any questions
signal Jack so that he can run a mic up
to you
what kind of um Technologies do you use
with events in terms of like message
cues or um or are there other methods I
haven't worked much with events so I may
not even know the right question to ask
but um yes yeah yeah that's a great
question
um so events
um
uh really the the technology that a lot
of people use now is Kafka right so it's
a distributed log you can just read the
events off the log and it's pretty easy
to use and I would recommend yeah Kafka
or if you're in the cloud you know
manage services Kafka or some of the the
other Cloud native
um examples like that
um cheap and cheerful is the RSS feed
right
um you request and you get back an XML
document and it's got events listed in
it that's effectively what Kafka is like
a high performance version of an RSS
feed uh some people use web hooks web
hooks have got some downsides in the
sense that you've got to send a web hook
to a subscriber you're assuming the
subscriber can have an always available
server waiting for the web hook so that
can be problematic
um
uh message cues I I love message queues
I've spent my career with message queues
but in reality I think message cues now
these days are probably good for
commands and they're good for high
availability or high performance
applications if you're doing banking
systems or your Google or something like
that so I tend to use message cues less
and less these days
um but I'll use you know rest apis and
I'll use um Kafka for events
hey how's it going um that wasn't the
question
um
is there a vocabulary for event driven
systems like when you're talking about
an API driven system you're quite used
to talking about get or push or patch or
query or mutate when it comes to events
there's event
um what sort of
conceptual distinctions are there
between types of events you might be
publishing what sort of vocabulary is in
use when you're building this kind of
thing yeah a good question
um so events I think there's really two
types of event there's
event carried State transfer which is a
real mouthful so when you publish an
event so I've you're publishing
something about the the resource you're
looking at okay with an API we're
operating on the resource with an event
with no defined changes to a resource so
that ride request with all the details
about it where to pick up where to drop
off Etc
when I publish an event do I just
publish an event with a link to the
resource like uber did and that's that's
small it's lightweight it's easy to work
with but then you're often getting you
might get a message storm of requests
coming after it or do I package up the
whole resource and send the whole thing
and that's one choice that people have
to make so packaging up the whole
resource event carried State transfer or
an event might just be something like
that the tooling and the terminology is
still young and evolving we've got Cloud
events as a good standard for having an
envelope for what an event should look
like so if you Google that cloud events
um
async API is is an approach to give us
the Swagger for an event okay
async API is the standard so that's just
like open API specification is for
request reply apis async API is the
equivalent for events and that's a an
emerging standard as well so yeah the
tooling is kind of evolving and that's
why it's it's it's more fun right
thank you again so all right and on
behalf of the conference thank you for
talking enjoy your Adelaide python mug
okay thank you
foreign
[Applause]