foreign
had a lovely lunch
uh here we're about to hear from huon
who has done work for some very big
names including
Apple Cyro and so on he's going to give
us a talk about
packaging with
serverless for serverless because that
must be easy right we've all done that
all right if you're ready yeah take it
away
[Music]
thank you
you know that nightmare where you're
standing in public speaking in public
and you realize you're not wearing any
clothes at EXO flare we were living in a
bit of a nightmare it was taking
embarrassingly in frustratingly long to
build our artifacts for our AWS
standards
but we woke up and we put our pants on
by adopting the pants build system we
made that process effortless
I'm hewan Wilson and this is Sterling
our Corgi
at the moment I'm working for exoflare
which is a software company trying to
make biosecurity easy so we're reducing
the risk and prosperity of animal
diseases that's what we're trying to do
I'm mostly writing python as a back-end
engineer but I'm also doing front end
I'm doing uh data science and I'm doing
devops
that's a bit of a disclaimer I'm now a
pants maintainer so this talk will be
about this pants build system tool but
when we were evaluating paths to work
out what tools to use I had no
connection to it as part of that
adoption process I found enough bugs
contributed enough fixes that I was
nominated to become a maintainer as well
for exoflare I've worked on compilers
and developer tools at Mozilla and apple
and machine learning at csiro
over the next half an hour we'll remind
ourselves what AWS Lambda and similar
products are are
and we'll work out way Python and
interpreted language needs a build
system
then we'll look at the conventional
wisdom for doing that build system which
is using pip install and we'll see why
it's slow and fiddly
and then I'll talk about the solution we
we really enjoy which is using pads
sound good let's get into it
functions as a service is a bit of a
silly name but it seems to be the
accepted
category name for tools like AWS Lambda
Google Cloud functions
um whatever the other Cloud providers
offer in this space
at exoflare we mostly use AWS so I'm
going to say Lambda throughout the talk
but I think most of the lessons will
apply to the different products I just
can't speak to them personally
so Lambda is effectively a snippet of
code it could be a dozen lines it could
be a dozen megabytes that you upload to
the cloud provider and then they'll
manage sort of invoking it for you and
setting up the interpreters and all that
stuff
we use it for things like cron jobs so
some cleanup that happens every hour
we use it for some event processing and
we use it for integrating with the cloud
provider so when we do our deploy with a
crowd formation we have some custom
resources and AWS makes it easy to back
those custom Resources with lambdas
this talk is about serverless but it's
not saying microservices save the world
or that you should use serverless for
everything
we use Docker images for other things we
only have a few Landers for different
um different uh problems we solve
this talk is about if you are using
Lambda or other serverless offerings
maybe you can learn from the suffering
we went through and the solution we
found
let's make this concrete so here's an
example of a Lambda
the details don't matter too much but it
takes in a bit of Json message then some
string and then we'll return that string
so message hello returns hello
I personally don't like passing Jason
myself I don't want to like check that
the message key exists I don't want to
check that the value is the string
so we Define use one of these libraries
a bit like pedantic that was referenced
in earlier talks that lets us Define the
structures
more declaratively
the last piece this is a Handler
function so this is what AWS will invoke
to actually get our Lander to do its
thing
the details of that don't matter too
much but what does matter is that this
message spec Library so the the thing
I'm using for manipulating the Json
that's an external Library it's from Pi
Pi
when we have an external Library we need
to specify our dependency somehow so we
could use pipeproject.tomel we could use
a pip file but we'll keep it simple for
these examples and just use
requirements.txt
so this defines our Lambda we've got
requirements.txt with our message spec
dependency
and we've got our actual source code
sounds pretty good and we can
hypothetically get this into AWS somehow
and invoke it and it would give the
right behavior in theory
but you'll notice that the the way to
get into AWS is either a Docker image
which has some downsides or zip files
we use zip files so that's what I'm
going to talk about in this talk
the zip file has to have a particular
format so the format has to be
of course our source code has to be in
there somehow so we've got Echo dot Pi
in our echo.zip for our Echo Lambda
and
we have to include all of the source
code of the dependencies so our
echolanda uses the message spec
dependency
and thus we need to include its source
code
this is why we need a build system for
python so what we had in our repository
that was Echo dot pi and
requirements.txt that's different to
what goes in the zip file so we need
some sort of a system for building a zip
file that has the right format
suppose we have a system then we get a
zip file and we can upload that to AWS
somehow so we could use the console we
could use the command line interface we
could use terraform we could use cdk
there's all sorts of tools that will let
us take code on our disk and put it into
the into the cloud but that's not the
focus of this talk the focus of this
talk is turning our source code in our
repository into a zip file ready to
upload into the cloud provider
if we look through the AWS documentation
we can work out a system for doing this
so this is a script up to still down
first we go pip install our requirements
into the the package folder then we can
copy in our own source code so copy
echo.pi into that folder and then at the
end we zip it all up this gives us
echo.zip
if we were to look into this ZIP file
just like with our normal computer
investigation skills we could
notice that it has basically the format
that we were just discussing so we'll
have our echo.pi in it and it will have
the source code of the message spec
dependency in it too
that's uploaded to AWS let's invoke it
and oh we get an error
no module named massive respect dot
underscore core
that's a bit weird because when we look
in the zip file there is a file that
looks pretty related message back flash
underscore core
that that sounds like it matches up
but a bit further in the file name it
says Darwin Darwin's another name for
the Mac OS operating system
Lambda is running the Linux operating
system so when I ran these commands this
pip install the build script
on my Mac machine
it shows code that is specific to Mac
and that won't work in Lambda
understandably the Lambda python
interpreter is ignoring this module
doesn't want to try and run Mac specific
code on the wrong operating system
okay well we can fix this using Docker
Docker for Mac makes it really easy to
run Linux
virtual machines commands on Linux while
on Mac
so we can go Docker run some arguments
to get our source code in so choosing an
image one provided by AWS seems pretty
plausible use our script if we run this
we get out another zip file we get our
echo.zip version two I guess
when we upload this to AWS and we invoke
this we get
the same error
but we know where to look now message
spec slash underscore call blah blah
Linux so we've got the right operating
system
but it says Ah 64. that's a reference to
the arm CPU architecture
lambdas at least by default will run
x8664 or Intel CPU so we've fixed our
operating system mismatch
but we still have a CPU mismatch
the fix is easy enough we had yet
another argument to our Docker command
We Run The build we upload to AWS we
invoke it and we're Victorious we
finally have a working Lambda so we
invoke it with message hello we get back
hello
very cool we now have somewhat of a
build system we could probably make it a
bit more General if we wanted to to
actually be a system but we've got
something that can produce functional
zip files that work in Lambda
unfortunately functional is just the
minimum Bar for a build system
performance is important too
no one wants a slow build right
this is the performance of the three
different steps we ran so if you say
that running natively on my Mac the very
first thing we did I about half a second
but it gave us a zip file that didn't
work so it's just taunting us
running on the arm docker took three
times longer but again it's taunting us
because the zip file doesn't work to
actually get a working zip that gave us
a working Lambda
takes us five seconds
this is because it's having to run a
virtual machine that's emulating the CPU
this comes with significant overhead
five seconds would be tolerable if it
was only five seconds but it gets worse
imagine we're heading a second Lambda so
a feature request has come in that we
need to do some reporting that reporting
is generating a plot so let's add we
start with our echo.pi and we've got our
requirement subtext let's add a new file
plot.pi we're generating a plot so we'll
import matplotlib
another library from Pi Pi that is
helpful for generating plots in the name
it's from Pi Pi so we better add it as a
requirement
um
so this is now our pair of lambdas we've
got echo.pi for our Echo Lambda we've
got plot.pi for our plot Lambda and
we've got the requirements required for
both of them
we can tweak our build system to also be
able to build the plot Lambda and look
at its performance
it's got far worse
running natively on Macs that's as fast
as we can go it takes about seven and a
half seconds
but to get a zip file that works because
this plot Lambda has exactly the same
platform dependent code issue
to get a zip file that works it takes
100 seconds
that's a very long time and no one wants
to wait for a build that long
why is this one so much slower so our
Echo Lander used to build in five
seconds this one builds in 100 seconds
well the plot Lambda is using matplotlib
matplotlib refers to other labs
libraries like numpy
what this means is that the final zip
package is almost 50 megabytes
for the echolanda that we had before
it was like 200 kilobytes that is a huge
amount larger
manipulating all that extra data is
going to take extra time and we can see
it on this graph
this graph is nominally about the plot
Lambda
but it's actually
by adding the new dependency to our
requirements file we've made the
echolanda really slow too
when we build the echolanda we installed
the requirements from this file so we
installed message spec that's what the
echolanda needs
but we also install matplotlib
even though the echo Lambda doesn't need
map
so we spend 100 seconds manipulating
installing matplotlib
for no reason
not only that but the echolanda zip file
went from 200 kilobytes to almost 50
megabytes
how large is it means slower uploads to
AWS and it also means slower cold starts
so when a Lambda starts for the first
time it has to initialize their the
system and that takes longer when you
have a larger deployment package
so preferably we're not we would prefer
not to be spending 100 seconds building
Echo pointlessly and we would probably
also prefer not to be spending 100
seconds building the plot Lambda
we can solve one of those problems by
splitting our requirements so on the
left we've got our echolanda and we
could have our Echo dot Pi file plus the
requirements that that file made
specifically
and on the right we have our plot Lander
and the requirements plot needs
specifically
if we do this now the echo Lambda will
be back to building in five seconds
the plot land will still take 100
seconds to build but like at least with
half the time that the total build
process will take
this can work
but it gets harder as soon as we start
trying to share code so imagine we
wanted to do logging in the same way
from both lambdas
so we have some logs.pi utilities
and in particular we want to do
structured logging
so log stop Pi Imports strap lock
struck log is yet another library from
Pi Pi
so we better make sure it's installed in
the lambdas
if it's not included in the Lambda zip
file then Echo dot Pi will import
logs.pi logs.pi tries to import struck
log struct log isn't available and the
Lambda crashes
one way to do this would be to remember
which lambdas we need to which lambdas
are using logs.pi and go and add it to
all of those requirements
but whenever we say remember and expect
humans to do it that's always a losing
proposition it's easy for us to forget
to do something
like if we hadn't added it to the echo
requirements then Echo would crash
this is fiddly this is annoying and it's
still not solving the problem with the
virtual machines
there's other ways to solve this problem
too like you could have sort of a shared
set of requirements so you have your
logs.pi with its own requirements and
then when you build the echo land you
install the shared requirements and the
requirements specific to to Echo and
that's basically what we were doing at
exoflare but even that is fiddly and
definitely doesn't spark Joy
so we've looked through the conventional
wisdom and we've looked at it in detail
there's tools like AWS cdk offers sort
of Packaging
um for python where it can wrap up this
process but under the hood it's
basically doing the same thing last time
I checked it was still using Docker to
run pip install
the impact of this is our cross-platform
builds are really slow
and when we have multiple lambdas with
different dependencies
we can either end up with unnecessarily
slow builds and unnecessarily huge Zips
if we include dependencies a Lambda
doesn't need
or we end up with a world of really
fiddly dependency management and having
to remember to update to updates 10
different files whenever we make a
change
at EXO flight we were trying to optimize
our build times with that fiddly
approach but we still had so it took us
three minutes to build our
infrastructure before deploying and
worth when we were trying to test it
locally it'll take five minutes to test
our infrastructure
that is not very pleasant
so we looked around for Solutions and we
settled on the pants tool bills itself
is the economic build system and that
seems to be true so we use it for things
like running tests running code linting
running code formatting but most
importantly building artifacts so pants
is the build tool we use for turning our
source code into zip files ready to
upload to AWS
as we're adopting pants and evaluating
different tools we of course ask
questions like what tools are available
how do we use these tools and how do we
adopt them incrementally to start seeing
benefits earlier
but we then asked Howard Sterling wear
pants
my wife thinks number three I reckon
number seven but I'm told that those are
overalls not the paths
but more seriously we asked what
performance Improvement would we see so
this is the examples we had the example
lambdas we had before
it would take five seconds to build the
echolanda using docker
with pants takes two seconds that's a
nice speed up
but more significantly the plot Lambda
will take 100 seconds to build with
docker
it now takes less than eight
that's a huge speed up
most of that hundred seconds is spent
manipulating dependencies
many playing Matlock modifying numpy
pants is able to do this natively
without having to run in a virtual
machine and without having to run and
without having to emulate the CPU
so we're not having to we're able to run
natively on Mac and see that really fast
performance instead of having to um run
virtual machines
as I said this was mostly manipulating
dependencies so let's look at that in a
bit more detail
here's the pipei page for message spec
on the left there's a download files tab
when you click on that you see a long
list of files it starts with Source
distribution and then there's a long
list of Wheels dot WHL files
when you run pip install these are the
files that pip can choose from so when I
ran a pip install on my Mac it would
actually choose this wheel where it says
something about Mac OS arm 64.
this is
this whale includes the
platform-specific code that was causing
problems at the very start so this is
the this is where we're getting that
Darwin file from
the fix and what we need to be doing is
actually choosing this wheel at the top
so Lambda is running on Linux and
running the x86 64 CPU
so we need to choose the appropriate
wheel the one that has the platform
specific code for that platform
as far as I know the only way to pick
convince pip to choose a particular
wheel is to be running on the matching
platform that's why we had to have a
virtual machine so that we can pretend
to be Linux that's why we had to have
the x8664 CPU emulation so that we can
pretend to be matching that CPU
the magic of pants via the related PEX
project is that it understands the
difference between the computer we're
currently running on and the target
environment
so we're targeting Lambda that's Linux
that's x8664.
we can choose the appropriate zip and
mash that up into the appropriate format
even if we're not running on that
platform so we can still run on Mac but
we choose this ZIP and then
turn it into the appropriate zip file
for AWS
this is why pants is able to run those
builds so much faster because there's
none of that overhead of virtual
machines or CPU emulation
sounds good how do we adopt pants let's
go back to the the simplest version of
the two lambdas that we had where we had
our Echo dot pi and we had our plot.pi
with our source code and then a single
requirements file that contains all the
requirements required
to start using it we first add a pants
dot tummel which specifies some Global
configuration like with Japan's version
we're using
and we also add a build file
the build file basically tells pants
like here's my source code here's what
you should be paying attention to
so we need to say two things we say
python requirements which will pick up
our requirements.txt file and say python
sources which will pick up the two dot
Pi files
once we've done this we've now told
pants about our python code and in
theory if we had any tests we could run
those tests
but we don't care about tests what we
care about is building Landers
to do that we activate the built-in
Lambda back end
and then we Define the two lambdas we
want build
so these the configuration is a little
bit overwhelming here but like it's
simple enough we say it's python AWS
Lambda function we need to configure two
things we need to give it a name so Echo
for the echo Lambda and we also need to
tell parents where to find the Handler
function
so to handle the function is in Echo dot
pi and it's called Handler
we need to do the same configuration for
plot so we say call a plot find the
Handler function in plot.pi please
this is enough to Define our lambdas so
we haven't had to write any shell
scripts we haven't had to fiddle around
with docker and this is indeed the Dozen
lines of configuration that I was using
to run those performance benchmarks we
mentioned we saw before we're driven off
this configuration
to actually do the build we run one
command to package colon colon
when we do this it takes about eight
seconds there's some streaming output
I've trimmed it down to what's
interesting let's walk through it step
by step
first we can say we make the minimum Bar
for a build system it's functional it's
produced our two zip files so we've got
echo.zip got plot.zip we can upload
those those to AWS and
we've got working lambdas
foreign
there's something that was missing from
the configuration on the previous slide
we didn't have to say anything about
message spec we didn't have to say that
Echo uses message spec we didn't have to
say that the plot Lambda uses matplotlib
what pants has done is it's inferred the
dependencies so it actually looks at our
echo.pi file and observes it as an
import message spec lying in there it
understands this is a reference to our
requirements so it knows that that's it
to run the echo dot Pi file you'll need
to have message spec available and thus
when we build the echo Lambda
pants make sure to include the message
spec library in that Lambda
there's no other import statements in
that file there's no input matplotlib so
matplotlib doesn't need to be included
buff dependency inference means that our
Lambda is automatically including only
the dependencies that it actually needs
we're solving that problem of
accidentally having larger builds than
necessary
the same process applies for the plot
Lambda so pants has looked into the
plot.pi file and observed that
matplotlib is imported therefore it
better be included in the Lambda
let's talk about performance because
performance is the the next bar or above
functional for a build system
the total build Tech took about eight
seconds and eight seconds is less than
it would take to First build echo which
was two seconds and then build plot
which was also about eight seconds
what's going on here
well instead of building them one by one
it's not going Echo then plot
pants is automatically building these in
parallel so it's building Echo and pants
at the same time Echo M plot at the same
time
um
parallel is nice it doesn't make too
much of a difference for these lambdas
but um it uh
but if you had more it would make a much
bigger difference
when we're talking about performance we
better run things multiple times so if
we run pants package again it gets so
much faster what's going on there
well pants has built-in caching
the first time we ran the package
command we had to run build so we
started with echo.pi and plot.pi
and we built our zip files that will
take two seconds and eight seconds they
run in parallel
when we run it the second time we
haven't changed code we haven't changed
configuration
so instead of having to build the things
again pants understands the inputs are
the same therefore the outputs will be
the same this happens so much quicker
it's just copying files around rather
than having to build like actually do
the expensive build process
this happens in a fine-grained way so if
we had changed echo.pi imagine we'd
added a feature to our Echo Lambda but
we've left plot as it was
when we ran this when we would package
this again
the plot Lander would still be cached
because we haven't changed any code that
affects the plot map
the echolanda has changed so we need to
run that build but now the whole process
takes two seconds and still gives us all
of the zip files that we requested
this caching applies to more than just
building packages so we use it for
testing we use it for linting and the
caching applies to that as well
caching is the first and Panthers
performance Powers so caching means that
pants never really does work that it's
already done
when you when Pants is running a process
it's usually because it has to it's
um
it's new work
the next performance power is
parallelism so when work has to happen
happens in parallel
the third one surprisingly enough is
dependency inference depends the
inference is what ensured that we did
the minimum amount of work possible our
Echo Lambda only included the dependency
it needed to include
the final one which is the one we sold
first is native execution we're not
having to do virtual machines we're not
having to emulate CPUs
so the work that needs to run is run it
runs in parallel
it's the smallest amount of work
possible and it runs as fast as possible
that makes for a happy developer
for our system this is the impact it had
so I remember how I set our builds we're
taking about three minutes before
now they take about 90 seconds and this
is actually happening already on Linux
so this is all because of parallelism
and it's all because of the dependency
inference reducing how much work happens
far more significantly though when we're
testing our infrastructure locally
previously it will take us five and a
half minutes and that's mostly because
we would be building and rebuilding
lambdas pointlessly every time we run
tests
now Panthers caching kicks in so instead
of having to rebuild the land that's
already been built it just gets served
out of the cache thus our tests take 10
seconds instead
that's much more enjoyable
in summary this is about as much
clothing as we could get on Sterling
he's not going to wear pants he barely
tolerates his harness
but actually at exoflare we saw that
when we adopted pants using it for
building lambdas it dramatically reduced
our build times and our test times for
our infrastructure as well as keeping
our configuration really simple
EXO flare has a strong open source
mindset and understands how important it
is such as me contributing to
um pants as well as paying for our
dependencies
and where please come and talk to me if
you're interested in biosecurity if
you're looking for a job
um and just generally if you want to
have a chat
the code examples and some more links
are available in this Repository
thank you very much
unfortunately we haven't got enough time
for questions but I'm sure you can find
you on out and about in the conference
if you have any further questions on the
topic yeah please chase me if you have
questions uh thank you we get the
speakers
thank you thank you guys thank you very
much