[Music]
hello everybody Welcome to the last talk
in this session just before lunch we've
got a great talk coming up this is Ana
Tish who's here to talk to us about
performant python please welcome her to
the
stage thank you thank you uh good
afternoon as you heard my name is Anna
and today we're going to do a brief
introduction on doing performance
analysis of python
code okay so I work for Microsoft on the
Azure SDK for python uh so throughout
this talk you'll probably hear me refer
to customers or the customer experience
and that's because my background is in
uh building python libraries that we
ship on pii for other developers to use
uh so the word customer may mean
different things to you depending on who
it is that you write python code for uh
so we're going to talk about what
performance means uh why it matters and
then we're going to look at how to
profile your code uh and how to process
and interpret those results and actually
turn it into improved performance uh and
we are going to focus on cpython today
so let's Jump Right In what do I mean
when I say performance performance can
mean different things to different
projects it can refer to Resource use
like CPU utilization memory usage uh
Network bandwidth uh or it could mean
something like throughput or
latency for this talk we're going to
focus on the efficient execution of CPU
bound python code uh so we're not going
to cover IO or um concurrent programming
like asynchronous or multi-threaded uh
but the tools we look at uh can support
those as
well uh so why do we care about this um
python doesn't have a great reputation
for performance and it's usually not the
language of First Choice uh for people
looking to build really performance
systems uh and even if we are using
python we can just use a c extension to
improve
performance but it doesn't actually have
to be this way cpython is built on top
of c and if we know how to really
Leverage that uh and write our python
code carefully we can actually get
pretty awesome performance uh if you're
writing code for customers they might
have existing projects or ecosystems
already built in Python uh or they may
just be purely python trained divs and
asking them to rewrite components in
another language just to achieve
performance might not even be
feasible also just because performance
might not be the primary focus of your
project uh that doesn't mean it needs to
run slow we really like it when all of
our code runs fast and uh if you do want
to fallback to using a c extension there
can be some drawbacks to that uh which
I'll talk about in a little
bit if you have a large codebase you
don't need to worry about doing
performance analysis for all of it uh it
can be quite an expensive timec
consuming process so you really want to
be strategic and focus uh that time
where it's going to matter in your
critical path or your hot path through
your code or areas of really high reuse
um or areas that you're going to
distribute to other teams to use and
that way if it's running as efficient as
possible everybody
benefits so let's jump straight into how
how do we analyze our code for
performance uh to start with we need to
know what it is we're going to measure
um this could be throughput so how much
data we can process in a certain amount
of time uh this could be latency so how
quickly are are we responding to
requests um once we know what we're
measuring we need to know what success
looks like uh is there a specific
benchmark we need to meet uh maybe we
simply want to check our code and
minimize any bottlenecks uh or maybe we
have a certain uh window of time two
weeks or a Sprint uh and we want to make
as much progress as we can in that time
frame so the example we're going to work
through today is actually um a real
issue that a customer opened on our
GitHub uh repo between version one and
version two of a library that we shipped
the customer reported a 20x performance
regression in model deserialization so
that is uh in this case the process of
loading Json data from a file on disk
and then uh turning that into a python
object uh so that is what we're trying
to measure here that is our performance
metric um the speed of deserialization
from uh raw Json data into python object
and our goal here is to reach parity
with version
one so this is the code um as you can
see here we have our model document
analysis uh what this model represents
is a deconstructed PDF document so it
breaks down individual pages and what's
on each page paragraphs of text images
tables and things like that uh so this
is the loading of the Json data from
disk and then this is the one
problematic line uh so this is just
constructing an object by passing in
that raw Json
data so how do we profile this we've got
a few tools uh that we can use and the
one we're going to focus on today is C
profile and that is the profiler that
ships with
cpython um if you have access to py
chart py Cham has a really awesome Suite
of performance analysis tools uh the
default profiler with py Cham is yappi
so you can also use yappi directly if
you don't have pyam uh snake viz and viz
Tracer are two visualization tools uh
snake viz is the one we're going to use
today uh but I use both regularly um
they have uh they operate on the outputs
of C profile and Austin is another
profiler and this one has a really nice
uh vs code extension when you're doing
profiling it's really important that you
try and uh run these tests on an
isolated uh well-defined and
reproducible environment I do not
recommend doing this on your local Dev
machine or laptop uh you will get very
inconsistent and inaccurate
results so how do we run the profiler uh
very simple command line we've got here
so we're just invoking the C profile
module giving it an arbitrary output
name this can be anything and then we're
passing in the script we just looked at
before perf test.py we can tell it's run
because we've got one print line there
and then to load that into snake B we're
just passing in the output file that we
defined and we get this nice big table
of numbers so what are we looking at
here uh the first thing we can see is we
have function names uh and line numbers
uh we can see at the top here per test.
Pi that's the module we were running so
that looks good uh next we have this
Constructor uh so that is actually the
Constructor of that document analysis
model so again that's what we would
expect third thing down we have another
Constructor and this is actually the
Base Class of that model so that model
inherits from this one uh so again that
one's also expected and we have a whole
bunch of uh utility functions all called
some variation on deserialize so that
sounds reasonable um and then we have a
couple of copy functions at the bottom
here so we're going to get to that in a
bit uh so the numbers what numbers do we
have first colum we have is uh number of
calls so this is how many times each of
these functions were called uh perest
module was run once that sounds about
right uh model Constructor that also was
one once perfect uh base model
Constructor we can see that was
originally called once uh but the second
number here indicates that it was called
recursively 709 times uh and what that
means is that even though we have one
class that we're creating uh that is
actually made up of 79 smaller classes
that are all being instantiated as part
of that deserialization process
uh next column we have total time and
this is the length of time spent running
a function exclusive of any sub
functions that it calls into so these
are the numbers we want to use if we're
looking at just isolating those lines of
code in the function to figure out how
long they take to run and then we have
that broken down on a peral basis next
column we have is the cumulative time
and so this is the amount of time spent
in that function inclusive of any sub
functions that it calls into and so we
want to use these numbers when we're
trying to get a sense of where time is
being spent uh to achieve certain tasks
and again we have that broken down per
call so how do I use this information so
my process when I am profiling code uh
is there are three red flags that I look
for uh the first one is probably very
obvious which areas are computationally
expensive so this is how much time
things are taking to run and for this
we're using our total time and our
cumulative time and the per call run
times uh the next thing I look for is
areas that are excessively repetitive
and they're uh functions that have being
called more times than seems reasonable
for what they're doing uh and that's
with the uh in calls column and lastly
uh this one can have some interesting
results and that's time wasted
unproductively so this is when I'm
looking for function calls that uh don't
look like they should even be there or
at the very least shouldn't be
represented so high in the numbers
so before we start iterating on this
code we're going to clean it up a little
bit because before we were just running
uh from the command line and that was
analyzing the entire script when in
reality we really only want to uh
analyze the performance of that one line
that Constructor for the object so we're
going to import C profile as a module
and we're going to run it in a context
manager like this and then that we can
output our file at the end uh I'm also
going to take that one line and I'm
going to run it in a loop and there's
two reasons for this uh one is that it's
going to average out any variation in
the runtime a little bit the first time
it runs it's going to be a wee bit
slower because it's doing some caching
and we want to sort of average that out
uh the second reason is that we're
dealing with very very tiny numbers here
a lot of this is in the millisecond and
the microsc range so by uh running it
more times we boost it into the more
human readable seconds so that we get a
better sense of what we're
doing okay so we've re rerun our
profiler results and they're a little
bit cleaner now we no longer have the uh
the mod module at the top there we've
jumped straight into our Constructor and
we can see as expected that ran 50 times
in our Loop now the cool thing about
snake viz and a lot of the profiling
tools will give you sort of a similar
output is that we can have this kind of
graphical breakdown as to where time and
resources are being spent uh when it
runs the code uh we can see at the top
here for 36 and 1 12 seconds we spent in
our model Constructor so that is the
entire deserialization process and again
all of that time is shared with how much
time was spent in the Base Class
Constructor from there the process
breaks into multiple paths um depending
on the call stack uh and immediately we
have a bit of a red flag here um why is
our process spending so much time
copying data this is making up a huge
chunk of our runtime so the first thing
I'm going to do with these results is
figure out what's going on
there and this is actually the base
class uh for our model and we can see
every time this is constructed we're
doing a full copy of the Json data that
is being passed in and if you recall
from the previous table this function is
being called 709 times on that data uh
so in this kind of deserialization
process it might be reasonable to copy
the data once because a dictionary is a
mutable object uh definitely not 79
times uh so first step we're just going
to clean that up and rerun our profiler
immediately you can see we've dropped
down to 5.92 seconds so one iteration on
this profile data we found a massive
bottleneck uh and simply removing that
has cleaned up our data a lot we're
spending a lot more time now actually
doing the
deserialization uh now we have this sort
of Big Orange rectangle to the side here
so that looks a we bit
suspicious uh so we run our numbers
again this time we've uh organized it
for the uh total time and we can see
we've got this serialized function here
and it's one of the most expensive
functions um which strikes me as a bit
odd because we're deserializing so why
have we got uh such an expensive
function here that's doing serializing
so let's take a closer look at
that and this is the function in
question uh and it turns out this was
actually being used as a fallback so
anytime we had uncontrol data that we
didn't know what it was or what to do
with it it was calling this function uh
and as you can see it's sort of a big if
else statement it's doing a whole lot of
instance checks is it a list is it a
dictionary uh is it a date time is it a
Time Delta and if it's none of these
things we get to the end of the function
and do absolutely nothing um so we're
returning the data completely unaltered
uh I happen to know that the data set
we're working on is almost entirely
strings integers and floats and so this
function was being called on all uh
every uh data entry in that file
and doing absolutely nothing with it so
there are two things we could do with
this we could either refactor it to xit
earlier in the case of a simple data
type uh or we could refactor this
function out
completely and Not only was serialize
one of our most expensive functions uh
but we can see here that serialized date
time also featured as well as time Delta
so that one function was accounting for
three of our Most Wanted expensive
functions so we got rid of
that run our profiler and now we're down
to 4.18 seconds so we're shaving off a
little bit more
time uh so now we're spending a lot more
time actually doing the deserialization
and that's the goal here is to maximize
the amount of time that we're actually
doing the work we want to be doing um
once we've achieved that we can actually
start uh fine-tuning that workload as
well so this deserialized with callable
as one of our most expensive functions
and we can see that here when we
organize for total time uh again it's
showing up as one of our most expensive
functions and it's run like 161,000
times so we should try and optimize that
a we
bit uh this is that function and it
looks a little bit similar to the last
one we've kind of got this if elf
structure going on uh again we've got
more of these is instance checks um in
Python uh we have a preference for the
pattern of uh ask forgiveness rather
than permission instead of look before
you leap which is what we're seeing here
so we can refactor this method a little
bit to we be a wee bit more assertive
about the data types where we're working
with uh and we can also Shuffle the
order a wee bit around um so I happen to
know that this scenario here happens
more frequently than this scenario here
so in that case we should move that
further up and so this function will
exit faster on the most high trffic
scenario so this is our restructured
function uh we moved to a tri except
style rather than an is instance style
and we've reordered some of these
operations so that the function is going
to be a little bit more efficient in our
uh most frequent
scenario and we have rerun the numbers
again we are nowed down to 3.38 seconds
so we've made a wee bit more
progress uh and this process we can keep
iterating on I would say my next uh
Target would probably be these light
blue boxes here would like to figure out
what's going on there uh if we organize
our table for number of
calls um this is another bit of a red
flag uh we're running as
instance 736 th000 times um so that
would probably be another thing I start
looking into now to be clear we are
talking about microc seconds here so I
could refactor away all of these is
instance uh calls and I'm saving a total
of16 seconds on 50 models uh so the
point in here is not necessarily to
remove all of those calls and shave off
a tiny amount of time rather when I look
at that um that to me indicates that
maybe we're doing something else very
inefficiently that requires us to check
the type that many times and so perhaps
a more substantial refactor could result
in more substantial uh improvements of
performance so what are we doing here uh
every time we uh see one of these
expensive functions in our profile
results we want to optimize it uh and
the first is the question why the method
is even there is it necessary could we
simp refactor it out completely like
that serialized function um Can the
function be replaced with a faster one
can we utilize something from the
built-in library or um that's already
optimized by C to make this run faster
or maybe we can refactor it to run fewer
times uh next if it's one of these big
uh multipath functions like those big F
statements can we optimize that so that
the majority case is going to be the
most
efficient and lastly are we doubling up
on any logic uh are we doing a lot of
type verification in multiple places on
the same data that could be
simplified so once we've started
iterating on that code we want to verify
that we've actually made improvements uh
so we're going to have a look at the
time it module and again this ships with
the standard Library uh and it's a
really useful tool for doing ab
comparisons on code paths uh and it's a
great way of validating whether you've
met your success
criteria can also be a really good tool
to help you prevent
per progressions going forward so that
you can make sure that once you've
optimized your code uh over time it
doesn't slip back into poor
performance uh also here the results can
be very inconsistent I wouldn't
recommend running this on your laptop uh
if you can use an isolated environment
that uh can be easily reproduced that
would be give you much more um accurate
and reusable
results uh how do we use time it uh very
simple we're importing it as a module
and then we're just passing in the
function we want to measure in this case
the construction of our object and the
number of times we want to run it uh so
we've got AB here with our original
model and our improved one and we can
see here from these numbers we've
achieved about a 4X performance gain so
we've still got quite a long way to go
before we hit that uh 20x um metric that
we want to meet to resolve this customer
issue we haven't closed this issue yet
this is an ongoing
investigation uh so step five is
um as you can see we did about four
iterations and managed to make some
massive improvements and often those
first few iterations are going to yield
some loow hanging fruit that can give
you some really big gains with minimal
um minimal code
changes each iteration is likely to give
you smaller Returns on investment uh or
might result in bigger structural
changes in order to see those per
gains um so if you do find yourself
needing to make massive refactoring do
make sure you have a very thorough test
Suite in place before you do uh and it's
worth noting that there is actually no
end point here um there is no maximum
performance it can always be faster you
will always find something else that you
could tweak to shave off an extra few
microseconds and in some cases uh that
level of fine-tuning will be
important uh so here's a couple of rules
of thumb uh for how to um when you're
writing your code uh things to keep in
mind so that you don't have to do
performance analysis later because your
your code will always be running at its
most
optimal uh to start with you want to
make use of as many of the built-in
functions as python in python as
possible uh as we mentioned cpython it
is built on C and A lot of those
built-in functions are actually uh
highly um fine-tuned C code under the
hood so the more we leverage them the
more we're making use of Python's raw
performance um in terms of the code that
you write a couple of Tricks uh a local
variable lookup is going to be faster
than a global variable lookup uh however
a hardcoded value is going to be even
faster index access and iteration is
faster in a list but lookup is faster in
a
dictionary dictionary lookup is faster
than accessing an object attribute uh
and instantiating a dictionary or a
tuple is going to be faster than an
object so if you're creating a lot of
objects in your
code um and using that as a means to
pass around data you might find you get
a really nice performance Improvement
simply by replacing them with
dictionaries uh if you're concatenating
data bite array is going to be faster
than
bytes however if you're slicing data
memory view is the way to
go Loop and dictionary comprehensions
are highly op in Python so if you're
doing a lot of looping or building
dictionaries or things like that um try
and squeeze that code into a
comprehension um because that way you're
going to leverage a lot of uh Python's
native
performance uh if you have a lot of
Branch en code paths like those if El
statements we saw before try to optimize
them so that your highest traffic
scenario is going to be the first to
exit uh this might be sacrificing some
perform perance of your low traffic
scenarios uh so depending on uh your
needs uh would be whether you want to
make that
compromise and lastly do try and use the
most recent version of python that you
can um if we are writing really good
pythonic code that is leveraging uh the
performance of um C standard library
then uh the more we can upgrade the more
we can make use of whatever performance
improvements are built into python
itself uh especially if you can be on
3.11 or higher because that included
some really great perf
work so there is a downside the more you
iterate on your code to improve
performance um the more you might find
it goes against your instincts as a
programmer so we mentioned before how a
uh Global variable is a lot less
performant than say a hardcoded value uh
and object attribute lookup is a lot
more expensive than just using a
dictionary uh and this goes on um the
most efficient code you will find is one
giant long flat function uh whereas it's
much more easy to read more modular more
testable uh to break that out into lots
of utility functions um so there is
compromises here depending on where you
fall between code readability code
maintainability versus Raw performance
and so where that line is for your
project is going to depend on your
needs um you can compensate for this a
little bit with lots of inline
documentation uh and if you find
yourself making a um non-developer
friendly design decision in order to
achieve Optimal Performance make sure
that is documented in the code and that
way uh if someone else comes along to
maintain it and decides to clean up for
you uh they're not going to revert one
of your your performance
enhancements um so yeah uh what your
needs are for the project is going to um
determine where you fall between code
maintainability and performance
um so that brings us to whether you
should just uh use an
extension so there are a couple of
downsides to using a compiled extension
uh one is that it's difficult to build
from source code unless you have all of
the build tools um installed on your
machine um the this may or may not be a
problem depending on who you plan to
distribute the code to uh if you want it
to be widely distributed for example uh
to publish this as a package on pii you
will need to build test and maintain
many wheels across many platforms um so
that requires you know some extensive CI
setup and infrastructure in order to get
that uh properly maintained and tested
across all of those
platforms uh when errors do occur it can
be more difficult to uh for customers to
diagnose that themselves if they are
pure python developers uh and we did
find in the projects that we have used C
extensions that it can be harder for
people to engage in the open- source uh
process if uh there we have components
in a language that they're not fluent in
uh so it can be harder for them to file
issue reports and open Community
contribution PRS to help resolve
them having said all
uh if those are not if those cave Arts
uh do not impact your project you can
achieve orders of magnitude performance
gains using an extension uh in the
projects that we have used them my
recommendations are to keep it very
targeted so try and find very specific
functions that you want to increase the
performance of with very defined inputs
and outputs and then use those in your
say C
extensions uh don't try to wrap uh very
large portions of code with complex
logic uh because it can make the
maintenance burdens more extensive going
forward especially if uh the maintainers
of this project are likely to change
over time uh and it might be inherited
by a new set of developers who need to
understand uh the goals of that
code
um lastly always do provide a pure
python fallback so in the case of a
build failure installation failure
incompatibility or anything like that uh
even if it is much slower having an out
giving your customers an option to fall
back to a pure python uh implementation
that they know will always work and they
will always understand uh is a really
customer friendly thing to
do so in conclusion um this is the
process that I use when I am doing
performance analysis of my code uh so to
start with I figure out what it is that
uh is important performance-wise for for
that um for that project uh what what we
what it is we want to measure um and
what success looks like what uh
Benchmark are we trying to reach then we
profile it we find out what the
expensive functions are and how many
times they're being called and we
optimize them as much as possible and
then we verify make sure that we have
actually made improvements and we keep
going and we keep going and we keep
improving and hopefully we hit that
Target number um
or maybe it's time boxed and your
manager says
enough uh so that covers pretty much uh
everything I wanted to go over today um
uh these slides will be in my my GitHub
repo uh if you want to click through to
any of the links to the tools and things
like that um my GitHub Alias is just my
name uh big thank you to swa and Adam
who are both um current and former
colleagues who helped contribute to this
content and a big thank you to my team
who allow me do keep writing python
[Applause]