foreign
[Music]
welcome to the Saturday afternoon track
here at Halls C my name is Maya
um and I'm the person you probably want
to come and hassle if you couldn't
decide which talk to go to
if it makes you feel any better as the
program chair I inconvenienced myself as
much as anyone I haven't been managing
to catch almost any of the talks I
actually want to see
um anyway I'm here to actually introduce
Dave hello Dave this is Dave Coles Dave
and I worked together at thoughtworks
for a while and here's the director of
the data and AI practice at thoughtworks
um actually I just remember Dave as
being the person who created the most
high value content on the internet at
work he's full of whimsical technical
data about which he is about to tell you
all
thanks Maya
and
thanks thanks for joining me everyone
lovely to see you here today and I'll do
my best to keep us all awake in this
slot after lunch and thank you Maya for
that as well
so today's about math swimsy with python
but before I start I'd like to
acknowledge that we're meeting on the
traditional country of the current
people of the Adelaide Plains and I pay
my respect to Elders past and present I
recognize and respect their cultural
heritage beliefs and relationship with
the land
so today is about me sharing my
experience with Whimsy with maths with
python libraries how those things come
together in a Sprint or a short
development cycle to generate outputs
learning and applications beyond that
today's quite indulgent it's quite
personal but I hope it's fun for us all
as well and I hope you'll take away from
this maybe a bit of inspiration
some guidance around a process that's
worked for me and might work for others
and resources with python for your own
Journey
when I talk about Whimsy I'm not talking
about being capricious or changeable I'm
talking about things that are light or
fanciful things that acquaint are
unusual that have some interest in their
own right but then it's a bonus if we if
we're able to leverage them beyond that
and a little about me as my said I'm a
director in the dardana data and AI
practice at thoughtworks Australia in
New Zealand thoughtworks is a custom
software technology consultancy in my
role I help our team help our clients
solve their problems with data maths and
AI
and it might be worth talking a little
bit about the path to this point so why
am I talking about Mass Whimsy in Python
today I think I can trace it back to
high school in the 90s where I probably
started with math swimsy and basic in
those days I enjoy doing things like
double pendulum simulations and drawings
of Penske triangles
and that took me into engineering and
computer science in my undergraduate
which involved a lot more engineering
maths using in those days C plus plus
and Java and an early career into games
engines CAD Cam and simulation
then that my career took me to a place
that was a bit Matt's light after being
very Mass heavy for those years
uh and in my when I joined thoughtworks
13 years ago I actually joined as a
project manager and I spent most of my
time working on delivery Improvement and
organizational change agile
transformation was a big thing that a
lot of organizations were doing back
then after some time I moved into the
data and AI practice role and in that in
those early days I was helping others uh
deliver on data and ml for our clients
and typically making slideware thank you
for the introduction Maya because you
know no slideway did become a bit of a
meme at the time this from one of the
team lamenting me going on Long service
leave and what would happen to the the
quality of our slideway over that period
so how did I go from that point of you
know what changed in that I was making
slideware day in day out to actually
doing some work in Python and some
Whimsical work in python as well
well I um found myself on a solo
advisory engagement so I was the only
only thought worker on that engagement
with a client that very quickly became
maths heavy and this was the kind of
thing where um qualitative explanations
and what I could deliver
um in terms of ideas communicated
through documentation wasn't enough I
actually needed to be able to prototype
and express my thinking as code instead
so at that point I picked up a Jupiter
notebook with some scikit-learn to
demonstrate to this particular client
what it would look like to extend their
current process into the world of
machine learning how they needed to
think about measuring success and the
types of types of processes they need to
put in place to manage the data
that was around about the time actually
I think of the last in-person python
conference and soon after that we moved
into covert lockdowns and covered
lockdown presented me with a lot of
spare time to pursue this newfound
interest in Python
so let's look at where Whimsy struck in
covert lockdown
in the first instance uh obviously I saw
a lot of my children in those days
um with remote schooling and one day one
of them said you know Dad for school I
made a square planet and you can see
here it's a pretty cool Square Planet
it's even got a little compass rose or
slash GPS device on top to tell you how
to navigate from one face of the of this
square or cubic planet to another but it
posed a question
um in its in its very form it posed a
question of what happens at the edges
when you come to the edge of one of the
faces uh it's standing on a flat plane
and you're looking down over a sheer
cliff and with the danger of falling
down that cliff and off the edge of the
planet entirely or is it something of it
different is it more like maybe reaching
the crest of an A-frame roof and so we
talked about this for a little while you
know we talked about considerations of
symmetry that might lead you to choose
one over the other but ultimately I
thought that was too simple a resolution
of the question and it needed some maths
and so we could start with uh we I
decided to model it as a grid and in
this grid that represents space some of
the cells have mass if they're if the
planet occupies those spell those cells
then for each of the grid cells if we
can calculate the resultant force of
gravity then we can build up a picture
of what gravity looks like in the region
of the planet and we should be able to
answer the question of what happens at
the corners thanks to Newton we've got
the Maths for over 300 years we've known
how gravitational attraction works and
if we look at that over the the grid
then we've kind of got a we've kind of
got a double Loop for each grid cell and
we sum up the contribution of all of the
masses to the gravitational field at
that point
and
then the python to add to the maths so
given the uh given the the discrete
nature of how it set up the problem it
seemed to lend itself well to numpy
which is a library that suits faster
than scalable numerical processing
especially arrays of data
and because I wanted to answer the
question what does it look like
matplotlib it's a really useful library
for visualization as code everything
from simple charts up to complex 3D
animations
so with those ingredients in place we
could start working through the process
and the process was to start as simple
as possible and then take small steps in
either problem complexity or solution
complexity so I knew a little bit about
for loops with my programming background
and that's where I started looping over
each location and the contribution of
every Mass to that location and I could
validate that I could get something that
far from a square Planet at least look
like the gravitational field you'd
expect to see around a regular spherical
planet and so I could confirm that I was
able to reproduce the the real world
phenomena I was expecting but I did want
to move beyond for Loops they were very
slow
and I did want to start learning numpy
so the next step was to try and
reproduce that first solution increase
the solution complexity in numpy instead
and the first step I took in numpy was
to use mesh grid and which some of you
may be familiar with is a technique for
uh discretizing a plane or a a region of
space and being able to do a calculation
at each point doing that I was able to
reproduce the for Loop and actually
reduce the time by a factor of about 30.
so I was quite pleased how much more
efficient that was
the next step in that process so I was
limited to square planets and square
grids I realized actually there was a
slightly more sophisticated mesh grid uh
implementation where we could mesh
indexes on both the locations in space
and the and the locations of the planet
and so that was another step in solution
complexity but I was still working on
the on the simple problem however that
step allowed me to tackle a harder
problem which was not just a square
Planet but any weird shaped planet in 2D
cross-section let's see what the
gravitational field looks like around
that and continue to extend the field
the solution further
and so if I visualize it it looks like
this we've got the square planet
as the starting point and the most
simple solution the for Loops then
getting slightly more complicated with
the numpy version one and numpy version
two we get to version two then we can
start moving in the problem complexity
space and look at extending the 2D
solution into 3D and actually get the
picture of a cubic Planet but also
choose any sort of arbitrary shape and
in this case I I chose to just take an
image and threshold it and anything
inside the above the threshold would be
mass of the planet and then we would
look at the boundary of that and figure
out how gravity varied around the
boundary
so to get back to the original question
uh of what does it what does a gravity
field look like around a square Planet
well this is this is the the rendering
uh with matplotlib of that numpy
calculation and we can see that at the
corners uh the gravity Vector indeed
points uh into the corners in a
symmetrical way rather than suggesting
that there's a cliff at any of the
corners and as we get further away from
the planet it looks actually more like
the typical gravity field that we'd find
around a spherical Planet so that's also
a good sign
so we can answer The Whimsy and we can
answer the question what happens at the
edges it is actually like you're at the
top of a roof but if you if you consider
that if you consider how the gravity
varies along the edge it's pretty weird
and pretty cool uh at the if you're
sitting at that point on the edge you'd
be looking down what appeared to be a
flat 45 degree slope that you could
slide off the end of and into space when
you reach the far end of that face if
you did start sliding however by the
time you reach the middle of the face
you wouldn't be accelerating anymore
you'd feel like you're on flat ground
and as you continue to travel along that
face you'd actually slow down before
coming to a stop at the far end of that
face and then oscillating backwards and
forwards this sounds like a pretty cool
Fairground ride to me
and so then we can so with that question
answered of course we can go a bit more
complicated uh in the in the problem
space and and take any take any image as
I said so the rubber duck here using
pillow uh to do the image processing and
give us a threshold and we can then
perform the same calculations uh find a
path around the boundary of the object
which it might appear to somebody
walking on the surface uh they could see
how the gravity varies in strength and
direction
so that was a bit of Whimsy a bit of
maths and a bit of python and what
outcomes did I get from that well I
board my kids with all the minutiae this
solution which is a pretty common
scenario in the Cole's household I must
admit
um but through the process I learned a
lot about numpy operations especially
How To Be Clever with indexing into
arrays which I've been able to use over
and over again and kind of established
this pattern for learning works and as I
said I really want to visit a square
Planet slide down and back up the sides
so I hope I'll be able to do that one
day but in covert lockdown I was pretty
close stuck pretty close to home in
Victoria we had the five kilometer limit
and the 60 even the 60 Minute limit per
day within those limits I was trying to
find some kind of exercise to do
and I hit upon the idea of practicing
wheelies on my bike in the back streets
of the neighborhood but also the maths
question of what have I measured uh how
I was doing wheelies with the
accelerometers of my phone artfully
attached with gaffer tape there to the
top tube of my bike and so this was
another another bit of Whimsy to pursue
and another another project that I could
pursue in small chunks as well slightly
bigger than the the square Planet but
small chunks nonetheless nonetheless the
first one was to build a data collection
Pipeline and so each time I went out
with my phone I could download the
results from the sensor log app that I
was using into something of a training
diary or a Strava for wheelies as I
called it this made use of pandas sci-fi
and a runtime a run length encoding
Library called python rle to uh to from
the so sampling the pitch basically what
we're looking at here is sampling the
pitch of the phone at one tenth of a
second intervals and then from that time
series data extracting useful features
such as the length of a wheelie for
further analysis
and then the second so so then having
that data those richer features I could
move on to can I build myself a wheelie
coach uh with machine learning to help
me improve my performance still further
and so the first iteration of this
wheelie coach was taking the half a
second where at the front wheel of the
bike is going up and based on the the
time series trace of that can we predict
how long how long the wheelie is going
to last and so that was I was able to
predict that better than random guessing
with an ml model so that that was an
improvement
The Next Step Beyond that was then to
say okay we've got the front wheel up
and we're balancing uh how might we
predict how the wheel is going to go
from any point in the balance phase so
am I likely to keep going or am I likely
to actually lose my balance and put the
wheel down again
so this was a slightly more complicated
ml approach which is a series to series
prediction but as you can see I'm
gradually able to layer on more and more
by biting off small chunks at each time
and what I found actually uh so some of
those things I've planned ahead of time
but then other things emerge as you're
going and so one of those was actually
realizing that when I had enough wheelie
data I could build my own generative Ai
and I could actually create more wheelie
data based only on the duration and
create some synthetic data so this is
using an auto encoder which takes 10
seconds of Time series data boils it
down into just three data points and
from what I could tell the model was
recognizing that wheelies are either
short or long they had a gradual or a
sharp Landing or they were low and
smooth or high and Wiggly
and then we can generate new Trace new
traces on demand if I'd built it today I
might have called it wheelie GPA
so those were some
um some examples of you know just pure
Whimsy uh for interest sake uh you know
I got some learning that I was maybe
able to apply indirectly but nothing
that I was able to apply directly
unfortunately we don't have too many
clients who are working on their
wheelies at thoughtworks so but there
are some other questions that came up
that could find Direct application in
client work and so one of those was the
question of how to communicate the
performance of machine Learning Systems
to clients a typical way to do this is
with a confusion Matrix and so confusion
Matrix is a great because they show us
when we're getting the answer right with
a classifier but most importantly they
also show us when what types of mistakes
we're making because often in
classification mistakes have asymmetric
costs in some cases false positives more
costly in some other cases false
negatives are more costly but it's
actually quite hard for non-experts to
access a confusion Matrix and to to
figure out what it's trying to tell them
in other areas we've found that Sankey
diagrams which show flows or resource
the flow of resources between states or
parts of the system are actually really
easy and intuitive for clients to
understand interestingly the underlying
data for both confusion Matrix and
Sankey diagrams is in the same form as a
matrix so I wondered what would it look
like to try and draw a confusion Matrix
as a Sankey diagram
and the answer is it actually works and
it helped us have some richer client
conversations where in this case we're
comparing two different versions of a
classifier and we're comparing them to
an existing process the existing process
is at both ends so we can see manually
which decisions have been given a true
or a false label and then in on the left
in the middle we can see method one
which has better accuracy so it gets
fewer things wrong overall and then we
have method two which has worse accuracy
that actually has no false positives in
this case and if false positives are
particularly costly then maybe we'd go
with method two instead by having them
all on the same chart instead of
separate confusion matrixes we can
actually also then compare method one
and Method two in the middle and say
what's happening where they disagree on
a prediction what are the factors that
drive one model to make a decision one
way or the other model to make a
decision the other way and so this was
something that was a bit easier a bit
more accessible for clients that weren't
expert you know evaluating machine
Learning Systems to talk about how well
different machine learning approaches
might work in their case
and it's also good for wheelies as it
turns out so looking at that that second
stage of the coach of predicting what's
going to happen in the balance phase
instead of this being a binary confusion
Matrix we see there's it's a multi-class
classification from continuing to
balance or to or to put the wheel down
on the ground and we can also show that
as a confusion Matrix instead and I
think what's nice as a Sankey diagram
instead I think what's nice here is that
you visually get a sense of the size of
the different classes which can be
harder to do from what's typically the
heat map coloring in a confusion Matrix
so that was it was good to see how
things are starting to come together a
little bit techniques driven by one
piece of Whimsy start to be applicable
somewhere else
and so
you're diving a bit deeper into the
different outcomes which I might
describe as outputs learnings and
applications from each of these mini
projects in terms of outputs I would
typically get python notebooks and
modules and then also typically write
those up into blogs or or talks like I'm
doing today quite matter
but beyond the beyond the artifacts as
outputs the learning of my own and that
I could share with others and then
applications in my day job to supporting
we might do pocs or content creation for
developing points of views for marketing
uh so there's a potential to apply these
uh these projects in that space setting
up engagements for Success being able to
anticipate problems that might come up
and plan plan those in the early days of
engagements and also during engagements
be able to support better outcomes
so lots lots of different applications
and having having dived into a few
specific examples we can look at the
overall learning pattern now as well
just to just to reinforce what I was
talking about
and so it's pretty simple I see
something interesting uh ask if there's
some maths involved and what type of
maths that might be and then also ask if
there's any prior art out there and so
when it comes to confusion Matrix as a
Sankey diagrams those things have been
done before but not necessarily in the
form that I was looking for in this case
and then uh and then when uh I guess
I've sort of evaluated that there's some
value in in doing something in this
space the next question is can I break
it down small enough or frame it in a
way that it fits within a Sprint and I'm
talking about like a typical Fortnight
period but as I said I have a day job
and so it's typically not uh like it's
not it's not an 80-hour Sprint it's
probably more like a 20 hour Sprint
spread over the course of two weeks and
so this requires the opportunity to
present itself at times in my career
I've had a lot of spare time to work on
these things at other times I haven't
covered obviously provide a lot of spare
time to work on these things since then
I've had to be a little bit more
opportunistic
but I think it's important to note that
you can have a backlog of ideas and then
wait until you have the time to pursue
them if they do if you then either then
recognize that I've got the time to
pursue something I can kick off a Sprint
uh solve the problems code them up in
small steps as I've said write it up as
well and then look to see if there are
any interesting extensions which might
lead me back to the start and hopefully
over time we'll find some applications
and it's been my experience that I have
found applications over time although
that is disconnected it's not tightly
coupled to the process of on one
particular project
the um and so this process is
because it's iterative and you know
maybe multi-scale as well
I have some projects in their own repos
but I have this this specific maths
Whimsy repo and that's why how I came to
think about talking about it here and so
the mass Whimsy repo is really to lower
the any barriers to to getting anything
started and written in mathswimsy I
might just write a notebook which is a
handful of lines of code to do something
like eliminate a node from a graph
um it's uh these are things that might
lead into bigger projects or they might
actually have come from the bigger
projects as a sub problem to be solved
but one that's reusable and repeatable
so I think it lives better in math
squimsy so you can see some examples
there from the from what I've showed you
that sit sort of on the bigger project
repo side or on the math swimsy repo
side
and how and one last set of examples of
how things might move from one to the
other
um so I've just discovered another area
that I tend to explore is puzzles and
automating or augmenting Human Solutions
to puzzles this actually started back in
my math swimsuit with basic days there's
a cube puzzle called instant insanity
and rather than go insane I decided to
write a basic solver to solve it for me
but in recent times I've was interested
in Samantha that's another another game
that can drive you insane trying to
solve it and thinking about the the
embedding space the 300 dimensional
embedding space how to actually navigate
that in an automated way to to find a
solution and so that was something that
started as just a bit of math Whimsy and
as I thought about different ways of
solving it and the fact that I could
actually plug in the solver into a game
with friends and play it live uh that
that then iterated and became its own
project
the one on the bottom uh asteroid Escape
it's a game we have at home 109 tips to
solve the hardest one I thought that was
much better automated than solved
manually and but looking at the catalog
of solutions they had I thought wouldn't
it be interesting to generate more
puzzles with interesting Solutions so
that might be that might be part two of
that particular project
and so again looking at looking at
individual examples and how they connect
together I'd now like to take a look at
the bigger picture uh the the Cascade as
I've hinted about how different projects
come together
and
as I said it gets sometimes talking
about things qualitatively helps and I
started thinking about this
qualitatively but then I realized I
could do it quantitatively as well and
what's the maths in this case a
Knowledge Graph so I thought let's
actually take a Knowledge Graph to all
of these projects that I've played with
over the years and this is what it looks
like for the square Planet example from
earlier so I have an idea and some maths
Square Planet idea the Newton gravity
there's a couple of python libraries
there matplot live and numpy as I
described they come together into a
Sprint which delivers a notebook and
also delivers some learning numpy
development to matplotlib development so
if I'd only done one project this would
be the extent of the knowledge graph
but then as I uh in the case of the
square Planet I extended it into a weird
planet and also then did some uh
rendering of LaGrange points around a
planet Moon system out of Interest then
it starts to build some more connections
so we start to see some more inspiration
and we start to see cycles of numpy
learning for instance numpy development
learning back to using use of the numpy
library
and then as we start to bring in more
examples at first the graph might have
separated uh groups of components but we
can see that we're starting to to add
more nodes and more edges
as we gradually add more projects then
it becomes richer and I've actually gone
through the whole contents of the maths
Whimsy repo but not all of those other
projects that I showed you on the right
hand side of the diagram
so this is what it looks like with
everything in the math Whimsy repo in
there and so what comes out and you know
maybe it's a little bit of an artifact
of how I modeled it but definitely that
the libraries and the Sprints are
Central Concepts in the graph so there's
where that's where things come together
where ideas come together and get turned
into outputs and applications
and the learning that I modeled uh I'd
it participates in Cycles so I find the
learning comes out of the process but
also goes back into the process as an
input The Whimsy and the maths of the
sources and the outputs and applications
of the sinks in this graph and there's
yeah there's a bunch of applications
beyond that
um as I showed you comparing multi-stage
classifiers I was able to
reuse some of this work in uh entity
resolution work that we did with clients
some of it's contributed to practical
data privacy book from my colleague
Catherine Jamil
and we've also been able to use it in
tools for Designing EV charger networks
as well so it's been great to see all
these different applications uh coming
out of the work
and with a quantitative representation
like this we can as well as asking
qualitative questions we can start to
ask quantitative questions as well and
so looking at the graph the libraries
that are most represented there map slot
level numpy as you might expect also
Network X I can highly recommend and or
tools have come up useful for a number
of projects
um if I think back to that early part of
my maths intensive career a library was
a building that you went to in the hope
of finding a textbook that might have an
algorithm in pseudo code or C that you
would then have to translate into your
application and exhaustively test for
weeks before uh being able to release it
into a production Solution by comparison
python library today is incredibly easy
to get started so instead of a once a
week cycle time we're up and running in
minutes
the documentation is generally very good
and supported by lots of examples and of
course you can always go and play if you
have an interesting problem to learn
more about them as well and in general I
find them very very mathematically
expressive so I actually guide you to
the form of the solution that you need
as well and python as a language is
quite nice in that respect as well
things look like things in code look
like they would in maths as well
so the overall lessons from today and
thank you for indulging me
to be driven by interest to be
opportunistic with energy
and small steps in problem and solution
complexity are pretty important to be
able to take those small steps to
deliver outputs I've seen the learning
Cascade through that process and the
real world applications becoming
apparent especially as you then know
where to look for them after after doing
projects like this
and I hope uh you're able to take away
that you can find inspiration anywhere
that you're curious that with a
structured process even for a nebulous
question or a vague starting point you
can make progress in demonstrable small
steps that there's a lot of resources
out there in the python ecosystem those
libraries that I shared from the maths
Whimsy repo are only a small subset of
what I've used across all those projects
and that those returns accumulate so the
more you the more you spend time in the
ecosystem the more you learn what it's
possible of achieving
I'd summarize it maybe as uh if you look
at simulation as a technique that we
might use it's a great technique for
learning whenever it's hard or costly to
access the real world
and these math swimsies uh simulations
of real world problems that circumvent
the hard or costly bit and so help to
accelerate my learning and with a
similar approach you might accelerate
your learning
so uh that's it from me today you can
find the examples at the mass Whimsy
repo and the write-ups at my blog there
if you would like to
[Applause]
thanks Dave I'm afraid we oh we probably
have time for one question
there we go there's one over there
yes
uh you appear to be presenting this as
kind of the way you have learned new
technologies explored ideas and whatnot
um I wonder if you had any thoughts
about this is an approach or as a
technique either in an in a formal
education setting or in an in-house you
know staff and staff development kind of
capacity using using similar techniques
in that kind of space
again I think it's applicable in both
very much
um it yet open-ended project-based
learning uh really really helpful
informal settings and uh it it is advice
I give to people I work with and uh
within our team and at client side as
well if they're looking at picking up
new technologies as as well as formal
learning materials or theoretical
learning Pathways to pick projects that
they're interested in because those
offer and those often uncover things
that aren't covered in the formal
learning pathway and and clean data sets
that might be provided with difficult
tutorials it's you know maybe it's uh
there might be a parallel that we might
draw with the performance of um uh say
chat GPT and co-pilot for for
reproducing code examples they're good
at doing the things for which there's a
lot of support a lot of resource out
there but if you want to solve new
problems and atypical problems you've
actually got to get your hands dirty
with the outside of toy well I mean
these are toy examples but they're
they're new and they're they're
different they're not as curated
um as as you might find in a tutorial
thanks everyone we've got a couple of
moments of change River before the next
talk
thank you