foreign
ER on the micropython project and a
part-time physics student
um a student of part-time physics and
then formerly SRE at Google and uh
programmer of group learning
um thank you thanks very much Jim
okay thank you very much for the intro
okay so people who are in this room are
going to feel a lot of deja vu from the
last half an hour
um we're gonna go in a slightly
different direction but a lot of the
topic is going to be the same I might
reinforce a couple of the same ideas if
if you're new to wasm that might be
useful but
um bear with us and and save some
questions at the end we'll try and
together and answer them
this is a talk about Distributing native
code in your python wheel so when I
install a package from pip I install it
it downloads
some untrusted piece of code from the
internet that I am about to run on my
computer
um
but sometimes I want to put native code
in that but where Python program is why
do why do we want to do this
and the answer is that there's just some
things that we cannot do in Python
um two main reasons one is we need
performance and the second is that the
code is already written in another
language be it C or rust
and it's just not feasible for us to
rewrite that in Python and we want to
distribute it as part of our Python
program
and this talk is going to build built
around a project that I worked on for
micropython this is not a talk about
micropython but this is the thing that
we ran into while building a micropython
tool so we have this tool called Empire
cross it is the micro python cross
compiler what it does is it takes python
source code and emits python micro
python bytecode so the output of this
program is a DOT mpy file that you can
copy onto your embedded device and then
have The micropython Interpreter on the
device execute that code
it's a very useful tool because even
though the devices have a compiler on
them and it's very useful to be able to
put a py file straight on the device
it's bigger and takes more time and uses
more memory and so forth
but this compiler is written using the
same code base as micropython itself
it's written in C and it's not feasible
to tell people here's our GitHub
download it and run make install the
tool chain all that sort of thing the
second thing is people want to use this
as a building block in their own python
tooling someone might write a
micropython IDE and need to generate
micro python bytecode so we actually
publish it as a library for python users
as well and it turns out pip is just a
really great way to distribute software
everyone in the python community and
micropython is part of that micropy of
the part of the Python Community knows
how to use pip to install packages and
tools
um it's also an interesting case that it
doesn't actually need to be really
really really fast most times you
compile a python source code that might
be at most a few kilobytes long we can
we can run quite slowly and still have
an instantaneous response time
it's also interesting because it has no
dependencies
um it it needs a c compiler to compile
and it doesn't do anything to the
operating system it doesn't use any
other complicated libraries so it's a
self-contained piece of code that does a
calculation this is a bit different to
Katie's talking that in that we're
talking about running a program that
does all sorts of operating system
functionality and and YZ provides an
interface to do that micro python the
cross compiler you can think of much
more as a
um a thing that does a calculation and
produces a result it's just code rather
than anything else
and what we currently do is we build it
as an executable you can run it at your
command line on Windows it's a DOT XC on
Linux it's a an a DOT out and we write
we have a python library that wraps that
and invokes it with sub process so if I
want to invoke uh when I invoke the
thing that I installed by by pip it's a
python thing with a main function that
just calls sub process with with its
arguments straight through we could have
written this as I see python extension
and then I would have a python library
that Imports the C python extension that
is the same code AS Empire cross instead
of being packaged in a standalone
executable it's a it's an it's a just a
piece of code that I could load into
python somehow
um this is almost the exact same thing
that a lot of other libraries do there's
a lot of things with native code on on
on on Pi Pi numpy is probably the
example that everybody's come across
um or at least is most likely to
compress a numpy's a dependency on so
many things in the python ecosystem
and numpy has both of those requirements
one is that it's very high performance
code couldn't be written in pure Python
and the second is that it's code that
was written decades ago in Fortran and
has been perfected and evolved and
nobody understands it anymore and nobody
wants to modify it or change it in any
way
but it works and it's perfect it's it's
probably perfect and and and and can be
used as is
but it's really hard to distribute code
it's unbelievable to be standing here in
2023 and say I don't know how to
distribute code on the Internet or like
distribute my software on the internet
um does anyone here know what a mini
Linux is does that word mean anything
okay so simple to build native packages
um
this is numpy on Pi Pi
there are 24 different Wheels to install
pipei and depending on what platform OS
architecture
phase of the moon
one of those files gets downloaded and
installed on your computer
now you can see here that this I have to
Target different C python versions
because it's a c python extension so the
ABI changes I have to Target different
architectures there's arm there's x86
um someone said to me earlier this week
so you can tell your compiler person
because when I said arm you said which
arm and there are multiple versions of
arm ah 64 is different if you're on a
Raspberry Pi versus if you're on an M1
Mac for example
um there's even different variations of
the same platform in the middle there is
two versions of Linux there's muza Linux
and many Linux different glibc versions
um to answer the question many Linux is
a very carefully maintained 2014 era
image of a Linux machine that if you
build software on that it will still run
today
um on any on a Linux today but if I
compile a binary on my PC now it would
not run on
other other linuxes and especially older
linuxes
imagine you're a project like us we
write embedded code we don't think about
software distribution we're not part of
an ecosystem like this we cannot take on
board the maintenance of building a CI
Pipeline and a build system that can
produce 24 different things so we were
talking about how to solve this and
thinking about how do we actually
distribute code and because we're
embedded people the conversation went
something like what if we wrote a risk 5
emulator in Python and compiled Empire
cross because we because we're a
microphone there is an embedded software
project and we target risk five and arm
and cortex and stuff like that
um and we're like no that's a bad idea
and then
um another well but we thought about it
um somebody else sent me this the other
day which is if you take a Windows
binary from like the early 90s it will
run on whatever the latest Windows 11
today they have done an incredible job
of Ford's compatibility and they can use
wine on pretty much every other platform
to run that 1995 here at Windows binary
what if we just made a Windows
compilation of Empire cross and just if
you did that
thing maybe why is themselves this
problem
and it turns out it wasn't to solve this
problem
so what is wasm you should go back in
time and watch Katie's talk and learn
what was them is but this is a quick
recap
um
we have a compiler M scripton this is a
replacement for your C compiler GCC or
whatever and built on the amazing
infrastructure of llvm and clang and it
can produce what was originally a thing
that ran in the browser JavaScript and
now is a dedicated
um
binary format webassembly the browser
support and as we're going to see things
as Katie mentioned things outside the
browser now support
now
this is not a talk about running
code in the browser but related to this
we're all this is why was kind of on our
radar is that we also run micropython in
the browser as part of a thing called Pi
script it's a totally different problem
a totally different thing that we're
solving but it's really cool what you
can do with that wasm
um
within the day we just distribute code
and
um and you have to
have a way to run that code
um I'm starting to think of computers
less as I have a computer that is like
an x86 or a PC or a Mac or whatever
rather I just distribute code and I
happen to sometimes have Hardware
acceleration for it and
wasm is just one thing that can generate
a thing that my computer can run and it
just so happens that we have a common
way of running that on lots of different
types of computers now
as I said earlier Empire cross is just a
thing that does a calculation it takes a
byte stream as an input and produces a
byte stream of output
but sometimes we want to do regular C
program things or regular rust program
things and so Wazi which we're not going
to talk about much is a standard for how
to replace all those things that I
expect my operating system to provide in
this particular case of doing something
like a computation I don't need any of
that I just need a way to call a
function and a way to get a result
so there are actually four different
ways to run wasm in Python so to recap
what we have is I have a Python program
that I want to use a piece of native
code inside and so previously what I
would have done is written a python
extension a a native extension and
linked that into my wheel
but now what we're going to do is we
we're going to compile our code to wasm
and we're going to try and call
functions inside that wasm binary
so these four runtimes are piwasm which
is a pure python implementation
otherwise and runtime this is really
cool because it has no dependencies
other than things that are already on Pi
Pi like like numpy but as you can
imagine writing a Python program to
emulate a machine to run webassembly
code it's really slow
it's very easy to use and works very
well and it's a great way to learn about
wasm
although we don't need micropython cross
compiler to be really fast it's too slow
for microphone
the next three are all really really
good options so piwasm three is
distributed as a source binary so as a
source wheel so it will install
everywhere the problem is is that
because it's distributed to Source
binary it needs to compile so you must
have GCC and the python Dev like the
development headers installed so as a
fallback as a way to run wasm as a
dependency to have to your program so
that your program can run wasm it's kind
of a universal run Universal will run
anywhere on time
plasma and wasm time are the sort of
slightly more sophisticated ones
um based on based on projects that are
built to run wise them in lots of
different places and Katie already
mentioned wasm time
these use quite sophisticated things
like crane lift so they can generate
native code they'll take your wasm
binary and re-emit that as native code
for your machine so you get nearly
performance parity
um and nowadays with things that the web
the wasm has extensions for things like
simd and Vector instructions
um it knows how to do things like
um set jump and and all sorts of
floating Point features and whatever and
so you can actually
run
your original code as fast as it would
have run
if you just compiled it directly
I'm going to talk about the last three
and highlights as an API sort of study
of how the three different projects work
but all you really need to know about
running wasm code is that a wasm binary
is a DOT wasm file that Imports a bunch
of functions that it expects you to
provide it exports a bunch of functions
that you can call
everything is integers it doesn't know
it's just machine code it knows how to
take an integer as an argument and
return as an integration argument if you
want to do something more sophisticated
like deal with strings
it's an integer that points to memory
that's the start of the string and it
tells you how long the string is so it's
quite low level and and built on on
these fundamentals but as far as your
python interrupt goes you only have to
deal with int and then a day memory just
looks like this big
list of integers that you can read and
write to
so I'm going to give a quick
um
demo
so here is a piece of native code that
is doing a very very sophisticated
calculation in this case it's
calculating the greatest common divisor
of two numbers using Euclid's algorithm
I would write this in Python and there
would be no reason not to write this in
Python just to be really clear but I'm
giving an example of how to how to how I
might be starting from some point
and what I've done is I've added this
notation to annotation to it to say in
script and keep alive and that says this
function needs to be exported from my
wise and binary
um it's the it's it's an entry point to
this to this program this program
doesn't have a main I can't run it
directly I'm building it as a if you're
from Windows you'd call it a dll Linux a
bit.so file
and it takes an end it takes two
integers and returns an integer
um normally I would compile C with with
GCC but in this case I compile with EMCC
and I've got this extra flag that says
no entry it doesn't have an entry point
um like a main a main function and this
produces this wasm file
and what's quite cool about the wasm
tooling is it's quite inspectable and
and debuggable there's a wasm
disassembler
and
this is what Katie showed in her slide
I've got in her in her talk it exports a
function called gcd
um which is the which is the number one
function and this function is number one
is defined and it does it does some
computation I'm not gonna
spend time looking into this but
um this is the text-based representation
of my binary gcd uasm file so if I do a
hex dump
is that that's actually what the file is
it's it's it's it's it's it's it's in
code
um
then
I can use this code from python
so what I'm doing here
is I'm creating I'm importing wasm3
which is the pierson3 library and I'm
creating a wasm environment so the
environment kind of represents the state
of the of how to create a machine I'm
going to configure it with things like
stack sizes and resource limits and
things like that related to what Katie
said about sandboxing you can also you
can also do quite fine grain control of
things like how much memory the thing is
allowed to allocate and how much
gas it's allowed to use it's kind of a
weird concept but you can use that as a
kind of a way to limit how much work it
can do
um I'm loading
um assist.v1 so I'm going to give the
name of the dot wasm file as bytes and
I'm parsing that into a module so I now
have this was a module which is the
python run times representation of my
wasm file
uh I'm creating a runtime from it um
giving it 2K of of Stack size
and loading my module into it
and then I'm asking get me the gcd
function
and if you've ever used any kind of like
native interop or or things like Swig
and stuff like that this is kind of
magic I just said find me the function
called gcd and call it and it knew
automatically what what arguments it
takes what types it returns and I can
invoke that function as gcd
um if I run this
yeah I've got to give it the 1005
gcd.orgasm
so gcd of 57 and 190 is 19.
um
that's all you have to do you can
compile your C code right literally 10
lines of python and and execute compiled
C code in in your in your in your wheel
the only extra step you have to do is
add that wyzen file to your wheel which
is one line in your automobile file
um
I have a quick demo here which is that
in theory I could have written
um I could have written this from
scratch what I'm trying to highlight
here is that it's not that I had to
write it in C lots of languages can
generate wasm
um and I could I could write you could
even write it by hand and some sort of
very optimized whatever
um
and
if I compile this to Dot gcd from
scratch.wasm I could run it with with
same the same tool
but this is not a very interesting
example because I took two integers in
return to integer I'm going to be
working with more complicated types like
in Empire cross I have to take a python
source code file in and return compiled
bytecode out
but very quickly I'll just show you so
this is pi wyzen 3. and then why some
time is a very very similar kind of set
of steps I'm building something that's
like an environment in this case it's an
engine in a store I'm loading a module
same concept
and then I'm
asking for things that are exports and
call in the gcd function
so the next example I'm going to give is
a function that that does something more
interesting with calling external
functions and dealing with memory
so this is a function I'm going to
export called calculate bytes and I'm
going to give it a number of bytes and
it's going to call a function that I'm
going to provide with some chunk of data
that that's calculated based on those
inputs in this case it's going to write
the numbers 0 through n into my byte
array and an invoker function when the
data is ready so this is sort of
building up to what Empire cross does
um
as before this is my uh exported
function that I can call from python
um this is saying I expect python to
provide this function to me I'm going to
call this function when I'm ready
and
I said I only take everything works in
ins it's an it's a un8 pointer and a
size T these are both just integers so
um it it
um fits into this model of everything
it's just integers
um now what I have to do though is tell
the inscript and compiler and in
addition this JS Library flag
and even though I'm doing nothing to do
with JavaScript and nothing to do with
the browser inscription is configured
with a JavaScript library file that says
this is the function that I'm going to
be providing from outside from outside
wasm
and then
exactly as before I create an engine but
now I'm defining a Linker and so the
Linker is what lets me link my python
functions into my wise and functions and
I'm saying that there's this function
called bytes ready that you expect me to
provide
um it takes
sorry yeah the function takes two
integers and returns nothing
and I'm going to
um Define that function on the Linker
that when I instantiate my module the
dot wasm file
it will it will now know how to call
that and so when I call calculate bytes
with 20
um
what we're printing when the function
gets called is the memory I'll come back
to what this means but we're putting the
other hex representation of my data so
just to confirm that we're getting what
we want it's the bytes 0 through 20 in a
20 byte in a 20 byte um
um
chunk of memory and the way it works is
that my my buffer was in it was a una
pointer but a pointer is an address in
memory and so I can index memory
by doing mem.read so mem came from my
things exported from the wasm runtime
and I read from buff to buff plus buff
line so the start pointer to the end
pointer
and this gives me back a thing that
looks like a bytes object in Python and
I'm printing the hex representation of
it
um and so finally I'm going to run
through this very quickly which is that
um another quite neat thing you can do
is you can use wasm as a way to run a
self-contained piece of code that I can
completely throw away and and run again
later with with a brand new state so a
lot of time when you when you work with
native code like you're linking it into
your process and it's
um and it's it's permanent in my process
whereas I can use wasm as a way to
temporarily run a Sandbox piece of code
and then run the same piece of code
again so this is a function that has
some Global this is a piece of code that
has some Global state
but when I load it
um three different times I'm going to
instantiate this module that I've loaded
and run it with a completely fresh State
each time so imagine I could write a
plug-in or whatever and then the plugin
can do whatever it likes with global
variables and Global State and just
instantiate it and run it as many times
as I like
um and
uh
yeah the libraries are subtly different
so then in in wazen 3 you access memory
just like a memory view the result you
get back from this is in Factor memory
view instance and so
um and this one uses a sort of a strange
syntax where you have to tell it the um
you have to tell it the uh the signature
of the function
um wasm wasma the other Library uses
python type annotations so you annotate
your functions with with tie pints and
it uses that to figure out what the
interop is between the different
runtimes
uh and then finally
as a very quick sort of explanation of
how we implemented this in Empire cross
so Empire cross is a c program it has a
make file so we would just write a
second make file that is identical to
the first make file except for we say
that our C compiler is EMCC instead
um we had a couple more things which is
that we had to add that JS Library flag
to say what imports and exports are
and
uh there's a no there's no entry flag as
well somewhere here yeah no entry as
well
um
that's literally all I had to do I had a
c program that was written that that was
written C I copied it to make file and I
added a couple of flags and changed what
the compiler is
and now when I compile it I get
um Empire cross dot wasm instead of
Empire cross.exe or or whatever
and then where we used to have
um this this script this this is this is
our python package that we published it
has a main that calls main main works
like you'd expect for any other Python
program and parses its inputs and then
used sub process
our early sub process I know compiler
native sorry uh
subprices.check output to run a native
binary
we've now implemented three different
backends the idea being that for any
Target that we support one of those
three ways and runtimes will be
available for you
um which sounds terrible but each one of
these literally took half an hour to
write it's it's exactly the demo code I
just showed you
um with slightly more functions it has
to call and Export and import but you'll
see the exact same stuff around creating
a store and creating a module and so
forth
um
uh and then in our toml file
we have this one extra step
which is that we copy the dot wasm file
into the into the output directory that
sits alongside the dot py files so what
the code does
is it looks in compile it at pi
it can look in um
in
the same directory as so you can kind of
see what's going through the absolute
path of file and so it looks adjacent to
my DOT Pi files to find it
so is this a good idea
um
we have to take a fairly pragmatic
approach on micro python like I said
earlier we can't with a small project we
can't spend time maintaining build
systems and things for for things a long
way away from the the type of things
that we work on
um
the performance is excellent we we you
cannot as a user you do not notice that
this is any different to the regular
tool you'd use even when you're
compiling a large number of files or big
files or whatever
filaments will only get better as was
evolves to support more instruction
types and it learns more about the
original code
my one little hesitation here is that by
doing this ourselves the old-fashioned
way we could have supported more targets
and more architectures because we could
just enumerate all 24 possible abilities
that we needed or whatever we're now
limited to about seven but I think those
seven are the most common ones and for
users that have the option to install
GCC and the python Dev headers the
piwasm 3 runtime will work
there's a big to do here which I would
have loved to have presented about but I
haven't quite figured out the details
yet is figuring out how to write a
requirements.text that figures out
exactly which run time to install but I
think this can be solved I've had a look
at it at a few things and and that will
be a so an interesting code to this
um we haven't shipped this yet
um we're going to find out and um and I
guess
hit me up on Mastodon or something or um
uh or getting cut touch and find out how
it goes for us
but the conclusion is it's it's so much
better than what we have at the moment
which is what we're telling people is
download this get repository and run
this make file and people look at us
like
did you just tell me to write a map
reduce in erlang like what are you
talking about and
um
and it's very very simple it's it's one
python dependency and uh it's quite easy
to introspect the tooling is good um
yeah so that's it
um I'd love to answer questions Katie
will come up and help join
[Applause]
thanks very much Tim
I'll invite Katie up and we'll share the
mic or she can have the micromote
um do we have questions hands up yep
just at the bank
you actually missed some methods that
you are not providing and where they're
coming from where's Malik and free
coming from
uh so malakan free provided because uh
the runtime provides the runtime
provides a way to get memory and Malik
is available to you in your wazen binary
it's actually a very good observation um
in my bytes example
um what happened was I wanted to return
a byte array so I allocated it
um the wazen run time is just this giant
array of of of bytes that it can
allocate and grow as it needs to and it
provides the way the amount the airlock
implementation for you
what's quite useful and what we do in
Empire cross is that we actually just
export malloc directly to the python
level and so when python wants to get
some memory to read and write into it
calls malloc directly
um I was wondering what the debugging
story is like and how you go about
getting errors back from your library to
users
so when you like when you make an error
for example in a function that you're in
the the Wizard module Imports if you
have an exception it gets very very
confused
um because you've now got a stack Trace
that involves python back through the
map wasm runtime through C then back
into the your python code that uh
initiate it
um
it's quite challenging
um and the other thing is when you when
you disassemble the wasm code you you
can't uh that's it what I've been doing
is is debugging the C code using my
normal build
um but I'm not aware of for example a
wasm debugger
um one thing you can do is you can take
your wasm file and you can load it
directly in the browser at the end it is
just wazm you have to and if you provide
the same inputs and exports and I think
there are better tools in the browser
um but yes debugging is is not a well
answered story at this point
um I haven't had much experience
debugging wasm directly because I've
mostly just been running C python
um but yes the browser tools are there
you have the same kind of interaction
between if you're running JavaScript and
you're calling a wasm function you also
need to sort of allocate the memory and
copy the strings across and then send
pointers into webassembly
um it's just for when you're building
for the browser those those functions in
JavaScript are kind of created for you A
lot of the time so if you build rust and
you compile it to webassembly it has
convenient functions for those but yes
there are I think there is some kind of
debugging tool for webassembly directly
but I've not used it so I don't know
yeah I have an anecdotal story of one of
the runtimes the very first one that's
written in pure python it didn't
Implement some part of the wasm spec I
didn't know that and I ran Empire I
wrote for my test programs and around
Empire cross and it crashed with the
legal instruction
um so the answer to that is
I had to debug the way you would debug
python code to yeah
we're out of time I'm afraid okay
um but thank you so much for the
questions thanks KD thanks Jim thank you
um once again and um we're gonna
reconvene again oh I need to give you
the
cool thank you so much ceremonial mug
and thank you card thank you
very good
[Applause]