foreign
speaker for today will be Owen who will
be giving us a talk on uh rushed for the
python Dev
what are you cool thank you very much
[Applause]
[Music]
[Applause]
cheers thank you everyone
um I guess I'll give you a very super
quick background on myself
um a bit of a academic bioinformatics
natural language background before
heading towards python for quite a while
I was into kind of games technology and
serious games development at simulator
companies before trying to Pivot more
towards that data sciency and python
Tech stack so I used to do a lot of C
plus plus which is where I heard about
rust and it sounded awesome because it
spoke to all the pain points and
pitfalls that I experienced as a c plus
developer
um but yeah the irony is I I looked at
rust first before I ever did anything
serious in Python but since that time
pattern yeah I've Loved python haven't
looked so much at rust keep making false
start attempts to try and learn more of
it and this is my latest attempt at
pushing myself to do some more in it so
yeah I'm still consider myself quite a
newbie at rust but hopefully you'll get
something from this anyway
I should acknowledge that this rust for
the python developer is a very very very
well covered topic as you'll see there
are many other pycon presentations about
this and Pi data presentations about
this and other presentations about this
so um I guess the good news for you is
if you don't like my version of this
talk you've got lots of other choices to
go with and the challenge for me is to
try and figure out how to come up with a
some sort of new spin or angle on this
topic
um just touching I haven't even watched
all of these myself
um Samuel Colvin I'm a big fan of fast
API and pedantic by the way um his talk
is great if you're into the kind of the
trade-offs of working in Rust and what
you lose in development time but what
you gain and kind of runtime performance
and that's something I'll touch on a
little bit but he probably does it much
better than me
and talk down the Bottom by David halter
is great if he's definitely done a whole
lot more rust coding than me and he can
speak a lot more to getting into the
weeds of kind of um lifetimes and and
ownership sort of syntax and semantics
of rust so if you're interested in
getting more into the weeds of it um
check out that talk it's it's pretty
good and I'm sure the others are really
good too I've watched a few of them to
different degrees
we'll say even if you've never
downloaded the rust compiler and touched
anything with it chances are you're
using something made with rust or you're
using some tools made with rust or
you're using some python packages that
are actually rust under the hood
so there's a few of common of the better
known examples that I've listed up here
so there's rough which is like a it's
both a linter and a code formatter and
it's yeah it's a really cool project it
it goes really fast partly because it's
written in Rust and partly because it
only has to do the parsing of your
python code the once as opposed to
running lots of tools like black and
eyesore and what pie dock style and
Flake 8 and all those
it's really neat
um yeah I said I refer to Samuel
colvin's talk so there's the pedantic or
specifically pedantic V2 core which is
now written in Rust
um tokenizers I haven't I used to do
natural language processing I haven't
done anything much in it lately but
apparently tokenized this package on the
hugging face uses rust
there's polars which is kind of an
alternative to pandas it's data frame
um package it supports really huge kind
of out of core memory so you can work
with tabular data that's much bigger
than can fit in the RAM on your computer
and it does a whole bunch of cool tricks
and I guess because it's newer it has
gone like a nicer cleaner interface than
pandas which has accumulated a lot of
stuff over how long it's lived for
um cryptography package that's that was
one of the other talks on the previous
slide I haven't worked with that much
but apparently it's one of the most
downloaded packages on pipe bi and it's
now partly implemented in Rust
um our Jason is a fast Jason
deserializer serializer I believe data
Fusion is like a SQL query engine for
parkai and a lot of those structure
um structured file types and I believe
it works with that Arrow memory models
under the hood and I didn't have it
before but as of yesterday's talk I had
to put Robin up there as well which is
kind of your HTTP server backend with
python front-end rust on the back
and many many more
so I guess a very quick history of rust
um You probably see it gets quite a lot
of good press and um the one thing that
gets cited quite often is that is the
most loved or I guess recently rebranded
most admired programming language by its
own developers so I guess rust
developers like rust better than most
other developers like their clothing
languages which is kind of a good sign
of gas
rust is probably older than what you
think it is I know there's there's
probably
hard to decide where to draw the line
but where it starts but I know it
started development at Mozilla and some
of the earliest versions of it I
believed that all the way back to
2006. so it's new ish but not new new I
think doing the math as of today it's
about half the age of python
I mentioned it became a whole lot better
known and popular sometime around it's
1.0 release which happened in 2015 and
indeed that's when I started hearing
about it later that year
I think I don't know the full history at
this point but at some point 2021 the
Russ Foundation was started so I give it
became less of a Mozilla specific
language and it became adopted by a lot
of the other
um larger tech companies and I should
have chosen a better acronym site
because I know Microsoft uses it as well
not just the Fang ones
of course I thought I'd do a very quick
compare and contrast I had a bit of a
random question first if you could do me
a show of hands
any of you familiar with the Starcraft
computer game or the Starcraft universe
a fair few of you
I changed this slide in my original
version I thought it would be funny to
do a parody of Starcraft I was going to
have python being the Zerg of
programming languages and rust was more
like the the pro task or python kind of
spreads everywhere and it evolves really
quickly and rust is kind of elegant and
advanced but a bit kind of very strict
and regimented and stuff
and that kind of idea of a programming
language of perfect Essence in perfect
form seemed to kind of fit the the
python rust analogy but that's why too
many of you wouldn't get the references
so I just went with a boring button
diagram instead apologies
um but I guess I'll walk down kind of
the sides of it
um probably if you have some familiarity
with C Python and I'm just going to talk
about vanilla C python not all the other
more exotic flavors
um and probably a lot of you are
familiar with this to some extent but I
said so see python it's compiled
compiles to high level bytecodes which
are then interpreted whereas rust is a
low-level virtual machine compiled
language it compiles the native binaries
on the platform that you're on I'm
actually curious to hear afterwards I've
never really understood llvm tap because
vmil was assumed implies that you
compile to byte codes and not actual
machine code so if someone can explain
that to me afterwards I'd be curious to
learn more how that works
um python the reason I love it and I
think lots of other people love it is
it's quite well optimized for learning
it's very easy language to pick up it's
very expressive it's it's high level
rust kind of aims are more about
performance and safety both safety in
the concurrency sense safety and the
memory access
um sense and that was a bunch of the
things that bit me as a C plus plus
developer I once spent probably over two
days tracking down a bug that just
happened to be an uninitialized variable
and trying to follow that code and the
intermittent errors that caused as it
bounced across the dll boundary was a
nightmare and I'm sure I've been out at
the C plus plus game for a while I'm
sure there's much better static
analyzing tools these days that would
pick up that sort of thing but yeah that
definitely bit me and a bunch of other
errors that happened at runtime
I guess python is is very diverse and it
has in some ways a lot of
self-competition so there's competing
interpreters there's competing ways to
install the same interpreter there's
competing packaging formats there's kind
of competing deployment tooling and
stories one thing and that kind of takes
up a lot of debate and oxygen one thing
that's kind of nice when you go to rust
is that everything is consistent again
there's only one way to download it one
way to install it one way to deploy it
one linter
um
depending on that trade-off you might
like that that it takes away a lot of
the choices and I guess they've
definitely learned from Python and that
along the way I guess one thing that's
kind of I guess technology specific is
that python oil is obviously a garbage
collected language and I guess if you're
new to python you might not even be
aware of a thing that is a garbage
collector that cleans up all your
variables over time and returns that to
the operating system and that point
means the garbage collector is doing its
job well rust on the other hand is more
like kind of C and C plus plus and that
you manually manage the memory you
allocate it you deallocate it I guess
the difference being in Rust is it has a
very very clever compiler and it tries
to take away a lot of the kind of
mistakes that you can make in C and C
plus plus when you do this thing
I mean because there is some despite
them being technically very very
different languages that there is some
commonality to them both they both have
very kind of active and cool communities
they both have a large ecosystem of
packages across many different problem
domains
they do share some obstructions and
syntax
I'll talk a little more to that later
and they do have c-type support So rust
and python do play quite nicely together
on that topic of rust and python playing
nicely together as I'll speak to there's
quite a variety of ways to make them
interoperate with each other probably
the most popular ones that I'll speak to
briefly are the pi03
um crate and Maturin tool so Pi 03 is
giving you leave its python bindings for
your rust and probably the most typical
use case is when you want to call a rust
function from your python code and you
want to leverage some of that
performance that rust can give you
um that would be the typical use guys
can actually work the other way around
as well that you can embed a python
interpreter or runtime into a rust
application that's not such a common use
case I guess based on my own background
in games technology I have seen
such use case or similar use cases and
typically in games technology we want to
write the game engine the physics
simulation the graphics and we want that
to be all very highly performant code
because it has to keep updating
refreshing consistently 60 Heights a
second you only have a few milliseconds
to finish drawing the next frame to show
but some elements of the game like the
game mechanics and like the non-player
character kind of interaction and that
you might want designers to work with a
higher level more easier to work with
scripting language which could have been
lower and it could have been python so
that's more a use case where you might
want to have python embedded in
something that's predominantly a rust
application
um maturing you can check it out this is
probably the most recommended tool for
making building say python wheels or at
least setting up the structure of python
wheels that you can use with rust
functions matern is a command line tool
it's pip installable I guess you
probably want to pip X install matern in
this way and it has I won't go into much
more detail than that but it has really
great tutorials and documentations you
can try it out it's very fast to kind of
set up a skeleton wheel slash rust
project that you can then build and
start trying out the Integrations
and I've only experimented with a little
about that that's probably one of the
ways I've heard recommended as the best
way to get started in to rust if you are
a python developer
there are a few other options well I
said there's lots of other options
there's a couple of other options I'm
going to list here that give you kind of
less boiler by even
arguably quicker slash dirtier ways to
integrate rust and python with each
other so there's the rust import tool
which lets you almost
as I've seen it work you can import
functions from rust source files almost
like it was just our python module and
use it and under the hood it is using
pi03
and you can read about that one more on
um it's
um blog that one looks really
interesting I believe it doesn't
currently support Windows so you're on
your posix operating systems your Mac or
your Linux OS for that last time I
checked
and vice versa if you want to run
Snippets of python inside your Rush
program you can check out inline python
it's a rust crate you can download and
it gives you a macro that you can put in
Python code and still access some of the
rust variables from in there and you can
get more information on that checking
out Mara's blog Mara does a lot of kind
of um great materials for the rust
community
as I said the languages are very
different but occasionally in some
places in some contexts they can look
very very similar so you have your
fairly common symbols and Syntax for
conditionals fairly common math
operators and the rust type annotations
and the python type hints use almost
exactly the same colon and dash greater
than sort of Arrow symbols so I did a
very quick example of a very inefficient
factorial recursive function and you can
see the rust version up the top and the
python version underneath and almost
line for line they are the same in this
case I guess one interesting thing to
note in the rust version whatever the
last expression is in the function is
implicitly returned you don't need to
have an explicit return on that in that
case but but otherwise all the same
uh Joyce I mentioned yeah so rust also
has that sort of async awake keywords
pattern matching they both Implement
structural pattern matching rust had
that earlier I believe like many modern
programming languages they steal ideas
of other programming languages so
there's many instances of both rust and
python stealing the same idea from other
programming languages in the case of
async and away and structural pattern
matching
some of the semantics and ideas are the
same but it plays out a little
differently between rust and python
although I won't go into those details
there are some very different concepts
or abstractions and ideas that are very
very unique to us that don't have a
parallel in Python so I will touch on
these briefly rust has a concept of a
borrow Checker as I said it's all about
sort of memory safety you can have
multiple read-only references to one
object but you can only have one mutable
reference to that object and that
mutable reference can be passed around
from one place to another
and it has a very very smart compiler
that will enforce this and it's this is
one of the advantage
one of the cool things about rust is
yeah that it does have this very smart
compiler it does have really nice error
messages that will make it explicit so
I'll get to that example in a moment
um what else so Russ doesn't have
classes at all it does have structs so a
bit like a data class in Python and you
can have functions associated with those
trucks although they are typically
defined outside destruct definition and
they just refer back to it
there's no inheritance in Rust it does
have this concept of things called
traits which are kind of like an
interface or an abstract based class
where you can say that these drugs will
Implement these particular interfaces or
methods
kind of and there's no exceptions so
definitely python is very exception
heavy language and it uses them for all
sorts of things Russ doesn't have that
concept it does have a way of kind of
bundling up errors with the returns from
its functions and passing them through
and I guess the nice thing is it can
either Force the caller to deal with
those exceptions like if something can
potentially return the equivalent of a
null value you can say the caller must
have a conditional to handle it being
returned as a null or some other error
and um
yeah that's pretty much it I guess I I
didn't quit summons showing
rust I guess kind of like CC plus plus
it has the curly braces variables get
scoped to those curly bases in this
example is showing me an error because
I'm trying to refer to an out a variable
inside a scope that or has gone
potentially out of scope
and that's probably what this borrow
Checker is checking
I guess that one of the downsides of
rust compared to say other equally
compiled languages like your C and C
plus plus and go is very smart compiler
and it will try and take away a lot of
the foot guns
but because it's smart and it does lots
of checks it tends to compile quite
slowly I believe that story has improved
it's not as bad as it was in the early
days when I first discovered it but it
still is a common criticism of the
language that it compiles quite slowly
relative to some of the less safe
languages
I guess um yeah a little bit of talk
about the the garbage collector so
system in Python that manages cleaning
up your variables once they're no longer
referenced and that I said I originally
came from a a games type background and
we were very much C plus plus C plus
plus C plus plus we can manage our own
memory and and they did
especially if you look at sort of
domains like console game development
sorry domains like console game
development they some of those
developers Are wizards and they do crazy
sort of memory allocation I don't know
if you ever look at the C python
internals and that and things like Arena
and how python manages like
pre-allocating chunks of memory and that
but the um console game development they
kind of take that up to level 11 where
they like pre-allocate all the memory
and have their own obstructions for
managing it and not doing any sort of
allocations or deallocations on the Fly
um so yeah garbage collected languages
like C Python and Java it's kind of
great that it frees you up from having
to think or worry about the garbage
about this manual memory management but
I guess the reason that it wasn't used a
lot in games Tech is when the garbage
collector does finally run it can give
you kind of unexpected runtime
performance or pauses and that's what
they were really trying to avoid in the
games where you want to keep that
consistent frame rate and you don't want
any unexpected so stops
um so I guess that the different
trade-off that rust gives you is it
tries to it takes away the garbage
collector you have to manage your own
memory again but it's also going to try
really hard not to let you make any
mistakes with that memory management you
shouldn't get dangling pointers anymore
you shouldn't get multiple threads
trying to update the same chunk of
memory at the same time and the race
sort of race conditions that that can
cause
so yeah I experienced that a lot as a c
plus developer so it makes memory
management safer not necessarily easy
though you have to be more conscious of
your lifetimes indeed a lot of rust is
about that sort of Lifetime management
of variables and another another
I guess common issue or complaint is a
lot of people end up fighting the rust
compiler trying to figure out the right
sort of Lifetime syntax and annotations
to use on their variables to make it
happy but so if it compiles it's
probably going to work but you might
have a very hard time getting your code
to compile until you have a very good
grip on this concept of Lifetime and
ownership
I guess one thing I'm about to put a fan
in the python world and I love breaking
out Jupiter and doing just some quick
hacky interactive clothing when I'm
prototyping or learning new libraries
and that and usually one downside of
working with these eagerly compiled
languages like C C plus plus and that is
that you lose that kind of very
interactive Rebel or Jupiter experience
of just hacking running a single line
looking out the output and iterating
fast but I did discover this evaluation
context for rust which both gives you a
raffle that you can use with to
interactively write and execute single
lines of rust it also offers you a
Jupiter lab kernel that you can install
which I've shown up here and you can use
it in a similar way and this is
um yeah I think it looks really cool
this is one of the main things I was
looking for and I think one of the main
helpers when you're starting out in a
language is it's so much nicer and
easier to learn Concepts when you can
just try small code Snippets at a time
and get the really instant or near
instant feedback on how it runs and
stuff so definitely recommend checking
out this project if you're getting
started in Rust and you're trying to
experiment and learn new Concepts I
don't pretend to understand what sort of
magic it does to let you run single
lines whether it's actually compiling it
under the well I assume it is
but I don't pretend to know it but it's
cool that it exists
another project that's kind of
interesting is that there are many
versions of the Python interpreter one
of them that is quite popular would not
see python popular but still relatively
active as a repo and interesting is the
rust Python and they've actually done a
they appear to have done a really good
job there like they're keeping up with
the python versions so they do other an
implementation of python 311 for last
time I checked
um I did listen to some like interviews
of the guys that started this
um and I think they were just starting
they were looking for a project to just
get into and start learning rust and it
blows my mind that someone would pick
re-implementing the python interpreter
as a starter project for learning a new
language but yeah super impressive that
they've done it and that yeah you can go
try that out it's still after all it's
been up for many years now I think they
still say it's not ready for production
use but it's it's an active repo and
they do keep updating it and fixing bugs
and um yeah it's super impressive and I
like I doubt it ever will be a Contin a
contender for a new reference
implementation of python but I I like
the idea that there might be a serious
Challenger for C python one day
cool and I thought for most of the
remainder of the talk I would just do a
bit of a mini awesome list of crates in
the rust ecosystem and
spread over a few different domains my
interests are fairly white once again
this is Broad and very very shallow I
don't pretend to have used a lot of
these crates but I do spend an awful lot
of time just trawling through GitHub
looking at what repos are popular in
both uh Python and the rust side so I
thought I'd pick out some ones that do
seem popular and interesting and speak
to them a little bit so if you are
interested in getting into rust
hopefully some of these packet crates on
the different slides that I'll show you
might take your interest and you'll want
to play with them some more
I won't talk to all of these ones on
back-end web development rust has a very
rich ecosystem of kind of like HTTP
servers that you can check out and they
are some of the most performant ones
that you can use
but I guess I'll speak very quickly to a
few so the rocket one that looked
interesting to me because I recognized I
said I'm a big fan of fast API and I did
notice quite a lot of fast API style
syntax functions for rest endpoints
arguments to those functions mapped to
the parameters for those endpoints and
it would do type checking fast API style
based on the types of parameters those
endpoint functions had and what
arguments they would actually accept
um let's keep on the next ones fairly
quickly but there's hyper which is a
popular low level HTTP client and server
and a couple of crates built on top of
that that axum and warp ActiveX web is
both HTTP and streaming web framework
and the note has a lot of Integrations
with other popular rust crates and this
last one leptos look kind of interesting
to me it reminded me of the amble
platform if anyone's tried that on
python which is kind of a and amble was
end-to-end python front end to the back
end you you add python functions on the
front end side and it's almost like your
remote procedure calling functions on
the back end so it almost abstracted
away the distinction between what was
client code and what was server code and
this laptop looks like something similar
in that vein but for rust so I thought
and all of these are
at least get how popular type repos so
it looks cool
quickly more on the front and side so
there's Rec West don't know how to say
it but it's like the rust equivalent for
the request Library it does handle
asynchronous request so I guess
technically it's maybe more like HTTP X
but that looks cool if you want to try
it out there's a u framework which is
all about rust running on the front end
as it didn't really speak to it but rust
supported webassembly quite early on and
you can compile to webassembly targets
and run your rust apps client side in
the browser so yeah that you looks
interesting if you actually wanted to do
more of the UI work and the front end
work purely and rust and yeah it has for
those of you that have worked in react
before it has the jsx light templates
um oh orms and database access so a
couple of popular crates in this space
are diesel I'm not that familiar with it
but yeah it supports postgres MySQL and
SQL Lite this SQL X which is like a
query Builder I think it supports almost
all the popular relational dbms's
um apparently it does compile time
validation I think against that database
schema so that's kind of interesting if
you want to check out more of that
database and database access side
this is one of my thing I came into
python more from the data sciency Honda
side so this is more my bread and butter
and it's interesting I already mentioned
polars which is like an alternative to
pandas but it has it is more tightly
focused just on data frame operations
whereas pandas gives you a lot of extra
stuff like data visualization and that
as well but polars it interacts with
pandas so you can quickly convert a
pandas data frame to a Polish data frame
vice versa it's much faster like it'll
use all your Hardware much more
effectively than pandas I guess that's a
common criticism of pandas that it's
single threaded
um and yeah and it's out of core two
which is really neat that it can it does
a lot of kind of lazy operations
and that and generates them only as it
needs to
see it and it's yeah it'll load data
from your disk as it needs to so you can
work with data that's bigger than the
RAM on the machine that you're working
on
ndra seems to be like the closest
parallel to numpy in Python so it's your
linear algebra Matrix Library I believe
this might actually be a few related
crates on the rust side that would give
you different parts of that ecosystem in
terms of a parallel for scikit learn
there's linfa and it's the there is a
growing kind of data science machine
learning sort of community and ecosystem
happening on Raspberry it's definitely
not as mature as pythons and not as big
but lympho is yeah it diff I think it
even documents itself as being psychic
learn inspired so it will give you a lot
of the basic ml models and ways to fit
them and use them for more of that
deeper learning sort of pie torch
tensorflow style stuff there's this burn
package
and I'm still trying to understand its
level of abstraction because it actually
can hook into and it can use different
linear algebra
um libraries under the hood so it can
use ND array or it can use Pi torch and
there's a few others I think it can hook
into underneath that's your thing
there's Lance which is our Columbus
store format kind of like parquet and
supposedly according to their Market
materials it's even more performant than
parquet and lastly um data Fusion which
I believe I've mentioned in one of the
earlier slides to this is like a SQL
query engine that works with arrow type
so you can use it against parquet files
CSV files and the like for your kind of
structured tabulator
I also mentioned that I was really into
games technology even though I've been
out of that space for a long time but
here's a few of the popular game engines
in the rust ecosystem probably the most
well known from what I can see is this
Bevy one
it's a fully cross-platform game engine
so you can you can compile it and run it
on each of your major sort of Linux OS X
windows and and also it can work yeah
webassembly so you can do client-side
games running in the browser
it seems to have one of the best the
largest community and that arguably
maybe one of the best documentations and
also fireworks is is very similar it can
go across those same platforms as well
um I believe fireworks just has slightly
different slightly smaller Community
slightly different obstructions to how
the game engine is architectured so you
if you were to pick between them you
probably just want to look at both and
see which one fits your brain a bit
better
Nano is like a lot of games and in
technology but it's more focused on
real-time multimedia so your your
visualizations and sound and stuff
unlike a game engine is not going to be
about Gamepad controllers or anything or
physics engines like the above 2 have
options for and the last one is piston
which is a looser collection from what I
can see it's still
quite popular as a as a code repo but I
don't know it's relative popular it
doesn't seem quite as popular as the
above options like bevian fireworks
and I guess I got to some of the
foundational quotes in Rust there's
probably a heap more I'm still learning
as well but you'll see this I hope it's
pronounced sir serializer deserializer
this is like you can you do your Json
serialization deserialization but it
actually supports dozens and dozens of
different formats that you can serialize
rust structures to and deserialize them
from and Tokio is um that's used by it
seems to be half the crates in the rust
ecosystem and that's all about doing
asynchronous on
so oh
sorry
[Applause]
[Music]
[Applause]
um unfortunately we won't have time for
questions but thank you oven cool thank
you very much sorry for missing my queue
[Applause]