[Music]
I love that intro welcome back to the
second talk of shiny block uh on Deck we
have no CTS with what python can learn
from other languages please welcome
Noah see excellent I am indeed Noz I'm
an S geom magical Labs we do computer
vision and augmented reality stuff for
Ikea but I'm not here to talk about that
today and I'm also not really here to
talk about python python is great it's
my primary tool for solving most
technical challenges but there's a lot
of other programming languages out there
and some of them have features or
workflows that python
lacks the goal of this talk is not to
immediately Advocate that we turn each
of these features into a pep we want to
broaden our exposure to programming
Concepts and talk about the trade-offs
being made if any of these ideas really
grab you try messing around with them in
a library or make a patch or cpython but
really we're just here to learn I'm
going to begin with PHP and I don't mean
this ironically or to be some kind of
language hipster PHP is pretty cool I
spent the last few years on a side
project almost entirely in PHP and it's
given me a refreshed
appreciation the biggest thing I've
grown to like about it the simplicity
each file maps unambiguously to a single
URL and deployment is as simple as
updating that file in any way you prefer
most editors these days support remote
editing via SFTP so you just click on
new file you start coding and that's it
this puts even low boilerplate web
Frameworks in Python to Absolute shame
and once you get going changes in the
code they just show up you don't need to
restart a server or configure automatic
reloads or whatever you just write code
and it runs simple as that in Python we
like to say that explicit is better than
implicit it and overall I agree but man
our juggling Imports annoying sometimes
even with editor help you've got a dance
of you write a new line of code you
realize you don't have the import in
place you scroll up to the top of the
file you add the import you scroll back
down to where you were you've lost your
train of thought it's really jarring
autoloaders bridge this Gap while
keeping things mostly explicit if you
enforce a onetoone mapping between uh
code symbols and file system paths you
know exactly where a symbol load is
going to end up this is also deeply
integrated into the PHP package manager
called composer toe bake the mapping
between those code symbols and the
underlying file system paths so the
import doesn't even have to Traverse a
package directory I don't actually know
if I would honestly prefer this in
Python but it's at least something to
think
about the standard library for PHP is a
little bit different from python it's
evolved towards a single Niche web
application development so it biases
towards things needed in that space so
for example my SQL and postgress clients
come by default there's also a high
performance HTTP client and then beyond
that there's this thing called the PHP
extension community library or Pele an
extension PHP is very similar to an
extension library in Python but the big
difference between PE and pii is that
for the a lot of the big core supported
extensions the documentation for them is
Right alongside the main PHP docs you
don't have to search for a library and
then find its read me or read the docs
or whatever it's all just there Peck
effectively bridges into a broader
secondary standard
library and alongside the standard
Library being adapted for its major use
case PHP as a language is itself too
it's web first so there's a simple
templating framework baked directly into
the language in Python we have we have F
strings now and they've matured over
time we've definitely started moving in
this direction but relatively complex
PHP code is again going to run circles
around Python's explicitly simplest web
framworks Ruby is probably the second
most language used over my career and
while the days of python and Ruby being
thought of as relatively neck and- neck
have wed as Python's taken over more and
more niches it's still a vibrant
language in
community symbols are thing in a lot of
languages but I'm going to highlight
them here at their simplest symbols are
just fancy strings but internally
they're D duplicated so checking
equality between two symbols is a
constant time operation we do have this
in Python cy. intern and intern strings
but they're very rarely used but there's
this deeper level to symbols having a
dedicated syntax for this makes it e
easier to differentiate between text
strings and for lack of a better term
symbolic strings these often serve very
different functions in code and even if
they're the same underlying data type
they they really shouldn't be mixed we
do already kind of have this a little
bit in Python with the uh keyword
argument syntax those are still strings
under the hood but they're clearly
intended to be used for a different
purpose and you don't uh you create a
boundary between what kinds of use cases
you want those strings to be
used blocks are something much more
specific to Ruby deep down we do have
this in Python you can write a Lambda
expression as an anonymous function
value uh but blocks do a lot more than
that first off they're not limited to a
single expression that's handy for
anything complicated but more
importantly that calling syntax with do
you can hang a block off any function in
Ruby now you can pass around Lambda
literals in Python it's technically
possible we all sometimes do it when
we're writing sort keys but the utility
falls off really quickly after that and
so Community Practice is that you'll
write a named inline function and then
you'll pass it around as a value totally
works but it does have this problem that
the read order is Now inverted from the
execution
order and this gets even worse when we
start talking about method chaining
something Ruby really excels at I'm not
even really going to touch the python
buil-in functions which are completely
backwards from this and I hate it but
you can write a method chaining API in
Python for instance the Jenga omm but
blocks allow it to go a lot further
without additional
effort and another language that has a
different take on this is Dart and it's
double do cascading operator this allows
interrupting a method chain to go one
level up so to speak like in a file
system path performing a sequence of
operations on the same object rather
than to propagate the chain through
every Link in it this allows method
chaining in Dart with zero extra work by
API
authors another place really makes
excellent use of blocks is its version
of the Builder pattern many complex
object instructors take a block which
exposes a little configuration API
almost a DSL this can allow for very
complex initialization uh while still
being Compact and without sacrificing
flexibility for an usual conditions and
it's very nice for things like static
analysis type
checking going a bit broader than Ruby
as a language one big thing that Ruby
has as a community that I wish python
was better at is the Ruby for good
organization it's a dedicated support
structure for Ruby projects work with
which work with specific nonprofit
organizations we have a lot of paths for
advancing python itself and some of the
bigger languages like Jango but
sometimes it's nice to do real tangible
good in the
world after Ruby the next logical place
to look for ideas are the JavaScript and
typescript worlds I'm not going to
repeat every feature shared by all these
languages like how JavaScript also has a
great Anonymous function syntax we'd be
here all day I'm just going to focus on
some of the interesting
bits this one has been discussed at
Great length in the python Community but
I personally think they should be added
and it's my talk so one more time null
cesing operators are very similar to an
or but they're checking explicitly for
nulles rather than any kind of falsy
value safe navigation operators build on
top of that you can access a deeply
nested chain of fields but if anywhere
there's anywhere in the chain there's a
null the whole thing stops and just
returns a null right there as you can
see in this very small example safe
navigation operators dramatically
streamline code this is something I
deeply miss when I'm working in
Python typescript has always had a fine
line to walk it applies type semantics
to JavaScript but it can't fundamentally
reshape the language or be too
alienating so they knew the majority of
object passed around within a codebase
were going to be the JavaScript
equivalent of just a plain dict no
matter what they also wanted to allow
typing information be gradually added to
a code base out of this comes the
synthesized requirement to support type
DEC
against arbitrary inline dicks in Python
terms this is kind of similar to a data
class but it's completely ad hoc and
without requiring either the upstream or
Downstream code to change this certainly
isn't as nice as a fully typed code base
but it gives a smoother on-ramp to doing
one bit of code at a
time I'm kind of cheating here we do
have promises in python as part of the
con Futures library and perhaps their
moment is over anyway uh with the
continued up uptake in explicit asyn A
we as Chris so rightfully pointed out uh
most of those apis are now what you'd
use in Python and even in other
languages too but they're still worth
studying even as only part of our
Collective history promise chaining is a
compact way to handle multiple
interacting concurrent operations
without dedicated syntax I'm pulling my
punches with this a little bit this is a
really simplistic example and in
practice many promise chains were a lot
harder to read but perhaps the technique
we'll find another application
someday on the topic of concurrency go
it's no secret that go structured
concurrency has made it a frequent
choice in complex system code and
honestly I think Python's already
heading in a lot of the same
directions if you've seen anything about
go it's probably this go routines are
concurrency mechanism more or less the
same as we have with async functions
except they're truly concurrent and
channels are cues to move small bits of
data between them I think this model is
about to get very interesting in python
as we've seen the addition of both Gil
isolated sub interpreters and the new no
Gill mode there's a future where we get
something very similar to the code on
screen but with full multicore
concurrency
runtime type checks are something python
has always kept at arms length would you
technically have it there's that runtime
checkable decorator that comes from the
typing module uh but as the docs warn
you very emphatically all this does is
check that the right method names exist
not their type signatures compare this
To Go's runtime casts which do a more
complete check and they can either fail
with an error message or just straight
up crash your program whichever you
prefer this would requ that there would
be something in cpython itself that
understands how to compare type
signatures but perhaps we've come enough
in our type hinting journey to be ready
for
that in a totally different direction go
is among the Best in Class alongside
rust with preparing easy Standalone
executables as well as crossplatform
compilation in the server world this
makes deployment really easy inside
minimalist containers all you need from
the OS is some TLS certificates maybe a
time zone definition file or two and for
command line tools you can distribute a
single binary without concern for the
system Library versions or anything else
uh briefcase has relatively recently
added some simple CLI packaging support
and I'm hopeful that it will improve
things here make Russ make this
better uh unic kernels build on static
compilation and take it a level deeper
rather than linking in the language
runtime and maybe libc and some other
files what if we statically link an
entire operating system unical
Frameworks take your app code and they
turn it into a bootable OS image
designed to run in a hypervisor instead
of
Linux while containers kind of took the
sales out of this technique uh it can
still matter when you want to get every
last bit of performance out of your
Hardware while also ensuring absolute
maximum security isolation there are
tools out there like uncraft that can do
this today but they're all general
purpose tools they lack the Deep
integration into python to really make
them
shine if you're only here for things
that have even a tiny chance of
happening in Python I wish you well but
now I want to look at a little bit of
the long taale weirder features you
maybe have heard about but it's really
unlikely that python will actually
probably discuss adding these at any
point uh even against those long odds I
still think there are things here to
learn on the complete opposite end from
packaging is easy on liners Pearl has a
lot of interesting Design Elements but
by far the most useful for me day to-day
is Pearl pie it's a sequence of command
line arguments that says take this list
of input files run this expression
against every line of every file and
then write the results back to those
files taken together this allows for a
super set of things like said and O and
Bash but in an incredibly compact
package and without the Myriad of
Provider differences between all those
tools that have occurred over time now
making a python version of this in a
library would be easy and they are out
there you can use them but one of the
real values of pearl pie you can drop me
on to pretty much any server made from
1995 onwards and there's probably going
to be a pearl binary on there somewhere
maybe if we added this to python in the
standard library now it could be
similarly Omni present in 10
years link is short for language
integrated query and it's basically what
if SQL was our API this is most direct
when it's used as a type of omm and that
is definitely the most common use case
for it but C took this even further and
you can build arbitrary providers on top
of link to make it do anything you want
python has taken some baby steps down
this path we have cross Library
standards for data frames and you can
put duck DB on top of those to get SQL
like exploration uh but what if that was
a language syntax that anything could
plug
into call with current continuations
will take some explaining so just bear
with me at a basic level it's a kind of
non-local return like an exception and
indeed we can do a very basic
implementation on this in Python using
exceptions so we take a function we want
to run that function the function will
receive one argument which is itself a
callable when the function wishes to
return it should call that callable with
the parameter being that the value that
it wants to return this alone would be
pretty boring though the real power of
continuations is that in some languages
they can be moved around stored as
variables or even serialized to disk all
allowing you to jump right back to the
point of the code where the continuation
was generated this makes it something
like a structured go-to imagine being
able to save and restore the state of a
generator function as if were any other
object uh I wrote that line but you
don't really have to imagine because
Chris showed us how to do that but what
if it wasn't like
that
um while continuations are mostly
associated with scheme they are very
effectively used in the small talk web
framework Seaside in seide the whole
state of the application is managed via
pickable continuations so in that small
talk code up there that invocation of
Color Picker this is inside a web app so
that with Color Picker is going to
return an HTP response to the client
telling some JavaScript to please open a
modal showing a Color Picker wait for
the user to do a thing get a response
then send that back as another request
up to the server where the server will
unpickle the the continuation and pick
up right where it left off in the code
so this is all the benefits of the
cleanliness of a single page client side
app but in server side
code and because I really love how small
talk does flow control here's a really
rough Port of some of it to python the
short and important version of this
small talk doesn't have special Syntax
for flow control so for example if is a
method of The Bu object and so is while
there's another an example of each of
those up there if you can read them uh
in Practical implementations these don't
actually happen they get compiled into
special constructs because we want code
to run quickly but it's a great example
of doing more with less in language
design and if you want to go even more
minimal you can look at tiny fourths you
can bootstrap a whole language
interpreter with a few hundred bytes of
assembly 20 to 30 keywords from those
you build a basic parser and then an
interpreter Loop and then some more
control flow and then an entire program
fourth Kernels have been written as
small as 400 bytes of machine code
minimalism isn't really a specific goal
of python but it's a good reminder of
how little is needed for a productive
system what would a minimal python look
like what's something that would be just
powerful enough to write all of the rest
of
python another interesting blend of
language features and Frameworks is LL
Lang's open telephony platform or
OTP speciic specifically the supervisor
system earling programs are to use
Python terms a series of interacting
async tasks all running in parallel we
can do that in Python today but what
happens when one of those tasks crashes
it's often difficult to sort out how to
respond exactly OTP applies similar
Concepts from the server and container
management world but to individual tasks
within a complex application we can set
the supervisor to restart a failed task
automatically or have a custom restart
Behavior like waiting a certain amount
of time or we can set up a dependency
tree so that if any in the tree crashes
it restarts the entire set to ensure
consistency this is how llang builds
incredibly High uptime systems like
phone switches without magical
programmers who don't write
bugs this could mesh well with the sub
interpreter work we looked at before and
in fact Go's co- routines and channels
were heavily influenced by otp's server
and mailbox model there could be a
future where we have something like task
groups not just as a synchronization
mechanism but also for runtime
supervision python has made incredible
inroads as being the language of choice
for s scientific Computing and data
science but in almost every one of those
applications you'll find Fortran under
the hood most people think of Fortran as
part of our technical past but it's very
much alive and for good reason forr is
almost perfectly designed to be able to
write complex mathematical algorithms
which can be compiled to the fastest
possible implementation on each platform
so rather than maintaining a separate
linear algebra implementation in
assembly on every OS we have a single
one in Fortran and we trust in the tool
chain even C can't compile things as
effectively 4 Trend in a lot of cases
despite most of us thinking of C as the
lowest level thing out there specialized
use cases need specialized
tools and while Fortran is still alive
and well cobal is mostly though not
entirely in the rearview mirror at this
point but like we talked about with PHP
it's fun to look at language features
that grow from a tool with a single
primary purpose in Cobalt I think the
most interesting part is the data
division in modern terms every Cobalt
program carries around the schemas of
the databases it expects to read from
and write to
while these days it would more often be
a network API rather than a structured
file on disk We have basically the same
problems to solve with producers and
consumers of data agreeing on schemas
while still allowing that schema to
evolve over time Cobalt's implementation
isn't perfect it comes from a time where
those schema changes were expected to
take months instead of hours but I've
worked on a lot of projects that are
decades newer and have substantially
worse schem of
management homomorphic encryption is
probably the most out there of all these
it's barely even a programming feature
and more of a cool math trick the idea
is doing computation on encrypted values
without decrypting them so this means
that you have data processing code that
takes encrypted data that it does not
have the decryption keys
for the core way that this works is
instead of thinking of your code AS
lines of code compiling to machine
instructions that are evaluated on some
kind of virtual machine we're instead
going to compile your code into
effectively a virtual Asic digital
circuit Gates that'll Implement a
homeomorphic encryption flow given those
constraints there are of course very
hard limitations in the kind of
computation that you can do and also
it's still pretty slow but in an
increasingly privacy conscious World
maybe this is worth more of a look from
all of
us I started off by saying I wasn't
aiming for you to go off and turn these
ideas into peps and while that is true I
wish it wasn't a few of these ideas
could probably be prototyped in
libraries and gain some traction there
but most of these to really succeed they
would need a core language change the
process for that has improved over the
years but the steering Council has to
walk a really fine line adding anything
into cpython is a big commitment it
can't just be experimental and it'll go
away in a little bit so this leaves us
in kind of a bind to get peer support
for a change we need to prove that it's
a good idea and for that we need people
to be able to play with it but for
people to be able to play with it we
need to get a Core change in and we
can't do that until we have peer support
and now we're
stuck macros are a language construct to
allow metaprogramming inside the
language itself adding a new
experimental syntax or changing some
syntax that's already there or writing
your code in rout 13 the sky the limit
the two best places to look for ideas
about macros are rust and lisp C does
heavily feature macros but it takes a
much less structured approach treating
source code more as just text to be
templated uh and that does technically
work but Lis and rust show us a deeper
power defining macros in terms of the
symbols of a language but not its syntax
so that we can create our
own python does have a proposed macro
system heavily influenced by rusts but
it has fallen into the same trap as we
were trying to escape by having a macro
system in the first place and has
received little discussion over the
years I want to talk about lisp does
macros before we can do that we need to
talk about how lisp code is structured
and before that we need to talk about
how python code is structured so like
most other languages Python's parer
produces an abstract syntax tree this
isn't bite code this is a highly
structured representation of your code
text and it's ready to be handed off to
the compiler and other layers for
forther execution or or
optimization lisp does have something
like that but it's way simpler lisp has
almost no actual syntax everything in
lisp is one of two things an atom which
is their word for a symbol which we
talked about before or a pair of two
things Each of which can again be either
an atom or a pair of two
things this is just the barest bit of
structure but it also means that macros
and lisp can do almost anything as long
as the input and output are both s
expressions and basically everything can
be an S
expression the way def macro Works in
lisp so we've parsed the code into an S
expression and something is going to
start spidering out over that code
looking for special atoms def macro it's
self is a special atom and it takes the
name the parameters and the body of the
macro and it registers that name as a
special atom and it's going to continue
spidering out of the code if it sees
that name it'll say it's a special atom
I know what to do I will go find the
body of that macro I will take the
arguments which remember are still
symbolic the code isn't running it's
just been parsed so we have the symbolic
arguments we'll slam them into something
we'll evaluate the body of the macro
we'll get back another expression and
we'll just plug it right in where
macro's special atom
was the important takeaway here is that
macros break the usual flow of things
usually our code is going to move
linearly from tokenizing to parsing to
compiling to executing there's a couple
of newer steps in there you'll see jit
compilers and stuff but the key with
macros is they put a little subl Loop
inside the parser step compiling and
executing the body of the macro and
returning the output to the parser this
may seem like a minor thing but it's the
pressure relief valve that lets us
extend the parser without explicit
support from the language
rust has two main types of macros
procedural macros are the equivalent of
what we saw in lisp uh there's a couple
flavors depending on exactly what kind
of parsing you want to hook into but
they all take a stream of tokens and
they return a stream of tokens which
means they can do basically anything
they can even operate on things that
aren't valid rust syntax in the first
place and that's cool sometimes but most
of the time when we're writing a rust
macro we just want to slightly tweak the
syntax that's already there and we use
declarative macros with this thing
called macro
rules declar macros are very similar to
Python's match case statements but again
the code isn't running it's just been
parsed so the parameters are symbolic
expressions or types and the code block
is returned rather than being executed
it does support a little bit of like
templating expansion basically although
I can't really go into the full power of
this in 30 seconds so hopefully this
gives you just the barest idea the core
idea is that basically anything can be
prototyped inside a macro without
explicit support from the rust
team so why at all of this personally I
just think it's fun to see what else is
out there but if that's not enough I
certainly hope that learning about these
patterns and systems can help you better
organize your code and apis in python or
at least shows you what other tools are
out there and in the end that makes us
all better developers thank you very
much any
questions thank you Noah we do have time
for
questions okay we have a we have a
question
hello um of those lists you mentioned
this is python ideas but not python
ideas which of those python ideas do you
think would be most likely to be
implemented or to to to actually land in
core and which one do you think would be
the highest yield if it were reflecting
on Chris's comments about the fact that
generators were implemented which then
made all these other things possible
um I think safe navigation I mean I
wasn't kidding that I think they are it
is it is a mistake that we have not
included them um I think that would
probably be the highest value one uh it
it makes it much more likely that people
will write good error checking in deeply
Nest exploring deeply nested objects um
which is a thing that a lot of code
these days does especially as we're
trying to push more people into using
structured typing things like data
classes or adders we can't just call do
getet on there and assume that it'll
figure itself out um so I think safe
naav operators for the second one the
one that is most likely I I don't even
know anymore uh the the the core team is
an inscrutable box to a lot of
us uh somebody is pointing that way
pointing
yep yeah hi um so you talked about what
you'd like to see added to the language
I'm interested if you've got any ideas
of things you'd like to see removed
maybe something that's non-standard
compared to other languages that you'd
like to see standardized across the
board
uh in in Practical terms I would like
nothing removed because I remember the
Python 3 uh migration period and
breaking compat like that should never
happen again um and I'm pretty sure that
we have all convinced ourselves that it
never will again so in Practical terms I
think anything that has been out there
that ship has sailed
um if I if I could wave a magic wand
though uh the one I actually ask that
question when I'm doing interviews the
big one would be packaging uh I think no
one is under any impression that python
packaging is in an amazing State um I
would make that not I would I would make
python push all of that into a better
Direction um if I could remove one thing
it would probably be multiple
inheritance I think it does more harm
than good and always has um class
decorators should have been the correct
solution there and I think most people
these days they use that instead but
multiple inheritance just breaks a lot
of stuff in mysterious ways uh my
understanding of that is that g read a
paper thought it was really cool and
implemented it that night and it was
early on enough in the the python
ecosystem that no one could tell him no
so he just merged it um basically
because he thought it was shiny which
not really how we do language design
anymore
hi I'm one of the duck DB devs I'm
curious to see what you would what you
think that would look like integrating
duck DB or another SQL engine into
python itself uh probably a lot like it
does in C um you know link there is a
special keyword I think it is literally
just select because they wanted to model
it off SQL because that's what most
people know and once it sees that
keyword it moves into an alternate
parsing mode that again very closely
resembles SQL and it would just probably
the simplest version is just that would
be a very fancy quoting syntax that
would just hand that off to something
else there would be a provider
registered um we've talked about this I
think uh a bit with tagged strings of
various types um being able to register
like we have F strings and R strings you
could have a duck string um and that
would certainly be a lot closer uh but I
I like things like typechecking and it
would be a lot closer to being a core
language Fe if it was more deeply
exposed as part of the
language thank you very much I think
that's
it give it up to Noah