[Music]
Hello, PyCon.
Welcome. Welcome back to another round
of PyCon sessions. I hope you've all had
a good break and that you're ready for
another block of amazing talks. I do
have a programming note here um which is
that Tessa Bradbury who was going to be
talking about open telemetry here
unfortunately could not make it um and
luckily Noah was able to step in and
he's going to talk about the long hello
world now give him a bigger round of
applause and then we'll get going
All right.
As I said, uh I am Noah Canrretz. I'm an
SR at Geomagical Labs, uh which is a
part of IKEA. I do computer vision and
augmented reality stuff for IKEA, but
I'm not here to talk about that today.
Hello World is a starting point. Now,
most of the time it's a starting point
for talking about functions and strings
and all of programming, but we're going
to use it as a starting point for
something else. Rather than looking
outward, we're going to look at uh the
stack below this. We're going to look
inward at all of the moving pieces that
make up this innocuous line of code.
To set the stage a bit, we're going to
look at the specific case of Linux and
Gibb C. Python runs on many other
platforms and combinations, but we want
to look at specifics, so we've got to
pick one. All code samples are correct
as of writing in September 2025. If
you're watching this in a video in the
future, please check the repositories.
And our point of departure, we are in a
Python interactive shell. We have typed
in print hello world and we have just
pressed enter. What happens next? Not
saying it prints hello world. What
really happens next? This is our first
stop. Slightly simplified. The heart of
the interactive interpreter is going to
be a while loop that it's calling input
gets our line of code which in this case
is going to be print hello world and
then it does something with it. In this
case calling self.push.
Every one of these layers is going to
involve a lot of calls. So, we're going
to skip around a little bit just this
doesn't take hours. But following down
the call stack from that push method,
here's the part we care about. We have
the code and we're going to call the xc
builtin on it. This is going to parse
and execute our line of code. What does
that mean? It's theoretically possible
to build a language which just executes
code one bite at a time. But this makes
any kind of complex syntax really hard
to implement. So basically all major
languages go through a number of steps
to add additional uh information to the
code as it goes through the processing.
Decoding bytes to unic code characters
technically happens first but that's
been covered many times in many places.
So let's look at the other steps in a
little more detail. Lexing or tokenizing
which are the same thing just different
words. That's the step that goes from a
sequence of characters to a sequence of
tokens. We'll talk about tokens. And
then parsing turns a list of tokens into
a syntax tree.
To reach the lecture, we need to jump a
ways down the stack. Uh, and in
practical terms, the lecture is actually
more like a Python generator. Although
it is written in C, it produces tokens
on demand as the parser needs it. But to
get normal mode is the bulk of the
logic. So what's this going to do? It's
going to go through each character one
at a time, group them together, and tag
them with this is the kind of thing that
this is. The token module in the
standard library exposes all of the
available token types. So we can see a
few kinds of patterns here. So the first
are names. This is anything that looks
like an identifier, function names,
variable names, class names. Uh numbers
and strings. Those are just literals.
And then syntactically significant
punctuation, parentheses, colons, the
plus sign. The lex also emits indent and
ddent tokens so the parser doesn't have
to care about tabs versus spaces. And
this is where line continuations happen
again so the parser doesn't have to
worry about that.
The tokenize module lets us experiment
with alexer and see the resulting token
list on the command line. So let's feed
it our line of code. We end up with six
tokens. So the first is a name token
which matches up with the characters P R
I N and T. Then a left PN token with the
left PN character. Then our string
literal token, a right PN, a new line,
and an end of input token, which is just
an opaque way of saying you have run out
of input. This is more structured than
our original string. We know a little
bit more about what's going on here, but
this isn't really ready to execute as
code. We've still got parsing to go.
So far the representations we've been
talking about have been linear. A
sequence of bytes, then a sequence of
characters, then a sequence of tokens.
But when we think about complex code,
it's not really flat. We would think
about the body of a function being
inside the function definition or the
parameters for a function call are
inside the function call. An abstract
syntax tree extends this concept. So we
want to build it up as a recursive tree
of all of the syntax in our code.
Python's parser is built around a
parsing expression grammar or peg input.
This file python.gg uh it has all of the
different syntax of Python in terms of
the tokens we saw before. Now parsing is
an incredibly dense topic. This is going
to be a surface level read on all of
these pieces. So the short version is
that we want to match them from left to
right in a particular order. Each RAM
rule will then say what type of syntax
node to create and has a little bit of
extra information about how to recurse
down into the next piece of syntax that
it should be parsing for. In the end, we
end up with this tree structure that is
our abstract syntax tree for the code.
As with the tokenizer before it, we have
a command line module we can use to play
around here. So, we pass our code into
the a module. We end up with a module
containing a single expression which is
a call with a name of print and a single
argument with a constant of hello world.
Now this is as structured as Python code
gets and it's ready to march forward on
its journey towards running.
Many words have been said over the years
about how Python is or isn't an
interpreted language. And obviously I've
lost interest at this point. But we can
show very clearly here that Python is a
compiled language because our next stop
is the Python compiler. And before we
can talk about compiling Python, what is
that compiler for? Now, when we compile
CC code, we are emitting CPU
instructions for the physical hardware
in your machine.
CPython has what's called a virtual
machine. Now, this is different than a
virtual machine like running Windows on
your Mac laptop, but it's the same idea
of being a very fancy fake computer that
we can write instructions for. The core
of the CPython virtual machine is the
stack. If you've worked with fourth or
an RPN calculator before, you are one,
very old, but two, this might look
familiar. Uh, in a stackbased system,
basically every operation is one of
three things. You're either pushing
something onto the stack, you are
operating on the top of the stack, or
you are popping something from the
stack. CPython does track some
additional things, call frame, metadata,
thread state, stuff like that. But the
heart of it is really the stack.
And because our virtual machine can be
way more complicated than any physical
computer we could build, our low-level
instructions can be pretty specific to
Python's patterns. They're called op
codes, and they are the building blocks
of all of Python. The disc module
documentation lists all of them. Uh, and
here's a few to set the tone. You can
see they vary widely in complexity from
that simple push pop to as complicated
as create a generator object. That is
one assembly instruction as far as
Python is concerned. Uh, new op codes
get added as new features are added, old
ones are removed, and any given time
there's usually about a 100 Python op
codes.
Okay, so we're back into run mod in
Python Run.c. And we want to call down
into the compiler. Again, this is just
showing the call stack from top to
bottom. Uh, the core of the codegen
system, which is the heart of the
compiler, is a visitor pattern. So, it's
going to walk down the tree, and it has
a callback for each type of syntax node
that the parser can emit. With each of
those nodes, it's going to create zero
or more op codes and push them into a
code object that'll eventually get
bundled into a function which will get
bundled into something else like a
module.
Again, the compiler could be a whole
talk, but let's just look at our bite
code for our hello world. Again, the
disc module has a command line interface
just like the other two so that we can
pass our code in and see what bite code
we get out. So, we end up with these op
codes. Load name looking for something
called print. Then a pushnull. This is
new in 313 because the calling
convention changed. All that is is
saying Python doesn't actually take a
self argument. So we just give it none
instead. Uh then we load our hello world
constant string. Then we call the
function print. Uh we ignore the return
value because we don't care if print
fails. And then we return none because
everything in Python has to return
something. All right. Finally, this is a
program. We can actually run this.
So, we're back in runmod, but we're
going to descend down a different piece
of the code tree. Uh, we're looking for
the part of Python that actually runs
this stuff. This leads us to the
function pie eval eval frame default,
which is possibly the most complicated C
function in all of Python. This function
and its friends unpack each compiled op
code and perform whatever computation it
requires. So, as an example, here's a
little bit of how the load name op code
is implemented. We've got a little bit
of weird stuff with stack things, but
the core of it is the bottom three
lines. We call pi eval loadname to get
some kind of object and then we push
that object into the stack.
This is also where the JIT happens and I
wanted to show some code samples but
honestly the JIT is spread out over a
million places and there isn't really
one thing to show you. So the really
quick version, whenever a jump backwards
op code is executed, the exeutor will
look to see if it is part of a
frequently used loop. If it is, it will
pause execution, cop snip out all of the
op codes that are part of that loop,
convert them from Python op codes into
CPU assembly for your specific platform,
and then paste them back in where they
came from. This is extremely
complicated, but also extremely useful.
All right, so we're back to executing
our first op code. We want to load
something named print onto the stack
like we saw two slides ago that uses pi
eval loadname. In here is where we see
the three lookup scopes that exist in
Python. So first it's going to check the
locals then the globals and then the
built-ins. Now our hello world doesn't
declare any variables local or global.
So it's probably going to come from that
third one the built-ins. And indeed in
the list of built-in methods registered
when Python starts up we find one named
print. load name will push a function
object referencing this onto the stack
and then that load const op code will
push our literal string hello world and
then we get to that call op code. When
we execute the call op code, it's going
to start running this function built-in
print imple. Now most of the code in
there is dealing with parsing and
validating the arguments passed into it
because print is an extremely
complicated function. But deep down we
see some of the shape that we expect.
We're getting a reference to something
called std out from the CIS module and
then we're calling write object on each
argument in turn. We're 24 slides in.
We've got to be close to seeing our
hello world, right?
All right. If we're getting a reference
to sedd out, where did that come from?
When any program starts, it receives
three special already open file handles
from its parent. These are standard in,
out, and error. And they are
respectively numbered 0, 1, and two.
Now, Python doesn't know anything about
what's in those. All it's doing is
grabbing it, wrapping it in some
encoding and buffering helpers, and
stores a reference in the CIS module.
Another small aside, we are definitely
down in Code. We're going to be seeing
mostly Code from this point out, but uh
a lot of this reads really similar to
Python. That is not by accident, and it
is very useful. There going to be some
other things in there. You'll see ref
counting, macros, and error checks. But
a good uh mental trick when you are
reading through Python's C code, you can
usually just recompile this in your head
into the equivalent Python. It makes it
a lot easier to read.
All right, once again, let's fast
forward down the stack a little bit.
We're going to see those encoding and
buffering helpers that we wrapped
standard out in before. We're going to
eventually end up in this method py
write imple. Now, this is a core
function. It's used for all kinds of
file IO inside Python. So it is
absolutely covered in special cases for
weird operating systems. Fortunately,
Linux is pretty simple. All it's going
to do is call this function write with
the file descriptor for standard out,
which as we said before is one, a buffer
with the bytes in hello world and the
count of the number of bytes to write,
which is 12. Don't forget there's a new
line. This is the outer edge of Python.
We could stop here, but what fun would
that be?
All right, to dive into the write
function, we need to talk about lib C.
Libby Cy is a standardized API exposing
helper methods, operating system
interfaces, math functions, and lots
more stuff. Libby is really just a
specification anyone can implement,
right? All you have to do is implement
some kind of function that matches that
declaration. But it's been documented
and formalized a few times. Most
commonly is the portable operating
system interface or POSIX specification.
So when you hear pix, that is a type of
libc specifying document. Now, in terms
of implementations, we're usually going
to be using GNUIB C, which is shortened
to GIB C. You will, just for
completeness, sometimes also see muscle.
If you ever use the Alpine container
images, those use muscle instead of gibb
C, but we're not talking about muscle
today.
All right, this is the implementation of
write in GIC C. We see the expected
three arguments. The file descriptor
we're writing to, the address, the
number of bytes, but this function is
not really doing very much. It's just
forwarding our arguments something
called a sis call. What's that?
Our programs cannot be trusted. All
normal programs run with a bunch of
hardware level security restrictions.
For example, we can only read memory
that is assigned to our process. We
can't just willy-nilly start reading
from someone else's RAM. That would be
very bad. This is called user space. It
is incredibly important for digital
security. But sometimes we want our
programs to be able to do something a
little bit more dangerous like touching
a file. So there is kernel space. This
is a different mode when you are running
the actual kernel, the the core of the
operating system with all of its
elevated permissions. Okay, back to sis
calls. A sis call you can think of as
the window in front of a bank teller. It
is a way that you can pass a request
into the kernel to do something using
all of its fancy permissions on your
behalf. Just like at a bank teller, you
can hand them a dollar bill, but you
cannot jump over the counter and do
whatever you want.
All right, Gibbsc has a couple wrapper
functions to get through in the call
stack. Uh because write is a special
kind of sys call a cancelellable sis
call. For example, you could have a
blocking right that has a timeout in
user space. So we need extra helpers to
be able to smoothly abort it just in
case. But eventually we get to the sys
call itself. Now this is written in
assembly for each language sorry for
each platform that gibbc supports. For
x8664, the way sis calls work is that
you put the sis call ID into the rax CPU
register. That ID is going to be one for
write. Then you put your other
parameters into a bunch of other uh CPU
registers and then you make the sys call
hardware instruction that tells the
hardware to jump into kernel space for
you.
As we land from the sysol CPU
instruction, we are now inside the Linux
kernel. We are running in kernel space.
We have unlimited cosmic powers. There
is a single landing pad for all sis
calls and inside that is a dispatch
table. So it's going to look at the ra
register and figure out what sis call
was this and where should I send it. Sis
call ID1 that's right that is mapped to
the cis write function inside the kernel
which is a oneline wrapper for ksis
write for some reason. Um what that's
going to do is look up our file
descriptor in the list of files that the
colonel has been kindly keeping on our
behalf behind the scenes. This is the
literal meaning of file descriptor 1.
That means that we want the second entry
in that files list.
Then it's going to pass that down to
something called VFS, write, short for
virtual file system, write, because we
are now in VFS land.
Just like we had a dispatch to a call
back based on the sys call ID. We're
going to do that again, but this time
based on what type of file it is.
Roughly a million years ago, I said that
our Python process receives its standard
out from its parent. Now, it's usually
going to be a shell of some kind. bash
or zsh or fish if you're extra fancy.
But what did it give us?
The os.fstat method lets us investigate
any open file descriptor. So looking at
standard out, we can see it's got a
funky file mode that identifies it as a
device file, which is how Linux exposes
drivers. Each device file has a device
number, which is broken down into a
major in the high bite and the a minor
in the low bite. So here we have major
136 and minor zero. This is new
information but doesn't really tell us
anything about what this file is yet.
To figure out where VFS write is going
to send our call, we need to look at the
device driver registrations. So these
happen in the kernel at startup. Each
subsystem can register a driver with a
given major and also attach a set of
function pointers to that device driver.
So because our file has major 136, it is
going to populate the fops or file ops
in any open file handle with these
functions. So we can see that the next
stop for our right is TTY write.
In the olden days, the way you
interacted with a computer was through a
modified electric typewriter called a
teletype. This is a way to send input to
your program by typing on it and receive
output by printing to literal paper. Uh
this one here was the model 33. It was
quite snazzy for 1963. Teletype
eventually got shortened to TTY because
our industry has literally always been
like this.
But I'm guessing most of you don't work
this way anymore. Or at least I hope
not. Usually we'll instead have a
software terminal. Xterm terminal.app
command.exe.
But because everything in computers is
built on the bones of the past, we've
left all of the terminal interfaces
pretty much unchanged since the 70s and
just added an emulated version. This is
a pseudo TTY shortened to pty.
For the most part, it's just a fancy
pipe. Anything that you write into one
side, you read on the other. In terms of
implementation, this means it's a shared
memory buffer of some kind. one side can
read can write into and another can read
from. Technically, this is also
birectional. There's one buffer for each
direction, but we're only looking at
output. So, we're just going to kind of
skip past that for right now. This is
what our standard out actually points
to. It is a pty device running in Unix
98 compatibility mode.
All right, back to the kernel in our
call stack. So, vfs write is calling to
tty write, which is then going to call
into pty write. And if we go one layer
deeper, we find the somewhat wordy TTY
insert flip string flags. This has a
call to the helper function memcopy. So
in here we have our helper string in
cars and we have our memory buffer in
TB. This is the final destination for
our print. This is where it's actually
going. But there's one more step. If
you're familiar with C programming, you
may recognize memcopy as a helper
function to copy uh bytes from one
location to another in memory. However,
the kernel cannot depend on lib C
because that would cause an infinite
loop. That is bad. So the kernel has its
own implementation in assembly. Hello
world plus a new line as previously
mentioned is 12 bytes. So we're going to
end up in this section of the kernel
implementation of mencopy designed for
copying between 8 and 15 bytes. So the
final destination for our hello world is
these four move Q instructions. That is
short for move quadword. This is it.
This is where our hello world ends up.
We could technically go even deeper. We
could look at uh what transistors
implement move Q or how do transistors
work at all. Unfortunately, AMD and
Intel don't actually release very much
information about how their cores work
anymore. So, I'd have to get really
handwavy. I think this is probably as
far down as we can go today. All right,
our right has completed. We're going to
rush back up the stack. We're going to
see every layer in reverse. We clean up
some kernel locks. We jump from kernel
space back to user space. We finish this
call op code. Our print function
returns. We've been in this for 20
minutes. The ripple loops and just waits
for future input. But okay, we kind of
skipped. We showed where the write goes,
but how does it get to the screen?
All right, we talked about pty is just a
fancy pipe. One side writes, another
side reads. Okay, so in our terminal
application, it's going to be reading in
a loop somewhere. This is going to use
the read sis call. that's going to dive
all the way back down through lib C into
the uh kernel, make a SIS call, get to
the TTY subsystem, the PTY subsystem.
It's going to eventually find that exact
same memory buffer as we copied our
hello world into. And it's going to do
the same thing in reverse. Return those
hello world bytes all the way back out
to our terminal program. And then your
terminal program will have to render
them on screen. Exactly how that works
is up to your terminal. Here's just an
example from Xterm.
And so at long last, we can see our
hello world exactly as expected. Simple,
right?
[Music]
All right. Uh, so why do we care about
all of this? Uh, everyone here could
probably have told me what hello world
does. This wasn't a puzzle to be solved.
Programming is full of amazing
abstractions and we couldn't build
things without them. But there will come
a day where every single one of those
abstractions will fail you. Maybe your
program is crashing in a weird new way,
or you need to squeak out a little bit
more performance out of your web app, or
something new we haven't even imagined
will fail. Uh, everyone should feel
comfortable peeling back the layers of
abstractions until you find what you
need. This was certainly overstatement
for dramatic effect, but I've had to
work with basically every one of these
layers at some point in my career,
except
uh, play with your tools. Tinker, fix,
experiment. You aren't just a Python
developer, you are an everything
developer. And inside every abstraction
is a new lesson to be learned. Thank you
very much.
Thank you very much for that, Noah, for
that uh worldw
through the Python internals. I was
going to make a joke about here about
quantum physics, but you already took
that away. We do have plenty of time for
questions. See if anybody's brain is not
as scrambled as mine after this. Please
um raise your hands. Yes. Uh just wait
for a microphone here.
Not a question, but I've used fourth.
How dare you?
And I am old.
Thank you.
All right, any other questions?
Let me have a quick look to see if we
got questions on Discord.
Um so of course why is Python slow? Um
we are built on a stack of abstractions
of abstractions of abstractions.
So many
which ones of those could we strip out
if we actually started clean and and
make Python fast all of a sudden?
I mean so there are a suite of
technologies. It's not really one thing
but unicernels are a way to try to build
all of this into kernel space. Um I do
not believe there are any currently
active Python unicel projects. Um there
were in the past I think they are all
sleeping right now. Um most unicronel
work is being done in okamel and rust at
this point. Um so the idea being that if
there is only the kernel there are no
sis calls. We can just dispense with all
of that weirdness and dispatching and
just compile it all into one thing. U
that is certainly a very compelling
story. I don't think any Unicernel
project has quite worked out a set of
smoothing tools to make this not
extremely disruptive. But certainly if I
needed absolute maximum performance,
that is the direction I would go.
Awesome talk, Noah. Thank you very much
for that. Have you got some time just to
go and talk about um the JIT side that
you um briefly went over? If you can
take another half an hour, that would be
really interesting.
I I slightly understand the JIT. Uh I'm
not sure uh I am qualified. So the the
general idea is what I said of so it's
looking for hot loops. So it means that
the first time it runs a jump backwards,
it's not going to do anything. Uh it
waits until a certain threshold of
identical jump backwards have happened.
So in each jump backward op code, it is
basically keeping track of how often
have I run. Uh, and if it sees, oh, this
jump backward has happened 50 times in
the last, I don't know, thousand uh,
instructions, okay, this is probably a
hot loop. You've got a for loop or a Y
loop that's going really fast. So,
that's what activates the JIT as far as
I know. Um, and so it will pause. It
will try to I mean, so this is the op
code level. So, it doesn't really know
what a for loop is anymore. That's
already been expanded, but it's trying
to reverse engineer what probably was
the body of the for loop. put all of
those op codes into an array, ship them
off to the actual JIT thing, the thing
that turns the op codes into assembly,
which itself is kind of complicated, but
you can imagine how that works. Like
that's just doing all of the same code
as the op code execution would be, but
hardwired into assembly. And then there
is a special thing that it can shove
back where the op codes were. That is,
no, actually, please just literally jump
into this block of assembly. I promise
it's safe.
That's that's basically how the JIT
works as far as I know. Uh it is
evolving extremely rapidly. So again, if
you are watching this on a video and it
is past September 2025, please go check
the documentation because it might be
something new by then.
All right, another question in front
here.
Yeah,
PEP 744 if you want to look up more info
about the JIT. And I'll chuck a link
into the Discord as well.
uh highly highly recommend. She is far
more knowledgeable than I.
All right, we've got time for one or two
more questions if there are any.
No. Uh in that case, uh let's all give
another hand to Noah for his wonderful,
wonderful talk.