foreign
welcome everyone thanks for coming to my
talk my name is Damien I'm the creator
of micro python which I started about 10
years ago some of you may have seen me
give a talk before and some maybe not
this talk is
supposed to be non-technical there's a
few graphs and plots of some data but
otherwise it's mostly just about my
experience over the past 10 years
so what is macro python so for some
people this is new and some people it's
it's not
um this is
just an introduction
now micropython is two things micro and
python micro is sort of small and also
means microcontroller and python is as
you know the language that we're here to
talk about
and micropython is a complete
re-implementation of python from the
ground up no lines of code are shared
with python it's it's a it's a complete
uh a rewrite uh and The Way It Was
Written was so that it was small and
efficient
um with the resources that you have at
hand so when you're running on a really
small computer a microcontroller you
don't have much memory you don't have
much hard disk space you don't even have
a hard disk and you don't have much CPU
so there's not many resources at your
disposal to do things but you still want
to run a high level language in this
case Python and so micropython was
implemented with the aim to be able to
do this to run python you do list
comprehensions for example and async IO
to run that in the context of a tiny
little embedded microcontrol that may
power for example a coffee machine
um
and this started uh during my time as a
theoretical physicist which was my
previous life and I got sidetracked when
I did a Kickstarter in 2013 uh to start
to bring micro python sort of true
fruition uh it took about six months to
develop from April 2013 is when it
started and then the kickstarter was at
the end of that year and that was quite
successful
um and then there was another
Kickstarter a few years later after that
and then many collaborations
um since then with with various
companies and people and institutions
and people such as yourself contributors
hackers uh Educators who've contributed
and used micro python so it's been been
quite a journey in 10 years and I didn't
really expect it to go where it's where
it is today but I guess I didn't expect
anything if you're asked what's going to
happen in 10 years you don't really have
such expectations but uh here we are and
I'm just trying to sort of convey this
things that have happened in those 10
years with a sort of a name towards
describing what it's like to actually
maintain a big open source project
um
the uses of micropython today are pretty
varied so we did quite a bit of work
with the European Space Agency to make
micropython robust enough to run in
space and it's actually used as like a
high level application control language
in satellites
uh it's the reason there is so that it's
easy to update things
whilst it's flying because traditional
space-based programming you have you
know in C or C plus plus or more ancient
languages and it's very inflexible but
these days they want things to be much
more flexible so they can update code
when it's out there and you know it
takes 10 minutes or something to
actually send data to it the other
things have been in LEGO Mindstorms so
Lego have used micropython to implement
their new Mindstorms kits for teaching
in schools
also various commercial products like
smart vending machines various medical
devices
um
vehicle counting and weighing devices on
the autobarns in Germany it's what it's
used for a lung elastomer to measure
lung elasticity came out recently
there's many many applications also in
teaching in research Labs people use it
to you know heat up DNA samples and and
things like this so it's it's become
quite widely used because it's
piggybacking on the success of python
itself as a really you know easy to use
language and then people who want to
build something for physical with the
hands like a robot or you know a toaster
or something
um
they don't necessarily have a software
engineering background but python allows
them to actually program something to
make it work and if you could mix you
know this ease of programming with
actual Hands-On you know make an LED
blank and a motor turn people can make
their eyes make their ideas
real in a very
a very straightforward way so that's
kind of the key selling point of
micropython is it's mixing
microcontrollers and the Python language
together
um you can try it out yourself there are
many many different Hardware platforms
that you can get started with some of
them are very cheap only a few dollars
uh some of them you know more expensive
you can run it on lots of different
things with lots of different uh plugins
Motors lights whatever you like you can
actually try it online and that bottom
link there
so if you haven't played with it it's
very easy to get started
now
um this talk is trying to give a bit of
an insight into what it's like to
maintain a big open source project like
micropython I mean it's not as big as
python itself for example or the Linux
kernel but it's still you know
reasonably
um scienceable project and there are
lots of challenges that come with
maintaining the source code uh
developing the code and then maintaining
the community and building the community
around it because obviously you don't
really have a project if you don't have
people using it or you can have your
side hobby projects but you know a
project that's out there in the world
the users are really the big part of the
project and we all know this with python
and conferences like this
so the social aspects of software
engineering is I guess
what I'm trying to convey here I'm a
very technical person and I love working
on micropython because it's very
technical but
as part of that I've been thrown into
this you know thing of building a
community and doing social sides of of
um of computers and technology and I
think it's it's really interesting
um and really eye-opening so also at the
end I'll give a bit of advice if you
were ever going to set up your own
project
um what sort of things you can do at the
start to make it easier later on
so the main I guess the place where I
spend most of my days on GitHub
at this webpage or at my terminal and
GitHub is is the sort of the Nexus of
where everything happens uh it's
micropython is free and open source so
free means you can use it without having
to pay and there's no licensing it's MIT
license and open source means the source
code is there and available and anyone
can access it those two things are not
always the same and you can have one
without the other but micropython is
both of those and you can access the
code if you go to this GitHub website
now
um the open source development that we
use uh is uh well it's based on git
Source control uh if you're not familiar
with this
git is sort of it changes your way of
thinking about code or at least it has
for me and that code is not about the
code well it's about the code you have
but it's also very much about the
stepping stones that you make every day
to revolve your code from one spot to
the next so a commit is a small stepping
stone that you make
your code has this form you make a
commit and it changes its form and to me
the commits the Delta the change the
stepping stones are really the important
part
um because you can look at them in one
go and say this is what I'm changing and
it comes with a test and it comes with
documentation and this is how the
project evolves
and we're very very strict with commits
and how they're formatted and and The
Styling and the commit messages
and I think this this is really sort of
part of keeping the community on track
when it's developing a project
um
the having a very strict git history and
a strict history of Stepping Stones
allows you to easily track down bugs in
the future when they do pop up so you
can go back and use git bisects which is
when you like we have a bug we know that
this code is good this code is bad so
somewhere in here the stepping stone
went wrong and you can easily track it
down and then fix it and it's also good
just for you know people who are new to
the project and they want to see well
why was this change introduced they can
go back in time and have a look and
usually hopefully the commit description
explains why this change was made
so I think having a having this really
strict git workflow is really important
for us we have a lot of testing we've
done a lot of work in testing part of
the work with European space agency was
to add a lot of tests to get close to
100 coverage
which is really important so every line
of code has a test associated with it
now the coverage is never perfect
because
there are still many more scenarios than
just making sure every line of code is
run but we do have very
good coverage and we are very strict to
try and keep that up and the continuous
integration whenever there's a new
command made there's a lot of tests that
are run automatically and you have to
pass those tests to get that commit in
to get that next stepping stone done
um it's so it's a really good safety
harness you can be really confident that
if you make a change to the code even if
you don't know what's going on in the
whole code base if you don't break any
tests then probably your code's okay
and by the same token like you can't you
can't just go randomly deleting lines or
anything because of the test will break
so it's the test harness is really there
it's like a safety barrier to make sure
we don't do things wrong
Hardware in the loop is something we
don't have yet but we do have offline
Hardware testing which is important as
well
and the releases uh the release cycle is
not very strict but we try and aim for
releases maybe every two three four
months and the releases do take a lot of
effort because you've got to get to a
point where you're confident with the
code and you know you're going to
release it and people are going to
upgrade and maybe they're gonna you know
the coffee machines now are going to run
the new version of macro Python and you
don't want it to break down because you
know embedded system is really hard to
debug so we're very strict to make sure
that the releases are equality and you
know known to work and hopefully not
have any bugs or at least no bugs that
we know about
um or if we do we can document them so
the continuous integration testing we
have helps us there but then also
there's a lot of hard testing on real
Hardware before we actually make a
release
it's also fun to write the release notes
although that takes me a couple of days
uh of full-time work to actually make a
full set of release notes and it's good
because it looks back on the past few
months of what you've been doing and
it's like this is a summary of my life
these release notes but it's yeah it's
good and it's nice to make a you know a
summary and then put it out to the world
and everyone reads it and they're like
okay I like this new version because it
has this and this feature
um so this is this is sort of the
day-to-day grind of of working on a
piece of code
um
one thing that is really important here
is funding so it's it's been 10 years
since I've been working on this uh and
you need a source of funding if you have
a open source project that is you know
going to live for so long because you
know you need to live
um and you can see here I've got we've
got uh GitHub sponsors uh which is a
program on GitHub where people can
sponsor for like a dollar a month or a
hundred dollars one off or whatever you
like and that's a continual source of
revenue for us now since 2020 and you
can see there how much that's in
Australian dollars how much we're
receiving from GitHub sponsors so 2023
is small because it hasn't finished yet
um but we get you know a sizable amount
of money there from GitHub sponsors and
that helps to pay sort of two part-time
maintainers
um not myself technically
um
so although yeah there is a bit bit left
over there but it is good the other
sources of funding we have are Hardware
Sales because it's a hardware software
project
we can have Hardware that we sell and
some of the money that from that can be
used to support the development which is
great but there's also a lot of others
contract work that I do and that other
parts of the members of the team do as
well so people come to us and say we
want macro python in our product or we
want this feature added to micro python
or we want this thing improved and uh
you know they pay us to do that work for
them and usually that work that they've
paid for goes back into the open source
community in the sense that it's not
hidden behind anything it's
contributions made to micropython that
everyone can benefit from so that's
really great but you know that someone
can pay for something to be to be done
um so I yeah I can't really stress
enough how important it is to have
funding and I think there are a lot of
pure software projects out there that
find it really hard to get funding
um and and to continue on
um there's been lots of stories you know
about about one-man projects that die
because that person doesn't want to do
it anymore and then people are relying
on this project so where does it go so
we're very fortunate to be able to
continue and to sort of Thrive and have
enough money to live and continue doing
what we love which is develop um
code
the other point to make is about
licensing which is a really deep topic I
have a single line we use the MIT
license the GPO license is kind of the
opposite open source license to MIT if
you want to say that we try we steer
clear of GPL because in an embedded
environment it's really dangerous but
difficult for customers who want to use
micropython to accept the GPL because it
means that any code they write like if
they're making a coffee machine again
um then they have to open source all the
code for the coffee machine which
doesn't work they want to sell a coffee
machine they don't want to have to open
source the code for it so MIT is sort of
yeah the choice that we've made and I
think that was a good choice because
it's allowed us to be successful people
use micro python because they know they
don't have to
release their code to the public
yeah as I said it's a big topic
licensing
um
so the community the community is very
important as I mentioned before and in
micropython there are sort of these four
portals of the community mostly can
interact with the project
um apart from we also have in-person
things events uh once a month at the
hacker space in Melbourne but online
there's GitHub which I told about before
which is where the code is there's also
GitHub reviews also for discussions
which is like a forum we used to have a
phpb forum but it was getting a bit old
so we switched to GitHub discussions we
also use GitHub for the wiki we had a
separate Wiki at one point but we again
tried to consolidate there and we use
Discord now for chat we used to use
slack but we're using Discord so we have
moved around uh you know it's um sort of
this is another part like it's it's not
easy to choose a technology to use to
interact like we had to make choices
here and some people were not happy with
those choices
um they were like we want to stick on
slack or we want to stay with phpb but
as you know the maintainer and open
source project you have to make
decisions to keep things moving and
there are reasons we went
um
one way or the other to keep up with you
know the pace of technology and change
but these sorts of things you don't
really think about when you're thinking
about coding and software engineering
and I don't think you know any
University course would teach you you
know how to choose a Wikipedia page or a
you know a chat interface for all your
users but it's it's really taxing and
you know the the team was we spent quite
a lot of time discussing this and
thinking what should we do and that
takes away from our time you know we
need to make a living but do we spend
you know 10 hours discussing what
platform to use so you have to make
decisions about what you spend your time
on and these things these things you
have to be
um conscious to take up your time
this is a graph here of the monthly
contributions to micropython over 10
years
um which is the
taken from the the git repository so all
the stepping stones that we've made
each the blue and the orange uh the
commits made per month so the scale goes
up to there to a few hundred so there's
a few hundred commits per month
the blue is the ones made by myself and
the few other core maintainers and you
can see that sort of trending downwards
the Orange is by a sort of contributors
who are not the core maintainers and
that is trending slightly upwards there
towards the end which is really great
um the green line is actually on the
right hand scale there that's the number
of unique contributors per month
so who've made a commit to the
repository so that's going up as well so
this is a good Trend in that the
maintainers are doing less commits Excel
people doing more commits and there are
more unique people contributing to the
project so
that's sort of
you know that is not without a lot of
hard work to Foster an environment where
that can happen where external
contributors can come in and make
contributions to a project especially
one where it's
um
you know you have to make sure that the
code is still space grade in the sense
that you can actually go up in a
satellite and run and not crash the
satellite so we have to be very careful
about introducing commits loot
repository introducing code changes and
I think it's uh something to be proud of
that we can get to a stage where we have
a lot of external contributors so you
know we have in
in a year we have over maybe 150 to 200
people contributing and we have around
an average of around about three commits
a day which is a lot like if you sustain
three commits a day for a year over 10
years that's a lot of work
um
this here so this is a plot of the
number of people who have downloaded
some micropython firmware to put on a
piece of Hardware I like um an esp32
board or something the different colors
are the different boards that your
different ports
um but different boards that you can use
different targets architectures so the
orange line is the Raspberry Pi rp2040
which is a really popular board that
came out at the start of 2021 and that's
sort of taken off a lot that got a lot
of popularity the blue line just below
it is esp32 which is espresso's new
Wi-Fi board well not that new but you
can see that started around 2017. the
green was their previous Wi-Fi board the
esp866 which started off earlier than
that then the red is then stm32 which is
the pi boards
architecture so this is showing
obviously that we're having more users
or at least more people downloading but
hopefully it's a good indication of of
users so downloads per month so we've
had over three million total downloads
in
so this is eight and a half years
because we didn't start downwards you
know immediately
um so this is showing that the community
is definitely people using is growing
and the previous
um
that graph was showing you know people
contributing to the code uh the way
that's trending
so
building a community uh it's it's really
hard and people have to do you know
marketing branding Website Maintenance
um I didn't really like doing those
things but you'd have to do them anyway
um some people are very understanding
when you're part they're part of a
community and they're like yes I
understand that this is your time that
you're spending on this
um and that's really great
um some people are less understanding
most people have good intentions but
sometimes people just like don't
understand and get stuff wrong or get
confused and are rude and you know
they'll say oh this doesn't work and it
should work and this is a terrible
project or you know they could they get
the bad end of the stick they come to
some part some corner of micro python
that's just not well tested and doesn't
work and it gives them a bad impression
of the entire thing
um so it there is a lot of a varied sort
of interactions that you have and people
have different experiences
um
for example to make a bug report people
are supposed to
you know tell you what the bug is and
make it so you help help you to fix it
we haven't we have an issue template
where if you want to submit a report
there's a template that you fill out
that was a we added that so that people
would actually give good bug reports
um but it's been almost completely
ineffectual ineffective people basically
just delete the template and write
whatever they want and usually don't
even say you know what version they're
using what Hardware they're using this
so like my Wi-Fi doesn't work
um and you're like the complete loss to
help them because there could be so many
reasons why the Wi-Fi doesn't work
um so some people you know most people
have good intentions but
um usually the good people are the
people who don't say anything and I just
quietly understanding the people who are
who don't understand are the people who
make you know take your time and you
need to help them but you don't need to
but you choose to
um
it's important to exit expectations but
people rarely read documentation or
anything so even if you have a Manifesto
people are not going to read it and
they're just going to behave the way
they do anyway
um
and some people well a lot of people
with micro python come to it to try and
learn like how do I turn an LED on and
how do I blink an LED and then they get
stuck and then they ask for help and
complete I mean this is great we're
teaching people here but who has time to
help them
like do you take it out of your time to
do that when you have other pressing
things to do to write code or to
maintain a website that's you know
crashed or something so you really have
to decide where your time goes and you
know what you want to do and with a big
Community hopefully there are other
people that can step in when the
questions are simple and as a maintainer
you know you're needed for when the
questions are really hard or there
really is a deep problem to solve
um
so I think the main one of the main
points is that as a maintainer uh
it's important to make sure that there's
Direction and that people don't get lost
and be like oh there's no decisions here
uh everyone has different opinions and
we don't know what to do so to maintain
his role really to set the direction to
say out of all these choices we have
we're going to choose this one because
of this and this some people are going
to be upset some people are going to be
happy but at least this is a decision
made and we can move forward and then
people can deal with the decision and
say okay well this is how it works and
I'll I will work around that
um but if you just let it sort of
flounder and there's no Direction then I
don't that's really not a good way to
run a project
but it is very rewarding I mean I you
know there are times when it's tough and
you don't want to open your email
because there might be emails that are
people you know not abusing you but you
know it's not really nice to read
responses sometimes
um but then there are other times when
you get really nice fruit replies from
people or you know thank you it's a
great project and things work really
well and I've used it for this and that
and it's really helped me a lot so it is
very very rewarding when things work
um a lot of people are thankful and
grateful which is nice and there are
definitely some standout community
members that help and go out of their
way without any without asking them and
they help to you know Foster the
community and answer questions and be
there all the time which is really great
um and as a maintainer you have to
recognize those people and the people
who have expertise and are willing to
help and really allow them to do their
thing and Foster them and Foster their
activities and say yeah well just step
back and let them do what they do well
and also you know I ask people when I
don't know something
um I ask them and say you know you know
more about me than this topic so can you
please help
um
so that's that's good
uh this is a an obligatory uh AI
generated image
it's such a fantastic technology but
anyway this is just uh to show you the
next bit is about bug reports and
contributions uh
so when as I said before bug reports
some people do not give very good bug
reports because it's not reproducible if
you're going to give a bug report please
make it reproducible and then at least
someone can try and fix it uh but then
on the other side there are
contributions a pull request someone who
has an idea about how to change the code
and they're like I would like to change
the code this way and when someone makes
a pull request
um you're like okay here's another 10
hours of my life gone because Apple
request it I mean that was joking in a
way but a progress is a lot of work to
go through uh sometimes it's you know
it's a few hours reviewing at the code
giving some feedback and then they go
back and forth for maybe a few weeks
where you say you have the assessor
changed has to be tested you review the
code again and eventually it's good
enough to merge and then you go through
the last little details of making sure
the commit message is right and all the
tests passed properly and maybe the CI
broke down a little bit so you have to
fix that and it takes it takes a lot of
time to get a contribution from someone
in and that's not to discourage
contributions because they're obviously
what make the project work but as a
maintainer and if you maintain or a
one-day maintain a project you have to
be aware that you need to do a lot of
work to you know Foster People Foster
the community and get
the code in and get it working sometimes
it's easier to do the work yourself and
just say okay I don't know how to fix
this I'll just do it but you can't do
that forever so you really have to
learn how to get people uh
efficiently contributing to the project
and one thing we do down the bottom
there is we we try and automate as much
as we can in terms of contributions so
people will have
uh a pull request and it will go through
the you know code formatter and spell
checker and code styling and even commit
formatting and making sure they've
signed it off and everything and those
things are all automated so we don't
have to worry about that and people can
check their code formatting make sure it
is correct fix it if it's not and once
all those things pass what's left is to
review the actual code which is you know
the the meat of the of the problem
um one thing
to mention is that sometimes you know
you get sort of lost up in a lost in a
conversation or a heated argument or a
discussion or something about code and
you know I think a lot of us have
probably been there and one good thing
to try and do is to distance yourself
from the actual personal interaction and
talk about the code rather than the
people and instead of saying why does
your code do this you say why does that
code do this
so don't talk about the person's code
you just talk about the code and that
removes the sort of human
there is human element there and that's
important but
when you say code doesn't work
you can almost feel like an attack on
someone who wrote that code so and say
instead of saying why doesn't your code
work why doesn't that code work
so some maybe take home points about
what I've learned it's hard to maintain
open as a project it takes a lot of
effort but I'm still here after 10 years
and it's I still love the technical side
and the community side is really great
it's really rewarding you can't please
everyone there are always people who
will think that you've made a bad
decision but you just have to go with it
um
if you're passionate and just follow
your instincts and have some nice sort
of like a Manifesto or a goal or a real
overarching principles in this case it's
micro python so microcontrollers and
small and python so we've got to try and
Implement python but we've got to do it
in a way that's small so we're not
trying to re-implement python so when
people say oh it's missing this and this
and this and we need that to match I'm
like we can't because it won't work then
like it won't work on a microcontroller
so we have these two pillars micro and
Python and we have to make sure we
follow those for all the decisions that
we make
um
so I think we're very fortunate to have
a project that has strong ties to
hardware and software so it's a real
physical thing
you can make something and it's also a
software technical project and that
really gives a good balance
and funding as I said before if you want
to do a project long term you really
need to think about how you'll find the
project maybe you can do it in spare
time but in this case the project is
large and we have multiple maintainers
and some people working on it and that's
the only thing they're doing so we need
to be able to pay people to do that
um if you're going to set up a project I
suggest you know you choose your license
from the beginning write a code of
conduct pick a git workflow that you're
comfortable that you like and stick with
it uh make sure you have a nice
directory structure so you don't have to
move files around at the end
uh version well from the beginning
hopefully using semv if you don't uh
read about it if you don't know
um have some really good code formatting
and linters and things set up from the
beginning uh Choose You Know pick black
as your python code formatter and clang
format or something else
um and stick with that because that will
really help you from the beginning
and then I think it's just some final
words that
I think sum up a lot of the you know a
lot of the time during the day when I'm
working
um I feel like this is that when you put
yourself out there online when you've
made some code or when you make a change
when you make a comment or a post in a
community you're really putting yourself
out there
um
and you know that's hard because you put
your you know your person out into the
world and
when you're maintaining an open source
Community I think you have to be aware
of that and make Foster an environment
where people are comfortable to put
themselves out so when they put up a
piece of code to review you realize that
maybe put a lot of time into this or
they've made a bug report which is
really detailed and they put their self
out there so if we have a community
where people are comfortable to put
themselves out there you know you can
build that Community up and make it a
really nice place to live well to live
online and to work and I hope that's
what we've done with macro python over
the past 10 years has fostered a good
Community where we have members who
enjoy being part of it because yeah I do
and I think it's it's it's been yeah a
very rewarding 10 year so far and I hope
that there is more more to come there
definitely is
so thanks thanks for listening
[Applause]
thank you very much Damian and we have
time for one quick question
anyone would okay oh look
yeah I think I'll go for the close one
sorry
thanks Damian um how have you uh engage
people to you know assist you with
testing on I guess the variety of
hardware and applications
yes uh that's a good question we we have
automated testing which is obviously
people don't have to to work with but
for Hardware testing uh
people
there are community members or people
who do that when there's a like a when
there's a pull request or a piece of a
commit for a particular piece of
Hardware that I may not have or that may
be difficult to test other people will
test it um you can just you know at them
on GitHub and say at you know X can you
please test on your hardware and they do
that and that's great yeah so there are
people who will test on various pieces
of hardware for us and applications
sorry application applications you mean
um so people have got micro python
applications and how do they test them
yeah is there any pre-release testing
that we have I have a few uh private
applications that I do tests on before
we make a release to make sure that
they're still functioning and they yeah
they do good integration testing of the
uh well the whole system yeah
um
usually people will test the non-release
code like before we make a release like
they'll be like we need to up we want to
use the latest micropy because it has a
feature that we need and I'll test it on
their project and then there'll be a bug
report the next day about how something
broke and we fix it so it is a really
continuous process there's always
generally a lot of testing from other
people
which is very helpful
thank you
thank you so much and here are your
speaker gifts and thank you very much
presenting today
[Music]
thank you