[Music]
Welcome back everyone. Another brief
reminder for those sitting in the masked
area, please wear your masks.
And now it is my pleasure to introduce
our next speaker, Sheena, who is going
to be talking on scaling Python powered
machine learning with Snowflake. So a
round of applause please.
[Applause]
Hello everyone. Thank you so much for
joining in. So I'm Sheena and today I'll
be talking about how you can scale your
ML pipelines because when you really
start working on it or you are building
a prototype, you don't really think
about this enterprise scale level of
problems that can really come to you. Um
so my idea of the session is also to
give you like a view of how we solve
those kind of uh data scaling problems.
All right. Who am I? I am an AI field
CTO in Snowflake. So I work with a lot
of enterprise customers over APJ and um
we help them build a IML solutions with
their best practices and things like
that. So this is today's agenda. I'll be
going through just to give you an intro
of what Snowflake is and then um how we
scale different parts of the ML life
cycle which is like data processing,
feature engineering, model development,
inference and finally the consumption
part and we'll give you a summary of it.
All right. What is Snowflake? Anybody
working on Snowflake? Anybody heard
about Snowflake? Okay, few hands. Good.
So Snowflake is a unified platform where
you can bring in all your data and you
can build your AI on top of it. And
finally you can also build your apps
because the consumption part is very
important. If you cannot build something
that can help the users to use your AI
that is where the value is not going to
come out. So you can build the entire
thing in snowflake.
Now what are the key messaging of
Snowflake is that we are a very easy
platform. We are a SAS platform. We
manage all the infrastructure. So you
don't have to worry about provisioning
it, maintaining, tuning it. So if you
want CPUs, GPUs, you can just go in and
leverage it and use it on the go. And we
are completely connected. We give you
options where you can just go and share
your data, collaborate with different
other uh customers, vendors, etc. And
it's trusted. We take um security very
very seriously in Snowflake. So it's a
trusted platform as well. Now I'm going
to talk about distributed and scalable
Python in snowflake. So we'll go through
each of these u sections and see how it
is done. First one data processing.
How do we do data processing today?
That's first thing we are going to
address. Pandas. Who all are using
pandas here? Okay. I'm with the right
audience. Awesome. So pandas is one of
the most popular libraries. I'm sure you
all are using it for processing the
data. Now what's the problem with
pandas?
If you write a code in pandas and if you
want to do an enterprise level
productionalization, we usually see like
an average of 6 months has been taken to
rewrite that code and write it in a
enterprise uh level production code.
It's all because of how really pandas
work. So, pantas is working in memory
and you mostly get out of the memory
errors. I'm sure some of you might have
already got this. There is no way if
you're working with pandas, you have
never encountered an out of the memory
error. It's very uh common to working
with pandas and pandas is single
threaded. Doesn't matter how many cores
of CPU that you have, it's not going to
take it. So, it will always take one
core of that CPU and keeps on working.
So no distributed processing at all
as a result it cannot scale even on GBs
of data millions of rows it's not going
to happen and encountering with most of
these problems like you will have to do
a retransation of your code from pandas
when you go into productionization on an
enterprise scale and also it is very
difficult to experiment debug loss of
productivity etc.
Now how does Snowflake helps our
customers or anybody who is building
that data processing part in Snowflake?
We have what is known as Snowpark Python
API. Think of it as like a Python API
which is a wrapper on top of um the most
common Python libraries and things like
that. Now under Snowpark API we have two
main points um I want to highlight here.
One is Snowark helps you to run the data
processing and transformation in a
distributed fashion. And the second
thing is we are more moving towards
executing your code near to the data. So
usually think about it when you're
writing a code in your laptop your data
is in a database you're just going to
download that data put it in your local
memory and then read csv or something
like that and then you work on it
locally but here the concept is like
pushing this code back to where the data
is residing.
Now we have two main offerings under the
snow park API which is pandas API and
dataf frame API. So for the pandas API
it is distributed. It solves all the
problems that we discussed for the open
source pandas. And then we also have a
dataf frame API which is u lazily
evaluated something very similar to
spark if you have done that.
How do I get started with pre-processing
in in snowflake? So I need an
environment to write it. It can be your
own ids or snowflake itself has a
notebook inside. So you can leverage
those or use your own ids like Jupyter
notebook, VS code, anything to write
your code.
Now you have options. So I want to
highlight again if you ever want to just
stick on to the normal pandas python
code OSS version and then want to do the
pro pre-processing. It's completely
fine. So these are the extra options
that we provide in snowflake to the
customers to do the things on a very
scalable manner. So we have snow park
data frame APIs. You can see a sample of
code there. DF.filter df.tate is equal
to WA. So we are sort of like um using
it very similar to if you're a user of
pandas. And then we have other option
which is pandas which is built on top of
the open source ones. Uh we maintain
this the name of the functions to be
very similar to the open source version.
So you don't have to change much of the
code. And in case if you want to write
custom Python code, very custom logic,
we also have something known as
open-source um sorry userdefined
functions and things in um snowflake as
well. Now the the best part is that if
you write code in these APIs or in this
manner, it always gets pushed down to
snowflake compute. So you write the code
and there is a client level libraries
which will push the code and the
execution always happens inside
snowflake.
Now we'll go double click into a little
bit more on the pantas on snowflake part
how it works how is it distributed and
how it becomes scalable
in this case again it's an extension of
the snow park API which is modeled on
top of the oss uh pandas as well now
anybody heard about modin
mo yeah okay awesome so mod was an um
open source initiative where building
this pandas to be on a distributed
scalable way Um it has been over 5 years
research and things like that. So modin
is also available as an open source. We
recently acquired modin as well and
snowark panners is already available in
snowflake. So what essentially this desk
does is you can keep on using your code
as such and the main thing that you need
to change is your import statement. So
you can see in the uh in the uh picture
there. So you are basically changing the
import pandas to import modern.pandas
pandas and the rest of the functioning
and everything remains same but you will
still leverage the distributed and
scalable processing behind the scenes.
The oss version is still available if
you want to check it out. Yeah, please
feel free to check it out as well.
Now how does it really work? How does it
really scale in snowflake? What is going
to happen is like when you write a
statement like pd.conat concat um df1
df2. What really happens is like there
is a query translator that is literally
taking your statement and then we have a
connector which connects to the
snowflake and it automatically pushes
down as a SQL query. So your pandas is
getting translated into a SQL query and
it is pushed down to the snowflake
processing engine.
Now the processing engine consists of
multiple nodes. It is distributed. So
Snowflake behind the scenes
automatically distribute and scale the
things for you. What are the advantages
to it? Seamless prototype. You don't
have to worry about refactoring your
code and everything and zero data moment
because we are pushing it towards the
snowflake and then you can just keep on
using your pandas which you're very
familiar with keep on writing the code
in that.
Now this is the main um highlight of
what is the difference when you switch
between these two libraries when the
difference is mainly like a import
statement. What happens is like let's
take a look at the second bar graph
which is about 10 GB of data. So the
x-axis is the data size and the y-axis
is the second. When you look at the 10
GB data the processing is 30 times
faster than using the normal oasis
pandas. So you can see the blue colored
one is the snowflake model um and the
other one is using the pantas osis wash
and the last one when it goes to one TB
you can see the processing gets finished
under 50 seconds for snow park pandas
but whereas other one it didn't finish
you will mostly encounter out of the
memory errors
now going to the feature engineering any
data scientist here
okay Good. So when it comes to data
science, it is very hard to do certain
kinds of feature engineering techniques,
we call it like one hot encoding and
things like that. These are very
computationally expensive operations. So
how do we help for those kind of
operations is that we have another API
called Snowflake ML Python package. So
what it essentially does is it is again
a wrapper that is built on top of the
open-source ones scikitlearn mainly and
uh we also have the rappers on top of XG
boost scikitlearn then live GBM all
these packages that helps you do
different kinds of pre-processing as
well as the ML training part in a very
distributed fashion. So under the
pre-processing you can see most of the
uh pre-processing things that you use
like standard scalar ordinal encoding
all those things are already there. So
you just have to use the functions and
behind the scenes we handle the
distributed things for you. Now again
how does it work behind the scenes or
how does it distributed the same logic
ML pre-processing gets again passed
through a query translator which
translate into a SQL and pushed back
into snowflake engine and then execution
happens whereas for the model training
some training we cannot distribute it
honestly um so if it is like a XGB
regressor or something it will just push
down as Python byte code and it will get
executed we'll go into more on how to
distribute the ML part as well later in
the slides. All right. Now, what is the
difference? Why should a person use like
snowpark ML for feature engineering
compared to the other scikitlearn or
anything? So, this is experiment we did
on uh 200 million rows of data which is
16 GB data in the memory and the only
difference between the two bar graphs
that you see. The first one is using uh
skarn standard scalar library. So
everybody is familiar with that. And
then when you do a scalar function there
are usually two um ways to it. One is
like you you train it and then you fit
it. That is why you see the two colors
which is red and the blue color. So
there is a fitting time and there is a
transforming time as well. Now the same
data when we run with snowpark ML
standard scalar which is the second one
here you see like the difference is
quite significant. It is about 25 to 50x
of speed up. And another operation we
also ran is one hot encoder. So these
are again very computationally expensive
feature engineering operations that data
scientists use and you can see the
significant difference there too. And
another one in the end which is the
purple bar graph. It is about
correlation. So in skarn also you can do
the correlation of features. Um this
becomes like very very hard when the
number of features increases. So in this
case we did the experiment on a 1
million of rows with 512 columns running
in a medium snowpark optimized
warehouse. warehouse is our the name of
our compute that we use and you can see
the result is that once 1,024 columns
there is usually getting out of the
memory errors for scikitlearn packages
but if you use it um you can get
everything still ongoing and without
getting a memory error you can execute
those correlation metrics and things.
Now going to the model training in case
of model training we offer a container
runtime for ML. Now whatever code you
have however optimized way that you
write if you don't have a scalable
infrastructure on which you want to you
can run the code it's not going to be
effective. So that is why we come up
with this container runtime for ML where
you have CPU and GPU pools which you can
configure out of the box and then it is
uh powered by ray ray compute cluster
and when you spin up a container what
really happens is like it comes with
most of the python libraries that you
really use it for. You can see skarn XG
boost light GBM etc. It comes as the
pre-built packages and we also have
optimized data injection if you want to
get a one TB of data or a huge size data
into the container runtime. It is much
easy and then we have scalable training
APIs over top of it etc.
Now how do we distribute it? So I have
an XG boost model and I want to
distribute the training of this XG boost
over multiple nodes. So usually outside
snowflake or if you want to do it with
os python it is quite difficult. So this
is why we again build a wrapper um on
top of it. So you can just keep on
writing your any opensource code and you
can just scale it easily by using the
scaling configuration. So you have seen
the only difference is that you have to
specify this particular scaling
configuration where you can specify how
many GPUs you want, what is the
estimators and scaling configuration
particularly that will help you to do
everything easily and um all the infra
management all the scaling everything is
done by Snowflake and also Snowflake
manages how to distribute the training
across your multiple nodes which you can
just freely use this particular
distributed training APIs and it just
runs.
Now what's the difference? So you can
see the first the blue line which is
using the snowflake ML API that I just
mentioned and also the other experiment
is with the oss um library for the xg
boost. So you can see the difference
when the x-axis is like the size of the
data and the x boost training time as
well. when the size of the data
increases, it's going to take more time
for the oss version to finish the
training and things like that. But in
case of snowflake, it can automatically
distribute. And you can see how wide the
gap is.
All right. Now, we also have a
hyperparameter tuning API. The one that
we discussed before was to run an XG
boost in a distributed fashion. Now, how
about if I have multiple hyperparameter
tuning? This is a a very common
technique all the data scientists use um
where you can run multiple
configurations of the model and see
which is the best one and you pick the
best among that. Now this is also a very
difficult technique but very essential
if you want to build a model you have to
test different hyperparameter
configurations and select the best one.
So you can easily scale hundreds of
model training across this one by using
it. Again just leverage your normal
Python OSS code. You can bring in any
package or anything and you will write
your define train function and you'll
write your opensource Python code and
under the tuner you basically specify
what configurations or what
hyperparameters you want to use to train
this models in a parallel fashion and
you can also um define different
parameters metric etc. So it will
automatically distribute for you and you
will get different models run parallelly
and finally you can pick the optimized
best one out of all your hyperparameter
configurations. You should also be able
to do like grid search strategies,
random search strategies. So these are
the strategies normally data scientists
use to um train different kinds of
parameter tuning and then pick the best
out of it.
Now we also have as a part of the
snowflake ML API a bit of hyperparameter
optimization that is coming out of the
box. So you can see the code is very
simple and very similar to when you use
scikitlearn or anything. So you can see
we are doing a grid search CV and you
are just specifying all these parameter
but behind the scenes what we do is like
we are running everything in a
distributed fashion. So you can see the
same code when you increase the number
of nodes or the size of the compute from
medium to large to extra large the time
is going to go down because the
automatic distribution happens behind
the scenes for you in snowflake.
Now this was a question I encountered
when I was uh talking with one of our
customers. So their question was like
hey I want to build 60 million models
for my use case. there's only like one
single use case but there has to be 60
million models that needs to be built
and um this use case is more a
hyperpersonalization use case where I
want to build a model for every single
customer and I want to understand how
much they are going to spend or are they
going to churn so I don't want like a
generic model so how do we do this so in
snowflake we did this um solve this
problem using a partitioned model
training feature that we use and for
this one when they were running it in
another platform without any
parallelization any distribution. It was
taking 18 hours to run. Now what we did
is like we ported the code in snowflake
and we use this feature and we could
finish it under 3 hours. So that's the
difference we are talking about.
Now this is very helpful particularly in
the cases like there is a partition key
or a partition that you can really
leverage. For example, in this one,
think about like different stores. You
want to understand what is the uh what
is the requirement or a demand for
different stores and you want to build
it on a store level or even on a product
level, right? I want to understand what
is the demand prediction for my product.
Then you have to go into each product
level and build it. So this is how we
can do that. Your training data just
remains the same. It's the data in the
table. You don't have to partition it.
You don't have to do anything. But what
we do is like we will do uh we will
create the partitions for different
stores and we'll create the subm models
for each of the stores and finally
everything gets pushed into our model
registry as one single model. So anytime
you want to do inference you call that
single model from the registry and it
will pick the corresponding submodel for
that particular store and give you the
results. So that is how it works and
there are two versions of it stateless
and stateful. So imagine you're building
60 million models. It is hard to save
all the models 60 million. Uh every day
you are running it. Then you don't want
to like save all the models all the
time. So it has to be like training on
the go and just done with the inference.
That's it. Next day you repeat the same.
So those kind of models are called
stateless models. You don't need to save
it. Then you have stateful moders. um I
want to save that particular model
version which is the normal behavior
when you do a build a data science model
you save it and then you call it and do
the inference so if you want to do that
that is also fine but it's considering
when you are building millions of model
it's going to take time um and uh it
will affect the performance as well
all right um so how do we do this many
model partition inference in snowflake
it is very same as you can see here you
write a python class which is example
forecasting model just um another
example and you can put a decorator here
the custom model decorator and then
under that function predict you can
write any Python code that you want any
algorithm any packages that you want to
use normal osis python xg boost anything
you can write your code underneath it
and after that you can log that model
into the registry so even before the
training we are logging the model into
the registry because we're building 60
models on parallel and then you're going
to push it to the registry. So registry
handles everything behind the scenes for
you. So in this case what I'm going to
do I'm going to push this model first
into the registry. So if you're not
familiar with model registry think of it
like a centralized place where I put all
my models. So I will have later like a
auditability traceability. It also acts
as a handshake points between different
teams. So I'll push my model into this
registry and say like hey apps team or
ML engineering team you can take the
model from there and predictionalize it
and things like that. So I do that and
then next one is I can just go in and
call the function run and just specify
the partition column. This partition
column can be store name. So which is
very easy. You don't have to physically
partition it. You just go here and one
line of code and you're specifying what
is the partition column you want to use.
Now tomorrow I decided I don't want to
do this particular modeling on a store
level. I want to do it on a country
level. So you can come here and you
change your partition column to the
country. So it only needs like a a
partition key that it can use and the
code remains the same. You don't have to
change anything.
So that's how we build millions of model
in parallel in snowflake.
All right, inference. When it comes to
inference, we provide two kinds of
inference as um usual ways which is one
is batch inference. So batch inference,
we provide it through warehouse. In this
particular kind of an inference, it
comes out of the box as distributed.
Let's say you have a million rows and
you are just using the warehouse
inference. Then it will automatically
behind the scenes distribute your data
and do the prediction for you. So you
can see the inference code here which is
model predict and you are just running
the function that's it you just get your
data test data and then you uh try to
predict with the model then behind the
scenes snowflake handle the distributed
processing and then finally merge all
the results together and then give it to
you as a one single table or result. Now
going to the next one which is noar
container services that is something we
use if you want to go for a GPU level
inference or let's say you want to go
for a realtime inference. So in case of
a realtime inference um there is
something you need to balance the load
as well. For example now I only have
10,000 users so I'm only getting 10,000
API calls. So tomorrow I'm starting to
get 20,000. So how do you scale it? So
that is also handled by snowflake. So
you can configure different uh number of
nodes or scalability. I want like
minimum nodes, maximum nodes and then
once you start inference in snow
container services, it will
automatically scale it for you according
to the load that is incoming. So you
don't have to worry about those things.
All right. And finally, this is also the
most important part. How do I make
available the result of my AI or ML to
the customers or the users, business
users in a company. So that is a very
important part here. So how do we do
that? You might have seen in every
single presentation today or at least
whatever I attended in this uh
conference, everybody talks about
Streamlit,
right? Anybody knows like Streamlit um
is acquired by Snowflake?
Okay. Okay, only one hand. Yeah. So, we
acquired snow uh streamlit um and now it
is available as a part of snowflake. We
call it cis streamlit in snowflake and
you also have the open source version
still available. Uh but uh inside
snowflake we provide it as an
enterprisegrade level app. We have a
native app framework where you can
easily build your streamllet inside and
share it across um very seamlessly with
all the rback control o and etc handled
by snowflake and then you also have your
open source version which you all are
leveraging I can see that in the most of
the talks.
Now why should I do streaml and
snowflake? We come with a top of
additional advantages when you compare
to your oss version which is fully
managed. We are also working on a
feature where you can run your streamlit
on our container runtime. Now that again
means it's going to be very very
scalable for you. Um you don't have to
worry about the load handling and things
like that. Governance and security. You
don't have to configure out of the box O
or anything who can log into it and
things like that. That will also be
taken care by Snowflake. You can set our
back level permissions. Now it's
completely integrated with Snowflake. So
it is easier to build it. We come with
an interface where you can go in and
start coding your python uh streamllet
and then um on the same window itself
you can see the build that is being
happening for this particular dashboards
in streamllet and also it's integrated
with our notebooks as well. If you code
in notebooks it's also easier for you to
see it there.
Now summary.
So Snowflake provides a very simple coy
platform that helps you to scale every
single part of your ML workload. And in
case of if you want high performance
pre-processing that is where you can
leverage the APIs that we provide out of
the box. Snow park API, snowark pantas
and also snowpark ML APIs to do your
feature engineering and processing. But
you're always free to use Python OSS
version and um bring in your existing
code or write anything that you want and
execute in in Snowflake.
Now going to the distributor training
and tuning, you can leverage your CPUs
and GPUs very easily in Snowflake
without setting any um any of these
libraries. For example, if you want to
use GPU, you have to get started with,
you know, installing CUDA, Kurin and all
those libraries and set it up and get
from the scratch. But in Snowflake, it
all comes out of the box for you. So you
can right away get started.
Seamless deployment and serving. So you
can serve your models on top of CPU, GPU
in whichever way you want. And out of
the box, we will provide the distributed
uh things for you. It will be handled
behind the scenes. For the user, it is
very similar to writing a normal Python
code with any other libraries.
Streamlit, I don't think I have to talk
much about streamllet. Everybody is uh
using streamllet now. So um it will help
you to build all those apps and
prototypes and easier and then share it
with the your wider users to leverage
the results of your AI ML etc. All
right, that is all for the session
today. If you're curious about knowing
more about Python in snowflake, so this
is the QR code that you can scan and
there is a lot of Python recipes that
you can get on that particular GitHub
link which will help you to build all
this uh Python in a very distributed
fashion. All right, that's all for
today. Thank you.
[Applause]