Speed Up Your Python Data Applications By Parallelizing Them With Bodo

Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Unknown (00:13):
Hello and welcome to Podcast Dot in It, the podcast about Python and the people who make it great.
When you're ready to launch your next app or want to try a project you hear about on the show, you'll need somewhere to deploy it. So take a look at our friends over at Linode.
With the launch of their managed Kubernetes platform, it's easy to get started with the next generation of deployment and scaling powered by the battle tested Linode platform, including simple pricing, node balancers, 40 gigabit networking,

(00:38):
dedicated CPU and GPU instances, and worldwide data centers.
Go to python podcast.com/linode,
that's l I n o d e, today and get a $100 credit to try out a Kubernetes cluster of your own. And don't forget to thank them for their continued support of this show.
Your host as usual is Tobias Macy. And today, I'm interviewing Ehsan Totoni about Boto, an inferential compiler for Python that automatically parallelizes your data oriented projects. So Ehsan, can you start by introducing yourself? Hi, Tobias. Thanks for having me. I'm Ehsan Tutuni, cofounder and CTO of Bodo dotai.

(01:15):
And do you remember how you first got introduced to Python?
A long story, I think it's worth digging into.
I was a PhD student at the University of Illinois, Urbana Champaign
and working on
high performance computing
on large supercomputers.
So we had these codes with MPI

(01:35):
and FORTRAN and c plus plus
very low level code, hand optimized,
running on
100 of thousands of cores sometimes.
But you see everywhere, even in,
an HPC oriented
school like University of Illinois, Urbana Champaign,
all the scientists and data scientists

(01:55):
and computational
scientists
were using scripted languages because they are much simpler. It was usually MathLab or Python or sometimes R.
So that's where I got interested in scripting languages in general
and started thinking about
how to
bring
the power of high performance computing and parallel computing

(02:19):
to scripting languages to make it easier
for regular programmers
to take advantage of these compute resources and scale their code and run their code as fast as possible.
So after
getting my PhD, I really wanted to bring this compute power
to programmers

(02:40):
of scripting languages, so I joined
Intel Labs.
We started
trying to optimize Julia
at first.
The promise of Julia
was that it's a scripted language
that is fast and lets you
run code with the speed of c. Simplicity of scripting language, but speed of c.

(03:03):
So we were working with Julia team
at MIT.
We built several packages, compiled technologies
to paralyze Julia, optimize Julia,
but we saw that Python is taking over
and is becoming standard for all kinds of programmers.
So we brought

(03:23):
our compute
technologies to Python instead of Julia.
And it was a
nice surprise for me in terms of how
nice and easy to use Python is
in terms of us developing
compiler technologies in Python and for the user in terms of abstractions.

(03:44):
So we thought some of the ideas that we had in Julia can be applied to Python, and that's what happened.
So in terms of some of those ideas,
the main idea in Julia is the fact that
you
can compile
individual
functions based on the type of their inputs.

(04:05):
So you don't have to compile everything like c
and lose some of the flexibility of dynamic and scripting languages, but
you can compile
individual functions and have different versions of functions based on the type of input that goes to function.
So the first time
you call the function, the system compiles it, puts it in some table, and it reuses it afterwards,

(04:31):
as this function is being called. So this is sort of the
architecture
of making the dynamic language
compilable.
And we saw that, basically, in Python,
the Numba project
does that for us.
And that's how we got started in Python and brought

(04:51):
ideas from Julia to Python.
And I believe this is the
right way to have
the power of a dynamic scripting language, but at the same time, take advantage of compilation and optimization
and get the speed of c, essentially,
for your programs.
And so in terms of
the

(05:12):
opportunities
for
the sort of utility of Bodo for Python applications. I'm wondering what are the
motivating factors
for using something like Bodo? And what are some of the elements of the Python language and runtime that cause it to be slow for these data heavy workloads?
What Bodo is maybe I can start by introducing Bodo. Bodo is a new inferential

(05:37):
compiler technology.
What it means is that
Modo can infer the structure of the program
and
convert the function to an optimized and paralyzed version.
As an HPC expert,
we wrote the code for you in MPI,
c plus plus,

(05:58):
but it's done automatically
and, transparently in real time for you.
So this was the result of 4 years of research
at Intel Labs and Carnegie Mellon University where I was.
And when the research was successful,
I joined forces with my cofounder who is an experienced business executive based on mastery, and started Bodo in mid 2019.

(06:25):
So
Bodo
is the first
compiler system that can automatically parallelize code. This has been the holy
grail of
1 of the holy grails of computer science
for decades
that we have achieved for the first time, and Python has made it possible that I will

(06:47):
explain in a minute. But in terms of
what we are building at Bodo is a platform
for data engineers and data experts
to scale their code
and run it at HPC speed
on thousands of cores. Doesn't matter. Scales linearly from a single core on your laptop

(07:08):
to 8 cores of your laptop, then a single cloud instance, and thousands of cores. We have results from enterprise customers up to 10, 000 cores, and we have published some of that. So that's what's Bodo. We think we can improve
very challenging data problems like ETL data prep
and speed them up by orders of magnitude, and increase their reliability

(07:31):
and predictability, and lower the cost all at the same time with this technology.
So
the motivating factor for Bodo was that we saw
data workloads are growing. Big data emerged
basically
in 2000,
and there was no good way to scale code to big data because

(07:53):
Moore's law slowed down in terms of sequential performance of CPUs. You had to use more CPUs
to process larger datasets.
And,
the industry went towards
libraries
like MapReduce, then Hadoop, then Spark,
where these
libraries
provide some high level APIs,

(08:15):
started with MapReduce. Now there are data frame APIs
that take your code
and break it up into tasks,
go across the cluster, run those tasks, and come back. So the driver program is a single process.
It's not actually parallelized,
but compute power is delivered through these tasks.

(08:38):
However,
we saw that
compared to native parallel code with things like MPI that has been standard of parallel competing for decades,
these systems like Spark are orders of magnitude
inefficient.
So some MPI code takes a few seconds, Spark can take hours.
And this was a huge shock for us, and that's what motivated

(09:02):
me to create Vodle
and replace all these inefficiencies of these big data frameworks.
But back to Python
and
how Vodou
delivers performance for Python.
Let's take a step back. Why is Python slow to begin with? So in c,
the program has all the data types

(09:25):
that translate to machine code. So you have integers, you have floating point values, classes
that are made up of those things, and the compiler knows
all the details of those data types
to take the program and generate a binary
that runs on bare metal. In Python,

(09:46):
we have an interpreter,
tip typically, c Python interpreter
that takes the code, and 1 by 1,
it runs the instructions.
It means that for every variable, it has to figure out the data type of that variable
and call the appropriate method of that variable.
So if you have, like, addition of 2 scalars,

(10:08):
integers, in worst case, it can become
check for data type of that value,
go to
the method for that
addition, call that method a lot of overhead.
And that's why sometimes
Python is
100 of times
slower than something like c.

(10:29):
However,
we can fix that. And numbers show that for certain class of applications, we can
convert the Python code
to something like c code under the hood and compile with LLVM to a binary
because
we can infer the data types
of the program in some

(10:51):
situations
and compile it as if the code was c. So you don't have all this interpreter
overhead.
The other approach that has
existed is, what if we create high level APIs
in Pandas and NumPy,
And instead of
running them on top of the interpreter,

(11:14):
when you call that API, some c libraries
actually
call them. That's how the code is run. So you have
array
let's say, 2 NumPy arrays. You add them. It doesn't happen through a loop in the interpreter.
The loop is actually a native code loop. Because if you do the loop thing in the interpreter,

(11:35):
the interpreter has a representation of the program, very high levels. It's like white code,
stack based
white code, pop up the instruction from the stack, figure out what it is, what are the variables, run that. If you do a loop like that, it's very slow. So that was 1 of the approaches that, actually,
in addition to bringing performance,

(11:57):
it brings simplicity as well because
there is motivation for building these high level APIs that are simpler for the program. Everyone knows that Pandas
is very simple for dealing with data. So given
this
status of the Python
ecosystem, which is
there are some ways to compile Python. It's not perfect, but with Numba,

(12:21):
there is some infrastructure.
And we have high level APIs
that are more understandable for the programmer,
it turns out those APIs are more understandable for the compiler as well. And that's what made Bodo possible because
Bodo
can take those APIs,

(12:42):
understand their semantics. Let's say you're joining 2 data frames
and
figure out
if the data structures can be distributed,
do the distribution for you,
and do the right communication across processors to make the computation possible
and
deliver it to you. So you don't have to worry about those aspects, and you don't have to write low level code to make that happen.

(13:07):
So Python
and this approach of using high level APIs is very instrumental
to enabling Bodo and this new approach of
scaling
computation and data intensive applications.
In terms of the sort of scalability
of Python data intensive operations, there are a few other systems that have started to gain popularity, most notably Dask and Ray and some other systems that layer on top of those or have similar approaches to horizontally scaling out the computation and having

(13:41):
largely compatible APIs to some of the bigger packages that are used in the system, in particular, NumPy and Pandas.
And I'm wondering what are the sort of deciding factors where somebody might go with Dask or Ray as compared to the use cases that Bodo is uniquely suited for?
Systems like Dask and Ray

(14:01):
are
very similar to Spark at the architectural level and different than Spark in terms of use cases
and some of the actual implementation. So,
basically, the idea behind
all the different frameworks out there, as far as I know, for big data is that
the program

(14:21):
runs on the driver process,
and there are some framework APIs
that are backed by a the Suites system library,
which is you know, you call this
data frame dot
add or data frame dot join.
Some library figures out

(14:42):
some tasks that need to be done, sends those tasks across the cluster,
and comes back the driver.
So this approach
is a task scheduling
job scheduling type approach
that we believe is not true parallel computing.
So if you give the same code to a parallel computing expert,

(15:05):
what would they do? They would write code in NPI
where
the program
actually runs
on all the processes.
There are no tasks like that. We have processes.
There is no driver. There is no scheduler.
So the MPI expert would divide up the data

(15:28):
across
the processes
and add communication
code
for
the parallel portion so processes can talk to each other and figure out what data structures should be replicated across these processes, the temporary small data structures,
versus what is the big data that needs to be distributed.

(15:51):
So that's what the PEL computing expert would do.
Bodo
automates that process
and gives you the true high performance computing and parallel computing performance and scalability.
These other frameworks, we believe, are trying to approximate
parallel computing
using distributed computing.

(16:13):
Distributed computing is not fit for data workloads. For distributed computing,
there is the assumption
that
the tasks are fully independent.
You have a web server or web application.
So the tasks of different people
coming to your website is independent.
That's 1 assumption.

(16:34):
The other assumption is
the system components
are heterogeneous.
You have a mobile device or a laptop, and then you have a server in the back end doing the work of the website.
And your network is slow and unreliable
through the Internet with TCP.
All these assumptions

(16:54):
are baked into these distributor libraries,
and
these are all wrong for parallel computing.
In parallel computing,
the
compute tasks have dependencies,
so you can't fire them off and forget about them.
That's 1 assumption.
The other wrong assumption is heterogeneity.

(17:16):
In parallel computing,
typically, things are in bulk synchronous manner
where
at the same time, all processors are doing the same thing. You don't have a set of processors doing something. You as I said, doing something else. It's usually
1 thing in 1 computation. You are paralyzing 1 algorithm.

(17:36):
So the compute is homogeneous,
and the hardware is homogeneous as well. You have a cluster in the cloud,
and all the cores are
some sort of CPU cores that are the same. You typically want your cluster to be homogeneous.
So that heterogeneity assumption
is wrong as well.

(17:56):
The other assumption that's wrong is network. In the cloud,
you don't have to use
TCP. You don't have to assume you're running over the Internet
and pay the overhead.
Even today,
on AWS,
you you have access to better network protocols
with things like Elastic Fabric Adapter,
EFA,

(18:17):
of AWS
that is much faster than running TCP code on AWS.
And on Azure, you have InfiniBand,
which is, you know, high performance network protocol
and has capabilities
like processes can actually write to into each other's memory.
So
the performance profile of a truly parallel computing and high performance computing, like, solution like Bodo

(18:44):
is very different than Spark, TaskRape,
all of these distributed task scheduling libraries.
And we believe that parallel computing has existed for decades.
There is a lot of research. There's a lot of development,
and we should reuse
what what has come
before, and we should stand on the shoulders of giants.

(19:06):
More specifically,
we believe we should use MPI for data workloads
because
MPI has all the primitives that you would need.
Let me share an example.
A key
bottleneck of data workloads is shuffling data,
where you're, let's say, joining 2 tables,
all the keys with the same value

(19:28):
have to be on the same process for join to happen
algorithmically.
So
in these task scheduling frameworks,
they fire off these tasks, and the tasks are asynchronous,
and they can't communicate with each other. So they
use workarounds.
And and main workaround is writing some shuffle files in case of SPA.

(19:51):
And they have spent
many years optimizing that. So you write shuffle files through tasks, and some other tasks come read the files and merge them together to achieve your shuffle.
However, this is a well known
parallel computing pattern,
and you can look
at parallel computing kind of resources and research papers from eighties

(20:14):
describing
how this should be done in parallel.
And the operation is called all to all communication.
So in the MPI world,
because the tasks there are no tasks. These are processes,
and they are working a bulk synchronous manner. They go to this all to all collective
operation,
and processes exchange messages in an optimal way. So without

(20:39):
writing files, without crashing JVM, without crashing your Linux, hitting file descriptor limits, all those issues,
communication happens
much faster
than
these frameworks, and we believe this is the right way. And by the way, there has been a lot of research, a lot of different
to do this more efficiently, and we are just reusing it.

(21:01):
And we are not reinventing the wheel. But back to DASK and Ray more specifically,
we don't
believe the use cases are similar.
So Spark and Bodo are built for large data engineering ETL,
data prep tasks,
but DAS can already, in our experience,
cannot even perform those tasks. And we have benchmarks

(21:23):
that we will be sharing where
scaling them beyond beyond a single node is not even practical
for these type of data heavy workloads with large joins and things like that. The difference is 100 of times
if they work. So both are 100 of times faster.
In terms of the software patterns that are common in these

(21:46):
data processing workflows,
what are some of the elements that contribute to the sort of scaling complexity and the algorithmic slowdowns that you're able to optimize in Boto?
So
performance,
I usually divide it in 2
orthogonal dimensions, what's related. 1 dimension is sequential

(22:06):
performance.
So the compute that's happening on
each core,
how fast is it? The other dimension is parallel performance,
which is how is communication happening across processors,
and is it done efficient? Is the code scaling well with linear scalability?
On the sequential

(22:27):
performance side,
a key
pattern that slows down
data workloads is whenever you have custom code,
and it's for real applications, very often,
you have to process some string value,
see
where it fits, return some code number out of it. Just 1

(22:49):
example.
So these sort of things
typically are written in terms of user defined functions
in Pandas, let's say, data frame dot apply
or some SQL user defined function.
In these cases,
there is no library
to run the code for you because it's custom code. It can't into some c library,

(23:10):
and you're
limited by the interpreter
speed, which is very slow. And every time for every row,
the interpreter is calling your function, you're paying a function call overhead and the overhead of running the function. All those compiler capability
to translate this to native code
can give you 100 of time speed up on these sort

(23:32):
of complex custom codes. So that's 1 key pattern.
There are some other patterns such as expressions.
Now if you run your expressions
in regular NumPy or pandas,
the code is essentially calling
libraries after library method after library method
on these data frames and arrays.

(23:55):
And, you know, you load data, store something, load data, store something, you lose cache efficiency of your processor,
and Bodo is able to fuse these
and optimize them. So that's the other pattern to improve cache efficiency. And in general, computations on strings, date, time, things like that,
Bodo is able to optimize

(24:16):
the computation, eliminate the interpreter overhead,
restructure the the computation
in a good way for cache efficiency,
avoid object overheads of Python, and things like that. So that's the sequential
performance dimension.
The other dimension
is parallel performance.
In data workloads,

(24:38):
a key pattern is shuffling data
whenever you have join,
group by, any of these operations.
And this is where a lot of the problems happen.
The application slows down, something fails.
Bodo
is, as I mentioned, using MPI, very robust for these

(24:58):
cases.
It can scale using MPI very well,
and you don't have any of these issues. For example,
at a large enterprise company,
the IT department
really liked using TerraSource Benchmark, which the source
large dataset random dataset.

(25:19):
And this
small benchmark
doesn't have
any
complex code for Bodo to optimize. It's just 1 library called load parquet files called sort, data frame that sort values.
But Boldo was 8 times faster than their optimized
Spark implementation, highly tuned Spark implementation

(25:41):
on 4 to 500
cores on AWS
infrastructure
they had
because
of this parallel aspect and the fact that Bodo uses MPI
to scale computation. So no tuning necessary,
scaled to 4 to 500 cores. They literally changed the number of cores in their launch command

(26:02):
from 1 core to 1, 000 to 4 to 500
without tuning it. Just because when you do parallel computing the right way,
you scale easily and much more efficiently,
And you can take advantage of better network protocols like EFA to scale. So for both dimensions of sequential performance and parallel performance,

(26:25):
although addresses the
challenges of data processing
data processing code. By the way,
loading data in parallel is also
a major problem, although it's storage agnostic and can load data from all sorts of storage systems such as parquet on s 3 or data warehouses like Snowflake.

(26:45):
So when you have a parallel computing infrastructure
like Bodo, it's easier to optimize
load of data. Another pain point for large scale data workloads that need to load terabytes of data at the same time. And now digging more into Boto itself and in particular the compiler aspects of it, I'm wondering what are some of the

(27:08):
sort of challenges
that are
sort of inherent in being able to
parse the sort of semantics and structure of Python code given its dynamic capabilities
and turn it into something that you're able to statically compile and optimize?
So the main challenge is data types.

(27:29):
For any compiler solution,
you have to have data types
to
do anything.
Without data types, the compiler can't do anything.
So
that's the key challenge.
And in this case,
the main data structures
for data processing workloads is data frames.

(27:50):
And for data frames,
the data type of each column and the names of the columns are also part of the data type.
So for every operation,
Modo has to know what the data frame input is and what the data frame output is. So we have a lot of compiler techniques
to figure out the data types. That's

(28:11):
1 of our daily challenges
that, you know, in Python and Pandas is not designed to be compiled
necessarily.
There are these patterns by the users. For example, very common thing. You have a data frame. You assign a new column to it. You add a new column to it. That's changing the data type, and it can break the compiler system. So how can we avoid that? So

(28:34):
we automate all of these patterns. However,
there are cases
where
we can't do it. The data type based on some
condition in the program changes.
And in those cases,
we
we have to throw the right error
for the user to know what's going on and point exactly where the problem happens

(28:58):
and where they need to slightly change their code for the compiler to work.
So throwing the right errors with the good
user experience
is 1 of the challenges we are focused on on a daily basis to make it easier for
Python developers to get up to speed with Godot
and understand this g technology and take advantage of it. For things like the Pandas data types, I'm wondering how things like the extension arrays capability that was added somewhat recently impacts your ability to be able to

(29:30):
understand the data type, particularly if the data type is, like, a complex object, for instance, with the GeoPandas plugin that adds support for
geographical information like, you know, shape files or plots, you know, lat and long coordinates and being able to optimize for that in something that might be using that extension to a pandas data frame?

(29:52):
Odo has its own internal representation
of
various types of data
and these arrays.
So we have,
obviously, numerical
arrays,
NumPy like arrays.
We have
more complex arrays like nested arrays, where each element of the column is an array or list by itself.

(30:15):
We have map arrays where each element is like a dictionary
and things like that. So Bodo has optimized representation of these arrays.
And these new array types of pandas
are actually making our job easier.
So
a recent 1,
you mentioned some of the
more complex ones

(30:36):
that I also mentioned some examples,
but a recent 1 was just nullable integer arrays. It made our job so much easier
because
in pandas,
if you have an integer array and it's backed by a NumPy array,
when you have
nulls in your column, you introduce nulls,

(30:57):
it becomes all of a sudden float,
which has other problems like numerical accuracy
and all the issues.
And Pandas
would convert to float and assign
NAND as the
sentinel value as indicator of null.
So this was causing a lot of issues for Bodo, this type change throughout computation.

(31:19):
And you can't have a single optimized data structure for Notable integer array solve that problem.
So this advancement of pandas towards that direction
of having proper data structures
and not just relying on NumPy,
actually help Bodo.
But for any of these, we have to add internal support to Bodo to understand these data structures,

(31:45):
and that's our
usual
kind of expansion of Bodo's functionality
based on Pandas advancement.
Stepping through more of the actual compilation process, I'm wondering if you can talk about the sort of
parsing and the ways that you understand some of the semantics of the code to know
when you're able to parallelize certain operations and when they need to be executed sequentially and

(32:11):
how the sort of output of the compilation step interacts with the original Python code?
Yeah. Definitely.
The compilation process is actually very interesting.
So
when
a JIT function, Bodo JIT function is called starting from the very beginning,
So the function is being called with some arguments.

(32:33):
1st, it goes to Numba. So Numba takes the function
that Python's interpreter has parsed.
It's in bytecode format,
a stack based bytecode that I mentioned.
So Nava takes that and converts it to an internal intermediate
representation
that corresponds directly to that bytecode.

(32:56):
And Numba has facilities for type inference and various things that we take advantage of. So Bodo takes that,
and
it goes through the program. I'm overly simplifying it, but it looks at all the operations that it understands.
So it understands
a large portion of pandas,

(33:16):
the common APIs used for data processing.
It understands
a lot of NumPy, the data processing portion of NumPy. It understands
scikit learn,
and our documentation has a reference for all of these.
So for these operations,
Godot knows how to parallelize them. In the next stages of the pipeline,

(33:39):
the compiler,
using these parallelization
semantics,
has an algorithm
to figure out how the program has to be structured,
what data structures have to be distributed across processors,
what data structures have to be replicated, and the corresponding
computations
that need to be distributed.

(34:00):
So Bodo, based on those semantics of Pandas APIs,
can do that automatically,
and then Bodo translates the code to a distributed code, which is
process
0
on the chunk of the data, it will load only that portion of the data from file, do the computation on that portion, and so on and so forth. And there are some data structures that need to be replicated across processors, smaller data structures,

(34:27):
and Bodo will insert
the communication
necessary
for
the computation to happen. For example, if you have a join of data frames, auto inserts the shuffle steps and all the other things around join
so that join is done correctly.
So
auto
finishes this
distributed parallel computing transformation,

(34:50):
and then goes back to Namba.
Numba translates this representation
into
LLVM
IR, and LLVM translates
the representation
into
actual assembly and binary.
So given the Python function, an optimized binary goes out. The way

(35:11):
this new function is called
is that
the Python arguments
are
so called unboxed
into their native representation,
it's number of framework terminology,
the native function is called
Python is great that it can interact with native code very easily

(35:33):
and c code.
The function finishes. The return of the function is so called boxed into
Python object and goes to regular Python.
So the workflow
of this
JET
optimized code is very seamless
in terms of interaction with regular Python.

(35:54):
So the goal is to provide Python experience,
but speed up high performance computing
in this process.
You mentioned that Python is able to very easily interact with
c libraries and other native code. And I'm wondering
how packages such as NumPy and Pandas that do rely on those

(36:14):
native compiled extensions
to the Python library, how Bodo is able to interact with those types of systems or if it is able to sort of simply ignore them and just focus on the actual Python bytecode and all of the native binaries are treated the same by the are treated the same by the Python interpreter?
So Bodo looks at the
APIs of Pandas and NumPy

(36:35):
and makes a decision in terms of how to implement them. We may ignore the implementation
of pandas and NumPy altogether
because both can do something more optimized and paralyzed.
A lot of the times, that's what happens.
But sometimes,
there is an optimized implementation
of something complicated
in c library somewhere,

(36:56):
and we just call that from Bodo. We can call any Python or c code inside this JIT context.
Calling into Python has a little bit of boxing and unboxing
overhead that I mentioned,
but a lot of the times, it can be negligible.
This is a great advantage for us because
a lot of the operations,

(37:18):
things like date and time processing
are already done in native code and c code in pandas,
and we just take advantage. We don't need to
reinvent the wheel.
So interaction with those extensions easy. And anytime there's a custom
piece of code that Bodo doesn't understand, the user can just come back to regular Python and run that. And this generality

(37:43):
helps
to deal with all the complicated situations that can happen in real
real world applications.
And so for people who are using Bodo, can you talk through the overall workflow of starting with a Python project that might already be fully functional, but it maybe doesn't run as fast as they want it to. And then applying Boto to it and being able to sort of determine what are the

(38:07):
sort of levels of speed up that it's getting and the overall process of debugging any errors that might come up as they're either
changing the program or in the process of applying Boto to it and challenges that come about because of the increased parallelism?
So, typically, the processes a data engineer
prototypes

(38:28):
the workload in regular Python,
and there's no need for using both of our initial development.
So all the testing of
actual
computation can happen in regular Python tools.
Other frameworks
don't provide that because they can't handle
native APIs.

(38:48):
But afterwards,
for using Bodo,
typically,
they have to look at
where the actual computation happens and where the large data structures are.
We factor those pieces of the
application
into functions
that can be jitted.
So the Pandas, NumPy

(39:11):
work on large data structures, terabytes of data,
are putting functions and, Bodo decorator
is put on top of those. And sometimes,
there may be some typing issues
that need to be refactored
for the code to work in Bodo. But the Bodo code,
you can actually comment out the Jits, decorate, or even disable it from environment variable. It's regular Python code. Even after the factoring for Jits, we can still test it in regular Python and make sure it works.

(39:43):
From that point on,
once it works on a single core with JIT,
you can test it on 2 cores and 3 cores and 4 cores.
Typically, there are no issues. Once it works, you can scale it on any number of processors without any problem. So we haven't had
too many issues of the parallelization per se. It's the first step of

(40:06):
converting code to JIT, needs attention and
work, but afterwards it's very easy because Bodo
takes care of the rest.
Sometimes,
there are structural issues in the program
that
the parallel APIs
are not used
properly
by the programmer,

(40:27):
Bodo
automatically
figures those situations out and throws
warnings.
This function could not be paralyzed.
Look at these diagnostics
of why
the operations you are using are not paralyzable,
and it points exactly the line. So we have extensive diagnostics for parallelization

(40:48):
capability.
Afterwards,
the code works. It's parallel.
There are no issues,
and the deployment
is the standard
MPI deployment.
And there are so many resources on MPI
that if there were issues, typically, with a simple
search, you can find out various resources for MPI.

(41:11):
So that's the process of using Vodou. And, basically, it's about the Jits aspect, jitting the code, and making sure there are no parallel issues based on compiler
messages.
Afterwards,
it's done. There is no
other steps that are necessary.
And for debugging,
you have access to print.
And so in terms of logs and printing and all of those things, but you also have access to all the tools

(41:38):
in the Python world and the MPI world for debugging at the same time,
If necessary, we haven't
seen much of parallel issues. It's about making the code
work sequentially
on a single core, from that point on, it
almost always works perfectly on any number of processors.

(42:00):
You've mentioned a few times the use of Bodo for data engineering workflows, and I'm wondering what are the opportunities for being able to apply it to machine learning or deep learning projects using frameworks such as PyTorch or TensorFlow or MXNet.
So our focus
is on
large scale data processing,
ETL,

(42:20):
Dataprep featurization,
and Bodo is often used for machine learning applications in general.
The last step of the pipeline is obviously
training a machine learning model,
and we have support for scikit learn, for
more classical machine learning algorithms.

(42:40):
But for deep learning,
we don't
compile
the APIs of deep learning frameworks because they already do that,
and they they can optimize for the GPUs. We integrate with them.
So although because it's MPI based, it works with Harvard
natively,
and we have support for

(43:02):
transferring the data from CPU after you do the processing in Bodo,
transfer the data to GPUs,
and manage these GPUs,
but the rest of the compute
is the same. It's your usual PyTorch and TensorFlow code. So that's how
the machine learning kind of deep learning pipelines
are built with Bodo today.

(43:24):
It's through integration with existing tools, and Bodo takes care of scaling
beyond the single node and single GPU for you using Carver.
In terms of being able to integrate with something like a workflow orchestrator such as Daxter or Airflow, what is the process for being able to use 1 of those tools to be able to drive the execution of Boto or a program that was compiled with Bodo and you're then using the Jitd output?

(43:54):
So Bodo's workflow
is standard Python,
and any
environment that supports Python Bodo is installed as a standard Python package. So any environment that supports Python
can run Bodo.
The difference of Bodo is that instead of a single Python process, you're launching

(44:15):
many more, potentially 1, 000. And you can
launch it through command line with MPI exec. So instead of saying Python this file, you say MPI exec this file, and MPI just needs a host
list to make that happen. So we have been able to
launch all the jobs in any environment on premises, in the cloud, standard tools, nonstandard

(44:38):
tools because the requirements are so simple. And it's the same for the workflow
management tools
as well. All of them support Python or just running a command line, and that's the simplest way
to just launch Bodo in any environment.
But there are cases where

(44:59):
you want more integration
where
there are notebooks
and there are interactive jobs and things like that,
and those become more interesting,
we are working with the open source community and developing Py Python parallel
for those cases, more interactive
cluster cases

(45:20):
that just launching a command line may not be enough,
and that's going very well. I encourage,
Python developers to check out the IPY Parallel. It's a very cool project
and makes PEL notebooks
very easy and interesting.
In terms of your experience of working with Bodo and seeing people apply it to their different data engineering challenges, what are some of the most interesting or innovative or unexpected ways that you've seen it applied?

(45:47):
So for Bodo, something that was very interesting and unexpected to me
was that
given this
new
powerful
tool,
data engineers
basically change the
architecture
and just use a different
stack altogether to solve the problem.

(46:10):
So
I saw that a lot
of SQL based systems are being replaced with Bodo or graph
databases
or
even change the storage.
So if you can run Python at scale on parquet files,
you may not want
a SQL based system.

(46:30):
And if your Python runs in real time and very fast, you may not want a Presto
installation and deployment.
These were very interesting and unexpected to me,
and I didn't see
Bodo being used for
such diverse set of applications and changing
the stack so much just because Bodo is so much simpler and so much faster. I didn't

(46:54):
know you can
decide to change a different type of storage
for these kind of use cases that need real time or use Goto for these complex visualization
pipelines
or business intelligence and those sort of workflows.
These were unexpected to me how much impact Bodo could have
on the compute environment.

(47:16):
And in your experience of building the Bodo project and working with the Python community to understand its use cases and capabilities. What are some of the most interesting or unexpected or challenging lessons that you've learned in the process?
So a key challenge
for us
introducing Bodo as a new
platform

(47:36):
to the world is that we are
making Python faster.
Right? And whenever we say the word Python,
everybody
thinks about
slow computing on small data.
This has been very challenging to explain that we bring simplicity of Python,
but

(47:57):
performance of high performance computing,
and we are here to solve bigger data engineering problems.
We are not here to solve small data problems
of Python on your laptop just, you know, per se. You could use Bodo to make data science workloads on laptop faster,
but we are here mainly to solve the bigger challenges

(48:19):
that
are causing a lot of issues for you on terabytes of data and 1, 000, of course.
So this understanding
that Python
can be used for large problems,
and Python can be used for production
workflows,
that has been
challenging, and I didn't expect it to be this difficult

(48:41):
to convince
various personas that Python
should be a first class
citizen and default
language for production. You don't need to rewrite your code in Scala and
see things like that.
It's difficult
to
convince barriers, especially IT leaders,

(49:03):
that you shouldn't make your programmers who love Python
rewrite the code in Java and Scala,
and Python can deliver the results that you would expect.
Python
is very productive,
very simple to use.
The challenge is performance, and we have solved it. And there is a challenge of data types and robustness,

(49:23):
which we have solved as well because we compile the code and check the types
and throw the right compiler errors
so that we don't have failures in production. So we have solved that problem as well. So we think Python should be the language for production.
For people who are interested
in Boto and they are looking to be able to optimize and accelerate their Python projects, what are the cases where Bodo is the wrong choice and somebody might need to actually go down the path

(49:53):
of writing an optimized native extension for a, you know, a tight loop for some computation heavy aspect or, you know, explore some of these scale out architectures
such as DASK or Ray?
Our focus
is on data intensive
workloads in data engineering,
ETL, data prep, and those sort of things.

(50:14):
And we think Vodle is the
best solution for those.
If the workload doesn't fit the
typical
data
problems,
for example,
it's a
web application,
web server. It clearly doesn't fit Bodo because it's not a data problem.

(50:35):
But sometimes,
there are things in between.
Things like scientific workloads
can look similar to data problems, and they may not be data problems. And we don't target
some of those scientific workloads that are not data problems. So anything that's working on a lot of data,
things like pandas,

(50:55):
data frames,
billions of rows,
ETL,
join of tables, processing tables, processing rows,
those are great for Bodo.
If
it's truly a distributed computing problem
that there are independent things that need to be done
across the system,
and it's not a parallel computing problem.

(51:18):
That's where
Dask and Ray are
the preferred solution,
and Bodo is not the right answer.
But whenever it's
large scale compute and data, terabytes of data, that's where Bodo
shines. And as you continue to iterate on the Bodo project and optimize its capabilities and explore how to expand its capabilities and compatibility. What are some of the things you have planned for the near to medium term?

(51:45):
So our goal is to bring
compute power
of high performance computing
to as
many programmers as possible.
So we want to lower the barrier
to this kind of compute
to,
regular Python programmers,
and help them become data engineers, essentially,

(52:07):
with as
little learning
and barriers as possible. So in the short term, we are working on improving the user experience of Bodo,
basically, automatically
compiling as much as possible without
changes in the code
where it's actually practical.

(52:27):
And when it's not possible to compile
throw the right errors
and have the right documentation
to explain
what could be done in the program. So
the
programmer experience
is
our number 1 focus in the short term. We are also
developing Motto platform

(52:48):
to be as complete
as possible.
We are fully storage agnostic,
so
we support
parquet on object storage like s 3 or ADLS.
We support various data warehouses like Snowflake,
and, we want to make this storage support as seamless and as efficient and scalable as possible.

(53:11):
But we are also building on managed cloud platform on AWS and Azure
to have a lot of the right features
in terms of managing clusters,
notebooks,
managing users.
And we have a new
notebook experience that I'm very excited about. In a few months, it will be out, and

(53:33):
I'm very excited to see
what kind of applications
and what kind of users this new experience can really enable
because it's a much lower barrier
for programmers than before.
I believe that
any
Python programmer can become a data engineer with both or with

(53:54):
much lower
amount of learning and barriers
than something like Spark.
And they can basically,
hopefully, double their salary
without learning all of these complex big data frameworks.
So that's that's our goal. And are there any other aspects of the work that you're doing on Bodo or the overall space of using Python for data engineering and accelerating

(54:18):
its computational speed that we didn't discuss yet that you'd like to cover before we close out the show? I think
the biggest
aspect
is
the mindset
of various
members of the data team.
So
at a high level, IT leaders
need to
pay attention to the needs of the developers

(54:39):
because IT leaders care about time to business value. And if we explain that
Python should be the default language to reduce that time,
that's where the value for IT leaders is. So that's 1 aspect
to focus on, and
the programmers
using Bodo, they can really argue that
we can

(55:00):
reduce
development time and time to production.
That's 1 key aspect
to focus on. The other aspect is the cost of infrastructure
for organizations.
If some computation is
10 times faster, we have had a 100 times, but let's say 10 times faster with Bozzo,
it means that your cloud bill is 90% lower.

(55:22):
So that's a key aspect that IT leaders are focused on, but I think
developers have to be focused on as well because the cost
grows significantly. Datasets are growing on a daily basis.
So we think
this change of mindset in terms of time to value

(55:42):
and the cost in the infrastructure
can really change the landscape of analytics, and Bodo is the catalyst
to make this transformation
happen
and democratize
data
and machine learning for all enterprises.
Alright. Well, for anybody who wants to get in touch with you and follow along with the work that you're doing, I'll have you add your preferred contact information to the show notes. And so with that, I'll move us into the picks. This week, I'm going to choose, recently started working on,

(56:13):
experiment
playing around with building paracord
bracelets and sort of
different interesting combinations of survival tools to put in there and just a fun thing to do away from the computer. So been enjoying that. Definitely worth taking a look at for just a fun little quick project you can do. So with that, I'll pass it to you, Ehsan. Do you have any picks this week? The NBA season is starting soon.

(56:36):
So I'm
addicted to basketball and
looking forward to that.
That's something I'm looking forward to.
Alright. Well, thank you very much for taking the time today to join me and share the work that you're doing at Bodo. It's definitely very interesting project and an interesting product, and I'm excited to see the
capabilities that it unlocks for the Python community and ecosystem and excited to see how it grows along with the sort of seemingly never ending growth of the Python language. So thank you for all the time and effort you've put into that, and I hope you enjoy the rest of your day. Thank you, Tobias. It was great speaking with you, and I encourage the Python community

(57:12):
to check out our community version and let us know what they think. We are listening, and we would like to know where this project should go to address their needs.
Thank you for listening. Don't forget to check out our other show, the Data Engineering Podcast at dataengineeringpodcast.com
for the latest on modern data management.

(57:34):
And visit the site at pythonpodcast.com
to subscribe to the show, sign up for the mailing list, and read the show notes.
And if you've learned something or tried out a project from the show, then tell us about it. Email host@podcastinit.com
with your story.
To help other people find the show, please leave a review on Itunes and tell your friends and coworkers.

All Episodes

Summary

Announcements

Interview

Keep In Touch

Picks

Closing Announcements

Episode Transcript

Popular Podcasts

Stuff You Should Know

Dateline NBC

The Clay Travis and Buck Sexton Show

.css-15opob5{left:0;position:absolute;top:0.8rem;} All Episodes

.css-14f5ked{margin:0;word-break:break-word;display:-webkit-box;-webkit-box-orient:vertical;box-orient:vertical;-webkit-line-clamp:2;overflow:hidden;}Speed Up Your Python Data Applications By Parallelizing Them With Bodo