December 18, 2024 • 39 mins
An evolution of asynchronous programming techniques to boost your Shiny apps, how a puzzle from over a century ago could tip the scales for mathematics algorithms solvable in R, and solving the recent 2024 Advent of Code puzzles with data.table.
Episode Links
Supporting the show
Episode Links
- This week's curator: Colin Fay - @colinfay@fosstodon.org [@_ColinFay]](https://twitter.com/_ColinFay) (X/Twitter) @colinfay.bsky.social
- Parallel and Asynchronous Programming in Shiny with future, promise, future_promise, and ExtendedTask
- Writing Signed Trinary: or, Back To the Four Weights Problem
- Advent of Code with data.table: Week One
- Entire issue available at rweekly.org/2024-W51
- Futureverse - A Unifying Parallelization Framework in R for Everyone https://www.futureverse.org/
- Non-blocking operations with ExtendedTask https://shiny.posit.co/r/articles/improve/nonblocking/
- Asynchronous Shiny apps with crew and mirai https://wlandau.github.io/crew/articles/shiny.html
- Bachet’s Four Weights Problem https://rworks.dev/posts/four-weights/
- Advent of Code 2024 https://adventofcode.com/
- Lime Bikes Summary Dashboard https://jokasan.github.io/Lime_Dash/
Supporting the show
- Use the contact page at https://serve.podhome.fm/custompage/r-weekly-highlights/contact to send us your feedback
- R-Weekly Highlights on the Podcastindex.org - You can send a boost into the show directly in the Podcast Index. First, top-up with Alby, and then head over to the R-Weekly Highlights podcast entry on the index.
- A new way to think about value: https://value4value.info
- Get in touch with us on social media
- Eric Nantz: @rpodcast@podcastindex.social (Mastodon), @rpodcast.bsky.social (BlueSky) and @theRcast (X/Twitter)
- Mike Thomas: @mike_thomas@fosstodon.org (Mastodon), @mike-thomas.bsky.social (BlueSky), and @mike_ketchbrook (X/Twitter)
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
(00:02):
Hello, friends. We are back with episode 190 of the Our Weekly Highlights podcast.
This is the weekly podcast where we talk about the terrific highlights that have been shared in this week's our weekly issue available at ourweekly.org.
My name is Eric Nantz, and, unfortunately, I am fine solo again this week because my awesome co host, Mike Thomas, is
(00:24):
currently a knee deep and finishing up some really high priority work projects.
I know from experience that the end of the year can be quite intensive to get these things wrapped up because anything can happen. Right? So
I'll be holding the fort down today, but as usual,
I never do this alone in terms of the project itself because
(00:45):
this week's issue was curated by the esteemed Colin Faye,
who had tremendous help from our fellow Aroki team members and contributors like you around the world with your awesome pull requests and suggestions.
And one little bit of admin note before we go on further,
I am now recording this
on my media box that my media server, I should say,
(01:09):
that is just as of yesterday of this recording,
been refreshed from the ground up to be powered by Nix OS instead of the standard Ubuntu Linux
distribution.
So, hopefully,
you won't notice a thing in terms of listening to this, but I have been knee deep in getting things reconfigured. And I think everything's working, but, hopefully, this episode goes without a hitch.
(01:33):
Speaking of our curator of the week, the esteemed Colin Faye himself is leading off our highlights because he has authored a terrific blog post on the think our blog
on a very, very important concept to think about
with not only within
the computation
of various data analysis task in R itself,
(01:56):
but how it relates to you as a Shiny developer
to give your users the best experience and the best response times.
This is a very difficult task, and he leads off the post with,
you know, an a very humorous saying about the
the 2 or 3 things that are hardest to do in computer science.
(02:17):
And, yes, a couple of them do relate to performance, and the one we're gonna be focusing on today
is asynchronous
computing.
And, yes, there is a lot of ways we can tackle this. But first,
Colin sets the stage, if you will, to get us all, you know, in the right mindset of the different types of what I'll call
(02:39):
performance enhancements
or high performance computing aspects that you can give
to your computer science type programming task.
One of those is parallel computing.
This is tailored more for when you have, say, a group of things that
maybe it's, you know, a set of countries in the world with a data set.
(03:01):
Maybe it's the states in the United States in a data set. Some kind of grouping
where you could do an analysis on one particular group,
but that other group doesn't have any impact on it. In other words,
while you could do an analysis sequentially across these different groups,
it may be more advantageous
(03:21):
to lurk look at all these groups in parallel
and do separate, you know, analysis tasks for each of them.
So very much the
split, apply, combine
kind of
mapping, reduce out, you know, paradigm
that he has some nice illustrations
to
show how that works.
And there is a very important R ecosystem
(03:45):
that helps with this framework immensely,
and that is the future
set of packages. You might call it a future verse, if you will. That's, originated by Henrik Bengtsson, who I've had the great pleasure of meeting at previous RStudio Conferences.
But there are many additional packages in the future ecosystem
(04:05):
on top of the future package itself that kind of tailor to different
back ends or different paradigms that you can do
parallel computing.
And, yes, with certain parallel tasks, you can get massive improvements in execution time
versus doing the sequential approach. But,
like everything, there's trade offs. Right? So
(04:28):
maybe parallel computing may not be the best when you have to deal with
kind of the upfront cost
of what Colin calls the cost of transportation,
meaning
that you have to somehow allocate to these different processes
that are being launched
kind of the setup to get things going.
(04:49):
1st, analyzing the code itself to figure out
what are the functions that it depends on, what are packages that depends on, any
data objects or other R objects that it depends on,
and then
writing or sending out those prerequisite
objects
to disk into so it can be loaded into a new R session.
(05:12):
And then when everything's done, you've gotta assemble everything back together.
This is pretty much negligible when it comes to, like, more simpler computation
task.
Not so much for a task that maybe
has that upfront cost
that maybe takes a bit of time to transfer,
(05:32):
and it may end up being faster if you just did it all yourself anyway.
I know how that goes in real life too, and Colin's got a great example
to illustrate that.
So, again, the other key part to keep in mind for parallel computing is even though you are launching these different, you know, group processing
on different sessions, whether it's multiple CPU cores or multiple servers,
(05:56):
in the end, you're still waiting for that result. You just hope that you don't have to wait much longer.
The other paradigm, especially the the meat of this post, is the concept of
asynchronous programming.
This is a little different
because
it's not so much dividing
an overall, you know, dataset or or or other object into groups and then launching those in parallel,
(06:22):
you have maybe various tasks that are, you know,
maybe they're related, maybe they're not related.
But you wanna make it so that you can launch those in the background
and then at some
point, get a check to see if they're readily available to you.
That's the concept of asynchronous programming.
(06:43):
And, yes, the the future package can help with this too, and he's got examples in the code
to show what this might look like with some contrived sleeping
and and messaging.
Now, another thing to keep in mind is that this is not blocking
the r session. You could type over r code in the console, and then when that task is finished that you you launch in the background, a message will pop up depending on how you configure the function
(07:12):
to do this.
Now, this could be a bit of a challenge
to figure out how do you manage a lot of this together,
how do you track when a task is done in this asynchronous approach, and then what do you do when you actually get those results?
(07:32):
That's where the promise package comes in
and promises, I should say, which is very much inspired by, you know, JavaScript,
utilities such as promises
that come fundamental
of modern JavaScript languages.
The promises package kind of gives you that same paradigm where you can define then
(07:53):
what happens for a given function
that when the result
is done
or when things go wrong, I e, what should what should it output or what should it return when there's an error in that execution?
And then what do you want the user to see
or return from when everything is done, whether it worked or not? So there's a good example here about how to tie the 2 together, future and promises
(08:21):
in an arbitrary type of function setup.
And you can see
that you can, you know, define these 3, you know, these 3,
scenarios.
What happens when it works? What happens when it doesn't work? And no matter what, what happens when everything is done?
This, you may be wondering, okay. That's great, Eric. What does this have to do with Shiny? Right?
(08:46):
Well, in Shiny,
you definitely want to think about asynchronous
programming,
especially in the scenario that Colin outlines here
when you have multiple users,
you know, re using your app at the same time,
the traditional
paradigm in r, which is affected by Shiny as well,
is it's single threaded, I e
(09:08):
one overall
process
running that execution.
Hence, if you don't build asynchronous programming in your Shiny app, if you have 2 users and 1 user starts to launch a task, it maybe takes, you know, a handful of seconds or even a minute to complete.
That other user, when they launch that same Shiny app and they hit that button,
(09:30):
they have to wait to even start their back their analysis until that first user has completed their task because
there's one session
per app in a typical execution in R itself.
So the good news is everything we just talked about of asynchronous programming
can be used in Shiny as well.
(09:51):
The previous generation of this that he starts off with was
this combination of future and promises,
and it was many years ago at a Rstudio conference
that Joe Cheng, the author of shiny himself,
had this interesting example of what he called the cranwhales
app
to show how he could optimize a shiny app in production, and it was using
(10:15):
the same integration
of future
and promises.
Now the way they configure this is, again, building upon the
the more,
streamlined example of the r process itself.
But Colin has an example here where it's using
what he calls a multi session future plan, which means that
(10:37):
asynchronous task is gonna be spun on multiple r processes
with 3 workers. We'll keep that in mind for later.
And then
based on hitting a button that will basically,
format the the current date and time,
he defines then the function
wrapped in or the set of code wrapped in future
(10:57):
to generate a random normal distribution.
With a set of numbers and then defining what happens when the result works,
when the result fails,
and then what happens when everything is done.
This takes a bit of bookkeeping to set up,
but there is one important assumption to make here.
(11:21):
If there are more than 3 users, let's say a 4th user goes to this app, the first three users are able to launch their task when they hit that button
in parallel.
But that 4th user,
oh, no. Sad panda face. They have to wait because everything is blocked
because
that meant there were 4 future sessions
(11:43):
launched in 3 sessions, and when those sessions are busy that other users gotta wait.
So there could be improvements in this, and yes, there are.
There is a wrapper
combining these two concepts of future and promise
called future underscore promise
is that you can now launch this asynchronous
(12:06):
type processing,
get a queue going,
and then
as
these,
sessions finish,
it can then launch new ones that come
as soon as one of those workers become available instead of all the workers being available.
That's a nuance there, and he's got a nice illustration to show this
(12:28):
in action along with the revised code to to launch this.
So, again, the setup looks pretty similar. You're just swapping in, future underscore promise instead of future.
This is working better.
But there but, again, that 4th user
will still have to wait a little bit until at least one of those three tasks finishes from the previous three users.
(12:55):
But there is a new paradigm, something we've covered on our wiki highlights a while back this year,
the extended
task function that now comes native
since shiny version 1.8.1,
which is giving you an r six class
to provide native asynchronous
support
without a lot of the
(13:16):
what I'll affectionately
call
somewhat odd setup that the future and promises,
package and a shiny or a combination
and a shiny app required you to do. You will see the example when you read the post, but it was never quite intuitive to me.
Extended task is in the right direction.
I would say to make things a bit easier because you're creating a new task
(13:39):
with an r six object,
putting in your future promise, where, again, some of that boilerplate still in there. But this is going to
help the user experience quite a bit because you get some nice helpers with this as well,
such as having a button in your Shiny app that can be automatically
(13:59):
relabeled on the spot when the user clicks it to give them an indication that something is happening. So you get a little processing message
on that button when they hit the button
so this is again a great improvement to help unify things a bit more
and extended tasks has some really nice documentation that we'll link in the show notes as well.
(14:23):
So when I saw this post, of course, I love seeing,
you know, very, you know, detailed,
you know, overviews of asynchronous programming because it is such an important concept in Shiny development.
But I did have another thing in mind that could be put
in this same paradigm, but I'm gonna flip the script a little bit.
(14:45):
Colin is coming at this from multiple users,
accessing an application at the same time.
That's not always the case. I have situations, and I know many others in it in the in the industry do,
where it's not so much many users hitting the app at once,
but one user,
(15:05):
their experience of running the app, they may want to launch something that's gonna take a while, but yet do other things in the app without the whole session being blocked. So what's I call within session,
asynchronous programming is desired.
And for this, yes, extended tasks can be used in this as well, but I do dare say that the combination
(15:29):
of Will Landau's
crew package
with Charlie Gao's MirrorEye package
can be a really attractive paradigm for you as a shiny developer.
If you know there are gonna be tasks launched in your shiny app that could be as simple or extremely
complex,
crew with MirrorEye is gonna give you an elegant way
(15:53):
to launch these workers
without really knowing ahead of time how many to launch per se. It's gonna kind of adapt to it. It's got some defaults that you can tweak,
but the example that I'll link to in the show notes is from the vignette of KRU
where they have a shiny app where you can hit a button and and
simulate basically 1,000 coin flips.
(16:15):
The default version of this, the output of, like, the heads, tails,
and the number of flips
is very laggy, the update, because everything's kind of being done
in one overall process. But with crew and MirrorEye,
when you hit that button, it's like looking at a a speedometer,
(16:35):
and it just continuously
increases
faster, and you could be able to type the the numbers. It is amazing to watch.
So I would say if you're in this paradigm of
within session
trying to optimize the performance as best as possible,
KRU and Mirai is a combination you definitely want to look at.
(16:57):
And the good news is that Will and Charlie have also talked to Joe Cheng about this, and he's been playing with it too. So there may be more integrations between these two paradigms
of shiny's extended task and Mirai and Krue in the near future.
Nonetheless, a terrific post by Colin on what can be an extremely difficult concept to master
(17:19):
as a shiny developer
for many many different reasons,
but this is a great way to get started and I invite you to check out these terrific resources in the blog post as well as
Will's and Charlie's crew and Mirai package respectively.
(17:51):
Oh, there's that time of year wherever you're, you know, getting some much needed time off from the day job, spending time with family or friends or whatnot. You might get a little downtime once in a while, and you wanna, you know, maybe do something
interesting and challenging for yourself.
There's definitely an appetite to look at different, you know, various puzzles, especially with coding, which we'll get to in a little bit.
(18:15):
But there are some really interesting ways to challenge yourself yourself
from very, you know, historical yet very interesting publications
from centuries ago.
And our next highlight here is giving you a little brain teaser to work on, and, yes, you can use R to help out with it.
(18:36):
Data scientist Nina Zumo,
who is a consultant for the WinVector
consultancy firm,
she has a new post on the Rworks
blog, which, by the way, I wanna give kudos to R Works because I don't think we've talked about very specifically on this on the show.
This is kind of the spiritual successor
(18:57):
to Joe Rickert's immensely
successful and highly acclaimed
our views blog posts that you would have from his days of revolutions computing in that posit as well. But this is kind of like the next generation of that, and Joe is one of the editors of this along with Isabel Velasquez, but they
give you the opportunity to put posts on their blog if you if you wish to contribute. It looks like Nina took them up on this offer multiple times.
(19:24):
Nonetheless, what Nina's talking about here is a a great brain cheeser with mathematics
that you can, you know, you actually use r to help solve,
and this is rooted into
the 4 weights problem,
which has a long history in it itself.
This was from a article
(19:45):
author by Henry Dunanese,
from The Strand Magazine
back in December of
1908. That's over a 100 years ago, folks.
But, yes, this has all been digitally archived, so we can learn from this too.
And the basic premise of what it's called the 4 weights problem,
think of having a scale that can be bound a weighing scale that has, you know, two sides left and right,
(20:12):
and the object is to find 4 weights. They all can't be the same of 4, say, weighted objects
that in the integer range of 1 to 40
can be put on these two levers of the weighted scale
and still be balanced together.
That sounds it already sounds complicated to me. I mean, this would take me a few maybe a few hours, if not longer, to sit down and do.
(20:40):
But
you want to be able to find what are the best ways to get to this solution.
So you one thing is just find the values that could work that could, you know, add up to the value of 40 without going over.
But then how do you balance that together? So
so the fundamental
paradigm behind a teaser, a brain teaser like this, if you will,
(21:04):
is that this is kind of based on a a different type of arithmetic, maybe, than what you may be doing in your day to day
because,
you know, that she she introduces that there are weights of 1, 3, 9, and 27
that could satisfy this, but
you have to figure out then
(21:26):
what this unknown quantity x
to make sure that on one side of the scale that we can distribute these weights so that the scale actually balances. And, yes, that is somewhat contrived because you wouldn't have to weigh everything if you already know it ahead of time,
but this is rooted into modular
arithmetic for how you can solve this.
(21:48):
So she starts with kinda, you know, figuring out the best way to kind of change our thinking of this
with this unknown quantity x, and how do we shift our lens of this
to a base n type representation.
This is where it'll get, you know, a little complicated to summarize in audio.
(22:09):
But the idea is that you have this you want to represent a non negative integer
with up to m digits and binary
representation.
And if you recall,
binary digits
can be represented by any number in a range
from 0
to some arbitrary m of 2 to the power n minus 1.
(22:32):
So there's some pseudo code here
to look at the different ways we can transform this x into base 2
notation,
and in the end get to a trinary
representation
of this that can satisfy
these requirements. And, yes, there is some neat r code.
To do this, she's got a handy function called basen
(22:55):
that will convert that x to it's a base end representation.
And it's got a, you know, a simple while loop
that based on certain conditions trying to figure out the the best way to represent that.
And we can see then
that when we
want to convert 13
to binary,
(23:16):
we can use this function feed in 13 as a verse parameter,
and then the base of this notation is usually 2 by default
and the number of digits for maximum.
And in the end for this example of 13 we get the result of 1, 1, 9, and 1
and then that can be converted to what's called a ternary
(23:39):
representation
with another
function that she has
constructed called 2 decimal,
where we're gonna feed in
the the number of digits to put in that, and hence we get
9,298
as the result to convert that to 18.
(23:59):
So
now that that gives us ground in for it to come into base functions.
Going back to that 4 weights problem,
we've got to now figure out with an equation that she writes here
these
for the for the x these different kind of coefficients, if you will,
in front of these 4 weights
(24:21):
and figuring out what are
the the the coefficients that we can put in front of that. So, again, some nice r code to kind of loop through this
with a a function that she wraps. She calls it way
to take this x.
And, again, we got, you know, bounded by 40 or or or less. Otherwise, it'll be out of the range.
(24:44):
She's got some arithmetic
to figure this out, as well as
scaling the notation
as well, which can be
helpful to get that trinary
representation
in that. And then she gives some examples of trying these functions out,
and you can see that we've got a few few different options here.
(25:04):
And in the end,
the answer that she comes up with
is on one
side of the weight scale. You put 1 and 3, weights of 1 and 3,
and then the other side is just gonna be the the leftover of x.
So there's a well, a few different ways that they carry this out. She's got multiple examples here.
(25:27):
But,
in the end, I think this is something you're gonna have to sit down with it a bit and maybe, again, try to challenge yourself in some of these other puzzles that could be found in publications like this. But it goes to show you that with basic arithmetic,
yeah, R can do just as well as any other programming language to
convert, you know, the the pseudo code that you're kind of going through in your head as you come up with an algorithm
(25:52):
to fit for purpose functions that can tease this out. And we'll have a link in the show notes as well to Nino's,
first Nino's, first blog post about this. It gives more background
on kind of the the other ways you could tackle this.
But, yeah, very good brain teaser. I didn't do it justice in audio, but you're invited to check it out if you wanna challenge yourself a bit when you get some downtime this, holiday season.
(26:33):
And speaking of challenging yourself, I did allude to a bit earlier,
this is also a very popular time of year to challenge yourself with some coding puzzles, too.
And the one, you know, area that gets,
gets spun up almost every holiday season for the past many years
has been the advent of code,
(26:54):
where for every day in this advent calendar, a new coding puzzle is released, and you're invited to use whatever language suits your fancy
to solve these these challenging problems.
I admit it's on my bucket list to be able to do this someday. I just haven't had as much time yet to do it, but maybe someday I will, maybe in retirement or whatever. But
(27:15):
that's not that's not soon for sure. Nonetheless,
there is there is some great opportunities to leverage your favorite data science language, such as R, to handle this. And, of course, within R itself,
there are many different ways to tackle this too with various packages and whatnot.
And so our our last highlight today is authored
(27:35):
by Kelly Baldwin, who is a assistant
professor of statistics
at the California
Polytechnic
Unif State University,
and she's actually authoring this on the data dot table blog, the our data table community blog, to be specific,
because
she wanted to challenge herself.
(27:57):
She's, again, always been, you know, very happy about the Advent or Code series of puzzles,
and she wanted to see,
you know what? Just see what it takes to use data dot table and some of these puzzles.
So that is really awesome. This blog post has actually been updated with
with a couple solutions here based on the puzzles that started on December 1st.
(28:22):
And so the first puzzle in this advent of code for this year is a bit of a warm up if you will where you have 2
lists. You might say 2 columns of numbers,
and the goal is to compute the distance between
the adjacent elements of these lists. But you want to do it from
the least, you know, the smallest number first and one list and then the smallest number on the right list. And then kind of keep, you know, sorting accordingly
(28:51):
to figure this out in the most elegant way that you can. And so with data. Table, Kelly shows that this can be a pretty straightforward
operation where you're just simply using its,
sort function to
basically mutate in place, if you will, or change the variable in place after reading this text file,
and then create a new variable called diff, which is again the absolute value between
(29:14):
those two numbers, and then computing the sum of that at the end.
That was pretty straightforward. Basically, four lines of code with data dot table, you know, not too challenging there.
And then part 2
is a a little bit different
because you can do things a little
because in part 2
in part 2, she is now looking at the kind of distance
(29:38):
or similarity,
if you will, between these these two
these different types of distances that have been computed.
And sure enough, that's yet another variable that you can assign in data dot table
as the sum
of the previous
of the variable
where the or the sum when the value in the first column is equal
(30:00):
to the
the
the various values in the second column
only when they're equal, multiply that by the other variable variable one value itself, but then grouping it by those unique values
in the first list and then summing up that similarity
calculation.
Again, two lines of code for that one.
(30:21):
So, again, pretty pretty straightforward once you open that text file to
solve that with data dot table.
And then there is yet another puzzle that has multiple parts to this as well from December 2nd,
where the background of this this puzzle is there are certain reports
that basically can be represented,
(30:43):
by
one line in a in a matrix,
and that
the report is a list of numbers called levels, which are basically the columns. So in the example that I will link to in the in the show notes or, I should say, the problem in the show notes has 5 levels, 5 columns for each report list,
but the goal is to, quote unquote, find when these variable reports are safe,
(31:08):
which basically means that the levels are either all increasing
or all decreasing,
and
any 2 adjacent levels differ by at least 1
and at most 3.
So those are 2 constraints to keep in mind.
So this this could be a bit of a challenge. Right? So again, what can data dot table do here?
(31:33):
So after importing this,
this, set of data,
Kelly defines a function called check report,
where it's gonna determine if these
this vector of numbers, in this case kind of like the list in that report vector, if they have negative or positive integers,
and then be able to figure out then
(31:55):
for whether it's too big
or whether it's increasing or decreasing,
an indicator for that, and then returning
whether that was
increasing or decreasing,
and if it's not too big. So basically,
the report will the check report function will return true if those conditions are met and false otherwise.
(32:18):
So she takes this report,
the dataset,
transposes it first,
and then with data dot table basically computes then
for the different,
for the different deltas between
these different
columns in in the dataset
(32:39):
applying wherever it works in a check report fashion. So it's kind of doing column wise,
but applying that with the apply function
on each of these and sure enough she was able to determine that
those are those are working well.
And then in part 2, she has a function of testing
all this
to make sure
(33:00):
that this is working as well, and sure enough it it is as well.
So she wasn't very happy about using the apply function for some of this,
so
she wanted to, you know, make see if there's another way to do it. And, yes, in data dot table,
there is yet another way,
(33:22):
another way to do it with data dot table itself where she's using the call sums function.
You know, I should say the call sums function wrapping
those,
those various calls or those indicators
that you can use after you compute those deltas.
And, again, very very similar.
And she hasn't done anything with data dot table yet for December 3rd or 7th, but we'll check back on the blog as as we progress in Advent of Code to see where this shakes out. So, again,
(33:52):
Advent of Code, fun little puzzles to entertain yourself,
you know, with some fun challenging exercises.
I heard I believe they get harder as as the
as the time goes on. But again, there's no
nothing's too easy here, so to speak. So
great way to challenge your challenge your, skills here and use a little r and your favorite package like data dot table in the process.
(34:17):
And, of course, there's a lot more in this terrific issue of rweekly
that Colin Faye has curated for us this week. So I'll take
another minute here for my additional
find that I wanna talk about here in this issue.
And you know me, I always like to see a really great showcase of data science in action with analysis and visualization,
(34:37):
and there is a wonderful
dashboard that has been contributed
to this issue
called the Lime bike analysis.
This has been authored by Nils Indetrin.
And if I had to guess, yes, indeed, my spider senses are correct. This is actually
actually a flex dashboard.
(34:59):
Retro, so to speak, with our markdown.
But when you look at this, it looks absolutely terrific.
And that what a Lime
bike actually is,
It's an electric bike and scooter sharing company,
and they have, you know, they are in various cities around the world.
And so they're look this dashboard is looking at from a single Lime bike account in London
(35:24):
the different total distances by month and day of week that have been traveled,
a nice little set of dashboard icons at the top, and an interactive map
looking at the concentration
of the trip start and end locations,
and another tab for estimated costs.
And boy oh boy, these charts are powered by e charts for r and high charter and leaflet for the math visualization.
(35:50):
Really elegant looking dashboard. Lots of great ideas that you could take everyone is stick with flex dashboard
or even use this with quartile dashboards as well.
Just love the interactivities.
These plots just look amazing here, like, interactive
clicking. I'm playing with it right now as I speak.
Really fun to explore here. So credit to Niles for creating this terrific dashboard. And, again, there's no shiny involved here. This is rendered, if I had to guess, via either GitHub actions or just rendered on the spot
(36:23):
when he committed this on on the repo. But lots of lots of great ideas to take from the source code here, so I invite you to check that out
as well as the rest of the r weekly issue. We've got a lot of terrific
content here for you to look at, but
also
it is getting near the end of the year, and we may have one issue left for the rest of the year, but we could always use your help
(36:46):
for helping with the project, and that can be through your
great blog post that you've either authored or you found online, maybe a great new package that has been very helpful to your workflows,
or even other great content out there in the art community.
We'd love to hear about it, so you can send us a poll request
at r wiki.org.
(37:06):
There's a little Octocat icon at the top that'll take you to this week's current issue draft where you can get a template going when you submit a poll request. We have a nice
set of pre filled, you know, notes that you can use to figure out which category your contribution goes in, but our curator, Faric, will be glad to merge that in for you.
(37:27):
And, also, we love hearing from you on this podcast. We have a lot of a few different ways to do that.
You can get in touch with us via the contact page and the episode show notes on your favorite podcast player here. It'll be directly linked there.
You can also send us a fun little boost along the way if you're listening on a modern podcast app like Podverse or Fountain, Cast O Matic. There's a whole bunch out there. You have all details in in the show notes as well.
(37:53):
And you can find me on social media these days. I am mostly on Mastodon.
I am with the handle at our podcast at podcast index.social.
You can also find me on Blue Sky recently, or I'm at our podcast
dot bsky
dot
social, I believe. Either way, it's in the show notes. I just haven't memorized it yet. You can also find me on LinkedIn. You can search my name, and you'll find me there.
(38:19):
And just a little programming note,
this may be the last episode of 2024
because my,
my 2 little ones are not so little anymore, but their time off of school is starting later this week, and we want to spend some good quality time together. So I'll probably be not doing as much technology things and won't have as much focus time to do an episode. But, nonetheless,
(38:42):
I definitely thank each and every one of you that have stuck with us throughout the year and have enjoyed our weekly and contributed to the project
and contributed your kudos to this podcast.
Again, we love hearing from you. It always is a a great pick me up when we see these great comments, whether it's
Blue Sky or her Mastodon or LinkedIn or whatever. We we enjoy the feedback.
(39:06):
So with that, I'm gonna close-up shop here. I I fly solo today, and it probably showed, but I did the best I could nonetheless.
But, again, hope you all have a wonderful rest of your year in 2024 with however you wanna celebrate
holidays, time with family and friends.
Enjoy it. Hopefully, you get some good rest and recharge with next year.
(39:27):
But, nonetheless, that's gonna close-up shop here for episode 190 of our weekly highlights,
and we'll be back with a new episode of our weekly highlights
in 2025.
Hello, friends. We are back with episode 190 of the Our Weekly Highlights podcast.
This is the weekly podcast where we talk about the terrific highlights that have been shared in this week's our weekly issue available at ourweekly.org.
My name is Eric Nantz, and, unfortunately, I am fine solo again this week because my awesome co host, Mike Thomas, is
(00:24):
currently a knee deep and finishing up some really high priority work projects.
I know from experience that the end of the year can be quite intensive to get these things wrapped up because anything can happen. Right? So
I'll be holding the fort down today, but as usual,
I never do this alone in terms of the project itself because
(00:45):
this week's issue was curated by the esteemed Colin Faye,
who had tremendous help from our fellow Aroki team members and contributors like you around the world with your awesome pull requests and suggestions.
And one little bit of admin note before we go on further,
I am now recording this
on my media box that my media server, I should say,
(01:09):
that is just as of yesterday of this recording,
been refreshed from the ground up to be powered by Nix OS instead of the standard Ubuntu Linux
distribution.
So, hopefully,
you won't notice a thing in terms of listening to this, but I have been knee deep in getting things reconfigured. And I think everything's working, but, hopefully, this episode goes without a hitch.
(01:33):
Speaking of our curator of the week, the esteemed Colin Faye himself is leading off our highlights because he has authored a terrific blog post on the think our blog
on a very, very important concept to think about
with not only within
the computation
of various data analysis task in R itself,
(01:56):
but how it relates to you as a Shiny developer
to give your users the best experience and the best response times.
This is a very difficult task, and he leads off the post with,
you know, an a very humorous saying about the
the 2 or 3 things that are hardest to do in computer science.
(02:17):
And, yes, a couple of them do relate to performance, and the one we're gonna be focusing on today
is asynchronous
computing.
And, yes, there is a lot of ways we can tackle this. But first,
Colin sets the stage, if you will, to get us all, you know, in the right mindset of the different types of what I'll call
(02:39):
performance enhancements
or high performance computing aspects that you can give
to your computer science type programming task.
One of those is parallel computing.
This is tailored more for when you have, say, a group of things that
maybe it's, you know, a set of countries in the world with a data set.
(03:01):
Maybe it's the states in the United States in a data set. Some kind of grouping
where you could do an analysis on one particular group,
but that other group doesn't have any impact on it. In other words,
while you could do an analysis sequentially across these different groups,
it may be more advantageous
(03:21):
to lurk look at all these groups in parallel
and do separate, you know, analysis tasks for each of them.
So very much the
split, apply, combine
kind of
mapping, reduce out, you know, paradigm
that he has some nice illustrations
to
show how that works.
And there is a very important R ecosystem
(03:45):
that helps with this framework immensely,
and that is the future
set of packages. You might call it a future verse, if you will. That's, originated by Henrik Bengtsson, who I've had the great pleasure of meeting at previous RStudio Conferences.
But there are many additional packages in the future ecosystem
(04:05):
on top of the future package itself that kind of tailor to different
back ends or different paradigms that you can do
parallel computing.
And, yes, with certain parallel tasks, you can get massive improvements in execution time
versus doing the sequential approach. But,
like everything, there's trade offs. Right? So
(04:28):
maybe parallel computing may not be the best when you have to deal with
kind of the upfront cost
of what Colin calls the cost of transportation,
meaning
that you have to somehow allocate to these different processes
that are being launched
kind of the setup to get things going.
(04:49):
1st, analyzing the code itself to figure out
what are the functions that it depends on, what are packages that depends on, any
data objects or other R objects that it depends on,
and then
writing or sending out those prerequisite
objects
to disk into so it can be loaded into a new R session.
(05:12):
And then when everything's done, you've gotta assemble everything back together.
This is pretty much negligible when it comes to, like, more simpler computation
task.
Not so much for a task that maybe
has that upfront cost
that maybe takes a bit of time to transfer,
(05:32):
and it may end up being faster if you just did it all yourself anyway.
I know how that goes in real life too, and Colin's got a great example
to illustrate that.
So, again, the other key part to keep in mind for parallel computing is even though you are launching these different, you know, group processing
on different sessions, whether it's multiple CPU cores or multiple servers,
(05:56):
in the end, you're still waiting for that result. You just hope that you don't have to wait much longer.
The other paradigm, especially the the meat of this post, is the concept of
asynchronous programming.
This is a little different
because
it's not so much dividing
an overall, you know, dataset or or or other object into groups and then launching those in parallel,
(06:22):
you have maybe various tasks that are, you know,
maybe they're related, maybe they're not related.
But you wanna make it so that you can launch those in the background
and then at some
point, get a check to see if they're readily available to you.
That's the concept of asynchronous programming.
(06:43):
And, yes, the the future package can help with this too, and he's got examples in the code
to show what this might look like with some contrived sleeping
and and messaging.
Now, another thing to keep in mind is that this is not blocking
the r session. You could type over r code in the console, and then when that task is finished that you you launch in the background, a message will pop up depending on how you configure the function
(07:12):
to do this.
Now, this could be a bit of a challenge
to figure out how do you manage a lot of this together,
how do you track when a task is done in this asynchronous approach, and then what do you do when you actually get those results?
(07:32):
That's where the promise package comes in
and promises, I should say, which is very much inspired by, you know, JavaScript,
utilities such as promises
that come fundamental
of modern JavaScript languages.
The promises package kind of gives you that same paradigm where you can define then
(07:53):
what happens for a given function
that when the result
is done
or when things go wrong, I e, what should what should it output or what should it return when there's an error in that execution?
And then what do you want the user to see
or return from when everything is done, whether it worked or not? So there's a good example here about how to tie the 2 together, future and promises
(08:21):
in an arbitrary type of function setup.
And you can see
that you can, you know, define these 3, you know, these 3,
scenarios.
What happens when it works? What happens when it doesn't work? And no matter what, what happens when everything is done?
This, you may be wondering, okay. That's great, Eric. What does this have to do with Shiny? Right?
(08:46):
Well, in Shiny,
you definitely want to think about asynchronous
programming,
especially in the scenario that Colin outlines here
when you have multiple users,
you know, re using your app at the same time,
the traditional
paradigm in r, which is affected by Shiny as well,
is it's single threaded, I e
(09:08):
one overall
process
running that execution.
Hence, if you don't build asynchronous programming in your Shiny app, if you have 2 users and 1 user starts to launch a task, it maybe takes, you know, a handful of seconds or even a minute to complete.
That other user, when they launch that same Shiny app and they hit that button,
(09:30):
they have to wait to even start their back their analysis until that first user has completed their task because
there's one session
per app in a typical execution in R itself.
So the good news is everything we just talked about of asynchronous programming
can be used in Shiny as well.
(09:51):
The previous generation of this that he starts off with was
this combination of future and promises,
and it was many years ago at a Rstudio conference
that Joe Cheng, the author of shiny himself,
had this interesting example of what he called the cranwhales
app
to show how he could optimize a shiny app in production, and it was using
(10:15):
the same integration
of future
and promises.
Now the way they configure this is, again, building upon the
the more,
streamlined example of the r process itself.
But Colin has an example here where it's using
what he calls a multi session future plan, which means that
(10:37):
asynchronous task is gonna be spun on multiple r processes
with 3 workers. We'll keep that in mind for later.
And then
based on hitting a button that will basically,
format the the current date and time,
he defines then the function
wrapped in or the set of code wrapped in future
(10:57):
to generate a random normal distribution.
With a set of numbers and then defining what happens when the result works,
when the result fails,
and then what happens when everything is done.
This takes a bit of bookkeeping to set up,
but there is one important assumption to make here.
(11:21):
If there are more than 3 users, let's say a 4th user goes to this app, the first three users are able to launch their task when they hit that button
in parallel.
But that 4th user,
oh, no. Sad panda face. They have to wait because everything is blocked
because
that meant there were 4 future sessions
(11:43):
launched in 3 sessions, and when those sessions are busy that other users gotta wait.
So there could be improvements in this, and yes, there are.
There is a wrapper
combining these two concepts of future and promise
called future underscore promise
is that you can now launch this asynchronous
(12:06):
type processing,
get a queue going,
and then
as
these,
sessions finish,
it can then launch new ones that come
as soon as one of those workers become available instead of all the workers being available.
That's a nuance there, and he's got a nice illustration to show this
(12:28):
in action along with the revised code to to launch this.
So, again, the setup looks pretty similar. You're just swapping in, future underscore promise instead of future.
This is working better.
But there but, again, that 4th user
will still have to wait a little bit until at least one of those three tasks finishes from the previous three users.
(12:55):
But there is a new paradigm, something we've covered on our wiki highlights a while back this year,
the extended
task function that now comes native
since shiny version 1.8.1,
which is giving you an r six class
to provide native asynchronous
support
without a lot of the
(13:16):
what I'll affectionately
call
somewhat odd setup that the future and promises,
package and a shiny or a combination
and a shiny app required you to do. You will see the example when you read the post, but it was never quite intuitive to me.
Extended task is in the right direction.
I would say to make things a bit easier because you're creating a new task
(13:39):
with an r six object,
putting in your future promise, where, again, some of that boilerplate still in there. But this is going to
help the user experience quite a bit because you get some nice helpers with this as well,
such as having a button in your Shiny app that can be automatically
(13:59):
relabeled on the spot when the user clicks it to give them an indication that something is happening. So you get a little processing message
on that button when they hit the button
so this is again a great improvement to help unify things a bit more
and extended tasks has some really nice documentation that we'll link in the show notes as well.
(14:23):
So when I saw this post, of course, I love seeing,
you know, very, you know, detailed,
you know, overviews of asynchronous programming because it is such an important concept in Shiny development.
But I did have another thing in mind that could be put
in this same paradigm, but I'm gonna flip the script a little bit.
(14:45):
Colin is coming at this from multiple users,
accessing an application at the same time.
That's not always the case. I have situations, and I know many others in it in the in the industry do,
where it's not so much many users hitting the app at once,
but one user,
(15:05):
their experience of running the app, they may want to launch something that's gonna take a while, but yet do other things in the app without the whole session being blocked. So what's I call within session,
asynchronous programming is desired.
And for this, yes, extended tasks can be used in this as well, but I do dare say that the combination
(15:29):
of Will Landau's
crew package
with Charlie Gao's MirrorEye package
can be a really attractive paradigm for you as a shiny developer.
If you know there are gonna be tasks launched in your shiny app that could be as simple or extremely
complex,
crew with MirrorEye is gonna give you an elegant way
(15:53):
to launch these workers
without really knowing ahead of time how many to launch per se. It's gonna kind of adapt to it. It's got some defaults that you can tweak,
but the example that I'll link to in the show notes is from the vignette of KRU
where they have a shiny app where you can hit a button and and
simulate basically 1,000 coin flips.
(16:15):
The default version of this, the output of, like, the heads, tails,
and the number of flips
is very laggy, the update, because everything's kind of being done
in one overall process. But with crew and MirrorEye,
when you hit that button, it's like looking at a a speedometer,
(16:35):
and it just continuously
increases
faster, and you could be able to type the the numbers. It is amazing to watch.
So I would say if you're in this paradigm of
within session
trying to optimize the performance as best as possible,
KRU and Mirai is a combination you definitely want to look at.
(16:57):
And the good news is that Will and Charlie have also talked to Joe Cheng about this, and he's been playing with it too. So there may be more integrations between these two paradigms
of shiny's extended task and Mirai and Krue in the near future.
Nonetheless, a terrific post by Colin on what can be an extremely difficult concept to master
(17:19):
as a shiny developer
for many many different reasons,
but this is a great way to get started and I invite you to check out these terrific resources in the blog post as well as
Will's and Charlie's crew and Mirai package respectively.
(17:51):
Oh, there's that time of year wherever you're, you know, getting some much needed time off from the day job, spending time with family or friends or whatnot. You might get a little downtime once in a while, and you wanna, you know, maybe do something
interesting and challenging for yourself.
There's definitely an appetite to look at different, you know, various puzzles, especially with coding, which we'll get to in a little bit.
(18:15):
But there are some really interesting ways to challenge yourself yourself
from very, you know, historical yet very interesting publications
from centuries ago.
And our next highlight here is giving you a little brain teaser to work on, and, yes, you can use R to help out with it.
(18:36):
Data scientist Nina Zumo,
who is a consultant for the WinVector
consultancy firm,
she has a new post on the Rworks
blog, which, by the way, I wanna give kudos to R Works because I don't think we've talked about very specifically on this on the show.
This is kind of the spiritual successor
(18:57):
to Joe Rickert's immensely
successful and highly acclaimed
our views blog posts that you would have from his days of revolutions computing in that posit as well. But this is kind of like the next generation of that, and Joe is one of the editors of this along with Isabel Velasquez, but they
give you the opportunity to put posts on their blog if you if you wish to contribute. It looks like Nina took them up on this offer multiple times.
(19:24):
Nonetheless, what Nina's talking about here is a a great brain cheeser with mathematics
that you can, you know, you actually use r to help solve,
and this is rooted into
the 4 weights problem,
which has a long history in it itself.
This was from a article
(19:45):
author by Henry Dunanese,
from The Strand Magazine
back in December of
1908. That's over a 100 years ago, folks.
But, yes, this has all been digitally archived, so we can learn from this too.
And the basic premise of what it's called the 4 weights problem,
think of having a scale that can be bound a weighing scale that has, you know, two sides left and right,
(20:12):
and the object is to find 4 weights. They all can't be the same of 4, say, weighted objects
that in the integer range of 1 to 40
can be put on these two levers of the weighted scale
and still be balanced together.
That sounds it already sounds complicated to me. I mean, this would take me a few maybe a few hours, if not longer, to sit down and do.
(20:40):
But
you want to be able to find what are the best ways to get to this solution.
So you one thing is just find the values that could work that could, you know, add up to the value of 40 without going over.
But then how do you balance that together? So
so the fundamental
paradigm behind a teaser, a brain teaser like this, if you will,
(21:04):
is that this is kind of based on a a different type of arithmetic, maybe, than what you may be doing in your day to day
because,
you know, that she she introduces that there are weights of 1, 3, 9, and 27
that could satisfy this, but
you have to figure out then
(21:26):
what this unknown quantity x
to make sure that on one side of the scale that we can distribute these weights so that the scale actually balances. And, yes, that is somewhat contrived because you wouldn't have to weigh everything if you already know it ahead of time,
but this is rooted into modular
arithmetic for how you can solve this.
(21:48):
So she starts with kinda, you know, figuring out the best way to kind of change our thinking of this
with this unknown quantity x, and how do we shift our lens of this
to a base n type representation.
This is where it'll get, you know, a little complicated to summarize in audio.
(22:09):
But the idea is that you have this you want to represent a non negative integer
with up to m digits and binary
representation.
And if you recall,
binary digits
can be represented by any number in a range
from 0
to some arbitrary m of 2 to the power n minus 1.
(22:32):
So there's some pseudo code here
to look at the different ways we can transform this x into base 2
notation,
and in the end get to a trinary
representation
of this that can satisfy
these requirements. And, yes, there is some neat r code.
To do this, she's got a handy function called basen
(22:55):
that will convert that x to it's a base end representation.
And it's got a, you know, a simple while loop
that based on certain conditions trying to figure out the the best way to represent that.
And we can see then
that when we
want to convert 13
to binary,
(23:16):
we can use this function feed in 13 as a verse parameter,
and then the base of this notation is usually 2 by default
and the number of digits for maximum.
And in the end for this example of 13 we get the result of 1, 1, 9, and 1
and then that can be converted to what's called a ternary
(23:39):
representation
with another
function that she has
constructed called 2 decimal,
where we're gonna feed in
the the number of digits to put in that, and hence we get
9,298
as the result to convert that to 18.
(23:59):
So
now that that gives us ground in for it to come into base functions.
Going back to that 4 weights problem,
we've got to now figure out with an equation that she writes here
these
for the for the x these different kind of coefficients, if you will,
in front of these 4 weights
(24:21):
and figuring out what are
the the the coefficients that we can put in front of that. So, again, some nice r code to kind of loop through this
with a a function that she wraps. She calls it way
to take this x.
And, again, we got, you know, bounded by 40 or or or less. Otherwise, it'll be out of the range.
(24:44):
She's got some arithmetic
to figure this out, as well as
scaling the notation
as well, which can be
helpful to get that trinary
representation
in that. And then she gives some examples of trying these functions out,
and you can see that we've got a few few different options here.
(25:04):
And in the end,
the answer that she comes up with
is on one
side of the weight scale. You put 1 and 3, weights of 1 and 3,
and then the other side is just gonna be the the leftover of x.
So there's a well, a few different ways that they carry this out. She's got multiple examples here.
(25:27):
But,
in the end, I think this is something you're gonna have to sit down with it a bit and maybe, again, try to challenge yourself in some of these other puzzles that could be found in publications like this. But it goes to show you that with basic arithmetic,
yeah, R can do just as well as any other programming language to
convert, you know, the the pseudo code that you're kind of going through in your head as you come up with an algorithm
(25:52):
to fit for purpose functions that can tease this out. And we'll have a link in the show notes as well to Nino's,
first Nino's, first blog post about this. It gives more background
on kind of the the other ways you could tackle this.
But, yeah, very good brain teaser. I didn't do it justice in audio, but you're invited to check it out if you wanna challenge yourself a bit when you get some downtime this, holiday season.
(26:33):
And speaking of challenging yourself, I did allude to a bit earlier,
this is also a very popular time of year to challenge yourself with some coding puzzles, too.
And the one, you know, area that gets,
gets spun up almost every holiday season for the past many years
has been the advent of code,
(26:54):
where for every day in this advent calendar, a new coding puzzle is released, and you're invited to use whatever language suits your fancy
to solve these these challenging problems.
I admit it's on my bucket list to be able to do this someday. I just haven't had as much time yet to do it, but maybe someday I will, maybe in retirement or whatever. But
(27:15):
that's not that's not soon for sure. Nonetheless,
there is there is some great opportunities to leverage your favorite data science language, such as R, to handle this. And, of course, within R itself,
there are many different ways to tackle this too with various packages and whatnot.
And so our our last highlight today is authored
(27:35):
by Kelly Baldwin, who is a assistant
professor of statistics
at the California
Polytechnic
Unif State University,
and she's actually authoring this on the data dot table blog, the our data table community blog, to be specific,
because
she wanted to challenge herself.
(27:57):
She's, again, always been, you know, very happy about the Advent or Code series of puzzles,
and she wanted to see,
you know what? Just see what it takes to use data dot table and some of these puzzles.
So that is really awesome. This blog post has actually been updated with
with a couple solutions here based on the puzzles that started on December 1st.
(28:22):
And so the first puzzle in this advent of code for this year is a bit of a warm up if you will where you have 2
lists. You might say 2 columns of numbers,
and the goal is to compute the distance between
the adjacent elements of these lists. But you want to do it from
the least, you know, the smallest number first and one list and then the smallest number on the right list. And then kind of keep, you know, sorting accordingly
(28:51):
to figure this out in the most elegant way that you can. And so with data. Table, Kelly shows that this can be a pretty straightforward
operation where you're just simply using its,
sort function to
basically mutate in place, if you will, or change the variable in place after reading this text file,
and then create a new variable called diff, which is again the absolute value between
(29:14):
those two numbers, and then computing the sum of that at the end.
That was pretty straightforward. Basically, four lines of code with data dot table, you know, not too challenging there.
And then part 2
is a a little bit different
because you can do things a little
because in part 2
in part 2, she is now looking at the kind of distance
(29:38):
or similarity,
if you will, between these these two
these different types of distances that have been computed.
And sure enough, that's yet another variable that you can assign in data dot table
as the sum
of the previous
of the variable
where the or the sum when the value in the first column is equal
(30:00):
to the
the
the various values in the second column
only when they're equal, multiply that by the other variable variable one value itself, but then grouping it by those unique values
in the first list and then summing up that similarity
calculation.
Again, two lines of code for that one.
(30:21):
So, again, pretty pretty straightforward once you open that text file to
solve that with data dot table.
And then there is yet another puzzle that has multiple parts to this as well from December 2nd,
where the background of this this puzzle is there are certain reports
that basically can be represented,
(30:43):
by
one line in a in a matrix,
and that
the report is a list of numbers called levels, which are basically the columns. So in the example that I will link to in the in the show notes or, I should say, the problem in the show notes has 5 levels, 5 columns for each report list,
but the goal is to, quote unquote, find when these variable reports are safe,
(31:08):
which basically means that the levels are either all increasing
or all decreasing,
and
any 2 adjacent levels differ by at least 1
and at most 3.
So those are 2 constraints to keep in mind.
So this this could be a bit of a challenge. Right? So again, what can data dot table do here?
(31:33):
So after importing this,
this, set of data,
Kelly defines a function called check report,
where it's gonna determine if these
this vector of numbers, in this case kind of like the list in that report vector, if they have negative or positive integers,
and then be able to figure out then
(31:55):
for whether it's too big
or whether it's increasing or decreasing,
an indicator for that, and then returning
whether that was
increasing or decreasing,
and if it's not too big. So basically,
the report will the check report function will return true if those conditions are met and false otherwise.
(32:18):
So she takes this report,
the dataset,
transposes it first,
and then with data dot table basically computes then
for the different,
for the different deltas between
these different
columns in in the dataset
(32:39):
applying wherever it works in a check report fashion. So it's kind of doing column wise,
but applying that with the apply function
on each of these and sure enough she was able to determine that
those are those are working well.
And then in part 2, she has a function of testing
all this
to make sure
(33:00):
that this is working as well, and sure enough it it is as well.
So she wasn't very happy about using the apply function for some of this,
so
she wanted to, you know, make see if there's another way to do it. And, yes, in data dot table,
there is yet another way,
(33:22):
another way to do it with data dot table itself where she's using the call sums function.
You know, I should say the call sums function wrapping
those,
those various calls or those indicators
that you can use after you compute those deltas.
And, again, very very similar.
And she hasn't done anything with data dot table yet for December 3rd or 7th, but we'll check back on the blog as as we progress in Advent of Code to see where this shakes out. So, again,
(33:52):
Advent of Code, fun little puzzles to entertain yourself,
you know, with some fun challenging exercises.
I heard I believe they get harder as as the
as the time goes on. But again, there's no
nothing's too easy here, so to speak. So
great way to challenge your challenge your, skills here and use a little r and your favorite package like data dot table in the process.
(34:17):
And, of course, there's a lot more in this terrific issue of rweekly
that Colin Faye has curated for us this week. So I'll take
another minute here for my additional
find that I wanna talk about here in this issue.
And you know me, I always like to see a really great showcase of data science in action with analysis and visualization,
(34:37):
and there is a wonderful
dashboard that has been contributed
to this issue
called the Lime bike analysis.
This has been authored by Nils Indetrin.
And if I had to guess, yes, indeed, my spider senses are correct. This is actually
actually a flex dashboard.
(34:59):
Retro, so to speak, with our markdown.
But when you look at this, it looks absolutely terrific.
And that what a Lime
bike actually is,
It's an electric bike and scooter sharing company,
and they have, you know, they are in various cities around the world.
And so they're look this dashboard is looking at from a single Lime bike account in London
(35:24):
the different total distances by month and day of week that have been traveled,
a nice little set of dashboard icons at the top, and an interactive map
looking at the concentration
of the trip start and end locations,
and another tab for estimated costs.
And boy oh boy, these charts are powered by e charts for r and high charter and leaflet for the math visualization.
(35:50):
Really elegant looking dashboard. Lots of great ideas that you could take everyone is stick with flex dashboard
or even use this with quartile dashboards as well.
Just love the interactivities.
These plots just look amazing here, like, interactive
clicking. I'm playing with it right now as I speak.
Really fun to explore here. So credit to Niles for creating this terrific dashboard. And, again, there's no shiny involved here. This is rendered, if I had to guess, via either GitHub actions or just rendered on the spot
(36:23):
when he committed this on on the repo. But lots of lots of great ideas to take from the source code here, so I invite you to check that out
as well as the rest of the r weekly issue. We've got a lot of terrific
content here for you to look at, but
also
it is getting near the end of the year, and we may have one issue left for the rest of the year, but we could always use your help
(36:46):
for helping with the project, and that can be through your
great blog post that you've either authored or you found online, maybe a great new package that has been very helpful to your workflows,
or even other great content out there in the art community.
We'd love to hear about it, so you can send us a poll request
at r wiki.org.
(37:06):
There's a little Octocat icon at the top that'll take you to this week's current issue draft where you can get a template going when you submit a poll request. We have a nice
set of pre filled, you know, notes that you can use to figure out which category your contribution goes in, but our curator, Faric, will be glad to merge that in for you.
(37:27):
And, also, we love hearing from you on this podcast. We have a lot of a few different ways to do that.
You can get in touch with us via the contact page and the episode show notes on your favorite podcast player here. It'll be directly linked there.
You can also send us a fun little boost along the way if you're listening on a modern podcast app like Podverse or Fountain, Cast O Matic. There's a whole bunch out there. You have all details in in the show notes as well.
(37:53):
And you can find me on social media these days. I am mostly on Mastodon.
I am with the handle at our podcast at podcast index.social.
You can also find me on Blue Sky recently, or I'm at our podcast
dot bsky
dot
social, I believe. Either way, it's in the show notes. I just haven't memorized it yet. You can also find me on LinkedIn. You can search my name, and you'll find me there.
(38:19):
And just a little programming note,
this may be the last episode of 2024
because my,
my 2 little ones are not so little anymore, but their time off of school is starting later this week, and we want to spend some good quality time together. So I'll probably be not doing as much technology things and won't have as much focus time to do an episode. But, nonetheless,
(38:42):
I definitely thank each and every one of you that have stuck with us throughout the year and have enjoyed our weekly and contributed to the project
and contributed your kudos to this podcast.
Again, we love hearing from you. It always is a a great pick me up when we see these great comments, whether it's
Blue Sky or her Mastodon or LinkedIn or whatever. We we enjoy the feedback.
(39:06):
So with that, I'm gonna close-up shop here. I I fly solo today, and it probably showed, but I did the best I could nonetheless.
But, again, hope you all have a wonderful rest of your year in 2024 with however you wanna celebrate
holidays, time with family and friends.
Enjoy it. Hopefully, you get some good rest and recharge with next year.
(39:27):
But, nonetheless, that's gonna close-up shop here for episode 190 of our weekly highlights,
and we'll be back with a new episode of our weekly highlights
in 2025.
Popular Podcasts
On Purpose with Jay Shetty
I’m Jay Shetty host of On Purpose the worlds #1 Mental Health podcast and I’m so grateful you found us. I started this podcast 5 years ago to invite you into conversations and workshops that are designed to help make you happier, healthier and more healed. I believe that when you (yes you) feel seen, heard and understood you’re able to deal with relationship struggles, work challenges and life’s ups and downs with more ease and grace. I interview experts, celebrities, thought leaders and athletes so that we can grow our mindset, build better habits and uncover a side of them we’ve never seen before. New episodes every Monday and Friday. Your support means the world to me and I don’t take it for granted — click the follow button and leave a review to help us spread the love with On Purpose. I can’t wait for you to listen to your first or 500th episode!
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com