October 1, 2025 • 44 mins
The ducks have made their presence felt in the world of databases, but now you can take a dip in a new Ducklake all within R! If you ever felt someone should write a book on the many ways you can slidecraft Quarto presentations, we have good news for you. Lastly, a spectacular series on harnessing the power of LLMs in your Shiny apps concludes with a powerful and very relevant use case to help review your next presentation slides.
Plus … a popular segment returns 😅
Episode Links
Plus … a popular segment returns 😅
Episode Links
- This week's curator: Eric Nantz: @rpodcast@podcastindex.social (Mastodon) & @rpodcast.bsky.social (BlueSky) & @theRcast (X/Twitter)
- Jumping in the Ducklake with nothing but R on
- Slidecrafting - Making beautiful slides with reveal.js and Quarto
- The Shiny side of LLMs Part 3
- Entire issue available at rweekly.org/2025-W40
- Use the contact page at https://serve.podhome.fm/custompage/r-weekly-highlights/contact to send us your feedback
- R-Weekly Highlights on the Podcastindex.org - You can send a boost into the show directly in the Podcast Index. First, top-up with Alby, and then head over to the R-Weekly Highlights podcast entry on the index.
- A new way to think about value: https://value4value.info
- Get in touch with us on social media
- Eric Nantz: @rpodcast@podcastindex.social (Mastodon), @rpodcast.bsky.social (BlueSky) and @theRcast (X/Twitter)
- Mike Thomas: @mike_thomas@fosstodon.org (Mastodon), @mike-thomas.bsky.social (BlueSky), and @mike_ketchbrook (X/Twitter)
- Relm's Weepy Loop - Final Fantasy 6 - MkVaff - https://ocremix.org/remix/OCR00178
- Serenity & Grace - Chrono Cross - timaeus222 - https://ocremix.org/remix/OCR04143
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
(00:02):
Hello, friends. We are back with episode 212
of the R Weekly Holidays podcast. Boy, that number keeps going up and up, and sometimes it surprises even me. But I'm happy to have you join us wherever you are around the world. And if you're new to the show, this is the weekly podcast,
usually weekly podcast,
where we talk about the excellent highlights and other resources that are shared on this week's our weekly issue at ourweekly.0rg.
(00:28):
My name is Eric Nance, and like I said, I'm happy to have you join us. It's already been a hectic morning for me personally where I was part of a R pharma diversity alliance hackathon with version control, and I'm happy to say
no fires or get had to be put out. And we got three poll requests in. So I'm gonna call that a win. So already a a nice day going so far, but day's getting even better because who's joining me here on the virtual screen here? It is my awesome cohost, Mike Thomas.
(00:56):
Mike, do you have any git fires today? I hope everything's going well over there.
No git fires. Today's a good day because I am writing code. I'm doing targets quarto
types intersection,
and everything is working swimmingly
for now.
I feel like I should say, oh my, afterwards, but that is a very gnarly combination. But, yeah, I'm really interested to see the fruits of those labors, and I've been knee deep in many workflow
(01:22):
actions myself, whoever in targets or even in GitHub actions proper, and debugging those is always an adventure. But that's not the focus today. We got our weekly to talk about today.
As always, let's check our notes here. Who was the the individual that curated this?
Oh, it's a goof me. Okay. It was my turn in the rotation. So it's about every, what, eight weeks or so, I'm called upon again. So as usual,
(01:48):
this is never an effort I do alone. We have,
terrific help from our fellow contributors,
your curators, I should say, as well as contributors like all of you around the world
with your terrific poll requests,
other suggestions,
and even helping the site itself. So special shout out to good friend of mine from the shiny community, Dean Atelli, who I had a great chance to meet again at POSITCOMF.
(02:12):
He sent a PR to make our site looking even better, especially when you look at the all,
tab of the r weekly and you get all those issues. I've loaded up once some nice CSS formatting there
as always. Anything to improve the site, we are more than happy to get in there. But let's improve your our knowledge, shall we? So let's dive into our highlights today, and we lead off with one of the hot topics that definitely
(02:37):
Mike and I saw at Pazacampa this year and the year prior.
Man, there is a ton of momentum around duck DB folks. DuckDV is
becoming, you might call the new SQL lite on steroids. It is just doing so much
in the community for in memory data management,
and it also has some awesome integrations with WebAssembly,
(02:59):
other great things that we heard even at the conf with integrations with Shiny,
so many integrations, even semantic searching. There's I could go on and on.
But there is a new part of DuckDV that
definitely flew under the radar for me,
But, this first highlight is talking about
a new capability called Duck Lake.
(03:20):
And what does this mean to an R user? So let's find out, shall we? This highlight has been authored by Novica Natkoff, who is, I believe, a data scientist at the Norwegian veterinary
Institute,
and he calls this post jumping into the duck lake with nothing but our on. So
this has been, I guess, a newer capability offered by the the company behind DuckDV.
(03:46):
You I guess the way to describe this, and I'm very new to this,
kind of this hybrid of what you hear in what's called a data lake where you often have a huge collection of data,
but also some metadata associated with it to help with the storage and retrieval
of various records.
Well, DuckDB is getting in that town so to speak with
(04:07):
Duck Lake
bringing in these two concepts together.
Now how do you actually get started with this? Well, that's where in the R community we've had the duck DB package
available to us for, gosh, probably almost two years now I'd guess, somewhere around those lines.
But within the package itself of course you get duck DB ready for you.
(04:28):
But one of the selling points of duck DB is the extension system.
There are extensions for all sorts of things.
I remember playing with the extension for object storage retrieval and and and sending of of parquet files and whatnot. That's played nicely.
Well, there's a new extension
called the
Duck Lake extension,
(04:50):
and that's where once you load up DuckDV
in your R process
and you create a database, usually an in memory database to play with things,
you can execute via DB execute
similar to the DBI syntax
the call to install literally the Duck Lake extension from the official
(05:10):
DuckDV extension repository.
With that, you can now attach
a new database.
They call this ducklake dot metadata dot ducklake
as a as a new
in memory database,
and this will create now this duck lake version on your disk.
(05:31):
Certainly in production usage, this Duck Lake artifact would live on probably object storage of our cloud provider and whatnot.
But with that, you can now play with
this in your inyourrcom.
So again, it's written locally to disk on a temporary
basis to start with.
So what's
why do you need to care about this? So in the example in the blog post,
(05:55):
Novica talks about let's load the m t cars set. You can load that in with a typical DB write table call,
and then you can now
create additional
tables with this,
say with a different color or a different categorization.
Where in this case, he's looking at a a derivation of the miles per gallon
(06:16):
over the mean of the miles per gallon as a new as a new variable
and then creating a summary of that with some averaging
and then grouping that by the cylinders. So
you've got now
a layer of raw data. He's calling that bronze in this case,
but this second layer that I just talked about
(06:38):
is sort of some cleaning and normalization
typical of any data kind of pipeline.
And now if you look inspect that that new
file based database duckleg that you've ex that you created here,
you've got three files associated with these different layers of the table
that he has created here. And in each of each of that you've got a parquet file. So it's using parquet
(07:03):
as a storage format.
Again, to me, parquet is like the new CSV. It is just everywhere now. You can't escape it. Not that you want to escape it. It makes your life
a heck of a lot easier, I would say, once you get to know that better.
So now
those are three kind of separate tables based on these derivations,
(07:23):
but you want to
figure out then
something that I think is still on the bleeding edge here
is being able to track kinda how these tables are related to each other. He does have some code that,
that would be in the ideal case of how you define, like, parent tables,
child tables when it's created over metadata.
(07:46):
I believe that's not quite working as expected,
but there are some additional metadata you can put in the table,
you know, Duck Lake, I should say, nonetheless,
such as
executing
different
values
into the filtering operation
(08:06):
and then visualizing then the relationship
amongst these tables. So So you got these parquet files that are kind of stored on disk. Right?
And that's where you can now
select into these from these relationships that you've established. And, again, some code works. I believe some doesn't. Haven't tried it myself.
But then you can create then
(08:28):
a graph of the kind of ontology
of the the way these tables relate to each other. And there's,
there's an example here in the blog post. It looks like a kind of a DAG of sorts
where it goes from one part of the table and I have to zoom in on my browser because it is kind of small on my screen here.
From the the the first level, he called it the bronze version of MT cars,
(08:52):
and then it branches off into separate branches
of these different derivations
that have been made here with this silver layer. And then from one of those layers that branches off again
to this gold layer at the very end,
calling it kind of a a catalog of sorts when visualizing
how those are related.
(09:13):
So what he thinks the value of this is is that when you have these this metadata
establishing the relationship amongst these tables,
you can now put this relationship
diagram of sorts, whether it's like this graph here in in the post
or elsewhere,
that could go in, like, your internal documentation for this, you know, set of databases or set of tables.
(09:37):
Thinking of Quarto that Michael Mike mentioned he's been working on. Maybe a Shiny app that visualizes the relationships amongst these.
There are a lot of things that this could be used for,
and in fact, he even puts that at the end on top of the gist that has the code for the blog post.
Maybe you want to fold this into part of an existing data pipeline.
(09:59):
We covered in, I think, last year's our weekly highlights
a package called Maestro that helps you kinda take a workflow like approach
to building data pipelines.
This could be a good fit for those as well.
So my take on this is that, a, it's standing on on well established format for saving these the disk, a parquet,
(10:20):
but it's not just the derivations themselves.
You are able to
add kind of relationships
for these tables as metadata
and then have that available
however you choose
to visualize it.
So I'm still very new to this. I do want to play with this a bit once I get into more complex setup,
(10:41):
but it's great to see that with an extension away, you can now play with a little bit at your fingertips
of what the Duck Lake Promise sets to be. So really informative post and and shout out, Novica who sent this my way on blue sky, and I'm like, yes, please. Get me all the Duck DB content you can you can pass at me. So I'm enjoying this, but, yeah, Mike, does this get your creative juices doing what or going what you do with DuckDV?
(11:09):
It does very much so. And I'm loving DuckDV.
It's still pigeonholed on a lot of projects with clients that we
have on some of the larger,
you know, data lake, if you will, like engines and providers or Databricks and your Snowflakes of the world. And I just anytime I'm working on one of those projects, I always feel like if I had the opportunity to start now from scratch,
(11:32):
we wouldn't need it. We could just use DuckDV,
Parquet files, and things like DuckLake here
to essentially do most of the same stuff for free in a completely open source way that we would have more control over as well. And and I think the benefits of DuckDV are not something that we need to go into again because I've probably,
(11:54):
talked about that way too much on this podcast. I just talked about that probably for five or ten minutes at Positconf,
last week, really, how great of a project that is. And
there's a lot more coming out on that project as well. Duck Lake being one of them. I think Mother Duck as well as sort of a tangential
project that allows you to leverage, you know, cloud compute,
(12:17):
and distributed computing, I believe, as opposed to needing to run things on your own laptop if you want. So a lot of advancements,
going on in this space. And I think one of
the the real pillars of of what Duck Lake is able to do and what Novica is, demonstrating in the blog post is kind of this concept, and you may have heard it if you're familiar with things like Databricks or Snowflake,
(12:39):
of of time travel, really. Such that you can start out with an initial dataset or a table, which is under the hood gonna be represented in a parquet file, stored in a parquet file. And any changes that you make to that, you know, through SQL update statements, through SQL,
insert or delete statements
will also be tracked in subsequent parquet files within that same storage.
(13:05):
And there's this abstraction layer that you can write SQL sort of on top of that storage,
and the way that the files are named and organized
in this,
Duck Lake convention, which is is not necessarily new as you mentioned.
This this data lake protocol has been around kind of for a while. But the,
sort of the protocol allows you to write SQL on top of it to say, you know, query,
(13:30):
this table as of this particular time or as of this particular version.
And under the hood, it can figure out what,
data it needs to query, what inserts to apply, what deletions need to be applied as well to show you the data that you're essentially asking for.
So it kind of builds up this set of parquet files
(13:50):
in your storage.
And I I think this kind of time travel notion is the biggest
sort of selling point of, what Duck Lake has to offer right now. And I really appreciated Novica taking that one step further
in places where, you know, I'm not sure,
DuckDV has, like, visual representations yet, but trying to actually show lineage
(14:14):
as well beyond that. So I thought that that was really cool. And,
you know, DuckDuck, I think, out of the box, tries to recommend leveraging Parquet for file storage and Postgres
for essentially your data catalog,
and allowing you to or your metadata, if you will, on top of your, data warehouse or your your data lake that you're creating. So that combination is is interesting as well, maybe good to know in terms of context here. But I I really enjoyed,
(14:42):
this write up because I I haven't seen a whole lot of, you know, diving into
some of the the weeds in these new developments in what's in DuckDV. I've seen a lot of getting started, but, in terms of getting started with, Duck Lake here, it's not something I had seen a whole lot of before. So I really appreciate Novoca,
putting this together for us.
(15:04):
Yeah. I'm definitely gonna pursue this and maybe some lower
lower risk projects just with some assorted simulated data, but I could I could see this time travel aspect.
Not some ideal with a whole lot on, you know, strictly clinical data kind of stuff I do for most of the the day job. But certainly as we look at different evolving stages
(15:27):
of, say, assumptions about data and different derivations
across its life cycle,
this this could be a great tool in our toolbox. So this is this is in the highlights for a reason. I think this is a a great way to to start off this episode.
(15:58):
Well, Mike, you mentioned you had a a spectacular presentation that you had at Posikov, and and I again, I watched the recording again. It was just as I thought when it was live.
But you and I had something in common with with that other than just, of course, us being presenters at Pazitconf.
We really like using Quarto. We've revealed JS to create our slides these days. It is just
(16:20):
been an absolute pleasure
compared to what I usually have to do with the day job, which I will save that rant for another time. You probably know what I'm talking about.
But it is really, really handy to take what I'm comfortable with, mark down syntax,
and break these up into literally different sections of my slides.
If I need to style it up, I sure can. You know, if I don't need to style it up and I just want nice convenient things like two column layouts,
(16:48):
embedding code chunks,
annotating through those code chunks, highlighting things as I progressively build,
all that comes out of the box of reveal JS and quarto. Absolutely love it.
And I have bookmarked a lot of stuff
in my collection of bookmarks with the quarto tag,
and they're either
(17:09):
from people that made quartal or those that are like us making slides of quartal and they do some neat little trick and I'm like I gotta bookmark that.
There's a lot of these tips out there. A lot of them are quite scattered, but there's been one person at the forefront
of collecting these in such a way that we could benefit as a community.
And boy, he has taken that even up a 100 notches here.
(17:33):
So we are speaking
of the newly released online book
called slide crafting,
authored
by who has been one of the ones at the forefront of this.
Emil Fithfeld,
who is on the as a posit
engineer
at Posit
does a lot of work with the the tiny models team, but he has also been
(17:56):
making
so many great resources about Quarto and in particular making slides of Quarto.
His blog post that he called slide crafting had been featured as previous highlights before but now he has assembled them into one place called the slide crafting book
and this is still early we always saw this come out I believe about a week ago
(18:19):
or so it was after comp for sure
and it's got a collection of all these tips that have been scattered
or he has mentioned in his previous blog post
but it's of it's not feature complete yet but what it has here is of definitely enough to get you started on the right foot.
All with kind of a crash course of creating
(18:40):
your own custom theme for your coral slides in ten minutes or less. It literally has from start to finish what that process looks like, and then the rest of the book goes a deeper dive
into each of these. What goes into the theme? Right? What colors do you select?
What kind of fonts are you using?
(19:01):
And how big or small are certain elements?
Also, like I mentioned, what is the CSS styling associated with it? All of these are dedicated sections in this book,
and I've already, just looking at this,
have learned so much already that I'm gonna take into my next portal
presentation.
(19:21):
And that's just the theming side of it. There's also the content block where we're gonna look at different ways
that you can display your code,
different ways you can highlight certain elements in your deck,
changing
layout. Where do things go? Left, right, moving it up and down?
All these things
that you can do with Quarto
(19:43):
are all
documented here. Like I said, it's not quite done yet.
It's even got,
other sections
such as when you're showing a code demo using things like ASCII cast to help with that kind of recording like setup of a code chunk,
ways you can bring interactivity
via fragments and highlighting certain blocks of of of of elements,
(20:08):
changing the styling on the fly,
and even the the start of an extensions,
section which is still not there yet, but I am looking at I'm gonna keep watching this quite closely.
There's a lot here.
I've always wanted something like this because I want something where I can go to just in one bookmark,
(20:29):
get all these tips.
At some point, a lot of it will become muscle memory. It hasn't yet. So every presentation, I've learned something new. I do something different.
In the past one at POSITCONF, I did the the highlighting of the code and narrated through each of those sections of, like, a shiny state block,
but there's just so much. There's so much to learn. I think this is the start of something huge here. So, yeah, Is the slide crafting? Yes, it is, and I am here for it. What do you think, Mike?
(20:58):
Yeah. I I can't agree more. I am slightly
upset
that this came out how many weeks after Positconf?
I think maybe a week after.
Oh, man. I could have used, like, so much of this in my presentation. This is this is fantastic. And it's so many it answers so many of the little questions that I've
(21:19):
had and just kind of not done the deep research into to figure it out, like image placement,
layouts,
things like that that are just, these great quality of life improvements for your presentation that I knew I needed to incorporate, but just didn't have the the bandwidth or didn't necessarily have a resource like this that I could go to
(21:42):
to quickly get answers to my questions. And don't get me started on what the LLMs
fail to know about, Quarto,
reveal JS
at this point because I I tried and and struggled quite a bit with some of the responses I was getting. But, anyways, this is it's just gonna be this is almost like a big book of type of a resource, and and I it's not super dense either. I mean, there's a ton of content here, but it's pretty easy and well organized to get to what you might be looking for. And even if you're not looking for something specific, you know, like I did, in preparation for the show today,
(22:20):
I found myself, you know, scrolling through every section, you know, taking two minutes
per section and and picking up something,
constantly
in terms of,
theming
and SCSS,
which is, I promise, not as scary as it it may sound if you're not familiar with CSS or or SCSS.
It's maybe a little different to look at at first, but the amount of control that it gives you
(22:45):
to incorporate
your, you know, companies,
or your organization's
brand
and logos and colors into your slide presentation that that that go a long way, believe it or not, to
getting folks to pay attention to what you're presenting to them.
That all everything is essentially right here.
And some fantastic little things that I could have certainly
(23:09):
used. I had, some slides in my presentation that
I needed to kind of manually we had a clicker that we were holding, but I needed someone may out there may have the same type of situation.
But
some had elements in the slide that I needed to scroll
down, and your clicker can't necessarily do that. You would have to go to your mouse or your touchpad
(23:32):
on your laptop to be able to do that. It's not just advancing a slider, not advancing a slide. Well, of course, ML has a section in
here, I believe, under,
fragments potentially
that shows me exactly how I could have done that.
You know, advance a tab set or change tab sets, I should say, or scroll,
(23:54):
on some some content that's on your slide
through, you know, the back and forward,
buttons in the presentation.
Oh, man. I could've used that so much. But, anyways,
just a fantastic
resource.
I can't even pick sort of one area to highlight because I was just picking up stuff everywhere
(24:14):
throughout this book. So I'm excited to see how it continues to expand, but a ton of content here, a great start. I think it's easily digestible.
And if you go to this URL and don't find something,
that, you know, you learned that you didn't know before, I'll give you a dollar. It's how confident I am. This is amazing here. And one thing I'm
(24:35):
as I'm assembling my notes through various presentations,
even if I see a potential to hopefully add maybe to this, I'm hopeful that this will be a community driven effort as well. I may have some things to add around the video,
playing when you get embed videos
in coral. There's some neat tricks you can do with that as well, which were really handy
(24:57):
for my talk with the the the Mega Man inspired app demos that went without a hitch. Thank goodness for that because I was a late night hack to get there, but that that that would be another time.
But, yeah, even you, Mike, you utilize videos nicely in your part of the presentation and do a poll request and and issue notes. So, yeah, I could see. I yeah. This is definitely growing, but, boy, there is enough here now
(25:21):
that for your next quartal presentation, whenever you're doing this at the day job or you have an external
conference you're presenting at, I think you're gonna have enough here to do a a terrific job from both the UX side of it and, like I hope, have fun doing it because it is a lot of fun to make these.
(25:55):
Well, if I'm the curator for our weekly issue, you know I'm gonna have something with Shiny involved in it
and, and, and, podcast. So, luckily, this was voted by others as well, not just me.
But especially at Pazitconf and many avenues before this, we are seeing
very much the intersection of large language models to enhance both development
(26:16):
but also giving unique experiences to Shiny apps that just simply were not possible
even just a couple years ago.
There has been a terrific series
that kinda takes you from literal start to finish, so to speak,
on how you might approach
building LLM
AI model capabilities
(26:38):
in your next Shiny app. And so we are talking about the third
in this series called the shiny side of LLMs,
and this has been authored
by Vera Elfink van Leeenpoot who has been
a prolific member in the shiny community since near the beginning.
She has done tremendous resources.
(27:01):
She's also
doing courses about Shiny as well. We have a platform called Athletics that I've heard great things about, but in any event she has been
embarking on this journey to
bring up to speed
somebody who's new to the world of language models,
what are the nuts and bolts behind how it works, but now we're at the third part where we actually plug this in to a shiny app. So the first parts are talking about what elements actually do and what they don't
(27:30):
And then the next part talked about going from prompts that you put into the LMS
interfaces
and how the responses are constructed.
Like I said, now it's time to put this in action.
And this comes full circle when we last talked about because the use case for this
is going to build a Shiny app
that will let you upload the contents of your quarto presentation
(27:56):
and get feedback from the LOM on what's working well and what might be tweaked. So it is in essence
a peer reviewer
of your next portal slide presentation.
Oh, boy. You told all things I wish I had had sooner. Maybe I could have used this before the conference talk, but nonetheless,
better late than never.
(28:16):
So she's calling this app deck check,
and it's gonna start with
some narrative here and then the post
about first why we choose Shiny.
Well, I think the question is why not, but that's another point for another day. The point being
is that within the ecosystem, now we have packages both on the our side and the Python side
(28:38):
to integrate these chat like interfaces
into your app this is called shiny chat on your side and chat last on the Python side
so once you have that ready to go and and install either of those packages for your use case,
you've now got to choose the LOM provider, of course.
(28:59):
I've been a big fan of Anthropics
Cloud Code.
Their models have done really well for me,
but, nonetheless, whichever
either hosted platform you choose, you're likely gonna have to get an API key because this is gonna go
into your configuration
based with the Elmer package to make all this work.
But like I said there are code examples here to build this app literally from beginning to end
(29:23):
and we start off with getting the chat interface going outside of shiny
so I'm gonna focus on the r tabs here in this what looks like a cordial blog post
with Elmer and configuring the chat
tie into anthropic so there's a chat underscore anthropic
object that you create where you just say the model
(29:45):
the system prompt that you give it, again, plain text,
and any additional parameters specific to the model itself.
And then this is something new. It's not just the chat object,
but you want to plug this in the Shiny. Right? And for this, you want to be able to kind of have that nice streaming effect
as the response comes in, and that's where Elmer comes with the stream method of this chat object.
(30:11):
And then that's where you feed into it some language that you want displayed
in the app itself,
and then be able to tell the LOM what to do with that.
I've never played with the stream object before, so I'm gonna or stream methods. I definitely want to play with this
after after this, after this episode probably.
(30:33):
It does some unique generator based on the coral package.
I believe that's a way to kind of do the looping correctly,
but this is this is really neat. So it gives you that natural look and feel
when the LOM is responding back after the user uploads their coral presentation.
The other thing to keep in mind is that you don't want to have a bottleneck in your app as this response comes in.
(30:58):
You want to take advantage of what's called asynchronous
processing. This is great for the UX of all this
so that your chat
doesn't
grind things to a halt with the rest of the app as the response comes in
that's where it's a combination
of again the Elmer package and in this case the promises package
(31:18):
on the our side
to do the special chat underscore
async
function.
So this is flagged to do asynchronous
mode for this and it returns what's called a promise object
afterwards
that will be
rendered
or resolved
when the app needs it, but it's not gonna stop
(31:39):
everything else.
So with that, you've got kind of the building blocks in place, and now it's time to build the UI.
This is where, for preparing my talk at the conf,
I did a, vibe coding session to prepare the UI in my demo and I got like 90% of the way there. But here she's done the hard work for you.
(32:00):
A simple layout would b s lib,
got a nice of sidebar
or page fluid layout, I should say,
where we've got a module
for the chat component
and a header at the top
kinda at, like, a little placeholder for the user to ask anything about the presentation.
Very simple UI. Again, a module chat mod UI is gonna drive most of this from the Shiny chat package.
(32:27):
On the server side, it's making that connection to the anthropic model
and then rendering the server side of this chat interface,
chat mod server.
They've done the hard work for you. This is a module plug and play. You don't have to reinvent the wheel here.
Really great to see the progress on this because I saw the early days of it. It was a lot of custom code. Now, right off the shelf with ShinyChat.
(32:53):
So once you have that, then you've got the scaffolding of a mess of a new UI. It's very straightforward right now. We're just seeing,
does this work?
And then you see in the screenshot here
the prompt that was added as part of that metadata going into the chat object
and then the response
from the LOM
(33:13):
based on the title
that the user entered
they call it the shiny side of LOM so what's your feedback
and it's got the response there
so we know the
the plumbing works right now it's time to make the app look nice we can't just do that one column interface right
so now she does a little whiteboard in here a sketching here of what the app will look like, a nice sidebar layout
(33:39):
with cards that are added to the application. We got a main content
and and then there's a nice animated screen of what they're working towards, the app in action.
The LOM
prompt is on the left, the chat interface, there's a nice little CSS loader icon,
and then once it's done it's got a whole bunch of like a series of value boxes at the top. Things like length of presentation
(34:04):
and other content,
and then the actual
advice
that it has for the the given presentation
with various pieces of feedback. So all the code is here
for both R and Python,
and you get again this very nice UI
and the app itself is doing a lot under the hood. Right? It's actually taking that uploaded quarto file
(34:29):
and then it's running quarto under the hood to get the HTML version
and then
it's getting that that HTML file
which is then fed to the l o m
based on what they've added in in the back end of it. So this is a nice kind of diagram showing
how this process works but in the end you can copy this app
(34:50):
right from the blog post and try it out for yourself providing you've got that key to anthropic or whatever AI service
you want to you want to do here. So there's other enhancements that she talks about here,
such as a loading experience where you get that nice little
loading icon when things are happening. So the user's not left with just a white screen doing nothing.
(35:13):
Those nice value boxes at the top,
nice refinements. Again, just giving that extra nudge,
that extra pizzazz to
the user experience.
But there's a lot more to this. There's styling. There's how you handle error checking.
But this is as comprehensive of a post I've ever seen
to take one of these apps
(35:33):
from start to finish.
So there's some nice nice examples, like I said, for error checking.
And then when you're ready to go, there's a great section,
what you would do to deploy this.
Obviously, there are platforms like Pasa Connect
that would be able to take this, Shiny apps IO,
the internal
I should say the internal Pasa Connect or the external Pasa Connect cloud.
(35:57):
But if I dare say so myself, another way you could host this would be containers as well and plug that into whatever app, you know, of service a, lets you host a container.
Again, you build the app. You've got many choices where you host it.
Either way, as always, if you're gonna put something like this for public facing, wherever internal your organization
(36:19):
or in the public domain,
be wary of the usage of that API key because you don't wanna break the bank too. But in the end, for a small app that doesn't have more than a handful of users,
I doubt you'll you'll do very much.
Add Jason to this post. I did I had a colleague ask me, hey. You've been using Anthropic for a while. What's been your cost of this?
(36:43):
Looked back at my cost.
I kid you not, Mike. From January to now, my cost for anthropic
has been a whopping $3.20.
It's time you did some explaining. That's less than a lunch at the cafeteria. So I dare say I'm doing things right, I hope.
I don't believe I believe it.
Alright. But yeah. Yeah. That's so that's where just from little me developing and using it with Shiny or Positron assistant
(37:09):
and in some very low level apps that that's pretty tangible. But, again, fantastic
series of posts here.
This is a great, great example of taking the concepts
from the first parts. Now put it in action
for a literal use case that I most definitely will be using before my next presentation.
(37:30):
But, Mike, you've been we've been watching this space for a while. What did you think of Verway's exploration
here? It would be one thing for Vira to just, you know,
use some sort of toy
example and go through the entire process
of creating both
R and Python
Shiny apps with the LLM capabilities in the back end, the ability to render the Quarto document.
(37:56):
Doing that all in just one language would be incredible.
She's done it in both languages
and with a use case that is incredibly,
I don't know, like, relevant or or tangible,
practical,
for anybody that's having to make a quarto presentation,
which, you know, we just talked about in the last blog post and we've both done recently. And by the way, your presentation at Pazitconf was was fantastic. I think you gave me a little too many flowers there, compared to your presentation. But,
(38:28):
this is,
as you mentioned, probably one of the most comprehensive
blog posts I've ever seen. I think it's well deserved that it's on,
Pozits,
series of shiny blog posts. And a I'll just pick out maybe a couple things that I I really like.
I think the error handling is something that we talk about a lot, Eric, and ensuring that your users aren't suddenly just seeing, like, red letters and error messages
(38:55):
that are cryptic on their screen,
and you do your best to try to this is an approach we always take is maybe pop up a modal when there is an error that tells them, hey. Something went wrong.
Maybe you're able to to provide
some pass or if statements that categorize
what went wrong to give them a message, or maybe you even just voice the the message itself, you know, that you would see in the the terminal in that modal, but it's a a much better user experience than the whole app essentially just breaking. Right? We certainly don't want that to happen,
(39:28):
if possible.
The the UI and the design here, I appreciated the time that she took to wireframe, what she wanted to put together,
before that. And
it's also
you know, I don't think this is the purpose of the blog post, but even just
some of the ways that she went about crafting
her her prompt,
(39:50):
yeah, a system prompt, if you will, to sort of instruct
the LLM
what
it should do,
you know, how it should respond and things like that were really interesting to me. She does have a system prompt in there, and then she has sort of a a more verbose
regular prompt.
And the results look
fantastic.
(40:11):
I'm very excited. I have my cloud API key ready, and it seems like I can just sort of copy and paste this this code locally and try to run it myself. Myself. And I'm quite sure I'm gonna be blown away here, but a fantastic
deep, deep dive
into the art of the possible
with Shiny,
for R, Shiny for Python,
(40:32):
and their associated
LLM,
counterparts these days.
Yeah. I am, literally, this afternoon, gonna build a custom next dev environment with Rex just for this because I am that eager to try it out. So, absolutely, that is on my
on my list here. And, again, this was the third part, the culmination of a three part series. So definitely, like I said, if you're new to this, definitely check out her existing material. It does a great job of breaking down
(41:01):
what can be very ambiguous
terms if you're new to this. I mean, even me personally, I still get tripped up by some of the terminology thrown left to right on LinkedIn posts or whatever else in conversation. So I definitely like to have it explained to me practically, but not not in a condescending way. Verily does a terrific job with this. So, yeah, this is
(41:24):
this is right in my wheelhouse, and, yeah, I'm
doing still doing chordal slides. Might have to do one again before the end of the year, so this is this is gonna be utilized for for darn sure. But, we hope you also utilize the rest of the r weekly issue. We are running a bit strapped for time so no additional fines here. I do well, maybe selfishly speaking,
(41:45):
I do have in this issue my adventures
with switching the shiny dev series site to corridor so apropos of what we talked about today lots of interesting findings there
and I'm back to blogging now. I got a blog component of shiny dev series so,
who knows how long it'll last but I'm gonna try and be diligent with it, but it was it was it was good to get get the creative juices going again. So that's in there but also many many other things too. So
(42:12):
with that we will, wrap up here but, of course, we love hearing from you in the community
and we got many ways to get in touch with us. You can send a poll request.
That's time that's definitely appropriate after what I went through this morning, a poll request to the markdown files
that comprise the our weekly project, especially the next issue. We have a draft markdown file.
(42:33):
If you find a great resource, new package, great blog post, maybe you wrote a great blog post you want it featured,
yeah, just send a poll request to that draft issue template
and the next curator will be glad to merge it in.
And, also, you can get in touch with us on the social medias out there. You can find me on Blue Sky these days.
I am at r podcast dot b s k y dot social.
(42:57):
Also, I'm Mastodon
at rpodcast@podcastindex.social.
That's a mouthful sometimes. And I'm also on LinkedIn. You can search my name and you find me there. Mike, where can they get a hold of you? You can find me on
blue sky at Mike dash Thomas dot b s k y dot social,
or you can find me on LinkedIn if you search catch Brook analytics,
(43:17):
k e t c h b r o o k. You can see what I'm up to.
Awesome stuff. Then again,
big shout out to all the listeners out there that said, hi, nose back at the conf. I will always appreciate
all those kind words. We're trying to get back to the regular schedule here where we're doing our part, trying to anyway. But, nonetheless, we'll close-up shop for episode 100 212,
(43:39):
I should say, of our weekly highlights,
and we will be back with another episode of our weekly highlights
next week.
Hello, friends. We are back of episode 102
212.
(44:02):
Well, they so I had my speaker
notes,
you know, on one screen Right. And then my slides on the other screen. And I thought that if I
pressed play on the video on my speaker notes that it would translate into the video. They're separate. They're separate. I found out the hard way you know before. Yep.
Oh, man. Okay. I didn't know he had videos in yours. I would have told you, but No, I know. I'll have to check yours out. You're good.
Hello, friends. We are back with episode 212
of the R Weekly Holidays podcast. Boy, that number keeps going up and up, and sometimes it surprises even me. But I'm happy to have you join us wherever you are around the world. And if you're new to the show, this is the weekly podcast,
usually weekly podcast,
where we talk about the excellent highlights and other resources that are shared on this week's our weekly issue at ourweekly.0rg.
(00:28):
My name is Eric Nance, and like I said, I'm happy to have you join us. It's already been a hectic morning for me personally where I was part of a R pharma diversity alliance hackathon with version control, and I'm happy to say
no fires or get had to be put out. And we got three poll requests in. So I'm gonna call that a win. So already a a nice day going so far, but day's getting even better because who's joining me here on the virtual screen here? It is my awesome cohost, Mike Thomas.
(00:56):
Mike, do you have any git fires today? I hope everything's going well over there.
No git fires. Today's a good day because I am writing code. I'm doing targets quarto
types intersection,
and everything is working swimmingly
for now.
I feel like I should say, oh my, afterwards, but that is a very gnarly combination. But, yeah, I'm really interested to see the fruits of those labors, and I've been knee deep in many workflow
(01:22):
actions myself, whoever in targets or even in GitHub actions proper, and debugging those is always an adventure. But that's not the focus today. We got our weekly to talk about today.
As always, let's check our notes here. Who was the the individual that curated this?
Oh, it's a goof me. Okay. It was my turn in the rotation. So it's about every, what, eight weeks or so, I'm called upon again. So as usual,
(01:48):
this is never an effort I do alone. We have,
terrific help from our fellow contributors,
your curators, I should say, as well as contributors like all of you around the world
with your terrific poll requests,
other suggestions,
and even helping the site itself. So special shout out to good friend of mine from the shiny community, Dean Atelli, who I had a great chance to meet again at POSITCOMF.
(02:12):
He sent a PR to make our site looking even better, especially when you look at the all,
tab of the r weekly and you get all those issues. I've loaded up once some nice CSS formatting there
as always. Anything to improve the site, we are more than happy to get in there. But let's improve your our knowledge, shall we? So let's dive into our highlights today, and we lead off with one of the hot topics that definitely
(02:37):
Mike and I saw at Pazacampa this year and the year prior.
Man, there is a ton of momentum around duck DB folks. DuckDV is
becoming, you might call the new SQL lite on steroids. It is just doing so much
in the community for in memory data management,
and it also has some awesome integrations with WebAssembly,
(02:59):
other great things that we heard even at the conf with integrations with Shiny,
so many integrations, even semantic searching. There's I could go on and on.
But there is a new part of DuckDV that
definitely flew under the radar for me,
But, this first highlight is talking about
a new capability called Duck Lake.
(03:20):
And what does this mean to an R user? So let's find out, shall we? This highlight has been authored by Novica Natkoff, who is, I believe, a data scientist at the Norwegian veterinary
Institute,
and he calls this post jumping into the duck lake with nothing but our on. So
this has been, I guess, a newer capability offered by the the company behind DuckDV.
(03:46):
You I guess the way to describe this, and I'm very new to this,
kind of this hybrid of what you hear in what's called a data lake where you often have a huge collection of data,
but also some metadata associated with it to help with the storage and retrieval
of various records.
Well, DuckDB is getting in that town so to speak with
(04:07):
Duck Lake
bringing in these two concepts together.
Now how do you actually get started with this? Well, that's where in the R community we've had the duck DB package
available to us for, gosh, probably almost two years now I'd guess, somewhere around those lines.
But within the package itself of course you get duck DB ready for you.
(04:28):
But one of the selling points of duck DB is the extension system.
There are extensions for all sorts of things.
I remember playing with the extension for object storage retrieval and and and sending of of parquet files and whatnot. That's played nicely.
Well, there's a new extension
called the
Duck Lake extension,
(04:50):
and that's where once you load up DuckDV
in your R process
and you create a database, usually an in memory database to play with things,
you can execute via DB execute
similar to the DBI syntax
the call to install literally the Duck Lake extension from the official
(05:10):
DuckDV extension repository.
With that, you can now attach
a new database.
They call this ducklake dot metadata dot ducklake
as a as a new
in memory database,
and this will create now this duck lake version on your disk.
(05:31):
Certainly in production usage, this Duck Lake artifact would live on probably object storage of our cloud provider and whatnot.
But with that, you can now play with
this in your inyourrcom.
So again, it's written locally to disk on a temporary
basis to start with.
So what's
why do you need to care about this? So in the example in the blog post,
(05:55):
Novica talks about let's load the m t cars set. You can load that in with a typical DB write table call,
and then you can now
create additional
tables with this,
say with a different color or a different categorization.
Where in this case, he's looking at a a derivation of the miles per gallon
(06:16):
over the mean of the miles per gallon as a new as a new variable
and then creating a summary of that with some averaging
and then grouping that by the cylinders. So
you've got now
a layer of raw data. He's calling that bronze in this case,
but this second layer that I just talked about
(06:38):
is sort of some cleaning and normalization
typical of any data kind of pipeline.
And now if you look inspect that that new
file based database duckleg that you've ex that you created here,
you've got three files associated with these different layers of the table
that he has created here. And in each of each of that you've got a parquet file. So it's using parquet
(07:03):
as a storage format.
Again, to me, parquet is like the new CSV. It is just everywhere now. You can't escape it. Not that you want to escape it. It makes your life
a heck of a lot easier, I would say, once you get to know that better.
So now
those are three kind of separate tables based on these derivations,
(07:23):
but you want to
figure out then
something that I think is still on the bleeding edge here
is being able to track kinda how these tables are related to each other. He does have some code that,
that would be in the ideal case of how you define, like, parent tables,
child tables when it's created over metadata.
(07:46):
I believe that's not quite working as expected,
but there are some additional metadata you can put in the table,
you know, Duck Lake, I should say, nonetheless,
such as
executing
different
values
into the filtering operation
(08:06):
and then visualizing then the relationship
amongst these tables. So So you got these parquet files that are kind of stored on disk. Right?
And that's where you can now
select into these from these relationships that you've established. And, again, some code works. I believe some doesn't. Haven't tried it myself.
But then you can create then
(08:28):
a graph of the kind of ontology
of the the way these tables relate to each other. And there's,
there's an example here in the blog post. It looks like a kind of a DAG of sorts
where it goes from one part of the table and I have to zoom in on my browser because it is kind of small on my screen here.
From the the the first level, he called it the bronze version of MT cars,
(08:52):
and then it branches off into separate branches
of these different derivations
that have been made here with this silver layer. And then from one of those layers that branches off again
to this gold layer at the very end,
calling it kind of a a catalog of sorts when visualizing
how those are related.
(09:13):
So what he thinks the value of this is is that when you have these this metadata
establishing the relationship amongst these tables,
you can now put this relationship
diagram of sorts, whether it's like this graph here in in the post
or elsewhere,
that could go in, like, your internal documentation for this, you know, set of databases or set of tables.
(09:37):
Thinking of Quarto that Michael Mike mentioned he's been working on. Maybe a Shiny app that visualizes the relationships amongst these.
There are a lot of things that this could be used for,
and in fact, he even puts that at the end on top of the gist that has the code for the blog post.
Maybe you want to fold this into part of an existing data pipeline.
(09:59):
We covered in, I think, last year's our weekly highlights
a package called Maestro that helps you kinda take a workflow like approach
to building data pipelines.
This could be a good fit for those as well.
So my take on this is that, a, it's standing on on well established format for saving these the disk, a parquet,
(10:20):
but it's not just the derivations themselves.
You are able to
add kind of relationships
for these tables as metadata
and then have that available
however you choose
to visualize it.
So I'm still very new to this. I do want to play with this a bit once I get into more complex setup,
(10:41):
but it's great to see that with an extension away, you can now play with a little bit at your fingertips
of what the Duck Lake Promise sets to be. So really informative post and and shout out, Novica who sent this my way on blue sky, and I'm like, yes, please. Get me all the Duck DB content you can you can pass at me. So I'm enjoying this, but, yeah, Mike, does this get your creative juices doing what or going what you do with DuckDV?
(11:09):
It does very much so. And I'm loving DuckDV.
It's still pigeonholed on a lot of projects with clients that we
have on some of the larger,
you know, data lake, if you will, like engines and providers or Databricks and your Snowflakes of the world. And I just anytime I'm working on one of those projects, I always feel like if I had the opportunity to start now from scratch,
(11:32):
we wouldn't need it. We could just use DuckDV,
Parquet files, and things like DuckLake here
to essentially do most of the same stuff for free in a completely open source way that we would have more control over as well. And and I think the benefits of DuckDV are not something that we need to go into again because I've probably,
(11:54):
talked about that way too much on this podcast. I just talked about that probably for five or ten minutes at Positconf,
last week, really, how great of a project that is. And
there's a lot more coming out on that project as well. Duck Lake being one of them. I think Mother Duck as well as sort of a tangential
project that allows you to leverage, you know, cloud compute,
(12:17):
and distributed computing, I believe, as opposed to needing to run things on your own laptop if you want. So a lot of advancements,
going on in this space. And I think one of
the the real pillars of of what Duck Lake is able to do and what Novica is, demonstrating in the blog post is kind of this concept, and you may have heard it if you're familiar with things like Databricks or Snowflake,
(12:39):
of of time travel, really. Such that you can start out with an initial dataset or a table, which is under the hood gonna be represented in a parquet file, stored in a parquet file. And any changes that you make to that, you know, through SQL update statements, through SQL,
insert or delete statements
will also be tracked in subsequent parquet files within that same storage.
(13:05):
And there's this abstraction layer that you can write SQL sort of on top of that storage,
and the way that the files are named and organized
in this,
Duck Lake convention, which is is not necessarily new as you mentioned.
This this data lake protocol has been around kind of for a while. But the,
sort of the protocol allows you to write SQL on top of it to say, you know, query,
(13:30):
this table as of this particular time or as of this particular version.
And under the hood, it can figure out what,
data it needs to query, what inserts to apply, what deletions need to be applied as well to show you the data that you're essentially asking for.
So it kind of builds up this set of parquet files
(13:50):
in your storage.
And I I think this kind of time travel notion is the biggest
sort of selling point of, what Duck Lake has to offer right now. And I really appreciated Novica taking that one step further
in places where, you know, I'm not sure,
DuckDV has, like, visual representations yet, but trying to actually show lineage
(14:14):
as well beyond that. So I thought that that was really cool. And,
you know, DuckDuck, I think, out of the box, tries to recommend leveraging Parquet for file storage and Postgres
for essentially your data catalog,
and allowing you to or your metadata, if you will, on top of your, data warehouse or your your data lake that you're creating. So that combination is is interesting as well, maybe good to know in terms of context here. But I I really enjoyed,
(14:42):
this write up because I I haven't seen a whole lot of, you know, diving into
some of the the weeds in these new developments in what's in DuckDV. I've seen a lot of getting started, but, in terms of getting started with, Duck Lake here, it's not something I had seen a whole lot of before. So I really appreciate Novoca,
putting this together for us.
(15:04):
Yeah. I'm definitely gonna pursue this and maybe some lower
lower risk projects just with some assorted simulated data, but I could I could see this time travel aspect.
Not some ideal with a whole lot on, you know, strictly clinical data kind of stuff I do for most of the the day job. But certainly as we look at different evolving stages
(15:27):
of, say, assumptions about data and different derivations
across its life cycle,
this this could be a great tool in our toolbox. So this is this is in the highlights for a reason. I think this is a a great way to to start off this episode.
(15:58):
Well, Mike, you mentioned you had a a spectacular presentation that you had at Posikov, and and I again, I watched the recording again. It was just as I thought when it was live.
But you and I had something in common with with that other than just, of course, us being presenters at Pazitconf.
We really like using Quarto. We've revealed JS to create our slides these days. It is just
(16:20):
been an absolute pleasure
compared to what I usually have to do with the day job, which I will save that rant for another time. You probably know what I'm talking about.
But it is really, really handy to take what I'm comfortable with, mark down syntax,
and break these up into literally different sections of my slides.
If I need to style it up, I sure can. You know, if I don't need to style it up and I just want nice convenient things like two column layouts,
(16:48):
embedding code chunks,
annotating through those code chunks, highlighting things as I progressively build,
all that comes out of the box of reveal JS and quarto. Absolutely love it.
And I have bookmarked a lot of stuff
in my collection of bookmarks with the quarto tag,
and they're either
(17:09):
from people that made quartal or those that are like us making slides of quartal and they do some neat little trick and I'm like I gotta bookmark that.
There's a lot of these tips out there. A lot of them are quite scattered, but there's been one person at the forefront
of collecting these in such a way that we could benefit as a community.
And boy, he has taken that even up a 100 notches here.
(17:33):
So we are speaking
of the newly released online book
called slide crafting,
authored
by who has been one of the ones at the forefront of this.
Emil Fithfeld,
who is on the as a posit
engineer
at Posit
does a lot of work with the the tiny models team, but he has also been
(17:56):
making
so many great resources about Quarto and in particular making slides of Quarto.
His blog post that he called slide crafting had been featured as previous highlights before but now he has assembled them into one place called the slide crafting book
and this is still early we always saw this come out I believe about a week ago
(18:19):
or so it was after comp for sure
and it's got a collection of all these tips that have been scattered
or he has mentioned in his previous blog post
but it's of it's not feature complete yet but what it has here is of definitely enough to get you started on the right foot.
All with kind of a crash course of creating
(18:40):
your own custom theme for your coral slides in ten minutes or less. It literally has from start to finish what that process looks like, and then the rest of the book goes a deeper dive
into each of these. What goes into the theme? Right? What colors do you select?
What kind of fonts are you using?
(19:01):
And how big or small are certain elements?
Also, like I mentioned, what is the CSS styling associated with it? All of these are dedicated sections in this book,
and I've already, just looking at this,
have learned so much already that I'm gonna take into my next portal
presentation.
(19:21):
And that's just the theming side of it. There's also the content block where we're gonna look at different ways
that you can display your code,
different ways you can highlight certain elements in your deck,
changing
layout. Where do things go? Left, right, moving it up and down?
All these things
that you can do with Quarto
(19:43):
are all
documented here. Like I said, it's not quite done yet.
It's even got,
other sections
such as when you're showing a code demo using things like ASCII cast to help with that kind of recording like setup of a code chunk,
ways you can bring interactivity
via fragments and highlighting certain blocks of of of of elements,
(20:08):
changing the styling on the fly,
and even the the start of an extensions,
section which is still not there yet, but I am looking at I'm gonna keep watching this quite closely.
There's a lot here.
I've always wanted something like this because I want something where I can go to just in one bookmark,
(20:29):
get all these tips.
At some point, a lot of it will become muscle memory. It hasn't yet. So every presentation, I've learned something new. I do something different.
In the past one at POSITCONF, I did the the highlighting of the code and narrated through each of those sections of, like, a shiny state block,
but there's just so much. There's so much to learn. I think this is the start of something huge here. So, yeah, Is the slide crafting? Yes, it is, and I am here for it. What do you think, Mike?
(20:58):
Yeah. I I can't agree more. I am slightly
upset
that this came out how many weeks after Positconf?
I think maybe a week after.
Oh, man. I could have used, like, so much of this in my presentation. This is this is fantastic. And it's so many it answers so many of the little questions that I've
(21:19):
had and just kind of not done the deep research into to figure it out, like image placement,
layouts,
things like that that are just, these great quality of life improvements for your presentation that I knew I needed to incorporate, but just didn't have the the bandwidth or didn't necessarily have a resource like this that I could go to
(21:42):
to quickly get answers to my questions. And don't get me started on what the LLMs
fail to know about, Quarto,
reveal JS
at this point because I I tried and and struggled quite a bit with some of the responses I was getting. But, anyways, this is it's just gonna be this is almost like a big book of type of a resource, and and I it's not super dense either. I mean, there's a ton of content here, but it's pretty easy and well organized to get to what you might be looking for. And even if you're not looking for something specific, you know, like I did, in preparation for the show today,
(22:20):
I found myself, you know, scrolling through every section, you know, taking two minutes
per section and and picking up something,
constantly
in terms of,
theming
and SCSS,
which is, I promise, not as scary as it it may sound if you're not familiar with CSS or or SCSS.
It's maybe a little different to look at at first, but the amount of control that it gives you
(22:45):
to incorporate
your, you know, companies,
or your organization's
brand
and logos and colors into your slide presentation that that that go a long way, believe it or not, to
getting folks to pay attention to what you're presenting to them.
That all everything is essentially right here.
And some fantastic little things that I could have certainly
(23:09):
used. I had, some slides in my presentation that
I needed to kind of manually we had a clicker that we were holding, but I needed someone may out there may have the same type of situation.
But
some had elements in the slide that I needed to scroll
down, and your clicker can't necessarily do that. You would have to go to your mouse or your touchpad
(23:32):
on your laptop to be able to do that. It's not just advancing a slider, not advancing a slide. Well, of course, ML has a section in
here, I believe, under,
fragments potentially
that shows me exactly how I could have done that.
You know, advance a tab set or change tab sets, I should say, or scroll,
(23:54):
on some some content that's on your slide
through, you know, the back and forward,
buttons in the presentation.
Oh, man. I could've used that so much. But, anyways,
just a fantastic
resource.
I can't even pick sort of one area to highlight because I was just picking up stuff everywhere
(24:14):
throughout this book. So I'm excited to see how it continues to expand, but a ton of content here, a great start. I think it's easily digestible.
And if you go to this URL and don't find something,
that, you know, you learned that you didn't know before, I'll give you a dollar. It's how confident I am. This is amazing here. And one thing I'm
(24:35):
as I'm assembling my notes through various presentations,
even if I see a potential to hopefully add maybe to this, I'm hopeful that this will be a community driven effort as well. I may have some things to add around the video,
playing when you get embed videos
in coral. There's some neat tricks you can do with that as well, which were really handy
(24:57):
for my talk with the the the Mega Man inspired app demos that went without a hitch. Thank goodness for that because I was a late night hack to get there, but that that that would be another time.
But, yeah, even you, Mike, you utilize videos nicely in your part of the presentation and do a poll request and and issue notes. So, yeah, I could see. I yeah. This is definitely growing, but, boy, there is enough here now
(25:21):
that for your next quartal presentation, whenever you're doing this at the day job or you have an external
conference you're presenting at, I think you're gonna have enough here to do a a terrific job from both the UX side of it and, like I hope, have fun doing it because it is a lot of fun to make these.
(25:55):
Well, if I'm the curator for our weekly issue, you know I'm gonna have something with Shiny involved in it
and, and, and, podcast. So, luckily, this was voted by others as well, not just me.
But especially at Pazitconf and many avenues before this, we are seeing
very much the intersection of large language models to enhance both development
(26:16):
but also giving unique experiences to Shiny apps that just simply were not possible
even just a couple years ago.
There has been a terrific series
that kinda takes you from literal start to finish, so to speak,
on how you might approach
building LLM
AI model capabilities
(26:38):
in your next Shiny app. And so we are talking about the third
in this series called the shiny side of LLMs,
and this has been authored
by Vera Elfink van Leeenpoot who has been
a prolific member in the shiny community since near the beginning.
She has done tremendous resources.
(27:01):
She's also
doing courses about Shiny as well. We have a platform called Athletics that I've heard great things about, but in any event she has been
embarking on this journey to
bring up to speed
somebody who's new to the world of language models,
what are the nuts and bolts behind how it works, but now we're at the third part where we actually plug this in to a shiny app. So the first parts are talking about what elements actually do and what they don't
(27:30):
And then the next part talked about going from prompts that you put into the LMS
interfaces
and how the responses are constructed.
Like I said, now it's time to put this in action.
And this comes full circle when we last talked about because the use case for this
is going to build a Shiny app
that will let you upload the contents of your quarto presentation
(27:56):
and get feedback from the LOM on what's working well and what might be tweaked. So it is in essence
a peer reviewer
of your next portal slide presentation.
Oh, boy. You told all things I wish I had had sooner. Maybe I could have used this before the conference talk, but nonetheless,
better late than never.
(28:16):
So she's calling this app deck check,
and it's gonna start with
some narrative here and then the post
about first why we choose Shiny.
Well, I think the question is why not, but that's another point for another day. The point being
is that within the ecosystem, now we have packages both on the our side and the Python side
(28:38):
to integrate these chat like interfaces
into your app this is called shiny chat on your side and chat last on the Python side
so once you have that ready to go and and install either of those packages for your use case,
you've now got to choose the LOM provider, of course.
(28:59):
I've been a big fan of Anthropics
Cloud Code.
Their models have done really well for me,
but, nonetheless, whichever
either hosted platform you choose, you're likely gonna have to get an API key because this is gonna go
into your configuration
based with the Elmer package to make all this work.
But like I said there are code examples here to build this app literally from beginning to end
(29:23):
and we start off with getting the chat interface going outside of shiny
so I'm gonna focus on the r tabs here in this what looks like a cordial blog post
with Elmer and configuring the chat
tie into anthropic so there's a chat underscore anthropic
object that you create where you just say the model
(29:45):
the system prompt that you give it, again, plain text,
and any additional parameters specific to the model itself.
And then this is something new. It's not just the chat object,
but you want to plug this in the Shiny. Right? And for this, you want to be able to kind of have that nice streaming effect
as the response comes in, and that's where Elmer comes with the stream method of this chat object.
(30:11):
And then that's where you feed into it some language that you want displayed
in the app itself,
and then be able to tell the LOM what to do with that.
I've never played with the stream object before, so I'm gonna or stream methods. I definitely want to play with this
after after this, after this episode probably.
(30:33):
It does some unique generator based on the coral package.
I believe that's a way to kind of do the looping correctly,
but this is this is really neat. So it gives you that natural look and feel
when the LOM is responding back after the user uploads their coral presentation.
The other thing to keep in mind is that you don't want to have a bottleneck in your app as this response comes in.
(30:58):
You want to take advantage of what's called asynchronous
processing. This is great for the UX of all this
so that your chat
doesn't
grind things to a halt with the rest of the app as the response comes in
that's where it's a combination
of again the Elmer package and in this case the promises package
(31:18):
on the our side
to do the special chat underscore
async
function.
So this is flagged to do asynchronous
mode for this and it returns what's called a promise object
afterwards
that will be
rendered
or resolved
when the app needs it, but it's not gonna stop
(31:39):
everything else.
So with that, you've got kind of the building blocks in place, and now it's time to build the UI.
This is where, for preparing my talk at the conf,
I did a, vibe coding session to prepare the UI in my demo and I got like 90% of the way there. But here she's done the hard work for you.
(32:00):
A simple layout would b s lib,
got a nice of sidebar
or page fluid layout, I should say,
where we've got a module
for the chat component
and a header at the top
kinda at, like, a little placeholder for the user to ask anything about the presentation.
Very simple UI. Again, a module chat mod UI is gonna drive most of this from the Shiny chat package.
(32:27):
On the server side, it's making that connection to the anthropic model
and then rendering the server side of this chat interface,
chat mod server.
They've done the hard work for you. This is a module plug and play. You don't have to reinvent the wheel here.
Really great to see the progress on this because I saw the early days of it. It was a lot of custom code. Now, right off the shelf with ShinyChat.
(32:53):
So once you have that, then you've got the scaffolding of a mess of a new UI. It's very straightforward right now. We're just seeing,
does this work?
And then you see in the screenshot here
the prompt that was added as part of that metadata going into the chat object
and then the response
from the LOM
(33:13):
based on the title
that the user entered
they call it the shiny side of LOM so what's your feedback
and it's got the response there
so we know the
the plumbing works right now it's time to make the app look nice we can't just do that one column interface right
so now she does a little whiteboard in here a sketching here of what the app will look like, a nice sidebar layout
(33:39):
with cards that are added to the application. We got a main content
and and then there's a nice animated screen of what they're working towards, the app in action.
The LOM
prompt is on the left, the chat interface, there's a nice little CSS loader icon,
and then once it's done it's got a whole bunch of like a series of value boxes at the top. Things like length of presentation
(34:04):
and other content,
and then the actual
advice
that it has for the the given presentation
with various pieces of feedback. So all the code is here
for both R and Python,
and you get again this very nice UI
and the app itself is doing a lot under the hood. Right? It's actually taking that uploaded quarto file
(34:29):
and then it's running quarto under the hood to get the HTML version
and then
it's getting that that HTML file
which is then fed to the l o m
based on what they've added in in the back end of it. So this is a nice kind of diagram showing
how this process works but in the end you can copy this app
(34:50):
right from the blog post and try it out for yourself providing you've got that key to anthropic or whatever AI service
you want to you want to do here. So there's other enhancements that she talks about here,
such as a loading experience where you get that nice little
loading icon when things are happening. So the user's not left with just a white screen doing nothing.
(35:13):
Those nice value boxes at the top,
nice refinements. Again, just giving that extra nudge,
that extra pizzazz to
the user experience.
But there's a lot more to this. There's styling. There's how you handle error checking.
But this is as comprehensive of a post I've ever seen
to take one of these apps
(35:33):
from start to finish.
So there's some nice nice examples, like I said, for error checking.
And then when you're ready to go, there's a great section,
what you would do to deploy this.
Obviously, there are platforms like Pasa Connect
that would be able to take this, Shiny apps IO,
the internal
I should say the internal Pasa Connect or the external Pasa Connect cloud.
(35:57):
But if I dare say so myself, another way you could host this would be containers as well and plug that into whatever app, you know, of service a, lets you host a container.
Again, you build the app. You've got many choices where you host it.
Either way, as always, if you're gonna put something like this for public facing, wherever internal your organization
(36:19):
or in the public domain,
be wary of the usage of that API key because you don't wanna break the bank too. But in the end, for a small app that doesn't have more than a handful of users,
I doubt you'll you'll do very much.
Add Jason to this post. I did I had a colleague ask me, hey. You've been using Anthropic for a while. What's been your cost of this?
(36:43):
Looked back at my cost.
I kid you not, Mike. From January to now, my cost for anthropic
has been a whopping $3.20.
It's time you did some explaining. That's less than a lunch at the cafeteria. So I dare say I'm doing things right, I hope.
I don't believe I believe it.
Alright. But yeah. Yeah. That's so that's where just from little me developing and using it with Shiny or Positron assistant
(37:09):
and in some very low level apps that that's pretty tangible. But, again, fantastic
series of posts here.
This is a great, great example of taking the concepts
from the first parts. Now put it in action
for a literal use case that I most definitely will be using before my next presentation.
(37:30):
But, Mike, you've been we've been watching this space for a while. What did you think of Verway's exploration
here? It would be one thing for Vira to just, you know,
use some sort of toy
example and go through the entire process
of creating both
R and Python
Shiny apps with the LLM capabilities in the back end, the ability to render the Quarto document.
(37:56):
Doing that all in just one language would be incredible.
She's done it in both languages
and with a use case that is incredibly,
I don't know, like, relevant or or tangible,
practical,
for anybody that's having to make a quarto presentation,
which, you know, we just talked about in the last blog post and we've both done recently. And by the way, your presentation at Pazitconf was was fantastic. I think you gave me a little too many flowers there, compared to your presentation. But,
(38:28):
this is,
as you mentioned, probably one of the most comprehensive
blog posts I've ever seen. I think it's well deserved that it's on,
Pozits,
series of shiny blog posts. And a I'll just pick out maybe a couple things that I I really like.
I think the error handling is something that we talk about a lot, Eric, and ensuring that your users aren't suddenly just seeing, like, red letters and error messages
(38:55):
that are cryptic on their screen,
and you do your best to try to this is an approach we always take is maybe pop up a modal when there is an error that tells them, hey. Something went wrong.
Maybe you're able to to provide
some pass or if statements that categorize
what went wrong to give them a message, or maybe you even just voice the the message itself, you know, that you would see in the the terminal in that modal, but it's a a much better user experience than the whole app essentially just breaking. Right? We certainly don't want that to happen,
(39:28):
if possible.
The the UI and the design here, I appreciated the time that she took to wireframe, what she wanted to put together,
before that. And
it's also
you know, I don't think this is the purpose of the blog post, but even just
some of the ways that she went about crafting
her her prompt,
(39:50):
yeah, a system prompt, if you will, to sort of instruct
the LLM
what
it should do,
you know, how it should respond and things like that were really interesting to me. She does have a system prompt in there, and then she has sort of a a more verbose
regular prompt.
And the results look
fantastic.
(40:11):
I'm very excited. I have my cloud API key ready, and it seems like I can just sort of copy and paste this this code locally and try to run it myself. Myself. And I'm quite sure I'm gonna be blown away here, but a fantastic
deep, deep dive
into the art of the possible
with Shiny,
for R, Shiny for Python,
(40:32):
and their associated
LLM,
counterparts these days.
Yeah. I am, literally, this afternoon, gonna build a custom next dev environment with Rex just for this because I am that eager to try it out. So, absolutely, that is on my
on my list here. And, again, this was the third part, the culmination of a three part series. So definitely, like I said, if you're new to this, definitely check out her existing material. It does a great job of breaking down
(41:01):
what can be very ambiguous
terms if you're new to this. I mean, even me personally, I still get tripped up by some of the terminology thrown left to right on LinkedIn posts or whatever else in conversation. So I definitely like to have it explained to me practically, but not not in a condescending way. Verily does a terrific job with this. So, yeah, this is
(41:24):
this is right in my wheelhouse, and, yeah, I'm
doing still doing chordal slides. Might have to do one again before the end of the year, so this is this is gonna be utilized for for darn sure. But, we hope you also utilize the rest of the r weekly issue. We are running a bit strapped for time so no additional fines here. I do well, maybe selfishly speaking,
(41:45):
I do have in this issue my adventures
with switching the shiny dev series site to corridor so apropos of what we talked about today lots of interesting findings there
and I'm back to blogging now. I got a blog component of shiny dev series so,
who knows how long it'll last but I'm gonna try and be diligent with it, but it was it was it was good to get get the creative juices going again. So that's in there but also many many other things too. So
(42:12):
with that we will, wrap up here but, of course, we love hearing from you in the community
and we got many ways to get in touch with us. You can send a poll request.
That's time that's definitely appropriate after what I went through this morning, a poll request to the markdown files
that comprise the our weekly project, especially the next issue. We have a draft markdown file.
(42:33):
If you find a great resource, new package, great blog post, maybe you wrote a great blog post you want it featured,
yeah, just send a poll request to that draft issue template
and the next curator will be glad to merge it in.
And, also, you can get in touch with us on the social medias out there. You can find me on Blue Sky these days.
I am at r podcast dot b s k y dot social.
(42:57):
Also, I'm Mastodon
at rpodcast@podcastindex.social.
That's a mouthful sometimes. And I'm also on LinkedIn. You can search my name and you find me there. Mike, where can they get a hold of you? You can find me on
blue sky at Mike dash Thomas dot b s k y dot social,
or you can find me on LinkedIn if you search catch Brook analytics,
(43:17):
k e t c h b r o o k. You can see what I'm up to.
Awesome stuff. Then again,
big shout out to all the listeners out there that said, hi, nose back at the conf. I will always appreciate
all those kind words. We're trying to get back to the regular schedule here where we're doing our part, trying to anyway. But, nonetheless, we'll close-up shop for episode 100 212,
(43:39):
I should say, of our weekly highlights,
and we will be back with another episode of our weekly highlights
next week.
Hello, friends. We are back of episode 102
212.
(44:02):
Well, they so I had my speaker
notes,
you know, on one screen Right. And then my slides on the other screen. And I thought that if I
pressed play on the video on my speaker notes that it would translate into the video. They're separate. They're separate. I found out the hard way you know before. Yep.
Oh, man. Okay. I didn't know he had videos in yours. I would have told you, but No, I know. I'll have to check yours out. You're good.
Popular Podcasts
My Favorite Murder with Karen Kilgariff and Georgia Hardstark
My Favorite Murder is a true crime comedy podcast hosted by Karen Kilgariff and Georgia Hardstark. Each week, Karen and Georgia share compelling true crimes and hometown stories from friends and listeners. Since MFM launched in January of 2016, Karen and Georgia have shared their lifelong interest in true crime and have covered stories of infamous serial killers like the Night Stalker, mysterious cold cases, captivating cults, incredible survivor stories and important events from history like the Tulsa race massacre of 1921. My Favorite Murder is part of the Exactly Right podcast network that provides a platform for bold, creative voices to bring to life provocative, entertaining and relatable stories for audiences everywhere. The Exactly Right roster of podcasts covers a variety of topics including historic true crime, comedic interviews and news, science, pop culture and more. Podcasts on the network include Buried Bones with Kate Winkler Dawson and Paul Holes, That's Messed Up: An SVU Podcast, This Podcast Will Kill You, Bananas and more.
24/7 News: The Latest
The latest news in 4 minutes updated every hour, every day.
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com