August 14, 2024 • 33 mins
As AI technology progresses, its impact on our daily lives—including how we consume our favorite sports— will grow alongside it. In this episode of Smart Talks with IBM, Jacob Goldstein, host of Pushkin’s own What’s Your Problem?, sat down with Brian Ryerson, Senior Director of Digital Strategy at the US Tennis Association. They discuss the impact of data on the fan experience, the role that storytelling plays in sports, and how AI has unlocked innovative features, such as AI Commentary and Match Reports.
This is a paid advertisement from IBM. The conversations on this podcast don't necessarily represent IBM's positions, strategies or opinions.
Visit us at https://ibm.com/smarttalks
See omnystudio.com/listener for privacy information.
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Speaker 1 (00:00):
Hey everyone, it's Robert and Joe here. Today we've got
something a little bit different to share with you. It
is a new season of the Smart Talks with IBM
podcast series.
Speaker 2 (00:09):
Today we are witnessed to one of those rare moments
in history, the rise of an innovative technology with the
potential to radically transform business and society forever. The technology,
of course, is artificial intelligence, and it's the central focus
for this new season of Smart Talks with IBM.
Speaker 1 (00:25):
Join hosts from your favorite Pushkin podcasts as they talk
with industry experts and leaders to explore how businesses can
integrate AI into their workflows and help drive real change
in this new era of AI. And of course, host
Malcolm Gladwell will be there to guide you through the
season and throw in his two cents as well.
Speaker 2 (00:43):
Look out for new episodes of Smart Talks with IBM
every other week on the iHeartRadio app, Apple Podcasts, or
wherever you get your podcasts, and learn more at IBM
dot com slash smart talks.
Speaker 3 (00:57):
Hello, Hello, Welcome to Smart Talks with IBM, a podcast
from Pushkin Industries, iHeartRadio and IBM. I'm Malcolm Gladwell. This
season we're diving back into the world of artificial intelligence,
but with a focus on the powerful concept of open
its possibilities, implications, and misconceptions. We'll look at openness from
(01:17):
a variety of angles and explore how the concept is
already reshaping industries, ways of doing business, and our very
notion of what's possible. I'm particularly excited for today's guest,
Brian Ryerson. He's Senior Director of Digital Strategy at the
US Tennis Association, helping to oversee one of the most
(01:38):
iconic events in the world of sports, the US Open.
Brian sat down with Pushkin's own Jacob Goldstein, host of
the podcast What's Your Problem. A veteran business journalist, Jacob
has reported for The Wall Street Journal, the Miami Herald,
and was a longtime host of the NPR program Planet Money.
IBM has been the official technology partner of the US
(02:01):
Tennis Association for more than thirty years, and the more
recent evolution into generative AI has enhanced the world class
digital experiences that help more than fifteen million fans from
all over the world enjoy the US Open Tennis Championships.
In this episode, we will explore how generative AI is
(02:22):
being used to generate match insights, spoken commentary for match highlights,
and postmatch summaries at scale for fans to enjoy through
the US Open app and website. We'll explore how these
AI solutions enable the editorial team to cover more of
the tournament than ever before, bringing fans even closer to
(02:43):
the game they love, and will learn more about one
of the engines behind this AI powered content creation, a
large language model from the ib M Granite family, which
is trained and maintained using the watsonex AI and data platform. Okay,
let's dive in.
Speaker 4 (03:02):
Brian, Welcome to the show.
Speaker 5 (03:03):
Thanks for having me. I'm excited to be here.
Speaker 4 (03:05):
Can you say your name and your job?
Speaker 5 (03:07):
Yeah, I'm Brian Ryerson. I'm Senior director of Digital Strategy
at the USTA.
Speaker 4 (03:11):
Some question, what's the USTA.
Speaker 5 (03:13):
The US Tennis Association.
Speaker 4 (03:16):
And tell me about the USTA, Like, what is it?
Speaker 5 (03:19):
Yeah? So the USTA is the governing body of tennis
in the US. Or mission is to grow the sport
of tennis across the US at all levels. Really, I
would say we're more like a health and wellness company
where tennis is the means to health and wellness. And
then the US Open is kind of our tenth pole
event that happens everyear and Flushing Meadows and is really
our chance to showcase the support of tennis at its
(03:41):
highest level to fans all around the world.
Speaker 4 (03:43):
Yeah, I mean the US Open. I assume most people
know this, but it's Grand Slam. It's one of the
what four biggest tennis tournaments in the world.
Speaker 5 (03:50):
Yes, yeah, every year, we especially the past couple of years,
we've seen immense growth and you know, we are very
hopeful this year and our big goals that have over
a million fires on site during the three week window
this year, So it's an amazing event. I always say
it's a food and wine festival where tennis is the
main attraction and it's a really fun, unique atmosphere.
Speaker 4 (04:10):
How did you get into the tennis business.
Speaker 5 (04:12):
It's a great question. It's not where I thought i'd
end up for especially being there for fourteen years. So
I was a marketing and technology major in school, and
I also played college lacrosse and sports was always a
big part of my life and always wanted to be
in the sports and entertainment world. I'm here from the
New York area. This is where I grew up so
I moved back home and had a few friends who
(04:32):
worked there, and I started out more on the number
side of things and really digital analytics. It was really
the start of when Facebook and Twitter is just starting
and digital marketing and all of that. And you know,
I went to my first year so Open not really
knowing what to expect, and again, I think the atmosphere
kind of captivated me and hooked me in. And I've
been there now fourteen years.
Speaker 4 (04:53):
And so your title is Digital Director. What does that mean?
What's your job?
Speaker 5 (04:58):
Yeah, so it's interesting one because it's tough to explain
to folks who are not in the weeds on all
things US open or even in the sports world. But
really I oversee all of our consumer facing digital property.
So that's the us open dot org, our website built
by IBM, as well as our mobile app. I oversee
our content strategy, our sponsorship integrations. So really anything consumer
(05:21):
facing that happens on the web is under my purview,
even some of our new platform extensions and gaming and
things like that. Anything that you can physically interact with
is kind of under my purview.
Speaker 4 (05:33):
And so you've been there now for fourteen ISSU years,
which in the digital world is a long time. How
has that sort of digital experience of sports changed over
that time.
Speaker 5 (05:46):
Yeah, it's obviously grown digital now, is what we say
and what my team says. It's the number one way
to engage with fans that can't make it to the event,
as well as those fans who are at the event,
and how to enrich their stay. So it's really kind
of you're tackling multiple personas. It's the international fan who's
staying up late to watch in other countries, to the
fan here who's maybe watching on broadcasts, and we go
(06:07):
in a company and enrich that broadcast with new stats
and insights. To the on site fan who bought a
ticket and maybe doesn't know what match is happening on
what court. We do have twenty plus courts happening at
a time, with all different matches, So we really try
to help all fans navigate the US Open the best
way possible.
Speaker 4 (06:24):
And so, like, what are some of the sort of
problems you're trying to solve. What are some of the
hard things about your job?
Speaker 5 (06:30):
Yeah, obviously technology changes at a rapid pace, right, So
I think part of it is how do we stay
on the forefront of that, and how do we do
that in the best way and make the best fan
experiences possible and the best user experiences possible. That's always
kind of driving factor number one. Then number two, it's
understanding and listening to our fans and what kind of
(06:51):
content they want. You hear me talk a lot about storytelling.
I feel like there's a lot of storytelling that happens
around the US open that we really want to bring
to fans. And that can be as simple as storytelling
of what's happening today and what you should be watching too.
Maybe it's your favorite players and what's going on behind
the scenes with them, to even introducing I want to say,
(07:11):
the casual fans to who they should be watching, why
they should follow certain players, and more bringing that player's
story to life.
Speaker 2 (07:19):
Yeah.
Speaker 4 (07:19):
I mean, I feel like almost the whole point of
sports is to like create stories for us to follow, right,
Like they're engineered to be stories. It's exactly, this thing
is happening in front of you and there are two
antagonists and the stakes are high, and you don't know
how it's gonna end, Like it's built to be a story.
Speaker 5 (07:37):
Yeah, And that's the main challenge of the job is
you can plan, plan, plan, but once you get on
two players on court and you don't know what that
outcome is going to be, it's now sitting and waiting
and watching and you become a fan yourself. And then
it's how do you really captivate that story and how
do you narrate it and how do you like translate
up to fans.
Speaker 4 (07:55):
And it's like you kind of have to do it
in real time, right, Like the whole point of sports
is you don't know what's gonna.
Speaker 5 (07:59):
Happen exactly, and that's the excitement. And it's also there's
so many different types of fans. You know, there's the
fans who want a lot of enriched data and their
tennis nerds for lack of better of saying it, and
that they really want to dive deep into the intricacies
of the game, versus the casual fan who maybe just
wants more of this high level storyline of what does
this mean? Why is it important? So it's really trying
(08:20):
to figure out how to deliver that at scale and
really help fans get what they're looking for and the
type of content they're looking for.
Speaker 4 (08:27):
So are there specific examples of you know how fan
feedback has led to specific features digital features you build.
Are there, like particularly popular features you've come up with, Like,
what are some specifics.
Speaker 5 (08:40):
Yeah, some low hanging fruit type things that came from
fan feedback. Is simple things sometimes like managing time zones
and when matches start.
Speaker 4 (08:47):
A persistent problem where those of us can work across
times exactly.
Speaker 5 (08:52):
And we do have, like I've mentioned, twenty plus courts
happening at a time, So it's a lot to follow,
and how do you translate that to a fan, whether
it's to their native language or to their time zone
or things like that. So that's one thing that came
through fan feedback, and another one a three to five
hour match, especially when you're having twenty plus of them
happening at a time, is there's too much for one
(09:12):
person to follow. So how do you start from an
editorial perspective really helping with that storytelling and guiding a
fan to like, all right, whether there's an upset about
to happen, or here's your matches to watch, or even
some of the predictions we're starting to put in, is
we really want to guide the fan before a match,
here's where you should tune in to even after a
(09:32):
match of here's what's happened, Here's what's important, and we're
really excited with some of the features we've built in
the last few years that I would say really helps
us do that at more scale than what we were
able to do with just writers following a match and
covering every single match.
Speaker 4 (09:45):
Huh. So I want to talk a little bit about
the partnership between IBM and the USTA, Like, just tell
me about the work you do together.
Speaker 5 (09:56):
So IBM is our official digital and technology partner and
innovation the US Open they predate me. It's a thirty
year partnership and it truly as a partnership. So I
view the IBM consulting team as an extension of my
USTA team, So we work with them year round. They design,
develop and deliver the digital properties. They help us provide
(10:17):
the tools to create content to do things at scale,
They help us from stats and information and really help
us push from an innovation standpoint to make sure that
we are staying on that cutting edge of technology. So
I would truly say it's much more than a sponsorship,
where it's truly a partnership to deliver that fan experience.
Speaker 4 (10:34):
And so What are some of the specific things that
you have done with IBM.
Speaker 5 (10:39):
Yeah, so, I mean there's countless ones to talk through. Obviously,
they thirty years ago. They helped us build our first
website and it's kind of grown from there over the
past few years. I would say, I think it was
twenty eighteen as we started AI Highlights, So that was
really when we were able to have all twenty matches
going at a single time. We were able to quickly
(11:00):
deliver succinct highlights to fans to our digital platform so
they could see highlights for every single core.
Speaker 4 (11:06):
Is that video highlights? Is that tech summaries? What does
that mean?
Speaker 5 (11:10):
At the time, it was video highlights, Okay, So it
was really taking that three to five hour match, let's say,
and cut it down to a three minute highlight that
could show up within moments after a match, ending to
our website and our mobile app, so fans could see
that all around the world and really kind of get
that three minute overview what happened in a match, and.
Speaker 4 (11:28):
Was that AI enabled? Was AI a piece of how
to do that?
Speaker 5 (11:32):
It was? It was probably our first foray into AI.
Speaker 4 (11:35):
Back twenty eighteen is relatively early, Yeah, exactly, for tennis exactly.
Speaker 5 (11:42):
Yeah. It really, I want to say, opened up our
ability to one again storytell but attract new fans too.
Is video has actually been our number one growth area
since twenty eighteen, and I think a lot of that
has to do with the scale of how we deliver
that content.
Speaker 4 (11:56):
Using AI and being able to deliver this sort of
video highlight reels at scale.
Speaker 5 (12:02):
Yeah, and do it quickly. Right. We've always had highlights,
but it was a manual process where you had a
video that or cutting through you know, a three hour match,
selecting the right scene, stitching together, it would have to
get voiced over, et cetera. We really have used AI
to make it, i want to say, much more efficient
and speed up that process and deliver it more quickly
to our fans.
Speaker 4 (12:22):
I mean, it would be a bummer to get scooped
by whatever NBC News or Yes Pen or whatever. I'm
sure there are all your partners and you love that most.
Obviously you want to have the video first, right, it's
your match.
Speaker 5 (12:33):
Yeah, And I think it's also important to us as
being the USTA is ensuring that it's not just you know,
the main marquee players, that every player and all those
storylines and that whether it's you know, the main singles
draw to our mixed doubles, et cetera. They all need
highlights and they all have their own stories to tell,
and how do we do that at scale? It was
(12:53):
something that before we had that product was not something
you were able to do.
Speaker 4 (12:57):
Great, So let's let's talk in some more detail about
what you're working on. Let's start with the app. Tell
me about the us Open app and the Companion website.
Speaker 5 (13:07):
Yeah, so I'll start with the app, and I feel
like they serve similar needs, but they're a little different
in their own respective manners. Is the app. Everybody has
a phone in their hands at this point. The app
is kind of their guide to when I say a
million fans on site, we view the app as we
want that to be their on site guide, and Companion
a million.
Speaker 4 (13:27):
Let's just pause on a million fans on site, right,
because like a big professional whatever, an NFL game or
something that's like one hundred thousand, this is ten x that.
Speaker 5 (13:37):
Yeah, and a three week window and a very succinct,
tight action packed window. There's a lot of action logistics.
Speaker 4 (13:45):
Okay, so keep going.
Speaker 5 (13:46):
So the app, you know, whether it's finding the schedules,
the live scores, what's happening on court. That's really the
focus point of the app, and what we're really focused
on this year is how do we build in some
of those map summaries into the app, into our slam
Tracker experience. So again, before match, that kind of match
preview of here's maybe if you have a ticket, here's
what to expect, here's you know are likely to win,
(14:09):
who we are predicting, so you can kind of get
some information heading in, and then after the match it's
more of what just happened, what it means for the
rest of the draw, who they're playing next, is this
the first time this has happened, et cetera, and really
enriching that experience as well. So the app is one
your guide to what you should be watching, but also
then giving you that insights and context of what's happening
(14:32):
on that court as you're.
Speaker 4 (14:32):
Watching, like the commentator in your pocket exactly. So you
used a phrase in there as if I already knew it,
and I love the phrase, but I want you to
talk more about it. That phrase is slam Tracker.
Speaker 5 (14:45):
Yes, So slam Tracker is our long standing live scores
I want to say Match Center. It is, okay, where
every single data point for every single match lives, and
it really it helps showcase what's happening to match. I say,
it's our brock Cast companions. If you're watching live, it's
our in stadium companion. And it's also the best thing
to have if you aren't able to watch.
Speaker 4 (15:06):
And so, like, I'm on the app and there's a
thing called slam Tracker, and it like taps slam Tracker.
What do I see on my phone when I tap
slam Tracker? You know, midday when the tournament's happening.
Speaker 5 (15:16):
So before match, that's where you get a lot of
pre match content. That's where those live kind of our
predictions are likelihood to win lives within that So likelihood
to win essentially pulls in a bunch of data points.
So pass matches, how many times these players have played
each other against each other, even some punditry and other
written articles that maybe our editorial team put out and
really kind of puts a prediction out there.
Speaker 4 (15:38):
And so it's just a percentage chance.
Speaker 5 (15:40):
Yes exactly, but it uses millions of data points to
come up with that. Yes, So it really helps you
kind of understand what you're getting into for that match.
During a live match, it is every single point, so
point by point scoring as well as in depth analysis
in point commentary where also this year have a live
visualization that accompanies that will really help bring the match together.
(16:02):
And what I mean by that is it uses our
ball tracking technology to really showcase the match in near
real time, so within seconds delay of where the ball's
being hit, where the players are, and really bring a
visualization to life and layered stats and data on top
of it.
Speaker 2 (16:16):
Huh.
Speaker 4 (16:16):
Is that sort of like when I'm watching a match
on TV and there's like a close call as the
ball in or out and they do that thing where
they kind of show a sort of video game version
of where the ball landed. Does it look like that?
Speaker 5 (16:27):
It's like that before every single shot, So it's not
just those close ones. It's our first foray to bring
that match to life.
Speaker 4 (16:34):
Huh. And so what do I see on that kind
of view that I don't see from whatever watching the video?
Speaker 5 (16:38):
Yeah, So one you'll be able just to see more
of the ball trajectory and where the ball is being hit,
but then you can also start layering things in stats
and insights on top of that, so how many times
has player A hit the ball on a certain baseline,
how fast are they hitting it, maybe their serve percentage
and a certain side of the court, et cetera. So
you can really start layering in for the ones that
(16:59):
really want to. I've deep into the for the nerds.
Speaker 4 (17:01):
It's for the information rich exactly.
Speaker 5 (17:05):
It's the strategy of tennis. It really should be an
interesting way to slice and dice a match.
Speaker 4 (17:10):
Huh.
Speaker 3 (17:11):
It's remarkable how the USTA is leveraging AI to enhance
fan engagement and deliver immersive experiences both on site and online.
Brian's emphasis on storytelling really underscores the evolution of sports marketing.
The slam Chakra feature particularly caught my attention. It's essentially
bringing the excitement of a tennis match to life in
(17:33):
your palm, moment by moment. As someone who appreciates the
narrative intricacies of sports, I find it compelling how AI
helps predict and analyze matches in real time.
Speaker 4 (17:46):
Tell me about the AI commentary feature.
Speaker 5 (17:49):
Yeah, I know. I mentioned AI highlights back in twenty eighteen.
It's now progressed for us. And again, if we go
back to before we had a highlights to have a
highlight ready for this was a video editor cutting the
highlight and getting voiced over and then being published aside,
and it took probably an hour plus for that highlight
(18:09):
to really be created. Now with AI commentary, not only
are we creating and cutting the highlights using our AI technology,
but it's now using all the data points that we
have around the match, whether it's our live scoring data,
our ball tra directory data, etc. And it's really creating
a script that helped storytell around that match. That's all
using Watson X technology and then using text to speech
(18:32):
we're able to actually then create the commentary on top
of that, which all happens now within minutes. So our
team's able to now create fully voiced highlights for every
men's and women's singles match to our site within minutes.
Speaker 4 (18:44):
So I know there's a new feature you're working on
for this year called match reports. What are match reports?
Speaker 5 (18:52):
It's our ability to succicktly tell the story of a match,
so everything happens in five hours within that match down
to a couple paragraphs that really helps a user understand
or a fan understand what just happened. Again, some key
stats what's upcoming really help us with that storytelling. In
the past, when we have twenty two courts happening at
(19:14):
a certain time, we would have to pick and choose
which stories we think or which matches we think are
going to have the best stories, and that's a really
hard thing to predict from an editorial perspective. With our
match reports now we'll be able to have full coverage
of every single match during the main draw.
Speaker 4 (19:29):
So, of course I want to talk about jeneritive AI.
How could we not talk about generative Of course, what
are you working on with jenitive AI?
Speaker 5 (19:36):
So match reports is the prime example of it. So
Match Reports will be completely using Watson next genera of
AI technology, And really again to us, it's how can
we do that storytelling at scale? Tennis is such a
data rich sport. All sports have data, but tennis has
a lot of shots and different shot types and ball
trajectory and live scoring data and umpire chair data and
(20:00):
and all that. Factoring in jeneral of AI really helps
us take some of that structured and unstructured data really
one organize it in a way, but then help us
quickly tell that story at scale to all of our fans,
and I think we're really just starting to scratch at
some of the capabilities, and we're really excited about where
we're being, but we also see the opportunity of even
(20:22):
how we can grow to new fans and new fans
around the world using jener of AI in the future.
Speaker 4 (20:28):
So I'm curious, and you alluded to this a moment ago,
but I'd like to talk a little bit more about
it because it seems interesting as a technical problem. Right,
is the nature of turning tennis matches into stories, which
is fundamentally what we're talking about here in different ways
in different media, is about taking both structured data, right
(20:51):
like the stats who you know, points stats matches, and
also unstructured data, right like commentary and articles and the
kind of fuzzier parts of storytelling. And so I'm curious
how AI kind of helps you manage both the structured
and the unstructured data.
Speaker 5 (21:08):
Yeah. So, I think the structured data is pretty self experimentatory,
but when you get into the unstructured data and some
of the punditry, that's where you get more of the
opinion pieces into it. Like a specific player matchup, this
player always plays well against so and so, or as
they play always played well at night, or they're a
fan favorite and the crowd, you know, adrenaline and the
crowd being behind you can really motivate you to play
(21:30):
a lot better. So it pulls in all those unstructured
pieces and helps us really put some more rigor around
it and help add and enrich our storytelling with it.
Speaker 4 (21:39):
And so I'm curious when you're starting to use generative AI,
you know, over the past few years, like, what were
your concerns going into that.
Speaker 5 (21:48):
I think our biggest concern is ensuring that one factually
it is correct, because it's only as good as the
data you feed in. And how do you really ensure
that your model's working right and that the output and
the data you're feeding it matches the output, and how
do you do that at scale? So we do have
a lot of human intervention. That's where the IBM consulting team,
they're on site with us for those full three weeks
(22:10):
really helping us review everything and we're constantly learning, especially
early in the tournament. And I would say the other
big concern, again it goes around to the data, is
what data do we have available that is trustworthy? So
you know, we are feel very confident with the data
that comes off of court, but when we get into
that unstructured piece, what are the right data sources? How
do we validate those data sources and how do we
(22:32):
ensure that they're accurate Because if the data that has
to go in has to be accurate for the for
the output.
Speaker 4 (22:38):
So how do you do that? That's the concern? How
how do you address it?
Speaker 5 (22:41):
Yeah, so I think there's there's a number of tools
that we use all within the Watson X umbrella. We
do a lot of training with the IBM team, so
we have to constantly train and retrain that model. I
think the other piece that we're doing is again as
we're creating that content and we have the IBM consulting
team on site helping us with that, is as we
see things and we see outputs, it's refeeding that back
(23:04):
into the model to make it better for the next time.
So it's a constantly learning process that we're undergoing.
Speaker 4 (23:11):
So I want to talk about scale. Yes, you have
like what twenty two different courts with matches going all
at the same time. You're trying to, you know, approximately
instantly generate summaries of all these matches in something like
real time, and I'm curious in particular how the IBM
(23:31):
models you're using, the IBM Granite models are helping you scale.
Speaker 5 (23:35):
Yeah. So I think one of the big learnings we
had with IBM granted models too is that we're able
to run it, you know, against last year's tournaments and
see what the expected outputs could be and really help
train that model heading into the tournament. Because as we
talked about in the beginning, is we can plan, plan,
and plan, but once two players get on court, the
outcome is unknown. So how do we really run it
(23:58):
through its paces and really make sure that whatever that
outcome could be and whatever that scenario is, whether it's
a fifth set tie break that happens, or maybe there's
a you know, a fault in the match or something
that we're not anticipating, that we have that accounted for
and that the a won't throw off that output. So
we really try to think through every scenario, which is
(24:19):
sometimes difficult, right because again live sports is the unknown
is the unknown that's what makes it fun. We do
spend a lot of time thinking through potential scenarios and
ensuring that we have the right data sets and the
model to predict that tell.
Speaker 4 (24:32):
Me about match reports and the generative AI model you're
using for that.
Speaker 5 (24:37):
Yeah, so match reports will be new for us this year,
So we're in testing right now, so we're really excited
around it. But the model that we'll be able to
use using Watson X will use a bunch of different
parts of the suite of tools A meaning that again
of taking some of that punditry and the unstructured data
and the editorial spend, take our structured data as well.
(24:57):
And really what we're working on right now is figuring
out the right prompts for the AI to really ensure
that it tells the right structured story, meaning what just happened. Right,
So our recap is pretty standard. Here's what the data
is telling us, who won, who lost, how many sets?
Speaker 4 (25:14):
Here's the score the structured data part, that's the easy part.
Speaker 5 (25:17):
Yeah, and then really where it gets exciting is then
what does this mean? Meaning what's upcoming? So there's all
these different scenarios when you get into you know, two
hundred and fifty four players and a large draw. This
allows us to distill that down and really tell kind
of what could happen upcoming. The AI helps us do
that at scale.
Speaker 4 (25:35):
So I want to sort of generalize for a moment
to talk about kind of you know, broader challenges with
AI and how you've solved them. You know a lot
of generative AI pilots fail because the data quality isn't
high enough, because the risk controls aren't there, and so
I'm curious how you dealt with those problems and are
(25:56):
dealing with them data quality.
Speaker 5 (25:58):
Again, we feel calm with the data that is supplied
from the US open and from the USTA. Right, So
we have again that's our structure, scoring data and all that.
I think what we're constantly looking at is when we
get outside of our known sources and out to third
parties is that's where a lot of the testing and
model work happens. So we pull in different data sources
(26:19):
and really try to work through how it changes that output. Again,
some of that comes down to where it's an open
model and the transparency that we have and the learning
that comes behind it. That's where a lot of that
confidence can come from, and it comes from a lot
of testing and feeding it more data. Your second question
was a little bit more around the output I believe.
Speaker 4 (26:38):
Right, Yeah, and risks right, So risk, I think of
risk more in terms of output, right, But the obvious
sphere is like what if it says something wrong? Yeah,
inflammatory or whatever like that seems scary?
Speaker 5 (26:51):
Yeah, it definitely is, and it's definitely one of our
largest concerns when we first took this. FORAY, I would
say a lot of that comes through our work with
IBM and I consulting team and really ensuring that again
they're an extension and the partnership there of our team.
So whenever we are creating let's say it's the match Report,
and we're going to be creating these excinct articles for
every single men's and women's single match that happens, is
(27:15):
all of those will have manual review and people looking
through them for accuracy to ensure that the model then
hallucinate or make up a factor or fill in the
gaps from things like that. That's the first step. And
then also when our editorial team goes to publish those
of the website, they're going to be checking it as well.
So there are manual interventions throughout that to really check
that model. But we feel that the ability to do
(27:37):
it at scale and with us more to check that
is the efficiency problem that we've been looking to solve.
Speaker 4 (27:43):
So the USTA and IBM have been working together on
digital innovation for like thirty years from you know, the
first website, yes for the USTA until now. So that's
the past thirty years. If you look ahead, what's the next.
Speaker 5 (27:57):
Thirty thirty years is a really long time?
Speaker 2 (28:00):
Agree?
Speaker 6 (28:01):
Yeah, I think you know where I get excited, and
I think I alluded to it in the beginning about
how I feel like we're just scratching at the surface,
especially with Journati of Ai, and where I see it
going is there's a lot of different fans out there,
and we're also very kindness in the us OP and
that we're a worldwide event, and that there's a lot
of different fans that were not necessary creating content for
(28:23):
bespoke meaning in their native language or maybe it's in
that native players language and things like that.
Speaker 5 (28:29):
Is where I get excited is we've seen immense growth
with a Highlights and the ability to now do highlights
at scale. Is the ability for us to start creating
content in different languages, maybe covering different parts of the match.
So maybe you do have that stats junkie you really wants,
just it's the fastest serve and here's the deep insights
versus the casual fan who's looking for more of the
(28:50):
storytelling around how a player trains and what leading up
to it was like and what it means for them
afterwards and things like that. A lot of that takes
a lot of time. Now we're able to solve that
efficiency problem and do it in multiple languages, we can
really create I want to say, personalized content to a
lot more fans all around the world, which again helps
(29:11):
us grow the sport of tennis great.
Speaker 4 (29:14):
So I want to finish with a speed round. Okay,
are you ready?
Speaker 5 (29:18):
I am ready?
Speaker 4 (29:19):
Okay, first thing that comes to mind, complete this sentence.
In five years, AI will.
Speaker 5 (29:26):
Transform many parts of the business.
Speaker 4 (29:29):
What is the number one thing that people misunderstand about AI?
Speaker 5 (29:34):
That it's supplemental, not replacing, meaning that it helps it
with efficiencies, but it doesn't necessarily replace the creativity.
Speaker 4 (29:43):
Right now, what advice would you give yourself ten years
ago to better prepare you for today?
Speaker 5 (29:50):
I think it would have been, especially now that we're
able to take so much of that unstructured data and
pass content that we were created to help tell st
was to I want to say archive more of that
in a way that we could be using that to
help pull from that now. So you know, we've seen
kind of a change in the guard from some of
(30:11):
our star players to now new and up and comers,
and it would be really fascinating to me if there
was a way to to cross sections some of that
and saying like what tra directories are certain up and
coming players maybe filing from others. So it's more I
wish we kept more of the content.
Speaker 4 (30:27):
We created back fave the data exactly. Well are you
saving it all now?
Speaker 5 (30:33):
Oh yeah, one hundred percent learned our lesson?
Speaker 2 (30:35):
Yes, yes.
Speaker 4 (30:37):
So on the business side of AI, what do you
think is the next big thing?
Speaker 5 (30:41):
I alluded to it earlier. I think it's personalization and
getting content that's catered to you at scale, whether you
know that's across the sports sphere or or any type
of written content or or news content. I feel like
the ability to really get contentated to the type of
fan you are and the insight you have is where
(31:01):
we're all headed.
Speaker 4 (31:03):
And in terms of your non work life, how do
you use AI day to day?
Speaker 5 (31:09):
It's funny, I was just having this conversation with a
friend the other day, and we were talking about sometimes
when you're starting something new, the hardest thing to do
is you have a blank piece of paper or a thought,
and how do you get started. Sometimes with these generative models,
the easiest thing and the best thing you can do
is it helps you get started. Meaning it may not
be one hundred percent with that first prompt, but it's
(31:30):
that efficiency of whether it's an outline for a new idea,
or it's a marketing brief you have to write, or
sometimes even if it's an email you have to write
for a personal something and you're not sure how to
word it the right way. It allows you to have
a start and then you can edit from there. So
again going back to my efficiency point, it helps you
become more.
Speaker 4 (31:49):
Efficient, solves the blank page problem.
Speaker 5 (31:51):
It does.
Speaker 4 (31:53):
Brian, it was great to talk with you. Thank you
so much for your time.
Speaker 5 (31:56):
Yeah, this was fun. Thanks for having me.
Speaker 3 (32:00):
Huge thanks to Jacob and Brian for the deep dive
into the cutting edge innovations transforming the game of tennis.
Brian shed light on how the US opens partnership with
IBM is harnessing data driven insights to reshape storytelling in sports,
from AI generated commentary to Match reports. As we look ahead,
(32:21):
I'm excited about the possibilities for personalizing content and reaching
fans in new ways. The future of AI promises more
than just efficiency. It's about enhancing fan experiences worldwide. Smart
Talks with IBM is produced by Matt Romano, Joey Fishground,
(32:42):
and Jacob Goldstein. We're edited by Lydia jen Kott. Our
engineers are Sarah Bruger and Ben Tolliday. Theme song by Gramascow.
Special thanks to the eight Bar and IBM teams, as
well as the Pushkin marketing team. Smart Talks with IBM
is a production of Pushkin Indie and Ruby Studio at iHeartMedia.
(33:03):
To find more Pushkin podcasts, listen on the iHeartRadio app,
Apple Podcasts, or wherever you listen to podcasts. I'm Malcolm Gladwell.
This is a paid advertisement from IBM. The conversations on
this podcast don't necessarily represent IBM's positions, strategies, or opinions.
Hey everyone, it's Robert and Joe here. Today we've got
something a little bit different to share with you. It
is a new season of the Smart Talks with IBM
podcast series.
Speaker 2 (00:09):
Today we are witnessed to one of those rare moments
in history, the rise of an innovative technology with the
potential to radically transform business and society forever. The technology,
of course, is artificial intelligence, and it's the central focus
for this new season of Smart Talks with IBM.
Speaker 1 (00:25):
Join hosts from your favorite Pushkin podcasts as they talk
with industry experts and leaders to explore how businesses can
integrate AI into their workflows and help drive real change
in this new era of AI. And of course, host
Malcolm Gladwell will be there to guide you through the
season and throw in his two cents as well.
Speaker 2 (00:43):
Look out for new episodes of Smart Talks with IBM
every other week on the iHeartRadio app, Apple Podcasts, or
wherever you get your podcasts, and learn more at IBM
dot com slash smart talks.
Speaker 3 (00:57):
Hello, Hello, Welcome to Smart Talks with IBM, a podcast
from Pushkin Industries, iHeartRadio and IBM. I'm Malcolm Gladwell. This
season we're diving back into the world of artificial intelligence,
but with a focus on the powerful concept of open
its possibilities, implications, and misconceptions. We'll look at openness from
(01:17):
a variety of angles and explore how the concept is
already reshaping industries, ways of doing business, and our very
notion of what's possible. I'm particularly excited for today's guest,
Brian Ryerson. He's Senior Director of Digital Strategy at the
US Tennis Association, helping to oversee one of the most
(01:38):
iconic events in the world of sports, the US Open.
Brian sat down with Pushkin's own Jacob Goldstein, host of
the podcast What's Your Problem. A veteran business journalist, Jacob
has reported for The Wall Street Journal, the Miami Herald,
and was a longtime host of the NPR program Planet Money.
IBM has been the official technology partner of the US
(02:01):
Tennis Association for more than thirty years, and the more
recent evolution into generative AI has enhanced the world class
digital experiences that help more than fifteen million fans from
all over the world enjoy the US Open Tennis Championships.
In this episode, we will explore how generative AI is
(02:22):
being used to generate match insights, spoken commentary for match highlights,
and postmatch summaries at scale for fans to enjoy through
the US Open app and website. We'll explore how these
AI solutions enable the editorial team to cover more of
the tournament than ever before, bringing fans even closer to
(02:43):
the game they love, and will learn more about one
of the engines behind this AI powered content creation, a
large language model from the ib M Granite family, which
is trained and maintained using the watsonex AI and data platform. Okay,
let's dive in.
Speaker 4 (03:02):
Brian, Welcome to the show.
Speaker 5 (03:03):
Thanks for having me. I'm excited to be here.
Speaker 4 (03:05):
Can you say your name and your job?
Speaker 5 (03:07):
Yeah, I'm Brian Ryerson. I'm Senior director of Digital Strategy
at the USTA.
Speaker 4 (03:11):
Some question, what's the USTA.
Speaker 5 (03:13):
The US Tennis Association.
Speaker 4 (03:16):
And tell me about the USTA, Like, what is it?
Speaker 5 (03:19):
Yeah? So the USTA is the governing body of tennis
in the US. Or mission is to grow the sport
of tennis across the US at all levels. Really, I
would say we're more like a health and wellness company
where tennis is the means to health and wellness. And
then the US Open is kind of our tenth pole
event that happens everyear and Flushing Meadows and is really
our chance to showcase the support of tennis at its
(03:41):
highest level to fans all around the world.
Speaker 4 (03:43):
Yeah, I mean the US Open. I assume most people
know this, but it's Grand Slam. It's one of the
what four biggest tennis tournaments in the world.
Speaker 5 (03:50):
Yes, yeah, every year, we especially the past couple of years,
we've seen immense growth and you know, we are very
hopeful this year and our big goals that have over
a million fires on site during the three week window
this year, So it's an amazing event. I always say
it's a food and wine festival where tennis is the
main attraction and it's a really fun, unique atmosphere.
Speaker 4 (04:10):
How did you get into the tennis business.
Speaker 5 (04:12):
It's a great question. It's not where I thought i'd
end up for especially being there for fourteen years. So
I was a marketing and technology major in school, and
I also played college lacrosse and sports was always a
big part of my life and always wanted to be
in the sports and entertainment world. I'm here from the
New York area. This is where I grew up so
I moved back home and had a few friends who
(04:32):
worked there, and I started out more on the number
side of things and really digital analytics. It was really
the start of when Facebook and Twitter is just starting
and digital marketing and all of that. And you know,
I went to my first year so Open not really
knowing what to expect, and again, I think the atmosphere
kind of captivated me and hooked me in. And I've
been there now fourteen years.
Speaker 4 (04:53):
And so your title is Digital Director. What does that mean?
What's your job?
Speaker 5 (04:58):
Yeah, so it's interesting one because it's tough to explain
to folks who are not in the weeds on all
things US open or even in the sports world. But
really I oversee all of our consumer facing digital property.
So that's the us open dot org, our website built
by IBM, as well as our mobile app. I oversee
our content strategy, our sponsorship integrations. So really anything consumer
(05:21):
facing that happens on the web is under my purview,
even some of our new platform extensions and gaming and
things like that. Anything that you can physically interact with
is kind of under my purview.
Speaker 4 (05:33):
And so you've been there now for fourteen ISSU years,
which in the digital world is a long time. How
has that sort of digital experience of sports changed over
that time.
Speaker 5 (05:46):
Yeah, it's obviously grown digital now, is what we say
and what my team says. It's the number one way
to engage with fans that can't make it to the event,
as well as those fans who are at the event,
and how to enrich their stay. So it's really kind
of you're tackling multiple personas. It's the international fan who's
staying up late to watch in other countries, to the
fan here who's maybe watching on broadcasts, and we go
(06:07):
in a company and enrich that broadcast with new stats
and insights. To the on site fan who bought a
ticket and maybe doesn't know what match is happening on
what court. We do have twenty plus courts happening at
a time, with all different matches, So we really try
to help all fans navigate the US Open the best
way possible.
Speaker 4 (06:24):
And so, like, what are some of the sort of
problems you're trying to solve. What are some of the
hard things about your job?
Speaker 5 (06:30):
Yeah, obviously technology changes at a rapid pace, right, So
I think part of it is how do we stay
on the forefront of that, and how do we do
that in the best way and make the best fan
experiences possible and the best user experiences possible. That's always
kind of driving factor number one. Then number two, it's
understanding and listening to our fans and what kind of
(06:51):
content they want. You hear me talk a lot about storytelling.
I feel like there's a lot of storytelling that happens
around the US open that we really want to bring
to fans. And that can be as simple as storytelling
of what's happening today and what you should be watching too.
Maybe it's your favorite players and what's going on behind
the scenes with them, to even introducing I want to say,
(07:11):
the casual fans to who they should be watching, why
they should follow certain players, and more bringing that player's
story to life.
Speaker 2 (07:19):
Yeah.
Speaker 4 (07:19):
I mean, I feel like almost the whole point of
sports is to like create stories for us to follow, right,
Like they're engineered to be stories. It's exactly, this thing
is happening in front of you and there are two
antagonists and the stakes are high, and you don't know
how it's gonna end, Like it's built to be a story.
Speaker 5 (07:37):
Yeah, And that's the main challenge of the job is
you can plan, plan, plan, but once you get on
two players on court and you don't know what that
outcome is going to be, it's now sitting and waiting
and watching and you become a fan yourself. And then
it's how do you really captivate that story and how
do you narrate it and how do you like translate
up to fans.
Speaker 4 (07:55):
And it's like you kind of have to do it
in real time, right, Like the whole point of sports
is you don't know what's gonna.
Speaker 5 (07:59):
Happen exactly, and that's the excitement. And it's also there's
so many different types of fans. You know, there's the
fans who want a lot of enriched data and their
tennis nerds for lack of better of saying it, and
that they really want to dive deep into the intricacies
of the game, versus the casual fan who maybe just
wants more of this high level storyline of what does
this mean? Why is it important? So it's really trying
(08:20):
to figure out how to deliver that at scale and
really help fans get what they're looking for and the
type of content they're looking for.
Speaker 4 (08:27):
So are there specific examples of you know how fan
feedback has led to specific features digital features you build.
Are there, like particularly popular features you've come up with, Like,
what are some specifics.
Speaker 5 (08:40):
Yeah, some low hanging fruit type things that came from
fan feedback. Is simple things sometimes like managing time zones
and when matches start.
Speaker 4 (08:47):
A persistent problem where those of us can work across
times exactly.
Speaker 5 (08:52):
And we do have, like I've mentioned, twenty plus courts
happening at a time, So it's a lot to follow,
and how do you translate that to a fan, whether
it's to their native language or to their time zone
or things like that. So that's one thing that came
through fan feedback, and another one a three to five
hour match, especially when you're having twenty plus of them
happening at a time, is there's too much for one
(09:12):
person to follow. So how do you start from an
editorial perspective really helping with that storytelling and guiding a
fan to like, all right, whether there's an upset about
to happen, or here's your matches to watch, or even
some of the predictions we're starting to put in, is
we really want to guide the fan before a match,
here's where you should tune in to even after a
(09:32):
match of here's what's happened, Here's what's important, and we're
really excited with some of the features we've built in
the last few years that I would say really helps
us do that at more scale than what we were
able to do with just writers following a match and
covering every single match.
Speaker 4 (09:45):
Huh. So I want to talk a little bit about
the partnership between IBM and the USTA, Like, just tell
me about the work you do together.
Speaker 5 (09:56):
So IBM is our official digital and technology partner and
innovation the US Open they predate me. It's a thirty
year partnership and it truly as a partnership. So I
view the IBM consulting team as an extension of my
USTA team, So we work with them year round. They design,
develop and deliver the digital properties. They help us provide
(10:17):
the tools to create content to do things at scale,
They help us from stats and information and really help
us push from an innovation standpoint to make sure that
we are staying on that cutting edge of technology. So
I would truly say it's much more than a sponsorship,
where it's truly a partnership to deliver that fan experience.
Speaker 4 (10:34):
And so What are some of the specific things that
you have done with IBM.
Speaker 5 (10:39):
Yeah, so, I mean there's countless ones to talk through. Obviously,
they thirty years ago. They helped us build our first
website and it's kind of grown from there over the
past few years. I would say, I think it was
twenty eighteen as we started AI Highlights, So that was
really when we were able to have all twenty matches
going at a single time. We were able to quickly
(11:00):
deliver succinct highlights to fans to our digital platform so
they could see highlights for every single core.
Speaker 4 (11:06):
Is that video highlights? Is that tech summaries? What does
that mean?
Speaker 5 (11:10):
At the time, it was video highlights, Okay, So it
was really taking that three to five hour match, let's say,
and cut it down to a three minute highlight that
could show up within moments after a match, ending to
our website and our mobile app, so fans could see
that all around the world and really kind of get
that three minute overview what happened in a match, and.
Speaker 4 (11:28):
Was that AI enabled? Was AI a piece of how
to do that?
Speaker 5 (11:32):
It was? It was probably our first foray into AI.
Speaker 4 (11:35):
Back twenty eighteen is relatively early, Yeah, exactly, for tennis exactly.
Speaker 5 (11:42):
Yeah. It really, I want to say, opened up our
ability to one again storytell but attract new fans too.
Is video has actually been our number one growth area
since twenty eighteen, and I think a lot of that
has to do with the scale of how we deliver
that content.
Speaker 4 (11:56):
Using AI and being able to deliver this sort of
video highlight reels at scale.
Speaker 5 (12:02):
Yeah, and do it quickly. Right. We've always had highlights,
but it was a manual process where you had a
video that or cutting through you know, a three hour match,
selecting the right scene, stitching together, it would have to
get voiced over, et cetera. We really have used AI
to make it, i want to say, much more efficient
and speed up that process and deliver it more quickly
to our fans.
Speaker 4 (12:22):
I mean, it would be a bummer to get scooped
by whatever NBC News or Yes Pen or whatever. I'm
sure there are all your partners and you love that most.
Obviously you want to have the video first, right, it's
your match.
Speaker 5 (12:33):
Yeah, And I think it's also important to us as
being the USTA is ensuring that it's not just you know,
the main marquee players, that every player and all those
storylines and that whether it's you know, the main singles
draw to our mixed doubles, et cetera. They all need
highlights and they all have their own stories to tell,
and how do we do that at scale? It was
(12:53):
something that before we had that product was not something
you were able to do.
Speaker 4 (12:57):
Great, So let's let's talk in some more detail about
what you're working on. Let's start with the app. Tell
me about the us Open app and the Companion website.
Speaker 5 (13:07):
Yeah, so I'll start with the app, and I feel
like they serve similar needs, but they're a little different
in their own respective manners. Is the app. Everybody has
a phone in their hands at this point. The app
is kind of their guide to when I say a
million fans on site, we view the app as we
want that to be their on site guide, and Companion
a million.
Speaker 4 (13:27):
Let's just pause on a million fans on site, right,
because like a big professional whatever, an NFL game or
something that's like one hundred thousand, this is ten x that.
Speaker 5 (13:37):
Yeah, and a three week window and a very succinct,
tight action packed window. There's a lot of action logistics.
Speaker 4 (13:45):
Okay, so keep going.
Speaker 5 (13:46):
So the app, you know, whether it's finding the schedules,
the live scores, what's happening on court. That's really the
focus point of the app, and what we're really focused
on this year is how do we build in some
of those map summaries into the app, into our slam
Tracker experience. So again, before match, that kind of match
preview of here's maybe if you have a ticket, here's
what to expect, here's you know are likely to win,
(14:09):
who we are predicting, so you can kind of get
some information heading in, and then after the match it's
more of what just happened, what it means for the
rest of the draw, who they're playing next, is this
the first time this has happened, et cetera, and really
enriching that experience as well. So the app is one
your guide to what you should be watching, but also
then giving you that insights and context of what's happening
(14:32):
on that court as you're.
Speaker 4 (14:32):
Watching, like the commentator in your pocket exactly. So you
used a phrase in there as if I already knew it,
and I love the phrase, but I want you to
talk more about it. That phrase is slam Tracker.
Speaker 5 (14:45):
Yes, So slam Tracker is our long standing live scores
I want to say Match Center. It is, okay, where
every single data point for every single match lives, and
it really it helps showcase what's happening to match. I say,
it's our brock Cast companions. If you're watching live, it's
our in stadium companion. And it's also the best thing
to have if you aren't able to watch.
Speaker 4 (15:06):
And so, like, I'm on the app and there's a
thing called slam Tracker, and it like taps slam Tracker.
What do I see on my phone when I tap
slam Tracker? You know, midday when the tournament's happening.
Speaker 5 (15:16):
So before match, that's where you get a lot of
pre match content. That's where those live kind of our
predictions are likelihood to win lives within that So likelihood
to win essentially pulls in a bunch of data points.
So pass matches, how many times these players have played
each other against each other, even some punditry and other
written articles that maybe our editorial team put out and
really kind of puts a prediction out there.
Speaker 4 (15:38):
And so it's just a percentage chance.
Speaker 5 (15:40):
Yes exactly, but it uses millions of data points to
come up with that. Yes, So it really helps you
kind of understand what you're getting into for that match.
During a live match, it is every single point, so
point by point scoring as well as in depth analysis
in point commentary where also this year have a live
visualization that accompanies that will really help bring the match together.
(16:02):
And what I mean by that is it uses our
ball tracking technology to really showcase the match in near
real time, so within seconds delay of where the ball's
being hit, where the players are, and really bring a
visualization to life and layered stats and data on top
of it.
Speaker 2 (16:16):
Huh.
Speaker 4 (16:16):
Is that sort of like when I'm watching a match
on TV and there's like a close call as the
ball in or out and they do that thing where
they kind of show a sort of video game version
of where the ball landed. Does it look like that?
Speaker 5 (16:27):
It's like that before every single shot, So it's not
just those close ones. It's our first foray to bring
that match to life.
Speaker 4 (16:34):
Huh. And so what do I see on that kind
of view that I don't see from whatever watching the video?
Speaker 5 (16:38):
Yeah, So one you'll be able just to see more
of the ball trajectory and where the ball is being hit,
but then you can also start layering things in stats
and insights on top of that, so how many times
has player A hit the ball on a certain baseline,
how fast are they hitting it, maybe their serve percentage
and a certain side of the court, et cetera. So
you can really start layering in for the ones that
(16:59):
really want to. I've deep into the for the nerds.
Speaker 4 (17:01):
It's for the information rich exactly.
Speaker 5 (17:05):
It's the strategy of tennis. It really should be an
interesting way to slice and dice a match.
Speaker 4 (17:10):
Huh.
Speaker 3 (17:11):
It's remarkable how the USTA is leveraging AI to enhance
fan engagement and deliver immersive experiences both on site and online.
Brian's emphasis on storytelling really underscores the evolution of sports marketing.
The slam Chakra feature particularly caught my attention. It's essentially
bringing the excitement of a tennis match to life in
(17:33):
your palm, moment by moment. As someone who appreciates the
narrative intricacies of sports, I find it compelling how AI
helps predict and analyze matches in real time.
Speaker 4 (17:46):
Tell me about the AI commentary feature.
Speaker 5 (17:49):
Yeah, I know. I mentioned AI highlights back in twenty eighteen.
It's now progressed for us. And again, if we go
back to before we had a highlights to have a
highlight ready for this was a video editor cutting the
highlight and getting voiced over and then being published aside,
and it took probably an hour plus for that highlight
(18:09):
to really be created. Now with AI commentary, not only
are we creating and cutting the highlights using our AI technology,
but it's now using all the data points that we
have around the match, whether it's our live scoring data,
our ball tra directory data, etc. And it's really creating
a script that helped storytell around that match. That's all
using Watson X technology and then using text to speech
(18:32):
we're able to actually then create the commentary on top
of that, which all happens now within minutes. So our
team's able to now create fully voiced highlights for every
men's and women's singles match to our site within minutes.
Speaker 4 (18:44):
So I know there's a new feature you're working on
for this year called match reports. What are match reports?
Speaker 5 (18:52):
It's our ability to succicktly tell the story of a match,
so everything happens in five hours within that match down
to a couple paragraphs that really helps a user understand
or a fan understand what just happened. Again, some key
stats what's upcoming really help us with that storytelling. In
the past, when we have twenty two courts happening at
(19:14):
a certain time, we would have to pick and choose
which stories we think or which matches we think are
going to have the best stories, and that's a really
hard thing to predict from an editorial perspective. With our
match reports now we'll be able to have full coverage
of every single match during the main draw.
Speaker 4 (19:29):
So, of course I want to talk about jeneritive AI.
How could we not talk about generative Of course, what
are you working on with jenitive AI?
Speaker 5 (19:36):
So match reports is the prime example of it. So
Match Reports will be completely using Watson next genera of
AI technology, And really again to us, it's how can
we do that storytelling at scale? Tennis is such a
data rich sport. All sports have data, but tennis has
a lot of shots and different shot types and ball
trajectory and live scoring data and umpire chair data and
(20:00):
and all that. Factoring in jeneral of AI really helps
us take some of that structured and unstructured data really
one organize it in a way, but then help us
quickly tell that story at scale to all of our fans,
and I think we're really just starting to scratch at
some of the capabilities, and we're really excited about where
we're being, but we also see the opportunity of even
(20:22):
how we can grow to new fans and new fans
around the world using jener of AI in the future.
Speaker 4 (20:28):
So I'm curious, and you alluded to this a moment ago,
but I'd like to talk a little bit more about
it because it seems interesting as a technical problem. Right,
is the nature of turning tennis matches into stories, which
is fundamentally what we're talking about here in different ways
in different media, is about taking both structured data, right
(20:51):
like the stats who you know, points stats matches, and
also unstructured data, right like commentary and articles and the
kind of fuzzier parts of storytelling. And so I'm curious
how AI kind of helps you manage both the structured
and the unstructured data.
Speaker 5 (21:08):
Yeah. So, I think the structured data is pretty self experimentatory,
but when you get into the unstructured data and some
of the punditry, that's where you get more of the
opinion pieces into it. Like a specific player matchup, this
player always plays well against so and so, or as
they play always played well at night, or they're a
fan favorite and the crowd, you know, adrenaline and the
crowd being behind you can really motivate you to play
(21:30):
a lot better. So it pulls in all those unstructured
pieces and helps us really put some more rigor around
it and help add and enrich our storytelling with it.
Speaker 4 (21:39):
And so I'm curious when you're starting to use generative AI,
you know, over the past few years, like, what were
your concerns going into that.
Speaker 5 (21:48):
I think our biggest concern is ensuring that one factually
it is correct, because it's only as good as the
data you feed in. And how do you really ensure
that your model's working right and that the output and
the data you're feeding it matches the output, and how
do you do that at scale? So we do have
a lot of human intervention. That's where the IBM consulting team,
they're on site with us for those full three weeks
(22:10):
really helping us review everything and we're constantly learning, especially
early in the tournament. And I would say the other
big concern, again it goes around to the data, is
what data do we have available that is trustworthy? So
you know, we are feel very confident with the data
that comes off of court, but when we get into
that unstructured piece, what are the right data sources? How
do we validate those data sources and how do we
(22:32):
ensure that they're accurate Because if the data that has
to go in has to be accurate for the for
the output.
Speaker 4 (22:38):
So how do you do that? That's the concern? How
how do you address it?
Speaker 5 (22:41):
Yeah, so I think there's there's a number of tools
that we use all within the Watson X umbrella. We
do a lot of training with the IBM team, so
we have to constantly train and retrain that model. I
think the other piece that we're doing is again as
we're creating that content and we have the IBM consulting
team on site helping us with that, is as we
see things and we see outputs, it's refeeding that back
(23:04):
into the model to make it better for the next time.
So it's a constantly learning process that we're undergoing.
Speaker 4 (23:11):
So I want to talk about scale. Yes, you have
like what twenty two different courts with matches going all
at the same time. You're trying to, you know, approximately
instantly generate summaries of all these matches in something like
real time, and I'm curious in particular how the IBM
(23:31):
models you're using, the IBM Granite models are helping you scale.
Speaker 5 (23:35):
Yeah. So I think one of the big learnings we
had with IBM granted models too is that we're able
to run it, you know, against last year's tournaments and
see what the expected outputs could be and really help
train that model heading into the tournament. Because as we
talked about in the beginning, is we can plan, plan,
and plan, but once two players get on court, the
outcome is unknown. So how do we really run it
(23:58):
through its paces and really make sure that whatever that
outcome could be and whatever that scenario is, whether it's
a fifth set tie break that happens, or maybe there's
a you know, a fault in the match or something
that we're not anticipating, that we have that accounted for
and that the a won't throw off that output. So
we really try to think through every scenario, which is
(24:19):
sometimes difficult, right because again live sports is the unknown
is the unknown that's what makes it fun. We do
spend a lot of time thinking through potential scenarios and
ensuring that we have the right data sets and the
model to predict that tell.
Speaker 4 (24:32):
Me about match reports and the generative AI model you're
using for that.
Speaker 5 (24:37):
Yeah, so match reports will be new for us this year,
So we're in testing right now, so we're really excited
around it. But the model that we'll be able to
use using Watson X will use a bunch of different
parts of the suite of tools A meaning that again
of taking some of that punditry and the unstructured data
and the editorial spend, take our structured data as well.
(24:57):
And really what we're working on right now is figuring
out the right prompts for the AI to really ensure
that it tells the right structured story, meaning what just happened. Right,
So our recap is pretty standard. Here's what the data
is telling us, who won, who lost, how many sets?
Speaker 4 (25:14):
Here's the score the structured data part, that's the easy part.
Speaker 5 (25:17):
Yeah, and then really where it gets exciting is then
what does this mean? Meaning what's upcoming? So there's all
these different scenarios when you get into you know, two
hundred and fifty four players and a large draw. This
allows us to distill that down and really tell kind
of what could happen upcoming. The AI helps us do
that at scale.
Speaker 4 (25:35):
So I want to sort of generalize for a moment
to talk about kind of you know, broader challenges with
AI and how you've solved them. You know a lot
of generative AI pilots fail because the data quality isn't
high enough, because the risk controls aren't there, and so
I'm curious how you dealt with those problems and are
(25:56):
dealing with them data quality.
Speaker 5 (25:58):
Again, we feel calm with the data that is supplied
from the US open and from the USTA. Right, So
we have again that's our structure, scoring data and all that.
I think what we're constantly looking at is when we
get outside of our known sources and out to third
parties is that's where a lot of the testing and
model work happens. So we pull in different data sources
(26:19):
and really try to work through how it changes that output. Again,
some of that comes down to where it's an open
model and the transparency that we have and the learning
that comes behind it. That's where a lot of that
confidence can come from, and it comes from a lot
of testing and feeding it more data. Your second question
was a little bit more around the output I believe.
Speaker 4 (26:38):
Right, Yeah, and risks right, So risk, I think of
risk more in terms of output, right, But the obvious
sphere is like what if it says something wrong? Yeah,
inflammatory or whatever like that seems scary?
Speaker 5 (26:51):
Yeah, it definitely is, and it's definitely one of our
largest concerns when we first took this. FORAY, I would
say a lot of that comes through our work with
IBM and I consulting team and really ensuring that again
they're an extension and the partnership there of our team.
So whenever we are creating let's say it's the match Report,
and we're going to be creating these excinct articles for
every single men's and women's single match that happens, is
(27:15):
all of those will have manual review and people looking
through them for accuracy to ensure that the model then
hallucinate or make up a factor or fill in the
gaps from things like that. That's the first step. And
then also when our editorial team goes to publish those
of the website, they're going to be checking it as well.
So there are manual interventions throughout that to really check
that model. But we feel that the ability to do
(27:37):
it at scale and with us more to check that
is the efficiency problem that we've been looking to solve.
Speaker 4 (27:43):
So the USTA and IBM have been working together on
digital innovation for like thirty years from you know, the
first website, yes for the USTA until now. So that's
the past thirty years. If you look ahead, what's the next.
Speaker 5 (27:57):
Thirty thirty years is a really long time?
Speaker 2 (28:00):
Agree?
Speaker 6 (28:01):
Yeah, I think you know where I get excited, and
I think I alluded to it in the beginning about
how I feel like we're just scratching at the surface,
especially with Journati of Ai, and where I see it
going is there's a lot of different fans out there,
and we're also very kindness in the us OP and
that we're a worldwide event, and that there's a lot
of different fans that were not necessary creating content for
(28:23):
bespoke meaning in their native language or maybe it's in
that native players language and things like that.
Speaker 5 (28:29):
Is where I get excited is we've seen immense growth
with a Highlights and the ability to now do highlights
at scale. Is the ability for us to start creating
content in different languages, maybe covering different parts of the match.
So maybe you do have that stats junkie you really wants,
just it's the fastest serve and here's the deep insights
versus the casual fan who's looking for more of the
(28:50):
storytelling around how a player trains and what leading up
to it was like and what it means for them
afterwards and things like that. A lot of that takes
a lot of time. Now we're able to solve that
efficiency problem and do it in multiple languages, we can
really create I want to say, personalized content to a
lot more fans all around the world, which again helps
(29:11):
us grow the sport of tennis great.
Speaker 4 (29:14):
So I want to finish with a speed round. Okay,
are you ready?
Speaker 5 (29:18):
I am ready?
Speaker 4 (29:19):
Okay, first thing that comes to mind, complete this sentence.
In five years, AI will.
Speaker 5 (29:26):
Transform many parts of the business.
Speaker 4 (29:29):
What is the number one thing that people misunderstand about AI?
Speaker 5 (29:34):
That it's supplemental, not replacing, meaning that it helps it
with efficiencies, but it doesn't necessarily replace the creativity.
Speaker 4 (29:43):
Right now, what advice would you give yourself ten years
ago to better prepare you for today?
Speaker 5 (29:50):
I think it would have been, especially now that we're
able to take so much of that unstructured data and
pass content that we were created to help tell st
was to I want to say archive more of that
in a way that we could be using that to
help pull from that now. So you know, we've seen
kind of a change in the guard from some of
(30:11):
our star players to now new and up and comers,
and it would be really fascinating to me if there
was a way to to cross sections some of that
and saying like what tra directories are certain up and
coming players maybe filing from others. So it's more I
wish we kept more of the content.
Speaker 4 (30:27):
We created back fave the data exactly. Well are you
saving it all now?
Speaker 5 (30:33):
Oh yeah, one hundred percent learned our lesson?
Speaker 2 (30:35):
Yes, yes.
Speaker 4 (30:37):
So on the business side of AI, what do you
think is the next big thing?
Speaker 5 (30:41):
I alluded to it earlier. I think it's personalization and
getting content that's catered to you at scale, whether you
know that's across the sports sphere or or any type
of written content or or news content. I feel like
the ability to really get contentated to the type of
fan you are and the insight you have is where
(31:01):
we're all headed.
Speaker 4 (31:03):
And in terms of your non work life, how do
you use AI day to day?
Speaker 5 (31:09):
It's funny, I was just having this conversation with a
friend the other day, and we were talking about sometimes
when you're starting something new, the hardest thing to do
is you have a blank piece of paper or a thought,
and how do you get started. Sometimes with these generative models,
the easiest thing and the best thing you can do
is it helps you get started. Meaning it may not
be one hundred percent with that first prompt, but it's
(31:30):
that efficiency of whether it's an outline for a new idea,
or it's a marketing brief you have to write, or
sometimes even if it's an email you have to write
for a personal something and you're not sure how to
word it the right way. It allows you to have
a start and then you can edit from there. So
again going back to my efficiency point, it helps you
become more.
Speaker 4 (31:49):
Efficient, solves the blank page problem.
Speaker 5 (31:51):
It does.
Speaker 4 (31:53):
Brian, it was great to talk with you. Thank you
so much for your time.
Speaker 5 (31:56):
Yeah, this was fun. Thanks for having me.
Speaker 3 (32:00):
Huge thanks to Jacob and Brian for the deep dive
into the cutting edge innovations transforming the game of tennis.
Brian shed light on how the US opens partnership with
IBM is harnessing data driven insights to reshape storytelling in sports,
from AI generated commentary to Match reports. As we look ahead,
(32:21):
I'm excited about the possibilities for personalizing content and reaching
fans in new ways. The future of AI promises more
than just efficiency. It's about enhancing fan experiences worldwide. Smart
Talks with IBM is produced by Matt Romano, Joey Fishground,
(32:42):
and Jacob Goldstein. We're edited by Lydia jen Kott. Our
engineers are Sarah Bruger and Ben Tolliday. Theme song by Gramascow.
Special thanks to the eight Bar and IBM teams, as
well as the Pushkin marketing team. Smart Talks with IBM
is a production of Pushkin Indie and Ruby Studio at iHeartMedia.
(33:03):
To find more Pushkin podcasts, listen on the iHeartRadio app,
Apple Podcasts, or wherever you listen to podcasts. I'm Malcolm Gladwell.
This is a paid advertisement from IBM. The conversations on
this podcast don't necessarily represent IBM's positions, strategies, or opinions.
Stuff To Blow Your Mind News
Hosts And Creators
Robert Lamb
Joe McCormick
Popular Podcasts
1. Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
2. Stuff You Missed in History Class
Join Holly and Tracy as they bring you the greatest and strangest Stuff You Missed In History Class in this podcast by iHeartRadio.
3. Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations.