Episode 381 – Microsoft Graph Data Connect

Welcome to Episode 381 of the Microsoft Cloud IT Pro Podcast. In this episode we discuss Microsoft Graph Data Connect, a service that allows organizations to access and manage data from Microsoft 365 in a secure, scalable, and efficient manner. Like what you hear and want to support the show? Check out our membership options. Show Notes Elgato's New Stream Deck POWER-UPS make streaming even easier USB Hub - Multiport adapter for Stream Deck + Helping our customers through the CrowdStrike outage Azure status history Update on the SharePoint Files dataset Datasets, regions, and sinks supported by Microsoft Graph Data Connect microsoftgraph / dataconnect-solutions MGDC for SharePoint: New, Updated and Upcoming Datasets Step-by-step: Gather a detailed dataset on SharePoint Sites using the Microsoft Graph Data Connect About the sponsors Would you like to become the irreplaceable Microsoft 365 resource for your organization? Let us know!

Episode Transcript

(00:03):
Welcome to episode
381 of the Microsoft Cloud IT Pro Podcast,
recorded live on July 26,
2024.
This is a show about Microsoft 365
and Azure from the perspective of IT pros
and end users, where we discuss a topic
or recent news and how it relates to
you. Today, we have some updates to an

(00:23):
older, but not often shared feature of Microsoft
365.
So without further ado, we bring you an
introduction to Microsoft Graph Data Connect as well
as some recent announcements about data that is
accessible via the service.
We are live with another episode of the
Microsoft Cloud IT Pro podcast recorded live on

(00:46):
Friday, July 26th in the middle of a
thunderstorm
in Florida because
it's Florida, and it's afternoon. It is
starting
to sound eerily ominous out there,
and we are into
thunderstorms
in the afternoon season.
So if I drop out, it's probably
not you. The thunderstorm? It's me.

(01:09):
And my awesome
my awesome access to electricity over here on
the beach side of town. I just want
the thunderstorms to pass so I can go
play pickleball tonight. Oh, you're really dating yourself
if you're if you're if you've already started
playing pickleball.
My my parents play pickleball,
Ben. Pickleball pickleball is not for the youth
like us.
You know what? You say that, but it's

(01:31):
better than not doing anything, I figure, and
I actually do kind of enjoy it. Alright.
It's kinda fun. I can't play tennis. It's
so for me, I can't play tennis. I
love ping pong, but, look, I mean, I
don't get to work out playing ping pong,
and ping pong takes room indoors. So pickleball
is, like, a happy medium between for me,
where I can still carry over some of
my ping pong abilities

(01:52):
and get some exercise at the same time.
If you find that it's that much fun
call me old. You might have to invite
me. I mean, I have I have all
the gray hair, so, you know, if it
does end up being, like, an old person
contingent thing, you just pull me along, and
I'll I'll come play pickleball with you. I
think we have all the stuff in the
garage. My my my kids have a pickleball
set that they bought to play pickleball with
their grandparents. See, and you were just telling

(02:12):
me I was dating myself by playing pickleball,
but, yeah, your kids play pickleball. No. I
said, we have a pickleball set in the
garage
that was bought so my kids could play
pickleball with their grandparents. I didn't say that
my kids actually were playing pickleball with their
grandparents
or that that set was being used at
all.
Oh, okay. I stand corrected then. So speaking

(02:33):
of buying other things, I spent more money
today. Did you see did you get a
chance to watch the YouTube video I sent
you? Yeah. Elgato's got some new stuff out.
So they have
a new XLR interface
and a new USB hub, so I'm interested.
What did you pick up? We have all
our podcasts set up, so I was like,
I do not need an XLR
audio interface. However,

(02:54):
if you do podcasts or have an XLR
mic and you just need like a single
XLR port,
that is super
intriguing and interesting
to be able to incorporate an XLR audio
interface
straight into the Stream Deck. I picked up
the USB 1. So for me, it was,
what, I think it's like $59

(03:15):
for the USB
add on, and you essentially, like, unscrew the
base of the Stream Deck Plus
and then pop this attachment on the back
and screw it back in, and it has
2 USB C ports, 2 USB 3 ports,
and it actually has, like, an SD card
reader and all of that.
And here's my use case. I have 2
Stream Decks. I have, like, a USB

(03:37):
speakerphone on my desk. I charge my keyboard
sometimes off of USB, and I have all
these cables, like, running back and around
up
to docks hidden under my desk, behind my
monitor, all of that.
But there are times I, like, just want
to plug a USB device in that's right
here on my desk, so I can minimize

(03:57):
my USB cables running and snaking all around
the desk and get myself a couple USB
ports easily accessible,
like right in front of me on the
desk
without adding something extra to the desk because
they just kinda hide right in the back
of the Stream Deck there. You'll have to
tell me how this one works out for
you. Like, one of my frustrations with the

(04:18):
Stream Decks
is,
you know, they're they're bound to the software
on your desktop.
So you end up,
you know, in these kinda mismatched state areas
where sometimes, like, the button on your Stream
Deck thinks it's on when the device really
isn't on over here, like, the software is
not reporting things correctly. So one of the
things that I do with my Stream Deck

(04:39):
at least once a day is I unplug
it, and I plug it back in. And
I just do that by taking the USB
cable. So
out the back. So one of the things
so these new devices, I think they were
only applicable to the Stream Deck Plus, which
is fine. Like, that's what I have.
But the way they also plugged in was
basically
having, like, a permanent USB,

(04:59):
not cable, but just a plug sticking out
on the mount device, And then you're basically
mounting straight on top of that. So I
don't know if I wanna go from just
pulling a cable out once a day to
pulling the whole thing off the mount or
potentially figuring out how to, like, pull the
cable from the
cable kind of thing depending on where it
is. So I'd be interested in feedback once
you've had a chance to play with it
a little bit. And even on, like, the

(05:21):
XLR interface, like, I look around a lot,
and I'm like, you know, I drive 2
mics most of the day. So for my
meetings, when I'm at work, I typically drive
a shotgun mic.
So to give folks a sense, so if
I come over here, this is normally the
microphone that I use on Teams calls and
things like that just so I can be
hands free and not have this big honking

(05:41):
mic sitting in front of my face.
But then when you and I do these
things, you know, I I I bring the
the the nice ESOS out and and pull
that over for some of that, like, rich
deep,
you know, broadcasty goodness.
And
so I I really like, I I only
need 2 XLR interfaces, and I could probably
get by with 1. Like, there's really no
reason that I couldn't just use 1 and

(06:02):
and be done with it. And it would
buy back a bunch of space on my
desk. So the cool thing about these, by
kind of combining the functionality of a USB
hub or the XLR thing, it's so stupid.
It's completely low hanging fruit, but it makes
a 100% of sense. Like, in your use
case and in mine, it'd be buying back
desk space. Right? Like, if I could get
rid of this honking
device next to me that's driving, like, 4

(06:23):
XLR interfaces when I only need
to, that actually wouldn't be too bad.
So so that's the nice part. The bad
part is you're doing everything in software, and
then you have to contend with Elgato software,
which isn't the
most robust stuff from what I've seen. So
trade offs continue to abound in technology land.
Yeah. I'll have to let you know. So

(06:43):
I'm curious, like, for your use case of
unplugging USB and plugging it back in, I
think you would still do that. You would
just unplug it from
the USB hub instead of
the back of the device because it looks
like the cable just plugs into the USB
hub, and then there's, like, a pass through
into the stream deck for Yep. Connecting that.

(07:04):
So I think it's still but then you
disconnect
every USB device. That's the rub. Right? Like,
so today,
I'm only unplugging the Stream Deck, but if
I have a bunch of other stuff plugged
in there, like, let me see. What else
do I have USB? Oh, my my mouse,
my keyboard, like the little, you know,
the 2.4 gigahertz dongles for those hang off
a USB hub.

(07:26):
Yeah. So I I wouldn't wanna lose, like,
my mouse and my keyboard and everything while
I'm waiting for the whole hub to go
through its machinations
and
rebooting and all that stuff. So we'll see.
I'd be I'd be interested in the feedback
on stability specifically. I will let you know.
I might not be a good test because,
ironically,
I do not have the same issue you

(07:47):
do around
stuff getting out of sync. So this is
would be another curious thing as to why
that is. It's either sync or it just
straight up freezes.
Like, my stream deck my stream deck freezes
all the time.
I cannot remember
the last time I have unplugged and replugged
in my stream deck. It just keeps working

(08:09):
for me.
You're living a magical world, my friend, every
single day. Apparently.
But that being
said,
I will and I'm curious if this works
for you if if you've tried it. What
I do have to do every once in
a while because this to your point, it
kinda streams through the software,
is my software

(08:30):
loses
its connection
back to, like, my Philips Hue or it
loses my connection to my
companion app
or my Govee app, and I don't unplug
and replay in the device. I just kill
the Stream Deck software and reopen it, and
it reconnects everything back up and starts working.
I've become a ninja killing the back end

(08:50):
daemons for Stream Deck, particularly on, like, Mac
OS.
So, like, I drive a lot of my
automations either through HomeKit or through Home Assistant,
like, we've talked about both of those in
the past.
So there's a specific daemon that runs for
the Home Assistant plug ins, so you don't
actually have to kill your whole Stream Deck.
You just have to kill the Home Assistant
1, and then it magically

(09:10):
kinda comes back. But, yeah, I I have
a whole, like, I have a whole script
I run on the side. Like, just go
run this bash file,
and that that kills everything and then brings
it back to where it needs to be.
Nice. Unless it was CrowdStrike, in which case
there is no coming back. Another topic, another
day. There's coming back from CrowdStrike. It just
involves 20 reboots and a USB key and,

(09:34):
a whole bunch of extra steps Manually touching
every device and if you are a cloud
hosted
virtual machine. Yes. We could talk about that,
but it truth be told, it I mean,
it a pet impacted Azure. It didn't really
impact Microsoft 365. It was more Windows. We
have other topics, Scott. Maybe we should Mabel
would come back and talk about Crowd Strike.

(09:54):
Did the Crowd Strike impact Azure?
So there was an Azure outage, which doesn't
tie to the CrowdStrike outage. They were
unfortunately, like, within, like, minutes of each other,
like, 20 minutes or something like that. Yes.
But completely unrelated. And I did like, I
legitimately felt bad for Microsoft because I was
reading some of these news articles, and you

(10:14):
were getting these companies like,
this is my pet peeve with the media.
I get why people do it, but they're
always want to be first, and they always
wanna have the breaking news, they always wanna
have the most information.
But that inherently, I feel like leads to
them getting stuff really wrong sometimes.
And in this case, I read so many
articles where it was like, CrowdStrike outage takes

(10:36):
down Azure and Microsoft 365, and I'm like,
no. Microsoft 360 5 and Azure went down,
like, the day before due to another issue.
And from what I read, those 2 were
somewhat related. But then the whole CrowdStrike thing,
like you said, very unfortunate timing
happened right after Azure and Microsoft 365 got

(10:56):
came back up. And so many news companies
munged the 2 together
when the CrowdStrike thing really was not Microsoft's
fault at all. So the PIR
post incident review
for
the
US central outage
is up and out there. I'll I'll I'll

(11:18):
see if I can pull it up for
folks and at least include it in the
show notes or I will post it over
to Discord where if folks want to become
members of the show because Ben told me
I should talk about membership or,
Hey. Become a member. You'd ask us access
to Discord. Come and join us while we
record these shows live, and follow along and
chat with us and generally have fun.

(11:38):
How was that? Was was my spiel good
there? You did good. Good job, Scott. So
that being said, should we talk about our
topic today? This is like an old one
that has some new updates. I mean, that's
pretty much what we're good at, right, is
talking about old ones. That's Right.
Do you feel overwhelmed by trying to manage
your Office 365 environment? Are you facing unexpected

(12:01):
issues that disrupt your company's productivity?
Intelligink is here to help. Much like you
take your car to the mechanic that has
specialized knowledge on how to best keep your
car running, Intelligent helps you with your Microsoft
cloud environment because that's their expertise.
Intelligent keeps up with the latest updates in
the Microsoft cloud to help keep your business
running smoothly and ahead of the curve. Whether

(12:22):
you are a small organization with just a
few users, up to an organization of several
1000 employees,
They want to partner with you to implement
and administer your Microsoft Cloud technology.
Visit them atintellijinc.com
/podcast.
That's
intelligink.com/podcast

(12:44):
for more information or to schedule a 30
minute call to get started with them today.
Remember, IntelliJint focuses on the Microsoft cloud, so
you can focus on your business.
Talking about old stuff that resurfaced,
so this is an article
that I saw
published on July 11, and it brought back

(13:06):
memories and we started digging into it, was
there was a post from
Jose
Barreto,
that's probably how you pronounce his name,
in
tech community about updates on the SharePoint files
dataset. And I was like, the SharePoint files
dataset from the headline, I couldn't remember what
it was. And then I started reading it,
and it's under the subcategory

(13:27):
of Microsoft Graph Data Connect
for SharePoint. And if you go start looking
into this,
this ties back to
a feature that came out. I think it
was originally announced. I had the YouTube video
here, and then I lost it.
2021,
I believe, like, 3 years ago,
around this Microsoft

(13:48):
Graph Data Connect
and
allowing you to
essentially
go in and grab a bunch of analytics
data out of your tenant
around
various metrics.
And some of the metrics from back in
2021 when this was originally announced

(14:09):
was being able to go in and pull
datasets around
users,
direct reports, and managers from like an Office
365
people perspective, and then they had it for
Exchange Online being able to pull in datasets
around
calendars, events, messages,
contacts, mail folders, mailbox settings, sent items,

(14:30):
and then they announced that Teams chats would
be coming.
And I feel like when they originally announced
this back in 2021,
they,
like, it just I I don't re remember
hearing much about it over the last 3
years. And then this article popped up with
3 updates to it

(14:50):
with some public availability for these updates coming
here in the coming months up here in
August.
So this is kinda it's an interesting feature,
and I don't I'll be honest, I don't
know where to start. Do we wanna start
with the updates, or do we wanna kinda
start with, like, what is the Microsoft Graph
Data Connect? Like, what you would use this

(15:11):
for? We should go back, and we should
definitely start this one with,
what is this?
And it took me
more than a hot minute to wrap my
head around
what this is
and the value of this and and kind
of
I I think where it can inflict value

(15:33):
on customer workloads, and I I I I
kind of love features that inflict value.
And this was definitely one of those that
does this, but it's kinda so confusing with
the way it's marketed and it's put together.
So
give me your pitch for it and your
understanding. And then if that matches with mine,
we're good. But if not, I'll I'll give
you a kind of some of my take

(15:54):
and where I think this actually sits and
why I think it's
kinda cool and maybe underappreciated. Like, I'm surprised
I don't hear about it more based on
what I learned about it after you pointed
it out to me, and then spent a
week researching it and clicking buttons and playing
around. Okay. So here's my take on it.
Microsoft Graph has I mean, Microsoft Graph is
the underlying

(16:16):
it's not really the underlying data. It's the
underlying
API, essentially,
for all things
Microsoft 365.
Like, when you go
write different connectors, when you're going and writing
custom code, when you're doing stuff in Power
Apps, when you're doing stuff in PowerShell, ultimately
you're connecting to Microsoft Graph for performing certain
actions. The way I understand Microsoft

(16:39):
Graph Data Connect is all these APIs also
surface different things, and they have a whole
list of different
activities that are in micros in the Microsoft
Graph data. And my understanding is is that
this is probably where a lot of the

(16:59):
reports
come from when you are looking at reports
on Microsoft 365 or storage trends,
around mailbox reports,
emails sent, emails received, all of that. It's
coming from Microsoft Graph. What Microsoft Graph Data
Connect
does is,
by nature, all of that data has a

(17:20):
time frame. Microsoft does not let you keep
that data forever. They have to store it
somewhere. Understandably,
it's going to have a lifespan. Usually, it's
somewhere between 30 90 days.
What Microsoft Graph Data Connect does is allow
you to go
set up a connector
to these datasets in the Microsoft Graph and
essentially extract that data to whatever

(17:44):
storage location
you want to, and it's coming through something
like Azure Synapse or Azure Data Factory where
you go connect to Graph, pull this data
through one of those, shove it off into
some location
where now you control the location so you
can keep this data as long as you
want to, but you also have all these

(18:06):
different data sets and activities
that you can now
build your own reporting, your own dashboards,
view trends, all of that off of all
these different activities and data that is surfaced
through Microsoft Graph via this Microsoft Data Connect
functionality
feature, whatever you wanna call it. So that

(18:28):
is my not necessarily a 32nd synopsis, but
my
understanding
and rough how I would kind of describe
what it is. I think you
mostly
nailed it as far as my understanding. I
I I would extend on a couple things.
So this is called
Microsoft Graph Data Connect, and then it has
all these datasets that come from

(18:50):
services that support the Graph, Exchange, SharePoint, all
this stuff.
Well, it has Graph in the name.
The nice thing about it, and I'm gonna
say this is a nice thing,
is it allows you to not have to
work with the graph APIs directly. Right? There
is no
invoke web request. There is no going out

(19:10):
and figuring out, am I using the beta
endpoint versus the current GA endpoint?
Which version of which PowerShell module do I
use, and how much jankiness is there, like,
any of that stuff. Like, the these are
effectively
snapshots
of telemetry from the system
that are pre aggregated and then pushed out

(19:31):
on a schedule.
And
the the the cool thing here is they're
not only aggregated and pushed out on a
schedule, they're pushed out
in a format that's consumable
and with a defined schema
that has a hard contract against it. So
each one of these datasets,

(19:51):
like the Outlook dataset versus the SharePoint dataset
versus,
you know, some of the identity ones, they
have hard schemas that are associated with them
where you can choose as a customer to
consume the entire schema
or to only consume
portions of that schema. Right. So let's say
you had a table with,
you know, 50 columns in it

(20:13):
based on the schema. And it turns out
you only really need 10 of those.
Great. You only take 10 of those, and
that's what you get in your snapshot. And
that's the data that's kinda dumped out continuously
to get things going.
So I I actually, like, I I looked
at this, and I was like, why don't
more services
do this? It kinda makes sense if you
think about it because Oh, 100%.

(20:35):
You don't always need near real time
extracts or data flows. And in a world
where you don't require
near real time things,
near real time
aggregation, transformation, all those things, like, it's expensive.
It's it's a ton of telemetry to store.

(20:56):
It's a ton of telemetry to aggregate together.
Quite often, it's on, like, you as the
customer to put it together. Alright? So if
I was gonna take, like, Azure Resources as
an example let's say we took virtual machines,
and and we imagine something like that. Right?
Like, so I have VM 1 and VM
2. VM 1 and VM 2 can both
emit their telemetry, but I still need to
rationalize which telemetry is from which one. How

(21:17):
do I put it together? What are the
right aggregates that I need to get to?
Versus having something like this, say virtual machines
was part of this data connect mechanism. You
would just get one big dump daily
or on your schedule of, hey, here's all
your VM information as you wanted it. And
that's basically what this is doing
for these various datasets that sit out there.
So if you're looking at more, like,

(21:39):
long term reporting scenarios,
and long term is really, like, honestly, anything
past, like, a couple minutes, because, you know,
at that point, like, once you're outside of
a a couple minutes to a handful of
hours, like, you're not doing real time data
anyway. Like, figure out a way to snap
to, like, a daily process or a weekly
process or something like that, and then just
track

(22:00):
general trends and how those things go over
time. And then maybe fill the gap with,
like, some NRT stuff where you actually need
it, maybe with, like, searching audit logs or
real time metrics
or or things like that along the way.
So so it it's really cool because, like
you said, for me, 1, you don't have
to deal with the jank that is the
Graph API. And let's be honest, the Graph

(22:21):
API is pretty darn janky and continues to
be.
And it gives you this fixed, durable contract
that then you can go ahead and proceed
forward with.
You don't need to be a
developer,
you know, like, hands on, slinging code against
things. Like, it all comes out in common
formats,
and you can just wire that up immediately

(22:41):
to
a Synapse solution if you wanted to do
some further aggregates on top of it with,
like, Spark or something like that, stand up
a Databricks cluster. Great.
Or just pound point Power BI at that
folder in a storage account and let it
go. It's fascinating, and I don't know if
you watched,
I'm assuming in your studying too, you watched
this video, and even back then they talked

(23:03):
about,
1, not needing all the tables because
columns
in the table, all of the fields in
the datasets,
but also from a privacy perspective. They were
like, if you want to run analytics on
certain things,
email, for example, it contained the email body
in the dataset. But if you wanna filter
out the email body, you can filter out

(23:23):
the email body or filter out email subjects
and just see, like, who's sending an email
to who or whom is sending it to
whom. When are when are we supposed to
use who and whom? Neither here nor there.
And filtering out users and groups. So maybe
you're only pulling in telemetry
from certain groups of users or you're excluding

(23:45):
certain users from your telemetry, but
a lot of flexibility there. And this was
one. I'll pop this on the screen. We
can throw an image in the show notes
or something. But, like, this Power BI report,
I've never seen this before, and I'm like,
this is fascinating,
where it's a super zoomed out,
it's a
it's showing like a graph of all the

(24:06):
different users in a company
with different bubbles in terms of emails that
they've sent, but then connections of, like, who
is emailing who within your organization.
And they use this example as they could
see like the HR department was emailing everybody
else but wasn't ever emailing anybody else in
the company and developing

(24:27):
graphs of how different people were connecting, when
they were emailing people, who they were emailing,
how different departments were collaborating or not collaborating
together,
just fascinating
analytics that you can pull out of this
without
necessarily,
I mean, again, you wouldn't have to know
what they're emailing. It's just the fact that

(24:48):
emails are going from one person to other
people.
So, yeah, it's
again, I remember this coming out. I started
playing with it. This makes me wanna go
play with it a whole lot more.
Yeah. I it's
it's it's an interesting one.
I I also like that, like, the so
so beyond, like, just, like, the generic, like,

(25:09):
hey, it's a fixed contract and allows you
to do reporting kind of your way with
your tooling, things like that.
It's
also pretty agnostic.
Like, the datasets, they just publish out to
storage accounts, like you said, so super easy
to kinda pick them up and play with
them
once they're in there.

(25:31):
Open format, queryable, you can get out of
them with
ADF, you can get them out at them
with Synapse, you can get out of them
with Databricks, like,
tons and tons of options just to
make your life,
a little bit a little little bit easier.
So, yeah, it's the
the the pricing actually is isn't too bad

(25:52):
either.
Like, I I guess we should call out,
like, hey, like many things, this is an
additive service.
There is pricing. Yeah. Yeah. There there there
are some considerations
there for you
just as,
you know, how things marry up and ultimately
come together. So,
you know, you could be charged, and this

(26:13):
could be a little hard to rationalize, Like,
you'd be charged for storage consumption.
You're gonna be charged
for your usage based on the
the the Graph Data Connect stuff.
And then some of these, like, they have
one off pricing.
So, like, you opened with the update on,
like, the SharePoint stuff. So the SharePoint files
dataset has a different pricing construct

(26:36):
than
other parts
of Graph Data Connect, which can be
a little bit
confusing
as well. So, you know, you're you're looking
at things like everything is based on number
of objects copied and number of objects copy
number of objects enumerated on one side and

(26:58):
sent into, ultimately, like, the dataset before it's
pumped out, blah blah blah. So you're talking
about things like 0.35¢
to,
0.75¢,
like, per n object, like, n 1,000 objects
loaded
into things along the way.
So I I I think, like,

(27:19):
you know, I was, like, seeing things like
this too. Like, you're also seeing a little
bit of, like, the bifurcation and potentially, like,
the associated cost on the service side to
pull this stuff out. Like, you mentioned logging
is expensive. It's not just expensive for customers.
It's it's expensive for service providers as well,
right, to generate and store and and put
it all together.
So so you do see some of that
reflected kind of

(27:40):
in the
in in the pricing components as well. There
is some chat going on in Discord
asking about, like, is some of this pulling
into Veeva? Yeah. This is probably also where
they're getting some of the Veeva Insights stuff,
but they were also asking about in-depth analysis
of
SharePoint online environments, and I think that gets
into a little bit of what this

(28:03):
announcement was was kind of three announcements that
came out here a couple weeks ago
is
datasets
for SharePoint, and the SharePoint file datasets are
gonna be publicly available
on August 20,
2024,
so another month or so from now.
They released the SharePoint files dataset

(28:24):
pricing,
And like you said, Scott, it's slightly different.
And then the SharePoint files private preview extended
to August 19, 2024, which makes sense. I
essentially said private preview is gonna go right
up until the GA date. So I don't
know if private preview was before that. But
kind of what you're getting now in this
dataset, and this is where you get some

(28:46):
of this updated
information,
and
I think this is some of the stuff
you're also seeing if you've looked at SharePoint
Premium and some of the oversharing analytics and
some of that is
these updated datasets now for SharePoint
files
allow you to pull in, like, archive state

(29:06):
of the site, you can pull in things
like Recyco bin item counts, you can pull
in
communication sites, you can pull in if the
site is a OneDrive site,
if external sharing is enabled or not on
the site,
if the site is connected to a private
Microsoft 365 group,
privacy of the site, owners,

(29:29):
last access data,
and then you can pull in SharePoint permission
datasets where it's total users,
who something's created by,
who an object is shared with based on
the Azure AD object ID,
how many users that's shared with, so a
user count of how many
files a or how many users a SharePoint

(29:51):
file is shared with.
So a bunch of additional
data was added to the SharePoint datasets,
that you can now start going in and
pulling in once it becomes generally available here
in another month.
And, again, some of these are
especially around the shared with type of analytics

(30:11):
can help you
with some of that oversharing that has become
ever so popular to talk about with the,
increasing demand for Copilot
or even just general security. It's more like
back to kind of what I was thinking
earlier with just call it out again. So
you have the kind of the the long
term trending things that you wanna do. Like,

(30:33):
this this all lends itself to trend analysis
at scale within your environment. Like, it's definitely
for the more, like, reactive stuff or maybe,
like, more long term proactive.
Like, I wanna track consumption
of
I I wanna track consumed
size in my SharePoint sites versus, like, provision

(30:54):
size and then, you know, track that. I
wanna track number of users, number of share
requests, blah blah blah,
different user types. Like, maybe you're trying to
make a transition from,
like, old school
SPNs over to, like, MSIs. Right? And and
and do that kind of burn down and
track that. So it's all, like, very well
suited for that. It's also well suited for

(31:16):
working around any of the limitations in the
Graph API,
where it just can't keep up with the
polling rate, or you're just in such, like,
a high churn environment
that,
you're gonna be subject to, like, API throttles
or a bunch of other things that you
don't want. So
I I don't know that it's, like, the
end all be all, but it's definitely
complementary.
Like, I could totally see having like, like,

(31:38):
when I was looking at this, I could
totally see having a dashboard that does something
like tracking my identity consumption longer term and
some of the dimensions around that and then
having, like, a view in there that also
ties into,
you know, some of my more, like, proactive,
real timey reports, like risky sign ins and
things like that.

(31:58):
And then you can merge all that together
in a single view, a single report, a
single world. Like,
really cool stuff for being able to report
just kinda, like, line of business or rhythm
of business metrics
around your consumption
of
these services that ultimately, like, have some hook
or some data's
data point into graph land. Yeah. And that

(32:20):
was my thought with this because I've had
clients come to me and say, okay. We
have all these SharePoint sites. We have a
1000 SharePoint sites, 2,000 SharePoint sites, 3,000, whatever
it may be,
especially around sharing. Who is this shared with?
How many sites do I have that have
external sharing links?
How many are shared with guest users? Like,

(32:41):
to write a PowerShell script to do that,
1, like you said, you're gonna hit throttles,
but, 2, it just it's not fast
to loop through 3,000 SharePoint sites and pull
that data. So to be able to dump
this out and even provide regular reports
every week, every month or so of how
has that changed, what do we have out
there,

(33:02):
I see a lot of potential for this,
especially around
some of the SharePoint security.
And I really do wanna go jump into
some of the other datasets
because
there's a lot more datasets out there than
I remember the last time I looked at
this. Again, given that this has been around
for a few years now. There's a whole
bunch. There's a whole git repo that's full

(33:22):
of thumbs.
So I I think the other thing, like,
really, like, you should be screaming in the
back of your head when you're sitting down
and and you have to, like, rationalize, like,
hey, do I use something like this or
do I just use, like, in your case,
like, PowerShell and things like that?
It's also just durability.
Like,
you
know, I know, like, we like to talk

(33:42):
a lot about, like, API surfaces being contracts
and things like that. But the reality is,
like, if you wrote a PowerShell script today
that worked with the graph
in its current instantiation,
that that stuff could break, like, 5 minutes
from now. Right? And then you're just left
with the churn of having to
figure that out and build it back to
where it needs to be versus something like

(34:04):
this where you can truly take a
durable contract, like, hey. You're you're you're not
only paying for the data,
but, you know, it it's a fixed schema.
It's a known thing. So you can take
a little bit of a firmer dependency on
it, like, on the order of, like, months
to years
versus what sometimes, like, really like, I I
know it comes across as me being flippant,

(34:25):
but it does feel like minutes sometimes
in graph land, which is the way stuff
changes or moves from underneath you. Absolutely.
And I guess kinda to wrap it up,
unless you need anything else, if you are
looking to get started with this, we will
throw this link in the share notes. And,
again, I don't know how I missed this.
This is a step by step, gather a
detailed dataset on SharePoint sites using the Microsoft

(34:47):
Graph Data Connector back from February of this
year,
where it does walk you step by step
through setting up Microsoft Graph Data Connect,
enabling the right
features, services in Microsoft 365,
setting up Azure. I know some Microsoft 365
people, me included,

(35:07):
maybe aren't super familiar with, I think, setting
up like
Synapse workspaces
and how you would connect the data to
Synapse and pull the data
through, going through and building reports. Like, this
is an end to end step by step
walk through from
turning the feature on to connecting it to
Synapse

(35:29):
to walking you through building a Power BI
report.
It is a fairly lengthy web page,
but I would say a very good place
to get started if you're not familiar with
some of this stuff,
just to see a sample of how you
can start building this stuff out.
Yeah.
I saw this one. So so it exists

(35:49):
in docs as well, in like the public
docs, just not with the verbosity. Uh-huh. And
I really wondered like, why doesn't this sit
in docs? Because
and it was the piece that I was
missing when I first started going through it
until you pointed out the article. It was
like, hey, you you get to the end
and, like, there's no good, like,
how do you visualize all this stuff? Because
ultimately, that's the point. Right? Like, we get

(36:09):
to really decide, like, what's what's the
consumption and distribution model look like, and is
that Power BI? Is that Tableau? Is that
Grafana? Like, who you know, whatever it happens
to be.
And then I was going through this article.
I was like, oh, it's just because they
went down the happy path of, like, everything
the most expensive way they could. Right? Like,
synapse here, ADF here, this this this, all

(36:33):
these things, and all these things, like
so so it's a happy path. Like, I
will give them that 100%. Like, it it's
it's a happy path. It works. Like,
I can tell you, like, if you go
through that article, like, it'll lead you into
some bad practices,
you know, around the way, like app registrations
and things are configured. But whatever. Like, it
works on the other side. Right? Like, it

(36:53):
it does give you kinda, like, a point
in time
and get you where you need to be.
But be mindful, like, as you're standing this
stuff up, like, there might be other ways
to do
it. There could be less costly ways to
do it. Right? Like, you might wanna consider,
like,
you know, do you really need Synapse?
If you do need Synapse, like, what's the
size of your Synapse compute that you're gonna

(37:15):
pull in versus what's the size that's maybe
in, like, the random article on TechCommunity or
things like that. So just just keep that
stuff in mind too.
But I'll I'll tell you, like, I I
was actually
quite impressed. Like, this is, like,
kudos because it it was very turnkey and
and very easy to get going with. Absolutely.

(37:36):
So I already see Pirate in the chat.
He's like, you guys just filled up my
weekend for me now.
He's gonna be busy this this weekend playing
with it as well. So very
cool
tool features
that, like you said, I'm surprised
it doesn't have it hasn't gotten more attention,
visibility.
So if you haven't seen it, played with

(37:56):
it. It's probably worth a check checking it
out and seeing what you can do, especially
if you like numbers and reporting and analytics
Absolutely. Or on Microsoft 365. It is for
the data geeks. Get it get it going.
Alright. Well, thanks, Scott. Appreciate it. Glad we
were able to nail it and be on
the same page with what this actually is,
and hopefully bring some clarity and some visibility

(38:18):
to it for those of you that have
not played with it. Yeah. But with that,
go play with data this weekend.
Pull in a bunch of data, create a
bunch of reports, and show me what you
created next week. Homework. There we go. I'm
yeah. I wanna see screenshots in Discord next
week.
Blur them all out. They'll just be blurry
blobs of default Power BI colors. What we
talked about, just abstract the data. Just mask

(38:40):
usernames and all that and create some fun
charts. Mhmm. We'll see what see what we
can see in Discord beginning of next week.
So hope everyone enjoys their weekend, and we
will talk to you again soon, Scott. Great.
Thanks, Ben.
If you enjoyed the podcast, go leave us
a 5 star rating in iTunes. It helps

(39:01):
to get the word out so more IT
pros can learn about Office 365 and Azure.
If you have any questions you want us
to address on the show, or feedback about
the show, feel free to reach out via
our website, Twitter, or Facebook.
Thanks again for listening, and have a great
day.