Quantifying Lessons Learned in the European Space Agency with Stefano Scaglioni

Stefano Scaglioni is working at the European Space Agency (ESA) in Darmstadt, Germany. He covers the role of Product Assurance and Safety Manager, within the quality office of the ESA Operations directorate. His job consists in ensuring the quality of the space operations services (e.g.  flying  JUICE and EUCLID satellites) delivered to the customers. ESA

Episode Transcript

(00:12):
Pioneer Knowledge Services welcomes you to the next
because you need to know. I'm your host,
Edwin k Morris. I serve as president and
founder of this organization, and we are thrilled
to offer this educational program. These design conversations
bring you people's experiences from all over the
globe in the field of knowledge management,
non profit work, and innovation.

(00:41):
Hey fans of Pioneer Knowledge Services and Because
You Need to Know for the upcoming conversation
you're about to step into. Stefano
Scaglione
was a recording that I did in Portugal
at the 23rd
European
Conference on Knowledge Management, and Stefano
was doing his remotely from Germany. The name

(01:03):
of his paper is the effectiveness
of the lessons learned process in the European
Space Agency Directorate of Operations. He is a
co author of that paper. With delay,
much delay, I apologize.
Here it is right now.
Hello. My name is Stefanos Caglione.

(01:26):
I'm Italian. I'm working and living in Germany,
specifically in the town Darmstadt.
Darmstadt is close to Frankfurt, more or less
in the middle of Germany.
I'm working in European Space Operations Center,
which is a site belonging to the European
Space Agency, ESA. My job is a Prote
Assurance and Safety Manager. It's a strange title.

(01:50):
I'm about
quality.
Quality of our
services to the customer. Our services are satellite
operation,
and I'm ensuring that, what we deliver is
according to customer specification.
And in part, I also work for the
quality of the organization,
meaning that I support

(02:10):
the overall
setup of the organization,
the processes,
the audit on the processes,
the training of the people, and tools that
are useful for everyone.
So it sounds like you do everything.
I'm involved with many people,
several activities.
I have the chance, and this is a
pleasure. I have the pleasure to to work

(02:33):
more or less in every corner of the
organization.
And being a limited size of the organization,
we are at most
600 people.
It is a nice community,
and it's good to be friend of each
other and to know each other. It is
an asset for the job. Well, that's interesting
you say that because from a scientific community,

(02:53):
I'm labeling it a scientific community compared to
a lot of other type organizations.
It sounds to me this would be pretty
heavy to science. In a scientific community, I
guess my first guess would have been there
wouldn't have been so much camaraderie and networking
people to people, but you're saying that is
a key ingredient.
Yes.
The ingredient

(03:13):
to to work together
every time for every space mission
as a team, and there would be for
sure the director of the orchestra to drive
everyone, but everyone is playing, is contributing to
success.
And coming to lesson learned,
everyone is knowing from the experience of the

(03:33):
other team what is best to avoid, what
is best to acquire,
best to implement as a new
solution, new technology, new practice.
It's bet it's a good place to be,
is, the center for networking,
but networking not for my private interests. Networking
for
allowing people

(03:55):
to help each other
and to make the entire organization to grow
together. So this professional
family,
I'll I'll label it a family because it
kinda sounds like there's that level of connectivity
in some families. I'll say that's not for
all families.
But for some families, that's a big thing.

(04:15):
What do you find
is the most
dramatic thing you do?
Dramatic.
This depends on the interpretation of dramatic. Yeah.
Yeah.
I believe that I help people in doing
their job. I offer them tools that are
supporting their job.
I helping them in advising

(04:37):
how to proceed because I know the standards,
internal standard that we have written down, how
the process should be.
So I help them. And in fact, I
bring training every year. I provide public training
to the entire organization
about, let's say, problem management,
about risk management, configuration

(04:57):
management,
documentation production, and lesson learned.
I offer this as,
support to everyone
from the top down to the bottom according
to their responsibility. And I offer also tools.
Let's say, implement the processes.
So it sounds like you're there to empower
the people to do their jobs fully. Yes.

(05:21):
If you want, my visit card
is more like a policeman.
In this no. Yes. Because
I am quality. So quality is to check
the work of the others, that they are
doing the work correctly.
But I interpret this as a part of
the team. I am supporting the team. And

(05:42):
being part of the team, they are aware,
first, that I am present.
2nd, that I can support them. And 3rd,
how the things should be done correctly. This
is altogether.
And this position is what I consider dramatic
because I believe I can provide added value.
Let's bring
where you're from to where you should be

(06:02):
this week, but you'll not be here physically.
Then that is the European
Conference on Knowledge Management.
What is your capacity you're bringing to this
conference? I bring our experience. My paper is
about
a way of measuring how the process of
lesson learned
is, carried out and especially how the process

(06:25):
is measured.
Are we able to perform what is written
down in term of activities?
Are we doing what we commit to do?
So we put in place
a a mechanism of measuring
the activities
performed
in order to verify if we are really
performing what we commit to do. How much

(06:45):
of the lesson learned activity
is of your daily or weekly practice of
your work versus the lesson learned piece, but
versus best practices?
Does your organization
differ between what a best practice is and
what lessons learned is? The second part of
the question, I do not very well understand.

(07:07):
The first part, I can answer in the
sense that
lesson learned process take me,
let's say, 5% of my time. 5%
because
more important are other processes like, problem management,
risk management.
Overall, the lesson learned process is if you
want the lowest priority

(07:29):
process in the entire organization.
For example, no specific manpower
is allocated
for performing this process.
Everyone, me and all the players, are playing
in the sort of spare time.
Okay? But we commit.
We commit to assess our experience

(07:50):
in order to find out is there anything
to be learned
by ourselves, to be learned by our colleague,
to be learned by our customer, or whoever
partners is working with us. Well, if you
don't mind, I'll try to go, what I
would present as a best practice. So in
the United States Army, which I have experienced
with lessons learned, there's a difference between

(08:11):
lessons learned and best practice. A best practice
is something that is regional,
maybe time situated
versus
enterprise
and consistent, you know, evergreen.
Do you have a a best practices kind
of application, or is it all just considered
lessons learned? 1st, I have to explain to

(08:32):
you what is the meaning for us of
lesson learned. Okay. Good. For us, the meaning
is and specific for our organization
is just
a recommendation for a change in the future.
What is lesson learned is what we derive
from the experience,
we derive something negative. For example, we have
done a mistake, and we want to avoid

(08:53):
this mistake for the future.
Or we did have something positive.
We have learned there is a new things
that is valuable to be replicated in the
future, and we inform the others that this
is something to be taken into account.
So for us, lesson learned is
a change that we recommend
for the future

(09:14):
based on an experience. And the experience must
be unexpected
because if it is normal experience as expected,
then there is no lesson to be learned.
We divide knowledge management from lesson learned. Oh.
So it is specific just an example. How
to fly to Mars. Okay? We know how
to do. We have it is documented.

(09:36):
It is reported.
It is also in the brain of the
people. If we have a mission to fly
to Mars, and then we look at the
experience, how was this activity? Was it done
as expected
or not? If it's done as expected, this
means that from this mission to Mars, there
is nothing to be learned because it was
exactly as in the practices that we were

(09:58):
we have written down and we are expected.
But the knowledge of this mission to go
to Mars is present.
There is a knowledge both written down and
also in the brain of the people
that is added value into the organization. Lesson
learned instead are just the changes. We respect
what we have done and what we have

(10:18):
done with respect to the reference.
That clears it up for me because the
image I got is that now you've combined.
This is the way I think. I think
visually, if you think of a dumbbell, right?
Lessons learned is the weight on this side.
Knowledge management
is the weight on this side.
And that's the strength you're building in the

(10:39):
organization is you've combined
both intent and purpose, Kilometers,
and, you know, lessons learned. That that's a
brilliant concept, and I've not ever heard that
We do not want
to impose this concept, and we do not
want to say this is a standard.
Everyone for us, everyone is free to use

(11:00):
any meaning to lesson learned and also to
organize any kind of process to this
lesson learned. With my paper, I just present
how we are doing, but not to explain
what we are doing, just because we have
invented a way to measure that what we
are doing correspond to what we want. We
want to measure the effectiveness
of the process.

(11:21):
So we want to be sure that the
ability to achieve the intended product
is indeed there, or at least we have
data that show us if we are good
or not. That is a bridge that a
lot of organizations
would love to get their hands around
is to be able to collect the correct
data
that would show them positive or negative the

(11:42):
influence they had. And I think that's where
it sounds like the longevity
from the work you're doing at the paper
for this conference in Portugal is that you'll
establish an importance or highlight the importance of
proper measurement and value. Yes. Is that is
that right? Okay. Exactly. What do you think

(12:02):
is gonna change
based off the research you're presenting
this week in Portugal? What's gonna change for
who? And I say that, like, in your
mind, who do you think is gonna benefit
and why? I believe there are 2 aspects
at least. One aspect is,
I I try to remember the second. One

(12:22):
aspect is that we just offer an example.
We do not want to impose and say
that we are the best. We are proposing
offering an example of
measuring a process and is a process.
In term of meaning, lesson learned, it should
be known by everyone. And if someone is
interested,
they could take this example as,

(12:44):
an idea to implement in their organization.
But I'm not selling. I'm just Yeah. Yeah.
You're just saying here, you take a bite.
If you like it, great. If you don't,
that's okay. Yes.
The second aspect
is,
like we mentioned a few minutes before,
it could be interesting to start the discussion
about what is a lesson learned and what

(13:07):
is the best process
to collect, process, and implement a lesson learned.
This could be a secondary outcome. The first,
result is just offering this idea.
And if people thinks that this is, something
useful, they may ask a clarification
I can offer,
but no more than what we have done.

(13:28):
So how do you think you're gonna get
feedback from the attendees of the conference
and from anyone that reads your paper? How
will we know if you made an impact?
This I do not
know. For sure, at the beginning, there will
be the the questions, but I have no
idea if being, for example, at the end
of the morning, people are already sleeping or

(13:48):
or not. The the first indication is the
level of questions.
The second indication, I would expect that,
my email address is
circulated. And then if someone wants, you can
contact me. I have no problem. Well, let's
make sure we have all that content so
I can add it to the podcast website
that you'll have as your own show.

(14:10):
And we'll have all that data there also
to give you more distribution. Okay. Two things
that we'll wrap up the show with. With.
What's your definition of knowledge management?
This is a very difficult question
because
okay. I have my personal opinion, and then
I have to
disclaim for responsibility of the organization.

(14:44):
Terms of
documented data
and and documents and records,
but also in the people brain. Okay? The
asset knowledge. Right? Yes. The difficult part of
this definition is the word management. Because I
tell you honestly, today, in our organization,
I'm talking only about our directorate and do

(15:04):
not talk about the entire agency. In our
directorate,
we do not have a specific
formal
process of knowledge management. We do not have
activities
recognized and approved
on what to do. We have a representative,
a point of contact with respect to the
rest of the agency,
but so far, we have not yet formalized

(15:27):
what to do about knowledge management. We have
the process of lesson learned.
And if you want, this one is the
only thing, formally speaking, that we have in
place. And lesson learned process for us is,
in a way, a distribution
of the knowledge,
but on specific knowledge, the knowledge that is
coming from unexpected experience. Because the the key

(15:50):
of the lesson learned is
not what we have done,
what was unexpected, and therefore, what we have
to learn from the unexpected. How do you
collect that? How does all the workforce in
your directorate know that, you know, this this
didn't go the way I planned. I should
probably tell somebody. How does that work? In
our process, there are milestones during the activities,

(16:12):
and there are specific milestone where we ask
people
to make the exercise.
Please, team members, think what you have done.
Is there something to learn out of it
or not? But these are, let's say, at
specific milestone which are important, especially in a
project.
And we really formally speaking, we really distribute

(16:35):
an email. We call for a meeting.
We push the people, but we push the
people just to make the exercise. This does
not mean that they will find out something.
There was a case, one mission
arrived up to up to the launch.
Launch was successful. And I asked them, please
do the exercise. They have done the exercise
looking back to all the activities during the

(16:56):
development.
And the conclusion was there is nothing to
learn. This means what? This means that what
they have implemented
was according to the expectation. They follow everything.
And at the end, they didn't have this
unexpected experience. They didn't have this delta
where to tell someone else, look, pay attention,

(17:17):
or, you you understand?
So the exercise the exercise is is mandatory
at certain point, but it's not granted that
the outcome
is something. It could be null. You going
back to your role, as you stated, you're
the police of this organization,
at least in this function. Right? Yes. So

(17:37):
do you go to these events and you're
like, Hey, what was, or is it just
like, you're waiting for people to tell you,
do you ask specific questions? Is there somebody
that pokes them a little bit? Pokey Pokey
Pokey. Take into account that we have always
training. Okay.
We have training,
regular every year, but we have also training

(17:58):
4 teams, for example, before such of this
event.
In any case, hour is invitation.
Try to think about whether there is lesson
learned. We do not really push people or
we put people in a room, and then
you do not go out if you have
not found something. Okay?
We just invite people to think about their
experience.

(18:19):
Okay. People are free from the boss down
to the lower. We have some principle. For
example, we have the democracy principle.
The democracy principle means that everyone has the
right
to state something to learn, and the boss
cannot put a veto
on what the last person has raised as

(18:39):
something to learn. There will be the process
later to but inside the team, everyone of
the team has the same right to raise,
something. Well, that that takes a level of
trust. What you just explained
takes a lot of trust because Yes. I've
seen it in some organizations where somebody does.
That policy is kind of an, oh, you

(19:00):
see something? Say something. Please. You know what?
And then somebody says something, they're like, what
are you talking
no. No. No. We don't do that. You
know, there's, like, this culture sometimes where they
don't wanna hear the news. They don't wanna
hear what people say. And you use the
word to hear.
We use the word
to read because for us, to raise a

(19:21):
lesson learned is not just to speak and
communicate, but to write down a record.
I offer the access to a tool to
all the team members. So all the team
members,
even if by night or whenever they want,
they can type something. So then we have
records. And in front of the record, no

(19:41):
one can
stop. Okay. Well, that levels the field. Right?
That levels the availability of people to participate
and say, hey. I saw something. I wanna
say something. Exactly. So how long have you
been at this position? In my position,
I am since 2,001. So 2020
almost 22 years.

(20:02):
And we set up the process of lesson
learned in 2007. So we have already in
history. Yes. We have a lot of experience,
and the process today is much different from
the beginning in the sense that we learn
ourself.
We have internal discussion.
We have criticism.
We also interact with other organization or other

(20:24):
part of our agency in order to
understand each other,
and there is always an evolution.
The latest one, the one that I present
at the conference is the latest evolution. So
it's the way how to measure the process.
Well, it sounds like
this evolution
has produced
a very good result. The intention was good,

(20:44):
and and the intention
no. No. But, what I want to tell
you is that the intention represent
the overall
willingness
of the staff
to participate, to engage.
What is the challenge? The challenge is to
improve the organization. And overall,
since 2007,
we feel that we have evolved.

(21:05):
At the beginning, we didn't have any culture
of lesson learned. Now, yes, we do have.
I could say that still some areas are
a little bit reluctant
to raise, but okay, it is our family.
We will grow together. We will grow, and
we will bring together with us all the
children. Do you see any difference between generation

(21:26):
of who does this better? Do you see
younger people
doing this engagement better than say older folks
or is it doesn't there's no indication?
Yes. And I I believe that you are
referring mostly to the, reason why why to
raise a lesson learned. Okay?
In the past, for example, we have addressed

(21:47):
the topic of, reward.
For any new, good ideas,
it is important that the person is recognized
as the initial author of this idea. This
is why we introduce in our process, we
introduce always a feedback,
giving back the answer
to the original author

(22:08):
even in case of rejection. No. So bad
idea. But in in any case, to give,
at the end of the process, give a
feedback on how was the idea considered.
Now your example about young generation
and older people.
Yes. In general,
young people are always willing
to propose,

(22:29):
and sometimes, however, they do not know beyond
the boundary where they work.
Sometimes they propose something that maybe is already
present in beside area. But, again, I appreciate
I appreciate that someone
spend time to provide the contribution to the
improvement of the organization,
whatever it will be. Okay? Let me just

(22:50):
say, I I what you just described just
shows the level of participation.
Right? They wanna help. They they see where
they could add value. It may not be
great value, but at least they're participating.
What would you say the percentage of your
workforce
actively participate in this realm of what you're
just speaking of? You know, what would you

(23:11):
say? You say you had a few outliers,
but
As active, if if I if I have
to count
actively, I would say
half of the population
actively. At least I have record raised by
them, but, also, there are not just players
that are involved in at the level of
generating

(23:31):
records, but there are also players that are
involved in term of
assessing the record. And other that the boss,
the management
is involved in term of taking decision what
to do. And finally, others are involved in
term of implementing the decision of the boss.
Okay? I would say that at least today
is 50%
are
are actively involved in one way or the

(23:53):
other. Well, that's a that seems to be
a a very high number. In any organization,
I think that's a good number, you know,
to have that type of participation.
You know, we have already
something like, more than 15 years of experience
in this process.
Okay? There are also
weak point, and I will mention in my
my presentation. So it's not a process

(24:15):
that is,
successful
at every time, everywhere, in every corner. Okay?
But also when you say about motivation,
there were periods
where there was a decrease
of motivation in generating lessons, lesson to be
learned, especially when this lesson were going outside
our organization, were going to our customer

(24:38):
because we never received feedback from them. So
it was like to raise a lesson in
a black hole. Okay? When there are lesson
for our organization, we have our process, but
when we have lesson going outside,
so for the improvement
of others,
obviously, we have no authority. We just provide
as ideas.
We appreciate to receive a feedback, bad idea

(25:00):
or good idea. But if we receive no
feedback, this means that, they forget about us.
You have no idea what the value was.
There's just no reflection. There's nothing coming back.
You have no report. Yeah. No idea. Yep.
And that's a challenge I think a lot
of organizations
have. You know, the the concept in the
social media would be called a lurker. You
know, somebody doesn't participate. They just kinda you

(25:22):
don't know what value they're getting or what
value they're giving.
It's just the unknown. The maybe we'll never
get to that unknown piece, but maybe AI
will fix that. I don't know.
Alright. So I've got one more question for
you. Yes. What food item do you miss
the most from Italy? Oh,
I am, mostly

(25:42):
from Milan,
and my preferred food, Italian food, is risotto.
Risotto alla Milanese.
This is the best of my dream.
But but look. I have a very nice
wife, Italian wife, and she is preparing for
me,
and so I'm happy. I'm really happy.

(26:03):
And finally, where is the best wine from
in Italy?
I cannot answer this because there is a
good wine for every food. No. No. You
cannot
you cannot for example There can't be just
one. There just can't be one. No. No.
It is a culture. It is a culture.
Okay? You cannot associate

(26:23):
any the best wine with the best pizza.
No?
Or for the best pizza, you need a
certain certain thing. Okay?
So I I just tell you one which
is not the best wine, but is the
best from my town Okay. Where I came
from. I'm I'm born in Modena. Modena is
the town of, Ferrari, Pavarotti,

(26:44):
and, the food is very heavy,
a lot of fat. And and the why
the wine associated to this is very light
and also with sparkling
sparkling wine. It is called Lambrusco.
Lambrusco, if you want, is
the Italian Coke.

(27:05):
It is light. It's not appreciated
around in the world, but it's associated to
to this food is perfect.
Well, thank you for that advice and that
lessons learned about Italian food. I appreciate that.
Thank you for being here today, Stefano. Okay.
I hope I have answered everything. And.

(27:31):
Thank you for listening to because you need
to know, the reference podcast in knowledge management.
My name is Soni Tonebi. As an art
administrator,
because you need to know has been my
go to podcast and has helped me hone
my management skills. Please consider sponsoring the podcast
with your business.

(27:54):
Thank you for joining this extraordinary journey, and
we hope the experiences gained add value to
you and yours.
See you next time at because you need
to know. If you'd like to contact us,
please email
byntk@pioneer
dashks.org,
or find us on LinkedIn.