October 19, 2022 • 45 mins
It's dangerous to go alone! Take us with you!
In this episode, we're joined by our very own Jon West and Nikhil Punathil at Esper to discuss a key part of what we do here — getting AOSP up and running on ARM and x86 hardware. If you want to port AOSP, it's not as easy as just compiling an image from Google's git repos and slapping it onto a device.
- 05:59 - What is a device bring-up? What are some of the challenges in doing a bring-up?
- 10:58 - How do AOSP developers deal with a lack of kernel source code?
- 18:16 - How did Project Treble affect building AOSP?
- 26:54 - How does Android on x86 differ from Android on ARM?
- 29:48 - What problem does the Generic Kernel Image try to solve?
- 35:22 - How long does Google support a particular AOSP release? What can AOSP developers do once support has ended?
Android Bytes is hosted by Mishaal Rahman, Senior Technical Editor, and David Ruddock, Editor in Chief, of Esper.
Esper enables next-gen device management for company-owned and managed tablets, kiosks, smart phones, IoT edge devices, and more.
For more about Esper:
- Esper Blog
- Mobile Device Management (MDM) Guide
- Android MDM Guide
- iOS MDM Guide
- MDM Solutions
- MDM APIs & SDK
New from the Esper blog:
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
David (00:02):
Hello and welcome to Android by Its Powered by Asper.
I'm David Ruddick, and each week I'mjoined by my co-host Michelle Ramen,
diving deep into the world of Android.
And this time we're diving deep intothe world of Asper because I have one
fellow Asper, uh, team member next to me.
And we have another fellow, Asper, right?
Asper, I'm not surewhat we call each Asper.
Jon (00:22):
Anyon is what we usually.
David (00:24):
That's Peren.
Okay.
I like that.
It's kind of fun.
Sounds like an alien from Star Trek.
Michelle, would you liketo introduce our guests?
Yeah,
Mishaal (00:31):
thanks David.
So as David mentioned, we have twovery special guests and of course every
guest who comes on the show is a veryspecial guest, but these are even special
to us because they are colleagues.
We have Nicki Puntil andJohn West on the show today.
I'd like to introduce bothof you to Android Bites.
Thanks for joining.
Nikhil (00:49):
So, uh, I'll, I'll start.
I'm Nick Pun.
I run Esther's Innovation Lab, sothat's where we do all the fun stuff
and just go nuts with Android and,you know, everything technology.
Jon (01:02):
John West, I help run the Sper device targets.
So we grab new things coming in andsee what changes we need to make and
make sure everything's working with thedevices and can be provisioned just fine.
Thanks for, uh, having
Nikhil (01:17):
us.
Yep.
Happy to be here.
David (01:19):
And so I guess it's fair to say both of you on a regular basis are
answering the question, can we do this?
Is it
Mishaal (01:25):
possible
Jon (01:27):
And Yes, Yes, we can.
Mishaal (01:31):
Can we build it?
Yes.
You're all Bob, the builders here.
And y'all were doing thiseven before you joined Sper.
So where did you guys come from?
What were you doing before Sperthat related to Android development?
Yeah,
Nikhil (01:42):
so I started messing with Android really a long time ago.
Around the time, I think the firstversion of Android that I played
around with was, uh, gingerbreadwith my first Android device.
It was.
It was a really crappySamsung Galaxy MP4 player.
Like it was one of the, oneof the iPod Touch competitors.
(02:03):
Those it, The Galaxy Player.
The Galaxy Player 4.2.
That was my first Android device.
Seems like ages ago, but also that device.
Never got any updates.
It was shunned by the community.
And that's really what led tome trying to see, okay, I want
this to be better than it is.
Cuz getting a new device reallyat that point wasn't an option.
(02:25):
So you gotta deal with what you have.
And that's when I started, just likechecking out the community, see what
was going on, looking at Synogen mod,which was a huge thing back then rooting.
In fact, one of the weird things that Iended up learning a lot about was smelly
at its, which is essentially compiled.
Java code for Android and working withthat and like adding mods to your OS
(02:49):
through smelly mods, it's, it was apretty interesting time and it grew
from there to me, building Android fromsource building, unofficial builds of
signage mod for the devices I owned.
To me, joining the Carbon Rom teamas a maintainer and eventually a core
team member made my way through thecommunity and I finally found home at.
Mishaal (03:10):
I gotta say, I don't envy the things you guys have
to do back then to modify apps.
Small editing small is a pain to read.
I'm so glad there are tools like J X nowthat actually convert that to readable
Java code and like Jeb Decompiler, somuch better de compilation, reverse
engineering tools, these stays.
Oh, absolutely.
So what about you John?
What did you do prior to joining Sper?
(03:30):
Oh
Jon (03:30):
goodness.
Uh, you guys are gonna hateme, but probably start way
back with Windows Mobile.
Moved in to Android when the HTC one cameout and the HTC Hero came out after that.
Pretty much started on my own for awhile there, did a bunch of stuff, and
then started on Team Bliss just as thatproject was being brought to light.
(03:53):
And soon after that becameinterested in Android X 86.
So that's buildingAndroid for PC hardware.
Moving into the realm of taking thingswhere no man has gone before and
seeing how far we can get it going.
That ended up going onfrom 2013 to present.
It's still going under newmanagement now, new developments.
So we have a bunch of developersthat are working on that project
(04:16):
while I'm working for Sper here.
And yeah, that's prettymuch my, uh, experience.
I specialize.
And the X 86 PC hardware stuff,uh, melding the worlds with
Linux and Android into one.
And that's where I get my kicks.
So how I fit into Eser here iswe get in all these devices, POS
(04:37):
devices, and they might have goalsto be on a newer version of Android.
We can simply bring them up, getthem to work and make them provision
fine over ota, using new versionAndroids, which are always updated
thanks to our foundation project.
And yeah, making customers happy nicely.
Mishaal (04:56):
Yeah, so you both come from very different Android hacking backgrounds.
Like on the one hand we have.
Who's delved a lot of work with X86 hardware, which I think is pretty
rare in the Android community cuz mostAndroid, A O S P engineers kind of
deal with arm devices like Nicki did.
And so like if you wanna bring Android,Android challenge to niche hardware, you
know, you, you're gonna need a lot ofexpertise in compiling Android modifying
(05:21):
Linux, reverse engineering binaries,doing all sorts of just hacky things
to get Android up and running on a.
Where you either wasn't meant to run oryou're trying to run a vanilla flavor
of Android on a device that originallyshipped with a heavily proprietary version
with heavy modifications to the kernel.
So there's all sorts of differentconsiderations you need to make based
(05:42):
on the base device you're tryingto modify and bring Android onto.
And it's far from easy, and I'm sure like.
By the end of this episode, you'llappreciate just how much work has
to go into actually bringing Androidonto a device that wasn't originally
running on, or you know, it wasrunning something else Android based.
So I kinda wanted to start out, butjust like some basic terminology.
So this episode wementioned device bring up.
(06:05):
What does it mean to do a device bring up?
Jon (06:08):
Device Bring up pretty much starts with either A, A device, or
B, the released source for a device.
Unfortunately, in our world, wehave to do it both ways because
sometimes we'll get sourced withouta device attached, or sometimes we'll
get device without source attach.
So either A, you're pulling thedevice information and parsing
(06:33):
your device source from the deviceitself, or B, if source is provided,
you're using that along with aos P.
In order to compile Android with allthe device house vendors, info every,
all the information for that devicein place so that it works out at the
Mishaal (06:52):
box.
All right, so before we dive into theintricacies and the many ways that you
gotta fill in the gaps for yourself, Ikind of wanted to start with a golden
example of like, what's an exampleof where everything's done right?
You basically have to just follow thestep by step instructions that are listed.
You don't have to go diggingfor something somewhere else.
And I'm sure obviously the examplewould be like Pixel, right?
(07:14):
If you wanna do a ring up on pixel,everything is handed down for you.
Like, can you mention some of thethings that Google does to make
device bring up significantly easier?
And then we'll start from there andlike talk about, you know, where others
kind of struggle to meet those needs.
Can we start
Jon (07:30):
with, uh, get Attri.
Who adds and uses proper GI attribution.
So when you look at the device tree,every change that has potentially
been made in the past is stillintact within that GI history.
This includes things that have beenadded and taken away, or support that has
(07:50):
yet to be added, can be found on their.
Or their Garrett.
This is a great tool that we don't see toooften being used properly across the web.
So even in the open sourceworld, everybody has a problem
with proper get attribution.
If you get a bsp, sometimesthey come in a tarball and get
attribution is not attached.
So we don't know what changes theywere made to the device Source
(08:13):
street and when they were made.
So this is our number one and mostimportant thing that I like to.
When I'm doing a devicebring up, and Google
Mishaal (08:22):
does
Jon (08:23):
provide this handover hands better than anybody I've.
David (08:27):
Those who are maybe not quite as technical.
Just to give you an anecdote there,you could essentially think of this as
versions of revisions of a document.
And then every change that has beenmade is documented and assigned to
whoever, whatever, whoever's responsiblefor that change, why they made it.
Hopefully they have commentsexplaining why they did those things.
And context is obviouslyimmensely helpful in any kind of
Mishaal (08:50):
project.
Right?
And Google provides you all of thatversus like with any off the shelf.
You may get the document, but you maynot get its revision history, or you
might not even get the document atall and you have to go digging and
like begging for the OEM or the ODMwho made it, who actually developed
the software to release it to you.
Or if they do release it, it's broken andyou can't actually compile it or use it.
(09:13):
What do you do then?
You just have to keep begging and askingfor more releases, so, There's all sorts
of things where this can go wrong, butlike a proper version, not just a tar
ball, which is basically just an archiveof the kernel files, but the actual GI
repository that they were themselves areprobably working on having access to that.
So you can see everything that Davidjust mentioned is immensely helpful.
(09:33):
And, uh, just
Jon (09:34):
to add context, most of the technology being released is
not being followed through withwhat Google does for good attri.
For example, the kernel source, I think itwas zing that was having trouble or maybe
a different company that was having a hardtime producing kernel source for a device.
And, uh, one of the Twitterpeople, uh, Sexy Cyboard made a
(09:58):
video of her going into the place.
And Rick Westing, the colonelsource from the developer,
was like a week long ordeal.
She compiled into one video,but sometimes it takes all that
just to get what we should.
On every device
David (10:14):
release.
Yeah, and I mean, if you've beeninvolved in the Android community,
obviously Kernel source is a ragingwar that never ends with, especially
the customization community.
Companies like Motorola are infamous forjust lagging behind on things like this.
And I know, Show me, like you mentioned,they just stop releasing kernel
source a lot from what I understand.
Mishaal (10:34):
So to be fair, like a lot of OEMs either, as I
mentioned, don't really sit at all.
Release it months late orin a half broken state.
And there are a few goldenchilds who do everything really
well, like Google, of course.
But of course, if, if you're going withoff the shelf hardware from a lesser
known brand, then chances are youmight not ever get the kernel source.
(10:58):
And what happens if youdon't have a kernel source?
What happens if all you have accessto is the pre-compiled kernel
binary that ships on the device?
What can you do with.
If anything, you can actually
Jon (11:08):
pull that pre-compiled kernel and use it as a, uh, pre-built kernel
within the device tree that allows usto deal with the lack of responsibility
from OEMs and actually still
Mishaal (11:20):
make it work.
Nikhil (11:22):
That comes at the cost of not being able to make any
necessary kernel changes, right?
So if you're starting with Android 12and you have an Android 12 kernel from
the ODM or oem, you're probably finefor Android 12, but as soon as Android
13 comes along and you wanna do a bringup of that, chances are you need to
make kernel changes and well, you don'thave the source to make those change.
(11:44):
So now you're stuck.
And some very smart people in thecommunity have figured out ways to
essentially reverse engineer and figureout they, they'll take the kernel
source from a very similar device,probably running the same chip set if
possible, from the same manufacturer,but that's not always possible.
And just figure out what needs tobe done to add changes on top of
(12:06):
that and make it boot on a device.
It always impresses me when Isee things like that cuz that's
just dedication to the craft.
David (12:13):
Well it's like kind of reminds me of like it's the Chevy small
block of, basically there had to bedevices that were especially popular
to use as basically a kernel source.
Right?
Like over the years.
Nikhil (12:23):
Absolutely.
Yeah.
Well, and the other part of it isyou, you could be working with a
weird combination of vendors whereit could be an unsupportive OEM who
doesn't wanna release sources, butthey could be running a Qualcomm chip
in there, which Qualcomm is very.
One of the most open source friendlychip vendors for Android, and so you
have generic Qualcomm sources thatare just out there and available.
(12:45):
So you have something to start from.
Right?
That's usually the best case scenariowhen it comes to unsupportive
and no source situations.
Yeah.
With
Jon (12:53):
those you can just run a diff and grab what you're missing pretty much,
and then have that as a handful of oneor two commits to add on top of that
Nikhil (13:01):
specific device.
Right.
So that's exactly what I was saying,which is you have a foundation,
and that foundation is reallywhat makes all the difference.
And you'll notice that devices withnot much support are typically devices.
Don't have open source supportfrom the tip vendor themselves.
Right?
And so, I mean, you can look atthis as a bunch of layers, right?
(13:21):
Where the real original source comesfrom, the chip vendor, Qualcomm, or
Media Tech, or Brock Chip or whoever.
We will build a new chip and theywill have a generic set of what we
call board support package, right?
BSP is just a bunch of files that youneed to add on top of Android to make
Android work on that particular chip.
(13:42):
Now, this is generic Android.
There's no modifications otherthan hardware support stuff added.
And then this is what gets handedout to ODMs OEMs and everyone else
who uses that chip and they addtheir own sauce on top of that.
At one point it becomes very recognizable.
But you do know that under thehood was that original basic
bare bones BSP that came from thechip vendor that's being used.
(14:05):
So if you have that, youhave something to work with.
And I guess
David (14:09):
the context here of why this makes things difficult is that you then have
to learn everything and more that the OEMand ODM did as part of their bring up.
So you're not only repeatingthe process, you're doing it
with less information than they
Nikhil (14:22):
had.
Absolutely.
And, and a lot of times, like, I mean, I,I say Qualcomm is a good example, right?
But even Qualcomm, and rightfullyso, they don't actually have
fully open source binaries.
There are still closed sourceproprietary binaries where they have
their own secret sauce and their, Imean, big sense iss P gpu Exactly.
Things like that.
Things like that.
So especially when you're dealing with anend of life device or an end of life chip.
(14:46):
You don't have updates to those binaries.
Even with supportive situationslike that, you still have to do
some hacking to get stuff working.
Yeah.
And
David (14:55):
I think that getting us too far off track, that's part of why we're
seeing Google try to modularize Androidmore so that we can start updating these
things standalone and stop basicallylike making everything beholden to BSP
Jon (15:06):
version.
Absolutely.
And that was their goal for projecttravel or travel support when they
introduced that into uh, I think Android.
Was the first introduction for itthat made it separated, kind of where
we had a vendor partition that wouldcontain all the device information, ODM
stuff, and that would be static thatwould only be updated by the vendor
(15:29):
and then everything else around it,the system partition, product system
extended, et cetera, would all beupdated by the OEM or Google the source.
For Android.
So that made it kind of simpler forsome forward parts, but a little bit
less simpler cuz we still have topull that vendor image, parse it,
figure out what all is added to it.
(15:51):
And if we plan on doing anythingwith it, it has to be mostly private.
Unfortunately,
Nikhil (15:56):
and things are moving in a good direction too, right?
So now Google has thegeneric kernel image concept.
Where that would come in handy is inthe future where most devices do come
with gki support, which Google enforces.
You wouldn't even need the kernel sources,which technically you're supposed to
get, but a lot of times you don't get,as we talked about earlier, right?
You could still run a generic kernel.
(16:18):
Top of your device that'srunning the OEM specific kernel
image and still get it working.
And so now you have the ability toswitch out everything on your device
to generic versions and just swap outthe entire operating system basically
without any help from the OEM or oem.
I think that's the ideal goal thatGoogle is aspiring towards and
seems like we're getting there.
(16:39):
Right,
David (16:40):
and you know, just.
Again, is a kind like kindof very high level check.
This also assumes that you can get intothe device, which is a big assumption
we have to make for any of this, right?
So you may either need cooperationand existing exploit or something
else, which I don't, I'm not sure what
Nikhil (16:57):
other routes there would be.
Well, there's a few ways, right?
So at least in a professional space whenit comes to Android devices and building
a O S P, , you have two situations.
One is where you have the bsp,the whole package that the ODM
uses themselves or gives out toother people who need it, right?
In that situation, the images you buildand the output really is something
(17:22):
that you can term as a whole package,meaning like the entire operating
system files, including stuff outsideof Android that is needed by Android,
but it's technically not Android.
It's quite a full system image, right?
Boot loader and other.
In the device.
You have all of that and.
Basically use whatever flashingprocess that the chip vendor has
(17:43):
determined for their board to flash it.
And at that point, you'reacting like the odm, right?
You're doing what theywould do in the factory.
And so that's the more clean process.
But in the community and as aconsumer, you don't really get
access to that with any odm really.
Right?
And so at that point, you're workingwith a S P and you're working with
whatever sources they've dropped, andyou're just building what you need.
(18:05):
To swap out and bring it up to a newversion of Android or update whatever
you need to update and leave those otherauxiliary partitions just untouched.
So there's like two levels to this.
Mishaal (18:16):
We briefly touched upon project trouble before jumping into generic
kernel image and then you know this topic.
But I kind of wanted to step backand focus more on Project Trouble
because it's such an important.
Part of what makes device bringup pretre versus post trouble.
So different.
So I wanted to ask you, can youexplain how a device bring up differs
before product troubles introducedversus after it was introduced?
(18:38):
Like what is the fundamental differences?
Pretre,
Jon (18:42):
everything was pretty much built into system is root.
So all of your device information, all ofthe vendor applications were all included
in one system, partition on the device.
Post travel, all of that is separatedinto a system, a product, a system
extended, and a vendor bar issue whereyou have access to all the separate
(19:06):
parts, but they're mounted within thesystem a little bit differently and
separated on the device themselves.
So when we update on a projecttravel device, we're only
updating the system parti.
And possibly product, uh, vendor systemextended as well from the actual OEM or
the vendor that leads it to where theycan do security updates much faster
(19:30):
without having to rebuild everythingthey've added previously on top of
a brand new security release onp.
They can easily add those patchesinto the system and it's all kept.
Mishaal (19:43):
When Google introduced trouble, when they decided to strip the
vendor specific hardware abstractionlayers from the system image into its
own dedicated vendor partition, theyalso defined an interface, a standard
interface between the vendor partitionand the Android operating system.
So to allow the OS to communicatewith those hardware abstraction
(20:03):
layers in a standardized way.
And they call that the vendor I.
I wanted to ask you, like, can you talka bit about this vendor interface and how
its introduction affected device bring up?
Well,
Jon (20:15):
there were two ways of vendor interface was added.
It was, uh, added either a through thevendor image that the hardware vendor
would create and that would containall the added files that they needed.
For I D L interfacing, it would alsocontain some of the information that they
need for external apps that they're addingonto the system and services as well.
(20:40):
They also introducedthe V N D K interface.
And that is pretty much where allthe hardware interaction goes.
So anything that goes through V NDK has to be certified pretty much
to work with their device interface.
I don't have too much experienceworking with the dk, Maybe does.
Nikhil (21:05):
The BDK is less talked about aspect of travel.
You know, when we say travel, talkabout project travel, typically you
talk about how things are separatedbetween system and vendor, but.
Part of that separation was Googledefined that interface, like you said.
Right?
And VDK is an important part of it.
Vdk, by the way, stands forVendor Native Development Kit.
(21:26):
And what it basically does is it givesthese ODMs and chip side vendors is
standardized way to define their Halsand their hardware support libraries.
And what it also does is it enforcesbackwards compatibility, right?
So if you have a vendor that wasfor Android nine, Then by Google's
(21:47):
compatibility definition, youwould have to make sure that that
vendor is or compatible up tothree versions I think, of Android.
So that means you coulduse that same vendor.
And a system image from Android 12say, and it would work perfectly fine,
and they have to work well together.
And this is the sort of rulesthat Google is setting that
makes device bring up easier.
(22:08):
So if you have a device thatdoesn't get updated that often,
it's running on Android 10 andyou wanna run Android 13 on it.
You don't really have to messwith the vendor layer at all.
Maybe perhaps slight modificationsdepending on device specific works,
but for the most part you can leavethat vendor part untouched and work
on the upper layer, so to speak.
(22:29):
And, and that really is the beautyof trouble is there's definitions
now and there's standardization of.
This communication betweensystem and vendor, and I
David (22:38):
guess is a good way to describe that, really, how the hardware
extract itself to software andsays, I can do this, I can do that.
And here's how you call that, right?
Essentially like the camera can zoom.
Therefore you should have a hook inhere for your vendor, specific camera
implementation if you have a special one.
For Zoom, not to get into Googleor Android camera AP guys, cuz
(22:59):
that's a whole other thing.
But in general, conceptually, isthat kind of the way that works?
Nikhil (23:04):
Right.
And it doesn't really block manufacturersor Google from implementing new things.
It just makes sure that whenyou do implement new things,
you have to make sure that.
It doesn't break compatibilitywith the older versions of the vdk.
Right.
So that's why you see, I think ifyou're in the Android community now,
new versions of Android, especially likeCustom ROMs are dropping much sooner.
(23:25):
It's because they don't reallyhave to spend, That's really
where most of the time is, right?
Because the vendor layer and theHals and the hardware support
stuff, that's where all the IP is.
That's where you have the.
Source code access, and so that'swhere you have to do the most.
Hacks and hacks are usually trial anderror and usually take a lot of time, and
people who do this for free in their freetime tend to not have much free time.
(23:48):
That's really what helped things getbetter over the years, especially
with trouble because you don'treally have to do much anymore.
For example, if you're waiting onone plus to drop their Android 13
builds, you can still get Android13 as a custom realm on your.
While you wait for that Android 13,and when it does drop, you can have a
newer build, which uses that updatedvendor that's more suited for Android
Jon (24:12):
13.
A lot of times that updatedvendor will have updated firmware
for the device, et cetera.
So Nel Image, that type of stuff.
Mishaal (24:21):
Technically is a generic system.
In the one plus example with theirrecent devices, things are getting
more complicated with the Googlerequirements freeze program.
So a lot of modern flagship devicesare now actually shipping older
vendor software implementations thatwere built for an older release.
So for example, a devicethat launched with Android 12
that upgrades the Android 13.
Might still be using the samevendor software implementation that
(24:43):
was developed for Android 12 whenit upgrades, and that's a quirk
of Google requirements freeze.
We talked about that before, but I don'twanna get into the exact nitty gritty
of that aspect right now, but I kindof wanted to touch upon a bit more on
the genericization of the interfacebetween the hardware extraction layers
and the OS, and why that's so important.
So let's say pre-project trouble, ifyou were to try to just flash an A
(25:06):
S P system image onto the device andhave it booted up, if the OEM of that
device had some custom hardware andthey wrote their hardware abstraction
layer in a nonstandard way that a S Pwouldn't be able to interface with it.
Then when you boot up a P, that pieceof hardware that that abstraction
layer was written for just mightnot work or it might just crash.
So like if you had a camera andyou were expecting it to boot up,
(25:28):
when you open up the a SB cameraapp, you might just get nothing.
You might see nothing at all.
And that's obviously a hugeproblem because you're dealing with
fundamental hardware components ofa device that you would expect to.
Out of the box.
But with project trouble standardizinga lot of these base components and
Google enforcing this through thesevendor test suites, their, their vendor
(25:49):
interfaces, you know, all these thingslike the diversions that OEMs to follow.
Like it ensures that if you take amodern project trouble compatible
device and you boot AOS P onto it,Chances are that at least almost all
the basic hardware components will work.
So like for example, we have a Lenovotab K 10, that we all frequently use
for work purposes and that deviceship put Android 11 outta the box.
(26:11):
I've been able to boot Android 1212L Android, 13 generic system images
onto it and all the hardware works.
A s p works perfectly for just fine.
There's no issues with it at all,and that's thanks to project.
Nikhil (26:22):
And we're where to the point where you can't even tell the difference
between a GSI and a dedicated image.
Right?
So unless your device has specific,like when it comes to one plus devices,
they have that alert slider, or theyhave the in display fingerprint sensor.
Those are specific hardware choices thataren't really a part of a s P or gsi.
But for a generic device like aLenovo K 10, it's a decent tablet
(26:45):
that has everything you need.
If you go to Gs.
It works.
Every feature you want to work works.
And so at this point, I mean the sortof direction we're heading into is
we will just have a Windows or Linuxlike situation where you have a build
of Android that drops that you canjust install on any device and it
more or less behaves the same nextstep for, you know, hardware specific
Jon (27:06):
works.
Let's talk about the difference therethough with Windows, Linux, and Android.
So on Windows, it ships withmonolithic kernel a lot like Linux
does well, So that contains all thedevice information, the drivers, the
firmware for the drivers, et cetera.
And hopefully 90% of that is detectedupon a net, or when it first boots
(27:31):
up on windows, it detects a list ofwhat's available, downloads what it
can, after connection is available, andthen, uh, pairs it up based on that.
But with Android, it's a.
Find a bit different because itrelies on what's on the device.
None of that extra stuff is everincluded on the Linux NEL that's sent
(27:52):
with Android, at least for most devices.
If you're dealing with an AndroidX 86 device, we did switch to a
monolithic kernel and it kind of pairsup hardware available using mod probe
and if probe exists, mod probe this.
If found great at it to thelist, if not, great, ignore it
and then continue booting when.
(28:14):
So it'll go through every potentialconnection in your device until it
finds it all just like Linux doesand pairs it up with that device.
Hardware on Android, especiallyusing Project trl, it's a bit
different because you have thatproduct definition already set up.
So it's gonna be shipped with that productimage, vendor image, and that's gonna
(28:35):
contain all the information they needto load the kernel, load the drivers,
load any extra blobs, and drive devicehardware for those drivers, and then
can make the connections and then boot.
Probably
David (28:47):
philosophically there's some really good reasons that Android ended up this
way, aside from inheriting from Linux.
But number one, Android was alwaysdesigned to be ultra lightweight.
And so by being able to define the firmerimage at a very high level, you can strip
out everything that is unnecessary andhave a super lightweight device, which
was very important in early Android,especially . You did not want bloat.
(29:11):
Everybody hated bloat.
But I think the other side is probably.
For Google, you know, this is a betterway for them to pull the levers.
They want to controlling the ecosystem.
They can do it very high up as opposed togenericizing, which they have been doing.
Um, that's a harderproblem to solve, right?
It's a lot easier to make a ruleand say, You vendor can do this.
(29:33):
You vendor can't do that.
Rather than giving them a toolthat says, Okay, we can make this
more flexible, adaptable, which.
Really what trouble I think was probably
Mishaal (29:41):
about, This actually brings us back to the generic kernel
image topic and the reason whyGoogle went for that in the first.
Since, as you mentioned, Androiddevices, they generally chip with
their device drivers on the device.
Not everything is included in theLinux terminal, and if you just boot
Linux onto an Android device, youcan't expect everything to work.
So the problem is that a lotof open source developers have
(30:02):
been pushing for Android devicemakers and vendors for years to.
Upstream your drivers open source yourdrivers and submit them to Linux and
let us take care of maintaining that.
But that has never happened.
That pushback.
It just just didn't happen.
No, it's, There was no ip.
Jon (30:17):
All these device manufacturers want that IP to stay private.
So I
Mishaal (30:21):
completely understand why.
Yeah, exactly.
But like then that becomes a problembecause there's so much out of tree
coat or there used to be so muchout of tree code running on, you
know, in the Linux NEL that ships.
Your average Android device and, andso if device drivers aren't being
upstreamed, then it kind of limits thelongevity of those devices, the ability
to upgrade those kernels and implementsecurity patches, because there's so
(30:44):
much of a difference between the kernelthat ships on a device and the upstream
Linux kernel that a lot of work hasto be done to manage those merges.
So what Google decided to do is,Okay, we'll let you keep your kernel
drivers closed source, but instead yougotta ship it as a kernel module that
interacts with the generic kernel imagein a standardized way, specifically
Jon (31:04):
built kernel module that interfaces a specific way using gki.
Mishaal (31:09):
Right?
So on a modern gki deviceright now, so like the boot.
It has the, the generic NEL image.
So as Naqui mentioned, you can go,literally go and download Google
signed generic kernel image, andthat's going to be the same kernel
that ships on other gki devices.
But what actually enables the kernel totalk to the device specific hardware?
Is those kernel modules thatare specific to that device, and
(31:32):
that's contained on a separatepartition from the boot partition.
Jon (31:35):
And that's automatically detected and loaded as soon as Android
zygote loads on the boot to process.
Nikhil (31:41):
It's, It's interesting to see how we got here though, right?
So you were kind of touching onthat, David, but if you look at why
Android is the way it is, I thinkfundamentally, The goal with Android
was to appeal to developers, right?
So the architecture itselfis designed to make sure that
applications are easy to write, right?
So for example, instead ofhaving to interface with hardware
(32:03):
directly like Linux, you havehardware abstraction layer.
So your application doesn't have tochange depending on how the hardware
features are implemented per device.
But that means.
Device vendors have to put moretime and effort into defining
these hardware abstractionlayers specific to their devices.
And when they put in that effort, theydon't want to put all that information
(32:25):
out there for the world to see.
So now you have people movinga lot of hardware specific code
from the kernel where it wouldtypically reside in a regular device
to vendor binaries like blogs.
Ah,
David (32:36):
so it's kinda a shell game thing that
Nikhil (32:38):
they're encouraging.
Yeah, it really is.
Yeah.
And so now you.
The idea of close source as much aspossible with like minimal changes
to the kernel, which makes Androidas an OS very highly device specific.
. And so you don't have the conceptof a generic, Well, you didn't have
a concept of a generic OS imagelike you do with Windows of Linux.
Jon (32:59):
Yeah.
There is no one size fits all anymore.
And I guess you, but there kinda is.
That kind is, and there kind isn't.
You can't take a brand new devicethat has no information provided
for it from the manufacturer and getit working like you can with Linux.
Linux.
You'd be able to take what we workedon this last HP desktop for example.
(33:20):
And the Linux kernel that has all thathardware, the monolithic kernel, and
it'll automatically mod probe and lookfor any existing hardware, and that
can probably get 90% of it workingon a new device with a new chip set.
That is not doable with Android anymorethough, and that's one of the downsides.
I
David (33:39):
mean, there's also of the business and partner model of
Android is very different fromlike Microsoft and Windows, where
Microsoft says, You wanna work on us?
You gotta talk to Windows daddyand submit your driver for
signing . Like there's no other way.
Like that's how you make thingswork on Windows, Whereas Android.
There are so many vendors at somany different layers, and also you
have vertically integrated vendorswho are doing things that multiple
(34:01):
vendors would otherwise be doing.
Whereas with Windows, Microsoft hadthe clout in the authority to say,
You will build a Windows computerlike this, Android and Google.
They've had less authority, I think,until recently, the last five.
But how
Jon (34:14):
far innovation's gone by letting the OEMs have that ability?
Totally.
With Android devices, I mean,we've gone three, four times
what Windows was able to.
Right.
Yeah.
And I
David (34:25):
think, you know, one of the, where I was going with this, and I will stop
after this, is that smartphones come froma much more dedicated device mindset.
They come from telephones.
Telephones had a verylimited number of things.
They just started doing more andmore and more of them, and so
the smarter telephones got, youneed a firmware on them, right?
They got smart enough that you hadto have an operating system under
(34:46):
telephone, and so they come from that.
This is a device with a list offeatures that does these things.
You rely on it for that.
Windows comes from, this is acomputer, you use it to make stuff.
And I think that actually informedprobably the way manufacturers
look at firmware for these devices.
They probably looked at, especiallyearly on as fixed in place.
I make a product, it has firmware,it stays stable, which is in our
(35:08):
world something many people expect.
But in the consumer world, especially whenit comes to general computing devices, no.
They expect they get better.
And Android has started todo that with smartphone.
Obviously, but that's a pretty big sea
Mishaal (35:20):
change.
I.
Yeah.
David, you'd mentioned that you know,it's true that Google has less of an
iron grip over a s p and Android thanMicrosoft does over Windows, but they
still do have a very size of control.
Yeah.
Especially when it comes to requirementsto getting nude Android versions up and
running, or certified on an older device.
So for example, like.
(35:41):
It's true that Google is genericizedAndroid to the point where you can take
an A O S P GSI for like Android 14, forexample, and you could boot it on a device
that originally launched with Android 12.
But can you do that with Android 16?
Maybe not, because Google tends toonly support backward compatibility
for three letter upgrades.
(36:02):
So like if you have a device that launcheswith Android 12, Google will make sure.
That Android 15 won't include anynew requirements that break backward
compatibility with Android devicesall the way back to Android 12.
But they don't guaranteeany background fat ability.
After that.
It's generally just three letterupgrades and you can kind of see
this extended across everywhere.
Like if you look up, how long does Googleprovide security Backport to AO s p?
(36:26):
It's generally only threes versions.
So like right now, if you look onthe security bulletin, they support
Backporting patches to Android10, because that's three letter
upgrades before, but soon that'llstop once they launch Android 14.
And this also goes into, it'snot just becoming difficult
to port a new Android version.
Sometimes they'll just.
Entire features that you might depend on.
So for example, I think Lineage,when they released their Android 12
(36:49):
builds, they had to drop a ton ofolder devices because those devices
didn't have a Linux kernel version newenough to support EPF F, which is like
the Berkeley packet filter feature.
It is like a kernel feature that'susually for like network filtering
or firewalls and stuff like that.
And so they had to drop a ton ofdevices because of that, because
Android straight up acquired it forAndroid 12, and those older devices
(37:10):
with older carnivals just couldn't.
Nikhil (37:12):
It's interesting to look at Android as sort of a pre-market, right?
So you have Google try to enforce itlike the government with their GMs
requirements and the compatibilitydefinition document and whatnot.
And then you have the whole AOS feespace where there's no laws, It's
people doing whatever they want.
ODMs, they don't have tofollow any rules, right?
(37:33):
And so how do you enforce.
The concept of a standard in sucha space where essentially the
goal is to cut as many cornersas possible to a working device.
And we talked about how Android is morerigid in terms of as an operating system.
That really helped here, if youthink about it, because what's
the goal of an Android device?
Right?
It should run all the Androidapps that are available, right?
(37:55):
Right.
So if apps don't.
Then your device doesn't work.
And so because apps rely on this nicelayer of communication, the how and
the interface between the system andthe hardware, you need to write that
well enough where apps don't break.
And so it's an interesting problemfor Google to solve because.
They can only do so much in the aospspace and for an open source operating
(38:18):
system with tons of people using and tonsof vendors and tons of manufacturers.
It seems like it's worked outalmost as well as it could.
And
David (38:27):
I think to Google's credit, honestly, the fact that they
seem to have the confidence thatleaving AOS Aosp out there is not
something that's going to hurt them.
It's something.
The changes they make in proper Androidare going to percolate basically into the
way people use AO s p, because guess what?
The ecosystem is gonnamove in that direction.
If you're Qualcomm or media techor even rock chip, you are going to
(38:50):
encourage then ODMs and OEMs to dothings because you have to do them.
So there is definitely risingtides kind of effect sort of
thing, you know, follow the leader.
But yeah, I mean, aos P obviously hasled to so many different innovations.
Car OEMs build their infotainment onAosp and have for years because it was
a much more reliable and lightweightway to do it than building Linux.
(39:12):
Like it was just like, Oh, this is designwork with Bluetooth and wifi and all
the other like kind of communicationstandards you would expect of a modern os.
And that is because the Android wasdesigned to be totally wireless connect
with all standards of humanly can.
Just generally be pretty flexiblethat way because Google wanted
OEMs to experiment, so I do.
You
Nikhil (39:32):
mentioned the app ecosystem, right?
Right.
They don't have to build anything.
There's already a huge, probably thelargest app ecosystem out there that
you don't even have to go throughGoogle certification to access.
Right.
You could just side load an apk.
Right?
And you have YouTube running onyour OSP head unit and your card.
I think the modularity and the opennessof a USP is why it's super popular.
Mishaal (39:53):
Well, just a small caveat on that, a lot of apps will require
your device to have GMs or GoogleMobile services in order to access
the APIs provided through it.
Our previous episode on GooglePlay Services covers that in
detail if you wanna learn more.
And yeah, I also wanted to kind ofcaveat something I mentioned before.
You know, I mentioned that.
Google makes it difficult tosupport new Android versions.
They drop support for olderfeatures of, they update their
(40:16):
requirements after three letter upversions, but it's not impossible.
Like David mentioned, theAosp is a wild, wild west.
You can do whatever you want.
If you have the technical knowledge andexpertise, you can bring up Android.
For as many versions as you want.
If you have extensive Linux kernelexperience, you could keep porting newer
versions of Linux onto an older device.
(40:36):
Like it's not impossible.
It's just really, really,really freaking hard.
You can
Jon (40:41):
also do both and back Port Linux modularity from a monolithic kernel
into a newer Android type build.
And this is a lot of what we seehappening with our work here at
Sper and elsewhere in the communityfor all like Android X 86.
Project Way, droid, those types
Nikhil (41:00):
of projects.
Yeah.
And with the community, you seethese exceptional cases, right?
Where you have devices that werelaunched with ice cream sandwich running
Android 13, or dine running Android 13.
It's, it's insane.
And how these developers get Android,like a new version of Android
came out 10 years after the lastupdate was issued to that device.
(41:21):
Working as if it was built by the oem.
That's the beauty of open source.
Hey man, my
Jon (41:26):
HTC Hero was updated for like 10 years, thanks to the
Mishaal (41:29):
community.
Oh, you remember the HTC HD two, thatlegendary phones that would never with the
Nikhil (41:36):
slide.
David (41:39):
That was one of those
Jon (41:40):
at first came with Windows Mobile, I think.
Nikhil (41:43):
Yep.
The device I've worked with a lot was theSony Xperia Z one, and that beast was up.
I think the last update Sony gaveit was 5.1, so Snap Track in 8 0 1.
800.
800.
David (41:56):
Wow.
800,
Nikhil (41:57):
okay.
The first cryo chip.
Yeah.
Last I checked it was Android 12.
Right.
So from five to 12, this people havebeen, uh, I think I was involved all the
way until Android 10 and after that otherpeople have taken over and just, uh,
kept updating and Android 11, Android 12.
In fact, the whole Sony platformis one of a great example of the
(42:18):
communities coming together andjust giving life to a legacy device.
David (42:22):
And you know this, this kind of gets into the question of like,
why can't we extend Android forever?
And I think one of the reasons isAndroid, unlike Linux and Windows
does not run on devices that are likeeasily air gapped from the internet.
Android doesn't reallywork without the internet.
Let's be real.
An Android device without internet is kindof a brick, unless you wanna use it as a
word processor or calculator or something.
(42:44):
That means though, you have to keepup with the internet, and that's like
Michelle, you bring up that part of LinuxNEL for firewall, like packet handling.
That is an example of thingsthat you just like, well,
somebody would have to build it.
And nobody's gonna do it.
And that's just kind of partof the ever evolving nature of
connected and cloud technology.
If your device can't talk to thesystems that it needs to to do
(43:07):
those things anymore, you get stuck.
And that's what happenedwith the HTC Hero, I believe.
Basically, it couldn't talk to theGoogle servers anymore to like get
like wifi credentials basically,so it wouldn't get certifi.
You'd have to connect it viaethernet cable over USB otg.
And even then, thatdidn't really work, right?
Like we had Ryan, one of oureditors tried to do this at AP and
(43:27):
it was just a nightmare to try toeven get this phone to load Gmail.
But yeah, it's differentthan a Windows computer.
You can buy a 20 year old Windowscomputer, open up Internet Explorer,
and probably getting parts ofthe internet to mostly load or
download Chrome, whatever, you know.
Well, I think we are at about time.
This was a wonderful chat about, you know,I mean, I learned a lot in the last hour,
(43:50):
quite frankly, about the practicalitiesof building A O S P, which is really
at Asper, what we do on a daily basis.
Take things that run what you wouldn'tprobably Google wouldn't call Android,
but that for everybody else's purposes,Yeah, they run Android and there are
some things that you do have to change.
Now John, um, works on our X 86foundation product and obviously that
(44:11):
is so unusual in the world of Android.
Jon (44:14):
We are doing, it does use a monolithic nel, so that's the project that
melds everything together and some mashup.
And I had no idea.
Best of Linux, best ofAndroid, and we make
David (44:25):
it work.
Peanut butter and jelly.
So if you are interested in learningabout building from AO s p, and
like what it takes to make a deviceor to make a device work the way
you want, come talk to usper.io.
Mishaal (44:39):
All right, thanks David.
And thank you John and Nala for joiningus and uh, we do this with all our guests.
We wanted to ask you now, like,where can people follow you if they
wanna follow your work online orsee where you tweet or do whatever?
Jon (44:50):
I'm on Twitter for the most part, so Twitter at E L E c T r i k J e s U
Nikhil (44:57):
S and not on Twitter.
So and I'm.
I'm kind of thinking,where Can't you follow me?
I don't know.
GitHub.
Yeah, I
Mishaal (45:07):
was gonna say
. Jon: There's that too.
I'm also in GitHub.
Same place.
Well, maybe it's good that you'renot on Twitter because, uh, it's a
place where, uh, nuance goes to die.
All right.
Well, this show is a placewhere Nuance doesn't die.
We love talking to experts every week.
You know, every other week, kind of aregular schedule right now about Android,
(45:30):
A O S P, the whole ecosystem and platform.
So thank you again, both of you,for joining us to talk about,
you know, device bring up.
And I hope those of you listeninglearn a thing or two about this.
And if this sounds interestingto you, as David mentioned,
come check us out@esper.io.
Now show some interest.
Jon (45:45):
We wanna do more of these.
Hello and welcome to Android by Its Powered by Asper.
I'm David Ruddick, and each week I'mjoined by my co-host Michelle Ramen,
diving deep into the world of Android.
And this time we're diving deep intothe world of Asper because I have one
fellow Asper, uh, team member next to me.
And we have another fellow, Asper, right?
Asper, I'm not surewhat we call each Asper.
Jon (00:22):
Anyon is what we usually.
David (00:24):
That's Peren.
Okay.
I like that.
It's kind of fun.
Sounds like an alien from Star Trek.
Michelle, would you liketo introduce our guests?
Yeah,
Mishaal (00:31):
thanks David.
So as David mentioned, we have twovery special guests and of course every
guest who comes on the show is a veryspecial guest, but these are even special
to us because they are colleagues.
We have Nicki Puntil andJohn West on the show today.
I'd like to introduce bothof you to Android Bites.
Thanks for joining.
Nikhil (00:49):
So, uh, I'll, I'll start.
I'm Nick Pun.
I run Esther's Innovation Lab, sothat's where we do all the fun stuff
and just go nuts with Android and,you know, everything technology.
Jon (01:02):
John West, I help run the Sper device targets.
So we grab new things coming in andsee what changes we need to make and
make sure everything's working with thedevices and can be provisioned just fine.
Thanks for, uh, having
Nikhil (01:17):
us.
Yep.
Happy to be here.
David (01:19):
And so I guess it's fair to say both of you on a regular basis are
answering the question, can we do this?
Is it
Mishaal (01:25):
possible
Jon (01:27):
And Yes, Yes, we can.
Mishaal (01:31):
Can we build it?
Yes.
You're all Bob, the builders here.
And y'all were doing thiseven before you joined Sper.
So where did you guys come from?
What were you doing before Sperthat related to Android development?
Yeah,
Nikhil (01:42):
so I started messing with Android really a long time ago.
Around the time, I think the firstversion of Android that I played
around with was, uh, gingerbreadwith my first Android device.
It was.
It was a really crappySamsung Galaxy MP4 player.
Like it was one of the, oneof the iPod Touch competitors.
(02:03):
Those it, The Galaxy Player.
The Galaxy Player 4.2.
That was my first Android device.
Seems like ages ago, but also that device.
Never got any updates.
It was shunned by the community.
And that's really what led tome trying to see, okay, I want
this to be better than it is.
Cuz getting a new device reallyat that point wasn't an option.
(02:25):
So you gotta deal with what you have.
And that's when I started, just likechecking out the community, see what
was going on, looking at Synogen mod,which was a huge thing back then rooting.
In fact, one of the weird things that Iended up learning a lot about was smelly
at its, which is essentially compiled.
Java code for Android and working withthat and like adding mods to your OS
(02:49):
through smelly mods, it's, it was apretty interesting time and it grew
from there to me, building Android fromsource building, unofficial builds of
signage mod for the devices I owned.
To me, joining the Carbon Rom teamas a maintainer and eventually a core
team member made my way through thecommunity and I finally found home at.
Mishaal (03:10):
I gotta say, I don't envy the things you guys have
to do back then to modify apps.
Small editing small is a pain to read.
I'm so glad there are tools like J X nowthat actually convert that to readable
Java code and like Jeb Decompiler, somuch better de compilation, reverse
engineering tools, these stays.
Oh, absolutely.
So what about you John?
What did you do prior to joining Sper?
(03:30):
Oh
Jon (03:30):
goodness.
Uh, you guys are gonna hateme, but probably start way
back with Windows Mobile.
Moved in to Android when the HTC one cameout and the HTC Hero came out after that.
Pretty much started on my own for awhile there, did a bunch of stuff, and
then started on Team Bliss just as thatproject was being brought to light.
(03:53):
And soon after that becameinterested in Android X 86.
So that's buildingAndroid for PC hardware.
Moving into the realm of taking thingswhere no man has gone before and
seeing how far we can get it going.
That ended up going onfrom 2013 to present.
It's still going under newmanagement now, new developments.
So we have a bunch of developersthat are working on that project
(04:16):
while I'm working for Sper here.
And yeah, that's prettymuch my, uh, experience.
I specialize.
And the X 86 PC hardware stuff,uh, melding the worlds with
Linux and Android into one.
And that's where I get my kicks.
So how I fit into Eser here iswe get in all these devices, POS
(04:37):
devices, and they might have goalsto be on a newer version of Android.
We can simply bring them up, getthem to work and make them provision
fine over ota, using new versionAndroids, which are always updated
thanks to our foundation project.
And yeah, making customers happy nicely.
Mishaal (04:56):
Yeah, so you both come from very different Android hacking backgrounds.
Like on the one hand we have.
Who's delved a lot of work with X86 hardware, which I think is pretty
rare in the Android community cuz mostAndroid, A O S P engineers kind of
deal with arm devices like Nicki did.
And so like if you wanna bring Android,Android challenge to niche hardware, you
know, you, you're gonna need a lot ofexpertise in compiling Android modifying
(05:21):
Linux, reverse engineering binaries,doing all sorts of just hacky things
to get Android up and running on a.
Where you either wasn't meant to run oryou're trying to run a vanilla flavor
of Android on a device that originallyshipped with a heavily proprietary version
with heavy modifications to the kernel.
So there's all sorts of differentconsiderations you need to make based
(05:42):
on the base device you're tryingto modify and bring Android onto.
And it's far from easy, and I'm sure like.
By the end of this episode, you'llappreciate just how much work has
to go into actually bringing Androidonto a device that wasn't originally
running on, or you know, it wasrunning something else Android based.
So I kinda wanted to start out, butjust like some basic terminology.
So this episode wementioned device bring up.
(06:05):
What does it mean to do a device bring up?
Jon (06:08):
Device Bring up pretty much starts with either A, A device, or
B, the released source for a device.
Unfortunately, in our world, wehave to do it both ways because
sometimes we'll get sourced withouta device attached, or sometimes we'll
get device without source attach.
So either A, you're pulling thedevice information and parsing
(06:33):
your device source from the deviceitself, or B, if source is provided,
you're using that along with aos P.
In order to compile Android with allthe device house vendors, info every,
all the information for that devicein place so that it works out at the
Mishaal (06:52):
box.
All right, so before we dive into theintricacies and the many ways that you
gotta fill in the gaps for yourself, Ikind of wanted to start with a golden
example of like, what's an exampleof where everything's done right?
You basically have to just follow thestep by step instructions that are listed.
You don't have to go diggingfor something somewhere else.
And I'm sure obviously the examplewould be like Pixel, right?
(07:14):
If you wanna do a ring up on pixel,everything is handed down for you.
Like, can you mention some of thethings that Google does to make
device bring up significantly easier?
And then we'll start from there andlike talk about, you know, where others
kind of struggle to meet those needs.
Can we start
Jon (07:30):
with, uh, get Attri.
Who adds and uses proper GI attribution.
So when you look at the device tree,every change that has potentially
been made in the past is stillintact within that GI history.
This includes things that have beenadded and taken away, or support that has
(07:50):
yet to be added, can be found on their.
Or their Garrett.
This is a great tool that we don't see toooften being used properly across the web.
So even in the open sourceworld, everybody has a problem
with proper get attribution.
If you get a bsp, sometimesthey come in a tarball and get
attribution is not attached.
So we don't know what changes theywere made to the device Source
(08:13):
street and when they were made.
So this is our number one and mostimportant thing that I like to.
When I'm doing a devicebring up, and Google
Mishaal (08:22):
does
Jon (08:23):
provide this handover hands better than anybody I've.
David (08:27):
Those who are maybe not quite as technical.
Just to give you an anecdote there,you could essentially think of this as
versions of revisions of a document.
And then every change that has beenmade is documented and assigned to
whoever, whatever, whoever's responsiblefor that change, why they made it.
Hopefully they have commentsexplaining why they did those things.
And context is obviouslyimmensely helpful in any kind of
Mishaal (08:50):
project.
Right?
And Google provides you all of thatversus like with any off the shelf.
You may get the document, but you maynot get its revision history, or you
might not even get the document atall and you have to go digging and
like begging for the OEM or the ODMwho made it, who actually developed
the software to release it to you.
Or if they do release it, it's broken andyou can't actually compile it or use it.
(09:13):
What do you do then?
You just have to keep begging and askingfor more releases, so, There's all sorts
of things where this can go wrong, butlike a proper version, not just a tar
ball, which is basically just an archiveof the kernel files, but the actual GI
repository that they were themselves areprobably working on having access to that.
So you can see everything that Davidjust mentioned is immensely helpful.
(09:33):
And, uh, just
Jon (09:34):
to add context, most of the technology being released is
not being followed through withwhat Google does for good attri.
For example, the kernel source, I think itwas zing that was having trouble or maybe
a different company that was having a hardtime producing kernel source for a device.
And, uh, one of the Twitterpeople, uh, Sexy Cyboard made a
(09:58):
video of her going into the place.
And Rick Westing, the colonelsource from the developer,
was like a week long ordeal.
She compiled into one video,but sometimes it takes all that
just to get what we should.
On every device
David (10:14):
release.
Yeah, and I mean, if you've beeninvolved in the Android community,
obviously Kernel source is a ragingwar that never ends with, especially
the customization community.
Companies like Motorola are infamous forjust lagging behind on things like this.
And I know, Show me, like you mentioned,they just stop releasing kernel
source a lot from what I understand.
Mishaal (10:34):
So to be fair, like a lot of OEMs either, as I
mentioned, don't really sit at all.
Release it months late orin a half broken state.
And there are a few goldenchilds who do everything really
well, like Google, of course.
But of course, if, if you're going withoff the shelf hardware from a lesser
known brand, then chances are youmight not ever get the kernel source.
(10:58):
And what happens if youdon't have a kernel source?
What happens if all you have accessto is the pre-compiled kernel
binary that ships on the device?
What can you do with.
If anything, you can actually
Jon (11:08):
pull that pre-compiled kernel and use it as a, uh, pre-built kernel
within the device tree that allows usto deal with the lack of responsibility
from OEMs and actually still
Mishaal (11:20):
make it work.
Nikhil (11:22):
That comes at the cost of not being able to make any
necessary kernel changes, right?
So if you're starting with Android 12and you have an Android 12 kernel from
the ODM or oem, you're probably finefor Android 12, but as soon as Android
13 comes along and you wanna do a bringup of that, chances are you need to
make kernel changes and well, you don'thave the source to make those change.
(11:44):
So now you're stuck.
And some very smart people in thecommunity have figured out ways to
essentially reverse engineer and figureout they, they'll take the kernel
source from a very similar device,probably running the same chip set if
possible, from the same manufacturer,but that's not always possible.
And just figure out what needs tobe done to add changes on top of
(12:06):
that and make it boot on a device.
It always impresses me when Isee things like that cuz that's
just dedication to the craft.
David (12:13):
Well it's like kind of reminds me of like it's the Chevy small
block of, basically there had to bedevices that were especially popular
to use as basically a kernel source.
Right?
Like over the years.
Nikhil (12:23):
Absolutely.
Yeah.
Well, and the other part of it isyou, you could be working with a
weird combination of vendors whereit could be an unsupportive OEM who
doesn't wanna release sources, butthey could be running a Qualcomm chip
in there, which Qualcomm is very.
One of the most open source friendlychip vendors for Android, and so you
have generic Qualcomm sources thatare just out there and available.
(12:45):
So you have something to start from.
Right?
That's usually the best case scenariowhen it comes to unsupportive
and no source situations.
Yeah.
With
Jon (12:53):
those you can just run a diff and grab what you're missing pretty much,
and then have that as a handful of oneor two commits to add on top of that
Nikhil (13:01):
specific device.
Right.
So that's exactly what I was saying,which is you have a foundation,
and that foundation is reallywhat makes all the difference.
And you'll notice that devices withnot much support are typically devices.
Don't have open source supportfrom the tip vendor themselves.
Right?
And so, I mean, you can look atthis as a bunch of layers, right?
(13:21):
Where the real original source comesfrom, the chip vendor, Qualcomm, or
Media Tech, or Brock Chip or whoever.
We will build a new chip and theywill have a generic set of what we
call board support package, right?
BSP is just a bunch of files that youneed to add on top of Android to make
Android work on that particular chip.
(13:42):
Now, this is generic Android.
There's no modifications otherthan hardware support stuff added.
And then this is what gets handedout to ODMs OEMs and everyone else
who uses that chip and they addtheir own sauce on top of that.
At one point it becomes very recognizable.
But you do know that under thehood was that original basic
bare bones BSP that came from thechip vendor that's being used.
(14:05):
So if you have that, youhave something to work with.
And I guess
David (14:09):
the context here of why this makes things difficult is that you then have
to learn everything and more that the OEMand ODM did as part of their bring up.
So you're not only repeatingthe process, you're doing it
with less information than they
Nikhil (14:22):
had.
Absolutely.
And, and a lot of times, like, I mean, I,I say Qualcomm is a good example, right?
But even Qualcomm, and rightfullyso, they don't actually have
fully open source binaries.
There are still closed sourceproprietary binaries where they have
their own secret sauce and their, Imean, big sense iss P gpu Exactly.
Things like that.
Things like that.
So especially when you're dealing with anend of life device or an end of life chip.
(14:46):
You don't have updates to those binaries.
Even with supportive situationslike that, you still have to do
some hacking to get stuff working.
Yeah.
And
David (14:55):
I think that getting us too far off track, that's part of why we're
seeing Google try to modularize Androidmore so that we can start updating these
things standalone and stop basicallylike making everything beholden to BSP
Jon (15:06):
version.
Absolutely.
And that was their goal for projecttravel or travel support when they
introduced that into uh, I think Android.
Was the first introduction for itthat made it separated, kind of where
we had a vendor partition that wouldcontain all the device information, ODM
stuff, and that would be static thatwould only be updated by the vendor
(15:29):
and then everything else around it,the system partition, product system
extended, et cetera, would all beupdated by the OEM or Google the source.
For Android.
So that made it kind of simpler forsome forward parts, but a little bit
less simpler cuz we still have topull that vendor image, parse it,
figure out what all is added to it.
(15:51):
And if we plan on doing anythingwith it, it has to be mostly private.
Unfortunately,
Nikhil (15:56):
and things are moving in a good direction too, right?
So now Google has thegeneric kernel image concept.
Where that would come in handy is inthe future where most devices do come
with gki support, which Google enforces.
You wouldn't even need the kernel sources,which technically you're supposed to
get, but a lot of times you don't get,as we talked about earlier, right?
You could still run a generic kernel.
(16:18):
Top of your device that'srunning the OEM specific kernel
image and still get it working.
And so now you have the ability toswitch out everything on your device
to generic versions and just swap outthe entire operating system basically
without any help from the OEM or oem.
I think that's the ideal goal thatGoogle is aspiring towards and
seems like we're getting there.
(16:39):
Right,
David (16:40):
and you know, just.
Again, is a kind like kindof very high level check.
This also assumes that you can get intothe device, which is a big assumption
we have to make for any of this, right?
So you may either need cooperationand existing exploit or something
else, which I don't, I'm not sure what
Nikhil (16:57):
other routes there would be.
Well, there's a few ways, right?
So at least in a professional space whenit comes to Android devices and building
a O S P, , you have two situations.
One is where you have the bsp,the whole package that the ODM
uses themselves or gives out toother people who need it, right?
In that situation, the images you buildand the output really is something
(17:22):
that you can term as a whole package,meaning like the entire operating
system files, including stuff outsideof Android that is needed by Android,
but it's technically not Android.
It's quite a full system image, right?
Boot loader and other.
In the device.
You have all of that and.
Basically use whatever flashingprocess that the chip vendor has
(17:43):
determined for their board to flash it.
And at that point, you'reacting like the odm, right?
You're doing what theywould do in the factory.
And so that's the more clean process.
But in the community and as aconsumer, you don't really get
access to that with any odm really.
Right?
And so at that point, you're workingwith a S P and you're working with
whatever sources they've dropped, andyou're just building what you need.
(18:05):
To swap out and bring it up to a newversion of Android or update whatever
you need to update and leave those otherauxiliary partitions just untouched.
So there's like two levels to this.
Mishaal (18:16):
We briefly touched upon project trouble before jumping into generic
kernel image and then you know this topic.
But I kind of wanted to step backand focus more on Project Trouble
because it's such an important.
Part of what makes device bringup pretre versus post trouble.
So different.
So I wanted to ask you, can youexplain how a device bring up differs
before product troubles introducedversus after it was introduced?
(18:38):
Like what is the fundamental differences?
Pretre,
Jon (18:42):
everything was pretty much built into system is root.
So all of your device information, all ofthe vendor applications were all included
in one system, partition on the device.
Post travel, all of that is separatedinto a system, a product, a system
extended, and a vendor bar issue whereyou have access to all the separate
(19:06):
parts, but they're mounted within thesystem a little bit differently and
separated on the device themselves.
So when we update on a projecttravel device, we're only
updating the system parti.
And possibly product, uh, vendor systemextended as well from the actual OEM or
the vendor that leads it to where theycan do security updates much faster
(19:30):
without having to rebuild everythingthey've added previously on top of
a brand new security release onp.
They can easily add those patchesinto the system and it's all kept.
Mishaal (19:43):
When Google introduced trouble, when they decided to strip the
vendor specific hardware abstractionlayers from the system image into its
own dedicated vendor partition, theyalso defined an interface, a standard
interface between the vendor partitionand the Android operating system.
So to allow the OS to communicatewith those hardware abstraction
(20:03):
layers in a standardized way.
And they call that the vendor I.
I wanted to ask you, like, can you talka bit about this vendor interface and how
its introduction affected device bring up?
Well,
Jon (20:15):
there were two ways of vendor interface was added.
It was, uh, added either a through thevendor image that the hardware vendor
would create and that would containall the added files that they needed.
For I D L interfacing, it would alsocontain some of the information that they
need for external apps that they're addingonto the system and services as well.
(20:40):
They also introducedthe V N D K interface.
And that is pretty much where allthe hardware interaction goes.
So anything that goes through V NDK has to be certified pretty much
to work with their device interface.
I don't have too much experienceworking with the dk, Maybe does.
Nikhil (21:05):
The BDK is less talked about aspect of travel.
You know, when we say travel, talkabout project travel, typically you
talk about how things are separatedbetween system and vendor, but.
Part of that separation was Googledefined that interface, like you said.
Right?
And VDK is an important part of it.
Vdk, by the way, stands forVendor Native Development Kit.
(21:26):
And what it basically does is it givesthese ODMs and chip side vendors is
standardized way to define their Halsand their hardware support libraries.
And what it also does is it enforcesbackwards compatibility, right?
So if you have a vendor that wasfor Android nine, Then by Google's
(21:47):
compatibility definition, youwould have to make sure that that
vendor is or compatible up tothree versions I think, of Android.
So that means you coulduse that same vendor.
And a system image from Android 12say, and it would work perfectly fine,
and they have to work well together.
And this is the sort of rulesthat Google is setting that
makes device bring up easier.
(22:08):
So if you have a device thatdoesn't get updated that often,
it's running on Android 10 andyou wanna run Android 13 on it.
You don't really have to messwith the vendor layer at all.
Maybe perhaps slight modificationsdepending on device specific works,
but for the most part you can leavethat vendor part untouched and work
on the upper layer, so to speak.
(22:29):
And, and that really is the beautyof trouble is there's definitions
now and there's standardization of.
This communication betweensystem and vendor, and I
David (22:38):
guess is a good way to describe that, really, how the hardware
extract itself to software andsays, I can do this, I can do that.
And here's how you call that, right?
Essentially like the camera can zoom.
Therefore you should have a hook inhere for your vendor, specific camera
implementation if you have a special one.
For Zoom, not to get into Googleor Android camera AP guys, cuz
(22:59):
that's a whole other thing.
But in general, conceptually, isthat kind of the way that works?
Nikhil (23:04):
Right.
And it doesn't really block manufacturersor Google from implementing new things.
It just makes sure that whenyou do implement new things,
you have to make sure that.
It doesn't break compatibilitywith the older versions of the vdk.
Right.
So that's why you see, I think ifyou're in the Android community now,
new versions of Android, especially likeCustom ROMs are dropping much sooner.
(23:25):
It's because they don't reallyhave to spend, That's really
where most of the time is, right?
Because the vendor layer and theHals and the hardware support
stuff, that's where all the IP is.
That's where you have the.
Source code access, and so that'swhere you have to do the most.
Hacks and hacks are usually trial anderror and usually take a lot of time, and
people who do this for free in their freetime tend to not have much free time.
(23:48):
That's really what helped things getbetter over the years, especially
with trouble because you don'treally have to do much anymore.
For example, if you're waiting onone plus to drop their Android 13
builds, you can still get Android13 as a custom realm on your.
While you wait for that Android 13,and when it does drop, you can have a
newer build, which uses that updatedvendor that's more suited for Android
Jon (24:12):
13.
A lot of times that updatedvendor will have updated firmware
for the device, et cetera.
So Nel Image, that type of stuff.
Mishaal (24:21):
Technically is a generic system.
In the one plus example with theirrecent devices, things are getting
more complicated with the Googlerequirements freeze program.
So a lot of modern flagship devicesare now actually shipping older
vendor software implementations thatwere built for an older release.
So for example, a devicethat launched with Android 12
that upgrades the Android 13.
Might still be using the samevendor software implementation that
(24:43):
was developed for Android 12 whenit upgrades, and that's a quirk
of Google requirements freeze.
We talked about that before, but I don'twanna get into the exact nitty gritty
of that aspect right now, but I kindof wanted to touch upon a bit more on
the genericization of the interfacebetween the hardware extraction layers
and the OS, and why that's so important.
So let's say pre-project trouble, ifyou were to try to just flash an A
(25:06):
S P system image onto the device andhave it booted up, if the OEM of that
device had some custom hardware andthey wrote their hardware abstraction
layer in a nonstandard way that a S Pwouldn't be able to interface with it.
Then when you boot up a P, that pieceof hardware that that abstraction
layer was written for just mightnot work or it might just crash.
So like if you had a camera andyou were expecting it to boot up,
(25:28):
when you open up the a SB cameraapp, you might just get nothing.
You might see nothing at all.
And that's obviously a hugeproblem because you're dealing with
fundamental hardware components ofa device that you would expect to.
Out of the box.
But with project trouble standardizinga lot of these base components and
Google enforcing this through thesevendor test suites, their, their vendor
(25:49):
interfaces, you know, all these thingslike the diversions that OEMs to follow.
Like it ensures that if you take amodern project trouble compatible
device and you boot AOS P onto it,Chances are that at least almost all
the basic hardware components will work.
So like for example, we have a Lenovotab K 10, that we all frequently use
for work purposes and that deviceship put Android 11 outta the box.
(26:11):
I've been able to boot Android 1212L Android, 13 generic system images
onto it and all the hardware works.
A s p works perfectly for just fine.
There's no issues with it at all,and that's thanks to project.
Nikhil (26:22):
And we're where to the point where you can't even tell the difference
between a GSI and a dedicated image.
Right?
So unless your device has specific,like when it comes to one plus devices,
they have that alert slider, or theyhave the in display fingerprint sensor.
Those are specific hardware choices thataren't really a part of a s P or gsi.
But for a generic device like aLenovo K 10, it's a decent tablet
(26:45):
that has everything you need.
If you go to Gs.
It works.
Every feature you want to work works.
And so at this point, I mean the sortof direction we're heading into is
we will just have a Windows or Linuxlike situation where you have a build
of Android that drops that you canjust install on any device and it
more or less behaves the same nextstep for, you know, hardware specific
Jon (27:06):
works.
Let's talk about the difference therethough with Windows, Linux, and Android.
So on Windows, it ships withmonolithic kernel a lot like Linux
does well, So that contains all thedevice information, the drivers, the
firmware for the drivers, et cetera.
And hopefully 90% of that is detectedupon a net, or when it first boots
(27:31):
up on windows, it detects a list ofwhat's available, downloads what it
can, after connection is available, andthen, uh, pairs it up based on that.
But with Android, it's a.
Find a bit different because itrelies on what's on the device.
None of that extra stuff is everincluded on the Linux NEL that's sent
(27:52):
with Android, at least for most devices.
If you're dealing with an AndroidX 86 device, we did switch to a
monolithic kernel and it kind of pairsup hardware available using mod probe
and if probe exists, mod probe this.
If found great at it to thelist, if not, great, ignore it
and then continue booting when.
(28:14):
So it'll go through every potentialconnection in your device until it
finds it all just like Linux doesand pairs it up with that device.
Hardware on Android, especiallyusing Project trl, it's a bit
different because you have thatproduct definition already set up.
So it's gonna be shipped with that productimage, vendor image, and that's gonna
(28:35):
contain all the information they needto load the kernel, load the drivers,
load any extra blobs, and drive devicehardware for those drivers, and then
can make the connections and then boot.
Probably
David (28:47):
philosophically there's some really good reasons that Android ended up this
way, aside from inheriting from Linux.
But number one, Android was alwaysdesigned to be ultra lightweight.
And so by being able to define the firmerimage at a very high level, you can strip
out everything that is unnecessary andhave a super lightweight device, which
was very important in early Android,especially . You did not want bloat.
(29:11):
Everybody hated bloat.
But I think the other side is probably.
For Google, you know, this is a betterway for them to pull the levers.
They want to controlling the ecosystem.
They can do it very high up as opposed togenericizing, which they have been doing.
Um, that's a harderproblem to solve, right?
It's a lot easier to make a ruleand say, You vendor can do this.
(29:33):
You vendor can't do that.
Rather than giving them a toolthat says, Okay, we can make this
more flexible, adaptable, which.
Really what trouble I think was probably
Mishaal (29:41):
about, This actually brings us back to the generic kernel
image topic and the reason whyGoogle went for that in the first.
Since, as you mentioned, Androiddevices, they generally chip with
their device drivers on the device.
Not everything is included in theLinux terminal, and if you just boot
Linux onto an Android device, youcan't expect everything to work.
So the problem is that a lotof open source developers have
(30:02):
been pushing for Android devicemakers and vendors for years to.
Upstream your drivers open source yourdrivers and submit them to Linux and
let us take care of maintaining that.
But that has never happened.
That pushback.
It just just didn't happen.
No, it's, There was no ip.
Jon (30:17):
All these device manufacturers want that IP to stay private.
So I
Mishaal (30:21):
completely understand why.
Yeah, exactly.
But like then that becomes a problembecause there's so much out of tree
coat or there used to be so muchout of tree code running on, you
know, in the Linux NEL that ships.
Your average Android device and, andso if device drivers aren't being
upstreamed, then it kind of limits thelongevity of those devices, the ability
to upgrade those kernels and implementsecurity patches, because there's so
(30:44):
much of a difference between the kernelthat ships on a device and the upstream
Linux kernel that a lot of work hasto be done to manage those merges.
So what Google decided to do is,Okay, we'll let you keep your kernel
drivers closed source, but instead yougotta ship it as a kernel module that
interacts with the generic kernel imagein a standardized way, specifically
Jon (31:04):
built kernel module that interfaces a specific way using gki.
Mishaal (31:09):
Right?
So on a modern gki deviceright now, so like the boot.
It has the, the generic NEL image.
So as Naqui mentioned, you can go,literally go and download Google
signed generic kernel image, andthat's going to be the same kernel
that ships on other gki devices.
But what actually enables the kernel totalk to the device specific hardware?
Is those kernel modules thatare specific to that device, and
(31:32):
that's contained on a separatepartition from the boot partition.
Jon (31:35):
And that's automatically detected and loaded as soon as Android
zygote loads on the boot to process.
Nikhil (31:41):
It's, It's interesting to see how we got here though, right?
So you were kind of touching onthat, David, but if you look at why
Android is the way it is, I thinkfundamentally, The goal with Android
was to appeal to developers, right?
So the architecture itselfis designed to make sure that
applications are easy to write, right?
So for example, instead ofhaving to interface with hardware
(32:03):
directly like Linux, you havehardware abstraction layer.
So your application doesn't have tochange depending on how the hardware
features are implemented per device.
But that means.
Device vendors have to put moretime and effort into defining
these hardware abstractionlayers specific to their devices.
And when they put in that effort, theydon't want to put all that information
(32:25):
out there for the world to see.
So now you have people movinga lot of hardware specific code
from the kernel where it wouldtypically reside in a regular device
to vendor binaries like blogs.
Ah,
David (32:36):
so it's kinda a shell game thing that
Nikhil (32:38):
they're encouraging.
Yeah, it really is.
Yeah.
And so now you.
The idea of close source as much aspossible with like minimal changes
to the kernel, which makes Androidas an OS very highly device specific.
. And so you don't have the conceptof a generic, Well, you didn't have
a concept of a generic OS imagelike you do with Windows of Linux.
Jon (32:59):
Yeah.
There is no one size fits all anymore.
And I guess you, but there kinda is.
That kind is, and there kind isn't.
You can't take a brand new devicethat has no information provided
for it from the manufacturer and getit working like you can with Linux.
Linux.
You'd be able to take what we workedon this last HP desktop for example.
(33:20):
And the Linux kernel that has all thathardware, the monolithic kernel, and
it'll automatically mod probe and lookfor any existing hardware, and that
can probably get 90% of it workingon a new device with a new chip set.
That is not doable with Android anymorethough, and that's one of the downsides.
I
David (33:39):
mean, there's also of the business and partner model of
Android is very different fromlike Microsoft and Windows, where
Microsoft says, You wanna work on us?
You gotta talk to Windows daddyand submit your driver for
signing . Like there's no other way.
Like that's how you make thingswork on Windows, Whereas Android.
There are so many vendors at somany different layers, and also you
have vertically integrated vendorswho are doing things that multiple
(34:01):
vendors would otherwise be doing.
Whereas with Windows, Microsoft hadthe clout in the authority to say,
You will build a Windows computerlike this, Android and Google.
They've had less authority, I think,until recently, the last five.
But how
Jon (34:14):
far innovation's gone by letting the OEMs have that ability?
Totally.
With Android devices, I mean,we've gone three, four times
what Windows was able to.
Right.
Yeah.
And I
David (34:25):
think, you know, one of the, where I was going with this, and I will stop
after this, is that smartphones come froma much more dedicated device mindset.
They come from telephones.
Telephones had a verylimited number of things.
They just started doing more andmore and more of them, and so
the smarter telephones got, youneed a firmware on them, right?
They got smart enough that you hadto have an operating system under
(34:46):
telephone, and so they come from that.
This is a device with a list offeatures that does these things.
You rely on it for that.
Windows comes from, this is acomputer, you use it to make stuff.
And I think that actually informedprobably the way manufacturers
look at firmware for these devices.
They probably looked at, especiallyearly on as fixed in place.
I make a product, it has firmware,it stays stable, which is in our
(35:08):
world something many people expect.
But in the consumer world, especially whenit comes to general computing devices, no.
They expect they get better.
And Android has started todo that with smartphone.
Obviously, but that's a pretty big sea
Mishaal (35:20):
change.
I.
Yeah.
David, you'd mentioned that you know,it's true that Google has less of an
iron grip over a s p and Android thanMicrosoft does over Windows, but they
still do have a very size of control.
Yeah.
Especially when it comes to requirementsto getting nude Android versions up and
running, or certified on an older device.
So for example, like.
(35:41):
It's true that Google is genericizedAndroid to the point where you can take
an A O S P GSI for like Android 14, forexample, and you could boot it on a device
that originally launched with Android 12.
But can you do that with Android 16?
Maybe not, because Google tends toonly support backward compatibility
for three letter upgrades.
(36:02):
So like if you have a device that launcheswith Android 12, Google will make sure.
That Android 15 won't include anynew requirements that break backward
compatibility with Android devicesall the way back to Android 12.
But they don't guaranteeany background fat ability.
After that.
It's generally just three letterupgrades and you can kind of see
this extended across everywhere.
Like if you look up, how long does Googleprovide security Backport to AO s p?
(36:26):
It's generally only threes versions.
So like right now, if you look onthe security bulletin, they support
Backporting patches to Android10, because that's three letter
upgrades before, but soon that'llstop once they launch Android 14.
And this also goes into, it'snot just becoming difficult
to port a new Android version.
Sometimes they'll just.
Entire features that you might depend on.
So for example, I think Lineage,when they released their Android 12
(36:49):
builds, they had to drop a ton ofolder devices because those devices
didn't have a Linux kernel version newenough to support EPF F, which is like
the Berkeley packet filter feature.
It is like a kernel feature that'susually for like network filtering
or firewalls and stuff like that.
And so they had to drop a ton ofdevices because of that, because
Android straight up acquired it forAndroid 12, and those older devices
(37:10):
with older carnivals just couldn't.
Nikhil (37:12):
It's interesting to look at Android as sort of a pre-market, right?
So you have Google try to enforce itlike the government with their GMs
requirements and the compatibilitydefinition document and whatnot.
And then you have the whole AOS feespace where there's no laws, It's
people doing whatever they want.
ODMs, they don't have tofollow any rules, right?
(37:33):
And so how do you enforce.
The concept of a standard in sucha space where essentially the
goal is to cut as many cornersas possible to a working device.
And we talked about how Android is morerigid in terms of as an operating system.
That really helped here, if youthink about it, because what's
the goal of an Android device?
Right?
It should run all the Androidapps that are available, right?
(37:55):
Right.
So if apps don't.
Then your device doesn't work.
And so because apps rely on this nicelayer of communication, the how and
the interface between the system andthe hardware, you need to write that
well enough where apps don't break.
And so it's an interesting problemfor Google to solve because.
They can only do so much in the aospspace and for an open source operating
(38:18):
system with tons of people using and tonsof vendors and tons of manufacturers.
It seems like it's worked outalmost as well as it could.
And
David (38:27):
I think to Google's credit, honestly, the fact that they
seem to have the confidence thatleaving AOS Aosp out there is not
something that's going to hurt them.
It's something.
The changes they make in proper Androidare going to percolate basically into the
way people use AO s p, because guess what?
The ecosystem is gonnamove in that direction.
If you're Qualcomm or media techor even rock chip, you are going to
(38:50):
encourage then ODMs and OEMs to dothings because you have to do them.
So there is definitely risingtides kind of effect sort of
thing, you know, follow the leader.
But yeah, I mean, aos P obviously hasled to so many different innovations.
Car OEMs build their infotainment onAosp and have for years because it was
a much more reliable and lightweightway to do it than building Linux.
(39:12):
Like it was just like, Oh, this is designwork with Bluetooth and wifi and all
the other like kind of communicationstandards you would expect of a modern os.
And that is because the Android wasdesigned to be totally wireless connect
with all standards of humanly can.
Just generally be pretty flexiblethat way because Google wanted
OEMs to experiment, so I do.
You
Nikhil (39:32):
mentioned the app ecosystem, right?
Right.
They don't have to build anything.
There's already a huge, probably thelargest app ecosystem out there that
you don't even have to go throughGoogle certification to access.
Right.
You could just side load an apk.
Right?
And you have YouTube running onyour OSP head unit and your card.
I think the modularity and the opennessof a USP is why it's super popular.
Mishaal (39:53):
Well, just a small caveat on that, a lot of apps will require
your device to have GMs or GoogleMobile services in order to access
the APIs provided through it.
Our previous episode on GooglePlay Services covers that in
detail if you wanna learn more.
And yeah, I also wanted to kind ofcaveat something I mentioned before.
You know, I mentioned that.
Google makes it difficult tosupport new Android versions.
They drop support for olderfeatures of, they update their
(40:16):
requirements after three letter upversions, but it's not impossible.
Like David mentioned, theAosp is a wild, wild west.
You can do whatever you want.
If you have the technical knowledge andexpertise, you can bring up Android.
For as many versions as you want.
If you have extensive Linux kernelexperience, you could keep porting newer
versions of Linux onto an older device.
(40:36):
Like it's not impossible.
It's just really, really,really freaking hard.
You can
Jon (40:41):
also do both and back Port Linux modularity from a monolithic kernel
into a newer Android type build.
And this is a lot of what we seehappening with our work here at
Sper and elsewhere in the communityfor all like Android X 86.
Project Way, droid, those types
Nikhil (41:00):
of projects.
Yeah.
And with the community, you seethese exceptional cases, right?
Where you have devices that werelaunched with ice cream sandwich running
Android 13, or dine running Android 13.
It's, it's insane.
And how these developers get Android,like a new version of Android
came out 10 years after the lastupdate was issued to that device.
(41:21):
Working as if it was built by the oem.
That's the beauty of open source.
Hey man, my
Jon (41:26):
HTC Hero was updated for like 10 years, thanks to the
Mishaal (41:29):
community.
Oh, you remember the HTC HD two, thatlegendary phones that would never with the
Nikhil (41:36):
slide.
David (41:39):
That was one of those
Jon (41:40):
at first came with Windows Mobile, I think.
Nikhil (41:43):
Yep.
The device I've worked with a lot was theSony Xperia Z one, and that beast was up.
I think the last update Sony gaveit was 5.1, so Snap Track in 8 0 1.
800.
800.
David (41:56):
Wow.
800,
Nikhil (41:57):
okay.
The first cryo chip.
Yeah.
Last I checked it was Android 12.
Right.
So from five to 12, this people havebeen, uh, I think I was involved all the
way until Android 10 and after that otherpeople have taken over and just, uh,
kept updating and Android 11, Android 12.
In fact, the whole Sony platformis one of a great example of the
(42:18):
communities coming together andjust giving life to a legacy device.
David (42:22):
And you know this, this kind of gets into the question of like,
why can't we extend Android forever?
And I think one of the reasons isAndroid, unlike Linux and Windows
does not run on devices that are likeeasily air gapped from the internet.
Android doesn't reallywork without the internet.
Let's be real.
An Android device without internet is kindof a brick, unless you wanna use it as a
word processor or calculator or something.
(42:44):
That means though, you have to keepup with the internet, and that's like
Michelle, you bring up that part of LinuxNEL for firewall, like packet handling.
That is an example of thingsthat you just like, well,
somebody would have to build it.
And nobody's gonna do it.
And that's just kind of partof the ever evolving nature of
connected and cloud technology.
If your device can't talk to thesystems that it needs to to do
(43:07):
those things anymore, you get stuck.
And that's what happenedwith the HTC Hero, I believe.
Basically, it couldn't talk to theGoogle servers anymore to like get
like wifi credentials basically,so it wouldn't get certifi.
You'd have to connect it viaethernet cable over USB otg.
And even then, thatdidn't really work, right?
Like we had Ryan, one of oureditors tried to do this at AP and
(43:27):
it was just a nightmare to try toeven get this phone to load Gmail.
But yeah, it's differentthan a Windows computer.
You can buy a 20 year old Windowscomputer, open up Internet Explorer,
and probably getting parts ofthe internet to mostly load or
download Chrome, whatever, you know.
Well, I think we are at about time.
This was a wonderful chat about, you know,I mean, I learned a lot in the last hour,
(43:50):
quite frankly, about the practicalitiesof building A O S P, which is really
at Asper, what we do on a daily basis.
Take things that run what you wouldn'tprobably Google wouldn't call Android,
but that for everybody else's purposes,Yeah, they run Android and there are
some things that you do have to change.
Now John, um, works on our X 86foundation product and obviously that
(44:11):
is so unusual in the world of Android.
Jon (44:14):
We are doing, it does use a monolithic nel, so that's the project that
melds everything together and some mashup.
And I had no idea.
Best of Linux, best ofAndroid, and we make
David (44:25):
it work.
Peanut butter and jelly.
So if you are interested in learningabout building from AO s p, and
like what it takes to make a deviceor to make a device work the way
you want, come talk to usper.io.
Mishaal (44:39):
All right, thanks David.
And thank you John and Nala for joiningus and uh, we do this with all our guests.
We wanted to ask you now, like,where can people follow you if they
wanna follow your work online orsee where you tweet or do whatever?
Jon (44:50):
I'm on Twitter for the most part, so Twitter at E L E c T r i k J e s U
Nikhil (44:57):
S and not on Twitter.
So and I'm.
I'm kind of thinking,where Can't you follow me?
I don't know.
GitHub.
Yeah, I
Mishaal (45:07):
was gonna say
. Jon: There's that too.
I'm also in GitHub.
Same place.
Well, maybe it's good that you'renot on Twitter because, uh, it's a
place where, uh, nuance goes to die.
All right.
Well, this show is a placewhere Nuance doesn't die.
We love talking to experts every week.
You know, every other week, kind of aregular schedule right now about Android,
(45:30):
A O S P, the whole ecosystem and platform.
So thank you again, both of you,for joining us to talk about,
you know, device bring up.
And I hope those of you listeninglearn a thing or two about this.
And if this sounds interestingto you, as David mentioned,
come check us out@esper.io.
Now show some interest.
Jon (45:45):
We wanna do more of these.
Popular Podcasts
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
On Purpose with Jay Shetty
I’m Jay Shetty host of On Purpose the worlds #1 Mental Health podcast and I’m so grateful you found us. I started this podcast 5 years ago to invite you into conversations and workshops that are designed to help make you happier, healthier and more healed. I believe that when you (yes you) feel seen, heard and understood you’re able to deal with relationship struggles, work challenges and life’s ups and downs with more ease and grace. I interview experts, celebrities, thought leaders and athletes so that we can grow our mindset, build better habits and uncover a side of them we’ve never seen before. New episodes every Monday and Friday. Your support means the world to me and I don’t take it for granted — click the follow button and leave a review to help us spread the love with On Purpose. I can’t wait for you to listen to your first or 500th episode!