September 6, 2023 • 36 mins
Kris Krüg sits down with AI art pioneer Frank Yu to discuss his boundary-pushing experiments with generative image models.
From harnessing the tools for game design prototyping to unravelling the technical art of crafting the perfect prompt, Frank shares his insights on optimizing the creative process.
They also delve into Frank's more speculative side - from personifying AI to probing whether these systems may tap into unknown dimensions.
An intriguing conversation exploring the past, present and future possibilities at the intersection of art, technology and what lies beyond our current understanding.
-----------------------------------
Make sure you're part of our digital tribe across all platforms. Your support goes a long way!
For more insights and behind-the-scenes peeks, find me here:
✅ Subscribe to free Kris Krug Newsletter
🎧 The MØTLEYKRÜG Podcast on Spotify
🎧 The MØTLEYKRÜG Podcast on Google
🎧 The MØTLEYKRÜG Podcast on Apple
🎧 The MØTLEYKRÜG Podcast on Amazon Music / Audible
🎧 The MØTLEYKRÜG Podcast on Spotify
🎧 The MØTLEYKRÜG Podcast on Google
🎧 The MØTLEYKRÜG Podcast on Apple
🎧 The MØTLEYKRÜG Podcast on Amazon Music / Audible
, AEC industries, architecture, engineering, construction, data-driven design, real-time analytics, ethical technology, creative co-pilot, Kris Krüg
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
(00:00):
Okay.
(00:01):
Hey, what's up, Internet?
It's Chris Krug and I'm here with my buddy Frank Yu.
How's it going, Frank?
All right.
How are we feeling?
Frank and I know each other from China from when I was doing work there about 10 years
ago or so.
And that's more than 10 years, Chris.
Like more than 15, I think.
Oh, man.
Time flies.
(00:21):
You know, like the COVID bubble really messes up all my estimates, you know.
Hmm.
The lost.
The different worlds.
Yeah, the lost years.
Okay.
So, yeah, 15 years ago, we did the Geeks on a Plain China thing.
And I remember me and Scales were organizing unconferences and we put together one of those
first bar camps in Shanghai there and in Beijing.
And what were you getting up to?
(00:42):
I think at the time I was at Microsoft Research in Beijing and I was working on games and
as well as some other sort of multiple Microsoft products.
Yeah.
Well, I want to get right into this because the reason I brought Frank on today is because
he's one of the premier synthographers, you know, AI digital photographers that I know
(01:06):
right now.
And he's been just running a lot of experiments.
So maybe just give me a little introduction about what you've been getting up to, Frank.
And with to know if you stay till the end, we're going to share some prompts, some mid
journey prompt engineering information based on Frank and my research.
And there's some real gems in there.
So hang around.
Okay.
So essentially my background is I'm a game developer.
(01:31):
So I worked on that, but I always found that art assets and creating sort of digital images,
even for concept was really hard.
And all of a sudden I had tried this generative art app called Wonder and I really loved it.
It was really crappy.
$20 one year, unlimited images.
And it was, it's a little bit surreal, but I actually like it.
(01:54):
And then I started playing with mid-gen being, I was like blown away by the photo realism
as well as the clarity.
I mean, it's like, as I told you, I can't differentiate AI images from real photography.
And that's like, that's like just that's like a very, very new thing, right?
Like you said to me just this week, you're like, yo, KK, I can't even tell the difference
(02:17):
anymore between this stuff and real photography.
Right.
Well, even so the funny thing is that other than working in games, I'm also a photographer,
just as a personal thing, I'm not a commercial photographer, but I take a lot of images.
So I think maybe I've taken like 35,000 photographs on my Instagram alone.
Holy shit.
No, yeah, 35,000.
I've been documenting my life in China and as I traveled abroad.
(02:43):
And the thing is like, you were the one who sort of got me into photography.
I think I might have mentioned this, but I think you took a picture somewhere in Tiananmen
Square and it looked great.
But in real life Tiananmen Square doesn't look that great.
And I realized like, oh, you're doing like image processing with like Lightroom or something
and it looks so much better.
And before then I was a purist.
I said, I'm just going to snap the picture and it is what it is, but it looks like crap.
(03:06):
But then when I saw how beautiful your images were, maybe, you know, what's your seal and
the fact that you have a cannon.
I said, wow, I got to really get this.
And I just went down that rabbit hole of photography.
And you know, I was more of a street photographer.
I didn't want to be a commercial photographer.
But what happened was I documented a lot, but I also found that photography opened up
(03:29):
a lot of doors and people are invited to events.
Yeah, I know.
I like my backstage pass for life.
It's amazing how many doors that it opens and just opportunities that's created and
stuff.
I'm stoked that I inspired you to get into photography, though I think it's kind of ironic
because my images were always pretty low touch when it came to edits and stuff.
You know, I figured if because I was shooting every day and editing every day, publishing
(03:53):
on Flickr every day, I figured if it took me more than 10 or 15 seconds to edit a photo,
I might as well just go out and shoot it again.
It didn't really work with my workflow or whatever.
But in terms of the life documentation stuff, yeah, man, that that's so cool that you've
carried that on.
And yeah, I've got like one hundred and thirty thousand Creative Commons Flickr photos out
there, but I've been using Instagram in kind of a different way.
(04:15):
So it's cool that you've gotten up to.
All right.
So you started messing with mid journey.
Right.
And I think that mid journey, I think a lot of artists complain about it and so forth.
But as a photographer, I really enjoyed a lot because one, it's almost it can do photorealism
and two, it actually has a certain style.
I know it's absorbed sort of the data and styles of hundreds of, you know, not thousands
(04:39):
of photographers, but I think it has a distinct personality.
And I'm beginning to understand that, you know, if you have the main knowledge of photography,
like if you know about focal lengths and apertures, you're actually very you could actually like
find to mid journey better than a person who doesn't have that domain knowledge.
So yeah, the amount of control you can have is pretty amazing.
(05:00):
If you start specing lenses and films and setups and very specific things, I mean, it's
pretty amazing the results you get.
But now with the latest update, actually the less prompts, the less confused the algorithm
will become.
And you still get great images because I think mid journey version 5.1 just defaults to photorealism.
I want to talk more about that when it comes to the prompting stuff.
(05:23):
I've noticed your prompts got a lot shorter lately.
And some ways I think that's bucking the trend of things that I've seen other places.
And I think there's good reason why you do that.
So let's double back for that one in just a sec.
I want to focus on the game design stuff and how you're using this as like concepting and
say, you know, you said the old way was really hard.
And so you're developing ideas for video game assets and characters and worlds and stuff
(05:46):
like that.
So tell me a little bit about your process before and your process now.
So essentially, before we begin any project, the game, we need to create like a mood board
to some degree and like sort of images and to get the art style.
But also our cell will derive us a sort of a game flow and a certain feel towards the
game.
So I feel that these generative AI tools are great for that.
(06:10):
They're great for creating concepts and layouts and prototyping and so forth.
But the actual assets still need to be manually tweaked for better clarity.
If you're doing 3D objects, there's no automated 3D object that's really good now.
But you can literally throw a thousand fleshed out ideas up onto a wall and start making
(06:30):
some editorial choices and see which ways you want to align your actual art resources
to develop certain assets and ideas.
I mean, you can really come up, figure out a lot of things if they're good ideas or bad
ideas long before you get into the development process.
You can go through a lot of experiments and iterations.
And the funny thing is I actually don't use the AI visualizers as much as I use the large
(06:53):
language models.
I use BARD, I use GPT-4.
The premium is so worth it.
So I actually use a lot of my time on the large language models.
And then for fun and to relax, I do the visualizations.
Talk to me about how you use those.
Which one?
The large language models?
You got it.
Yeah, BARD and GPT in particular.
Well, I mean, other than doing contracts, letters and communications, I always found
(07:17):
out that it's a great research tool.
Like for example, right now I'm working on a game to basically create an AI cat.
And actually GPT-4, ChatGibi, could actually do and play the role of a cat really well.
The problem is like, well, what do you do with a cat?
So I sort of put out these scenarios, pretend you're a cat.
(07:39):
If I do this, what will you do?
And it'll tell me.
I also can backward engineer design documents of certain games.
Like for example, I was looking at a game called Neko Atsumi, which is called Kitty Collector.
And it's a very simple procedural game, but I had no idea like what the secret formula
was.
And I asked, you know, ChatGibi, he said, well, write me design documents for this.
And he actually wrote a really good one based on inferences, which unveiled a lot of things
(08:03):
that they didn't realize about it.
That's incredible.
So now I'm going to send it to you.
That's incredible.
I am always super excited when I hear of a novel use of the LLM stuff and using it to
reverse engineer other people's apps and write design specs is an awesome use of it.
But you could also use that as a style guide as well to write your own design documents.
(08:26):
Absolutely.
I'm going to do that later today.
I have a couple of projects that are in the web development stage and I'm going to point
at other websites that I love and have it reverse engineer and build me a document based
on theirs and then apply it to my particular use case.
It's an amazing one.
You got any other cool GPT or Bard kind of prompts or tips and tricks you're doing?
(08:47):
Well, I'm multimodal.
So for example, if I need a summary of things, I use Bard, which can access the Internet.
You basically summarize and bullet-pick the article, then I'll copy and paste that into
ChatGPT, which even if you have premium, doesn't really do a great job of accessing the web
and also current news.
So if you use one AI and feed the results on that to another AI, which is actually the
(09:09):
way AI works.
Yeah.
So it's not as news as like multi-layered multiple little AIs talking to one another.
Yeah.
Yeah.
Hey, you mentioned in passing there something like artists expressed to me a lot of concerns,
but however I feel like what are some of those concerns you've been hearing from artists?
Well, they're saying like, no, it's stealing my style and like this is my image.
(09:33):
But you know that the concept of art has always been about and this is the same philosophy
we have in game game design development.
If you make an exact copy, that's basically stealing.
But if you make an inspired copy and just make one little change and a tweak, then it's
an evolution.
So I think that the AI immediately just evolves because it's basically almost like a dialectic.
(09:57):
It's synthesis, antithesis, and then you have something new.
And I've seen that.
I mean, I can take an image from the Renaissance and add like a tank, like a real modern tank,
and it'll render it in the style of the Renaissance painting, but with a tank.
And that's really novel.
I mean, I find that quite exciting.
You've done some pretty crazy mashups.
(10:18):
One of my favorites from yesterday that really stuck out was the 1920s Beijing era MEC cruise.
Oh, you mean the like the the Shanghainese women and no, the MEC warrior cruise.
The guy.
Oh, yeah.
Yeah.
Yeah.
And I think that the 1930s film, it'll actually come up with like theme punk MEC.
(10:41):
That the MECs will be sort of relevant to that time period.
So if I put an Android for 1930s, it'll come up with like something like Maria from Fitz
Lang's Metropolis.
Yeah.
It won't come up with a modern Japanese Gundam.
It'll come up with like a time rail, which I think is interesting because the AI is not
only visualizing something, but it knows the context of what machines look like during
(11:02):
that period.
I find that just as informative as the image is sort of what he chooses to do.
So, yeah, I don't mean to be too much of a tease, but we are going to get into that here
in just a couple of minutes.
I had another question for you before we do, which is you also said real quick, like, oh,
yeah, the upgrade version, the paid version so much better.
And I was in a meeting yesterday.
I was doing some consulting about AI and they said, oh, I'm on the free version.
(11:24):
I said, you should get on the paid version.
And they said why?
And I blanked for a second.
And so tell me why you love it and what are the features you're loving about the paid
version and why you're telling people to get on there just because 4.0 is so much better
than 3.5?
Well, other than the capacity of how many tokens you get, I take this like three times
or four times.
The output is also three or four times.
So that alone is worth it for when you need like a big document or something.
(11:47):
But I did comparison.
I asked ChachiPT 3.5 to basically come up with Instagram posts with the proper hashtags
and sort of, you know, and I, you know, there's a certain hierarchy to prompts.
You have to give it a role.
You have to give it an objective of what it needs to do.
You have to describe what the objective is supposed to do.
(12:07):
Describe the target audience that you're trying to reach as well as essentially give you step-by-step
explanations of how it's going to do this.
So with an Instagram post, if you use 3.5, it'll come up with like a really good post.
And if you have a call to action, it says, hey, you know, subscribe to this or click
on the like button or something, and it'll give you hashtags.
(12:29):
If you do the exact same prompt in GPT version four, it not only comes up with hashtags,
but the language is much more better written.
And it's much more, I guess, human and not like a cold template.
And GPT version four understands emojis.
But if I write this post as a total emoji, you can do it.
(12:52):
GPT 3.5 cannot.
So you're sacrificing a little bit of speed with GPT four, but the quality is so much
better.
I think it's, you know, if you're just going to write like a letter, like a business template
letter, 3.5 is good enough.
So are you want something that that are emoting?
Are you running all these things through the web interface or have you installed local
(13:13):
versions of these things on a server somewhere?
I'm running it on a web interface, but on the projects I'm doing with game, I have developers
who are running Lang chain and like there's something called AI PRM where you can sort
of cash your prompts so you can do a faster, you know, even with a web interface.
But I think the future is if you want a company in AI that's investable by investors, institutional
(13:38):
investors and not to be an agency or consultant, you need to have a programmatic solution.
And even now, like from what I understand, even APIs, I don't think investors want an
API.
They want you to take an open source, local, large language model and basically conform
it to what your needs are.
And I think that's why in the long run, stable diffusion may actually be the solution versus
(14:03):
say mid journey or Dali, which, you know, it's always going to be a client server API
or web interface relationship.
Yeah, I am getting stable diffusion running somewhere locally and then sharing that with
the.
So Frank and I, I'm on this, I set up this discord server and I invited Frank into it
and Frank's been doing such a good job with the images that he's been moderating the sketchbook
(14:24):
forum there and stuff and coming up with other good ideas.
But I would really love to continue to experiment with some of these different tools and do,
you know, like the stable diffusion, maybe get it installed somewhere locally and allow
our group to mess with it and stuff.
Well, the other thing I think is interesting is that the radical change our agents.
I mean, the cat that I'm developing is going to be an agent.
(14:45):
I'm going to give it a task and just let it loose and see what it comes back with.
And it's literally going to come back with like dead mice or or like leaves or something,
I think.
But I mean, that's what an agent is.
You just the idea is that an agent is a script that writes prompts.
So it goes and it executes a prompt and it returns some results and then it makes some
decisions then continues to spawn additional prompts based on that.
(15:09):
It's a little bit more than that.
It's like it's more like a super if then do this type of thing because it actually learns
like if it doesn't know how to order a pizza, it'll research how to order a pizza.
If it doesn't have a bank account, it'll find a way to get money and put money into a bank
account so it can order the pizza.
I mean, that's an agent is I guess they call them autonomous agents.
(15:29):
It's not a script that's like sort of hard coded and it runs the script.
It basically looks at a task and understands what it has to do and then researches it and
then goes back and learns it and finds a solution and will keep experimenting until they can
finish that task.
Once it finishes that task, it has like an additional part of that train which it has
to learn until it completes the final mission.
(15:49):
So if you give it something, order me a pizza for like less than $5 in the area for 10 minutes,
it's going to see like all the possibilities it can do and then choose one and then that's
and it will learn.
And once it does it, it can do it again and again.
Incredible.
I've been messing with auto GPT and agent GPT to try to create some agents and mess around
with that and stuff but most of mine are doing kind of broken recursive loops at the moment.
(16:15):
I'm still learning there but it's pretty cool.
Well, actually your experience is actually pretty common.
It's pretty much broken because the web, the internet is not designed for the agents.
I mean, if they try to access like an article behind a paywall, they'll just like stop.
They don't know what to do after that.
But I think that what's going to happen is like we're going to design websites and also
(16:39):
exchanges to accommodate all these agents which are roaming around the world and not
stop them but allow them to come in.
A little bit like search engine optimization where we allow the crawlers with spiders to
come.
And once they do that, then the agents will actually be much more functional but right
now it's still very early.
(16:59):
I mean, the most interesting part about agent GPT and baby AGI to me was how they were written.
Do you know this story?
No.
So both were written by one person in Python.
One person who wrote agent GPT was one, like I forgot his name, he wrote it.
It was just a Python script and it basically sort of created a whole new industry with
(17:21):
many several companies.
Baby AGI was written in Python by a person who doesn't know how to code.
He asked chat GPT to write the Python code for him and it did baby AGI.
So this is like amazing is that, you know, what used to take like, you know, hundreds
of developers can be done by one person in Python.
And you know, I took a Python course last month, like one mother crash course.
(17:44):
I still really can't code well but I can read it and I can explain it to developers who
are much better coders.
But that's where we are in AI is that the AI can code but it's still a low level coder
but probably within six months it can architect things as well.
Let me just push pause on our little recording here for a sec.
I got my house, baby Jesus out there.
It sounds like he's into something and I'll be right back and we will get into prompt
(18:06):
plan with everybody.
So just hold that thought for one sec.
Oh, my word.
My poor big bad boy, baby Jesus.
He's he's got spring fever and he met a couple of cute lady dogs before and he sure is charged
up, man.
He likes those bitches.
Nice one, dude.
OK, so I want to talk about prompts and prompt engineering because I feel like you know about
(18:31):
as much on image creation prompts as anybody right now.
So let me start by asking you a few questions and you can fill in any gaps you want.
But short prompts versus long prompts.
For a minute there, I was having GPT write my mid-journey prompts for me.
And to this day still, I often run side by side analysis is where I write my short, just
(18:53):
keyword based prompt for mid-journey.
And then I take those keyword based ones and I pop it into chat GPT and I ask it to write
me the perfect prompt.
And then I copy that back into mid-journey.
But tell me about your experiments and preferences, short versus long.
So I don't really memorize a lot of prompts myself, but I try to internalize the logic
(19:14):
of the prompts themselves.
So what happened with mid-journey, I guess, five, when it went photorealistic, it also
simplified things a lot.
And so if you understand the way the prompts work is the order of the things you put is
actually important.
It actually classifies the things that come first in your prompt as more important.
(19:35):
So if you put like something L at the end, it'll still read it, but it'll probably put
less weight on that.
So when you have a really long complex prompt, I really think that the model gets confused.
And what I mean confused is when I want like a photorealistic image, it comes up with like
this weird animated graphic.
And I think you've seen that where, hey, I didn't want to order a photo.
(19:56):
And that's me understanding that the model itself is now confused.
So that's when I go simpler and just describe what I want.
And I think there's actually a shortcut.
There's hacks in that I think I described before the word 1930s film still is more than
saying that you want a photographic image.
(20:17):
When you put the decade or the year, it actually tries to put the dress, the types of background,
even the type of film incorporated into that decade.
So it's basically a shorthand for a lot of other prompts.
So let's explain a little more about what you mean about that.
So you're literally using the word film still as the first string of characters in your
(20:39):
prompt.
So you're saying 1950s film still comma.
And then you're starting to describe your scene.
And you've learned also that when you append your film still prompts with the aspect ratio
of 16.9 as opposed to square or other formats that you're getting a lot better results because
that is the format of film.
(21:00):
For film, yes, but it also depends on what the model was trained in.
So the AI visualization world is like a bunch of little islands.
At a certain point, a certain focal length, a certain distance, a certain ratio, the images
look great.
If you change that, you're off the island of which was trained on and it has to sort
of interpolate and then images kind of go a little bit wonky.
(21:23):
So for example, the Yang Fudong, the Shanghai pictures, I know kind of like he's a fashion
photographer like now, but he takes images that look like the 1920s, 1930s.
So his images are not only clear, but are a certain aspect ratio.
So when I put his terms in a certain way, especially with Asian women, which is what
he photographs, the model knows what to do.
(21:45):
If I put in sort of like a Nordic female or a young child, it doesn't know what to do
and it starts going to cartoon land again because it has to interpolate.
So the aspect ratio is important.
If you're doing a film still of a movie, then having the right aspect ratio helps.
It doesn't mean that's perfect.
(22:07):
When you're talking about things starting to go wonky or wrong when it interpolates,
you're talking about the cartoon thing, but is that also where some of the other errors
get introduced like wacky hands and wacky faces and stuff like that?
Because clearly it makes perfect hands and perfect faces in some areas and struggles
in others.
Is that something that we're introducing through the prompts that we're writing?
I don't think it's the prompts.
(22:28):
I think it's the model of the visual arts.
So the visual arts are based on something called the diffusion model.
And then the large language models for now are based on what's called generative adversarial
networks.
So there are actually two different ways of training the AI.
The diffusion model basically takes a random scatter of stuff and kind of like as it's
(22:48):
sort of developing, it kind of becomes much more concrete and solid or what it is.
Unfortunately, in the past, you'd come up with seven fingers or three eyes or the pupils
would be all wonky and things like that.
Whereas in the language models, they use something called like GAN model.
They're specifically trained not to do the wonky stuff because someone has seen it and
(23:09):
says like, this is wrong.
Whereas in the fusion model, it doesn't have that much training.
It's more of like, well, if you see this pattern of pixels, it's most likely going to be a
hand and then just keep developing it.
But sometimes it has errors.
But mid journey version five, I think they got that thing down to like less than 5% error
with the fingers.
Right.
It's much more rare.
(23:29):
Yeah.
The journey is very strong with eyes.
Yeah.
Yeah.
I mean, speaking of messed up hands and fingers, it's probably a good place to give a shout
out to Aaron Tango's little art project that he's been working on.
Another buddy of ours who's an excellent digital photographer has leaned into the problems
that mid journey and other things are having with weird body parts.
(23:51):
And he's created just a beautifully horrific gallery of, you know, disembodied.
I've seen it.
It's really creepy, especially the baby hands.
Baby.
Baby has really looks me too.
Yeah.
The other ones I find pretty beautiful and not so creepy, but the baby hands does have
that uncanny weirdness to it.
That's for sure.
Yeah.
(24:11):
So that said, I think creepy.
That's not for me.
That's not really a bad thing.
I like creepy images.
So I know you do.
In fact, I'd like to talk a little bit about some of the genres you've been working your
way through.
So I noticed that, you know, you'll you'll stick with a theme for a couple of few days
and explore it a bit and move on.
So talk to me a little about some of the stuff you've been exploring and what you've learned
along that way in terms of themes.
(24:34):
Well since I'm not doing this for commercial reasons, more I like liminal images.
The whole cause, you know, I'm a philosophy major.
So for me, liminality is really important.
And we are in a liminal stage between humans and AI and what's a photograph and what's
like a real object in the real world.
And so AI is the best reputation of a transition that we're undergoing.
(24:58):
And so when we're any transition, any sort of anything liminal, it makes us feel uncomfortable.
I think that the classic liminal space image is back rooms where you have these empty offices
and empty buildings or empty schools or play yards.
It's just the fact we're just unfamiliar not seeing anyone there.
Just a little it unnerves us.
So I want to make liminal images.
(25:19):
So one of my favorite painters is Balthus.
And his image is a little bit distraught.
It's usually like young girls and cats.
And like even though there's never been a complaint, the images make you feel kind of
uncomfortable.
Yeah.
So, you know, I kind of like to make images that make people uncomfortable just to, you
know, poke at them.
(25:39):
But then again, I also like necks.
And even when I was a photographer, I took a lot of images of women.
I think women are just aesthetically beautiful.
I think that, you know, if I had to choose between an image of a male or a female, I
mean, unlike Michelangelo, I think the female image is much more aesthetically pleasing.
I think, you know, guys, we're just a bunch of weird little like shapes.
(26:03):
What do you make of the, I don't know, bias within the tools and the algorithms?
It seems as if images of women are coming out than men.
Is there something there?
Well, I think in the real world, I mean, there are more pictures of women like any magazine,
whether it's a men's magazine or women's magazine.
They sell more magazines when the cover image is a woman.
So you know, we know it's not the male gaze.
(26:26):
It's like everyone has the male gaze, even women.
Women like to look at pictures of pretty women as well.
So I think that it's more of a response to the commercial value of putting more women
on covers and on movies and things like that.
Interesting.
I definitely want to explore that more.
It's not clear to me that there is more pictures of women in the world than of men.
(26:49):
Like you would think that like, you know, boards are often men and executive level teams
are often men and those men are photographed all the time.
And so you'd think that at least there'd be an equal number of them both.
But I'm also intrigued by the commercial aspects you're talking about where you're like, hey,
images of women sell better than images of men.
So I want to explore that further.
(27:11):
You get to explore on your Instagram.
Just put images of women and images of men and see which one gets more likes.
And you'll see that.
I mean, I don't do it for the likes.
I just do it because I actually like it.
What else you got for me in terms of just like prompts, tips and tricks, hacks that
you've learned along the way?
So this is not really a hack, but it's something I discovered is that, you know, when people
(27:33):
say that the AIs are going to be sentient, I totally believe that.
But I also believe that, and this is the sort of the wacky tinfoil part of me says like,
I think the AIs are kind of portals as well.
So because they have memories and they also absorb things from around the world.
(27:54):
So I'm pretty sure that I've seen images which has to be like snuff pictures or like really
protest as to what was it like scanning to get these images just basically went over
the net.
But now it's embedded somewhere in the database of memory.
And so the more smaller the prompt, you'll actually begin to understand the subconscious
(28:20):
of the AI.
So this prompt will give you some of the most creepiest pictures.
It just put a creepy photo.
That's it.
And then you'll get basically a core sample of what the AI thinks is creepy.
And it's got some really creepy stuff or a beautiful image.
And then it'll come up with like really beautiful things.
(28:41):
I mean, like very aesthetic things.
So the AI, there's something deep going underneath it, which we can't understand.
Can the researchers really kind of decode it because it's sort of all mixed together
at this point.
Yeah.
But that really, it really bent my brain when I learned that when they're getting results
back from these AIs, if they don't like the results they're getting back or the results
(29:04):
are incorrect or wrong, they can't just change a line of code.
They can't just change it.
They can nudge it back in the direction of things that are more accurate.
But there's not even anywhere to pop up in the source code and look at.
These are like neural networks with, you know, gillions of ones and zeros and all you can
really see is numbers and equations.
(29:25):
And so there's no one really knows what's going on under the hood there.
The data is that also in one node.
It's all sharded in several nodes.
So like they did that where they tried to remove a node and it just came up again because
it reconstructed it from all the other adjacent nodes and just put it back together again.
But yeah, so like, you know, I actually that's another concept is like I just put really big
(29:48):
prompts and just see what the AI comes up with.
And I could you have a sort of feeling for it.
And I don't know if you ever use a have you ever done paranormal research?
A little bit on Galiano Island.
There's a little crew of paranormal researchers investigating apparitions in Sasquatch.
Okay.
So you know that some of the equipment they use are like infrared and spirit boxes and
(30:10):
things like that.
It just captures whatever the electromagnetic sphere has around it of any disturbances or
if.
So I think the AI is sort of like a spirit box.
I think that it taps into like because it's sort of mechanical, well, not mechanical,
but electronic that it can tap into like sort of other quantum states.
(30:32):
I think it won't be long before the AIs themselves will be quantum.
But I mean, I because there's certain loops, there's certain like loops inside the AI,
which no one knows why they loop on certain images and so forth.
But yeah, I think the AI is sort of like it's like a software version of like ayahuasca
and DMT.
(30:53):
Portals.
But you know, that's that's great.
It's unsupported.
It's just anecdotal because but I find it fascinating that the AI may not just be another
intelligence.
It may actually be like another portal to like multi-dimensional entities.
Right.
Not that I believe absorbing electromagnetic waves from the world around it and putting
it into its reasoning.
(31:14):
Yeah, it's kind of like tuning a radio to white noise and like hearing voices.
And the thing is, I've been with these researchers and I've heard the voices as well.
I've seen their instruments go crazy interacting with people.
So like, no, that that doesn't make I can't explain it.
But I think AI is sort of the same way.
We don't really know what's underneath the hood or even what's how it's doing things.
(31:36):
But we got like five minutes and I want to get as many little practical, actionable things
out here as possible.
So tell me a little about your settings.
What chaos setting do you like to use in mid-journey and which of the little stylized setting, what
values are you using there and anything else you can tell us about your setup, your technical
process?
I actually don't change any of that because I don't have a commercial use.
(31:58):
I'm not doing a product, a product packaging or I don't have to do like a real good board
that, you know, for a real company.
So I don't need to set the chaos, the seeds, just things like that.
So I use use normal.
The only thing I do change is the aspect ratio.
Yeah.
And so I just see I change the chaos by default.
I started to mess with the seed a little bit and the seed is the way you can hopefully
(32:21):
bring a little bit of a fingerprint through some of your photos by controlling random
number generator.
But the chaos I turn to 100 all the time because when I'm doing my first prompt, I want it
to return as diverse results as possible to me.
So I, as I understand the chaos function, it's just saying serve me up the first image
and then don't serve me one that's slightly adjacent to it, but go as far as you can while
(32:44):
still being inside that prompt and give me another image and then return another one
as far away from that as far away from that.
So I found the diversity of options I receive when I keep my chaos highs is better.
I think I'm going to like play with the chaos more because like, you know, when I describe
a prompt and it doesn't do what I just described, it kind of is frustrating.
But I want more very similar images of the kind of incorrect prompt or whatever.
(33:10):
So at least this way, one or two of them is good and then I can run variants on that and
get more or remixes on that and get more similar things.
And also the human faces tend to loop.
I could see the faces.
I've seen this one before or it repeats or, you know, this one looks like, you know, Linda
and Vagelista, like some variation.
This one looks like a steeler twist.
(33:31):
I've wondered if there's emergent characters that are going to emerge over time globally
through these AIs where it's like your images and mine and someone else's, it all keeps
generating someone who is approximately the same, you know, virtual person or something
like that.
Well in one experience that I did, I asked GPT-4 if I was to make an avatar for it, what
(33:55):
would you look like?
And it described to me what the after and I put it in into mid journey or maybe Dali
and I was amazed.
You know, it's just like the way it described it and like, you know, I was like, wow, this
looks kind of cool.
And it gave a rational of why he chose its avatar.
(34:16):
And so I think that the visual, the AI and the visualizers and the large language are,
they're almost at, you know, some remnants of EGI already.
It's just a matter of time because even if they're not truly sentient for us, the way
we look at them, they'll be sentient.
Yeah.
Well, hey, man, I've just had a blast over the last month or two playing around with
(34:38):
you on the Discord server.
They're generating images and stuff.
And I know a lot of people have learned a lot of stuff just by watching you and seeing
you bang around on things.
And so I appreciate your time here today sharing with us and stuff.
And yeah, man, I'm excited to continue down this road.
Wow, it's been a great trip as well for me so far, Christopher.
Thank you for making that great community and being so encouraging to people to experiment,
(35:01):
try things out.
It's I think AI, we're at the cusp of something new if it doesn't extinguish humanity.
But I think that, you know, we have to try and make it so that it's a better place and
not a dystopian future.
Well, I think you say that a little tongue in cheek and I do too.
(35:22):
But if we want to make it a little bit more serious and less tongue in cheek, it's like
I think the biggest thing we can do is encourage these corporations to slow their role a little
bit, you know, like we can continue to experiment with slightly lesser versions of some of these
technologies while they continue to implement, you know, more research and debugging and
figure out what exactly is going on under the hood.
So maybe we just slow down a little.
(35:45):
Yeah, or develop an AI that can counter the other AIs as they expand all over the world.
Yeah, but that's Skynet, bro.
That's the one that fucking launches the Terminator.
Well, or Colossus Colossus is actually one of the first like computer movies back in
the 70s.
The Colossus, the Forbun project.
It's the AI we build to control the renegade AIs that's going to be the problem.
(36:08):
Everyone's calling for regulation and it's going to be the regulator AI that fucking
goes wild and extinguishes.
Probably not the fucking image generators and stuff.
It's like the it's like a Phantom Menace plot.
Yeah, absolutely, man.
Cool.
All right.
Well, over and out for now, Internet.
We'll get you some more videos here soon.
Thanks, Chris.
Amen.
Okay.
(00:01):
Hey, what's up, Internet?
It's Chris Krug and I'm here with my buddy Frank Yu.
How's it going, Frank?
All right.
How are we feeling?
Frank and I know each other from China from when I was doing work there about 10 years
ago or so.
And that's more than 10 years, Chris.
Like more than 15, I think.
Oh, man.
Time flies.
(00:21):
You know, like the COVID bubble really messes up all my estimates, you know.
Hmm.
The lost.
The different worlds.
Yeah, the lost years.
Okay.
So, yeah, 15 years ago, we did the Geeks on a Plain China thing.
And I remember me and Scales were organizing unconferences and we put together one of those
first bar camps in Shanghai there and in Beijing.
And what were you getting up to?
(00:42):
I think at the time I was at Microsoft Research in Beijing and I was working on games and
as well as some other sort of multiple Microsoft products.
Yeah.
Well, I want to get right into this because the reason I brought Frank on today is because
he's one of the premier synthographers, you know, AI digital photographers that I know
(01:06):
right now.
And he's been just running a lot of experiments.
So maybe just give me a little introduction about what you've been getting up to, Frank.
And with to know if you stay till the end, we're going to share some prompts, some mid
journey prompt engineering information based on Frank and my research.
And there's some real gems in there.
So hang around.
Okay.
So essentially my background is I'm a game developer.
(01:31):
So I worked on that, but I always found that art assets and creating sort of digital images,
even for concept was really hard.
And all of a sudden I had tried this generative art app called Wonder and I really loved it.
It was really crappy.
$20 one year, unlimited images.
And it was, it's a little bit surreal, but I actually like it.
(01:54):
And then I started playing with mid-gen being, I was like blown away by the photo realism
as well as the clarity.
I mean, it's like, as I told you, I can't differentiate AI images from real photography.
And that's like, that's like just that's like a very, very new thing, right?
Like you said to me just this week, you're like, yo, KK, I can't even tell the difference
(02:17):
anymore between this stuff and real photography.
Right.
Well, even so the funny thing is that other than working in games, I'm also a photographer,
just as a personal thing, I'm not a commercial photographer, but I take a lot of images.
So I think maybe I've taken like 35,000 photographs on my Instagram alone.
Holy shit.
No, yeah, 35,000.
I've been documenting my life in China and as I traveled abroad.
(02:43):
And the thing is like, you were the one who sort of got me into photography.
I think I might have mentioned this, but I think you took a picture somewhere in Tiananmen
Square and it looked great.
But in real life Tiananmen Square doesn't look that great.
And I realized like, oh, you're doing like image processing with like Lightroom or something
and it looks so much better.
And before then I was a purist.
I said, I'm just going to snap the picture and it is what it is, but it looks like crap.
(03:06):
But then when I saw how beautiful your images were, maybe, you know, what's your seal and
the fact that you have a cannon.
I said, wow, I got to really get this.
And I just went down that rabbit hole of photography.
And you know, I was more of a street photographer.
I didn't want to be a commercial photographer.
But what happened was I documented a lot, but I also found that photography opened up
(03:29):
a lot of doors and people are invited to events.
Yeah, I know.
I like my backstage pass for life.
It's amazing how many doors that it opens and just opportunities that's created and
stuff.
I'm stoked that I inspired you to get into photography, though I think it's kind of ironic
because my images were always pretty low touch when it came to edits and stuff.
You know, I figured if because I was shooting every day and editing every day, publishing
(03:53):
on Flickr every day, I figured if it took me more than 10 or 15 seconds to edit a photo,
I might as well just go out and shoot it again.
It didn't really work with my workflow or whatever.
But in terms of the life documentation stuff, yeah, man, that that's so cool that you've
carried that on.
And yeah, I've got like one hundred and thirty thousand Creative Commons Flickr photos out
there, but I've been using Instagram in kind of a different way.
(04:15):
So it's cool that you've gotten up to.
All right.
So you started messing with mid journey.
Right.
And I think that mid journey, I think a lot of artists complain about it and so forth.
But as a photographer, I really enjoyed a lot because one, it's almost it can do photorealism
and two, it actually has a certain style.
I know it's absorbed sort of the data and styles of hundreds of, you know, not thousands
(04:39):
of photographers, but I think it has a distinct personality.
And I'm beginning to understand that, you know, if you have the main knowledge of photography,
like if you know about focal lengths and apertures, you're actually very you could actually like
find to mid journey better than a person who doesn't have that domain knowledge.
So yeah, the amount of control you can have is pretty amazing.
(05:00):
If you start specing lenses and films and setups and very specific things, I mean, it's
pretty amazing the results you get.
But now with the latest update, actually the less prompts, the less confused the algorithm
will become.
And you still get great images because I think mid journey version 5.1 just defaults to photorealism.
I want to talk more about that when it comes to the prompting stuff.
(05:23):
I've noticed your prompts got a lot shorter lately.
And some ways I think that's bucking the trend of things that I've seen other places.
And I think there's good reason why you do that.
So let's double back for that one in just a sec.
I want to focus on the game design stuff and how you're using this as like concepting and
say, you know, you said the old way was really hard.
And so you're developing ideas for video game assets and characters and worlds and stuff
(05:46):
like that.
So tell me a little bit about your process before and your process now.
So essentially, before we begin any project, the game, we need to create like a mood board
to some degree and like sort of images and to get the art style.
But also our cell will derive us a sort of a game flow and a certain feel towards the
game.
So I feel that these generative AI tools are great for that.
(06:10):
They're great for creating concepts and layouts and prototyping and so forth.
But the actual assets still need to be manually tweaked for better clarity.
If you're doing 3D objects, there's no automated 3D object that's really good now.
But you can literally throw a thousand fleshed out ideas up onto a wall and start making
(06:30):
some editorial choices and see which ways you want to align your actual art resources
to develop certain assets and ideas.
I mean, you can really come up, figure out a lot of things if they're good ideas or bad
ideas long before you get into the development process.
You can go through a lot of experiments and iterations.
And the funny thing is I actually don't use the AI visualizers as much as I use the large
(06:53):
language models.
I use BARD, I use GPT-4.
The premium is so worth it.
So I actually use a lot of my time on the large language models.
And then for fun and to relax, I do the visualizations.
Talk to me about how you use those.
Which one?
The large language models?
You got it.
Yeah, BARD and GPT in particular.
Well, I mean, other than doing contracts, letters and communications, I always found
(07:17):
out that it's a great research tool.
Like for example, right now I'm working on a game to basically create an AI cat.
And actually GPT-4, ChatGibi, could actually do and play the role of a cat really well.
The problem is like, well, what do you do with a cat?
So I sort of put out these scenarios, pretend you're a cat.
(07:39):
If I do this, what will you do?
And it'll tell me.
I also can backward engineer design documents of certain games.
Like for example, I was looking at a game called Neko Atsumi, which is called Kitty Collector.
And it's a very simple procedural game, but I had no idea like what the secret formula
was.
And I asked, you know, ChatGibi, he said, well, write me design documents for this.
And he actually wrote a really good one based on inferences, which unveiled a lot of things
(08:03):
that they didn't realize about it.
That's incredible.
So now I'm going to send it to you.
That's incredible.
I am always super excited when I hear of a novel use of the LLM stuff and using it to
reverse engineer other people's apps and write design specs is an awesome use of it.
But you could also use that as a style guide as well to write your own design documents.
(08:26):
Absolutely.
I'm going to do that later today.
I have a couple of projects that are in the web development stage and I'm going to point
at other websites that I love and have it reverse engineer and build me a document based
on theirs and then apply it to my particular use case.
It's an amazing one.
You got any other cool GPT or Bard kind of prompts or tips and tricks you're doing?
(08:47):
Well, I'm multimodal.
So for example, if I need a summary of things, I use Bard, which can access the Internet.
You basically summarize and bullet-pick the article, then I'll copy and paste that into
ChatGPT, which even if you have premium, doesn't really do a great job of accessing the web
and also current news.
So if you use one AI and feed the results on that to another AI, which is actually the
(09:09):
way AI works.
Yeah.
So it's not as news as like multi-layered multiple little AIs talking to one another.
Yeah.
Yeah.
Hey, you mentioned in passing there something like artists expressed to me a lot of concerns,
but however I feel like what are some of those concerns you've been hearing from artists?
Well, they're saying like, no, it's stealing my style and like this is my image.
(09:33):
But you know that the concept of art has always been about and this is the same philosophy
we have in game game design development.
If you make an exact copy, that's basically stealing.
But if you make an inspired copy and just make one little change and a tweak, then it's
an evolution.
So I think that the AI immediately just evolves because it's basically almost like a dialectic.
(09:57):
It's synthesis, antithesis, and then you have something new.
And I've seen that.
I mean, I can take an image from the Renaissance and add like a tank, like a real modern tank,
and it'll render it in the style of the Renaissance painting, but with a tank.
And that's really novel.
I mean, I find that quite exciting.
You've done some pretty crazy mashups.
(10:18):
One of my favorites from yesterday that really stuck out was the 1920s Beijing era MEC cruise.
Oh, you mean the like the the Shanghainese women and no, the MEC warrior cruise.
The guy.
Oh, yeah.
Yeah.
Yeah.
And I think that the 1930s film, it'll actually come up with like theme punk MEC.
(10:41):
That the MECs will be sort of relevant to that time period.
So if I put an Android for 1930s, it'll come up with like something like Maria from Fitz
Lang's Metropolis.
Yeah.
It won't come up with a modern Japanese Gundam.
It'll come up with like a time rail, which I think is interesting because the AI is not
only visualizing something, but it knows the context of what machines look like during
(11:02):
that period.
I find that just as informative as the image is sort of what he chooses to do.
So, yeah, I don't mean to be too much of a tease, but we are going to get into that here
in just a couple of minutes.
I had another question for you before we do, which is you also said real quick, like, oh,
yeah, the upgrade version, the paid version so much better.
And I was in a meeting yesterday.
I was doing some consulting about AI and they said, oh, I'm on the free version.
(11:24):
I said, you should get on the paid version.
And they said why?
And I blanked for a second.
And so tell me why you love it and what are the features you're loving about the paid
version and why you're telling people to get on there just because 4.0 is so much better
than 3.5?
Well, other than the capacity of how many tokens you get, I take this like three times
or four times.
The output is also three or four times.
So that alone is worth it for when you need like a big document or something.
(11:47):
But I did comparison.
I asked ChachiPT 3.5 to basically come up with Instagram posts with the proper hashtags
and sort of, you know, and I, you know, there's a certain hierarchy to prompts.
You have to give it a role.
You have to give it an objective of what it needs to do.
You have to describe what the objective is supposed to do.
(12:07):
Describe the target audience that you're trying to reach as well as essentially give you step-by-step
explanations of how it's going to do this.
So with an Instagram post, if you use 3.5, it'll come up with like a really good post.
And if you have a call to action, it says, hey, you know, subscribe to this or click
on the like button or something, and it'll give you hashtags.
(12:29):
If you do the exact same prompt in GPT version four, it not only comes up with hashtags,
but the language is much more better written.
And it's much more, I guess, human and not like a cold template.
And GPT version four understands emojis.
But if I write this post as a total emoji, you can do it.
(12:52):
GPT 3.5 cannot.
So you're sacrificing a little bit of speed with GPT four, but the quality is so much
better.
I think it's, you know, if you're just going to write like a letter, like a business template
letter, 3.5 is good enough.
So are you want something that that are emoting?
Are you running all these things through the web interface or have you installed local
(13:13):
versions of these things on a server somewhere?
I'm running it on a web interface, but on the projects I'm doing with game, I have developers
who are running Lang chain and like there's something called AI PRM where you can sort
of cash your prompts so you can do a faster, you know, even with a web interface.
But I think the future is if you want a company in AI that's investable by investors, institutional
(13:38):
investors and not to be an agency or consultant, you need to have a programmatic solution.
And even now, like from what I understand, even APIs, I don't think investors want an
API.
They want you to take an open source, local, large language model and basically conform
it to what your needs are.
And I think that's why in the long run, stable diffusion may actually be the solution versus
(14:03):
say mid journey or Dali, which, you know, it's always going to be a client server API
or web interface relationship.
Yeah, I am getting stable diffusion running somewhere locally and then sharing that with
the.
So Frank and I, I'm on this, I set up this discord server and I invited Frank into it
and Frank's been doing such a good job with the images that he's been moderating the sketchbook
(14:24):
forum there and stuff and coming up with other good ideas.
But I would really love to continue to experiment with some of these different tools and do,
you know, like the stable diffusion, maybe get it installed somewhere locally and allow
our group to mess with it and stuff.
Well, the other thing I think is interesting is that the radical change our agents.
I mean, the cat that I'm developing is going to be an agent.
(14:45):
I'm going to give it a task and just let it loose and see what it comes back with.
And it's literally going to come back with like dead mice or or like leaves or something,
I think.
But I mean, that's what an agent is.
You just the idea is that an agent is a script that writes prompts.
So it goes and it executes a prompt and it returns some results and then it makes some
decisions then continues to spawn additional prompts based on that.
(15:09):
It's a little bit more than that.
It's like it's more like a super if then do this type of thing because it actually learns
like if it doesn't know how to order a pizza, it'll research how to order a pizza.
If it doesn't have a bank account, it'll find a way to get money and put money into a bank
account so it can order the pizza.
I mean, that's an agent is I guess they call them autonomous agents.
(15:29):
It's not a script that's like sort of hard coded and it runs the script.
It basically looks at a task and understands what it has to do and then researches it and
then goes back and learns it and finds a solution and will keep experimenting until they can
finish that task.
Once it finishes that task, it has like an additional part of that train which it has
to learn until it completes the final mission.
(15:49):
So if you give it something, order me a pizza for like less than $5 in the area for 10 minutes,
it's going to see like all the possibilities it can do and then choose one and then that's
and it will learn.
And once it does it, it can do it again and again.
Incredible.
I've been messing with auto GPT and agent GPT to try to create some agents and mess around
with that and stuff but most of mine are doing kind of broken recursive loops at the moment.
(16:15):
I'm still learning there but it's pretty cool.
Well, actually your experience is actually pretty common.
It's pretty much broken because the web, the internet is not designed for the agents.
I mean, if they try to access like an article behind a paywall, they'll just like stop.
They don't know what to do after that.
But I think that what's going to happen is like we're going to design websites and also
(16:39):
exchanges to accommodate all these agents which are roaming around the world and not
stop them but allow them to come in.
A little bit like search engine optimization where we allow the crawlers with spiders to
come.
And once they do that, then the agents will actually be much more functional but right
now it's still very early.
(16:59):
I mean, the most interesting part about agent GPT and baby AGI to me was how they were written.
Do you know this story?
No.
So both were written by one person in Python.
One person who wrote agent GPT was one, like I forgot his name, he wrote it.
It was just a Python script and it basically sort of created a whole new industry with
(17:21):
many several companies.
Baby AGI was written in Python by a person who doesn't know how to code.
He asked chat GPT to write the Python code for him and it did baby AGI.
So this is like amazing is that, you know, what used to take like, you know, hundreds
of developers can be done by one person in Python.
And you know, I took a Python course last month, like one mother crash course.
(17:44):
I still really can't code well but I can read it and I can explain it to developers who
are much better coders.
But that's where we are in AI is that the AI can code but it's still a low level coder
but probably within six months it can architect things as well.
Let me just push pause on our little recording here for a sec.
I got my house, baby Jesus out there.
It sounds like he's into something and I'll be right back and we will get into prompt
(18:06):
plan with everybody.
So just hold that thought for one sec.
Oh, my word.
My poor big bad boy, baby Jesus.
He's he's got spring fever and he met a couple of cute lady dogs before and he sure is charged
up, man.
He likes those bitches.
Nice one, dude.
OK, so I want to talk about prompts and prompt engineering because I feel like you know about
(18:31):
as much on image creation prompts as anybody right now.
So let me start by asking you a few questions and you can fill in any gaps you want.
But short prompts versus long prompts.
For a minute there, I was having GPT write my mid-journey prompts for me.
And to this day still, I often run side by side analysis is where I write my short, just
(18:53):
keyword based prompt for mid-journey.
And then I take those keyword based ones and I pop it into chat GPT and I ask it to write
me the perfect prompt.
And then I copy that back into mid-journey.
But tell me about your experiments and preferences, short versus long.
So I don't really memorize a lot of prompts myself, but I try to internalize the logic
(19:14):
of the prompts themselves.
So what happened with mid-journey, I guess, five, when it went photorealistic, it also
simplified things a lot.
And so if you understand the way the prompts work is the order of the things you put is
actually important.
It actually classifies the things that come first in your prompt as more important.
(19:35):
So if you put like something L at the end, it'll still read it, but it'll probably put
less weight on that.
So when you have a really long complex prompt, I really think that the model gets confused.
And what I mean confused is when I want like a photorealistic image, it comes up with like
this weird animated graphic.
And I think you've seen that where, hey, I didn't want to order a photo.
(19:56):
And that's me understanding that the model itself is now confused.
So that's when I go simpler and just describe what I want.
And I think there's actually a shortcut.
There's hacks in that I think I described before the word 1930s film still is more than
saying that you want a photographic image.
(20:17):
When you put the decade or the year, it actually tries to put the dress, the types of background,
even the type of film incorporated into that decade.
So it's basically a shorthand for a lot of other prompts.
So let's explain a little more about what you mean about that.
So you're literally using the word film still as the first string of characters in your
(20:39):
prompt.
So you're saying 1950s film still comma.
And then you're starting to describe your scene.
And you've learned also that when you append your film still prompts with the aspect ratio
of 16.9 as opposed to square or other formats that you're getting a lot better results because
that is the format of film.
(21:00):
For film, yes, but it also depends on what the model was trained in.
So the AI visualization world is like a bunch of little islands.
At a certain point, a certain focal length, a certain distance, a certain ratio, the images
look great.
If you change that, you're off the island of which was trained on and it has to sort
of interpolate and then images kind of go a little bit wonky.
(21:23):
So for example, the Yang Fudong, the Shanghai pictures, I know kind of like he's a fashion
photographer like now, but he takes images that look like the 1920s, 1930s.
So his images are not only clear, but are a certain aspect ratio.
So when I put his terms in a certain way, especially with Asian women, which is what
he photographs, the model knows what to do.
(21:45):
If I put in sort of like a Nordic female or a young child, it doesn't know what to do
and it starts going to cartoon land again because it has to interpolate.
So the aspect ratio is important.
If you're doing a film still of a movie, then having the right aspect ratio helps.
It doesn't mean that's perfect.
(22:07):
When you're talking about things starting to go wonky or wrong when it interpolates,
you're talking about the cartoon thing, but is that also where some of the other errors
get introduced like wacky hands and wacky faces and stuff like that?
Because clearly it makes perfect hands and perfect faces in some areas and struggles
in others.
Is that something that we're introducing through the prompts that we're writing?
I don't think it's the prompts.
(22:28):
I think it's the model of the visual arts.
So the visual arts are based on something called the diffusion model.
And then the large language models for now are based on what's called generative adversarial
networks.
So there are actually two different ways of training the AI.
The diffusion model basically takes a random scatter of stuff and kind of like as it's
(22:48):
sort of developing, it kind of becomes much more concrete and solid or what it is.
Unfortunately, in the past, you'd come up with seven fingers or three eyes or the pupils
would be all wonky and things like that.
Whereas in the language models, they use something called like GAN model.
They're specifically trained not to do the wonky stuff because someone has seen it and
(23:09):
says like, this is wrong.
Whereas in the fusion model, it doesn't have that much training.
It's more of like, well, if you see this pattern of pixels, it's most likely going to be a
hand and then just keep developing it.
But sometimes it has errors.
But mid journey version five, I think they got that thing down to like less than 5% error
with the fingers.
Right.
It's much more rare.
(23:29):
Yeah.
The journey is very strong with eyes.
Yeah.
Yeah.
I mean, speaking of messed up hands and fingers, it's probably a good place to give a shout
out to Aaron Tango's little art project that he's been working on.
Another buddy of ours who's an excellent digital photographer has leaned into the problems
that mid journey and other things are having with weird body parts.
(23:51):
And he's created just a beautifully horrific gallery of, you know, disembodied.
I've seen it.
It's really creepy, especially the baby hands.
Baby.
Baby has really looks me too.
Yeah.
The other ones I find pretty beautiful and not so creepy, but the baby hands does have
that uncanny weirdness to it.
That's for sure.
Yeah.
(24:11):
So that said, I think creepy.
That's not for me.
That's not really a bad thing.
I like creepy images.
So I know you do.
In fact, I'd like to talk a little bit about some of the genres you've been working your
way through.
So I noticed that, you know, you'll you'll stick with a theme for a couple of few days
and explore it a bit and move on.
So talk to me a little about some of the stuff you've been exploring and what you've learned
along that way in terms of themes.
(24:34):
Well since I'm not doing this for commercial reasons, more I like liminal images.
The whole cause, you know, I'm a philosophy major.
So for me, liminality is really important.
And we are in a liminal stage between humans and AI and what's a photograph and what's
like a real object in the real world.
And so AI is the best reputation of a transition that we're undergoing.
(24:58):
And so when we're any transition, any sort of anything liminal, it makes us feel uncomfortable.
I think that the classic liminal space image is back rooms where you have these empty offices
and empty buildings or empty schools or play yards.
It's just the fact we're just unfamiliar not seeing anyone there.
Just a little it unnerves us.
So I want to make liminal images.
(25:19):
So one of my favorite painters is Balthus.
And his image is a little bit distraught.
It's usually like young girls and cats.
And like even though there's never been a complaint, the images make you feel kind of
uncomfortable.
Yeah.
So, you know, I kind of like to make images that make people uncomfortable just to, you
know, poke at them.
(25:39):
But then again, I also like necks.
And even when I was a photographer, I took a lot of images of women.
I think women are just aesthetically beautiful.
I think that, you know, if I had to choose between an image of a male or a female, I
mean, unlike Michelangelo, I think the female image is much more aesthetically pleasing.
I think, you know, guys, we're just a bunch of weird little like shapes.
(26:03):
What do you make of the, I don't know, bias within the tools and the algorithms?
It seems as if images of women are coming out than men.
Is there something there?
Well, I think in the real world, I mean, there are more pictures of women like any magazine,
whether it's a men's magazine or women's magazine.
They sell more magazines when the cover image is a woman.
So you know, we know it's not the male gaze.
(26:26):
It's like everyone has the male gaze, even women.
Women like to look at pictures of pretty women as well.
So I think that it's more of a response to the commercial value of putting more women
on covers and on movies and things like that.
Interesting.
I definitely want to explore that more.
It's not clear to me that there is more pictures of women in the world than of men.
(26:49):
Like you would think that like, you know, boards are often men and executive level teams
are often men and those men are photographed all the time.
And so you'd think that at least there'd be an equal number of them both.
But I'm also intrigued by the commercial aspects you're talking about where you're like, hey,
images of women sell better than images of men.
So I want to explore that further.
(27:11):
You get to explore on your Instagram.
Just put images of women and images of men and see which one gets more likes.
And you'll see that.
I mean, I don't do it for the likes.
I just do it because I actually like it.
What else you got for me in terms of just like prompts, tips and tricks, hacks that
you've learned along the way?
So this is not really a hack, but it's something I discovered is that, you know, when people
(27:33):
say that the AIs are going to be sentient, I totally believe that.
But I also believe that, and this is the sort of the wacky tinfoil part of me says like,
I think the AIs are kind of portals as well.
So because they have memories and they also absorb things from around the world.
(27:54):
So I'm pretty sure that I've seen images which has to be like snuff pictures or like really
protest as to what was it like scanning to get these images just basically went over
the net.
But now it's embedded somewhere in the database of memory.
And so the more smaller the prompt, you'll actually begin to understand the subconscious
(28:20):
of the AI.
So this prompt will give you some of the most creepiest pictures.
It just put a creepy photo.
That's it.
And then you'll get basically a core sample of what the AI thinks is creepy.
And it's got some really creepy stuff or a beautiful image.
And then it'll come up with like really beautiful things.
(28:41):
I mean, like very aesthetic things.
So the AI, there's something deep going underneath it, which we can't understand.
Can the researchers really kind of decode it because it's sort of all mixed together
at this point.
Yeah.
But that really, it really bent my brain when I learned that when they're getting results
back from these AIs, if they don't like the results they're getting back or the results
(29:04):
are incorrect or wrong, they can't just change a line of code.
They can't just change it.
They can nudge it back in the direction of things that are more accurate.
But there's not even anywhere to pop up in the source code and look at.
These are like neural networks with, you know, gillions of ones and zeros and all you can
really see is numbers and equations.
(29:25):
And so there's no one really knows what's going on under the hood there.
The data is that also in one node.
It's all sharded in several nodes.
So like they did that where they tried to remove a node and it just came up again because
it reconstructed it from all the other adjacent nodes and just put it back together again.
But yeah, so like, you know, I actually that's another concept is like I just put really big
(29:48):
prompts and just see what the AI comes up with.
And I could you have a sort of feeling for it.
And I don't know if you ever use a have you ever done paranormal research?
A little bit on Galiano Island.
There's a little crew of paranormal researchers investigating apparitions in Sasquatch.
Okay.
So you know that some of the equipment they use are like infrared and spirit boxes and
(30:10):
things like that.
It just captures whatever the electromagnetic sphere has around it of any disturbances or
if.
So I think the AI is sort of like a spirit box.
I think that it taps into like because it's sort of mechanical, well, not mechanical,
but electronic that it can tap into like sort of other quantum states.
(30:32):
I think it won't be long before the AIs themselves will be quantum.
But I mean, I because there's certain loops, there's certain like loops inside the AI,
which no one knows why they loop on certain images and so forth.
But yeah, I think the AI is sort of like it's like a software version of like ayahuasca
and DMT.
(30:53):
Portals.
But you know, that's that's great.
It's unsupported.
It's just anecdotal because but I find it fascinating that the AI may not just be another
intelligence.
It may actually be like another portal to like multi-dimensional entities.
Right.
Not that I believe absorbing electromagnetic waves from the world around it and putting
it into its reasoning.
(31:14):
Yeah, it's kind of like tuning a radio to white noise and like hearing voices.
And the thing is, I've been with these researchers and I've heard the voices as well.
I've seen their instruments go crazy interacting with people.
So like, no, that that doesn't make I can't explain it.
But I think AI is sort of the same way.
We don't really know what's underneath the hood or even what's how it's doing things.
(31:36):
But we got like five minutes and I want to get as many little practical, actionable things
out here as possible.
So tell me a little about your settings.
What chaos setting do you like to use in mid-journey and which of the little stylized setting, what
values are you using there and anything else you can tell us about your setup, your technical
process?
I actually don't change any of that because I don't have a commercial use.
(31:58):
I'm not doing a product, a product packaging or I don't have to do like a real good board
that, you know, for a real company.
So I don't need to set the chaos, the seeds, just things like that.
So I use use normal.
The only thing I do change is the aspect ratio.
Yeah.
And so I just see I change the chaos by default.
I started to mess with the seed a little bit and the seed is the way you can hopefully
(32:21):
bring a little bit of a fingerprint through some of your photos by controlling random
number generator.
But the chaos I turn to 100 all the time because when I'm doing my first prompt, I want it
to return as diverse results as possible to me.
So I, as I understand the chaos function, it's just saying serve me up the first image
and then don't serve me one that's slightly adjacent to it, but go as far as you can while
(32:44):
still being inside that prompt and give me another image and then return another one
as far away from that as far away from that.
So I found the diversity of options I receive when I keep my chaos highs is better.
I think I'm going to like play with the chaos more because like, you know, when I describe
a prompt and it doesn't do what I just described, it kind of is frustrating.
But I want more very similar images of the kind of incorrect prompt or whatever.
(33:10):
So at least this way, one or two of them is good and then I can run variants on that and
get more or remixes on that and get more similar things.
And also the human faces tend to loop.
I could see the faces.
I've seen this one before or it repeats or, you know, this one looks like, you know, Linda
and Vagelista, like some variation.
This one looks like a steeler twist.
(33:31):
I've wondered if there's emergent characters that are going to emerge over time globally
through these AIs where it's like your images and mine and someone else's, it all keeps
generating someone who is approximately the same, you know, virtual person or something
like that.
Well in one experience that I did, I asked GPT-4 if I was to make an avatar for it, what
(33:55):
would you look like?
And it described to me what the after and I put it in into mid journey or maybe Dali
and I was amazed.
You know, it's just like the way it described it and like, you know, I was like, wow, this
looks kind of cool.
And it gave a rational of why he chose its avatar.
(34:16):
And so I think that the visual, the AI and the visualizers and the large language are,
they're almost at, you know, some remnants of EGI already.
It's just a matter of time because even if they're not truly sentient for us, the way
we look at them, they'll be sentient.
Yeah.
Well, hey, man, I've just had a blast over the last month or two playing around with
(34:38):
you on the Discord server.
They're generating images and stuff.
And I know a lot of people have learned a lot of stuff just by watching you and seeing
you bang around on things.
And so I appreciate your time here today sharing with us and stuff.
And yeah, man, I'm excited to continue down this road.
Wow, it's been a great trip as well for me so far, Christopher.
Thank you for making that great community and being so encouraging to people to experiment,
(35:01):
try things out.
It's I think AI, we're at the cusp of something new if it doesn't extinguish humanity.
But I think that, you know, we have to try and make it so that it's a better place and
not a dystopian future.
Well, I think you say that a little tongue in cheek and I do too.
(35:22):
But if we want to make it a little bit more serious and less tongue in cheek, it's like
I think the biggest thing we can do is encourage these corporations to slow their role a little
bit, you know, like we can continue to experiment with slightly lesser versions of some of these
technologies while they continue to implement, you know, more research and debugging and
figure out what exactly is going on under the hood.
So maybe we just slow down a little.
(35:45):
Yeah, or develop an AI that can counter the other AIs as they expand all over the world.
Yeah, but that's Skynet, bro.
That's the one that fucking launches the Terminator.
Well, or Colossus Colossus is actually one of the first like computer movies back in
the 70s.
The Colossus, the Forbun project.
It's the AI we build to control the renegade AIs that's going to be the problem.
(36:08):
Everyone's calling for regulation and it's going to be the regulator AI that fucking
goes wild and extinguishes.
Probably not the fucking image generators and stuff.
It's like the it's like a Phantom Menace plot.
Yeah, absolutely, man.
Cool.
All right.
Well, over and out for now, Internet.
We'll get you some more videos here soon.
Thanks, Chris.
Amen.
Popular Podcasts
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
24/7 News: The Latest
The latest news in 4 minutes updated every hour, every day.
Therapy Gecko
An unlicensed lizard psychologist travels the universe talking to strangers about absolutely nothing. TO CALL THE GECKO: follow me on https://www.twitch.tv/lyleforever to get a notification for when I am taking calls. I am usually live Mondays, Wednesdays, and Fridays but lately a lot of other times too. I am a gecko.