Unsolicited Feedback: Confluent's CPO Shaun Clowes [Summary + Transcript]

00:11
Fareed Mosavat
Welcome back for part two of our unsolicited feedback episode of me and Joff Redfern in conversation with Sean Close. This part two is a deep dive where we give unsolicited feedback to first Devin, a new software AI agent and development platform made by Cognition Labs, which made quite a splash last week. And we give some thoughts, expertise and feedback to Reddit on the eve of their IPO, which should be happening any minute now. We hope you enjoy this episode. We'll dive right in.

00:44
Fareed Mosavat
I think this might be a decent time for us to segue to the next topic, and one of the things that a lot of people are talking about this week is the launch of Devin, which is AI coder basically is the best way I can describe it, that claims to be able to do relatively complex programming tasks completely autonomously or mostly autonomously. And some of the demos for this seem pretty impressive. I've not been able to use the product myself and I think, Joff, to your point, of really technical, ambitious founding teams. The founding team here won multiple programming competition, super technical, and seems to have done some stuff that other people have struggled with.

01:38
Fareed Mosavat
I've played around with a lot of the baby Agis and GPT agent GPT type stuff, and the planning step is the place where these ten come apart at the high level planning, but they claim to have built their own thing for that and do some complex high level task planning, and it seems pretty cool. I'm curious about your reactions to this announcement and whether you believe it and whether you think it'll work and what implications you think it might have if it does.

02:09
Shaun Clowes
I really give credence when in the very first moment of announcing something, people are honest about, the numbers and the numbers are not blow you away. So in their announcement, I think the number was that it successfully completed 15% of tasks, which is still like wildly better than any competitor. I think that's double anyone else. More. Okay, wildly better is the 6% TP. I can't remember these something like this, but I remember they did share the numbers. It's funny because my first reaction is 15%. Okay, so that means it did successfully do that. But then the tasks that they're talking about are actually unbelievably complex. And so the fact that it can do that, the question then becomes, okay, is the path from 15 to 20 like a five year journey, or is the path from 15 to 60 a one year journey?

02:54
Shaun Clowes
And you could believe either thing, right? Because again, the devil is in the details. The devil is in the edge cases I've used actually llama two and Gemini and QBT four to write bits of code before. I've actually used them to write complete applications just for random tools that I need for my job. And I've actually been really impressed by them. Broadly speaking, Latin two has never been able to produce anything that would work directly, but it's usually got a few lines of code wrong. Gemini has come closer, usually with one or two things that have hallucinated made up APIs that didn't exist or whatever, but broadly speaking, whatever. TPT four, when I used it, actually got it all right. The code was terrible, but it ran. Literally ran. Like you just loaded it.

03:34
Shaun Clowes
It ran and you'd be like, wow, I would never write this as a professional engineer. But roughly speaking, this works. I have given it an abstract, complex technical problem, and it has built an end to end application that does that. And so when you think about what that means in terms of opening up the ability to program, the ability to make computers be able to do things in an unlimited number of times, to automate away portions of your life, that is just really, just unbelievable. But it's just so unknowable whether or not we're really making steps forward or we're just finding the local maximum. We're solving specific engineering tasks, or whether or not we are in fact producing, I guess, not an AGI, but a general programmer, like an AGP. Is that what we're producing? And how would we know?

04:18
Joff Redfern
Well, it's interesting, because I had this conversation with, he's a professor over at UPed, now he's a PhD at MIT, this guy, Daniel Rock. And he had written a paper with a bunch of his graduate students where they were looking at the economy, and they said, you can think about every job in the economy as a basket of tasks, or for product managers, basket of jobs to be done, then you can break those down. And if you think about how many jobs to be done there are as product managers, which we've been, there's not a lot. There really aren't. There's probably seven or eight big categories. Each of them probably have five ish underneath them. Let's say it's like 40. So you got that part of it.

05:00
Joff Redfern
And then the question is, some of these things are done by a human, some of them are done by an algorithm, and some of those roles are shared. So I think that the mental model I've been using is not what stuff comes out of the economy, what job gets killed. It's like, no I'm more comfortable sharing my job with an algorithm. There'll be some new stuff that comes out, there'll be some stuff that comes in, and I'll have a little slider for each one of those tasks. And that, I think, is a good way to perhaps think about it. On the devon stuff, just riffing off of some of the things that Sean had pointed out. And I'll go back to that analogy from my friend Bob.

05:41
Joff Redfern
It's like there was a little bit of techno Legos when I first looked at it, because, like Sean, I've been building tools. I've been coding again and solving my own problems, and I've been using vs. Code. And you go into vs. Code and non engineer going into vs. Code, you're just like, oh, my God, this is like Las Vegas. It's got so many knobs you can turn. It's like no one had to make hard product decisions because every engineer works differently. So we're going to expose all of it. And then on top of that, there's like thousands of extensions. So you get in there and you're like, what the hell is going on?

06:15
Joff Redfern
And then when I looked at what they had done is it felt like that team stepped back and said, hey, if I had AI as this co agent, what would I do? And they broke it down into the four primary tools that you would need. You need a chat window, you're going to need a terminal window, you're going to need a browser, and you're going to need an editor. And that felt like, so refreshing to me. I was thinking, man, okay, here's someone who really stepped back and is trying to reinvent for the future. And then the examples are, it's hard to say because you haven't played with it. But I will say one of the things that I quickly suffered getting back into coding is just setting up the production environment for the. I mean, it's better.

07:01
Joff Redfern
It's way better than when I first learned to code, but it's still a monster of a task, getting all your libraries set up. You've got some things that clash. You've got virtual environments. Then, of course, I've got a laptop and I've got a Mac. So I need those environments to match. And then at some point, I might put it up into the cloud, and that thing's like a mess. So this one example that I saw yesterday of setting up the environment I was really enamored with, I'm like, that's a real problem that I still face. And just the ease with which the program did it. And that focus on the planner I thought was really important. Because the thing about LLMs is you quickly realize they're good at one step, but they're not really great at multistep process.

07:48
Joff Redfern
And in order to do that effectively, you need a planner. And in order for the planner to work effectively, it probably does need to know more about your given context than your standard LLM might know. And I'm excited to try it out. I thought it was really well done.

08:03
Fareed Mosavat
Yeah, I think the UX decisions were really interesting that it's billed as this fully autonomous AI coder, but really there's an editor, there's a browser, there's terminal, et cetera. And so I think it is built for autonomous but directed, which I think is really smart to build some of that UX versus a lot of the early projects on this were like, I give it a task and it just runs and it goes and does things, and then it does them wrong and then it breaks and you don't know how to give feedback. And if you think about the interface with an engineer, the interface is not. Here's a spec. See you later. Here's twelve Jira epics or whatever. Go get it done. It's a constant back and forth, iteration, conversation, discussion, conversation.

08:50
Fareed Mosavat
So I think that's one area where I think they have done a cool job of building some of the natural user interface of how you would interact with code, but also like how you might interact with an engineer, which Aaron called it like the human interface. Right? I'm typing into a box and telling it what it should do next or what it's doing. I think that's really smart. And yeah, more Joff like you on the side of every other programming environment has made more engineers, not less. Why do we think that AI will make fewer? I think it's unlocking more people to have the power of code. Maybe that is a totally naive worldview about the doom of these things, but I think if you're a smart maker who's got creative and interesting ideas, these tools should unlock more power to you.

09:39
Fareed Mosavat
Because I don't have to spend. I spent 4 hours setting up a python environment so I could write like a 300 line python script for a silly side project golf thing. I was doing like simulation thing I wanted to build. I used GPT to do it. It took me like hours to do the silly setup work. Now replit other tools do help with that, but there is something to set me up the environment for this kind of application that, God, that's got to save a ton of people.

10:08
Joff Redfern
Well, the school system, too, has already been preparing people for that because lots of schools along the way have had a programming prereq that when I was growing up, we certainly didn't have. And they're using interfaces like MIT scratch to teach you how to program. So it's more and more part of the regular coursework of high school. So you've got people that have enough of an aptitude and influency with programming to invite more than the tens of millions of developers that exist in the world today into the system. And I think with that democratization, to your point, you kind of unlock another gate, which I personally am excited about because it's enabling me to do things that weren't fun enough for me to do a few years ago. So I always deprioritized it.

11:00
Shaun Clowes
It's kind of a doom and gloom alternative version of this as well. Call it evil AI version. And I don't mean like rise of the machines or whatever. I mean in general, the creative process, whether it's programming or even it's writing a business document, is about ordering a set of thoughts, deciding what you're going to write about, writing about the thing, revising the thing, figuring out where you're wrong. Like kind of an iterative process. There's an iterative process that goes on to go from nothing to something. And the thing is that the iterative process feeds on itself. And so there's an alternative world in which if we end up writing, I'm going to start with English first, but I'll end up at programming.

11:39
Shaun Clowes
There's an alternative world if we start with English and documents and we move from writing a bunch of essays in high school by hand to writing a bunch of memos that no one ever reads at work, to writing better memos that people maybe read, to writing strategic documents, to eventually becoming senior in your organization, because you've shown that you can think that way and whatever, there's a world in which if you skip those bits, like you skip the essay writing because you have a computer writer, you skip the deep thinking because you had a computer do it for you, then in the end, you actually don't have the capabilities to do those things. And at all, they don't even, you know, Noam Chomsky, it's very debated in the line of linguistics, but anybody who's followed linguistics, he's like, language is thought.

12:21
Shaun Clowes
Language and thought are the same thing. They are not different. They are thought. Language is thought. Language is equal to thought. And so he's like, you cannot think without language. It's like, literally a tool of your mind. It's the way your mind actually works. And so if you lose the middle, you might not have the ends at all. You could imagine a world in which we actually end up with a lot less developers, a lot less engineers. We end up with a lot less sophisticated thinkers, because the middle doesn't exist anymore. Engineering, that's obviously true, because you write a code, that's one thing. But then you spend easily five times as much debugging it or iterating it as you do working in it in the first place.

12:54
Shaun Clowes
And so you still need kind of those capabilities to look inside the box, know what is wrong with the box, not whether or not the box is there or not, but what is wrong with it. So there's like this interesting kind of interplay going on there.

13:05
Joff Redfern
Well, yeah. Adjacent to that is you've got your typical organization. Large organization is going to bring in lots and lots of young people because it's really an apprenticeship. You have to get in there and you have to learn from the ground up. So you bring in lots and lots of people for that apprenticeship. So if you have a copilot, people will say copilot is like having an intern, or Copilot is like having a junior engineer on your team. They're really good at the first draft, but then you need to clean it up. And that would be one of the downsides as well, is if you were to bring in fewer of those people, because we need that in the organization, because we need the apprenticeship to grow up to be the senior engineer, then the manager, and so on and so forth.

13:50
Shaun Clowes
Yeah.

13:50
Fareed Mosavat
If you imagine that as you go up the stack, the jobs are more durable. Right. The work is more durable because it is really about strategic thinking. It's about deciding what to do versus just executing what it is. And those are complex problems. We talked about this in a previous episode. We talked about the idea of tacit knowledge that can only be gained by having actually done it. The things that come from experience, these are going to take longer. And also, where there's no right answers, there's actually equally good answers. And the decisions about what to do are nuanced. There is this worry I have, I think the same as you joff, that if every company is only hiring very senior IC product managers, for instance, principal or staff level people, because they can leverage small teams to get a ton of work done.

14:39
Fareed Mosavat
So I could ship a whole new product line for Atlassian as an individual person instead of a VP of product with four directors who each have four PMs underneath them, running a team of hundreds of engineers, or whatever it might be, I could do that with four engineers all using AI copilots to do things and iterate really quickly. How does anyone get to be that job, right? What is the pathway to that work? What would your advice be to a young person?

15:07
Joff Redfern
Let me say one thing about that, especially for our younger audience, because I've been through four downturns, this will be my fourth downturn. And the thing is, it's kind of unfortunate because we've got a pullback in the economy. At the same time we've got the introduction of a new technology that potentially threatens the structure of the economy or employment. And I think what's happening potentially is we're conflating these two things because you look at the pullback and the job layoffs, and they're saying those are the canary and the coal mine. Those are foreshadowing the AI thing. And I think it's highly probable that they're disconnected at this point. And what I've seen in the cycles that I've lived through is you go through a boom cycle and you just get rewarded for growth. It's like, go grow the business. Don't worry about profitability.

15:59
Joff Redfern
And then for a variety of reasons, you get a depression in the economy, and then people have to reverse course. And in reversing course, the shareholders, the people who are investing in your company, they value profitability more. So when they value profitability, they're saying, okay, well, let's go ahead and pull back. We've overhired and many of the CEOs that you've heard know, meta, Google, so on and so forth. They said, hey, we just misjudged, I think, another narrative, which is probably not helpful, is imagine you're the CEO and you have to explain why you're pulling back and laying people off. And a lot of the reason could just be like, you don't have a good business. But instead of just saying, I don't have a good business, you're saying, oh, I've got this automation stuff that's coming, or I'm going to automate these roles.

16:46
Joff Redfern
So I'm just preparing for it. And I think that's a little bit of an injustice. So if I have one takeaway, there is know that there are two things happening at once that may be correlated, but there's also a good probability that they're not. And that's not to say you shouldn't be worried about what the future looks like. It's probably more to what Sean was saying earlier. That saying it's like we tend to misjudge how much is going to happen in the short term and underjudge how much is going to happen in the long term. And I do think that we're still in the early innings. I do think it structurally will take a while yet for these things to roll through, different roles of the economy and so on and so forth.

17:29
Fareed Mosavat
Yeah, certainly a lot of the Twitter chatter is Devin is the end of boot camps, Devon is the end of undergraduate CS programs, et cetera. I think that's probably somewhat overstated. I do think those things will have to change. Just teaching somebody how to iterate a linked list or build a simple react web app is not good enough. My suspicion is the further up the stack you go, the more valuable you are, and that means you can do more on your own. So I think it's software building is still hard, useful software building, but code writing might be easy.

18:02
Shaun Clowes
Maybe this is just a motorcycle for the mind. You've jobs set a bicycle for the mind, maybe this is just a motorcycle or a jetpack for the mind. I think that humans do have this kind of constant ability to shake up whatever it is they see. When you give them a new tool, they create new things. We've been creating art for all of our existence as a species continue to create art, new forms of art, and new twists on art. And wasn't it Bill Gates who said that 640 ram would be enough? And another way of saying that he thought, well, there's only so many business apps the world needs. Like you only need Microsoft word, you only need exchange, you need Excel, and you're pretty much done.

18:37
Shaun Clowes
But it turns out that there's infinite niches and niches and niches of the problems and these tasks, these baskets of tasks that Joff was just talking about, there's like a truly enormous number of them. So maybe this will be a true renaissance. Enable a bunch of people to take advantage of automation and computers and become even more creative.

18:55
Fareed Mosavat
This is sort of a weird digression, and I'm sorry, but I saw an amazing thing on Twitter yesterday about this satellite company building these giant mirrors to be able to, like the moon, reflect sun back onto solar farms so that they could generate energy at night. This is the wildest thing I've ever seen. It blew my mind, right one to even think of that idea, right two, to believe that you could do it, and three, to actually execute it. There's so much to that. And I imagine, like, a world where that person with that brilliant idea and just enough technical knowledge can unlock so many more. No one will believe you. No one will believe you unless you can show that it's sort of possible. And so could that person build the prototype instead of for like millions of dollars or whatever?

19:46
Fareed Mosavat
I don't know how much it costs to get the first version of that with the balloons and all the things they tried going, but could do it for 10,000. How is that not good for the world?

19:56
Shaun Clowes
Right?

19:56
Fareed Mosavat
I think it's really interesting. Or if you imagined you could land a rocket on a boat in the middle of the ocean, and you could write the toy controller for that, because you had that idea in a simulation and prove that it could be done before anyone believed it was possible. I am more on the polytta side on these things. I just think, like, human creativity has always gone in one direction, and I don't see how new tools make it go the other way. Our capacity for new stuff is infinite.

20:27
Joff Redfern
That reminds me of so sean and I worked closely with founder Mike Cannon Brooks over atlassian. Scott and Mike founded the company. And Mike is such a bold and courageous both dreamer and doer. And I remember him telling me in one of our one ones, he's like, okay, so I'm thinking about investing, and we're going to have a solar farm in Australia, and then we're going to pipe the electricity underground over to Singapore. And you're just like, wait, what? The thing is, he's accumulated enough wealth to actually try to take a run at that. I don't know where that project stands, but I agree.

21:08
Joff Redfern
I think that some of these really wicked problems, there are so many permutations and unknowns in there that you tend to say, I'm not going to try to approach it, but if you had a system that had infinite knowledge, or at least all the knowledge that's been written down, and it has a speed in which it can process things, that you could be solving much harder problems than we're solving today.

21:40
Fareed Mosavat
Yeah, I actually really like using the Rumsfeldism or whatever as the thing that you should be focused on, which is the unknowns. AI is obviously great at knowns, but the unknowns are where all the cool stuff know. How might we make solar farms be able to have light at night? Maybe an LLM could help you solve some problems around that. I don't know. But it could at least make you wonder whether your crazy ideas are less crazy than you think they are.

22:07
Shaun Clowes
It's interesting to think if you can have more builder time or more thinking time, whether or not that would by itself inspire a whole bunch of kind of creativity.

22:15
Fareed Mosavat
Yeah, I think if I were a product manager right now, like a fresh, young two to three years in, relatively junior product, person or engineer, I would focus not on how to write great product briefs and memos like you described them, sean, but instead on how to build great prototypes, because these tools unlock prototyping for way more people. If you have one undergraduate CS class, you can probably use chat, GBT Devon, or a couple tools like that to build a not working software as we described, that does all of the important things, but certainly prototype the things that you think should exist much more easily. And I think prototypes are 100 times more powerful than product specs.

22:59
Joff Redfern
I think that's a great suggestion. Another one. What I would tell my young self is focus on teaching yourself to learn, like be adaptable, be able to be very scrappy and relearning something. And I think that would serve you well. And I would also try to nurture my right brain skills, not just my left brain. So my left brain is my analytical side, my right brain is my creative side. And I would do things on that front. And I think those are going to help as I think about the industry.

23:32
Shaun Clowes
Moving forward, only because I would agree with that Joff. But I would also kind of take your thing, Farid. And I would say even today in product, a lot of product leaders in the world, there's actually a little bit too much delivery, even in the form of prototypes. And not enough question discovery, like not enough discovery of problems, not enough discovery of the meaningfulness of those problems, how severe they are, who has them, where do they have that problem? What is the context? I actually think too much of product management is seen as mechanical.

23:59
Shaun Clowes
And so I think if I was going to give a product manager a little bit of advice about LLMs or whatever right now, I'd be like, think about the things you can do to bring as much information as you possibly can from anywhere you can find it to this LLM and say, tell me what you see. It'll be like, tell me the problems that you see in all of this data, all of these screen recordings, all of these videos, all of these audio transcripts, all of these verbatim feedbacks. We're living in a truly tremendous time. Many people probably use tools like user voice, or they've had NPS surveys or they've run surveys or whatever, and people really suck at getting great insights out of those things because it's really hard.

24:33
Shaun Clowes
It's like you're trying to take these quantitative things on top of this really qualitative feedback expressed in human language. This is the best moment of all time. To be able to do that, to really be able to ask interesting questions. Like, you can literally copy and paste stuff from Google forms into ChattbpT and ask it questions, and it will give you good answers. That is by itself a truly amazing capability.

24:53
Joff Redfern
Do what Sean said.

24:54
Fareed Mosavat
Yeah, do what said.

24:56
Joff Redfern
Yeah.

24:56
Fareed Mosavat
Then build a prototype from it. But don't just build a lot of prototypes.

24:59
Shaun Clowes
Exactly. Well, I mean, you have to go test your ideas, don't you? So I agree with you about the prototyping.

25:03
Fareed Mosavat
Yeah, I think that's great advice. So one of the things we do is we ask our guests to bring a topic that they're interested in or a product or something coming in, the news that they want to riff on. So I would love to have you kick this ONE off if you can.

25:16
Shaun Clowes
Yeah, I mean, I think the Reddit IPO is like a really interesting piece of news. I would argue there's even an AI side to it. There's an AI side to it. There's a social, there's an ad tracking angle to it. It'd be fun to TaLk about. Why now? Why Reddit? Yeah, what are the Assets? I mean, it looks like it's going to be very richly valued. It looks like there's even going to be sales of stock to moderators. Moderators are going to be able to participate in the IPO. So there's clearly going to be an interesting IPO, assuming it gets off the ground.

25:45
Fareed Mosavat
One of the fascinating things about this that I was just thinking about is that in a lot of ways, Reddit was the first social product before all the other ones that are already public. I think Reddit was founded before Facebook or around the same time. There was a point in time where people thought dig was going to win, not Reddit. I mean, that seems, like, wild to think about. This company's been through multiple wild sort of transitions. It was acquired by Conde Nas in the earliest days. It's also from the first YC batch.

26:20
Joff Redfern
That's the gift that keeps on giving. What's going on there?

26:23
Shaun Clowes
Yeah.

26:24
Fareed Mosavat
So Reddit was part of the first YC batch. I think it was YC's first exit when it sold to NAS. I believe that's true. And now has been through multiple CEO changes, founder changes, ins and outs, and crazy things, and now is finally public and. Yeah. Or will be soon. So it's kind of a wild story of just everything takes longer than you think it will.

26:47
Shaun Clowes
Yeah.

26:47
Joff Redfern
I mean, if you kind of go back to the heart of it's very much Internet 10. Right. It's a group chat product. Yahoo Groups was incredibly large at the time. There was no investment that ever went into it because it was unwieldy. We weren't sure if advertisers would invest in it because it had all sorts of, like, there was an underbelly to what was said on some of those communities. Some of it wasn't appropriate for advertisers, for example. And then later on, LinkedIn had groups like, they had professional groups, Facebook has groups. So the groups as a paradigm has been around for so long, the thing that they've done so well is they've had the right amount of moderation and now they've found the right amount of monetization against it.

27:44
Joff Redfern
And I think it's part of what Sean was saying where we've seen some changes in privacy that have meant that, okay, now, advertisers could really benefit from having communities that are interest based communities or narrowly defined and moderated. So they're on topic, and I think that's worked out pretty well. I mean, 804,000,000 in revenue growing 20% year over year, that's a real business there. And that 20% year over year, if I look at that whole software sector, public software sector, we've gone from really attractive 40% year over year gross that they're getting pulled back in the current economy. So it's not what we would have seen as a growth rate a few years ago as being wildly exciting, but it's a reasonably good growth rate for the shape of the macro economy right now.

28:42
Shaun Clowes
Yeah.

28:42
Fareed Mosavat
And I think the other piece that's interesting is it's been around for long enough that they probably could. I haven't looked that deeply into the numbers, but I suspect it's probably been growing pretty steadily for a very long time and sort of has great compounding growth loop dynamics. Right. More content equals more SEO. More SEO means more traffic. More traffic means more people being content creators, more engagement, more usage, better advertisers, et cetera, that you could probably make the case that like a Facebook or others, it has the dynamic of 20% year over year growth for a long time versus just explosive growth for a short amount of time. It has been around a very long time. It is a 15 year plus, slow grower by Internet standards, and the world.

29:35
Shaun Clowes
Kind of came to it. It's interesting because what Joff was talking about earlier in terms of moderation, as the Internet has become increasingly full of content designed for SEO, designed to bring you into the website, there's been a whole, I guess, degradation in just general content on the web, and then a bunch of it has gone behind. You know, Facebook content is not public at all. And then you've got a bunch of web content is no longer any good. It's all effectively spam. You're like, well, where do I go? And that's actually paid off in two ways. One is it's paid off because they have become a destination. So when people search for a review of a product anymore, I don't know about other people, you probably search for review product Reddit. There's literally a pattern. People do that.

30:15
Shaun Clowes
Well, at least whatever the review is on Reddit is going to be a moderated review. And assuming it's in a popular subreddit, I'm going to get something that's less likely to be just total spam and trash. And that's great. But that then pays off again, because it turns out that not only does Reddit end up collecting a valuable advertiser audience with intent, so, like, relatively, advertisers can go to get people who have an interest, it also means that they are sitting on top of a gold mine in terms of data that can be used to train AI. And I'm pretty sure that in their IPO prospectus, they included that they have a deal. I think it's Google, a multi billion dollar deal with Google to sell the data, basically to use the data to build AI algorithms and everything else.

30:54
Shaun Clowes
And so it's kind of like a bastion of human generated content. It's a bastion of community, it's a bastion of human generated, high value content. And it turns out that in the world with increasingly machine generated content, human generated content gets more valuable. It's just a very interesting business from that perspective.

31:11
Joff Redfern
Yeah, for sure. I don't know if this is right or not, but I was looking at what was the average revenue per user on Reddit, and that was actually a low number. That was like $3.42, where you get a premium in the US, but they have a large global audience. So that brings that average revenue down. But you compare that to something like Facebook, which again, I don't know if these numbers are right. This post was saying it was more like in the neighborhood of $68 in the US. So you're like, holy crap, that's a really big delta between advertising dollars. So if you can close the gap on that's a tremendous business opportunity.

31:51
Shaun Clowes
Yeah.

31:51
Fareed Mosavat
And Reddit is relatively new to the ad monetization game versus the longer arc. And so I think you probably have the case that while user growth, you can probably make pretty predictable assumptions about the revenue growth is a question mark, which makes it both. If you believe it's a great upside opportunity and if you think that's a sign that it will never get there, then it's a more complicated decision. I think the AI monetization piece is a really interesting one, not only just for Reddit, but for actually, honestly, all user generated content, human built companies. And the way I frame this is the trade used to be I put things on the public Internet because Google will send me traffic, and traffic is good for my business because I can monetize it in a variety of different ways. Right.

32:44
Fareed Mosavat
If I'm LinkedIn, I put all the profiles that I painstakingly took to get out there because I get sign ins or I get premium subscriptions, et cetera, like all that fuels my business, advertising, et cetera. Same is true if I'm yelp, same is true if I'm opentable, et cetera, cetera, et cetera. There are so many different. But now what you're seeing is a world where things like stack overflow have now been completely mined and now there's no trait. The LMS are answering the questions directly. There's no traffic going back to them, and they are really suffering. And so the question mark that I have is, are direct payments the solution to this? Why would I put something on the open Internet question, what is the incentive trade off?

33:27
Fareed Mosavat
Reddit seems to be doing it through direct partnership, which I think is very smart for them as a larger player. But what are the ways in which this will work for smaller companies and smaller teams? I think is an interesting long term question. And I think, Sean, to your point, you'll probably see more and more of these becoming closed gardens, so to speak, because they want their unique content to stay unique. And so I think it'll be interesting to watch the case study of what happens with them as they are both being scraped by scrupulous and unscrupulous AI type tools as well as signing these partnerships for it. I think the same question lives for a place like the New York Times or the Wall Street Journal.

34:08
Fareed Mosavat
It used to be I put my stuff on Google so it could get searched, so I could get traffic and I could monetize that through subscription. But if it's just getting pulled by LLMS and I ask chat GPT, what's the interesting stuff that happened this week? I now have a problem. So I think that's one of the interesting things about this 60 million or whatever the Number Deal is. They certainly have the Leverage to do that, and I think that's really smart. And is that the Pattern for how.

34:33
Shaun Clowes
This is going to work in the Future? Yeah, I mean, I think we're definitely going to see an increasingly closed Ecosystem for the Reasons you just talked about. I think when you think about the Trade with Reddit, there's Three Parties in the Trade. There's the Producer, there's the Community, and then there's the Advertisers or Reddit. Like there's three Parties in the Trade. And I would argue that Reddit, I think a BullCase for a Product like Reddit is Community. It's about Community. It's about Strength of Community. It's about quality signal over noise. Right. Can you bring in enough PEople? And we all know that. What's the Old Saying? PEople say 1% of People Write, 10% of People Comment and then Interact, and then the remaining 89%, Luck, something like that. It's Basically, yeah, you know, the Numbers, Jeff.

35:18
Joff Redfern
Well, I remember that originally the first Time I heard it was Katerina fake was telling me about it when Yahoo. Had acquired Flickr, and they were really one of the first social networks. I think in time it's changed a little bit. That 1990 was 1% are like creators and then 9% are liking and commenting, and then the 90% are viewing. But I think as social systems have become more of the norm, I think those numbers have grown. But I think your basic concept is still the same. There are different classes of users, and as you get closer to the creators, those people are really the originators of a community like Reddit because they're the ones that are creating content and moderating the content.

36:02
Shaun Clowes
Yeah, it's kind of like one of those. It's an unstable system in a way. It's a stable system and an unstable system because you have to keep those people coming. You have to keep establishing new subreddits, and those subreddits have to establish a following, and they have to be people who are willing to moderate those things, usually for no compensation, and they have to do a good job. There's a lot of different variables that go into these things. And it's weird because to some degree, that actually means that Reddit, because of its constituencies, it means that at any moment, if it annoyed enough of its moderators for long enough, you can end up with really weird things. Now, they did have the protests.

36:38
Shaun Clowes
You would remember the protests, I think they were last year, about the closing down of the APIs that prevented third party applications. I think Apollo was the one that shut down. And they did kind of face a revolt. But the revolt, as far as the data seems to show, didn't do anything. It seems like the temporary shutdown of the homepage was not enough, I guess because whatever it is the moderators are getting from this relationship, it wasn't worth them to continue the strike. So there is clearly an exchange of value going on. But there are also weird things, which is that constituency still exists. And so remember that means that group will still be arguing. Their point of view is they want the moderation experience to be easier. They ultimately want Reddit to stay the same.

37:21
Shaun Clowes
It's a little bit like where I'm going to go with this. Those are the old timers. They are the people that have been with Reddit since the early days or whatever. And so they are both the number one asset of Reddit in a way, and also its biggest risk, because if Reddit wants to make big changes, they want to change the way moderation works, they want to change the way karma works, they want to change the way the front page works, they want to change kind of anything. They're going to have this very important constituency pulling very hard against any of that. Yeah.

37:48
Joff Redfern
I always feel like this feels a little similar to open source. Right. Like when you have an open source project which then goes and gets monetized, there's always this delicate balance between, okay, this commercialization of the project could be really good for the project, but then you've also got, it was built off of a different frame, and that framing was one of openness, and the compensation wasn't monetary, it came for other reasons. There was value in building up my credibility or was a problem I was trying to solve for myself or whatever your reason is. So I always kind of watch these things as they unfold, thinking that there's going to be some sort of revolt. And usually there's an uprise rise. Like, people talk about it, but there's not really anything that significantly changes it. Like, life just goes on.

38:36
Joff Redfern
It's like we have our uproar, we talk about it, but then we kind of go back and continue along the way. So that would be another interesting aspect of this.

38:45
Shaun Clowes
Do you guys remember what killed dig? Was Deek, was Dee killed because they changed the front page and that, yeah, I think that was what killed them. But was it the moderators who killed them when they changed the front page, or was it just that all the users hated the front page? I can't remember.

38:59
Fareed Mosavat
I don't know.

38:59
Shaun Clowes
Because that'd be an interesting, I do.

39:01
Fareed Mosavat
Remember there being a redesign and that was the beginning of the end. Like, people hated it and it might have even been good, but they decided they hated it because it was different. I don't remember the details. I do remember being around the Internet at that time. And it is interesting because Reddit, I think they've made some changes, but is mostly the very familiar. If you go to Reddit.com, if you went to Reddit.com a decade ago, and I think part of that is the sensitivity of what is effectively a two sided marketplace. Right. There's a whole bunch of visitors, viewers, readers, but the content creators, moderators have a lot of power because without them, Reddit communities don't exist. Most people are just reading. Right, I suspect.

39:44
Fareed Mosavat
And I think this move to allow their moderators to be purchasers think, I don't know if they're buying it or granted stock, but is a really smart move for a product like Reddit because they do need alignment with this hardcore community. They need this hardcore community to be aligned with them long term. Now, is stock enough, financial incentives, et cetera, but it's like their super community being aligned with the long term viability and goals and monetization and other things of Reddit, I think is a really smart decision. It's not just goodwill. I think Uber probably was mostly a PR play. Like we're going to let the drivers have some stock in the know, makes it look like there's lots of winners.

40:28
Fareed Mosavat
But here they have real incentive for these revolts not to happen, for them to be able to do redesigns, changes, product changes, increased monetization, removal of APIs, et cetera, without a ton of backlash. And maybe that alignment of interests will be valuable.

40:46
Joff Redfern
Yeah, I think it was like 8% of the IPO shares are being set aside for the Reddit users and moderators. So that's nontrivial amount. Then another tranche of stock that was going to go towards community related programs. I think there's obviously sensitivity towards and thoughtfulness that it appears was given to making sure this doesn't go sideways.

41:08
Shaun Clowes
Yeah.

41:09
Fareed Mosavat
Are either of you BIG Redditors?

41:11
Shaun Clowes
I'm a lurker. I'm a pretty lurker. I like to see what's on the front PAge of Reddit.

41:17
Joff Redfern
I've been to 90% as well.

41:20
Fareed Mosavat
Yeah, I'M a LURKER as wEll. A couple of SubReddIts here and there.

41:24
Shaun Clowes
I think one thing that's really potentially not obvious to people that don't work in kind of these spaces or study these companies pretty closely is that it's quite possible for a company to make a lot of Money and make a lot of Product changes that nobody normally sees. Sorry, when I say nobody, I mean the end user does not see.

41:44
Fareed Mosavat
Right.

41:44
Shaun Clowes
So the perfect example, and this is the case with many different businesses like this, content businesses, they can do a lot of work on Relevancy. So the Front Page is perfectly tailored to you and that will definitely help them win. Right. They can also do work on advertiser experiences, like how to get your thing into the right place, and that will directly enable them to monetize. There's many different things that they can do that would look like nothing has changed. Twitter spent, I don't know, what, seven years where basically the products didn't appear to change to the end user, did not appear to change in any given way. But a bunch of changes were happening on the advertising side, the segmentation side, the moderation safety side and all that stuff. I'm not saying they did a great job.

42:27
Shaun Clowes
I'm not saying they necessarily were crushing it on the innovation perspective, but these products are asymmetrical. Like, you don't see all of them just when you're the end user of the product.

42:36
Joff Redfern
Yeah, it's interesting you say that, Sean, because one thing I did have a chance to look at, the s one filing there, and I noted that R and D is 54% of revenue, which is, like, way higher than what I thought it would be. I'm like, there's a bunch of community boards have been around forever. And sure, you got a mobile app and you got a website, and it's like, what gives? And I think, to your point, there's quite a bit of algorithmic work that goes into the ad system and serving system, and I'm sure there's plenty of tools for moderators, but that did stand out to me. I'm like, okay, what's going on there? What are they doing? What are they working on? How many are there?

43:15
Shaun Clowes
So it's a bit of a chestnut. Unhacking. Is that almost any application you look at, you're like, couldn't they just whip it up over the week? Yeah.

43:25
Joff Redfern
Every place I've ever worked, they're like, come on, dude, you can get away with like a quarter of what you have. Exactly.

43:33
Shaun Clowes
Right. It's always harder than you think it is.

43:37
Fareed Mosavat
Yeah. I mean, I think if I look armchair product here, I actually think in a lot of ways some of the charm of Reddit is how this is going to sound rude to people who work on it, but it does feel a little unpolished in a good way. It feels very community driven. It feels content driven. It feels authentic. It feels like the community kind of runs the show. Right. It's a little bit of a shell that you can kind of run your own thing in, which I think is kind of cool. And each of the SubReddits have a little Bit of their own Flavor Identity, et cetera, around how they operate. I think that's really positive. I think the biggest Question Mark for Reddit is, yeah, that MonetiZation Gap. I mean, Facebook is one of the best monetizing consumer products of all time.

44:25
Fareed Mosavat
It's a very high Bar to compare things to, but there's a huge Gap there. And a lot of the other social networks struggle with this. None of them are quite, you could do the, if I monetize that Facebook's rates for everybody, and it would be very high. But Reddit does strike me as having a unique personalized Ad Inventory that is not available or possible in other Places. So how would you monetize Superfans, super believers, super creators around certain Things. Right. I had a problem with my Headphone amp on my desk. I still have the problem. And I went around Reddit looking for other people who had that problem. They had it. And there's a whole Audio File, SubReddit, full of people who are really deep, ski reviews, music, these kinds of things, and they feel under monetized in those areas.

45:11
Fareed Mosavat
And I'm very curious why that is and whether it's a format problem, a sales problem, a maturity, a platform problem versus Facebook, or is it just Facebook so good it's hard to take any dollars away from them? And it may very well be that. I don't know. But I think that's where there's a ton of opportunity for the product. The other is, I think there's something around, like, I think Discord is probably chipping away slowly but surely on certain types of communities, especially not big, broad ones. Subreddits tend to be large, but I'm sure there's like sub audio file communities that are around certain brands, certain things, and Discord is probably the home for them and chipping away. So I'm very curious to see how that intersection happens from a product perspective.

45:59
Joff Redfern
I'm seeing more product companies do both customer support and beta programs out of discord in a way that I hadn't seen that before. So you just launch a product and you're a fan of the product, you should go to this Discord channel and we've got the team in there and they can answer any questions and that feels really good. That's a good medium to try to get feedback from your early customers. And then I think the mid journey, kind of a wonky place to start your photo generation program. But that too kind of brought a lot of visibility into it because people wanted to get at that image creation. And then you went in there and you joined the server and you're like, wait, this is od to me. Why are we doing it here?

46:47
Joff Redfern
Can't you just have a website where I put in my prompt and I get a picture back, but they've kept it there for a really long time. I know they're moving out to a website now, but I think that in some ways has created some aura around, hey, if you're doing AI stuff, you should really be over in discord. And the way that, hey, if you were a gamer, you should do discord stuff. So I think that's probably helped the cause over there.

47:14
Fareed Mosavat
Yeah, I think that's where a lot of interesting conversations are happening. Right? Like Reddit has always called itself the front page of the Internet, I think was one of their taglines, but it is the front page for a bunch of different communities. But I'm starting to find that these smaller, more authentic niche type boards or discords are chipping away at certain areas of this.

47:34
Shaun Clowes
Yeah.

47:35
Fareed Mosavat
So I think that's an interesting, if.

47:36
Shaun Clowes
There'S one thing about the Internet, it's niches on niches on niches. It's like a vandalbot Fractal. That's what were saying earlier. If you can use AI to go attack these little kind of niches that are now viable, because if you think one thing really all we're doing here is we're making things that were not viable before viable, we've basically changed the efficient frontiers. And so who knows what's on the other side of this? So it doesn't surprise me too much that we're seeing this splintering on the other hand, if there's one thing that's common about PEople, they want to find their People. They want to find their People. People want to find people. People really love their community.

48:08
Shaun Clowes
And so I think there must be some Sort of NatUral Limit to the Ability that it can Fragment in Discord or in Reddit or in any other Thing before it ceases to be big enough to have any Gravity. It ceases to be any MateRIal Kind of Ability to form a Community.

48:24
Fareed Mosavat
Yeah, I guess I'll close with one last Question. This is a tough One because we don't exactly know Price, but curious. Buy, Sell, hold on, Reddit, do you think it will outperform the Market over the next, let's say, Decade? Just for fun, Long Term, do you.

48:40
Joff Redfern
Have to have some Sort of disclaimer like, we do not do financial service Advice?

48:44
Fareed Mosavat
Yeah, we're not Investment Advice. We are not professional. Well, you're a professional investor, but we are not professional public market investors. I'm just curious whether you think, based on Things we've just talked about, MonetIZation Gap, et ceTera, that there's Upside here and growth.

49:00
Shaun Clowes
I think it's highly like a ten year time horizon is almost unknowable, particularly in the presence of all of this AI generated content were just talking about. I mean, it's kind of a dice roll, but there's definitely reasons to believe. There's reasons. You could be like, hey, is this a thing worth having for a little while and see where the particular wind blows? Certainly they're different than many of the other different companies that are in the social spaces. And so if you're interested in that area and you're like, hey, I want to belong that area, there'd be reason to believe. But who knows? If they appear with a really insane valuation, you might be like, well, let's wait a year. Let's wait two years, and then I can buy in at that rate. So it'll depend on their valuation.

49:41
Shaun Clowes
But I think there's definitely reason to believe, depending on kind of the way any individual person wants to invest.

49:47
Joff Redfern
I'm more of a believer than a disbeliever. You're right, the valuation is important. You name the terms, I'll name the price, I'll name the terms. But one thing that I've observed is that once these flywheels get put in place, these giant flywheels, it's almost like inertia continues to take them. When I was at LinkedIn, I remember we have somewhere a copy of the first dollar we made on the feed, and that feed is now generating billions of dollars. I went from seeing that business at hundreds of millions of revenue. I think Ryan said it was something like 14 billion. I think that was part of Microsoft's announcement or some crazy number like that. So once the thing is set in motion, and this thing's been in motion for quite some time, I do think there are network effects at play.

50:41
Joff Redfern
I think there are flywheel effects at play. I do think that there's an opportunity for that revenue number to increase. They've got some talented people over there. I used to work with Jim Squires. He was over at Yahoo. Then he was at Instagram. He's over there doing some revenue projects and he's a smart dude. So, yeah, I would be more bullish than bearish on this thing.

51:04
Fareed Mosavat
Yeah, hard to say versus market. I'm not a professional investor, all those sorts of things. I think the lessons that I've learned, and I was going to say Joffrey, are very similar to yours. Like, network effects are hard to break and they tend to be way more durable than people really think. Long after people say no one uses X, it turns out people still use X and it's still growing and it's still driving. And this company has proven to be not just a cockroach like a quiet grower for a very long time. And so I think that's very interesting. And I don't dabble individual stocks. I've learned my lesson on that. Angel investments and index funds. But I have a hard time betting against network effects and highly engaged communities. And Reddit has a subset of really highly engaged communities.

51:52
Fareed Mosavat
It's very hard to bet against a company like that. I know it's a trope, but you tend to figure monetization after you have a product with hundreds and hundreds of millions of users. So I'm bullish too, but I'm not a buyer.

52:05
Joff Redfern
All right, awesome. Well, thanks.

52:07
Fareed Mosavat
That was fun.

52:08
Joff Redfern
Yeah, super fun. Sean, it was amazing to see you again. I haven't seen you in a while and fried. It was great to finally spend time with you in person after you ditched me on the last time were supposed to hang out.

52:20
Fareed Mosavat
Yeah. Thanks so much for joining us this week on unsolicited feedback with me, Farid Mosevat, my co host Joff Redfern and Sean Close.

52:29
Shaun Clowes
See you next weekend.