High-tech Coolhunting

Many of the most important ideas in technology come from the fringes. How do we spot them in the early stages?

An idea is born somewhere relatively obscure (maybe in a garage somewhere), spreads to small communities of hardcore enthusiasts (like Kickstarter and Reddit), and sometime later takes the mainstream by surprise when it suddenly explodes into popularity.

The journey is familiar in the context of startups, but it applies to important ideas and technologies more broadly.

One example is bitcoin, which began as an interesting whitepaper published in 2008 that was circulated among cryptography experts for a few years before coming to the attention of the mainstream startup community. Now, it’s a cryptocurrency — and blockchain protocol — sensation where activity is being tracked closely and community splits are being chronicled by newspapers around the world including the New York Times. (The first mention of bitcoin there was actually four years ago, in the context of the TV show “The Good Wife”).

There are countless reasons why you’d want to know about the next big idea in technology, and as early as possible. Whether you’re finding them, inventing them, or building businesses based on them, ideas matter, as does “the idea maze” one travels to get to them. But is there a way to catch these ideas as they emerge, in their very early stages?It’s difficult, because the places where these sleeper trends begin are seemingly random and obscure. This is tautological in a way, because if something exciting comes out of an established tech center (like Stanford or MIT), the mainstream will pay attention very quickly. It’s only ideas that are significant and come from outsiders that take longer to surface and be understood.

Spotting these ideas has an element of serendipity and luck to it, but there are some things we can all do to improve our chances of finding an important trend before it hits the mainstream. The techniques aren’t different from what 1990s “coolhunters” or media and marketing trendspotters do to find pop culture trends early: It’s one part looking in the right places and cultivating the right sources, and one part noticing anomalies and acting quickly.

Where new ideas come from

The places where sleeper trends begin are by definition unpredictable, so it’s important to cast a broad net among interesting discussion groups orhobbyist communities — virtual or physical — that seem like good incubators for new ideas.

The next step is to keep track of what these groups are doing by setting up streams of information about them — anything from subscribing to newsletters and discovering good blogs in that space, to attending meetups and conferences.

One of the best ways to stay informed is by building a network of “social gateways”, people who are well connected in the communities you want to watch but that are also far enough outside your usual network that you are hearing about new things. Then, when a particularly compelling idea surfaces, you will hear about it early.

Some communities are far more likely to produce winning ideas than others. In his classic work The Diffusion of Innovations, sociologist Everett Rogers describes the characteristics of so-called “early adopters” — people who are more likely to find and use new technology.

These people are usually open-minded and scientific in their mindset, and have time or money to spend on trying new things. Any group with these characteristics is a good place for technologies to germinate, which is perhaps why college campuses make great testbeds for not only spotting but trying out new products.

According to Rogers, the best groups of early adopters are extroverted and have lots of social ties, because the more connected they are, the faster new ideas spread through the group. This is why trends often start with young people in cities, rather than in sprawling suburban neighborhoods, even though the latter group may be just as willing to try out the same new things.

Highly connected groups can be either offline (densely populated cities or other clusters) or online (tight-knit online communities). A really good sign of such a highly connected group is one that has with a newly formed, makeshift online presence, like a dedicated Slack channel or fast-growing subreddit (r/nameofgrouportopic). That usually means the group is both new and highly connected.

The number of small groups where ideas could surface is too large to watch them all, so it may be preferable to look further along the path, where ideas collect — in communities that aren’t very big, but have outsized importance or influence. The places to watch aren’t so much “gatekeepers” or curators of culture, like influential editors, as they are tastemakers.

A good analogy is what happened with The Velvet Underground’s “banana album” (so called because of the pop art banana cover by Andy Warhol); while the album only sold 30,000 copies in its early years, the people who bought it were the kind of people who started bands. It ended up “influencing the influencers” despite being relatively unknown.

The tech equivalent of people-who-start-bands are programmers and developers, which is why sites like StackOverflow and Hacker News — where those groups congregate — are good places to watch for trends. If a tool or technology is especially popular with the best engineers, it’s worth watching those forums. The opinion of programmers often determines whether a technology gets built or not, or whether it comes through in startup recruiting or through open source development. It’s hard to imagine Linux being as successful as it has been — without the willingness of enthusiastic developers dedicating hours of their free time in the early years.

How to tell a trend from a fad

Once you’ve built a pipeline of promising groups and information sources, how do you decide which ones are worth paying attention to?

With breakout ideas in particular, the most important thing to notice are signs of rapid growth. If you have the data, anything over 5% weekly sustained growth is anomalous and worth paying attention to. The next best thing is to compare leading versus lagging indicators, because a big mismatch between the two is often a sign of rapid growth.

Lagging indicators are things like brand recognition, prestige, and perceived importance. Leading indicators are more intrinsic to the idea or product itself, like how much their users care about it, how much better it is than the alternatives, and the volume of positive chatter about it. An example of lagging indicators outweighing leading indicators could be a film like Avatar, which had lots of marketing spend behind it, but appears to have had relatively limited cultural impact.

When I came across early but rapidly growing trends, like bitcoin in 2011, or the Oculus Kickstarter, they felt incongruous. In both communities, users and developers were going crazy for the new technology, which seemed to be a real breakthrough. Both trends seemed way too important to be something nobody outside the niche seemed to care about. In other words, the leading indicators far outstripped the lagging ones.

The natural instinct for most people is to ignore or dismiss this feeling, but you can train yourself to pay attention to it. In these cases it’s important to act quickly, because if something’s growing really fast, it’ll be common knowledge quite soon.

How to spot an important trend

In the early days of a new idea, it’s often the case that nobody is paying for anything. One way of guessing at economic impact, is by looking at “proxy for demand” — how much people will pay for similar alternatives — and on the other side, looking at “proxy for supply” — how expensive something was to produce before.

But it’s still tricky, because it’s so hard to estimate the economic changes brought by a disruptive technology. It can be dangerous to dismiss or embrace trends solely for this reason.

Finally, while rapid exponential growth is a good sign, it’s not everything. Internet memes have huge growth as well. Most ideas that quickly attract a large audience are actually just fads, and it’s important to be able to pick out the important ones. One good sign is evidence of a “secret”, a real discovery, or some plausible reasoning why this idea couldn’t have manifested itself before now. With bitcoin, the secret is the technical breakthrough described in the bitcoin paper; where previous efforts at distributed trust and decentralized resourcing failed, it coupled bitcoin (incentives) with the blockchain protocol (distributed ledger) to solve those problems.

Of the fast growing, real trends, only a few have the potential to be really, dramatically, world-changing. Every year, there are only a few really important macro trends, and just a handful of them in computer science. It’s unlikely a tech trend will be significant unless it benefits from one of these larger shifts, though it can do so in an oblique way. For instance, ridesharing apps were an important trend, but they were only enabled more broadly and recently by the larger trend of smartphones everywhere.

How trends go mainstream

Having a breakthrough or a community of early adopters clearly isn’t enough. So how does one tell a trend — something that continues to spread — from a fad — something that flares briefly and dies? The key is it needs to spread beyond the group of early adopters to the rest of the world, and there needs to be a real pathway for this to happen.

Before the internet, this path was from metropolitan centers, through the suburbs to the rest of the population. For ideas that spread purely online, the path could be through a big aggregator site like Reddit. Another pattern is spread by institutional similarity: Facebook was able to easily spread from Harvard to clusters of students in other schools, because of the structural similarity of most universities to each other despite other differences they may have.

Another path is by latching on to a different fast-growing community. In the 1990s, one of the big marketing successes of Sprite was advertising to the hip-hop subculture before it became mainstream. This type of path is especially important in technology, as subcultures that go through massive exponential growth are common. Targeting fast-growing communities is a common strategy for startups who want to see their userbase grow. Mobile developers were a niche group in 2007, but are a large, mainstream developer community now — — yet the today’s community still retains many of the tastes and technology preferences of the old one. Coolhunters can take the same approach but in reverse, by looking at fast growing communities and seeing what they’re using.

Whatever the path from early adopters to the mainstream may be, there are some qualities that some early adopters have some qualities that help the idea spread. In the fashion world, social media marketers often target internet personalities who project an aspirational ideal, often by posting Instagram-style pictures of food, live events, etc.

In technology, the kind of person who others want to copy may fit a different profile — might be famous through open source, might be a prominent blogger — but must have the same kind of influence. The same principle can apply to groups; Python programmers could be more influential than Java programmers, for example.

A contrarian view

You can get quite far in spotting new ideas just by watching developers, people in major cities or schools, other early adopters and tastemakers… but that’s where everyone’s looking. Part of finding the right people and places to watch necessarily requires you to have an alternative but correct view of the world — to form hypotheses about what’s overrated and underrated.

Gaming has been a good example of an underrated community for the last few years — despite little prestige, it’s a surprisingly large and influential subculture, and gamers have been early adopters of ideas like livestreaming and VR.

Why coolhunt?

Beyond any acquisitive value, finding tech trends also has broader applications. These ideas have huge, and often quite sudden, effects on the world, and it’s very difficult to tell in advance what they’ll be, or which industries they’ll affect the most. And while in an ideal world, futurists could rationally deduce what the next big trend will be- , the reality is that these systems are complex and more fluid, changes compound and build on each other, and there are lots of unknown unknowns to account for.

This means coolhunting can be a surprisingly good way to catch these monumental shifts, compared to traditional market research or deductive reasoning from experts. Not bad for a technique invented by ’90s fashion marketers!