There comes a time in every young startup’s life when it must truly confront the question: “Does this thing have legs?” You, your investors, and your team all believe so – but whether someone will actually pay you to solve their problem with your product is untested. Right now, you have a hypothesis, and the best way to test that hypothesis is through a pilot with a real, potential customer. 

Unfortunately, this world of pilots, POCs, and designer partners might as well be a foreign language to most technical founders. Why you should run pilots, how to find the right customers, and how to make pilots successful are not simple or easy questions. 

At Amplify, my job (and my passion) is to help these entrepreneurs learn the language and land their first few customers. Over the dozens of companies I’ve worked with – not to mention my 20+ year technical sales and engineering career – I’ve learned a bit about what it takes to run these pilots successfully. The goal of this guide is to codify that information and help you grow faster and stronger via well-targeted, efficient design partnerships.

This thing is organized into 4 main sections:

1. The What: what a pilot actually is, how it differs from a demo or a product, and how it works.
2. The When: when you’re ready to run a pilot, and how you can determine that.
3. The Who: finding your initial target customers and refining that audience over time.
4. The How: building a good pitch, running successful pilots, and turning them into paid customers.

Speaking of companies I’ve worked with, examples from two of them will be peppered throughout this guide to keep things grounded. Two Amplify founders – Barry McCardel, Co-founder & CEO of Hex, and Richie Artoul, Co-Founder of WarpStream – have been where you are today. You’ll find their advice (and commiserations) throughout, thanks to them generously sharing their time and minds.

The Why: why should you run a pilot?

Before we get into the mechanics of pilots and POCs, the obvious question to start with is: why do this at all? Why run a bunch of labor-intensive customer engagements when you could just launch your product into the wild and let people sign up?

The answer is feedback. As an early-stage founder, your singular product goal should be high-quality, actionable feedback, and pilots are the best way to get it.

One of the biggest blind spots that technical founders have is the assumption that the context for their product is obvious, and that once they get it out there, people will immediately understand what makes it special (and adopt it). This is a well-documented phenomenon and I call it the “if you build it, they will come” fallacy. In an era of a million developer tools, it’s a dangerous, company-killing misconception.

What this misconception does is plant a seed in a founder’s mind that launching widely is the best way forward, since smart people who see the launch will adopt the product. It implies that breadth is preferable to depth because the product is inevitable. 

But the opposite is true: because at this stage the story of your product is more important than your product itself, the ability to tell that story in a controlled environment to a highly targeted audience is the best way to determine whether you have something here. You’re looking for quality, targeted feedback on your core offering; you will get it through intensive pilot engagements, not from self-serve users poking around. 

You can think of the number one goal for your pilot as this: to learn something. 

To get more concrete, there are some specific questions you should be looking to answer with the feedback you get from your future pilots.

1. Does your product add value?

At the core of whether you’ll exist in 5 years is whether you actually add value to a potential customer’s business life. Even if your product isn’t in the state you would want to launch it in (more on that to come), a pilot should be able to tell you if you’re headed in the right direction. The last thing you want is to get too far down the road, where your product is mostly set in stone, and learn that it just isn’t going to add the value you hoped for.

Just because you had a problem at a previous job that your product solves doesn’t mean that’s going to translate to a customer, and definitely doesn’t mean that’s going to translate to a customer paying you. Until you’ve validated that your product adds value in a customer setting, all you have is a hypothesis. 

2. Do you need to improve your product or your pitch?

These early potential customers are your design partners: your product isn’t supposed to be ready; it should be rough around the edges, because that early feedback will help you shape and polish it. 

That feedback could take a few forms:

• Is your Ideal Customer Profile (ICP) correct? (More on this below)
• Does the product do exactly what you thought it would?
• Are the use cases you planned to address slightly different from what you imagined?
• What value are you actually able to add?

Yes, ideally, your pilot converts to a paid annual contract (more on that later). But the goal – the real goal – at this point is feedback.

Consider WarpStream – there were a million directions they could have gone in to start. But a very, very specific combination of design choices ended up making the difference for them;  without those early conversations with potential customers they wouldn’t have gotten this feedback (and the confidence it gave them).

1. Kafka protocol compatibility
2. Statelessness
3. BYOC architecture

The combination of all three was critical to its success: if any one of them had been missing, the business may not have made it.

It’s so easy to go too deep or far in a direction that’s missing something – making it risky to build for a year or more without showing your product to anyone or testing market demand. 

[fs-toc-omit]But we’re a product-led growth company!

Look, I get it. Your goal is to get users, and the product-led growth (PLG) approach is tempting because it’s deceptively easy. Instead of laboring over a POC and seeking out ICPs, why not spin up a sign-up form and let potential users come to you? Aren’t you leaving users (and money) on the table by creating friction? Well, yes — but early on, your company’s growth depends on friction. Because you don’t actually just want any users — you want useful users who you can collaborate with so that you can learn if your product and your story are worth anything (and how much). A founder’s primary goal before they have product-market fit is feedback. 

At this point, friction is not the enemy. Frictionless signups mean frictionless exits. People can come and go as they please and say nothing to you — and they will likely say literally nothing to you, nor will they respond after they have already left to tell you why. Instead, you need high-fidelity, carefully selected design partners who deeply experience the pain point your product solves and can tell you where it hurts. 

Feedback is worth much more than revenue or signups early on. Building meaningful relationships with these partners is the key to getting and maximizing that feedback. If you do PLG too early, you’ll never get the insight you need to determine whether you really have something.

The What: What exactly is a proof of concept?

Now that you know why you need a POC, it’s worth defining the boundaries of what a POC really is.

A proof of concept (POC) is just that — a miniaturized, boiled-down version of your product that proves that it actually achieves whatever you said it would. In a perfect world, your POC addresses whatever production problem you’ve set out to solve, just at a smaller scale and with less polish than your ideal final product. Your POC will typically lack many of the bells and whistles you hope to include, but it should, at its core, solve a real problem the customer actually has.

What is not a POC

In my years working with early-stage technical founders, I’ve come across several misconceptions about what a POC really is:

[fs-toc-omit]A POC is not a demo 

If you’ve started customer discovery or raised money, you probably already have a demo. A demo typically uses a fake data set (ideally Star Wars-themed). Generally, it shows prospective investors or customers how the product works and the types of problems it solves.

A POC is not a demo. It’s a real, working minimal version of your product that (at its core) legitimately solves a problem your potential customer actually has. Just because you have a demo does not mean it’s ready to bring to a customer. On the flip side…

[fs-toc-omit]A POC is not your entire product

POCs are less developed than a full product. Completely production-ready products are too complex and risky to build before you have meaningful feedback, and tend to waste time and effort on both sides. 

Instead, a POC is a smaller step in between: more ‘baked’ than a demo, but less onerous to build than your full-blown product. Something compact that works with a boiled-down, smaller subset of requirements that’s representative of the customer problem.

The goal of WarpStream’s early pilots, for example, was always to get their POC working with real data in a production-like environment. They didn’t try to prove themselves in actual production because getting there would take months of integration work (if they could even get approval). Instead they simulated production well enough to clearly show that the product would hold under real production conditions.

This approach will help you iterate faster and benefit the customer by getting to a bare-minimum, workable solution faster, preserving their time and effort. If they need to see that your POC can scale, that’s something you can agree on contractually.

The Technology Readiness Levels (TRLs) framework developed by NASA in the 1970s is one way to gauge your product’s stage, but don’t expect customers to be aware of or care about it (unless you’re working with a government agency).  

[fs-toc-omit]POC vs pilot

It’s common to use these terms interchangeably, but for consistency, we’ll use “pilot” throughout to refer to the process of trialing your POC with a design partner. 

To quote Alchemist Accelerator:

“A POC proves that a product could work, while a pilot proves that a product does work.”

The When: when are you ready to go to market?

How do you know if your product is ready for a pilot?

For many product-centric founders, there’s no good answer here. You’re never going to feel ready. Building an early-stage tech startup is like putting wheels on the car once you’re already rolling. Your product won’t be complete or shiny. 

Barry and the Hex team were already in the habit of practicing Commitment Engineering from their backgrounds at Palantir: the idea being that you build both for and with the customer — never in isolation. The prospect of starting those feedback loops with early users may be intimidating, but for many products there’s no possible better way to get the feedback you need from the people you need it now. 

So with that in mind: 
If your product is useful in some way, if you believe it’s ready to add some value, you’re probably ready for a design partner to give feedback on your direction. That is true even if:

• You haven’t realized the full vision yet
• It doesn’t look polished

Early adopters have patience for bugs and missing features; don’t be afraid of people seeing it unfinished. You are far more likely to prevent good things from getting out the door because your bar for quality is too high. 

When you see rough edges, you say, “not ready.” When I see rough edges, I say “Beta!” (Hey, Gmail was in Beta for half a decade.) Of course, if your product is in a state that is so buggy and cumbersome that even if it does add value, it’s a very bad user experience — don’t inflict it on anyone yet. You’re not going to get actionable feedback. 

How do you know when your company is ready to run a pilot?

Running an early pilot will take concerted effort beyond getting the POC ready — working with design partners, supporting them through onboarding, and gathering feedback are all tasks you may not feel you have the bandwidth for. 

It’s very easy then to say, “We’re not ready, because we haven’t hired enough engineers, or we don’t have cycles, or we have too many competing priorities, we need to get these features shipped, let me wait till I get past these next two sprints...”

All of this is probably true, but you still need to do it. Being understaffed and overburdened is the nature of running an early-stage startup, and unfortunately, it’s not an excuse not to talk to customers. You have to carve out a certain percentage of your time to do it. All of this other work you’re doing is going to be a complete waste of time if you don’t build the right thing for the right people in the right market; finding and completing POCs is arguably the only thing that matters right now.

[fs-toc-omit]Should we bring in an advisor?

When you’re feeling out of your depth, it’s natural to want to offload the task to someone more qualified and experienced. But in this case, no one is more qualified to run your pilot than you. There are plenty of people who would love to take your money for this, and far fewer who are actually good at it. Even great consultants and sales advisors don’t know your company and product as well as you do. Lean on your VC team for sure, take up any offers of help you can get, but don’t expect to outsource the work of finding design partners and working with them through your pilot.

The Who: ICPs and how you figure yours out

So, you’re as ready as you’ll ever be to run an early pilot. It’s time to find some customers. Who should you go after?

You should start with an ICP, or Ideal Customer Profile. It describes (via a few methods) who you think your product is right for, and who you’ll try to get it in front of. 

Two important points I learned about ICPs over the years: the best companies pick a highly specific ICP. They are open-minded and experimental towards evolving the definition over time.

1. ICPs must be highly specific. You’re trying to nail down a very, very particular persona. Why? Because you’re looking for directed, helpful feedback. That’s much harder to get if you’re talking to 18 different types of people. Your pilot is an experiment, and to make it as scientific as possible, you don’t want too many variables. 

2. An ICP is a living thing and will evolve as you conduct your pilot and learn more. Initially, your ICP will be your best guess as to who you built your product for. When you decided to start a company, you probably had a specific type of person in mind (probably you) to solve a problem you experienced in your previous job. This is a great place to start. 

Let us dive deeper into both of these.

How to define a highly specific ICP

A highly specific ICP will help you get to higher-quality feedback faster, and also help you make a more specific and targeted pitch instead of a broad one. 

So, how do you define this specific ICP? Most start with identifying a job title, maybe a tech stack, and a few companies you’re interested in targeting, and stopping there. It’s much more complex than that. Instead, factors like: 

• Number of employees
• How much funding they’ve raised
• The size of the team your ideal person is on
• What other stakeholders are making buying decisions besides your ICP (and whether their boss needs to approve)

Combine to create the specificity and experimentability you need for this to work.

A specific ICP is:

“Backend engineering team leads at US companies with $50M – $1B in annual revenue and mobile fleets of 1M+ devices”

A specific ICP is not:

“Developers at enterprise companies”

Which factors are essential to defining your ICP is extremely context dependent. One way to assess whether your ICP is specific enough is to see how much it narrows your list of potential design partners. If you’re looking at 200-300 companies, it’s probably right. If you’re trying to choose from a list of 15,000, you probably need to get more specific.

Note: How specific your ICP needs to be will depend on how specific the problem you’re solving is. If you’re building for a very specific use case (say, insured cloud commitments for RIs, it will be obvious who you need to target. If your product has multiple potential ICPs (anyone wanting to build an AI agent for their business) you can be more flexible, but make sure you narrow it down sufficiently for your pilot.

How to treat your ICP like a living, evolving thing

No matter how specific you get, you cannot forget that your first ICP is a thesis, and you’re not going to know if it’s right until you run with it. Some things, like the size of the company, will always be guesses until you gather more data. There are limits to your personal experience when it comes to the logistics of selling software. Think of this as an iterative experiment. It may take you several rounds to find the right niche for your product. 

Similarly, you may already know that your ICP today isn’t the ICP you’ll have in a few years. You may set out thinking your ICP is a very large enterprise, and that may be true in the long run, but that doesn’t mean you’re ready for that today (even Databricks started out as effectively just a hosted Spark service).

Instead, start with smaller, mid-market companies with higher risk tolerances. As a young company, your risk profile is high, there are more hoops to jump through, and depending on how critical a role your product plays, you might be too big of a bet for them to make.

In the early days at Hex, they initially spent a lot of time talking to a team at Salesforce. In retrospect, this was misguided, because there was no chance an enterprise that large was going to roll out a year-old data science prototype to their whole org; but they had the problem Hex was solving and were saying all the right things. Barry recalled that he just wasn’t willing to say no to a big logo at this point in time (and logically, who could blame him, right?). Hindsight 20:20, it would have been more helpful to have said no."

You need a smaller, more risk-tolerant group to get started with.

“There will be other people who have the right problem now, who accept the risk because the problem is really painful, or they see your product as offering some competitive advantage. Those are the people you want to start with.” – Richie, co-founder, WarpStream 

[fs-toc-omit]How many design partners do you need?

Generally, at least two, but 3-5 design partners is a good place to start. You might find that you start with three, discover along the way that a couple aren’t actually a good fit or you need to adjust your ICP, or the pain point you’re solving isn’t actually as desperate as you thought, and you start over with another set. 

Expect to go through several revisions of what a good design partner is to you before you’re really off to the races.

How to find and contact your ICPs

Once you’ve got your ICP nailed down, you need to find real people who match it. Obviously, warm introductions are the best way to get in touch with early design partners. You’re going to have so many things wrong early on: the pitch, value proposition, positioning, etc. You need to find someone who isn’t going to straight-up ignore you if your outreach is bad, which people will do if you’re emailing them out of the blue.

There’s a fine balance to strike here — while you want a design partner who’s predisposed to working with you, you don’t want someone friendly doing it as a complete favor. If they don’t genuinely have the problem your product solves (or aren’t willing to pay for a solution) they’re not going to give you the kind of feedback you need.

When I work with founders, we use a hodgepodge of different tactics to find this initial group of leads:

• Going through your LinkedIn and seeing any mutual connections
• Asking your investors (if you have them) for introductions
• Combing through anyone you’ve already spoken to
• Use Gem or some type of LinkedIn automation
• Working with a firm like TaskMinions to create bespoke searches

I’ve generally found that around 200 is the right number to start with if you want to end up with 15-20 conversations. 

Others may disagree, but cold outreach should be an absolute last resort. Unless you have solid evidence that your pitch and story are strong, I just don’t see good results with it. Even if you DO have a solid pitch and story, cold outreach is among the hardest things in all of GTM to do well at any stage of a company. Conversion percentages are usually in the low single digits, even when done properly, to give you a more concrete idea.

[fs-toc-omit]Send painstakingly personalized messages

Even if most of your outreach is via warm connections, acceptance is not guaranteed. The warm connection just gives you the luxury of a likely response (and that response can still be “not interested”). Messages cannot be templated and standardized. If you’re not going to take the time to handcraft a message, why would this person take the time to consider your pitch? You want the person on the receiving end to feel seen and understood. That’s not going to come across with “Hey, I saw you work at company X.”

Real personalization might mean pointing out a recent blog post the prospect wrote and what you like about it, commiserating over a mutual boss at an old company that everyone hated, noting a recent tweet about how the prospect has been building a sauna in their backyard, and wow coincidence, you happen to be building yourself too and should mention that a good foundation is essential because the wood is going to warp…that level of personalization.

[fs-toc-omit]You are not spamming people

Technical founders shy away from this kind of outreach because it can feel spammy to them. You wouldn’t want a random person to email you trying to sell you something, so why would you do it to someone else?

But if you’re doing this right, you’re not spamming. First, the goal is to go through 100% warm connections — you’re emailing people you already know, asking for introductions. And second, you’re coming with a highly targeted message: asking if someone has a problem and, if so, whether I can help them solve it. 

If you’re doing email right, it shouldn't feel spammy at all. Challenge yourself to write a note that you, yourself, wouldn’t mind receiving. They do exist and can be done properly; it’s just that most people don’t take the time and care to do so. 

How do you know if your ICP is wrong?

Assuming you got everything else right (a compelling story, a persuasive email), if you send out your carefully worded pitches and hear total crickets, you’re probably going after the wrong ICP. 

It’s unlikely that your ICP is wildly wrong — you had experience or a hunch that led you down this path. If you’re solving a common or sticky problem, you probably just need to make a small tweak, like the size of the company, or approaching engineering managers rather than engineers. I’ve seen countless examples in my outreach work where a small change made all the difference. 

That being said, if you revise your ICP multiple times and are still getting crickets, you might have chosen a problem that’s not really a problem. 

“If people aren’t willing to spend 30 minutes to take a demo, that’s a non-monetary signal that you’re not on the right track.” – Barry, Hex 

This sounds catastrophic, but it is actually a good thing to learn now. This is why it’s essential to be somewhat scientific about the whole thing — if you have limited the variables in each case, that makes it easier to hone in on what the issue might be. Your ICP can actually help to expose when you’re on the wrong track, and you want to find this out sooner rather than later.

The How: how to build your pitch

A good product will get you nowhere if you can’t pitch it right. So many founders work hard to build a great early product, only to fail to accurately explain how and why it solves a critical problem for a potential customer — and no dice on the POC.

Ultimately, the goal of your pitch is to get to the next meeting. If you’re sending a request for an intro email, your goal is to get to a meeting. If you’re pitching in your first meeting, the goal is to get to your next meeting. Everything else your story might do (appeal to the customer’s desire to save time or money, for example) is in service of getting to that next meeting. 

[fs-toc-omit]A pitch is a story, a promise

A pitch is fundamentally a story. You want your story to get the customer excited about the outcomes you’re selling so they keep talking to you. At this point, it’s irrelevant whether they understand anything about your product or your company — they just need to be excited enough about the promised land to consider spending more of their time learning about what you’re doing.

Your story also needs to be pitched at a level that excites engineers: it should deliver a practical, specific outcome that improves their day-to-day, while also being clear about how it impacts the business at a higher level. 

Think:

WarpStream decouples Kafka’s compute and storage layers by using object storage as the backend, eliminating the need to manage local disk and replica synchronization.

vs.

WarpStream cuts your Kafka bill and maintenance efforts in half via a diskless architecture built entirely on object storage.

You want somewhere in between too in-the-weeds and up-in-the-clouds. As an engineer, this is fairly intuitive. If someone tells you they will dramatically cut your cloud costs, you are, by default, skeptical until they explain how, at least at a high level. Similarly, if someone just pitches a random feature or piece of their architecture to you, you might think it’s cool, but that’s not going to lead to a purchase decision. A good pitch straddles this line.

The four elements of a good pitch

There are at least four elements to a good POC pitch:

1. Selling yourself and your personal experience
2. “There’s got to be a better way!”
3. Quantified, numerical outcomes
4. A peek under the technical hood

Make sure your pitch has at least a little bit of each of these, and you’ll be golden.

[fs-toc-omit]Personal experience

Part of getting people excited about your product is getting them excited about you. You know you are uniquely qualified to solve this problem. To build trust with potential customers, you want to communicate that you deeply, painfully understand their frustrations. You didn’t actually want to found this company, but you were compelled to! The outcome, the vision that you’re selling, is one you were able to conceive because you have a kinship with them. In a sense, your pilot customers aren’t just buying your product: they’re buying you.

For most technical founders, you’ve probably sat in the seat of the person you’re trying to win over — you’ve faced the challenges plaguing them. The only time you should spend talking about yourself here is to explain that you were working as an engineer (replace as needed) at company X, and were experiencing challenges trying to do Y. Your motivation for building this product stems from your own painful experience, which makes you the best person to provide the solution. 

[fs-toc-omit]There’s got to be a better way

A massive part of your pitch is selling the problem. Your first step is getting the person on the other end to agree with you that whatever the problem you’re solving is, indeed, a significant problem that they do, indeed, have. A great way to sell the problem is to paint a picture of ‘old world vs new world’. Set the scene by describing the problem and all the things you tried to do to solve it. All of those approaches had drawbacks, and nothing worked quite the right way. 

Successful products that truly change the game typically arise out of situations where someone takes a step back and asks, “What if there was a better way to solve this problem? What if someone actually addressed this need from the beginning?” Some phrasing that many companies I work with use: what if we reimagined [x] from the ground up?

Instead of making do with cobbling something together based on assumptions that were made at the outset (that ended up becoming incorrect or obsolete), there was probably a smart way to build the thing you need. No one ever built any of the existing facilities with the assumption that this workflow would ever exist.

For example: The most popular databases today — Postgres and MySQL — have (at least) one major thing in common: they were built in a dramatically different hardware reality, when hard drives and memory were scarce and expensive. CedarDB is the result of asking, “What would these databases look like if you created them today?” They’ve created an exceptionally fast database by challenging those assumptions, and designing every piece of the system from scratch to take modern hardware and software into account. 

[fs-toc-omit]Quantifying outcomes with numbers

People buy outcomes, not products as such. Your pitch should sell an outcome, and a specific one. Promises like making the customer’s team “more efficient” or “faster” will only land if you quantify them. 

Now, it’s tricky to quantify outcomes because there are so many variables that influence them — they’re subjective and nebulous. As a technical founder, you will probably feel uncomfortable speculating on a dollar number or how many engineer hours you’ll save. 

It is uncomfortable, and I’m sorry, but you do have to do it. You need to build your best estimate of the value case for an average organization, knowing that it’s likely not accurate for everybody. 

Some ways to make your claims less vague:

• Be specific about what you’re making faster or more efficiently. Builds, queries, tests, and so on are more tangible than “code”.
• In the absence of real customer examples, speak to your own experience. 

WarpStream demonstrated measurable outcomes before they had customer references by sharing the interzone networking costs of their own cloud account, showing an average of <$15/day, compared to $641 per day for an equivalent workload run using a Kafka cluster: 

Some standard measures of value among technical tools:

• Cost saved (usually in an infrastructure context — see above)
• Engineering time saved
• Conversion increased (usually in a performance context)
• Product brought to market faster
• Downtime prevented

You’re not writing a contract, you’re helping potential partners to visualize the outcome. It doesn’t have to be precise, but you have to give them something grounded in a realistic use case. Your potential customers need to be able to envision how big of an improvement they can expect.

[fs-toc-omit]A peek under the technical hood

Maybe the most common mistake technical founders make is making their pitch too technical; more on this later. But there are exceptions. You want to be technical enough to prove some sense of legitimacy and domain expertise. For example, if the technical details in your implementation of a solution are the thing that enables outcomes that were previously impossible to achieve, you probably want to mention them. 

If you are solving the problem in a fundamentally different way (like Modal’s novel approach to creating a place to run AI workloads), include some timely examples of how the product is built that enable this. It’s less about specific functionality, though, and more about the inspiration behind your idea, or as Richie calls it, “not just technicalities for technicalities’ sake.”

The balance here is this: people who are really early adopters won’t accept a “magic box”. If your sales calls are with fairly technical people, don’t be surprised if you get a lot of questions about how the magic box works. Especially with an infrastructure product, people still want a mental concept of how it works, even just so they can reason about performance or tuning. But if your entire pitch is about how the box works, you’ve already lost. 

Things most founders get wrong with their story

Pitching and storytelling are not easy, and many technical founders struggle with it. Even if you’ve already raised a round from investors, that kind of pitching is very different from the kind of story you’d want to tell your customers. Here are a few areas where I see founders go wrong:

[fs-toc-omit]Talking too much about the product and not enough about the problem and outcome

This one will really ruffle some feathers. If you’re truly selling a vision, ideally, you should not actually talk about the product at all in the first pitch. Don’t even mention a single thing about it. At this stage, the product itself doesn’t matter (if you engage a contractor to build your house, you’re probably not going to ask them which tools they’ll use). It’s the outcome that matters.

You want to align with the customer on desired outcomes, because if they aren’t interested in the outcomes that you’re selling, but are interested in your product nonetheless, you can end up putting a lot of work into this pilot only to realize that they’re not good customers for you. It’s not unusual for people to want to kick the tires and try cool new technology, but pursuing a pilot with them will just waste your time if they’re not going to convert.

[fs-toc-omit]Doing too much in the first meeting

Your first meeting could also be your only meeting. It’s tempting to use your one shot to present everything you think is awesome about you, your company, and your product. Unfortunately, this will more than likely just overwhelm them. If a potential partner walks away from the meeting confused, they’re quickly going to forget about you. 

Remember that the goal is to get people excited enough to take a second meeting. Listing a bunch of features or going in-depth into your tech stack choices won’t do that. Think movie trailer, not movie. A good trailer has you on the edge of your seat, so you’ll want to go see the whole film. 

[fs-toc-omit]Getting too technical

Which brings me to my next point: no matter how technical your product is, your pitch should not be. This isn’t Hacker News, nor are you selling to professors. If your story is too dense, if your deck looks like an academic paper with a bunch of graphs, you’re going to lose people. Getting into the weeds requires everyone to have a lot more context up front, and for a 30-minute call, they probably haven’t thought about you for more than 10 seconds. 

If you think your product is an exception to this rule, I'm going to say there’s a 90% chance you’re wrong about that. There may be cases where you need to be deeply technical in the first pitch, but I have yet to encounter one (and I’ve seen hundreds of pitches).

If you’ve simplified your pitch to focus purely on outcomes and someone actually asks you for technical details, that is a great sign. This is a potential partner who is very engaged and interested in your product! But they likely won’t get there if you overwhelm them with too much information first.

[fs-toc-omit]Slides are your friends!

Slide decks get a bad rap, and that’s usually because people tend to read off of them, which is boring for everyone involved. Done right, slides are actually a very important visual aid to your narrative. You’re selling a complex vision, and slides help to fill in the gaps that words can’t convey alone. When done correctly, that combination of words and visuals helps you reach the coveted next meeting. 

For Hex, their sales deck was a tool to qualify or disqualify potential design partners: a couple of slides about the problem they’re solving guided the conversation before a demo. You want to present potential design partners with the problem and let them validate whether it is a pain point for them.

The How: how to run a successful pilot

If you’re at the stage where someone has agreed to be your design partner and is excited about doing a pilot with you, congratulations! It’s not easy getting here. Now, the real hard work begins.

Before you send them off with the keys to take your POC for a test drive, you need to make a plan that details exactly how the pilot will proceed. In my experience, a pilot without a plan will fail nine times out of ten.

What goes into a pilot plan? There are two main things you want to agree on up front:

1. Success criteria

The first thing you want to lock down in a pilot plan is the success criteria: what outcome the customer wants to achieve, and how you will know when that outcome is achieved. Aligning on these at the beginning gives you the best shot at landing an actual customer at the end. Then you can say, “Hey, with this pilot we were able to successfully [reduce your TTR to under 24 hours], just like we said we would.”

If you have a surface-level discussion and agree to some hand-wavy success criteria, you risk missing the mark. Success criteria must be tangible and specific. Think of yourself as an investigative journalist, really getting to the heart of what your partner is looking for.  

These conversations can be a bit awkward to have upfront, because it’s not always human nature to agree on a numerical outcome for a relationship. But you must do it! Failing to have these hard conversations up front can result in a lot of wasted time and effort. 

Hex had a potential design partner they engaged with too early; they agreed to sign a contract and run a pilot in a specific environment, but the path to get out of that environment and into production would be a multi-year project. Hex ended up walking away from the pilot because that wasn’t a useful time horizon for them, and they weren’t convinced that the time and energy they would put in would actually net a new customer. Agree on your goals up front!

2. Scope

There are a million things you can do to help your customer achieve the agreed-upon goal (including secretly writing code for them), but what is your product going to do? What’s the minimally viable version of your product that will meet those criteria and be realistically achievable on a reasonable timeline?

You’re not trying to simulate production here. Failing to scope down sufficiently usually means you have to wait far too long for actionable feedback and risks the customer losing interest along the way. This is why you need to scope your pilot tightly: flesh out how you intend to integrate with their systems and other dependencies, divide those responsibilities, and agree on prerequisites and deadlines.

If this scoping doesn’t happen at the planning stage, you might find yourself in a position like the team at WarpStream, who had one pilot that dragged on for nine months, with each month revealing some new, as-of-yet undiscussed criterion they wanted to test. 

“Sometimes you do have to set rails on things — otherwise it can get to the point where you’re essentially a professional services arm.” – Richie, WarpStream

[fs-toc-omit]Failure modes

• Not getting genuine alignment: Success criteria and scope have to be genuinely mutually agreed upon. If you’re just uniquely gifted at persuading people to say yes to things, you likely won’t convert them, regardless of whether you fulfilled the terms of the agreement. 
• Not being ambitious enough: If the scope is too small, and your design partner will only commit to working with a dummy data set, you’re going to have a hard time graduating to a real production use case.
• Falling into the “innovation theatre” trap: Corporate innovation departments may experiment with startups, but don’t necessarily have the authority to roll out new solutions beyond their immediate team. Make sure you have the right stakeholders on board before you get started. 

With success criteria and scope agreed, you can move on to other practical matters:

Length of pilot

2-3 months is the norm: long enough for your design partners to give the product a fair shake and start seeing results; short enough to get meaningful feedback promptly. 

If meaningful results don’t seem attainable in 2-3 months, you probably have a scoping issue. Gently guide the conversation back to scoping the POC down to something that can conceivably happen in that timeframe. The aim is to get to the point as quickly as possible where the customer sees the impact and is eager to see a bigger, more fleshed-out solution. If they’re nervous about production use, you can include assurances about reliability or uptime in the contract (keep this in your back pocket). 

Pilots of 6/9/12 months can happen in some cases, but I’d encourage you to reconsider whether a customer needing a pilot that long is a good candidate. You can’t afford to spend upwards of six months evaluating just to see if your product has legs.

Timeline

With a span of time in mind, now you want to outline what should happen at various stages throughout the pilot. If you’re going to stick to your timeline, you will need approvals, integrations, and so on set up by a certain date, onboarding to begin, and evaluation to kick off so that you can conclude with a clear view of whether success criteria were met.  

One of the significant challenges of running pilots is delivering value throughout the process. If you only have a pot of gold at the end of the rainbow, it’s all too easy for the customer’s priorities to shift, they lose interest, or move on before proving your concept is worth their time. 

As such, ideally, you want to build small wins into your weekly timeline to generate a steady drip of excitement for the customer. That could be their successful data import, first user onboarded, first integration…you get the idea. If every week is too ambitious, aim to give the customer something to stay motivated every other week. You might have to get creative. 

You can use my pilot plan template as a starting point.

Make sure you’re ready to support your design partners

“No one is going to be successful without your help.” – Richie, WarpStream

Your design partners will rarely just “succeed” by putting your product in front of them. You can’t just agree on the plan, set them up with the product, and disappear. In fact, if this can happen, you probably spent too much time building your product before going out to the market.

Instead, you need to figure out what’s actively blocking your customers, read between the lines, and try every day (within reason) to help them succeed with the product. I encourage founders that I work with to communicate with their design partners as much as possible:

• Do whatever you can to get your design partner in a shared Slack channel with you.
• Respond to anything they say as immediately as humanly possible.
• Take every problem they raise seriously, no matter how minor.

Hands-on support is especially important if your product is complex or requires extensive integration. If there are many different components, packages, and libraries your customer needs to wrangle, it is your fault if they get stuck or do something wrong. 

An area where I specifically see technical founders fail is focusing too much on product fixes and not enough on hand-holding. A classic example:

Design partner: We’re having trouble connecting our database to the app.

Founder: We’re shipping a new onboarding experience next week that should help!

Coming from an engineering background, sometimes everything can look like a product problem. But with design partners, it’s not always the case. They just need you to walk through implementation with them, fix something manually, and generally do things that aren’t scalable.

Should you charge for a pilot?

This is not a popular take, but by default, pilots should be free. It is not your goal to make money off this (right now) — your goal is to get feedback (landing a paying customer after the pilot is a nice bonus). 

“The point of a pilot early on is not the money, it’s not the logo. It’s to validate that you’re actually solving the problem…If you’re right about the product, the space, or the category, whatever your early design partners represent in revenue is just a rounding error.” – Barry, Hex

If you think of the cost of running a pilot as whatever time you invest in helping your design partner get up and running with your product, it’s largely offset by the value you gain from validating that you’re on the right track.

With that in mind, there’s really no good reason to add the friction of money to the process. Charging for the pilot means your champion has to go through a procurement cycle, vendor review, get approvals — a bunch of things that slow you down and give people reasons to question why they’re doing this in the first place. 

Also, let’s be clear: nothing is free. Your design partner will pay with their time. Don’t underestimate the investment or the risk people are taking when they agree to a pilot with you. People are subject to so much marketing and spam that it’s not unusual for them to be skeptical. Just because a pilot is free, you will still have to sell it to potential customers — don’t make that task harder by adding another thing they might object to. You want to make it easier to say yes, start the pilot sooner, and then hopefully close a deal.

[fs-toc-omit]If we don’t charge them, how do we know they’ll stay?

If you’re concerned that the customer can just walk away at any time because they won’t have skin in the game, you have a problem. The pain point that you’re solving should be so critical that people will stay engaged even without money on the line. If not, charging for the pilot is a crutch to force commitment and will confound your ability to actually learn from it. 

Think of it as a test: If you can convince a design partner to show up and be engaged without a financial investment, that’s a great signal that your bet on the problem is solid.  

[fs-toc-omit]Exceptions

• In some cases, the cost of running a pilot might be extremely high due to your own COGS. If the customer understands that and the value proposition is right, they might still be willing to pay for the pilot because the upside is worth it. 
• If you’re facing a lot of demand (congrats), charging for a pilot could help weed out tire-kickers so you can focus on serious partners.  
• Right now, your only goal is to maximize revenue (a word of caution: charging for a pilot can also send the wrong impression that you’re desperate for cash. Don’t be surprised if people call you on it.)

Qualifying design partners

In a world where you are trying very hard to land your first few POCs, it’s easy to forget that not all of them are good for you. Some design partners are just bad fits: they don’t have the problem, they don’t have the people, they don’t have the time. And it can be really hard to walk away from a potential design partner, especially if you aren’t getting many meetings. The temptation is to jump through hoops to land them, but this is really down to a scarcity mindset:

“Not everyone has to be your customer early on, and the cost of letting someone poorly qualified go is so much less than the cost of getting yourself turned around and wrapped around the axle.” – Barry, Hex

Instead, you should approach things with an abundance mindset. There are plenty of potential design partners in the sea; you want to invest your time and energy in those who are a really good fit for your product. And if there aren’t enough in the sea, is this a sea you really want to be in?

With a robust pilot plan, you will likely convert a design partner into a paying customer because everyone is committed and on the same page. Of course, any number of external factors can prevent this happy conclusion, which we’ll get into in a moment. These are beyond your control and you shouldn’t feel bad if a pilot doesn’t convert for one of these reasons. 

But your success criteria and test plan should be so tight that if the customer does what is outlined, the pilot should be successful, period. If you’re not convinced that’s the case, you might need to revisit the plan and drill down even more on the criteria and scope, or reconsider whether this is the right design partner for you.

The How: how do you know your pilot worked?

Though the goal of your pilots is feedback, the end state you should aim for is converting them into annual contracts. Aiming to convert will give you the best chance of learning something practical, which is the actual real goal (sort of a “shoot for the moon and you’ll land among the stars” situation). Let’s look at a few scenarios:‍

[fs-toc-omit]Best case: Your design partner becomes a customer‍

If you convert your design partner into a paying customer, the pilot was successful. They accepted the criteria you outlined in your plan, you hit the criteria, and if you were right that the problem is important to them. If you actually solve that problem, they’re going to want to pay you to continue using your product. 

If you did a good job building your pilot plan, conversion is easy because you negotiated everything up front, and all you had to do was deliver. Now you’re off to the races.

[fs-toc-omit]Neutral cases: You failed, but you learned something

Even if your pilot doesn’t convert, it doesn’t necessarily mean the entire exercise failed. A few examples:

[fs-toc-omit]You picked the wrong design partner

Maybe they were just doing you a favor and weren’t really committed to the problem you’re solving, or you need to revisit and tweak your ICP. 

If you go through this loop multiple times, you might not have picked a problem that’s sticky enough, or you might not have viability as a company (some solutions work better as features of existing products than as standalone products). These are tough lessons to learn, but better to know now. 

[fs-toc-omit]The customer needs more integrations

This is a good problem to have! It just becomes a question of whether you want to build those connections and how to prioritize them.

[fs-toc-omit]The customer isn’t aligned with your vision 

You could close them, but shouldn’t, because you’ve learned enough about them and their needs to know they won’t be a good long-term fit for your offer. If your design partner isn’t really engaging with your core features and requests excessive customization instead, it’s probably better to part ways. There is a fine balance between accommodating them without extending so much that you end up outside of your vision.

“It’s natural to think the point of pilots is winning business. It’s not really winning business; it’s learning and getting feedback. Contorting yourself to win or making unkeepable promises isn’t actually a sign of business momentum.” – Barry, Hex

[fs-toc-omit]How much should you deviate from your roadmap?

Your customers will ask for things you weren’t expecting, and might not align with your roadmap. Should you build them?

I recommend reserving 10-20% of cycles for unexpected events. That could mean prioritizing items that were further down your roadmap, or it could be as extreme as adding some functionality that no one else will ever use, but is critical to closing an early customer. 

A lot of VCs will advise you never to change your roadmap for a customer, because that’s not how you build repeat business — and they’re right. This is great advice for a series A or series B company, but awful for a seed-stage company or anyone going from zero to one. If you are 85% of the way to landing your ideal, long-term customer and they’re asking for the final 15%, it is expected and encouraged to do some unnatural things to get them on board. 

Note: This is only for pilots! Once you start onboarding customers and have a solid market for your product, you should do less outside your roadmap. Realistically, you will need to reserve some bandwidth for customer requests that may not be repeatable. The goal is to decrease the cycles you spend this way over time, but don’t be in too much of a hurry at the expense of closing your first customers. 

[fs-toc-omit]Your product wasn’t ready 

The pilot may have gone well initially, but your design partner kept running into product issues: missing features, half-baked screens, and consistent bugs. They understood the value of what you were doing, wanted it to work, but the product just wasn't there — or if it was, they didn’t push through the cruft to find it.

You most likely went to market too early, and don’t have a basic viable product that can solve the problem. You can also go too far in the other direction, and put yourself in a cave working on a perfect product that nobody ends up wanting. But a failed pilot that teaches you that you need to spend time improving your product? That’s valuable.

Ultimately, you want to end a pilot with an answer to the question, “Does your product add value?”

Ideally, the answer is yes. A no is still a win if you can interrogate data points from the pilot to understand if your POC is truly busted, or how it’s missing the mark. We’d all love to hit a home run right out of the gate, but the reality is that many, many great companies were founded on lots and lots of initial failures with their pilots. Some of Amplify’s most successful portfolio companies have gone through multiple POCs and iterations. 

There are also instances where the product you’re building is something the market truly wants, but for any number of reasons beyond your control, you don’t end up with a paying customer. Maybe your champion left, the budget got pulled, or their own priorities changed — these are all “fail” states that don’t necessarily mean you’re not on the right track.

All of that said, if your product is truly adding value, it’s highly unlikely that a design partner will just walk away after evaluating it. If you’ve done a good job upfront of making sure that what’s on the end of the stick is actually a carrot for them, then failing to convert is an important data point you’ll be glad to have now rather than later. If you follow this guide and have a bulletproof pilot plan, you should be converting 100% of the time — that’s really the bar that you should be shooting for.