How We Launched A $3.2K/Month Automated Software For Testing Websites
Hello! Who are you and what business did you start?
Hi there! My name is Fitz Nowlan and I am one of two co-founders of Reflect. Reflect creates and manages automated web regression tests to verify a web application's appearance and functionality. Using Reflect, you can create automated web tests that replicate how a user actually uses your web application---all without writing a line of code. The benefits are two-pronged: 1) creating a Reflect test is faster than using a code-based automation tool, and 2) anyone in your organization can create a test because it requires no programming. Thus, businesses improve the quality of their web applications while simultaneously saving time and allowing anyone to contribute to their testing efforts.
Our customers are technology companies who sell either B2B, or B2C, and deliver their services or products through their web application. Therefore, they have a critical dependency on their web application behaving as expected in order for them to make money. They’ll usually have employees test (i.e., use) their web application to ensure it’s working, but the testers can range from technical folks like web developers and testing engineers, to non-technical folks like marketing and SEO specialists. As a result of the varied technical skills of application testers, we decided to make Reflect zero-code and zero-installation. This means anyone can use Reflect without coding or downloading anything, and create a test with nothing more than a URL!
Our customers access Reflect through a monthly, pay-as-you-go subscription that includes creating, executing and managing their suite of web tests. Reflect has been publicly available since November 2019 and generates around $3K in monthly recurring revenue. The cost of a subscription scales based on the number of test executions and the month-to-month plans range from ~$100/mo to $1,000/mo or more for our Enterprise customers. Our most exciting statistic is that we have not lost a customer yet!
Here’s an example of me creating a test for Starter Story, where I first validate that the headline text is present with a visual assertion. Then, I enter my email address into the form and Reflect captures the text input action:
Our cloud-based browser automatically captures those events and builds up the plain text description of the steps on the left-hand side. Once I save this test, Reflect will re-execute those actions whenever I want (e.g., every day at 8 AM) and alert me via email if any of the actions fail. Many of our customers use our scheduling functionality to have their tests run automatically every day or every week, but we support deeper integrations through our API.
Here’s a quick-start demo video that shows how to create your first test in Reflect:
What's your backstory and how did you come up with the idea?
I am from the Philadelphia suburbs and began programming in Java as a senior in high school. I graduated from Georgetown University in 2009 and went straight into the Computer Science Ph.D. program at Yale University. While earning my Ph.D. with a focus on networking, I took internships in the summers to work at Microsoft and Google. These experiences were eye-opening in many positive ways, but in particular, I knew I wanted to build software for sale rather than pursue a career in research. I loved the idea of taking some software that I wrote to market and “setting up shop” to sell it to customers. The plight of the entrepreneur excited me.
When I was deciding where to work after graduation, both my wife and I were feeling an internal pull to move back to the Philadelphia area where both of our families reside. I didn’t know much about the Philadelphia tech scene, but my mother sent me an article she found online about a digital marketing start-up named Curalate. (Yes, that is the exact article from 2013.) Obviously, I knew I wouldn’t be doing any work related to my research in networking, but I was hoping it would provide a broad software engineering education, of sorts, which I could one day in the future pair together with my highly-concentrated research expertise. It turned out to be a perfect fit! Curalate was a first-class engineering organization and I learned as much there as I could anywhere. It was instrumental in my growth as an engineer, and it’s also where I met my Reflect co-founder, Todd McNeal. He was Curalate’s very first engineering hire back in 2012.
Our idea for Reflect was born as a direct result of our experience building and maintaining software over the course of our careers. Developers have a lot of great options available when it comes to unit and integration testing, but for E2E testing they’re either investing a lot of resources to build and maintain automated test suites, relying heavily on manual testing, or are going without formal testing. It’s interesting that there hasn’t been much innovation in E2E testing vs. unit and integration testing, considering how critical it is to software organizations: whenever you deploy changes to your website or application, you run the risk of regressing functionality that used to work. Often, your customers will forgive you if some new functionality doesn’t work as expected, but they will be much less forgiving if you break functionality that they have used and relied on for months or years. Developers learn pretty quickly how important it is to test things end-to-end. You can spend all this effort unit and integration testing, but a user will still hit a bug immediately after that feature ships if you’re not testing like an end-user.
Fast forward to late 2018 and Todd and I began talking through start-up ideas. We had three criteria for potential businesses:
- We wanted to sell a software product,
- We wanted to sell to a market we understood, and
- We didn’t want to be first to market - i.e., we wanted someone to have already validated that a market for the product exists.
We spent some time ideating and researching and ended up kicking around 10 or so ideas one day at a coffee shop in Philly. The ideas were pretty varied - for example, one was building our own ISP using mesh networking. However, we kept coming back to, and finally settled on, the idea of a test automation product. There were already a ton of tools in the market---a lot of which we’ve tried over the years---and it felt like they’re all rehashing the same approaches over and over with the same limitations.
In some ways, it can be difficult to find the motivation to work on a side project because you’re not dependent on its success financially and you’re probably exhausted after a full day of work.
The code-based tools are really powerful but when you’re a small team the maintenance cost can increase pretty quickly. They also have some serious blind spots in terms of replicating how users interact with a website. All code-based tools can verify that an element exists on the page, but what you really want is to validate that the page appears visually how you’d expect, which is hard to do in code. This space also appealed to us because we believed (and still believe fervently) that being last-to-market is a massive advantage because we could make use of technology that quite literally was not available to our competitors when they started years ago.
Take us through the process of designing, prototyping, and manufacturing your first product.
We were quite familiar with the task of manually testing web applications from our experiences as Software Engineers and then later as Directors of Engineering. As we researched the existing no-code testing tools, we noticed that nearly all of them required users to download and install an extension as a means of capturing the tester’s actions. To us as users (i.e., testers) in those tools, it felt cumbersome. But worse, this approach opens up so many issues that can cause tests to not record properly. For example, we needed to effectively reset our browser state (e.g., clear cookies and site settings) before recording any tests. The reason is that when the tool later executes the test automatically, it won’t have your cookies or your browser state. So, the test is likely to fail. It’s the same issue if you change the browser dimensions mid-test, run behind a proxy, or use an ad-blocker---these all cause tests to be recorded incorrectly.
We decided the only practical way to avoid this class of failures was to create a cloud-based browser where the tester uses the website via proxy. Furthermore, if we instrument the cloud-based browser to capture the tester’s interactions on the webpage, then the tester doesn’t need to be able to “code” at all. They just need to use a website!
In addition to producing more resilient tests, this drastically lowers the barrier to entry for anyone wishing to create a test. We quickly decided this would be our differentiator in the marketplace. With Reflect, anyone in your organization can create and manage a test because there is no installation and there is no coding. For this approach to work, Reflect has to produce high-fidelity and maintainable tests. If it does just that, we unlock a ton of value for our users, since they can literally create a test in minutes and execute it as often as they want.
Once we determined a cloud-based browser was feasible, we began building our initial Minimum Viable Product (MVP). This took us basically from November 2018 to around April 2019, working nights and weekends on our own. Here are some screenshots of our original MVP application from April 2019 compared to our current application in February 2020:
The first pair of screenshots is the Test Recorder, which shows the webpage being tested on the right-hand side with the test steps on the left. The first screenshot shows a test verifying the Sign-In experience on nordstrom.com. Notice the captured click actions displayed on the left-hand side. The second screenshot shows a similar test recording with the current version of our app (Feb 2020).
The second pair of screenshots is the Test Detail page, which shows the results of a test execution. These results are for a different test on amazon.com (1) and homedepot.com (2). The test actions appear on the left as in the recording page, but the right side shows a video of the test executing in our cloud-based browser.
Describe the process of launching the business.
By April 2019, we had built a working MVP of Reflect and shown it to some friends and colleagues. At this point, we were mentally committed to starting a business around Reflect and eventually leaving our jobs at Curalate when the time was right. (More on this later.) We established the business entity, put up a landing page, and began emailing technology companies in the Philadelphia area. We had a surprisingly great response rate from local CTOs and engineers who were supportive and happy to check out the product and provide feedback.
This is basically what you would call a “soft launch”. We were still employed elsewhere, we probably weren’t ready to sign a customer, but we started our outreach anyway and sought feedback on our product. This was beneficial for at least a few reasons:
- We established some connections which ultimately led to our first sales, and
- We received valuable feedback from our target customers.
The other learning here is to not be shy about asking for feedback or for help, but make sure you do it once you have something to show or talk about. In other words, don’t email someone and ask, “I’d like to do X, how do I get started?”. Instead, email them and say, “I’ve already built X, but I’m struggling with some piece of it, Y. Do you have any ideas on how I can work around it?”. People generally remember what it was like just starting out themselves and they’re happy to give a boost, but you need to have already demonstrated consistent effort and initiative.
In terms of costs up to this point of the initial soft launch, we paid $500 for Stripe Atlas to register our LLC partnership and once we had our landing page live on AWS, we purchased a GSuite subscription so that we could send emails from our business domain (reflect.run). Stripe Atlas comes with a $5,000 credit for AWS, so we had $0 of monthly infrastructure costs and we’re almost a year in now and still have credit left! If you’re starting a technology business and AWS is right for you, you really can’t beat a $5,000 credit. Stripe Atlas actually has an even larger credit for Google Cloud if you are accepted into their Google Cloud for Startups program (very straightforward), but AWS was just a more familiar platform for us, so we went with that.
It was a great feeling to have a company, a live web application, email addresses, and everything. But obviously we knew the real decision was when to leave our jobs and work on Reflect full-time. We are both extremely fortunate to have working wives who are able to support our families (we both have toddlers) while we pursue this opportunity. Thanks to them, we were able to leave our jobs in September 2019 and start full-time on Reflect. In the span of about a month, we reworked the landing page, fixed a smorgasbord of bugs, and publicly launched on LinkedIn and Twitter on November 1st, 2019. We activated most of our network over email around that time and we signed our first customers that month!
The dominant challenge we’ve needed to overcome when selling our product is how to convince customers that our tool will reduce the time and financial investment of their regression tests and improve their efficiency.
Since launch, what has worked to attract and retain customers?
LinkedIn, Indie Hackers and direct email outreach have been our best performing channels for attracting customers so far. One of our IH posts yielded nearly 100 visitors that day. Additionally, having a geographical connection to Philly always improves the chances that a customer is willing to talk. Our personal networks have been valuable and led to a few customers as well, and we run search ads with a limited budget. As the steady-state of traffic volume to our marketing site increases, we’ll be able to experiment more with things like tweaking the landing page to see how that affects conversions. But we’re not quite there yet in our marketing sophistication.
Retention is first and foremost driven by whether your customers derive value from the product. In our case, this means does Reflect detect web application regressions and is it easy for our users to manage their tests over time. We gather insight into these questions through the test runs themselves, usage of our application and user feedback in emails. This is table stakes for any successful business: the product has to work and do what it says it will do.
The second factor in retention is customer service. We aim to be friendly, receptive and honest when interacting with our customers. We respond to emails promptly and we incorporate their feedback into the product. We also share our product roadmap with our customers. We have not lost a customer who has signed up for a paid subscription with us yet. Certainly, at some point, a customer will not renew with us, but our retention is an achievement we’re very proud of so far.
How are you doing today and what does the future look like?
We’re profitable in terms of Cost of Goods Sold (COGS), which means our revenue exceeds the sum of our expenses from infrastructure and the other platforms we use to conduct business (see Platforms below). In terms of Net Income, we’re not taking salaries at this point, but if we were, that would obviously make us not profitable. Our next goal is to get to the point where we can take small salaries, which would be around $8-10K in monthly recurring revenue.
It’s obviously only been about 4 months since we publicly launched and our customer acquisition has been slow, but steady. We always have a few demos or meetings with potential customers each week, but since our leads are primarily driven by referrals and email outreach (right now), we haven’t achieved a significant scale on our customer growth. Scaling our marketing efforts through better content and greater visibility on the web is our primary goal in Q1 2020.
Lastly, we’re coding and improving the product every single day.
Through starting the business, have you learned anything particularly helpful or advantageous?
Almost every business that we’ve come across values their employees’ time and this seems to be the number one reason they resist change. The financial cost of a tool or platform is definitely important for smaller companies, but the dominant challenge we’ve needed to overcome when selling our product is how to convince customers that our tool will reduce the time and financial investment of their regression tests and improve their efficiency. This, of course, makes sense since there are few businesses whose employees have extra time on their hands. But, I think I didn’t consciously realize how averse to change potential customers would be when it comes to their employees’ day-to-day tasks and time.
What platform/tools do you use for your business?
We’re a software business so the most important tool for us to manage and publish our code is GitHub. We used GitHub from Day 1 for tracking code and issues, and once we had a working prototype we published it to AWS. Then, we signed up for GSuite so that we could email prospective customers from our business domain (reflect.run). Fast forward a few months and we were ready to sign our first customer, so we used Stripe to process our payments and manage the monthly subscriptions. We also signed up for Postmark to send notification emails. By the end of our first year, we had revenue to report, so we signed up for QuickBooks to “balance the books” and produce the forms we need to file our tax return. Lastly, we’re ready to turn some attention to SEO and traffic acquisition, so we’re looking into tools like Moz for that.
The nice thing about each of these tools is that:
- The pricing scales to the complexity or volume of our usage, and
- They’re pretty much instantaneously usable.
So, we could delay signing up for Stripe until we had payments to process, but as soon as we had them, it only took us half a day to get off the ground and charge a card. Obviously, these providers have spent years honing these products to be as “instant-on” as they are, but it’s worth calling out that because of these services and the ecosystem of competitors, it’s never been easier to start a business than it is today.
What have been the most influential books, podcasts, or other resources?
We’re both daily readers of YCombinator Hacker News aggregator and believe in the start-up ethos embodied by a lot of the folks in that community. Paul Graham’s personal articles are inspiring and provide a great, even-keeled perspective for founders.
Another resource and community that we interact with daily is Indie Hackers. I’ve never come across a community that is as positive and substantive as IH. You can find a lot of blindly positive communities lacking in substance on the net. On IH, people are quick to respond to posts with constructive feedback or an encouraging word, and it isn’t cheesy or tinted with rose-colored glasses. It seems honest but nurturing. I’ve taken inspiration from folks there, especially the really small posts about signing the first customer or discovering marketing hacks. It’s an authentic collection of skilled people who are “fighting the good fight” and really grinding to succeed.
Lastly, the most influential e-book for us has been Refactoring UI, by Adam Wathan and Steve Schoger. We loved the small tips and best practices Steve shared on his site and bought the e-book with the Black Friday special discount. It’s been an absolutely terrific resource for us.
Advice for other entrepreneurs who want to get started or are just starting out?
We benefited immensely from working on our idea, exploring technical feasibility and researching the existing market while we were still employed full-time at our previous jobs. In some ways, it can be difficult to find the motivation to work on a side project because you’re not dependent on its success financially and you’re probably exhausted after a full day of work. But if you can learn to be disciplined in those moments when the pressure is not on, you will become more disciplined and methodical when the pressure is on. It also gives you a running start when you leave.
The second point that comes to mind relates to getting started. For me personally, I have a lot of ideas that seem promising but just an enormous amount of work to build. As in, I imagine the product being fully done and it would take 756,040 steps to build it and I think, well, I can’t do that tonight, so I don’t. The apparent size or total work of building something valuable is daunting and it overwhelms me into not building things, which is obviously bad. Everyone knows this feeling as “writer’s block”, but it happens in every discipline.
To combat this, my strategy is basically a blend of similar concepts that I’ve come across in a lot of different materials, including Pirsig’s Zen and the Art of Motorcycle Maintenance, Thiel’s Zero to One, and Matt D’Avella’s superhuman performance, to name a few. The idea is: do the smallest possible thing that brings you closer to your goal as a means of unlocking your internal drive, or flow, which will then propel and sustain you to achieve the much larger goal.
An illustrative example of this is publishing the first version of our web application. It’s daunting to imagine all of the features we ever could or should build around web testing. But it’s easy to ignore all of the operations or features and imagine, simply, a test. That immediately prompts the question: “what is a test”? That’s tough to answer, but “I know it has a name”.
So, I’ll publish a web application that just returns names. With that, my mind is racing to ask questions about what I can do with tests and what information I want to associate with them. Even if I stop right there (for the night) or have to redo everything later, I’ve mentally gotten closer to the solution than I was at the outset because I’ve considered the problem in greater detail.
Again, the strategy isn’t anything novel or mind-blowing, but I think it’s applicable to any work or task that a person is struggling to begin. If you think your idea is too big or difficult to ever be fully realized, then don’t think of it being fully realized. Think of the smallest possible piece of the idea that is guaranteed to exist in the fully realized version and start building that instead.
Are you looking to hire for certain positions right now?
We aren’t looking to hire yet but we hope to do so at some point. Since we’re bootstrapping the business, we need to reach a certain threshold of monthly revenue in order to feel comfortable bringing on an employee.
Where can we go to learn more?
Our company:
My personal info:
We love chatting tech and entrepreneurship in general, so please feel free to get in touch. Lastly, I’d like to extend a personal Thank You to Pat and the Starter Story team for giving us the opportunity to share our story!
- 4,818 founder case studies
- Access to our founder directory
- Live events, courses and recordings
- 8,628 business ideas
- $1M in software savings
- 4,818 founder case studies
- Access to our founder directory
- Live events, courses and recordings
- 8,628 business ideas
- $1M in software savings
- 4,818 founder case studies
- Access to our founder directory
- Live events, courses and recordings
- 8,628 business ideas
- $1M in software savings
- 4,818 founder case studies
- Access to our founder directory
- Live events, courses and recordings
- 8,628 business ideas
- $1M in software savings
- 4,818 founder case studies
- Access to our founder directory
- Live events, courses and recordings
- 8,628 business ideas
- $1M in software savings
- 4,818 founder case studies
- Access to our founder directory
- Live events, courses and recordings
- 8,628 business ideas
- $1M in software savings
- 4,818 founder case studies
- Access to our founder directory
- Live events, courses and recordings
- 8,628 business ideas
- $1M in software savings
- 4,818 founder case studies
- Access to our founder directory
- Live events, courses and recordings
- 8,628 business ideas
- $1M in software savings
