[chuckcode]

I like the concept. Yet it feels like an echo chamber when the YC saas review company only reviews other YC based web analytics products after leading off with talk from YC parter about how everyone is doing it wrong... Do you think your audience won't notice the heavy YC bias?

[Fission]

Actually, I totally agree with you. This is something that we thought hard about, because our initial three guides are dominated by YC companies.

I think the main explanation is that YC companies, to a much greater extent than other companies, tend to focus on early-stage startups. As our guides are currently focused on early-stage startups, we naturally see more YC companies represented in the guides.

If anything, I'd argue that we're so aware of this that we need to consciously avoid overcorrecting in the other direction.

[ijpsud]

Have you thought about collecting reviews from "real" users of the products to supplement your analysis?

There's potentially a misaligned-incentives problem with review sites which themselves do the reviews. The latest episode of the FYI podcast[0] goes through this problem with the founder of Capiche[1] (looks like a competitor of yours) and the founder was explaining how firms like Gartner basically have two sets of customers: The software buys and the software sellers. The software sellers can actually pay Gartner to get in front of an analyst and sell them on their product.

I don't know to what extent this makes Gartner's recommendations less useful, but it seems like it would make it hard for smaller companies to get on the radar. I think the fundamental problem is that the analysts are not actual users of the software.

So I like Capiche's direction, but I dislike the fact that they're a "closed" invite-only website.

[0] https://twitter.com/ARKInvest/status/1261421627717218305

[1] https://capiche.com

[awwstn]

Thanks for mentioning Capiche! I think there's so much opportunity in this space so I'm stoked to see anyone else joining in.

At the very start, I wanted Capiche to be essentially Wirecutter for SaaS. We ended up going in a different direction, but I'm super impressed with Satchel's execution here, and as someone who spends a crazy amount of time on this stuff, I learned a bunch of new things reading through the site today.

I really don't want to hijack @fission's launch discussion, but thought I'd just reply to your comment on the invite-only thing: our site will be wide open soon-ish. Just taking our time building a community around high quality discussion (most of which has little to do with comparisons/reviews), but eventually will open it all up. Think of it like Stack Overflow's private beta...just a step in the process.

[Fission]

Hey Austin, I appreciate the thoughtfulness and the kind words.

I think even though we're working in different directions, we share a lot of the same values and see similar issues with the status quo. When (I hope not if) quarantine's over, let's grab a coffee :) — my email's in my profile

[1cvmask]

Any chance we can join now?

[Fission]

I think there are a couple of great points here, and I'll try to break things down and address each point independently.

1. re: "real" user reviews — right now, we do do this, just in a different way vs. what I think you're thinking of. For all of our guides so far, we've conducted a lot of interviews with founders who've used each product. However, we ultimately synthesize this data, pair it up with our own findings from our testing, and draw our own conclusions.

We think this is close to the optimal way of doing things. We found that sites which rely 100% on user reviews usually hit a ceiling in terms of usefulness, as the reviews are typically on the shallower side, which compounds b/c high-level reviews typically end up talking about the same things, so they're often redundant as well. By getting founder feedback, we're able to get a lot of datapoints to guide our findings and to surface less common issues, but by testing and ultimately writing these guides ourselves, we're able to both spend the time to elaborate on important aspects in depth (e.g. caveats, background info, etc.), while also being able to present info from a bigger picture perspective (e.g. this is how X, Y, and Z compare, etc.). The main downside is that it requires a lot of additional work.

2. re: Gartner + incentives — I believe that you identified the problem correctly, but misattributed the underlying cause. Gartner's incentives are misaligned b/c vendors who pay have an edge up on those who don't, for reasons you correctly identified. Importantly, this is not isolated to Gartner — in fact, the incentives are just as misaligned for a lot of the 100% UGC review sites. For those sites, if a vendor pays, they're allowed to cherrypick their own users to invite for an "organic review." If the vendor doesn't pay, then their rating will be relatively lower b/c it becomes much harder for them to get their best users to write a review on the site.

So in general, the problem isn't caused by whether you're a UGC review site or write guides yourself — it's caused by, well, whether or not you're doing something slimy to make money.

Edit: Oh, and I just saw that you mentioned Capiche at the end. I just want to say that none of this comment is directed at them — I sincerely think that they're passionate, motivated founders who deeply care about doing good for the startup community.

[pc86]

I think it's possible to have a system where lesser-known products can pay to get reviewed, and have the review still be unbiased and truthful. The probably is that it's impossible to know where that stops and "pay 1x to get reviewed, pay 2x for a good review" begins, and I think the vast majority of people would be immediately distrustful of any system where I can pay for my brand new, single-customer startup to get reviewed by a giant like Gartner.

[shostack]

I'm not sure I follow. Presumably, more mature SaaS products have more widely available reviews, documentation, tutorials, complaints, etc. to help companies make more informed decisions.

They also seem like they might have more proven business models, which is important when you're relying on it for a core function of your business. The last thing you want is some fresh startup that craters and forces you to scramble to find and implement a replacement when they fail to close their next funding round and need to shutter things.

[Fission]

Would you mind elaborating on what you don't follow on? I generally agree with everything you've said, so normally it'd be A-OK, but I think there's a question here somewhere (though I'm not exactly sure what) and I want to make sure I can address it.

[OoTheNigerian]

It's more of a comment on your previous response than a question.

1. Older companies tend to have wider footprint of real world use in various contexts thereby having a wider "body of work" for objective analysis and review.

2. There is switching cost when you get the 'thank you from the incredible journey" email or blogpost. So it is important to have companies that are self-sufficient and aren't at the risk of being acquired and shutdown which new companies are much more prone to.

PS: It is acknowledged that new companies most likely have fresher takes on old problems but you should also cater for their negatives enumerated above.

[shostack]

The other person who responded nailed it.

There's definitely value and opportunity that can be found in using the newest tech from the freshest companies. You can often find great value from their pricing, better UI, improved results, etc.

So my comment was in no way meant to bash using a startup. It was meant more to point out that there's a double-edged sword here by focusing on startups as there are real risks and trade-offs.

[quickthrower2]

I agree this would be a problem. If I am using a review to pick a tool I want to know they’ve tried the old tool from Oracle and IBM and Microsoft and they are treating it as seriously as I would as the person that has to pay the bill.

[Fission]

I also agree that this is a problem. This is why we're evaluating the old-school solutions for each problem. For payroll, this is ADP + PayChex; for knowledge bases this is Confluence; for payment processors we're even dealing with Authorize.net, even though I'd think most startups would eliminate that choice based on repressed memories alone.

I believe we've also done a good job of this in our current guides: we discuss LegalZoom and lawyers in our incorporation service guide (even though bonafide startup incorporation services themselves are relatively novel), and for our store of money guide we even went so far as to stick our money into Fidelity (the definition of old school) to get a cash management comparable.

Our event-based analytics guide doesn't have any old school evaluation, but that's because event-based analytics tools, in the grand scheme of things, are really quite a recent development.

[caseyf7]

This really comes across in the choice of Brex for storing cash. I don't think most VCs would be comfortable with portfolio companies storing their millions with Brex (unless they are investors in Brex). Startup CFOs are not chasing yield with investor's money and really want the safest, most convenient option, that builds a relationship for the future if the company is successful.

[DjMojoRisin]

Money Market funds are a good option to store cash that one needs for the immediate term, and there is a wide variety of them. However, one should evaluate each fund and their holdings to understand the risk & liquidity profiles.

My colleague just wrote this explainer detailing how money market funds work and how one should go about selecting one, in case it's helpful - https://medium.com/@mikedombrowski/are-money-market-funds-th...

Full Disclosure: I'm the founder of InterPrime (yc w19). We provide treasury management services and bring the treasury tools that big companies have been using for decades to Startups, SMBs, Non-profits & Investment Funds. Happy to chat and help anyone with questions on these topics.

[Fission]

This is actually a really interesting comment, because it reminds me of a meta-observation that I've made over the past few weeks.

First off though, I want to clarify that the money market fund is typically not operated by the org. The GMMF choices for SVB are managed by BlackRock, Morgan Stanley, etc., for Brex it is BNY Mellon, etc. You really only get self-operated funds for larger financial institutions, e.g. Wells Fargo or Fidelity. Therefore, many of these products act as a software layer over some existing bank/fund/etc. I believe we explain this distinction in the article. There are some caveats, but this is the general gist.

But onto the meta-observation: given the current economic climate w/ depressed treasury rates, we've recently received a lot of questions about higher-yield options, specifically re: CDs which are currently hovering around 1.5%. These are from well-known, well-respected startups who presumably don't need the cash and don't need to deal with the high overhead and illiquidity of CDs. I think that this really demonstrates the survival instincts of later-stage founders being carried forward from their early days, and the fact that startups are functionally live optimization machines, not just at a product-level, but as an entire organization. It seems to us that the best startups eke out every advantage they can — with all respect to the nature of the power law, these small differences, in aggregate, might make or break the entire company.