Show HN: Visibly – More Collaborative and Efficient Code Reviews on GitHub

[nggonzalez]

Hey HN! I’m Nick, the founder of Visibly (https://visibly.dev). Visibly is a GitHub App and Chrome extension that makes code review more efficient, collaborative, and visible by enabling you to:

- See review comment thread statuses so that you can quickly address each thread and focus your attention on the ones that actually require it

- Indicate whether feedback is blocking or non-blocking so only crucial feedback blocks PR merge

- Ensure all blocking review threads are addressed before merge via a status check

- Track total active review time per reviewer so that you can understand the effort and review time going into each PR

- See reviewer presence so that you can see if someone is already reviewing and avoid unnecessary pings.

In addition to these workflow improvements, Visibly provides a personalized metrics dashboard so you can understand your real-time performance and better advocate for yourself and advance your career.

You can see a demo and a breakdown of all of the current features at https://visibly.dev.

I created Visibly after seeing the same pain points across Microsoft, Snap, Headspace, and Brex:

- Code review was frequently undervalued and invisible. Developers recognized this and prioritized code production instead, which resulted in knowledge silos, poor code quality, and a culture of rubber stamp reviews.

- Engineering performance review was broken. From the manager’s perspective, it was difficult to identify top performers and to provide timely, actionable feedback. From the IC’s perspective, feedback was infrequent and subjective around many aspects of the job. The process was also time-consuming and required everyone to sift through 3-6 months of work to quantify “impact and scope”.

Visibly solves these issues by using objective measures to make code review more visible and efficient.

Visibly values privacy and security above all else; it does not require source code access and requests the fewest possible permissions to provide these experiences. Visibly is also differentiated in its belief that individuals should know how they’re doing and have personalized metrics so that they can advance in their careers.

If you use GitHub, you can try Visibly today for free by heading to https://visibly.dev/login. Visibly is currently free, but we plan to introduce paid features and pricing plans soon. Our pricing is still under development, but based on existing customer feedback, we expect to price at $20/seat/month. For those that sign up over the next week, we plan to provide 50% off for the first year once our pricing plans launch.

We look forward to your thoughts and feedback!

[d4nyll]

One question and one comment:

Q) What's the difference between a blocking comment and a "normal" comment + rejecting the PR?

C) Exposing metrics like "Review time per reviewer" may, at the wrong hands, incentivise the wrong behavior. For example, a team lead may view long review times by a particular reviewer as them being slow, whereas they are just more thorough. Tracking the total review time for the PR (aggregating the times from multiple reviewers) is more useful.

[nggonzalez]

> Q) What's the difference between a blocking comment and a "normal" comment + rejecting the PR?

Good question. The granularity of the states are a bit different. Rejecting the PR doesn't capture individual thread states. You may even forget to look at thread later by mistake. With the blocking thread, the individual thread needs to be marked as addressed in order for the status check to go green. This helps ensure that the work gets done and also makes it possible to leave comments like: LGTM after comments or something similar.

Another difference is cultural. At a couple of the companies I was at, Request Changes/Reject on GitHub was viewed as passive aggressive and not used. Blocking and non-blocking comments help these team be more explicit and block on smaller changes in a friendlier way.

> C) Exposing metrics like "Review time per reviewer" may, at the wrong hands, incentivise the wrong behavior. For example, a team lead may view long review times by a particular reviewer as them being slow, whereas they are just more thorough. Tracking the total review time for the PR (aggregating the times from multiple reviewers) is more useful.

Visibly does both, it tracks cumulative time across all reviewers and the individual times. The reasoning is to help show the effort people are putting in to the reviews and the people they are reviewing and to understand, to some degree, if a review was thorough. The metrics help show trends and just surface more knowledge, they should still be a single input in a more holistic picture though.

A few examples that becomes possible with these metrics: (1) You can show why you couldn't get to a task sooner if you spent time reviewing a lot of PRs on the day. (2) You can understand if your PRs "cost" more than other PRs opened by the team. This can help you write smaller PRs or identify the differences between your work and others'.

We also plan to use this time to help surface an estimated "It will take you X min to review this PR" in the future.

[d4nyll]

Thanks for clarifying. I agree that rejecting a PR can be seen as negative in some companies, but I'd also argue that it's a symptom of a bad culture to view rejected PRs as something negative. And using tools to mask a a negative culture instead of surfacing it may not be wise.

> they should still be a single input in a more holistic picture though.

Whilst, in theory, no single metric should be used to determine performance, in practice it may (especially if your lead is inexperienced). But it can be prevented altogether by exposing individual metrics to indivuals only, and exposing the less granular aggregate metrics to the team and team lead.

> You can show why you couldn't get to a task sooner if you spent time reviewing a lot of PRs on the day.

I think in a culture where you have to justify AND PROVE why you couldn't do a task sooner shows a lack of trust, and the problem won't go away even if you are able to show, this time around, that the delay is justified.

I do think tracking the cost of a PR is important, as it will incentivize smaller PRs. But my point is that exposing the "wrong" metrics (like individual review times) isn't just a data point that people don't have to use, but that it can be harmful as it'll incentivize the wrong behavior (e.g. knowing the team lead value quick reviews, developers may be incentivized to review quicker and thus less thoroughly).

[brandonb]

Very cool! I like the idea of tracking time per active reviewer -- in addition to bringing visibility to the work of reviewing code, it doubles as a signal of how easy a particular PR was to understand. That's potentially valuable feedback to both the author of the code (e.g., if smaller PRs would be helpful), as well as to the team if there are parts of the codebase that systematically take longer to grok.

[nggonzalez]

Exactly! As we continue to iterate through the visualization side of the metrics, I think this will become even easier to see. Another benefit of the time tracking is that it helps lessen negative emotional responses when receiving a lot of comments - if you see that someone has spent 30 minutes of their time to review, it helps put it in context and show the effort they're putting into you. Without the time, its easy to feel that someone just quickly dropped a bunch of comments and doesn't care

[mtlynch]

>See reviewer presence so that you can see if someone is already reviewing and avoid unnecessary pings.

>Visibly values privacy and security above all else

These two things seem squarely at odds. Personally, I don't want my code review tool announcing to developers when I'm looking at their PR.

The fact that it's a Chrome extension would also be a big blocker for me. I run a dev team, and I wouldn't ask any of my teammates to install a browser extension to participate in code reviews, especially one that's only available for Chrome.

I care a lot about code reviews [0], but the features here don't really resonate with me.

- Blocking vs. non-blocking: This is useful, but the default should be that a comment is blocking. Otherwise, it makes the default that all the reviewer's notes are optional unless they go out of their way to mark them as important.

- Thread status history: I think the status history is much less important than the status right now. In other words, I'd like to know if a review is blocked on the author or reviewer. I'm not that interested in what the status was yesterday.

- Metrics: I worry that these focus on the wrong thing. I'm trying to maximize usefulness of my review rather than speedrun through them to improve my PR time.

One of the huge gaps in Github's tool is the ability to do multi-round reviews (e.g., "show me what's changed since my last round of notes"). You can kind of bend Github to show you the right diff, but then it can't show the comments in a sensible way. Two tools that solve this well are CodeApprove[1] and Reviewable[2].

[0] https://mtlynch.io/tags/code-review/

[1] https://codeapprove.com

[2] https://reviewable.io

[nggonzalez]

Thanks for the extensive feedback. Here are my thoughts on these things:

> Personally, I don't want my code review tool announcing to developers when I'm looking at their PR.

This feature is similar to Google docs presence and helps prevent unnecessary work and pings. In my experience, people generally want to know when someone is looking at their PR so they know whether they need to find another reviewer or follow up.

> The fact that it's a Chrome extension would also be a big blocker for me.

Can you expand a little more on this point? At my previous companies, almost all development and debugging was done through Chrome. Happy to expand to other browsers. To provide more context, the reason it is an extension is so that it can update GitHub in place, rather than forcing you to navigate to another website and learn that experience too.

> - Blocking vs. non-blocking: This is useful, but the default should be that a comment is blocking. Otherwise, it makes the default that all the reviewer's notes are optional unless they go out of their way to mark them as important.

The default is blocking. You make something non-blocking by unchecking the box or writing "nit:"

> - Thread status history: I think the status history is much less important than the status right now. In other words, I'd like to know if a review is blocked on the author or reviewer. I'm not that interested in what the status was yesterday.

The current status always visible in the thread header and footer. You can expand the history to see all statuses, but you don't need to do so to see the latest.

> Metrics: I worry that these focus on the wrong thing. I'm trying to maximize usefulness of my review rather than speedrun through them to improve my PR time.

Which aspects of metrics do you feel lead to that incorrect focus? The intention with the review time metrics is definitely not to speedrun, it is quite the opposite - with the review time you can see how much effort has gone into the review, and so it helps naturally push developers to smaller PRs and to also leave more thorough reviews. Without the time, it's really easy to review a 1000 line PR in less than minute because there's no accountability. With the time, it would be easy to spot these cases. This goes back to the classic meme of short PR == 20 comments and large PR === LGTM with no comments

[mtlynch]

>This feature is similar to Google docs presence and helps prevent unnecessary work and pings.

Right, I understand, and I think for developer tasks like reviewing code or reviewing design documents, that's a negative feature in docs.

When I'm reviewing someone's work (be it a design doc or a PR), I don't want them watching me as I do it. I want time to focus privately, and I want to be in control of when I share my feedback with them.

>In my experience, people generally want to know when someone is looking at their PR so they know whether they need to find another reviewer or follow up.

I'm having trouble imagining a scenario where that would be needed.

If it's an urgent PR that needs review ASAP (rare in my experience, but it happens), I'd ping them on another channel to get positive confirmation they're on it and understand it's urgent.

If it's a normal PR, and the norms on my team are that PRs are reviewed in <1 business day, and they're over the limit, and I'm blocked, I'd send them a ping to make sure they're on it. I guess Visibly would prevent me from pinging them if I caught them in the 60-90 min window where they're beyond the normal deadline but still in the middle of a review, but I'd expect those cases to be fairly rare.

What's the typical flow where the presence feature would save a ping?

>Can you expand a little more on this point? At my previous companies, almost all development and debugging was done through Chrome.

Through Chrome specifically or through a web browser? My teammates do a lot of their work in a web browser, but they use a variety of browsers.

I should clarify that I work with a team of freelance developers, so I don't like asking them to install unnecessary software on their systems. I definitely wouldn't ask them to install surveillance software, which it sounds like is what Visibly is.

We currently do code approve with CodeApprove, which everyone on the team likes. We do have to learn new UI flows, but they're pretty simple. It was a problem with Reviewable, which had a steeper learning curve.

>The default is blocking. You make something non-blocking by unchecking the box or writing "nit:"

Ah, okay. I like that behavior.

>The current status always visible in the thread header and footer. You can expand the history to see all statuses, but you don't need to do so to see the latest.

That sounds good, but my feedback was more about highlighting that as a first-class feature of the product. I don't think status history is an important thing to surface, so it might not be a worthwhile thing to highlight as a feature.

>Which aspects of metrics do you feel lead to that incorrect focus? The intention with the review time metrics is definitely not to speedrun, it is quite the opposite - with the review time you can see how much effort has gone into the review, and so it helps naturally push developers to smaller PRs and to also leave more thorough reviews. Without the time, it's really easy to review a 1000 line PR in less than minute because there's no accountability. With the time, it would be easy to spot these cases. This goes back to the classic meme of short PR == 20 comments and large PR === LGTM with no comments

Oh, I thought the PR times were only visible to the reviewer. It sounds like you're saying that the PR times are public because their purpose is to snitch on the reviewer to their teammates and reveal that they didn't spend enough time on the review.

Again, I feel like this is a big contradiction to the promise of "Visibly values privacy" because I definitely don't want a surveillance tool watching me do reviews and reporting my behavior to my teammates.

If the problem is that developers are speeding through reviews and rubber stamping them without really reading them, that's a culture problem and not a tooling problem. Attempts to reduce developer productivity down to simple metrics like "lines of code written" or "minutes per code review" are counterproductive.

[nggonzalez]

> When I'm reviewing someone's work (be it a design doc or a PR), I don't want them watching me as I do it. I want time to focus privately, and I want to be in control of when I share my feedback with them.

That's fair. I'll think about how I can address this longer-term. Would being able to enter a "privacy" mode that hides your presence help?

> What's the typical flow where the presence feature would save a ping?

The typical flow I envision is someone on the same team following up with a coworker after 30-40 minutes after that person has indicated that they will review. It helps them understand if they need to ping again as a reminder. The reason someone may ping in that window is because PRs behave like black boxes after the link is sent - they have no idea if someone is reviewing and how far along that reviewer may be are; the author just has to wait until the comments come.

We're currently working on real notifications pipeline, so this won't be a huge use case in the future.

A different use case where presence may be more useful is to prevent overlapping reviews when pings to codeowners teams are automatic. If you can see someone else is reviewing, then you can defer to them.

> I should clarify that I work with a team of freelance developers, so I don't like asking them to install unnecessary software on their systems. I definitely wouldn't ask them to install surveillance software, which it sounds like is what Visibly is.

I disagree on the surveillance comment - Visibly uses GitHub data that is readily available, augments it with the review time metrics, and allows you to see the progression over time. Beyond that, Visibly does not monitor your work; it does not track what sites you're on or how you do your work. Visibly is only active on github.com and only on PR pages atm. It also takes the most minimal permissions to do this.

Visibly is there to help you and your team understand how much time and effort reviews are taking so you identify trends and better optimize processes. It's also there to help the IC represent their work come perf time - it's really hard to figure out what to write in a perf review when you have to sift through months of work and reviews.

> Oh, I thought the PR times were only visible to the reviewer.

PR times are visible to those who have read/write permissions on the repo. Just like they can see comments / reviewers, they can see status and time. It's just another decision point.

> Attempts to reduce developer productivity down to simple metrics like "lines of code written" or "minutes per code review" are counterproductive.

I agree and not what Visibly is doing. Visibly surfaces metrics to give you more insight. If you don't know how much time reviews are taking, it's hard to accurately estimate timelines, hard to understand the thoroughness of reviews you receive, hard to see the cost of tech debt / complex code, etc. Visibly provides knowledge that you can use to adapt and improve collaboration.

[codeapprove]

Wow this looks great. I really like the additions of presence and the stats about review times. It seems seamlessly integrated.

As you might be able to tell from my username, I’m the founder of CodeApprove (codeapprove.com) which is meant to fix a lot of the same problems you’re going through with Visibly. I’m also passionate about code review and code review tools in general so I’d love to chat sometime, feel free to use the email in my profile.

[nggonzalez]

Thanks for the thoughtful comment. Sent you a DM

[Jochim]

What makes this not just another tool to be abused by micromanagers?

Not everything needs to be measured, and indeed, measuring some things can cause more harm than good, especially as the measurements are often devoid of any kind of context.

[nggonzalez]

> What makes this not just another tool to be abused by micromanagers?

Visibly is meant to create transparency and visibility for the IC. Today, a lot of managers secretly look at metrics in calibrations and beyond in order gauge productivity and be able to discuss their reports. However, they're seeing a very small piece of the picture and are making apples-to-oranges comparisons. For example, an engineer in a new repo will look way more productive than an engineer in a legacy repo if you look at just lines of code. Visibly's goal is to help make comparisons apples-to-apples and to help you, the IC, see how your tracking in real-time so that you're not surprised later. In addition, Visibly aims to make a more holistic picture of your work so that invisible work that you're doing, like code review, mentorship, design docs, etc, can be captured and used to your benefit.

The intention is to give teams and ICs more knowledge so that they can grow and adapt more quickly and not be surprised by perf at the end.

Moreover, today we don't have a manager portal and have prioritized the IC dashboard. Although it is on the roadmap, managers will see what you see and we are working to make it easy to share highlights and other positives of your performance to them to help avoid this.

I truly understand the concern, but I'm hoping the framing presented and the Visibly team's values around this help ease it for you.

[barefeg]

I often start reviewing in GHs UI but depending on the complexity of the code I might need to go to vscode for better readability and intellisense. Does Visibly have an extension for that too?

[nggonzalez]

We haven't gotten to a VSCode extension yet, but it is on the roadmap!

Visibly has some auto transitions built in (ie. auto transitions threads to `addressed` when the diff becomes outdated and auto transitions threads to `in-progress` when the author reacts / comments back) and allows you to change a thread's status via `/v <command>` in comments. These have been a workaround for some of our existing teams, but it is something we'd like to address in the future.

[smcleod]

Is there a link to the source code somewhere?

[nggonzalez]

Visibly is currently closed source, so I can't provide a link, but curious to hear more about how it may help if some aspects of it were available