Code review
In Reviewable, each review corresponds to a GitHub pull request. To initiate a code review, you’ll first need to create a pull request as usual and then access the corresponding review. This can be done either (a) through the link to Reviewable inserted into the PR, or (b) by clicking on the PR on your Reviewable dashboard.
In general, Reviewable keeps data synchronized between the review and its pull request for all compatible features, such as assignees, comments, and approvals. However, some features are unique to Reviewable (such as file review marks or discussion dispositions). Other Reviewable features cannot be mapped effectively due to GitHub API limitations, such as line comments that often become top level comments. Consequently, we don’t recommend mixing use of Reviewable and GitHub reviews on the same pull request as the experience will prove frustrating on both sides.
Reviewable can't operate directly on raw commits, since it doesn't actually clone your repo. It depends heavily on GitHub APIs that are only available for pull requests.
This chapter provides an overview of the review page, but the core features of reviewing files and managing review discussions are addressed separately in subsequent chapters.
User roles
Throughout this guide, we'll often refer to users by the role they play in a review. Reviewable automatically assigns one of three roles to every review participant:
- Author: the creator of the pull request. Note if an author marks a file as reviewed they'll become a reviewer.
- Reviewer: anybody that is not an author or mentionee.
- Mentionee: someone other than the author that was @-mentioned in a discussion (except the main top-level one). Note that a mentionee will become a reviewer if they start a new discussion or mark a file as reviewed.
It's possible for one person to be both the author and a reviewer in a “self-review” scenario. In that case, the “reviewer” behavior usually trumps the “author” path, but it's context-dependent.
Publishing your review
As you work through a review using the tools at your disposal, Reviewable will automatically save your changes but they won't be visible to others. To publish all drafts and other buffered state changes (including review marks, dispositions, and acknowledgements), click the Publish button. This action will reveal all of these to other authorized Reviewable users, and also post a single, combined message to the PR on GitHub.
To temporarily keep a draft from being published, switch its disposition to Pondering.
If you'd like to suppress posting the combined message to Github, e.g. because you want to separate the Reviewable review from the GitHub review process, you can do so on a PR-by-PR basis. Just add the Comments only in Reviewable label (exact spelling!) to the PR and neither batch published nor single-send comments will be posted to GitHub.
Some Reviewable inline comments won't appear as inline comments in GitHub and vice-versa, because the GitHub model for comment placement is poor and trying to conform to it would invalidate many of the best features in Reviewable.
You can open the dropdown menu off the Publish button to set a few extra options. The dropdown also shows you a preview of how your published message will appear on GitHub. You can click on any of your comments in this preview to navigate to the corresponding draft in the review.
Approval levels
Like for GitHub reviews, there are three approval levels you can choose from when publishing via the dropdown attached to the button:
- Comment — Submit general feedback without explicit approval.
- Approve — Submit feedback and approve merging these changes.
- Request changes — Submit feedback that must be addressed before merging.
Reviewable will select a default approval level for you according to your review marks and the disposition of any comments you’re sending. You can override this level in the Publish dropdown menu for the review that you are about to publish (your selection is not “sticky” for subsequent publications). This approval level will be visible to others, and may affect the review completion requirements for both GitHub and Reviewable.
You may only change the approval level if you have write permissions to the repository and are not the author of the PR. As a repository admin you can also disable the Approve and Request changes for everyone via the custom review completion condition if they are apt to mess up your team's workflow.
If you choose Comment, any previous Approve or Request changes will remain in effect; you cannot rescind your vote, only change it.
Requested reviewers synchronization
Reviewable maintains its own list of people whose action is needed on a review (as shown on the dashboard and in the participants summary), independent of GitHub's requested reviewers list. You can choose to synchronize the latter with the former by checking the Sync requested reviewers box. Doing so will request reviews from Reviewable's awaited reviewers, and cancel requests for people who have left Reviewable's list. The option shows you what changes it will make in GitHub and you can always override it with ±reviewer inline directives.
It's not possible to request a review from the pull request's author in GitHub, nor from people who aren't collaborators on the repo, even if the user in question is on Reviewable's list of awaited reviewers. Only users with push permissions on the repo can request reviewers.
Keeping requested reviewers up-to-date (rather than just requesting the initial review) can improve integration with other tools, such as Pull Reminders.
Repository admins can customize the list of awaited reviewers and, if desired, override the Sync requested reviewers checkbox in a custom review completion condition. For example, you may wish to remove other users from the list if the PR author is on it, or force this option on for everyone to maintain a consistent workflow.
Review summary
For your convenience, Reviewable will provide a review summary panel at the end of the review so you don't have to scroll back to the top of the page to leave a comment in the top-level review discussion.
The review summary doesn't show up if the review was forced into one file at a time mode, due to too many files showing.
The draft has an LGTM button that you can use to approve the pull request. (See the repository settings for information on how to customize this button.)
Below the draft, there are counters of unreviewed files and unresolved discussions. A red counter indicates that there's still something for you to do: either review some files, or read / respond to some discussions. When appropriate, you can hover over a counter to reveal a button underneath that will mark files as reviewed or discussions as read.
Grey counters — even if non-zero — are just an indication of the review's state, but don't imply that any action is required from you at this time.
Merging a pull request
When a review is complete, a victory graphic appears and you can merge the pull request directly from within Reviewable given sufficient permissions.
A review is considered complete when the first defined condition of the following is true:
-
You've defined a custom review completion condition and it has returned a verdict with
completed: true(irrespective of any other per-file or per-discussion flags). -
You've turned on branch protection for the target branch in this repo in GitHub, and GitHub says that a merge is permitted.
-
Otherwise, a review is considered complete when GitHub reports that merging the branch won't cause any conflicts, all files have been marked reviewed by at least one person at the most recent revision, and all discussions have been resolved. (In this case, the toolbar will show a green Checks donut, a grey Changes revision, and grey zero counters in the remaining boxes.)
To merge the pull request, click and hold the button for a couple seconds until the arming indicator fills with yellow and starts pulsing, then release. This procedure is in place to reduce the chances of accidentally merging a pull request without requiring a separate confirmation.
You can set merge options and edit the merge commit message via the dropdown attached to the button:
Here you can select between the usual GitHub merge styles (normal/full, squash, and rebase), and whether Reviewable should automatically delete the source branch for you if the merge is successful. Your selections are automatically persisted for this review, and the selections you made last will be applied to any new reviews. A custom review completion condition can force the merge style to use.
When using the rebase merge style, Reviewable will indicate if the merge will be a fast-forward by adding a small annotation under the style radio button.
If you've selected the full or squash merge styles, you can edit the automatically generated merge commit message as well, or generate your own default in your custom review completion condition.
Review toolbar
At the top of every review page you’ll find a floating toolbar with some core state indicators and buttons for common actions.
At the end of the toolbar you'll always find either a Publish or Merge button, depending on the review's state. Note that if you have drafts pending, you'll always see the Publish button even if the pull request is otherwise mergeable. See the sections above for details on these two operations.
If you scroll down a bit, a bunny shortcut menu icon appears at the left of the toolbar.
Click the bunny shortcut menu icon to drop down a panel in which you can jump to a file or quickly edit the top-level draft. (Again, this menu icon only appears after you've scrolled down the page a bit.)
We'll now look at the functionality of the different features moving from left to right along the toolbar.
Checks
This item summarizes the current condition of GitHub's CI commit statuses and checks, mergeability, and review completion. The donut chart icon indicates the relative proportion of successful, pending, and error/fail states.
Click the item to open a panel with more details:
These state icons may appear in the panel:
If you're a repo admin you can click on the small wrench icon next to the current review state to jump to the custom review completion condition editor in the repository's settings.
Also, if your PR branch is behind the target branch, you can merge the target branch into your PR by clicking the small merge button, next to the mergeability status. Repo admins can disable this feature in a custom review completion condition.
Changes
This item summarizes the changes you're currently looking at. The revision label indicates the revision that is the current right diff bound for all files, or r?? if it's a mix. If the label is red, then you have reviewed all files in your current diff but there are more unreviewed revisions that remain. Click to open the panel, and then click Show Unreviewed Diffs. Or, adjust the diff bounds manually yourself.
Click the item to open a panel with more details:
This is similar to parts of the changes summary box, but also includes a list of commits currently in scope.
Counters
The next three items on the toolbar are counters for files, discussions, and drafts. Red counters indicate that you must address the given number of items to advance the review. Grey counters indicate that other participants must address the given number of items, but you're in the clear. Grey counters with a red stripe indicate that you've deferred reviewing files or responding to conversations until others have acted, but otherwise work just like a grey one.
The counters take into account your unsent drafts, so somebody else may see different numbers on the same review.
The files counter displays the number of files that remain to be reviewed at the current diff bounds, either by you or by others. Click to cycle between these files (default keyboard shorcut: n). You're free to disregard these suggestions, of course, but if you find yourself doing so often you may want to check the review settings in the Changes summary box or customize your review completion condition, which also controls the per-file reviewed state.
If you can’t get things to work the way you want, have a look at issue #404 for a more thorough exploration of “to review” semantics and suggestions for alternative command bindings.
The discussions counter display the number of discussions that are waiting for your reply or that are unresolved. Click to cycle between these discussions (default keyboard shortcuts: j for next unreplied, ⇧J for next unresolved).
The main general discussion is always considered resolved.
The drafts counter displays the number of drafts you have pending, and also turns red if you have any buffered state such as review marks, disposition changes, or acknowledgements. Click to cycle between your drafts. You can publish all of your drafts and other buffered changes by clicking the Publish button.
Changes summary
This panel appears at the very top of the review page. The top line describes the current diff set you're viewing as succinctly as possible to help orient you. There's also a link to the pull request in the top-right corner..
The panel is divided into three sections, and has some auxiliary file matrix controls at the bottom and may also have some unique styling if a review has been deferred.
Files
This shows the number of files currently in the review.
Any extra obsolete files have been modified at some point during the PR, but those are now the same as in the target base branch (so no further review of those files is necessary). They're usually hidden by default but you can reveal manually by clicking a small link next to the count. If you are in commit-by-commit review mode then obsolete files will still be selected for review for you, until you reach the commit at which they've become obsolete.
The Mark as Reviewed button will mark all files as reviewed—up to the current right-bound of the diffs. This doesn’t necessarily mean that you think the files are ready for merging, but rather only that you reviewed them and added all of your comments. After clicking this button, a small Undo link will appear beneath it for a while. (Incidentally, this is the only action in Reviewable that offers an explicit undo method of recovery.)
To mark files as reviewed individually, click the buttons to the left of the file name in the File Matrix, or in the file headers and at the bottom of unreviewed files.
Revisions
This shows the total number of revisions in the review. Each revision is an automatic, unmodifiable capture of one or more commits. You’ll find the commits assigned to a revision in the Review Discussion box, and also in the Changes drop-down at the very top of the page.
The logic for grouping commits into revisions depends on the review style, number of commits pushed at the same time, commit authors, etc. There are also some safety limits for how many revisions Reviewable will create at one time.
A provisional revision is tentative, since it may still change up to the point at which someone begins reviewing it. The intent behind provisional revisions is to permit the PR author to self-review the changes and push fixes without polluting the revision list. Provisional revisions are italicized in the file matrix.
An obsolete revision is one that is no longer part of the pull request due to a force push that changed the commit history. It will appear crossed out in the file matrix.
The Show Unreviewed Diffs button (exact wording varies) in this section will set the diff bounds on all the files to the next range that Reviewable thinks you need to examine. When you first load the review page, this button has in essence already been clicked for you — that is, the initial diffs will be what Reviewable thinks you should be looking at, not necessarily the ones that you were looking at on your last visit.
When applicable, you’ll find a small Show full diff link beneath the Show Unreviewed Diffs button that will show the full diffs between the base and the latest revision for each file. This will show you exactly the full deltas that will be applied if the PR is merged.
When there is more than one reviewer participating in the review, a dropdown with three review overlap strategy options will appear beneath the button. This setting changes which file diffs are suggested for the user to review and implicitly sets the default for any future reviews.
- Skip files claimed by others suggests files that have been previously reviewed by you, or that nobody has reviewed yet. Select this if you want to divide work by skipping files already reviewed by someone at an earlier version.
- Review any unreviewed files suggests files to review if they have not been reviewed by anyone at the current revision. Select this if you want to combine efforts and review any file that needs reviewing.
- Review all files personally suggests files you have not personally reviewed. Select this if you want to ensure that you review every file yourself, ignoring other reviewers.
Regardless of which option you select you will still be able to manually diff and review any file you want.
Commits
This shows the current number of commits encompassed by the pull request, along with the source (head) and target (base) branch names. You can easily copy the source branch name (e.g., to check out locally) or change the target branch of the PR if you have the necessary permissions. Click any of the other links here to open the corresponding GitHub page in the same tab.
The Review Style dropdown lets you choose the style of this review, affecting how commits are grouped into revisions and the suggested sequence of diffs to review.
There are two review styles, and changing the style will require from a few seconds to a minute or so to restructure the provisional revisions in the review.
- Combine commits for review — review commits that are grouped together according to the time at which they were pushed and a few other factors. Keep in mind that some commits might not be accessible for diffing.
- Review each commit separately — a revision is created for each commit, even if a successive commit wrote over previous changes. We recommend choosing this review style only if the commits have been carefully organized. Keep in mind that there are some built-in limits on how many revisions can be created together. This means that commits may get aggregated if those limits are exceeded. Please contact support to discuss raising the limits for your repos if you feel this would be useful.
If you're a repo admin, you can also set the default review style for the repo via a small link under the dropdown.
Snapshotted revisions won’t get restructured, so you may encounter surprising results if you switch the review style after beginning the review. An exception to this is the case in which a revision was snapshotted only because somebody other than the PR author looked at it, in which case it appears snapshotted but is OK for restructuring. The purpose of this is to enable a reviewer to switch the review style, since just loading the page will show the diff and snapshot the revisions.
Labels
Any labels or milestones that have been applied to the PR appear in the Review discussion footer.
Participants
The participants panel lets you quickly track the status of each individual that is involved in the review. Every information cell has contextual help with more details on its contents. Some cells may also have a dropdown menu with relevant actions that appears on hover. You can copy all usernames, or a subset of sorted usernames, by clicking the button. You can sort participants by each category using the button.
For each participant the following information is provided:
| Icon | Participant is: |
|---|---|
| is responsible for the next step of this review. | |
| is not being waited on in the review. | |
| has approved the pull request. | |
| has requested changes to the pull request. | |
 | says 'Everything looks good to me!' |
| is blocking a discussion in the review. | |
| has no approval-related status. | |
| is an author of the pull request. | |
| is a reviewer. | |
| is a requested reviewer. | |
| has only been @-mentioned in a discussion. | |
| is not directly participating in the review. | |
| is assigned to this pull request. | |
| has either authored or submitted a commit in this pull request. | |
| was not involved with commits in this pull request. | |
| is not yet active in the review. | |
| has pending comments drafted that are not yet published. | |
| has no pending drafts. | |
| is not engaged in any discussions. |
Waiting on
Reviewable keeps track of which participants are needed to move a review forward and marks them with a pointing hand. The list is roughly the union of the following:
- All participants involved in unresolved discussions that are unreplied for them.
- All participants who most recently marked as reviewed a file that is not reviewed at the latest revision.
- All requested reviewers, or if none all assigned users, or if none and there are files with no previous reviewers or discussions with no participants besides the pull request's author then all reviewers .
However, if a user deferred by publishing a review when some files or discussions were still unreviewed or unreplied, then they'll be removed from consideration for the waiting-on list until the review's state changes. Finally, if all files and discussions in the review have been engaged with but the list of waited-on users is still empty, Reviewable will default to the pull request's author. (This happens most often when a review has been completed and the pull request is ready to merge.)
You can see the algorithm above written out as code here and customize it to better fit your workflow.
Actions
Discussions
If you have the necessary permissions on the repo, you can open the actions dropdown for another active participant and click dismiss from all in the drop-down, which will dismiss the participant from all unresolved discussions they are engaged in. You can undo this action until you publish.
Deferring a review
When you publish a review and you have files left to review or discussions left to reply (red counters), the review will be automatically deferred for you: the red counters will turn grey with a red stripe and the review will not be awaiting your action. This lets you post a partial review — perhaps requesting some larger design changes or putting off reviewing test cases — and visibly hand off responsibility for taking the next action.
A review will remain deferred until either a new revision of a file becomes available for you to review or a new comment is posted. When this happens, the review will be reactivated for you with all counters going back to being red (including for files or discussions you had deferred), and the review awaiting your action once more.
Note that you can continue manipulating a review as usual while it's deferred, except that Reviewable won't suggest the next set of diffs to review. If you wish, you can also reactivate a deferred review manually by clicking the continue review now button on the review page.
Sending an individual comment (via its dedicated send button) doesn't affect deferrals either way: it will neither defer a review, nor cause a deferred review to become active again.
Reviewing commit messages
Reviewable composes a system file used for reviewing commit messages. The commit file is denoted with a special commit icon prepended to its name and will always be listed first in the files list (matrix and review). This file will contain a list of all non obsolete commit messages that precede it, regardless of whether you're reviewing with the combined commits or commit-by-commit review style. It behaves just like a normal file in that it will be diffed against the selected revision and supports discussions, however it will not impact your repo or PR in anyway -- it's solely used by Reviewable for reviewing commit messages.
While the commit file is virtual, it still needs to be marked as reviewed as much (or as little) as normal files. It's included in review file counters, but not counted in review status messages unless it's the only unreviewed file. It's also handled separately from normal files when evaluating a custom review completion condition; see custom completion condition for details.
Keyboard shortcuts
There are a number of keyboard shortcuts available to power your way through a review. Type ? to display a popup that lists the current bindings. Learn how to modify the available commands and the corresponding keyboard shortcuts in the Custom key bindings section.
