A repo with the English translation of each of the rules files, using Google Translate: <a href="https://github.com/pramodbiligiri/open-code-review-rules" rel="nofollow">https://github.com/pramodbiligiri/open-code-review-rules</a>.<p>The original rules files (in Chinese): <a href="https://github.com/alibaba/open-code-review/tree/main/internal/config/rules/rule_docs" rel="nofollow">https://github.com/alibaba/open-code-review/tree/main/intern...</a>
Ran it on a subset of 10 of the 50 PRs in this benchmark <a href="https://codereview.withmartian.com" rel="nofollow">https://codereview.withmartian.com</a><p>- very good recall (~74%, e.g. found a lot of the golden issues)<p>- not so good precision (~12%, e.g. lots of false positives)<p>- the precision causes the F1 to tank (~20%, if this stays the same on the full 50 sample it would puts it almost last, even less than Kilo+Grok)
Rule files are in <a href="https://github.com/alibaba/open-code-review/tree/main/internal/config/rules/rule_docs" rel="nofollow">https://github.com/alibaba/open-code-review/tree/main/intern...</a> (in Chinese)
I like the pattern of making a dedicated cli/harness and just build a skill to teach coding agents to use it.<p>At $work we built a thorough workflow to do security reviews, which is a pure skill to simplify adoption <a href="https://www.synthesia.io/post/automating-code-security-reviews-with-claude-mythos-level-capabilities" rel="nofollow">https://www.synthesia.io/post/automating-code-security-revie...</a><p>But the user experience is tricky because if we aim for very low false positives the run time for this kind of workflows is too long, it's then hard to justify blocking PRs.
If you've codex what does it add over codex's default app? I am confused. Can't you simply ask codex in another tab to just do a code review?
Developers should definitely use whatever tool they use to review the code they (or the tool) just wrote. We have a skill that does this in a loop - spin subagents, review (based on our coding standards), triage the review in another subagent, fix what's applicable, push back on what's not, and we run this in a loop. This is before you even open a PR.<p>The idea of a PR is for others to find things that you have a blind spot to, and also leave some paper trail on the thought process. E.g. if something was not fixed, there is a history of a comment and a reason on WHY it wasn't fixed. If you do all that only locally, that context is lost.<p>We noticed that even after doing this self review loop multiple times, we still find issues (either via other models / tools or via humans that have the "tribal knowledge")<p>Maybe one day AI will write perfect code and can review itself, but even if it's 0.1% chance it has a bug, or 1 in a million it will do something a bit sinister (like open a backdoor just in case you try to shut it down) - then I really think there is always going to be a need for humans to review something.
Mechanics of running their command aside, I think the main value add is all the rules: <a href="https://github.com/alibaba/open-code-review/tree/main/internal/config/rules/rule_docs" rel="nofollow">https://github.com/alibaba/open-code-review/tree/main/intern...</a><p>Like with "SKILL" files in general, it's got to do with Prompt Engineering: <a href="https://en.wikipedia.org/wiki/Prompt_engineering#Rationale" rel="nofollow">https://en.wikipedia.org/wiki/Prompt_engineering#Rationale</a>
> Can't you simply ask codex in another tab to just do a code review?<p>You are likely to get better results if you do not use the same model for review that wrote the code. I typically use Opus for code editing and GPT 5.5 for peer review using an automation with skills.<p>Training set is different between models. If there are gaps in coverage in one model, you want a different model reviewing the work. The second model will its own gaps, but the gap list is not identical.
> You are likely to get better results if you do not use the same model for review that wrote the code<p>There’s no evidence of this. I guess you are anthropomorphising models (i.e., it’s good that - different human reviews your code)
Yeah, one model over another seems to matter less, they respond differently to the same prompts, so if anything, I'd use multiple prompts over choosing one model over another.<p>However, using two models to generate two reviews easily beats doing one model and one review, as some models seem to "care" more about certain things, but you'll just miss different things if you change the model rather than add more.
Results also depend on the prompt. You get different results if you ask to review the PR and focus on particular file than if you don't make it focus.<p>Or if you make it "be a security engineer" with particular focus points.<p>Or make it a grammar nazi, it will find way more typos than without such focus.<p>Of course all of those "focuses" needs to be in a separate context (agent/subagent) to make it work.
I would suggest that you reverse those roles. gpt-5.5 as the implementer and Opus as the reviewer.
They find different things, and there's no reason to use one model for review. You want to review it until there's nothing left to be unearth.<p>And if you put the review effort into polishing an impl plan, then it doesn't matter which model implements it either.
How come? I find Opus to have better taste and GPT to have more rigor.
Presumably nothing. Do note the publisher—Alibaba presumably would rather their own tools and models instead of licensing.<p>They do open source a fair bit of internal tooling, so it’s always interesting to see their approach
It can be used outside of local machine.<p>We built something similar, it looks for new PRs where the bot is added and does reviews. Makes the code more tuned toward similar rules. I can't assume that a developer run a code review tool himself (just as I don't assume he/she run a build - so we run builds also).<p>It is just another perspective for code review, besides human. Unfortunately it uses a lot of tokens, and considering that Anthropic, OpenAI and Github Copilot all moved to token based pricing, it is quite a money burner.
We'd need a benchmark to tell.
At a kill s@@s hackathon at work, I was able to build something that<p>uses a node image
installs claude code
runs a /review-like command
puts inline comments to PR
deletes old comments when rerunning<p>OCR seems cool, but overkill, and I'm definitely not using Code Rabbit after their CEO was on here acting snobbish a while back.<p>Point being AI code review in Git** itself isn't hard to do and can add a lot of value quickly.
Nothing against coderabbit or SaaS specifically, but this was one of the reasons I stopped using it <a href="https://kudelskisecurity.com/research/how-we-exploited-coderabbit-from-a-simple-pr-to-rce-and-write-access-on-1m-repositories" rel="nofollow">https://kudelskisecurity.com/research/how-we-exploited-coder...</a><p>It's very easy to build a basic code review tool.
It's hard to build one that developers won't ask you to turn off because of false positives (or one that will miss your next escaped bug)<p>I think if all the tool does is run a claude code level /review skill (which all developers should definitely run before they even open a PR) then isn't this a bit of a review theater? Just a guardrail to those developers who don't run a /review-triage-fix skill in /loop before they take the PR out of draft?<p>I wonder how many PRs in the world got to production where several developers commented on each other's code, and none of them read anything, just used their gh cli / MCP to post / answer comments / fix issues on their behalf.<p>There is going to be an exponential growth of code generated, and you can't escape AI code review, but also there is no real difference between having Claude Code write the code and review itself locally, vs communicating with itself via a slow and downtime prone medium of "PR comments"<p>tl;dr - without any human in the loop reviewing the AI code review, or skimming to see what the AI code review missed, there is no real reason to use a "code review" you can just run it as part of the CI/CD and hope AI won't miss anything (according to my linkedin feed, there are people out there who really thing this way...)
I think that in most cases you either agree on a PR comment or you don't. But it has to leave a mark in PR. This is how we do reviews, ignoring PR comment is one of the worst offenses one can make. I don't let it go.
Yes! Where it gets really interesting is the scenario in which every developer has their own unique review skill/workflow, so the reviews end up being different than you running it yourself, but nobody is reading them still.
How snobbish was the CEO acting?
I'm interested in trying this.<p>We have our own internal automated review which has shown positive results, but I would love to drop it if I find something better.<p>Code review is currently our bottleneck, so any possibility of better automating it is welcome.
Thermonuclear suggested by someone below is good. Matt Poccock did a demo/breakdown of that: <a href="https://www.youtube.com/watch?v=mh5XZ-L5SFQ" rel="nofollow">https://www.youtube.com/watch?v=mh5XZ-L5SFQ</a>. He has his own "improve-codebase-architecture" skill: <a href="https://github.com/mattpocock/skills/blob/main/skills/engineering/improve-codebase-architecture/SKILL.md" rel="nofollow">https://github.com/mattpocock/skills/blob/main/skills/engine...</a><p>Some of them are about general coding guidelines and code quality, not necessarily vetting your current PR against specs! There's AbsolutelySkilled with clean-code and clean-architecture. Linking to older version of repo because they seem to be no longer on trunk: <a href="https://github.com/AbsolutelySkilled/AbsolutelySkilled/tree/8f704b39e2c652ab199d3c4d6b6cb709426f692c/skills" rel="nofollow">https://github.com/AbsolutelySkilled/AbsolutelySkilled/tree/...</a><p>I've been creating some rules to help with my Java coding:
<a href="https://github.com/bitkentech/shipsmooth/tree/main/skills/experimental/refine/rules" rel="nofollow">https://github.com/bitkentech/shipsmooth/tree/main/skills/ex...</a>. These are assembled into a SKILL file when this skill file template is built: <a href="https://github.com/bitkentech/shipsmooth/blob/main/skills/experimental/refine/SKILL.jte.md" rel="nofollow">https://github.com/bitkentech/shipsmooth/blob/main/skills/ex...</a>
I've been liking this code review skill lately, it has pointed out some good improvements. <a href="https://github.com/cursor/plugins/blob/main/cursor-team-kit/skills/thermo-nuclear-code-quality-review/SKILL.md" rel="nofollow">https://github.com/cursor/plugins/blob/main/cursor-team-kit/...</a>
[flagged]
this is a great tool, until you try reading the rule files, I had find a translator to make heads of it. given that it is CLI tool is great dev the tinker with it at no additional cost.
i did something like this, but somewhat in reverse. you are the one that reviews the code and you instruct AI what to do through code review comments: <a href="https://parley.cloudflavor.io" rel="nofollow">https://parley.cloudflavor.io</a>.<p>thinking about it, it would be funny to first run alibaba's tool and then run parley after.<p>posted it here a few days ago: <a href="https://news.ycombinator.com/item?id=48369782">https://news.ycombinator.com/item?id=48369782</a>
i guess with AI there are too many Show HN now, and i never got any type of feedback.
Just a small note, the font on your site is very annoying to read, the characters are not aligned horizontally (Windows w Chrome). Looks to be a scaling issue, if I zoom to 200% it shows fine.
Is not working with gpt5.x models (Unsupported parameter: 'max_tokens' is not supported with this model. Use 'max_completion_tokens' instead.) which is hardcoded. I dont know why this is on the front page. My review-with-codex skill is working just fine, consuming my usage and not API tokens.
> After installation, the ocr command is available globally.<p>Wish they chose a different acronym...
We've been using Coderabbit, great deal ($30/mo/dev flat) and finds a lot.<p>I also built a skill I call `/meta-review` that asks Codex, Cursor, and Gemini to review the code (I use Claude Code). It always finds little things claude & I missed.<p>Coderabbit just came out with their own PR review UI that's great for big PRs, it groups files together etc. <a href="https://www.coderabbit.ai/blog/introducing-atlas-the-first-ai-native-code-review-interface" rel="nofollow">https://www.coderabbit.ai/blog/introducing-atlas-the-first-a...</a>
Not sure why you got downvoted, and I have nothing against CodeRabbit, but this comment feels a bit like a paid ad :)<p>How do you see CodeRabbit against other AI code review solutions? E.g. cubic.dev, Qodo, Graphite, Greptile, Baz, Augment Code...<p>An alternative UI to GitHub is well overdue. But once someone will get it right, everyone will copy them...
Is it actually flat fee? I loved Cursor bugbot which was flat fee but they moved to per-run and that killed it for me, but a lot of others are doing the same.
I've tried many AI code review tools. Nothing comes close to the depth of CodeRabbit reviews. It's the only such tool that can find real logical bugs. I'd love to be able to get Claude Code to do similar quality of review, but I can't get it right, no matter how I try.
I wonder how they do against this benchmark (not that I vetted this benchmark... but still interesting to know...)<p><a href="https://codereview.withmartian.com" rel="nofollow">https://codereview.withmartian.com</a>
I recently moved off Cursor's BugBot because it's no longer a flat $40, and I feel a little lost trying to find a viable alternative because there are so many and the pricing kind of sucks for all of them. Curious if anyone has a recommendation.
My team tried coderabbit and qodo and they are both trash compared to a tool we quickly built in-house that is more or less a thin wrapper around claude/codex, along with per-repo skills. PR review is triggered by webhooks from github to the review tool's web app. The tool shared by OP from alibaba certainly does some things ours does not and appears more sophisticated, but we have never had the problems they mention.<p>"The agent can read full file contents, search the codebase, inspect other changed files for context, and produce deep reviews — not just surface-level diff feedback." our tool does all this too. It catches dumb typos as well as more complicated bugs. Not to mention it is great as a ratchet (<a href="https://qntm.org/ratchet" rel="nofollow">https://qntm.org/ratchet</a>). It is not a substitute for reviews from other engineers though, since obviously it does nothing to achieve one of the main goals of code review, which is to socialize knowledge of the codebase.<p>Alibaba's work here is almost certainly more advanced than what we've done, but ours has been perfectly satisfactory and better than the paid offerings we've tried. I think <i>most</i> teams should not be paying SaaS fees for AI code review, that is the kind of business that mostly should not exist any more.
gitar.ai is flat with no limits
[flagged]
[flagged]
[flagged]
[flagged]
[dead]