Run a pilot
Documentation Index
Fetch the complete documentation index at: https://docs.sourcery.ai/llms.txt
Use this file to discover all available pages before exploring further.
We want Sourcery to earn its place on your team, and a pilot is how you find out if it does. Spend two weeks running it on your real pull requests. You’ll see whether it catches bugs and security issues your team would have missed, and whether code review gets faster.
Ask for help if you want it
Section titled “Ask for help if you want it”In-app chat is on every dashboard page, and during a pilot it’s the fastest way to reach us. If a finding looks wrong or you can’t find a setting, ask there. You can also email support@sourcery.ai.
Set up in the first few days
Section titled “Set up in the first few days”Turn on reviews and scans
Section titled “Turn on reviews and scans”Pick two to five repositories that get plenty of pull requests, and include one codebase the team knows well. Aim for at least 50 reviewed pull requests by the end of the pilot. The final numbers need that much volume to mean anything. If the repositories you picked won’t get there, add a few more. Walk through the quickstart for your platform to connect them, and Sourcery reviews every new pull request from then on.
Start a security scan too. Open Repositories and click Start Scan.
Bring in the team
Section titled “Bring in the team”Everyone in your organization has access during the trial. Bring in the engineers who open and review pull requests in the pilot repositories, plus a security lead for the scan findings.
Ask everyone to react to Sourcery’s comments with a thumbs up or down. Reactions teach Sourcery which comments the team wants, and they feed the comment-quality number you’ll read later.
Configure reviews to fit your team
Section titled “Configure reviews to fit your team”Sourcery reviews sensibly out of the box, and you can shape it further. Open Review Settings to choose what it posts and which pull requests it reviews, and set them up how you like. Recurring conventions can go in as review rules. Do this at the start, so the results reflect how your team actually runs Sourcery.
Read your stats at the end
Section titled “Read your stats at the end”By then the dashboards have enough to read. Two weeks is a short window, though, so the numbers are still forming. Read them for direction and rough size.
Code reviews
Section titled “Code reviews”The Analytics page holds the code-review numbers. It’s available for GitHub; if you’re piloting on GitLab, read the code-review results from the reviews on the merge requests themselves.
Sourcery Acceptance Rate, on the Overview tab, is the share of Sourcery’s comments that developers acted on, meaning the comment led to a code change. It shows whether the comments are worth acting on, and it’s the number you’d otherwise have counted by hand.
Sourcery Comment Quality, on the Code Reviews tab, is the thumbs-up ratio from the reactions the team collected.
Median Cycle Time and Time to First Review, both on the Overview tab, show whether review got faster. Each shows the change against the previous period.
Security findings
Section titled “Security findings”The Security Reports page covers the scan, on GitHub and GitLab alike. The Risk Impact tab shows Open Issues and the breakdown by severity, from Critical down to Low. The Trends tab shows New against Resolved, the findings that came in during the pilot and the ones that got fixed.
The report counts findings, but it can’t tell you which ones are false positives. Take a sample from the codebase you know best and judge for yourself which findings are real.
Check in with your teammates
Section titled “Check in with your teammates”The dashboards don’t tell you everything. Near the end of the pilot, ask the engineers whether the comments were worth reading and whether the reactions habit stuck. If the team wants to keep Sourcery, that tells you as much as any number.
Optional extras
Section titled “Optional extras”If Jira is your tracker, add Jira autocreate to the pilot. If your engineers fix issues with coding agents, try agent fix prompts.
What it costs
Section titled “What it costs”Costs shouldn’t be a surprise at the end of a pilot. Pricing is public on the plans page.