Custom Rules

Besides following general best practices, you can also define rules specific to your codebase.

You can define your own project-specific rules in your Sourcery config file in yaml format.

Fields Overview

Fields marked with * are required.

Field	Type	Description
id*	string	Unique, descriptive identifier
pattern*	string	Pattern syntax to search for in your code
description*	string	A description of why this rule triggered and how to resolve it
condition	string	Constraints on the pattern or its matches
replacement	string	Optional code to replace any search matches
explanation	string	Optional Markdown text to provide more info
tags	list of string	A list of tags to describe and group the rule
paths	object	Paths to include or exclude when running this rule
tests	list of object	Tests to run and check if the rule is correct

Complete Example

Using the following two rules, Sourcery will:

• replace assert statements with conditional raises of AssertionError
• flag any class definitions inheriting from Exception classes that are not named Error

rules:
  - id: no-assert-statements
    pattern: assert ${condition}, ${explanation?}
    replacement: |
      if ${condition}:
          raise AssertionError(${explanation})
    description: Do not use assert statements, except for tests.
    tags:
      - test-rules
    paths:
      exclude:
        - '**/test_*.py'
    tests:
      - match: |
          assert isinstance(value, Example)
      - match: |
          assert key in my_dict, f"{key} not in dict"
  - id: errors-named-error
    pattern: |
      class ${error}(${base}):
        ${statements*}
    condition: |
      (base.is_exception_type() or base.matches_regex("[A-Z][a-zA-Z]*Error")) and not error.matches_regex("[A-Z][a-zA-Z]*Error")
    description: |
      Exception names must end in Error
    explanation: |
      From Google Style Guide [2.4.4](https://google.github.io/styleguide/pyguide.html#244-decision)
    tags:
      - exception-rules
    tests:
      - match: |
          class Foo(ValueError):
            ...
      - match: |
          class ExampleException(CustomError):
            def __init__(self, msg):
              ...
      - match: |
          class InvalidName(Exception):
            ...
      - match: |
          class InvalidName(BaseException):
            ...
      - no-match: |
          class MyError(Exception):
            def __init__(self, msg):
              ...
      - no-match: |
          class NameError(AttributeError):
            ...
      - no-match: |
          class Dog(Mammal):
            ...

See Also

• Tutorial: Writing Custom Rules
• Reference: Pattern Syntax

Fields

id

required string

A unique, descriptive identifier for the rule.

The ID must be unique. It can't collide with:

• other custom rules' IDs
• Sourcery default rules' IDs
• Sourcery optional rules' IDs
• Sourcery JavaScript rules' IDs

Example

In the following example, the non-required fields have been omitted for clarity.

rules:
  - id: remove-debug-logs
    pattern: log.debug(...)
    description: Remove debug logs
    replacement: ""

pattern

required string

A code snippet matching undesirable code, written using Sourcery's Pattern Syntax.

Examples

In all of these examples, fields other than pattern have been omitted for clarity.

Using this example, Sourcery will search for code exactly matching the phrase raise NotImplemented.

rules:
  - id: do-not-raise-notimplemented
    pattern: raise NotImplemented
    replacement: raise NotImplementedError
    description: Do not raise NotImplemented

Using the next example, Sourcery will match any function definition named "get".

rules:
  - id: find-get-functions
    pattern: |
      def get(...):
        ...
    description: Find `get` functions

Using the next example, Sourcery will match any class inheriting from Exception.

rules:
  - id: find-exception-subclasses
    pattern: |
      class ${cls}(Exception):
        ...
    description: Find `Exception` subclasses

See Also

• Tutorial: Writing Custom Rules
• Reference: Pattern Syntax

description

required string

The description is displayed in the plugins and in the command line.

Example

rules:
  - id: do-not-assign-to-self
    pattern: ${var} = ${var}
    description: Variables should not be assigned to themselves

Extra: Using Captured Names

Descriptions may contain captured names from the pattern field. (See capture naming for reference on this feature.) When the rule description is displayed, the placeholders will be replaced by the contents of their captures.

For example, the following rule

rules:
  - id: do-not-assign-to-self
    pattern: ${var} = ${var}
    description: Variable "${var}" should not be assigned to itself

when applied to the snippet

x = x

will show a comment with the description Variable "x" should not be assigned to itself

Multiple captures are supported:

rules:
  - id: do-not-assign-to-self
    pattern: '${var}: ${ann?} = ${var}'
    description: |
      Variable "${var}" of type "${ann}" should not be assigned to itself.
      Assign "${var}" to something else.

This rule, when applied to

x: int = x

will show the description

Variable "x" of type "int" should not be assigned to itself.
Assign "x" to something else.

In addition, if applied to the snippet

x = x

the same rule would show as description

Variable "x" of type "<no-match>" should not be assigned to itself.
Assign "x" to something else.

because there is no match for the ann capture.

condition

string

The condition is used to restrict pattern matches according to boolean logic. The condition is written using a Python-like syntax based on methods of the names captured in the pattern. (See capture naming for reference on this feature.)

Default

None (unconditional match)

Example

rule:
  - id: no-global-variables
    pattern: ${var} = ${value}
    condition: |
      var.in_module_scope()
        and not var.is_upper_case()
        and not var.starts_with("_")
    description: Don't declare `${var}` as a global variable

See Also

• Reference: Condition Syntax
• Tutorial: Using Conditions in Custom Rules

replacement

string

The replacement guides Sourcery's behaviour when it finds some code that matches the pattern:

• If your rule contains a replacement: Sourcery will fix the issue.
• If your rule doesn't contain a replacement: Sourcery will only flag the issue.

The replacement needs to be valid Python code. It can refer to the captures declared by the pattern, but can't declare new captures.

Default

None (no replacement)

Example

rules:
  - id: simplify-range-generator
    pattern: (${i} for ${i} in range(...))
    replacement: range(...)
    description: Simplify a generator expression over a range by using the range itself

Extra: Removing Code with an Empty Replacement

The replacement can also be an empty string. In that case, Sourcery deletes any code that matches the pattern.

For example:

rules:
  - id: remove_breakpoint
    description: Remove breakpoints from production code
    pattern: breakpoint()
    replacement: ''

If a code block contained only the code matching the pattern, applying such a rule would remove the whole content of that block. In such cases, Sourcery will replace the rule with a pass statement. Further refactorings may then remove the whole block, if it's considered redundant.

if improbable_condition():
    pass
continue_logic()

explanation

string

The explanation provides background information to the rule.

The explanation is displayed in both the VSCode and JetBrains plugins. They both support the Markdown format: feel free to add code snippets, links, listings to your explanations.

A rule with an explanation shown in VSCode.

Default

None (no explanation)

Example

rules:
  - id: do-not-raise-notimplemented
    pattern: raise NotImplemented
    description: Do not raise NotImplemented
    explanation: |
      `NotImplemented` is reserved as a _return_ type for invalid binary operations.
      Perhaps you meant to `raise NotImplementedError`, which is used to indicate
      that a method or function has not yet been implemented.

tags

list of string

The tags provide a way of grouping rules together. This allows you to easily run groups of rules together, or exclude them using the --enable and --disable options of the CLI. Note that all of Sourcery's default rules have the default tag applied to them, so you can run only your custom rules by excluding this tag.

Default

None (no tags)

Example

rules:
  - id: do-not-raise-notimplemented
    pattern: raise NotImplemented
    description: Do not raise NotImplemented
    explanation: |
      `NotImplemented` is reserved as a _return_ type for invalid binary operations.
      Perhaps you meant to `raise NotImplementedError`, which is used to indicate
      that a method or function has not yet been implemented.
    tags:
     - common-gotchas

paths

optional object

The paths field defines where in your project directory the rule will be applied. It specifies two subfields:

• include: Run this rule only on the specified paths.
• exclude: Run this rule everywhere except the specified paths.

Wildcard patterns (glob style) like test_*.pyare supported.

Recursive globs using ** are not supported

Default

None (Sourcery will run the rule over all files in the project directory)

Example

In the following example, all assert statements in the app directory will be flagged, unless they're in a test file.

rules:
  - id: no-asserts
    pattern: assert ${condition}, ${message}
    description: Do not use assertions in application code
    paths:
      include:
        - app
      exclude:
        - app/*/test_*.py

Extra: More Examples

To ignore specific files for a rule, set the paths.exclude key:

rules:
  - id: no-asserts
    pattern: assert ${condition}, ${message?}
    description: Don't use asserts in app code
    paths:
      exclude:
        - setup.py
        - test/
        - benchmark/*/*.py

The above rule will run on all files except:

• The setup.py file
• Any files within the test directory
• Any file matching the benchmark/*/*.py glob syntax

If you only want to run a rule on specific paths, use paths.include:

rules:
  - id: no-asserts
    pattern: import app
    description: Don't import app into lib directory
    paths:
      include:
        - lib

If both include and exclude are used, exclude takes precedence over include.

rules:
  - id: hello-world
    pattern: print("Hello world")
    description: Include an exclamation in "Hello world!"
    paths:
      include:
        - test_*.py
        - test/
      exclude:
        - test_abc.py
        - test/app

These files will be excluded by exclude even though they match include:

• test/app/test_def.py
• test_abc.py

These files will be included by include:

• test_def.py
• test/abc.py

These files are not included because they don't match include

• test/ternary/test_abc.py
• abc.py

tests

list of object

A list of tests to validate the correctness of the rule. When the .sourcery.yaml file is loaded, the tests are run as part of a validation step. If any of the tests in your rules fail, Sourcery will not run.

Each test is either:

• A positive match, containing the match subfield and an optional expect subfield
• A negative match, containing the no-match subfield

The match and no-match subfields should contain valid Python code.

Default

(Empty list)

Example

rules:
  - id: logs-should-contain-extra
    pattern: log.${method}(${arg})
    description: Log methods should contain the `extra` field for analytics purposes
    tests:
      - match: |
          log.info("This log does not have extra")
      - no-match: |
          log.info("This log does have extra", extra={"protocol": "http"})

See Also

• Tutorial: Writing Tests for Rules

Tests in the IDE

If you're using a Sourcery plugin for an IDE, it will highlight failing tests in the .sourcery.yaml file.

Related Docs

• Guides: Custom Rule Recipes
• Tutorial: Writing Custom Rules