Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

linter: nested file configuration #7408

Open
camchenry opened this issue Nov 21, 2024 · 6 comments
Open

linter: nested file configuration #7408

camchenry opened this issue Nov 21, 2024 · 6 comments
Assignees
Labels
A-linter Area - Linter

Comments

@camchenry
Copy link
Contributor

camchenry commented Nov 21, 2024

This is the tracking issue for nested file configuration in oxlint.

@camchenry camchenry added the A-linter Area - Linter label Nov 21, 2024
@camchenry camchenry added this to the Oxlint Beta Milestone milestone Nov 21, 2024
@camchenry camchenry self-assigned this Nov 21, 2024
@camchenry
Copy link
Contributor Author

I did a bit of testing and thinking on this today. As it stands, these are my current thoughts on what we need to implement.

  1. Auto-detection:
    • When no config file is passed, we will automatically search for configuration files. (Linter should detect configuration file(s) if not provided #7101)
    • Otherwise, if a config file is passed via -c/--config, then we will NOT automatically search for configuration files in the current directory and below. This means that the single config file will be used for linting the entire project (which is the current behavior already).
      • Perhaps we should add a --no-hierarchy-config (example name) option that forcefully disables the config searching, to improve performance. May be unnecessary though if we support flat config.
    • FUTURE: When we choose to support ESLint V9 flat config in some capacity, then we should detect if the flat format is used and use this to automatically disable hierarchical config searching.
  2. Storage
    • I am not completely certain on how we should optimally store the configurations in memory for performance, but in general we will need to:
      • Create an Oxlintrc instance for every discovered config file
      • Create a ConfigStore for each root configuration
      • Merge/clone a ConfigStore for each directory containing a config file
    • Ultimately, despite hierarchy, we must be able to resolve the configuration completely for a given directory so we can print it out.
  3. Compatibility
    • I would propose that we standardize on a single expected file name for nested configurations. I don't have a strong preference, but I think that oxlintrc.json would be fine. This means that we would not automatically support .eslintrc.json, but it would make it simpler/faster to automatically search for
    • FUTURE: When we support V9 flat config, we could standardize on oxlint.config.json being the standard configuration file name. If oxlint.config.json exists at the root level, then we should disable nested config searching. Otherwise, if we find oxlintrc.json, then we will search for nested configs, unless only a single config file was passed.

@Boshen
Copy link
Member

Boshen commented Nov 23, 2024

@Boshen Boshen mentioned this issue Nov 23, 2024
@camchenry
Copy link
Contributor Author

camchenry commented Dec 3, 2024

This is a brief informal description of the nested file configuration specification for oxlint. It is primarily based on ESLint, but with some inspiration taken from the https://docs.astral.sh/ruff/configuration/#config-file-discovery as well. This should be documented officially on the website later

Decisions

These are the main decisions / important points:

  • oxlint will only search automatically for files named .oxlintrc.json
  • oxlint will automatically use the closest configuration file (see configuration file resolution below for what this means)
  • configuration files can be shared by using the extends property in .oxlintrc.json
  • configuration files will not automatically extend configurations in parent directories (unlike ESLint)
  • if a single configuration file is passed in to the CLI (via --config), it will be used to lint all files and subdirectories and turn off the automatic configuration searching and resolution.

Context

In large monorepositories (repositories containing multiple packages/projects), it is convenient for each project to have its own configuration files, rather than having a single shared file. This makes it easy to clone or work out of just a subdirectory and treat it as its own independent project. This means that each project directory should have its own .oxlint.json file at the root level. For example:

project-root/
  .oxlintrc.json
  package1/
    .oxlintrc.json
  package2/
    .oxlintrc.json  

However, it is not possible to lint the entire project-root directory at once, because we currently only search in the current working directory for a configuration file. So, we need to search each directory for a .oxlintrc.json file and use that as the configuration for linting. However, we then need to decide on how to handle multiple .oxlintrc.json files such as if there is one in the project root, and then another one in a subdirectory.

Resolving configuration files

In ESLint, configuration files are resolved by using the file in the current directory or nearest parent directory as the base configuration file, and then subsequently merging that with each file in each parent directory, up to the root directory. The base configuration file takes precedence over any parent configurations.

However, this can create situations where you have more rules enabled than you want, such as rules that shouldn't apply for test files, or rules for packages you aren't using in a subdirectory. ESLint solves this by supporting adding a root: true property to the configuration file, which indicates that it should not find and merge configuration files in parent directories. You can still extend other configuration files though.

In oxlint, we will not automatically search upwards in the file hierarchy for configuration files. We will only use the closest configuration file to the file currently being linted. Every configuration file will essentially be treated as if it already contained root: true. This makes configuration resolution faster, because we don't need to do any further searching once we've find the closest configuration file, and chances are that we've already parsed it and cached it, since it's in a parent directory. This also makes it easier for developers to understand how the final linting configuration is formed, because you can always just find the

In priority order, the "nearest" configuration should be defined as:

  1. .oxlintrc.json in the same directory as a file
  2. .oxlintrc.json in a parent directory of a file
  3. .oxlintrc.json in the root directory (or current working directory of the process)
  4. The default oxlint configuration

However, not automatically merging configuration files means that it becomes harder to share configuration across directories. So, we need to define a way to share lint configurations.

Extending configuration files

In ESLint, the extends property can be used in the configuration file to specify other configurations that should be merged with the current file to resolve the final configuration.

Oxlint will also support the extends property, but with reduced functionality initially, to make implementation easier. The extends property in a .oxlint.json is either a string or a list of strings representing paths to other JSON configuration files. For example: this is a configuration file using extends (where ../.oxlint.json, ../.config/react-oxlint.json, and ts-oxlint.json are all valid oxlint configuration files. )

{
  "extends": [
    "../.oxlint.json",
    "../.config/react-oxlint.json",
    "ts-oxlint.json"
  ],
  "rules": {
    ...
  }
}

The configuration file paths are resolved relative to the base configuration file. When extending other files, some general rules apply:

  1. Configuration files are merged in the written order, and later configuration files take precedence over earlier ones.
    1. For example: If the first extended config file enables a rule, and then a later extended config file disables that rule, then the rule is disabled.
  2. The base configuration file has precedence over any extended file.
    1. For example: turning off a rule or plugin in the configuration file will always disable the rule or plugin, regardless of which other files are extended.
  3. The files can be named anything and be located anywhere, as long as they are valid JSON and conform to the oxlint configuration file format
  4. Extended configuration files can also contain the extends property. The final resolved configuration for that file is what is used for merging.

Other details and context

How are CLI arguments resolved?

In general, CLI arguments take precedence over the final resolved configuration. For example, if the .oxlintrc.json file enables the no-console rule, but the CLI is passed as oxlint -D no-console, then the rule will be turned off.

What is the precedence of configuration resolution?

This is a list of the general precedence of configuration, in order from highest precedence to lowest:

  1. Command line arguments (example: oxlint -D no-console)
  2. Configuration file from command line (example: oxlint --config .oxlintrc.json)
  3. Configuration file in the same directory as file being linted
  4. Configuration file in a parent directory of file being linted (first one found is used)
  5. Configuration file in the current working directory (that is, where the process was spawned from)
  6. Default oxlint configuration

How does --config work with nested config files?

If --config is passed, then that file will be used to lint all files in the project, including subdirectories with .oxlintrc.json. CLI arguments can still override settings in this

Why .oxlintrc.json?

ESLint supports many different configuration file names and formats. This makes it more convenient for developers, but complicates implementations as the linter needs to include a JS engine (such as Node), a JSON parser, a YAML parser, support ES modules, and do lots of extra work for convenience.

The format and naming of the config file for oxlint as .oxlintrc.json is chosen for simplicity and convenience:

  • There is only one supported format: JSON. This makes the tool faster and leaner as we don't need to support other formats or evaluate JS code to determine the final configuration. For convenience, this is also the most popular format for configuring ESLint, so it should be familiar to most developers.
  • There is only one name. oxlint can be slightly faster (compared to ESLint) because it only needs to search for one file name. The name .oxlintrc.json is chosen because it is close to the most common ESLint config files which are .eslintrc or .eslintrc.js (or other variant), so it should be familiar and hopefully easier to remember.

The actual name and format are not important and we could always change the name later if we decide it's better.

@yisibl
Copy link

yisibl commented Dec 3, 2024

It would be nice if Oxlint could provide a migration program to convert YAML configuration(or other) files to JSON

@GiladShoham
Copy link

Hi,

First off, I want to express how much I appreciate the thought and effort that went into making this decision. It's great to see the structured approach taken here!

I do, however, have a few pieces of feedback that I think could help refine the priority order for configuration resolution:

1. Priority Order Adjustment

You’ve mentioned the priority order as follows:

  • .oxlintrc.json in the root directory (or current working directory of the process)
  • .oxlintrc.json in the same directory as the file
  • .oxlintrc.json in a parent directory of the file
  • The default oxlint configuration

Based on my experience with mono-repos across many organizations, I’d suggest adjusting this order. Specifically, I recommend prioritizing the current working directory (CWD) configuration after .oxlintrc.json in a parent directory of a file (essentially moving it from 1 to 3).

The reason is that in large mono-repos, it's common to run scripts from the root of the repository that are meant to target specific sub-projects. If the root config takes precedence, it can unintentionally override the configuration intended for a specific project.

For instance, when using tools like Bit, the CWD is often the root, but the script is executed for a sub-project. In such cases, prioritizing the CWD would lead to the scenario you’ve mentioned: “more rules enabled than desired.”

2. Explicit Configuration (--config <path>)

While it’s indirectly mentioned, I think it’s worth explicitly stating in the priority list (or at least in the final documentation) that if a --config <path> flag is passed, it should take absolute precedence over all other configurations. Making this explicit would avoid any ambiguity and align with the principle of direct overrides.

3. Extending Configurations and Overrides

It would be helpful to clarify how more complex configurations, such as those involving plugins or nested extensions, are resolved. For example:

  • If one of the extended configurations adds a plugin, but I want to disable that plugin in the base configuration, how would that be handled?
  • Similarly, how are overrides and merges processed when dealing with extended configurations across multiple levels?

Providing guidance on these scenarios could make the behavior more predictable for users and avoid surprises when resolving conflicts in advanced use cases.

Thanks again for the great work on this project and for considering these suggestions. I'm looking forward to seeing how this evolves!

Best regards,
Gilad

@camchenry camchenry changed the title linter: cascading file configuration linter: nested file configuration Dec 5, 2024
@camchenry
Copy link
Contributor Author

@GiladShoham Thanks for the feedback, I've updated the post with some extra details, but I'll try to add more in the future as well with more clarifications. I'm going to try and get started on implementing this.

You’ve mentioned the priority order as follows:

* `.oxlintrc.json` in the root directory (or current working directory of the process)

* `.oxlintrc.json` in the same directory as the file

* `.oxlintrc.json` in a parent directory of the file

* The default oxlint configuration

Based on my experience with mono-repos across many organizations, I’d suggest adjusting this order. Specifically, I recommend prioritizing the current working directory (CWD) configuration after .oxlintrc.json in a parent directory of a file (essentially moving it from 1 to 3).

This was a mistake on my part, I wrote down the list incorrectly. I agree that the root directory should be a lower priority than a config file in a subdirectory. I've updated the precedence list.

While it’s indirectly mentioned, I think it’s worth explicitly stating in the priority list (or at least in the final documentation) that if a --config <path> flag is passed, it should take absolute precedence over all other configurations. Making this explicit would avoid any ambiguity and align with the principle of direct overrides.

Agreed, I added a clarification on this, as well as a more general list of precedence in configuration.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-linter Area - Linter
Projects
None yet
Development

No branches or pull requests

5 participants