Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Design issues around moving IDE code style analyzers to CodeStyle Analyzers NuGet package #38480

Closed
mavasani opened this issue Sep 4, 2019 · 30 comments
Assignees
Milestone

Comments

@mavasani
Copy link
Contributor

mavasani commented Sep 4, 2019

As part of #34907, we want to move as many IDE analyzers as possible into the CodeStyle analyzers NuGet package so the rules can be enforced on build/CI. We have already moved IDE0055 (Formatting analyzer) to this NuGet package, and I tried to move an additional analyzer to the package (#38481), but ran into bunch of design questions/issues that need to be discussed.

  1. Diagnostic Analyzer: This is the core piece that needs to be ported from IDE Features project to CodeStyle project. Design questions in this effort:

    1. Should we re-use the same IDE diagnostic IDs for the ported analyzer?
      Design Proposal: "yes", otherwise we run into issue about identical diagnostics with just a differing ID for each violation, duplicate suppressions, duplicate configuration (editorconfig/ruleset/nowarn entries) etc.
    2. De-duping of diagnostics from different sources: If we re-use the diagnostic IDs for the ported analyzer, this will likely happen by default. We should confirm with @heejaechang.
    3. De-duping of diagnostic analyzers from different sources: Given that the analyzer assembly containing these analyzers have different name (CodeStyle versus Features), there will be no automatic-deduping of analyzers and the analyzer from both these sources will execute. Do we care about the performance impact from duplicate analyzer execution? Should we add custom logic to IDE diagnostic service to not execute the IDE analyzer when there is an identical analyzer (say with same name and reporting same supported diagnostic IDs)?
    4. Should we move the CodeStyle package to reference Microsoft.CodeAnalysis 3.x? Currently, the CodeStyle NuGet package targets Microsoft.CodeAnalysis 2.8.2 to enable running on Dev15 to promote dogfooding. However, this prevents porting of bunch of analyzers which use APIs from Microsoft.CodeAnalysis 3.x. We can take couple of approaches here:
      1. Move to Microsoft.CodeAnalysis 3.x and officially state that newer CodeStyle NuGet packages are only supported with Dev16 and 3.x compiler toolset. Dev15 users can use prior CodeStyle packages.
      2. Stay on Microsoft.CodeAnalysis 2.8.2, and use Reflection based approach to light up analyzers using newer APIs if user is on Dev16.
        Personally, I hate the reflection based approach and would strongly prefer just moving to Microsoft.CodeAnalysis 3.x.
  2. Code fixer: This is likely the most controversial and tricky piece to port, given that majority of our code fixers depend on internal IDE workspace and language services and extension methods, which in turn use bunch of internal utilities, helper utilities (for example, symbol equivalence comparer), etc.

    1. Should we port IDE code fixers to Code Style layer? Theoretically speaking, this is not required as code fixers never execute in CI. Following are the pros and cons of this porting effort:

      1. PROS:
        1. Code style layer can remain isolated from IDE, and can be on par with other third party analyzer/fix packages. This can help us identify potential IDE internal APIs and utilities that should be made public.
        2. Protects us against breaking change in the analyzer, where IDE code fixers do not need to handle all versions of analyzer (from different version of NuGet packages)
      2. CONS:
        1. Implementation cost: As mentioned above, porting code fixers would likely be the most costly part of the port effort, possibly also making this whole work item unfeasible and unrealistic.
        2. Unmaintainable code: The entire IDE Workspaces and Features layer would be filled with linked files, #ifdefs based on code style preprocessor directive, splitting of extension methods into two separate files, one included in both Code Style and IDE layers, and one only in IDE layer. IMO, the code keeps getting pretty mess to work with and maintain.
        3. De-duping code fixers: We will end up showing duplicate, identical code fixes from CodeStyle NuGet and IDE (need to confirm), which means we need to design a way to de-dupe code fixers and implement it. This would mean more work.

      Design Proposal: Given the extremely large number of highly impactful CONS and the overall implementation and maintenance cost likely being a deal breaker for this effort, I would like to propose that we don't port the code fixers to code style layer. We should harden the code fixers so they gracefully bail out if the diagnostic location/properties is not as it expects, so it can handle analyzer breaking changes gracefully, and we can recommend customers to just upgrade to latest analyzer NuGet package to light-up the IDE code fixers again.

  3. Unit tests:

    1. Should we port and/or clone unit tests for analyzers/fixers CodeStyle project? We have following possible approaches, each with its own PROS and CONS:
      1. Clone unit test files into CodeStyle project: This approach is only applicable if we decide that we are going to port the code fixers into CodeStyle NuGet layer, otherwise the unit tests need to continue using IDE code fixers and hence cannot be moved to CodeStyle unit tests project. Formatting analyzer port did not port any existing formatting tests, but added just a few additional formatting analyzer tests in the CodeStyle package for the ported analyzer. This approach will certainly not work for other analyzers, as we want to run every authored analyzer/fixer unit test for both the IDE version of the analyzer and the ported analyzer in CodeStyle package. Creating cloned test files in CodeStyle unit test project will likely be impossible to maintain and keep them in sync with Features unit test project. Additionally, if we decide not to re-use the IDE test framework, and use the new Microsoft.CodeAnalysis testing library, we will have to edit each and every ported unit test, which will enormously increase the port and maintenance effort.
      2. Link unit test files into CodeStyle project and continue using the IDE test framework: This is comparatively much easier to implement and maintain compared to the prior approach. Again this approach is only applicable if we decide to port the code fixers into CodeStyle NuGet layer. I think this might be the most feasible approach if we decide to keep the CodeStyle package and its test project isolated from rest of Roslyn.sln.
      3. Tweak the existing IDE test helpers in Features test project to test analyzers and/or fixers from both the Features and Code Style layer: With this approach, the core unit test helpers just have an additional test dimension where they run with CodeStyle analyzer and/or CodeStyle fixer (latter is needed if we decide to port the fixers). This would be similar to how our IDE analyzer/fixer tests run for both regular and script code kind by default. Each test file would just need to override a helper indicating additional analyzer/fixer against which to run all the tests. This approach would likely be minimal porting effort and also easiest to maintain.

Overall Design Proposal: I think we should try to keep a very good balance between ideal and practical aspects of the above approaches. I fear taking the ideal approach, which would be to port analyzer, code fixers and unit tests (also moving them to new analyzer/fixer test library) would make this effort almost impossible to complete.
IMO, we should do the minimal implementation/maintenance cost approach which would be feasible and also gives end user value add: port just the diagnostic analyzers and do not port the code fixers or unit tests, but just ensure that the code fixers are hardened against analyzer breaking changes and tweak existing IDE unit tests to also run with ported CodeStyle analyzers.

@mavasani
Copy link
Contributor Author

mavasani commented Sep 4, 2019

@CyrusNajmabadi
Copy link
Member

Can you leave me in the loop on this?

@CyrusNajmabadi
Copy link
Member

Note: i question this:

CONS:
Implementation cost: As mentioned above, porting code fixers would likely be the most costly part of the port effort, possibly also making this whole work item unfeasible and unrealistic.

I don't think the impl cost will be that high. Furthermore, i think there are several interesting potential approaches here that are feasible depending on the 'breaking change' bar we're willing to take.

For example, i think we could just literally extract a ton of helpers into a separate library and just make things public. We'd then reference this library appropriately. This would also be beneficial to many customers who've been asking for the extensions to be able to use. We would, however, be explicit that you could use these helpers but that they might change at any time. We'd rev the version of this as we wanted, and it would be up to third parties to update appropriately.

@CyrusNajmabadi
Copy link
Member

Should we move the CodeStyle package to reference Microsoft.CodeAnalysis 3.x? Currently, the CodeStyle NuGet package targets Microsoft.CodeAnalysis 2.8.2 to enable running on Dev15 to promote dogfooding. However, this prevents porting of bunch of analyzers which use APIs from Microsoft.CodeAnalysis 3.x.

Use this as an upsell opportunity. What is devdivs goals here? Widespread use of this package? Or more use of the higher version of VS?

@CyrusNajmabadi
Copy link
Member

Design Proposal: Given the extremely large number of highly impactful CONS and the overall implementation and maintenance cost likely being a deal breaker for this effort, I would like to propose that we don't port the code fixers to code style layer. We should harden the code fixers so they gracefully bail out if the diagnostic location/properties is not as it expects, so it can handle analyzer breaking changes gracefully

Ick. That's pretty awful. What happens when we want to change something that needs an update on the analyzer and fixer? We go through a multi-month process of fixing up the analyzer, then waiting till we can reference it, then updating the fixer to support the changes in the analyzer? It just seems hellish in temrs of supporting things.

Imagine if we had to do the same for adopting the compiler APIs and fixes. Yes, the fixers use a lot of internal stuff. But then we can address that. We move a fix at a time, moving the helpers that they need. THe vast majority of helpers are totally trivial to expose in a reasonable way. i.e. they're just extension methods, or helper services for answering questions about the language, or helpers for constructing/manipulating code. They're fairly well self-contained and would not be difficult to make available.

@CyrusNajmabadi
Copy link
Member

Unmaintainable code: The entire IDE Workspaces and Features layer would be filled with linked files, #ifdefs based on code style preprocessor directive, splitting of extension methods into two separate files, one included in both Code Style and IDE layers, and one only in IDE layer. IMO, the code keeps getting pretty mess to work with and maintain.

Yes. Please please please avoid this. It's really awful how messy some layers of the IDE have already gotten in hte last couple of years. There is just blatent layering violations everywhere. Some features just hardcode knowledge in of other features. There always seems to eb time to do things in a hacky fashion, but never time to properly do things, even if doing that would make efforts like this much simpler.

@mavasani
Copy link
Contributor Author

mavasani commented Sep 4, 2019

@CyrusNajmabadi Let me try to summarize your opinion here, please correct me if I am wrong:

  1. Refactor the Worksapces and Features API surface used by IDE analyzers and fixers into a separate package with public APIs with a weak compatibility contract.
  2. Clone/link the core analyzers and fixers into the CodeStyle layer. Both the IDE (Features) and CodeStyle layer analyzers/fixers will consume the above refactored package.
  3. Unit tests: Not sure if have any preference/suggestion

This seems like a reasonable approach as well, though I see a few difficulties upfront:

  1. This approach would lead to us splitting Workspaces and Features layer (6 assemblies considering the language specific ones) into possibly 12 assemblies (at the minimum 9). It will be hard to get RPS exception for these additional assembly loads just to aid refactoring our code base better.
  2. Given that Workspaces and Features NuGet packages will depend on this, it will be extremely tricky for users to understand what surface area is fully supported public API versus weakly supported public API.

We can certainly bring this up as another potential approach to tackle this work. We will keep this issue updated with any design decisions/updates before we start any concrete porting work. Note that my PR #38481 was just a quick prototype to gauge the required work and issues.

@CyrusNajmabadi
Copy link
Member

Clone/link the core analyzers and fixers into the CodeStyle layer. Both the IDE (Features) and CodeStyle layer analyzers/fixers will consume the above refactored package.

Can we not just move these into the CodeStyle layer? And then just ship the CodeStyle stuff in Roslyn itself?

i.e. the way i see it is that we have "features" today which is a bit too big. It's an aggregation of a ton of stuff, with analyzers/fixers being a large (but not whole) portion of it.

So we're just breaking out the analyzer/fixer portion of 'features' into its own layer.

@CyrusNajmabadi
Copy link
Member

This seems like a reasonable approach as well, though I see a few difficulties upfront:

This approach would lead to us splitting Workspaces and Features layer (6 assemblies considering the language specific ones) into possibly 12 assemblies (at the minimum 9). It will be hard to get RPS exception for these additional assembly loads just to aid refactoring our code base better.

It's really unfortunate to be constantly held hostage by RPS and to basically be expected to hack thnigs together to work around that problem. Factoring things into packages/layers is sensible and keeps it possible to efficiently keep delivering updates in a healthy way without building up debt. IIRC, if you look at something like .net core, the expectation is that a core app will end up referencing >100 dlls (with several of them going well into the multi-hundreds). Not being able to just break out a large, but cohesive, grouping of code when sensible, is very limiting.

RPS should be an approach to not accidentally regress perf. It shouldn't be used to disallow changes completely when they're sensible and appropriate.

@CyrusNajmabadi
Copy link
Member

Given that Workspaces and Features NuGet packages will depend on this, it will be extremely tricky for users to understand what surface area is fully supported public API versus weakly supported public API.

Say that the entire surface area of this DLL is weakly supported. If you want, put everything under a Microsoft.CodeAnalysis.Experimental subnamespace. Have a comment on every type that says "all functionality in this type subject to change at any point".

@CyrusNajmabadi
Copy link
Member

Unit tests: Not sure if have any preference/suggestion

If we can pull the analyzers/fixers out into the new layer, then i would recommend doing the same for tests. But i don't care a huge amount. The tests today are already in the wrong place (i.e. they're in the EditorFeatures test library, instead of the Features test lib), so i don't care much if they keep staying in the wrong place.

@CyrusNajmabadi
Copy link
Member

CyrusNajmabadi commented Sep 4, 2019

@CyrusNajmabadi Let me try to summarize your opinion here, please correct me if I am wrong

Overall the summary is mostly what i think. I've clarified some of my position above.

Basically, i'm hugely opposed to continually hacking away at this stuff. This is core critical functionality that the Roslyn team owns that should be well designed and implemented in a clean, cohesive, and healthy manner.

The latter bit is especially important. I've given feedback several times over the last couple of years how we just keep having feature goals that we're not willing to properly assign resources to. So we have cut corners all over the place and the entire codebase it just building up more and more debt. This debt doesn't seem to be acknowledged**, but it continues to cost and harm future development. Unfortunately, this then gets ignored with the next big piece of work (like LiveShare) where more debt is piled on.

I do not believe this approach is sustainable and I worry that the attitude from the top is somewhat of a "not my problem" take on things. i.e. the debt will keep piling up until it all has to be burned down. Instead, of spending the proper amount of tiem to do things right so that the same Roslyn we have now can be used 10+ years from now effectively.

--

** Specifically, the approach seems to be "we'll file bug, but then do nothing about it". Having the bug isn't the problem that needs addressing. It's the actual technical debt that is the problem.

@CyrusNajmabadi
Copy link
Member

Another way to look at it: we have parts of roslyn that are effectively so complex and difficult to work with that they ahve not received more than tiny updates in them in around 5years now. That's because the debt piled up so fast, and the people working on those parts moved on, and now no one can actually handle updating things in those areas. We've basically made it so that the cost of getting out of that debt is higher than any value provided in improving those feature areas. That's really bad, and shows us how easy it is to get to that point.

I would like to keep Analyzers/Fixers from descending into that. We need to stay in the position where it's simple and easy to author these, test these, update them for new use cases (or user reported issues), all while making it simple and easy for all of them to utilize and benefit from the shared infrastructure underneath.

@mavasani
Copy link
Contributor Author

mavasani commented Sep 4, 2019

Can we not just move these into the CodeStyle layer? And then just ship the CodeStyle stuff in Roslyn itself?

That would be my most preferred approach as well. However, note that we tried the exact same strategy for creating separate External Access projects/assemblies for each of our internal partners that had IVTs for Roslyn IDE layers, and this was rejected outright by RPS team due to additional assembly loads - this was the primary reason I did not mention this approach in the original description. We can probably pitch this refactoring as a requirement for editorconfig based CI support for IDE analyzers, and that might take it through, but we will be surely asked about potential alternatives that do not involve additional assembly loads.

@jmarolf
Copy link
Contributor

jmarolf commented Sep 4, 2019

We can merge assemblies together like the editor team does if it really matters. I would personally like us to have a more enforceable way to keep things separate

@CyrusNajmabadi
Copy link
Member

We can merge assemblies together like the editor team does if it really matters.

That would be great! AFAICT, roslyn should really only have '3 dlls' at the end of the day (through the merging of many layers). The core, language-agnostic, layer. All the way from teh compiler layer up through the VS layer. Then a dll for C# and a dll for VB. Those are the 'vertical slices' that you'll always have in VS, and it makes sense to deploy like that to avoid the RPS penalty.

@sharwell
Copy link
Member

sharwell commented Sep 4, 2019

🔗 Related to #18818

@sharwell
Copy link
Member

sharwell commented Sep 4, 2019

Given that the analyzer assembly containing these analyzers have different name (CodeStyle versus Features), there will be no automatic-deduping of analyzers and the analyzer from both these sources will execute

This is covered by #18818. Only one analyzer should run for a given diagnostic ID.

@sharwell
Copy link
Member

sharwell commented Sep 4, 2019

We will end up showing duplicate, identical code fixes from CodeStyle NuGet and IDE (need to confirm), which means we need to design a way to de-dupe code fixers and implement it. This would mean more work.

The design for this is complete in #18818, but it still needs to be implemented to match the feature specification.

@mavasani
Copy link
Contributor Author

mavasani commented Sep 4, 2019

This is covered by #18818. Only one analyzer should run for a given diagnostic ID.

@sharwell That will be true only if we take @CyrusNajmabadi's suggestion and have only one assembly with the analyzer/fixer implementaton. If analyzers are in different named packages (CodeStyle versus Features), there can be no automatic de-duping even if #18818 is implemented.

@sharwell
Copy link
Member

sharwell commented Sep 4, 2019

@mavasani #18818 defines and then explicitly states that the assembly name and identity is unrelated to the process of selecting diagnostic analyzers to run.

The assembly identity is generally irrelevant for determining the source of a diagnostic. Analyzer assemblies may be freely renamed, split, merged, etc., and/or change any other identity property without changing the expected behavior for the scenario described in this issue.

@mavasani
Copy link
Contributor Author

mavasani commented Sep 4, 2019

The assembly identity is generally irrelevant for determining the source of a diagnostic. Analyzer assemblies may be freely renamed, split, merged, etc., and/or change any other identity property without changing the expected behavior for the scenario described in this issue.

@sharwell I believe the above design requirement in #18818 is flawed. We cannot de-dupe and exclude analyzers from different analyzer assemblies based on just the fact that they happened to use the same diagnostic ID. I don't believe that issue has been brought for design discussion yet.

@vatsalyaagrawal vatsalyaagrawal added this to the 16.4 milestone Sep 4, 2019
@mavasani
Copy link
Contributor Author

mavasani commented Jan 27, 2020

Proposal

User Experience

  1. User creates a new C#/VB project and sees relevant code style suggestions (IDExxxx) and important code quality suggestions (CAxxxx) in the IDE live analysis.
  2. Users can configure severity for any code style and code quality suggestions in editorconfig using the light bulb actions to either disable rules or turn them into warnings/errors. Doing the latter automatically causes these rules to execute in CI and break build.
  3. User does not need to figure out what analyzer NuGet package to install to enable Microsoft recommended analyzers. In fact, then even don't not need to know what and how the analyzers are executing under the covers.
  4. User does not need to or rather does not have to differentiate between code style suggestions (IDExxxx) and important code quality suggestions (CAxxxx) or the underlying packaging schemes. These just light up automatically in the IDE as inbuilt functionality.
  5. Users can bulk configure severity of bucket of rules based on category such as style, security, performance, globalization, etc. from light bulb . They can even turn off all analyzers or escalate all analyzer diagnostics to warnings/errors based on their preference towards analyzers in their workflow. Core .editorconfig support for this has already been added with Add editorconfig options to bulk configure analyzer diagnostics #38886 (comment).

Packaging

  1. We are working towards re-branding the FxCop analyzers package into a separate analyzer assembly/package and inserting the analyzer assembly with all the CA rules directly it into the .NET SDK. Most CA rules in this analyzer assembly will either enabled as suggestions or disabled by default for new .NET projects. An MSBuild property will be added to guard enabling/disabling this analyzer assembly.
  2. We should try to follow the same model for IDE analyzers NuGet package - these get packaged into a specific NuGet package that gets inserted into the .NET SDK. So, our IDE analyzers light up in Visual Studio as NuGet based analyzers, not the VSIX based ones from Features.dll as we do today.

IMO, achieving the above packaging and user experience story for all of our analyzers requires the following from implementation perspective:

  1. IDE analyzers/code fixes reside in the same repository as our other analyzers/fixers. This might mean we move IDE analyzers and code fixes to roslyn-analyzers repo OR we merge all our analyzers/code fixes into the Roslyn repo, but with a different RoslynAnalyzers.sln.
  2. All the common analyzer + code fix utilities should be separated into its own projects (likely shared projects) that are then included in the relevant analyzer/code projects (more details later). We should likely also ship these utilities in its own NuGet package so that they can be used by external analyzer and code fix authors. However, these packages will have a weak API compat guarantee allowing public API breaks as appropriate, so the package consumers must have the right expectations.
  3. Move the IDE analyzer/code fix unit tests to be based on our new testing library Microsoft.CodeAnalysis.Testing. We already moved all analyzers/fixes in roslyn-analyzers repo to this library. Additionally, the unit tests for CodeStyle layer are also using the testing library.

Implementation

Achieving the above packaging and user experience in a single shot is going to be challenging. So, I propose dividing it into stages as follows:

  1. Stage 1: Refactor common utilities, extensions, IDE analyzers and code fixes in Roslyn repo into shared projects that are imported into Workspaces, Features and Code Style layers as appropriate (details later in this section).
  2. Stage 2: Move all IDE analyzers + code fixes into these shared projects so we have a NuGet package that users can explicitly add to their projects to enable IDE analyzers in CI.
  3. Stage 3: Migrate all the IDE analyzer/code fix unit tests to Microsoft.CodeAnalysis.Testing library.
  4. Stage 4: Unification of IDE analyzers and CA analyzers from roslyn-analyzers repo into a single repo, built on top of the same analyzer and code fix utilities and using the same test framework. These should be inserted into the .NET SDK with the same packaging and deployment mechanism.

Stage 1

  1. Create a shared project, say CompilerExtensions.shproj or CompilerUtilities.shproj, for all the extension methods, utility classes and helper classes for functionality in Workspaces.csproj that work on core compilation concepts, such as syntax tree, syntax node/token, semantic model, symbols, compilation, etc. Essentially the pieces that have no Workspace dependency.
  2. Create a shared project, say WorkspaceExtensions.shproj or WorkspaceUtilities.shproj, for all the extension methods, utility classes and helper classes for functionality in Workspaces.csproj that work on core Workspace concepts, such as document, project, solution, etc. This should also include useful language services and workspace services, such as say ISyntaxFactsService, ISemanticsFactsService, etc. that are good utilities by themselves for components such as code fixes that work at workspace layer.
  3. Create a shared project, say Analyzers.shproj, for all the IDE analyzers that currently reside in Features.csproj that we want to share between Features and Code Style layer.
  4. Create a shared project, say CodeFixes.shproj, for all the IDE code fixes that currently reside in Features.csproj that we want to share between Features and Code Style layer.
  5. Create a shared project, say AnalyzerAndCodeFixUnitTests.shproj, to share all unit tests for the above shared analyzers and code fixes. We possibly also need a shared project for shared AnalyzerAndCodeFixUnitTestUtilities.shproj.

NOTE: We will also need corresponding C# and VB counterparts for all of above shared projects for language specific stuff.

  1. Import CompilerExtensions.shproj and WorkspaceExtensions.shproj into Workspace.csproj
  2. Import Analyzers.shproj and CodeFixes.shproj into Features.csproj
  3. Import CompilerExtensions.shproj and Analyzers.shproj into the core CodeStyle analyzers project Microsoft.CodeAnalysis.CodeStyle.csproj
  4. Import WorkspaceExtensions.shproj and CodeFixes.shproj into the core CodeStyle code fixes project Microsoft.CodeAnalysis.CodeStyle.Fixes.csproj

Stage 2

  1. Move each analyzer and code fix from Features.csproj into Analyzers.shproj and CodeFixes.shproj respectively. Simulatenously, also move the required utilities into CompilerExtensions.shproj and WorkspaceExtensions.shproj as approproate.
  2. Link the existing analyzer/code fix unit tests into CodeStyle unit test projects, similar to the way done here: https://github.com/dotnet/roslyn/pull/38481/files#diff-a424f3f2c3fec886a04a0cccd917e366R25-R26. This is just temporary approach to ensure that we don't spend too many cycles in migrating/porting unit tests in this stage.

Stage 3

  1. Undo the linking of analyzer/code fix unit tests into CodeStyle unit test projects.
  2. Move the unit test file into AnalyzerAndCodeFixUnitTests.shproj and rewrite the tests to use the new test framework from Microsoft.CodeAnalysis.Testing library.
  3. Import AnalyzerAndCodeFixUnitTests.shproj into CodeStyle unit test project Microsoft.CodeAnalysis.CodeStyle.UnitTests.csproj
  4. Import AnalyzerAndCodeFixUnitTests.shproj into Features unit test project that currently has our IDE analyzer/fixer unit tests

Stage 4

<TBD>

@mavasani
Copy link
Contributor Author

@sharwell @mikadumont @vatsalyaagrawal I would like to bring up the new proposal from #38480 (comment) for discussion in today's design meeting.

@tmat
Copy link
Member

tmat commented Jan 27, 2020

@mavasani Could you list the IDE analyzers we thing we should move to a package?

@mavasani
Copy link
Contributor Author

Could you list the IDE analyzers we thing we should move to a package?

Ideally, almost all of them (except probably for things like EnC, RenameTracking, etc. which are pure IDE-only ones). I think we should start with IDE analyzers that do not have any workspace dependency. For example, I prototyped one of them in #38481 and moving the analyzer and code fix into CodeStyle package itself was quite straightforward. I believe once we have all the utilities and helpers/extensions sorted out, majority of analyzers should be easy to move and start providing value add to customers.

@sharwell
Copy link
Member

Ideally, almost all of them

We're going to need to list them individually. I'm concerned that many would not meet the expected bar for avoiding false positives in NuGet-installed analyzers.

@mavasani
Copy link
Contributor Author

mavasani commented Jan 27, 2020

We're going to need to list them individually. I'm concerned that many would not meet the expected bar for avoiding false positives in NuGet-installed analyzers.

@sharwell If any of the IDE analyzers are causing false positives, they should not be enabled in the IDE even today. I think we should move all (possible) IDE analyzers to NuGet package and decide to disable them by default if we think they are causing false positives. Enabled ones will still be suggestions or hidden diagnostics by default, so they cannot degrade the end user experience unless they explicitly bump up the severity of any diagnostic to be warning/error. Unless of course user is bulk configuring all style category diagnostics to warning/error by default.

@jinujoseph jinujoseph added the Need Design Review The end user experience design needs to be reviewed and approved. label Jan 27, 2020
@mavasani mavasani removed the Need Design Review The end user experience design needs to be reviewed and approved. label Feb 14, 2020
@mavasani mavasani assigned mavasani and unassigned sharwell Feb 14, 2020
@mavasani mavasani modified the milestones: Backlog, 16.6.P1 Feb 14, 2020
mavasani added a commit to mavasani/roslyn that referenced this issue Mar 17, 2020
All the analyzers in CodeStyle NuGet package are suggestions or hidden diagnostics by default. Hence, executing them in CI by default is an unnecessary performance overhead during build. This is especially critical when we attempt to move the CodeStyle NuGet package into the SDK as outlined in the User Experience [here](dotnet#38480 (comment)).
This change auto-generates and embeds a .props file into the CodeStyle NuGet package that disables all analyzers in the package by default when executing command line builds. Current approach generates a `NoWarn`, but once dotnet#42219 is implemented, we will replace it with a global/base editorconfig which can ship with the NuGet package.
@jinujoseph jinujoseph modified the milestones: 16.6.P1, 16.6 Mar 22, 2020
@mavasani
Copy link
Contributor Author

The design issues here have been resolved, and we have ported nearly half of the IDE analyzers to a pre-release CodeStyle NuGet package. Closing this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Archived in project
Development

No branches or pull requests

8 participants