NuGet strategy consolidation

fossas · Aug 21, 2024 · ff29631 · ff29631
1 parent cfbcd5b
commit ff29631
Show file tree

Hide file tree

Showing 14 changed files with 182 additions and 131 deletions.
diff --git a/docs/references/strategies/languages/dotnet/README.md b/docs/references/strategies/languages/dotnet/README.md
@@ -1,12 +1,12 @@
 # .NET Analysis
 
-There are several different methods of .NET analysis, that use both the `NuGet` (`nuspec`, `PackageReference`, `packages.config`, `project.json`, `project.assets.json`) and `Paket` package managers.
+There are several different methods of .NET analysis, that use both the [NuGet](./nuget.md) (`nuspec`, `PackageReference`, `packages.config`, `project.json`, `project.assets.json`) and [Paket](./paket.md) package managers.
 
 | Strategy                                 | Direct Deps | Transitive Deps | Edges |
 | ---------------------------------------- | ----------- | --------- | ----- |
-| [nuspec](nuspec.md)                         | ✅          | ❌        | ❌    |
-| [PackageReference](packagereference.md)     | ✅          | ❌        | ❌    |
-| [packages.config](packagesconfig.md)        | ✅          | ❌        | ❌    |
 | [paket](paket.md)                           | ✅          | ✅        | ✅    |
 | [project.assets.json](projectassetsjson.md) | ✅          | ✅        | ✅    |
-| [project.json](projectjson.md)              | ✅          | ❌        | ❌    |
+| [PackageReference](packagereference.md)     | ✅          | ❌        | ❌    |
+| [project.json](projectjson.md)              | ✅          | ❌        | ❌    |
+| [packages.config](packagesconfig.md)        | ✅          | ❌        | ❌    |
+| [nuspec](nuspec.md)                         | ✅          | ❌        | ❌    |
diff --git a/docs/references/strategies/languages/dotnet/nuget-qr.md b/docs/references/strategies/languages/dotnet/nuget-qr.md
@@ -0,0 +1,17 @@
+# Quick reference: NuGet
+
+## Requirements
+
+**Ideal/Minimum**
+
+One more more of the following:
+
+- [`.nuspec`](https://docs.microsoft.com/en-us/nuget/reference/nuspec) formatted file in your directory.
+- [Package reference](https://docs.microsoft.com/en-us/nuget/consume-packages/package-references-in-project-files) file present in your project. Commonly with an ending such as `.csproj`, `xproj`, `vbproj` and others.
+- `project.assets.json`
+- `packages.config`
+- `project.json`
+
+## Project discovery
+
+Directories containing any of the files listed above are considered NuGet projects.
diff --git a/docs/references/strategies/languages/dotnet/nuget.md b/docs/references/strategies/languages/dotnet/nuget.md
@@ -1,17 +1,22 @@
-# Quick reference: NuGet
+# NuGet Analysis
 
-## Requirements
+| Strategy                                 | Direct Deps | Transitive Deps | Edges |
+| ---------------------------------------- | ----------- | --------- | ----- |
+| [project.assets.json](projectassetsjson.md) | ✅          | ✅        | ✅    |
+| [PackageReference](packagereference.md)     | ✅          | ❌        | ❌    |
+| [project.json](projectjson.md)              | ✅          | ❌        | ❌    |
+| [packages.config](packagesconfig.md)        | ✅          | ❌        | ❌    |
+| [nuspec](nuspec.md)                         | ✅          | ❌        | ❌    |
 
-**Ideal/Minimum**
+NuGet analysis follows these strategies in sequence:
+1. `project.assets.json`
+2. `PackageReference`
 
-One more more of the following:
+`project.assets.json` files and their dependencies are generated from  `.csproj` files. `PackageReference` dependencies can be found in `.csproj`, `.xproj`, `.vbproj`, `.dbproj`, or `.fsproj` files. To consolidate findings from these two strategies, `project.assets.json` analysis is attempted first and falls back to `PackageReference` analysis. 
 
-- [`.nuspec`](https://docs.microsoft.com/en-us/nuget/reference/nuspec) formatted file in your directory.
-- [Package reference](https://docs.microsoft.com/en-us/nuget/consume-packages/package-references-in-project-files) file present in your project. Commonly with an ending such as `.csproj`, `xproj`, `vbproj` and others.
-- `project.assets.json`
-- `packages.config`
-- `project.json`
+The following strategies are executed independently::
+1. `project.json`
+2. `packages.config`
+3. `nuspec`
 
-## Project discovery
-
-Directories containing any of the files listed above are considered NuGet projects.
+`project.json` and `packages.config` files are deprecated in favor of `.csproj` and their usage of `PackageReference` format. `nuspec` serves as a manifest containing package metadata, used both for building the package and providing information to consumers. These strategies are isolated from the `project.assets.json` and `PackageReference` approaches and therefore run independently.
diff --git a/docs/references/strategies/languages/dotnet/paket.md b/docs/references/strategies/languages/dotnet/paket.md
@@ -1,5 +1,9 @@
 # Paket
 
+| Strategy                                 | Direct Deps | Transitive Deps | Edges |
+| ---------------------------------------- | ----------- | --------- | ----- |
+| [paket](paket.md)                           | ✅          | ✅        | ✅    |
+
 Paket is a dependency manager for .NET projects. Paket enables precise and predictable control over your dependencies
 
 Paket manages your dependencies with three core file types:

diff --git a/fossa-cli.generated.sln b/fossa-cli.generated.sln
@@ -0,0 +1,33 @@
+
+Microsoft Visual Studio Solution File, Format Version 12.00
+# Visual Studio Version 17
+VisualStudioVersion = 17.5.002.0
+MinimumVisualStudioVersion = 10.0.40219.1
+Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "test", "test", "{658B52A1-C5F7-4B7E-944F-053941594B2F}"
+EndProject
+Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "NuGet", "NuGet", "{3B3BB3E9-0869-4F9B-A4C5-C19DF124C9D2}"
+EndProject
+Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "test", "test\NuGet\testdata\test.csproj", "{EE31BA60-2E5B-4CF6-BD30-1EABD73BB0B6}"
+EndProject
+Global
+	GlobalSection(SolutionConfigurationPlatforms) = preSolution
+		Debug|Any CPU = Debug|Any CPU
+		Release|Any CPU = Release|Any CPU
+	EndGlobalSection
+	GlobalSection(ProjectConfigurationPlatforms) = postSolution
+		{EE31BA60-2E5B-4CF6-BD30-1EABD73BB0B6}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
+		{EE31BA60-2E5B-4CF6-BD30-1EABD73BB0B6}.Debug|Any CPU.Build.0 = Debug|Any CPU
+		{EE31BA60-2E5B-4CF6-BD30-1EABD73BB0B6}.Release|Any CPU.ActiveCfg = Release|Any CPU
+		{EE31BA60-2E5B-4CF6-BD30-1EABD73BB0B6}.Release|Any CPU.Build.0 = Release|Any CPU
+	EndGlobalSection
+	GlobalSection(SolutionProperties) = preSolution
+		HideSolutionNode = FALSE
+	EndGlobalSection
+	GlobalSection(NestedProjects) = preSolution
+		{3B3BB3E9-0869-4F9B-A4C5-C19DF124C9D2} = {658B52A1-C5F7-4B7E-944F-053941594B2F}
+		{EE31BA60-2E5B-4CF6-BD30-1EABD73BB0B6} = {3B3BB3E9-0869-4F9B-A4C5-C19DF124C9D2}
+	EndGlobalSection
+	GlobalSection(ExtensibilityGlobals) = postSolution
+		SolutionGuid = {FACD6BBC-C0EA-4521-9831-30EB13595A84}
+	EndGlobalSection
+EndGlobal
diff --git a/integration-test/Analysis/NugetSpec.hs b/integration-test/Analysis/NugetSpec.hs
@@ -6,7 +6,8 @@ module Analysis.NugetSpec (spec) where
 import Analysis.FixtureExpectationUtils
 import Analysis.FixtureUtils
 import Path
-import Strategy.NuGet.PackageReference qualified as PackageReference
+
+import Strategy.NuGet qualified as NuGet
 import Strategy.NuGet.PackagesConfig qualified as PackagesConfig
 import Test.Hspec
 import Types
@@ -25,7 +26,7 @@ serviceStack discoveryFunc =
 
 testServiceStackForPkgReferences :: Spec
 testServiceStackForPkgReferences =
-  aroundAll (withAnalysisOf $ serviceStack PackageReference.discover) $ do
+  aroundAll (withAnalysisOf $ serviceStack NuGet.discover) $ do
     describe "ServiceStack" $ do
       it "should find targets" $ \(result, _) -> do
         length result `shouldBe` 64

diff --git a/spectrometer.cabal b/spectrometer.cabal
@@ -485,6 +485,7 @@ library
     Strategy.Node.YarnV2.Lockfile
     Strategy.Node.YarnV2.Resolvers
     Strategy.Node.YarnV2.YarnLock
+    Strategy.NuGet
     Strategy.NuGet.Nuspec
     Strategy.NuGet.PackageReference
     Strategy.NuGet.PackagesConfig

diff --git a/src/App/Fossa/Analyze/Discover.hs b/src/App/Fossa/Analyze/Discover.hs
@@ -27,6 +27,7 @@ import Strategy.Maven qualified as Maven
 import Strategy.Mix qualified as Mix
 import Strategy.Nim qualified as Nim
 import Strategy.Node qualified as Node
+import Strategy.NuGet qualified as NuGet
 import Strategy.NuGet.Nuspec qualified as Nuspec
 import Strategy.NuGet.PackageReference qualified as PackageReference
 import Strategy.NuGet.PackagesConfig qualified as PackagesConfig
@@ -65,15 +66,14 @@ discoverFuncs =
   , DiscoverFunc Mix.discover
   , DiscoverFunc Nim.discover
   , DiscoverFunc Node.discover
+  , DiscoverFunc NuGet.discover
   , DiscoverFunc Nuspec.discover
-  , DiscoverFunc PackageReference.discover
   , DiscoverFunc PackagesConfig.discover
   , DiscoverFunc Paket.discover
   , DiscoverFunc Pdm.discover
   , DiscoverFunc Perl.discover
   , DiscoverFunc Pipenv.discover
   , DiscoverFunc Poetry.discover
-  , DiscoverFunc ProjectAssetsJson.discover
   , DiscoverFunc ProjectJson.discover
   , DiscoverFunc Pub.discover
   , DiscoverFunc R.discover

diff --git a/src/App/Fossa/Container/Sources/Discovery.hs b/src/App/Fossa/Container/Sources/Discovery.hs
@@ -36,11 +36,10 @@ import Strategy.Maven qualified as Maven
 import Strategy.NDB qualified as NDB
 import Strategy.Nim qualified as Nim
 import Strategy.Node qualified as Node
+import Strategy.NuGet qualified as NuGet
 import Strategy.NuGet.Nuspec qualified as Nuspec
-import Strategy.NuGet.PackageReference qualified as PackageReference
 import Strategy.NuGet.PackagesConfig qualified as PackagesConfig
 import Strategy.NuGet.Paket qualified as Paket
-import Strategy.NuGet.ProjectAssetsJson qualified as ProjectAssetsJson
 import Strategy.NuGet.ProjectJson qualified as ProjectJson
 import Strategy.Perl qualified as Perl
 import Strategy.Pub qualified as Pub
@@ -89,15 +88,14 @@ managedDepsDiscoveryF =
   , DiscoverFunc Maven.discover
   , DiscoverFunc Nim.discover
   , DiscoverFunc Node.discover
+  , DiscoverFunc NuGet.discover
   , DiscoverFunc Nuspec.discover
-  , DiscoverFunc PackageReference.discover
   , DiscoverFunc PackagesConfig.discover
   , DiscoverFunc Paket.discover
   , DiscoverFunc Pdm.discover
   , DiscoverFunc Perl.discover
   , DiscoverFunc Pipenv.discover
   , DiscoverFunc Poetry.discover
-  , DiscoverFunc ProjectAssetsJson.discover
   , DiscoverFunc ProjectJson.discover
   , DiscoverFunc Pub.discover
   , DiscoverFunc R.discover

diff --git a/src/Strategy/NuGet.hs b/src/Strategy/NuGet.hs
@@ -0,0 +1,91 @@
+module Strategy.NuGet (
+  discover,
+  findProjects,
+  getDeps,
+  mkProject,
+  NuGetProject (..),
+) where
+
+import App.Fossa.Analyze.Types (AnalyzeProject (analyzeProjectStaticOnly), analyzeProject)
+import Control.Effect.Diagnostics (
+  Diagnostics,
+  Has,
+  context,
+  (<||>),
+ )
+import Control.Effect.Reader (Reader)
+import Data.Aeson (
+  ToJSON,
+ )
+import Data.Foldable (find)
+import Data.List qualified as L
+import Discovery.Filters (AllFilters)
+import Discovery.Simple (simpleDiscover)
+import Discovery.Walk (
+  WalkStep (WalkContinue),
+  fileName,
+  findFileNamed,
+  walkWithFilters',
+ )
+import Effect.ReadFS (ReadFS)
+import GHC.Generics (Generic)
+import Path (Abs, Dir, File, Path, parent)
+import Strategy.NuGet.PackageReference qualified as PackageReference
+import Strategy.NuGet.PackageReference qualified as ProjectAssetsJson
+import Types (
+  DependencyResults (..),
+  DiscoveredProject (..),
+  DiscoveredProjectType (NuGetProjectType),
+ )
+
+discover ::
+  ( Has ReadFS sig m
+  , Has Diagnostics sig m
+  , Has (Reader AllFilters) sig m
+  ) =>
+  Path Abs Dir ->
+  m [DiscoveredProject NuGetProject]
+discover = simpleDiscover findProjects mkProject NuGetProjectType
+
+findProjects :: (Has ReadFS sig m, Has Diagnostics sig m, Has (Reader AllFilters) sig m) => Path Abs Dir -> m [NuGetProject]
+findProjects = walkWithFilters' $ \_ _ files -> do
+  case findProjectAssetsJsonFile files of
+    Just file -> pure ([NuGetProject file], WalkContinue)
+    Nothing -> case find isPackageRefFile files of
+      Just file -> pure ([NuGetProject file], WalkContinue)
+      Nothing -> pure ([], WalkContinue)
+  where
+    findProjectAssetsJsonFile :: [Path Abs File] -> Maybe (Path Abs File)
+    findProjectAssetsJsonFile = findFileNamed "project.assets.json"
+
+    isPackageRefFile :: Path b File -> Bool
+    isPackageRefFile file = any (\x -> x `L.isSuffixOf` fileName file) [".csproj", ".xproj", ".vbproj", ".dbproj", ".fsproj"]
+
+mkProject :: NuGetProject -> DiscoveredProject NuGetProject
+mkProject project =
+  DiscoveredProject
+    { projectType = NuGetProjectType
+    , projectPath = parent $ nugetProjectFile project
+    , projectBuildTargets = mempty
+    , projectData = project
+    }
+
+newtype NuGetProject = NuGetProject
+  { nugetProjectFile :: Path Abs File
+  }
+  deriving (Eq, Ord, Show, Generic)
+
+instance ToJSON NuGetProject
+
+instance AnalyzeProject NuGetProject where
+  analyzeProject _ = getDeps
+  analyzeProjectStaticOnly _ = getDeps
+
+getDeps :: (Has ReadFS sig m, Has Diagnostics sig m) => NuGetProject -> m DependencyResults
+getDeps project = context "NuGet" (getAssetsJsonDeps project <||> getPackageReferenceDeps project)
+
+getAssetsJsonDeps :: (Has ReadFS sig m, Has Diagnostics sig m) => NuGetProject -> m DependencyResults
+getAssetsJsonDeps = context "ProjectAssetsJson" . context "Static analysis" . ProjectAssetsJson.analyze' . nugetProjectFile
+
+getPackageReferenceDeps :: (Has ReadFS sig m, Has Diagnostics sig m) => NuGetProject -> m DependencyResults
+getPackageReferenceDeps = context "PackageReference" . context "Static analysis" . PackageReference.analyze' . nugetProjectFile
diff --git a/src/Strategy/NuGet/PackageReference.hs b/src/Strategy/NuGet/PackageReference.hs
@@ -1,86 +1,32 @@
 {-# LANGUAGE RecordWildCards #-}
 
 module Strategy.NuGet.PackageReference (
-  discover,
-  findProjects,
-  getDeps,
-  mkProject,
   buildGraph,
+  analyze',
   PackageReference (..),
-  PackageReferenceProject (..),
   ItemGroup (..),
   Package (..),
 ) where
 
-import App.Fossa.Analyze.Types (AnalyzeProject (analyzeProjectStaticOnly), analyzeProject)
 import Control.Applicative (optional, (<|>))
 import Control.Effect.Diagnostics (Diagnostics, Has, context)
-import Control.Effect.Reader (Reader)
-import Data.Aeson (ToJSON)
-import Data.Foldable (find)
-import Data.List qualified as L
 import Data.Map.Strict qualified as Map
 import Data.Text (Text)
 import DepTypes (
   DepType (NuGetType),
   Dependency (..),
   VerConstraint (CEq),
  )
-import Discovery.Filters (AllFilters)
-import Discovery.Simple (simpleDiscover)
-import Discovery.Walk (
-  WalkStep (WalkContinue),
-  fileName,
-  walkWithFilters',
- )
 import Effect.ReadFS (ReadFS, readContentsXML)
-import GHC.Generics (Generic)
 import Graphing (Graphing)
 import Graphing qualified
 import Parse.XML (FromXML (..), attr, child, children)
-import Path (Abs, Dir, File, Path, parent)
+import Path (Abs, File, Path)
 import Types (
   DependencyResults (..),
-  DiscoveredProject (..),
-  DiscoveredProjectType (PackageReferenceProjectType),
   GraphBreadth (Partial),
  )
 
-isPackageRefFile :: Path b File -> Bool
-isPackageRefFile file = any (\x -> x `L.isSuffixOf` fileName file) [".csproj", ".xproj", ".vbproj", ".dbproj", ".fsproj"]
-
-discover :: (Has ReadFS sig m, Has Diagnostics sig m, Has (Reader AllFilters) sig m) => Path Abs Dir -> m [DiscoveredProject PackageReferenceProject]
-discover = simpleDiscover findProjects mkProject PackageReferenceProjectType
-
-findProjects :: (Has ReadFS sig m, Has Diagnostics sig m, Has (Reader AllFilters) sig m) => Path Abs Dir -> m [PackageReferenceProject]
-findProjects = walkWithFilters' $ \_ _ files -> do
-  case find isPackageRefFile files of
-    Nothing -> pure ([], WalkContinue)
-    Just file -> pure ([PackageReferenceProject file], WalkContinue)
-
-newtype PackageReferenceProject = PackageReferenceProject
-  { packageReferenceFile :: Path Abs File
-  }
-  deriving (Eq, Ord, Show, Generic)
-
-instance ToJSON PackageReferenceProject
-
-instance AnalyzeProject PackageReferenceProject where
-  analyzeProject _ = getDeps
-  analyzeProjectStaticOnly _ = getDeps
-
-mkProject :: PackageReferenceProject -> DiscoveredProject PackageReferenceProject
-mkProject project =
-  DiscoveredProject
-    { projectType = PackageReferenceProjectType
-    , projectBuildTargets = mempty
-    , projectPath = parent $ packageReferenceFile project
-    , projectData = project
-    }
-
-getDeps :: (Has ReadFS sig m, Has Diagnostics sig m) => PackageReferenceProject -> m DependencyResults
-getDeps = context "PackageReference" . context "Static analysis" . analyze' . packageReferenceFile
-
 analyze' :: (Has ReadFS sig m, Has Diagnostics sig m) => Path Abs File -> m DependencyResults
 analyze' file = do
   ref <- readContentsXML @PackageReference file