refactor!: project system refactors (#1960)

docs/index.md

@@ -25,6 +25,7 @@
    userguides/scripts
    userguides/publishing
    userguides/logging
+   userguides/trace
 ```
 
 ```{eval-rst}
@@ -45,11 +46,10 @@
 
 ```{eval-rst}
 .. toctree::
-   :caption: Python Reference
+   :caption: Core Python Reference
    :maxdepth: 1
 
    methoddocs/ape.md
-   methoddocs/ape_accounts.md
    methoddocs/api.md
    methoddocs/cli.md
    methoddocs/contracts.md
@@ -59,3 +59,13 @@
    methoddocs/types.md
    methoddocs/utils.md
 ```
+
+```{eval-rst}
+.. toctree::
+   :caption: Plugin Python Reference
+   :maxdepth: 1
+
+   methoddocs/ape_accounts.md
+   methoddocs/ape_compile.md
+   methoddocs/ape_pm.md
+```

docs/methoddocs/ape_compile.md

@@ -0,0 +1,6 @@
+# ape-compile
+
+```{eval-rst}
+.. automodule:: ape_compile
+    :members:
+```

docs/methoddocs/ape_pm.md

@@ -0,0 +1,6 @@
+# ape-pm
+
+```{eval-rst}
+.. automodule:: ape_pm
+    :members:
+```

docs/methoddocs/managers.md

@@ -72,31 +72,7 @@
 ## Project
 
 ```{eval-rst}
-.. automodule:: ape.managers.project.manager
-    :members:
-    :special-members:
-```
-
-```{eval-rst}
-.. automodule:: ape.managers.project.dependency
-    :members:
-    :special-members:
-```
-
-```{eval-rst}
-.. autoclass:: ape.managers.project.types.BaseProject
-    :members:
-    :special-members:
-```
-
-```{eval-rst}
-.. autoclass:: ape.managers.project.types.ApeProject
-    :members:
-    :special-members:
-```
-
-```{eval-rst}
-.. autoclass:: ape.managers.project.types.BrownieProject
+.. automodule:: ape.managers.project
     :members:
     :special-members:
 ```

docs/userguides/compile.md

@@ -87,23 +87,8 @@ compile:
 ## Settings
 
 Generally, configure compiler plugins using your `ape-config.yaml` file.
-One setting that applies to many compiler plugins is `cache_folder`, which holds dependency source files the compiler uses when compiling your contracts.
-By default, the folder is in your `contracts/.cache` folder but there are times you may want to move this to another location.
-Paths are relative to the project directory.
-For instance, to move the dependency cache to the root project directory:
 
-```yaml
-compile:
-   cache_folder: .cache
-```
-
-```{caution}
-Changing the location of the dependency cache folder may alter the the output bytecode of your contracts from some compilers.
-Specifically, the [solc compiler will apend a hash of the input metadata to the contract bytecode](https://docs.soliditylang.org/en/latest/metadata.html#encoding-of-the-metadata-hash-in-the-bytecode) which will change with contract path or compiler settings changes.
-This may impact things like contract verification on existing projects.
-```
-
-As another example, when using the `vyper` plugin, you can configure settings under the `vyper` key:
+For example, when using the `vyper` plugin, you can configure settings under the `vyper` key:
 
 ```yaml
 vyper:
@@ -158,3 +143,19 @@ owner = accounts.test_accounts[0]
 
 instance = container.deploy(sender=owner)
 ```
+
+## Output Extra
+
+Sometimes, there are extra output styles you may want.
+For example, to output minified ABI JSONs, use the following config:
+
+```yaml
+compile:
+  output_extra:
+     - ABI
+```
+
+Then, after compiling, you should notice minified ABI json files in your `.build/abi` folder.
+This is useful if hosting these files on a web-server.
+
+To see the full list of supported output-extra, see [the OutpuExtra enum documentation](../methoddocs/ape_compile.html#ape_compile.OutputExtras).

docs/userguides/dependencies.md

@@ -1,10 +1,17 @@
 # Dependencies
 
-Ape downloads and caches dependencies in the `.ape/packages/<name>/<version-id>` directory where `<name>` refers to the name of the dependency and `<version-id>` refers to the version or branch of the package.
-When first downloading dependencies, Ape only places the source contents in the `sources` field of the `PackageManifest` and leaves the `contract_types` field untouched.
-This is because dependencies may not compile by Ape's standard out-of-the-box but their contract types can still be used in projects that do.
+Ape downloads and caches dependencies in the `.ape/packages` folder.
+There are three sub-folders in `.ape/packages` for dependencies:
 
-To use dependencies in your projects, you must configure them in your `ape-config.yaml` file.
+1. `projects/` - contains the raw project files for each dependency in subsequent `/<name>/<version-id>` directories (where `<name>` refers to the path-ified full-name of the dependency, e.g. `"OpenZeppelin_openzeppelin-contracts"`, and `<version-id>` refers to the version or branch of the package).
+   This location is where local project compilation looks for additional sources from import statements.
+2. `manifests/` - much like your local projects' `.build/__local__.json`, this is where dependencies cache their manifests.
+   When you compile a dependency, the contract types are stored in the dependency manifest's JSON file.
+3. `api/` - for caching the API data placed in `dependencies:` config or `ape pm install` commands, allowing dependency usage and management from anywhere in the file system.
+
+*NOTE*: You can install dependencies that don't compile out-of-the-box.
+Sometimes, dependencies are only collections of source files not meant to compile on their own but instead be used in projects via import statements.
+You can change the settings of a dependency using `config_override:` to compile dependencies after installed, if needed, and the `api/` cache always refers to the latest used during installation or compilation.
 
 ## Types of Dependencies
 
@@ -149,6 +156,16 @@ ape pm install gh:OpenZeppelin/openzeppelin-contracts \
   --config-override '{"solidity": {"version": "0.8.12"}}'
 ```
 
+You can also use Python to install dependencies, using `**kwargs` as the same fields you put in your `dependencies:` config:
+
+```python
+from ape import project
+
+project.dependencies.install(
+   github="OpenZeppelin/openzeppelin-contracts", name="openzeppelin", version="4.4.2"
+)
+```
+
 ### uninstall
 
 Remove previously installed packages using the `uninstall` command:
@@ -285,7 +302,8 @@ You can achieve this using the project manager:
 from ape import accounts, project
 
 # NOTE: This will compile the dependency
-dependency_contract = project.dependencies["my_dependency"]["1.0.0"].DependencyContractType
+dependency_project = project.dependencies["my_dependency"]["1.0.0"]
+dependency_contract = dependency_project.DependencyContractType 
 my_account = accounts.load("alias")
 deployed_contract = my_account.deploy(dependency_contract, "argument")
 print(deployed_contract.address)

docs/userguides/projects.md

@@ -17,64 +17,56 @@ project                             # The root project directory
 Notice that you can configure you ape project using the `ape-config.yaml` file.
 See the [configuration guide](./config.html) for a more detailed explanation of settings you can adjust.
 
-## Adding Plugins
+## The Local Project
 
-Your project may require plugins.
-To install plugins, use the `ape plugins install .` command.
-Learn more about configuring your project's required plugins by following [this guide](./installing_plugins.html).
-
-## Compiling Contracts
-
-The project manager object is a representation of your current project.
-Access it from the root `ape` namespace:
+After you have a local project and you are in the directory of that project, the global `project` reference in Ape will refer to this project.
+You can see this by typing `project` in the `ape console`:
 
 ```python
-from ape import project
-```
-
-Your `project` contains all the "relevant" files, such as source files in the `contracts/` directory.
-Use the following command to compile all contracts in the `contracts/` directory:
-
-```bash
-ape compile
+In [1]: project
+Out[1]: <ProjectManager ~/ApeProjects/ape-demo-project>
 ```
 
-For more information on compiling your project, see [this guide](./compile.html).
+In this case, my terminal's current working directory is the same as a local project named `ape-demo-project`.
 
-## Deploying Contracts
+## Other Projects
 
-After compiling, the contract containers are accessible from the `project` manager.
-Deploy them in the `console` or in scripts; for example:
+You can reference other local projects on your computer by using the `Project` factory class (notice the capital `P`):
 
 ```python
-from ape import accounts, project
+from ape import Project
 
-account = accounts.load("my_account_alias")
-account.deploy(project.MyContract)
+my_other_project = Project("../path/to/my/other/project")
+_ = my_other_project.MyContract  # Do anything you can do to the root-level project.
 ```
 
-**NOTE**: You can also deploy contracts from the container itself:
+## Project Manifests
+
+Ape stores and caches artifacts in an [EthPM package manifest](https://eips.ethereum.org/EIPS/eip-2678).
+When working with local projects, the manifests get placed in the `<project-path>/.build/__local__.json`.
+However, you may obtain a manifest from a different location.
+If that is the case, you can create a project directly from the manifest itself:
 
 ```python
-from ape import accounts, project
+from ape import Project
 
-account = accounts.load("my_account_alias")
-project.MyContract.deploy(sender=account)
+# Pass in a manifest (object or dictionary), or a path to a manifest's JSON file.
+project = Project.from_manifest("path/to/manifest.json")
+_ = project.MyContract  # Do anything you can do to the root-level project.
 ```
 
-### Dependencies
-
-To set up and use dependencies in your project, follow [this guide](./dependencies.html).
+## Dependencies
 
-## Scripts
+Use other projects as dependencies in Ape.
+There is an extensive guide you can read on this [here](./dependencies.html).
+But it is important to note that the dependency system largely is dependent on the project system.
+Dependencies are just projects after all; projects containing source files you both use in your projects or compile independently.
 
-The scripts folder contains project automation scripts, such as deploy scripts, as well as other executable jobs, such as scripts for running simulations.
-To learn more about scripting in Ape, see [the scripting guide](./scripts.html).
+For example, access a dependency project and treat it like any other project this way:
 
-## Testing
+```python
+from ape import project
 
-Use tests to verify your project.
-You can test your project using the `ape test` command.
-The `ape test` command comes with the core-plugin `ape-test`.
-The `ape-test` plugin extends the popular python testing framework [pytest](https://docs.pytest.org/en/6.2.x/contents.html).
-Testing is a complex topic; learn more about testing using Ape framework [here](./testing.html).
+dependency = project.dependencies.get_dependency("my-dependency", "1.0.0")
+contract_type = dependency.project.ContractFromDependency
+```

docs/userguides/publishing.md

@@ -16,7 +16,7 @@ Once your project has successfully compiled, you will have the start of your `Pa
 
 ## Tracking Deployments
 
-If your project contains deployments that you wish to include in its package manifest, use the [track_deployment()](../methoddocs/managers.html#ape.managers.project.manager.ProjectManager.track_deployment) method.
+If your project contains deployments that you wish to include in its package manifest, use the [project.deployments.track](../methoddocs/managers.html#ape.managers.project.manager.DeploymentManager.track) method.
 Example:
 
 ```python
@@ -26,7 +26,7 @@ account = accounts.load("mainnet-account")
 
 # Assume your project has a contract named 'MyContract' with constructor that accepts argument '123'.
 contract = project.MyContract.deploy(123, sender=account)
-project.track_deployment(contract)
+project.deployments.track(contract)
 ```
 
 If the contract is already deployed, you can use [Contract](../methoddocs/ape.html#ape.Contract) to get a contract instance:
@@ -35,7 +35,7 @@ If the contract is already deployed, you can use [Contract](../methoddocs/ape.ht
 from ape import Contract, project
 
 contract = Contract("0x12c17f958d2ee523a2206206994597c13d831e34")
-project.track_deployment(contract)
+project.deployments.track(contract)
 ```
 
 For more information on accessing contract instances, follow [this guide](./contracts.html).

setup.py

@@ -99,7 +99,7 @@
     install_requires=[
         "click>=8.1.6,<9",
         "ijson>=3.1.4,<4",
-        "ipython>=8.5.0,<9",
+        "ipython>=8.18.1,<9",
         "lazyasd>=0.1.4",
         "packaging>=23.0,<24",
         "pandas>=1.3.0,<2",

src/ape/__init__.py

@@ -37,7 +37,7 @@
 accounts = _ManagerAccessMixin.account_manager
 """Manages accounts for the current project. See :class:`ape.managers.accounts.AccountManager`."""
 
-project = _ManagerAccessMixin.project_manager
+project = _ManagerAccessMixin.local_project
 """The currently active project. See :class:`ape.managers.project.ProjectManager`."""
 
 Contract = chain.contracts.instance_at

src/ape/_cli.py

@@ -28,7 +28,7 @@ def display_config(ctx, param, value):
     click.echo("# Current configuration")
 
     # NOTE: Using json-mode as yaml.dump requires JSON-like structure.
-    model = ManagerAccessMixin.project_manager.config_manager.model_dump(mode="json")
+    model = ManagerAccessMixin.local_project.config_manager.model_dump(mode="json")
 
     click.echo(yaml.dump(model))
 
@@ -89,7 +89,7 @@ def invoke(self, ctx) -> Any:
         except click.UsageError as err:
             self._suggest_cmd(err)
         except ApeException as err:
-            path = ctx.obj.project_manager.path
+            path = ctx.obj.local_project.path
 
             # NOTE: isinstance check for type-checkers.
             if isinstance(path, Path) and handle_ape_exception(err, (path,)):

src/ape/api/accounts.py

@@ -280,10 +280,10 @@ def deploy(
         self.chain_manager.contracts.cache_deployment(instance)
 
         if publish:
-            self.project_manager.track_deployment(instance)
+            self.local_project.deployments.track(instance)
             self.provider.network.publish_contract(address)
 
-        instance.base_path = contract.base_path or self.project_manager.contracts_folder
+        instance.base_path = contract.base_path or self.local_project.path
         return instance
 
     def declare(self, contract: "ContractContainer", *args, **kwargs) -> ReceiptAPI:
@@ -462,7 +462,7 @@ def __getitem__(self, address: AddressType) -> AccountAPI:
               `ChecksumAddress <https://eth-typing.readthedocs.io/en/latest/types.html#checksumaddress>`__.  # noqa: E501
 
         Raises:
-            IndexError: When there is no local account with the given address.
+            KeyError: When there is no local account with the given address.
 
         Returns:
             :class:`~ape.api.accounts.AccountAPI`
@@ -471,7 +471,7 @@ def __getitem__(self, address: AddressType) -> AccountAPI:
             if account.address == address:
                 return account
 
-        raise IndexError(f"No local account {address}.")
+        raise KeyError(f"No local account {address}.")
 
     def append(self, account: AccountAPI):
         """
@@ -541,7 +541,7 @@ def __contains__(self, address: AddressType) -> bool:
             self.__getitem__(address)
             return True
 
-        except (IndexError, AttributeError):
+        except (IndexError, KeyError, AttributeError):
             return False
 
     def _verify_account_type(self, account):