CLI maybe?

[pipermerriam]

What was wrong?

It's still kind of hard for people to work with ethpm packages.

1. Vyper might add support for using EthPM for contract interfaces
2. Web3.py supports working with packages but right now you have to do a bit of configuration to make it work.
3. Maybe we get Solidity onboard at some point, or web3.js, or brownie, or ....

But other than the first two which are snake charmer projects, it's likely that the 3rd party projects end up implementing support in their own

How can it be fixed?

This has me thinking about the concept of filesystem installs for ethpm packages. I implemented this once in populus a long time ago:

https://github.com/ethereum/populus/blob/4e9cc1efcea2333f48ceec9081ef5392d87608e8/populus/packages/installation.py

I'm wondering if there's utility in shipping a CLI for installing packages to disk as well as going ahead and defining the specification for the layout of packages on disk so that 3rd party integration is well supported. This would lower barrier to using EthPM based features for both Vyper and Web3.py use case and should reduce some barriers to adding support to some of those other 3rd party frameworks.

Here's the general layout as I remember it from the Populus implementation.

ROUGH Filesystem Installation Spec

The below is written using UNIX environment variable syntax for simplicity sake.

• let $CWD be the current working directory.
• let $PACKAGES_DIR be the directory that packages are installed to. This defaults to the relative path ./ethpm_packages
  • If the path is relative it is resolved with respect to $CWD

To install a package a user must provide the following information.

• PACKAGE_IDENTIFIER: specifies the package to install
• ALIAS: an optional alias name to install the package under
  • This is needed to allow installation of multiple versions of the same package under the same the same namespace.
  • Subject to the same name validity rules as package_name from the EthPM spec

The PACKAGE_IDENTIFIER concept should be fully fleshed out, potentially using a URI scheme, however for this spec, I'll define two identifier types which can generally be mapped to URI schemes.

• By Content Hash (such as an IPFS or SWARM URI)
• A Registry URI:
  • NAME: The name of the package (as it exists in the registry)
  • REGISTRY_URI: the package registry to install the package from.
  • VERSION: The version identifier to use to resolve the package.
    • Things like =1.2.3, >=1.2.3,<1.8, etc

Installation of packages is a recursive operation, with the top level namespace being comprised of user specified packages and deeper name spacing being the dependencies of the user specified packages. Here is what a single package namespace looks like.

|--- CWD
  |---PACKAGES_DIR
    |--- ethpm.lock
    |--- <package-name>
    | |--- package.json
    | |--- SRC/<package-source-tree>
    | |--- PACKAGES_DIR  (same directory format containing installed dependencies for this package.
    |   |--- <...>
    |--- <...>
```

When a user installs a package, the directory structure above is lazily created.  Each top level package is installed to it's own directory as follows.

- `$PACKAGES_DIR/<package-name>/`

Within that directory, the following files and directories are created.

- `package.json`: The package manifest for the package
- A directory named `./src` which contains the fully resolved package source tree.
- another `$PACKAGES_DIR` which contains all of the installed dependencies for the package.
    - This could probably be omitted for packages without dependencies.

Each `$PACKAGES_DIR` contains a top level file named `ethpm.lock`.  The purpose of this file is track what packages are installed and what mechanism they were installed with.  This is necessary to preserve the information of where packages came from in for packages installed via mechanisms like registry URIs.  The file structure is out of scope for this spec but it would contain thing like the following.

- The registry address
- The package name specified by the user
- The alias for the package
- The version identifier specified by the user
- The version that was resolved (and installed)
- The resolved content hash (probably normalized)