Skip to content

Commit

Permalink
Merge pull request #164 from martinholmer/revise-local
Browse files Browse the repository at this point in the history 
Simplify --local option and generalize its version logic
  • Loading branch information
@martinholmer
martinholmer authored Apr 15, 2019
2 parents 7c5d5db + 9f96c5c commit 5280eeb
Show file tree
Hide file tree
Showing 4 changed files with 95 additions and 78 deletions.
27 changes: 13 additions & 14 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ and getting something like this screen output:
```
$ pbrelease --help
usage: pbrelease REPOSITORY_NAME PACKAGE_NAME MODEL_VERSION
[--help] [--local LOCAL] [--dryrun] [--version]
[--help] [--local] [--dryrun] [--version]
Creates conda packages named PACKAGE_NAME for the PSL model in REPOSITORY_NAME
that has a GitHub release named MODEL_VERSION. The packages are built locally
Expand All @@ -76,12 +76,10 @@ positional arguments:
optional arguments:
-h, --help show this help message and exit
--local LOCAL optional flag where LOCAL is name of top-level directory
containing the model conda.recipe directory used to build
the package that is installed on local computer; no --local
option implies model source code is cloned from GitHub
REPOSITORY_NAME for MODEL_RELEASE and packages are uploaded
to Anaconda Cloud PSLmodels channel for public distribution
--local optional flag that causes package to be built from current
source code and installed on local computer without
packages being uploaded to Anaconda Cloud PSLmodels
channel.
--dryrun optional flag that writes execution plan to stdout and
quits without executing plan
--version optional flag that writes Package-Builder release version
Expand Down Expand Up @@ -142,15 +140,17 @@ $ pbrelease Behavioral-Responses behresp 0.5.0
To use `pbrelease` to make a **local package** from the current (even
uncommitted) code on your current branch, make the top-level directory
of the repo's source-code tree the current working directory. Then
use the `--local .` option as shown here:
use the `--local` option as shown here:

```
$ cd Tax-Calculator
$ pbrelease Tax-Calculator taxcalc 0.0.0 --local .
$ pbrelease Tax-Calculator taxcalc 0.0.0 --local
```

This will produce a local package on your computer for testing or validation
work. You can uninstall this local package at any time using this command:
This will produce a local package on your computer for testing or
validation work. The above example specifies version 0.0.0, but you
can specify any version you want. You can uninstall this local
package at any time using this command:

```
$ conda uninstall taxcalc --yes
Expand Down Expand Up @@ -190,7 +190,6 @@ work correctly. Here are the model code criteria:
The best place to look at code that meets all these criteria is the
PSLmodels Behavioral-Responses repository.

Version 0.0.0 indicates a local package; the `pbrelease` tool will
Version 0.0.0 indicates the earliest version; the `pbrelease` tool will
automatically replace (in a temporary copy of your model code) the
0.0.0 with the X.Y.Z version specified when you execute `pbrelease`
without using the `--local` option.
0.0.0 with the X.Y.Z version specified when you execute `pbrelease`.
16 changes: 16 additions & 0 deletions RELEASES.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,22 @@ Go [here](https://github.com/PSLmodels~/Package-Builder/pulls?q=is%3Apr+is%3Aclo
for a complete commit history.


2019-04-15 Release 0.22.0
-------------------------
(last merged pull request is
[#164](https://github.com/PSLmodels/Package-Builder/pull/164))

**API Changes**
- None

**New Features**
- Simplify `--local` so that it is a simple boolean option and allow the locally installed package to be specified with any version
[[#164](https://github.com/PSLmodels/Package-Builder/pull/164)
by Martin Holmer]

**Bug Fixes**
- None

2019-03-13 Release 0.21.0
-------------------------
(last merged pull request is
Expand Down
34 changes: 16 additions & 18 deletions pkgbld/cli.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ def main():
"""
# parse command-line arguments:
usage_str = ('pbrelease REPOSITORY_NAME PACKAGE_NAME MODEL_VERSION\n'
' [--help] [--local LOCAL] '
' [--help] [--local] '
'[--dryrun] [--version]')
parser = argparse.ArgumentParser(
prog='',
Expand All @@ -45,16 +45,12 @@ def main():
'Example: 1.0.1'),
default=None)
parser.add_argument('--local',
help=('optional flag where LOCAL is name of top-level '
'directory containing the model conda.recipe '
'directory used to build the package that is '
'installed on local computer; '
'no --local option implies model source code '
'is cloned from GitHub REPOSITORY_NAME for '
'MODEL_RELEASE and packages are uploaded to '
'Anaconda Cloud PSLmodels channel for public '
'distribution'),
default=None)
help=('optional flag that causes package to be '
'built from current source code and installed '
'on local computer without packages being '
'uploaded to Anaconda Cloud PSLmodels channel.'),
default=False,
action="store_true")
parser.add_argument('--dryrun',
help=('optional flag that writes execution plan '
'to stdout and quits without executing plan'),
Expand Down Expand Up @@ -90,17 +86,19 @@ def main():
emsg += ('ERROR: Anaconda token file {} '
'does not exist\n'.format(pkgbld.ANACONDA_TOKEN_FILE))
if args.local:
if not os.path.isdir(args.local):
emsg += ('ERROR: LOCAL directory {} '
'does not exist\n'.format(args.local))
if version != '0.0.0':
emsg += ('ERROR: MODEL_VERSION {} is not 0.0.0 '
'when using --local option\n'.format(version))
cwd = os.getcwd()
if not cwd.endswith(repo_name):
emsg += ('ERROR: cwd={} does not correspond to '
'REPOSITORY_NAME={}\n'.format(cwd, repo_name))
local_pkgname = os.path.join('.', pkg_name)
if not os.path.isdir(local_pkgname):
emsg += ('ERROR: cwd={} does not contain '
'subdirectory {} '.format(cwd, pkg_name))
if emsg:
print(emsg)
print('USAGE:', usage_str)
return 1
# call pkgbld release function with specified parameters
pkgbld.release(repo_name, pkg_name, version,
localdir=args.local, dryrun=args.dryrun)
local=args.local, dryrun=args.dryrun)
return 0
96 changes: 50 additions & 46 deletions pkgbld/release.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -31,16 +31,16 @@
BUILDS_DIR = 'pkgbld_output'


def release(repo_name, pkg_name, version, localdir=None, dryrun=False):
def release(repo_name, pkg_name, version, local=False, dryrun=False):
"""
If localdir==None, conduct build using cloned source code and
upload to Anaconda Cloud of conda packages for each operating-system
If local==False, conduct build using cloned source code and
upload to Anaconda Cloud conda packages for each operating-system
platform and Python version for the specified Policy Simulation Library
(PSL) model and GitHub release version.
If localdir==string, build from source code in localdir and skip the
convert and upload steps instead installing the built package on the
local computer.
If local==True, build from source code in current working directory and
skip the convert and upload steps instead installing the built package on
the local computer.
Parameters
----------
Expand All @@ -54,9 +54,8 @@ def release(repo_name, pkg_name, version, localdir=None, dryrun=False):
model version string having X.Y.Z semantic-versioning pattern;
must be a release tag in the model repository
localdir: None or string
localdir containing model source code; None implies cloning
source code from GitHub repository
local: boolean
whether or not to build/install local package
dryrun: boolean
whether or not just the package build/upload plan is shown
Expand Down Expand Up @@ -84,25 +83,31 @@ def release(repo_name, pkg_name, version, localdir=None, dryrun=False):
raise ValueError('pkg_name is not a string object')
if not isinstance(version, str):
raise ValueError('version is not a string object')
if not (localdir is None or isinstance(localdir, str)):
raise ValueError('localdir is not None or a string object')
if isinstance(localdir, str):
if not os.path.isdir(localdir):
raise ValueError('localdir is not a directory')
if version != '0.0.0':
raise ValueError('version is not 0.0.0 when using --local option')
if not isinstance(local, bool):
raise ValueError('local is not a boolean object')
if local:
cwd = os.getcwd()
if not cwd.endswith(repo_name):
msg = ('ERROR: cwd={} does not correspond to '
'REPOSITORY_NAME={}\n'.format(cwd, repo_name))
raise ValueError(msg)
local_pkgname = os.path.join('.', pkg_name)
if not os.path.isdir(local_pkgname):
msg = ('ERROR: cwd={} does not contain '
'subdirectory {} '.format(cwd, pkg_name))
raise ValueError(msg)
if not isinstance(dryrun, bool):
raise ValueError('dryrun is not a boolean object')
pattern = r'^[0-9]+\.[0-9]+\.[0-9]+$'
if re.match(pattern, version) is None:
msg = 'version={} does not have X.Y.Z semantic-versioning pattern'
raise ValueError(msg.format(version))

# specify Python versions list, which depends on localdir
# specify Python versions list, which depends on local
assert sys.version_info[0] == 3
local_python_version = '3.{}'.format(sys.version_info[1])
python_versions = [local_python_version] # always first in the list
if not localdir:
if not local:
for ver in ALL_PYTHON_VERSIONS:
if ver not in python_versions:
python_versions.append(ver)
Expand All @@ -113,7 +118,7 @@ def release(repo_name, pkg_name, version, localdir=None, dryrun=False):
print(': package_name = {}'.format(pkg_name))
print(': model_version = {}'.format(version))
print(': python_versions = {}'.format(python_versions))
if localdir:
if local:
print(': Package-Builder will install package on local computer')
else:
print(': Package-Builder will upload model packages to:')
Expand All @@ -131,12 +136,12 @@ def release(repo_name, pkg_name, version, localdir=None, dryrun=False):
shutil.rmtree(WORKING_DIR)

# copy model source code to working directory
if localdir:
if local:
# copy source tree on local computer
print(': Package-Builder is copying local source code')
destination = os.path.join(WORKING_DIR, repo_name)
ignorepattern = shutil.ignore_patterns('*.pyc', '*.html', 'test_*')
shutil.copytree(localdir, destination, ignore=ignorepattern)
shutil.copytree(cwd, destination, ignore=ignorepattern)
os.chdir(WORKING_DIR)
else:
# clone code for model_version from model repository
Expand All @@ -151,26 +156,25 @@ def release(repo_name, pkg_name, version, localdir=None, dryrun=False):
os.chdir(repo_name)

# specify version in several repository files
if not localdir:
print(': Package-Builder is setting version')
# ... specify version in meta.yaml file
u.file_revision(
filename=os.path.join('conda.recipe', 'meta.yaml'),
pattern=r'version: .*',
replacement='version: {}'.format(version)
)
# ... specify version in setup.py file
u.file_revision(
filename='setup.py',
pattern=r'version = .*',
replacement='version = "{}"'.format(version)
)
# ... specify version in package_name/__init__.py file
u.file_revision(
filename=os.path.join(pkg_name, '__init__.py'),
pattern=r'__version__ = .*',
replacement='__version__ = "{}"'.format(version)
)
print(': Package-Builder is setting version')
# ... specify version in meta.yaml file
u.file_revision(
filename=os.path.join('conda.recipe', 'meta.yaml'),
pattern=r'version: .*',
replacement='version: {}'.format(version)
)
# ... specify version in setup.py file
u.file_revision(
filename='setup.py',
pattern=r'version = .*',
replacement='version = "{}"'.format(version)
)
# ... specify version in package_name/__init__.py file
u.file_revision(
filename=os.path.join(pkg_name, '__init__.py'),
pattern=r'__version__ = .*',
replacement='__version__ = "{}"'.format(version)
)

# build and upload model package for each Python version and OS platform
local_platform = u.conda_platform_name()
Expand All @@ -183,8 +187,8 @@ def release(repo_name, pkg_name, version, localdir=None, dryrun=False):
'--no-anaconda-upload --output-folder {} '
'conda.recipe').format(pyver, ANACONDA_CHANNEL, BUILDS_DIR)
u.os_call(cmd)
# ... if localdir is specified, skip convert and upload logic
if localdir:
# ... if local is True, skip convert and upload logic
if local:
break # out of for pyver loop
# ... convert local build to other OS_PLATFORMS
print((': Package-Builder is converting package '
Expand All @@ -209,16 +213,16 @@ def release(repo_name, pkg_name, version, localdir=None, dryrun=False):
ANACONDA_TOKEN_FILE, ANACONDA_USER, pkgpath
)
u.os_call(cmd)
if localdir:
if local:
# do uninstall and install on local computer
print(': Package-Builder is uninstalling any existing package')
cmd = 'conda uninstall {} --yes'.format(pkg_name)
u.os_call(cmd, ignore_error=True)
print(': Package-Builder is installing package on local computer')
pkg_dir = os.path.join('file://', WORKING_DIR,
repo_name, 'pkgbld_output')
cmd = 'conda install --channel {} {}=0.0.0 --yes'
u.os_call(cmd.format(pkg_dir, pkg_name))
cmd = 'conda install --channel {} {}={} --yes'
u.os_call(cmd.format(pkg_dir, pkg_name, version))

print(': Package-Builder is cleaning-up')

Expand Down

0 comments on commit 5280eeb

Please sign in to comment.