Old install vsc-base get precedence over newly installed versions

[kwaegema]

giving strange output:

[kwaegema@daenerys vsc-base (master)]$ python setup.py install --user
run_tests from base dir /home/kwaegema/gitrepos/ugenthpc/vsc-base (using executable /home/kwaegema/gitrepos/ugenthpc/vsc-base/setup.py)
generated packages list: ['vsc', 'vsc.utils']
cleanup build
cleanup lib/vsc_base.egg-info
makesetupcfg set to True, (re)creating setup.cfg
New target = {'maintainer': 'Stijn De Weirdt;Jens Timmermans;Andy Georges;Kenneth Hoste', 'version': '2.4.16', 'install_requires': ['vsc-install >= 0.9.8'], 'setup_requires': ['vsc-install >= 0.9.8'], 'maintainer_email': '[email protected];[email protected];[email protected];[email protected]', 'author_email': '[email protected];[email protected];[email protected];[email protected]', 'packages': ['vsc', 'vsc.utils'], 'package_dir': {'': 'lib'}, 'author': 'Stijn De Weirdt;Jens Timmermans;Andy Georges;Kenneth Hoste', 'cmdclass': {'test': <class vsc.install.shared_setup.VscTestCommand at 0x7f6132244120>, 'bdist_rpm': <class vsc.install.shared_setup.vsc_bdist_rpm at 0x7f61322440b8>, 'egg_info': <class vsc.install.shared_setup.vsc_egg_info at 0x7f6132291e88>, 'install_scripts': <class vsc.install.shared_setup.vsc_install_scripts at 0x7f6132291f58>}, 'download_url': '', 'url': '', 'test_suite': 'test'}
Searching for vsc-install>=0.9.8
Reading https://pypi.python.org/simple/vsc-install/
Found link: https://pypi.python.org/packages/source/v/vsc-install/vsc-install-0.9.12.tar.gz#md5=ff9e84e6f9c766f2d6f6e2307f80c5c6
Found link: https://pypi.python.org/packages/source/v/vsc-install/vsc-install-0.9.3.tar.gz#md5=867900d4e14dcc324adcfa41176742c7
Found link: https://pypi.python.org/packages/source/v/vsc-install/vsc-install-0.9.6.tar.gz#md5=8b8c84021d7741e1e2118b6d42e39747
Best match: vsc-install 0.9.12
Downloading https://pypi.python.org/packages/source/v/vsc-install/vsc-install-0.9.12.tar.gz#md5=ff9e84e6f9c766f2d6f6e2307f80c5c6
Validating md5 checksum for /tmp/easy_install-pWtQJn/vsc-install-0.9.12.tar.gz
Processing vsc-install-0.9.12.tar.gz
Unpacking vsc-install-0.9.12 to /tmp/easy_install-pWtQJn/vsc-install-0.9.12
Unpacking vsc-install-0.9.12/LICENSE to /tmp/easy_install-pWtQJn/vsc-install-0.9.12/LICENSE
Unpacking vsc-install-0.9.12/setup.cfg to /tmp/easy_install-pWtQJn/vsc-install-0.9.12/setup.cfg
Unpacking vsc-install-0.9.12/lib to /tmp/easy_install-pWtQJn/vsc-install-0.9.12/lib
Unpacking vsc-install-0.9.12/lib/vsc_install.egg-info to /tmp/easy_install-pWtQJn/vsc-install-0.9.12/lib/vsc_install.egg-info
Unpacking vsc-install-0.9.12/lib/vsc_install.egg-info/dependency_links.txt to /tmp/easy_install-pWtQJn/vsc-install-0.9.12/lib/vsc_install.egg-info/dependency_links.txt
Unpacking vsc-install-0.9.12/lib/vsc_install.egg-info/requires.txt to /tmp/easy_install-pWtQJn/vsc-install-0.9.12/lib/vsc_install.egg-info/requires.txt
Unpacking vsc-install-0.9.12/lib/vsc_install.egg-info/top_level.txt to /tmp/easy_install-pWtQJn/vsc-install-0.9.12/lib/vsc_install.egg-info/top_level.txt
Unpacking vsc-install-0.9.12/lib/vsc_install.egg-info/PKG-INFO to /tmp/easy_install-pWtQJn/vsc-install-0.9.12/lib/vsc_install.egg-info/PKG-INFO
Unpacking vsc-install-0.9.12/lib/vsc_install.egg-info/SOURCES.txt to /tmp/easy_install-pWtQJn/vsc-install-0.9.12/lib/vsc_install.egg-info/SOURCES.txt
Unpacking vsc-install-0.9.12/lib/vsc to /tmp/easy_install-pWtQJn/vsc-install-0.9.12/lib/vsc
Unpacking vsc-install-0.9.12/lib/vsc/install to /tmp/easy_install-pWtQJn/vsc-install-0.9.12/lib/vsc/install
Unpacking vsc-install-0.9.12/lib/vsc/install/shared_setup.py to /tmp/easy_install-pWtQJn/vsc-install-0.9.12/lib/vsc/install/shared_setup.py
Unpacking vsc-install-0.9.12/lib/vsc/install/headers.py to /tmp/easy_install-pWtQJn/vsc-install-0.9.12/lib/vsc/install/headers.py
Unpacking vsc-install-0.9.12/lib/vsc/install/testing.py to /tmp/easy_install-pWtQJn/vsc-install-0.9.12/lib/vsc/install/testing.py
Unpacking vsc-install-0.9.12/lib/vsc/install/init.py to /tmp/easy_install-pWtQJn/vsc-install-0.9.12/lib/vsc/install/init.py
Unpacking vsc-install-0.9.12/lib/vsc/fancylogger.py to /tmp/easy_install-pWtQJn/vsc-install-0.9.12/lib/vsc/fancylogger.py
Unpacking vsc-install-0.9.12/lib/vsc/init.py to /tmp/easy_install-pWtQJn/vsc-install-0.9.12/lib/vsc/init.py
Unpacking vsc-install-0.9.12/README.md to /tmp/easy_install-pWtQJn/vsc-install-0.9.12/README.md
Unpacking vsc-install-0.9.12/PKG-INFO to /tmp/easy_install-pWtQJn/vsc-install-0.9.12/PKG-INFO
Unpacking vsc-install-0.9.12/setup.py to /tmp/easy_install-pWtQJn/vsc-install-0.9.12/setup.py
Reading configuration from /tmp/easy_install-pWtQJn/vsc-install-0.9.12/setup.cfg
Adding new section [easy_install] to /tmp/easy_install-pWtQJn/vsc-install-0.9.12/setup.cfg
Writing /tmp/easy_install-pWtQJn/vsc-install-0.9.12/setup.cfg
Running vsc-install-0.9.12/setup.py -q bdist_egg --dist-dir /tmp/easy_install-pWtQJn/vsc-install-0.9.12/egg-dist-tmp-BEYFhc
INFO: This is (based on) vsc.install.shared_setup 0.9.12
INFO: run_tests from base dir /tmp/easy_install-pWtQJn/vsc-install-0.9.12 (using executable /tmp/easy_install-pWtQJn/vsc-install-0.9.12/setup.py)
WARN: cleanup lib/vsc_install.egg-info
INFO: generated list: ['vsc.install', 'vsc']
INFO: generated packages list: ['vsc.install', 'vsc']
INFO: makesetupcfg set to True, (re)creating setup.cfg
INFO: found license /tmp/easy_install-pWtQJn/vsc-install-0.9.12/LICENSE with md5sum 5f30f0716dfdd0d91eb439ebec522ec2
INFO: Found license name LGPLv2+ and classifier License :: OSI Approved :: GNU Lesser General Public License v2 or later (LGPLv2+)
INFO: setting license LGPLv2+
INFO: No name defined, trying to determine it
INFO: found match url https://github.com/hpcugent/vsc-install in /tmp/easy_install-pWtQJn/vsc-install-0.9.12/PKG-INFO
INFO: found match name vsc-install in /tmp/easy_install-pWtQJn/vsc-install-0.9.12/PKG-INFO
INFO: Removing None download_url
INFO: get_name_url returns {'url': 'https://github.com/hpcugent/vsc-install', 'name': 'vsc-install'}
INFO: using long_description vsc-install provides shared setuptools functions and classes for python libraries developed by UGent's HPC group
INFO: generated list: []
INFO: generated scripts list: []
/usr/lib64/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'setup_requires'
warnings.warn(msg)
WARN: zip_safe flag not set; analyzing archive contents...
WARN: vsc.fancylogger: module references file
WARN: vsc.install.shared_setup: module references file
WARN: vsc.install.shared_setup: module references path
WARN: vsc.install.shared_setup: module MAY be using inspect.getsource
creating /home/kwaegema/gitrepos/ugenthpc/vsc-base/.eggs/vsc_install-0.9.12-py2.7.egg
Extracting vsc_install-0.9.12-py2.7.egg to /home/kwaegema/gitrepos/ugenthpc/vsc-base/.eggs
Unpacking EGG-INFO/not-zip-safe to /home/kwaegema/gitrepos/ugenthpc/vsc-base/.eggs/vsc_install-0.9.12-py2.7.egg/EGG-INFO/not-zip-safe
Unpacking EGG-INFO/top_level.txt to /home/kwaegema/gitrepos/ugenthpc/vsc-base/.eggs/vsc_install-0.9.12-py2.7.egg/EGG-INFO/top_level.txt
Unpacking EGG-INFO/requires.txt to /home/kwaegema/gitrepos/ugenthpc/vsc-base/.eggs/vsc_install-0.9.12-py2.7.egg/EGG-INFO/requires.txt
Unpacking EGG-INFO/dependency_links.txt to /home/kwaegema/gitrepos/ugenthpc/vsc-base/.eggs/vsc_install-0.9.12-py2.7.egg/EGG-INFO/dependency_links.txt
Unpacking EGG-INFO/SOURCES.txt to /home/kwaegema/gitrepos/ugenthpc/vsc-base/.eggs/vsc_install-0.9.12-py2.7.egg/EGG-INFO/SOURCES.txt
Unpacking EGG-INFO/PKG-INFO to /home/kwaegema/gitrepos/ugenthpc/vsc-base/.eggs/vsc_install-0.9.12-py2.7.egg/EGG-INFO/PKG-INFO
Unpacking vsc/init.pyc to /home/kwaegema/gitrepos/ugenthpc/vsc-base/.eggs/vsc_install-0.9.12-py2.7.egg/vsc/init.pyc
Unpacking vsc/fancylogger.pyc to /home/kwaegema/gitrepos/ugenthpc/vsc-base/.eggs/vsc_install-0.9.12-py2.7.egg/vsc/fancylogger.pyc
Unpacking vsc/init.py to /home/kwaegema/gitrepos/ugenthpc/vsc-base/.eggs/vsc_install-0.9.12-py2.7.egg/vsc/init.py
Unpacking vsc/fancylogger.py to /home/kwaegema/gitrepos/ugenthpc/vsc-base/.eggs/vsc_install-0.9.12-py2.7.egg/vsc/fancylogger.py
Unpacking vsc/install/init.pyc to /home/kwaegema/gitrepos/ugenthpc/vsc-base/.eggs/vsc_install-0.9.12-py2.7.egg/vsc/install/init.pyc
Unpacking vsc/install/testing.pyc to /home/kwaegema/gitrepos/ugenthpc/vsc-base/.eggs/vsc_install-0.9.12-py2.7.egg/vsc/install/testing.pyc
Unpacking vsc/install/headers.pyc to /home/kwaegema/gitrepos/ugenthpc/vsc-base/.eggs/vsc_install-0.9.12-py2.7.egg/vsc/install/headers.pyc
Unpacking vsc/install/shared_setup.pyc to /home/kwaegema/gitrepos/ugenthpc/vsc-base/.eggs/vsc_install-0.9.12-py2.7.egg/vsc/install/shared_setup.pyc
Unpacking vsc/install/init.py to /home/kwaegema/gitrepos/ugenthpc/vsc-base/.eggs/vsc_install-0.9.12-py2.7.egg/vsc/install/init.py
Unpacking vsc/install/testing.py to /home/kwaegema/gitrepos/ugenthpc/vsc-base/.eggs/vsc_install-0.9.12-py2.7.egg/vsc/install/testing.py
Unpacking vsc/install/headers.py to /home/kwaegema/gitrepos/ugenthpc/vsc-base/.eggs/vsc_install-0.9.12-py2.7.egg/vsc/install/headers.py
Unpacking vsc/install/shared_setup.py to /home/kwaegema/gitrepos/ugenthpc/vsc-base/.eggs/vsc_install-0.9.12-py2.7.egg/vsc/install/shared_setup.py

Installed /home/kwaegema/gitrepos/ugenthpc/vsc-base/.eggs/vsc_install-0.9.12-py2.7.egg
/usr/lib64/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'setup_requires'
warnings.warn(msg)
running install
running bdist_egg
running egg_info
creating lib/UNKNOWN.egg-info
writing requirements to lib/UNKNOWN.egg-info/requires.txt
writing lib/UNKNOWN.egg-info/PKG-INFO
writing top-level names to lib/UNKNOWN.egg-info/top_level.txt
writing dependency_links to lib/UNKNOWN.egg-info/dependency_links.txt
writing manifest file 'lib/UNKNOWN.egg-info/SOURCES.txt'
reading manifest file 'lib/UNKNOWN.egg-info/SOURCES.txt'
writing manifest file 'lib/UNKNOWN.egg-info/SOURCES.txt'
looking for extra dist files
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
creating build
creating build/lib
creating build/lib/vsc
copying lib/vsc/init.py -> build/lib/vsc
creating build/lib/vsc/utils
copying lib/vsc/utils/affinity.py -> build/lib/vsc/utils
copying lib/vsc/utils/run.py -> build/lib/vsc/utils
copying lib/vsc/utils/daemon.py -> build/lib/vsc/utils
copying lib/vsc/utils/rest.py -> build/lib/vsc/utils
copying lib/vsc/utils/optcomplete.py -> build/lib/vsc/utils
copying lib/vsc/utils/docs.py -> build/lib/vsc/utils
copying lib/vsc/utils/frozendict.py -> build/lib/vsc/utils
copying lib/vsc/utils/testing.py -> build/lib/vsc/utils
copying lib/vsc/utils/fancylogger.py -> build/lib/vsc/utils
copying lib/vsc/utils/exceptions.py -> build/lib/vsc/utils
copying lib/vsc/utils/mail.py -> build/lib/vsc/utils
copying lib/vsc/utils/patterns.py -> build/lib/vsc/utils
copying lib/vsc/utils/missing.py -> build/lib/vsc/utils
copying lib/vsc/utils/init.py -> build/lib/vsc/utils
copying lib/vsc/utils/generaloption.py -> build/lib/vsc/utils
copying lib/vsc/utils/asyncprocess.py -> build/lib/vsc/utils
copying lib/vsc/utils/dateandtime.py -> build/lib/vsc/utils
copying lib/vsc/utils/wrapper.py -> build/lib/vsc/utils
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/vsc
creating build/bdist.linux-x86_64/egg/vsc/utils
copying build/lib/vsc/utils/affinity.py -> build/bdist.linux-x86_64/egg/vsc/utils
copying build/lib/vsc/utils/run.py -> build/bdist.linux-x86_64/egg/vsc/utils
copying build/lib/vsc/utils/daemon.py -> build/bdist.linux-x86_64/egg/vsc/utils
copying build/lib/vsc/utils/rest.py -> build/bdist.linux-x86_64/egg/vsc/utils
copying build/lib/vsc/utils/optcomplete.py -> build/bdist.linux-x86_64/egg/vsc/utils
copying build/lib/vsc/utils/docs.py -> build/bdist.linux-x86_64/egg/vsc/utils
copying build/lib/vsc/utils/frozendict.py -> build/bdist.linux-x86_64/egg/vsc/utils
copying build/lib/vsc/utils/testing.py -> build/bdist.linux-x86_64/egg/vsc/utils
copying build/lib/vsc/utils/fancylogger.py -> build/bdist.linux-x86_64/egg/vsc/utils
copying build/lib/vsc/utils/exceptions.py -> build/bdist.linux-x86_64/egg/vsc/utils
copying build/lib/vsc/utils/mail.py -> build/bdist.linux-x86_64/egg/vsc/utils
copying build/lib/vsc/utils/patterns.py -> build/bdist.linux-x86_64/egg/vsc/utils
copying build/lib/vsc/utils/missing.py -> build/bdist.linux-x86_64/egg/vsc/utils
copying build/lib/vsc/utils/init.py -> build/bdist.linux-x86_64/egg/vsc/utils
copying build/lib/vsc/utils/generaloption.py -> build/bdist.linux-x86_64/egg/vsc/utils
copying build/lib/vsc/utils/asyncprocess.py -> build/bdist.linux-x86_64/egg/vsc/utils
copying build/lib/vsc/utils/dateandtime.py -> build/bdist.linux-x86_64/egg/vsc/utils
copying build/lib/vsc/utils/wrapper.py -> build/bdist.linux-x86_64/egg/vsc/utils
copying build/lib/vsc/init.py -> build/bdist.linux-x86_64/egg/vsc
byte-compiling build/bdist.linux-x86_64/egg/vsc/utils/affinity.py to affinity.pyc
byte-compiling build/bdist.linux-x86_64/egg/vsc/utils/run.py to run.pyc
byte-compiling build/bdist.linux-x86_64/egg/vsc/utils/daemon.py to daemon.pyc
byte-compiling build/bdist.linux-x86_64/egg/vsc/utils/rest.py to rest.pyc
byte-compiling build/bdist.linux-x86_64/egg/vsc/utils/optcomplete.py to optcomplete.pyc
byte-compiling build/bdist.linux-x86_64/egg/vsc/utils/docs.py to docs.pyc
byte-compiling build/bdist.linux-x86_64/egg/vsc/utils/frozendict.py to frozendict.pyc
byte-compiling build/bdist.linux-x86_64/egg/vsc/utils/testing.py to testing.pyc
byte-compiling build/bdist.linux-x86_64/egg/vsc/utils/fancylogger.py to fancylogger.pyc
byte-compiling build/bdist.linux-x86_64/egg/vsc/utils/exceptions.py to exceptions.pyc
byte-compiling build/bdist.linux-x86_64/egg/vsc/utils/mail.py to mail.pyc
byte-compiling build/bdist.linux-x86_64/egg/vsc/utils/patterns.py to patterns.pyc
byte-compiling build/bdist.linux-x86_64/egg/vsc/utils/missing.py to missing.pyc
byte-compiling build/bdist.linux-x86_64/egg/vsc/utils/init.py to init.pyc
byte-compiling build/bdist.linux-x86_64/egg/vsc/utils/generaloption.py to generaloption.pyc
byte-compiling build/bdist.linux-x86_64/egg/vsc/utils/asyncprocess.py to asyncprocess.pyc
byte-compiling build/bdist.linux-x86_64/egg/vsc/utils/dateandtime.py to dateandtime.pyc
byte-compiling build/bdist.linux-x86_64/egg/vsc/utils/wrapper.py to wrapper.pyc
byte-compiling build/bdist.linux-x86_64/egg/vsc/init.py to init.pyc
creating build/bdist.linux-x86_64/egg/EGG-INFO
copying lib/UNKNOWN.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying lib/UNKNOWN.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying lib/UNKNOWN.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying lib/UNKNOWN.egg-info/requires.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying lib/UNKNOWN.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
zip_safe flag not set; analyzing archive contents...
vsc.utils.exceptions: module MAY be using inspect.getouterframes
vsc.utils.fancylogger: module MAY be using inspect.stack
vsc.utils.generaloption: module MAY be using inspect.stack
creating 'dist/UNKNOWN-2.4.16-py2.7.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing UNKNOWN-2.4.16-py2.7.egg
creating /home/kwaegema/.local/lib/python2.7/site-packages/UNKNOWN-2.4.16-py2.7.egg
Extracting UNKNOWN-2.4.16-py2.7.egg to /home/kwaegema/.local/lib/python2.7/site-packages
Removing UNKNOWN 1.7.6 from easy-install.pth file
Adding UNKNOWN 2.4.16 to easy-install.pth file

Installed /home/kwaegema/.local/lib/python2.7/site-packages/UNKNOWN-2.4.16-py2.7.egg
Processing dependencies for UNKNOWN==2.4.16
Searching for vsc-install==0.9.12
Best match: vsc-install 0.9.12
Processing vsc_install-0.9.12-py2.7.egg
Removing vsc-install 0.9.1 from easy-install.pth file
Adding vsc-install 0.9.12 to easy-install.pth file

Using /home/kwaegema/gitrepos/ugenthpc/vsc-base/.eggs/vsc_install-0.9.12-py2.7.egg
Searching for setuptools==17.1.1
Best match: setuptools 17.1.1
Adding setuptools 17.1.1 to easy-install.pth file
Installing easy_install script to /home/kwaegema/.local/bin
Installing easy_install-2.7 script to /home/kwaegema/.local/bin

Using /usr/lib/python2.7/site-packages
Finished processing dependencies for UNKNOWN==2.4.16

[boegel]

The problem here is that the shared_setup.py of the system vsc-base installation in /usr/lib/... is being picked up, rather than the one that comes with vsc-install.

@stdweird: didn't you have trickery in vsc-install which is supposed to prevent this from happening?

[stdweird]

@boegel yes, but not if you install vsc-base like this
installing vsc-base from git requires to install recent vsc-install first.
the proper way to install vsc-base is to easy_install vsc-base, than this should not happen.

[JensTimmerman]

so the official answer here is:
if you're installing things from git or a downloaded tarball of the sources, you should be aware of what you are doing (you're not just a user who does easy_install/pip install xxx)
so you should be able to figure out you need to remove these old versions.

[boegel]

An answer like this is not going to fly for EasyBuild, if I have a say in it (which is troublesome, since vsc-base is sort of part of EB, and not).

For our own tools, sure, makes sense.

[stdweird]

unless EB uses vsc-install, this has nothing to do with EB. vsc-base is just a dependency of EB, nothing more.

[JensTimmerman]

wel it would be nicer if the installation just failed somewhere instead of using UNKNOWN, so it's clearer that something is wrong.
I don't think this issue is anything that hasn't always been the case with python packages.

sometimes an old version in /usr/lib/python/site-packages wil still be used if you installed with --user.
This is an issue that has alweys been the case.