Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[solution] Habitat_sim installs failing in Colab #1818

Closed
srkiyengar opened this issue Jul 19, 2022 · 11 comments
Closed

[solution] Habitat_sim installs failing in Colab #1818

srkiyengar opened this issue Jul 19, 2022 · 11 comments
Assignees
Labels
bug Something isn't working solution proposed A solution or workaround is proposed but may not be merged to main. wontfix This will not be worked on

Comments

@srkiyengar
Copy link

Recently, habitat_sim colab installs seem to fail in "import habitat-sim" after successful installtion. It seems to be a complaining about missing library:
ImportError: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.26' not found (required by /usr/local/lib/python3.7/dist-packages/scipy/optimize/_group_columns.cpython-37m-x86_64-linux-gnu.so)

The two colab habitat-sim install that I have tried are,

  1. https://colab.research.google.com/github/facebookresearch/habitat-sim/blob/main/examples/tutorials/colabs/ECCV_2020_Interactivity.ipynb

  2. !curl -L https://raw.githubusercontent.com/facebookresearch/habitat-sim/master/examples/colab_utils/colab_install.sh | NIGHTLY=false bash -s

Error details while importing habitat-sim in ECCV_2020_Interactivity.ipynb

ImportError Traceback (most recent call last)
in ()
17 from PIL import Image
18
---> 19 import habitat_sim
20 from habitat_sim.utils import common as ut
21 from habitat_sim.utils import viz_utils as vut

11 frames
/usr/local/lib/python3.7/dist-packages/scipy/optimize/_numdiff.py in ()
6 from scipy.sparse.linalg import LinearOperator
7 from ..sparse import issparse, csc_matrix, csr_matrix, coo_matrix, find
----> 8 from ._group_columns import group_dense, group_sparse
9
10

ImportError: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.26' not found (required by /usr/local/lib/python3.7/dist-packages/scipy/optimize/_group_columns.cpython-37m-x86_64-linux-gnu.so)

Error details of "import habitat-sim" after install from "https://raw.githubusercontent.com/facebookresearch/habitat-sim/master/examples/colab_utils/colab_install.sh | NIGHTLY=false bash -s"

/content/oreo-pybullet/agent_oreo.py in ()
----> 1 import habitat_sim
2 import numpy as np
3 import quaternion
4 import math
5 import cv2

/usr/local/lib/python3.7/dist-packages/habitat_sim/init.py in ()
11 if not getattr(builtins, "HSIM_SETUP", False):
12 # TODO: kept only for compatibiliy with existing code. Please gradually remove
---> 13 from habitat_sim import (
14 agent,
15 attributes,

/usr/local/lib/python3.7/dist-packages/habitat_sim/agent/init.py in ()
7 # type: ignore
8
----> 9 from .agent import * # noqa: F403
10 from .controls import * # noqa: F403
11

/usr/local/lib/python3.7/dist-packages/habitat_sim/agent/agent.py in ()
29 )
30
---> 31 from .controls import ActuationSpec, ObjectControls
32
33 all = ["ActionSpec", "SixDOFPose", "AgentState", "AgentConfiguration", "Agent"]

/usr/local/lib/python3.7/dist-packages/habitat_sim/agent/controls/init.py in ()
8 from habitat_sim.agent.controls.default_controls import * # noqa: F401, F403
9 from habitat_sim.agent.controls.object_controls import ObjectControls
---> 10 from habitat_sim.agent.controls.pyrobot_noisy_controls import PyRobotNoisyActuationSpec
11
12 all = [

/usr/local/lib/python3.7/dist-packages/habitat_sim/agent/controls/pyrobot_noisy_controls.py in ()
18 import magnum as mn
19 import numpy as np
---> 20 import scipy.stats
21 from attr import Attribute
22 from numpy import ndarray

/usr/local/lib/python3.7/dist-packages/scipy/stats/init.py in ()
439 """
440
--> 441 from .stats import *
442 from .distributions import *
443 from .morestats import *

/usr/local/lib/python3.7/dist-packages/scipy/stats/stats.py in ()
41 import scipy.special as special
42 from scipy import linalg
---> 43 from . import distributions
44 from . import mstats_basic
45 from ._stats_mstats_common import (_find_repeats, linregress, theilslopes,

/usr/local/lib/python3.7/dist-packages/scipy/stats/distributions.py in ()
6 # instead of git blame -Lxxx,+x.
7 #
----> 8 from ._distn_infrastructure import (rv_discrete, rv_continuous, rv_frozen)
9
10 from . import _continuous_distns

/usr/local/lib/python3.7/dist-packages/scipy/stats/_distn_infrastructure.py in ()
22 # for root finding for continuous distribution ppf, and max likelihood
23 # estimation
---> 24 from scipy import optimize
25
26 # for functions of continuous distributions (e.g. moments, entropy, cdf)

/usr/local/lib/python3.7/dist-packages/scipy/optimize/init.py in ()
398 """
399
--> 400 from .optimize import *
401 from ._minimize import *
402 from ._root import *

/usr/local/lib/python3.7/dist-packages/scipy/optimize/optimize.py in ()
34 line_search_wolfe2 as line_search,
35 LineSearchWarning)
---> 36 from ._numdiff import approx_derivative
37 from scipy._lib._util import getfullargspec_no_self as _getfullargspec
38 from scipy._lib._util import MapWrapper

/usr/local/lib/python3.7/dist-packages/scipy/optimize/_numdiff.py in ()
6 from scipy.sparse.linalg import LinearOperator
7 from ..sparse import issparse, csc_matrix, csr_matrix, coo_matrix, find
----> 8 from ._group_columns import group_dense, group_sparse
9
10

ImportError: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.26' not found (required by /usr/local/lib/python3.7/dist-packages/scipy/optimize/_group_columns.cpython-37m-x86_64-linux-gnu.so)

Hopefully the details are sufficient to troubleshoot the colab environment working.The installs used to work nicely the last time I checked 3-4 weeks ago.
Thanks.

@dhruvbatra
Copy link
Contributor

@Skylion007 -- thoughts?

@srkiyengar
Copy link
Author

Strings command of /usr/lib/x86_64-linux-gnu/libstdc++.so.6 for the colab install.
image

@arjunmajum
Copy link

Here's temporary fix that worked for me from the second solution here.

Run this one time:

!add-apt-repository ppa:ubuntu-toolchain-r/test -y
!apt-get update --fix-missing
!apt-get dist-upgrade

Then, restart the runtime -- or -- modify the imports to automatically restart:

try:
  import habitat_sim
except ImportError:
  exit()

@srkiyengar
Copy link
Author

Thank you @arjunmajum.

@Skylion007
Copy link
Contributor

@Skylion007 -- thoughts?

Trying to find a long term fix here that doesn't require a Colab restart. I am really confused where the new LIBCXX is being installed / overwritten from.

@srkiyengar
Copy link
Author

@arjunmajum's suggestion of restart worked well and was very usable. But as of last evening, executing the install script to install habitat-sim is running into another issue.

PackagesNotFoundError: The following packages are not available from current channels:

  • numba=0.56.0

Any suggestion or work around will be appreciated.


!curl -L https://raw.githubusercontent.com/facebookresearch/habitat-sim/master/examples/colab_utils/colab_install.sh | NIGHTLY=false bash -s

--2022-08-02 13:43:47-- https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
Resolving repo.continuum.io (repo.continuum.io)... 104.18.200.79, 104.18.201.79, 2606:4700::6812:c94f, ...
Connecting to repo.continuum.io (repo.continuum.io)|104.18.200.79|:443... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh [following]
--2022-08-02 13:43:47-- https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
Resolving repo.anaconda.com (repo.anaconda.com)... 104.16.130.3, 104.16.131.3, 2606:4700::6810:8203, ...
Connecting to repo.anaconda.com (repo.anaconda.com)|104.16.130.3|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 76607678 (73M) [application/x-sh]
Saving to: ‘Miniconda3-latest-Linux-x86_64.sh’

Miniconda3-latest-L 100%[===================>] 73.06M 234MB/s in 0.3s

2022-08-02 13:43:48 (234 MB/s) - ‘Miniconda3-latest-Linux-x86_64.sh’ saved [76607678/76607678]

PREFIX=/usr/local
Unpacking payload ...
Collecting package metadata (current_repodata.json): done
Solving environment: done

Package Plan

environment location: /usr/local

added / updated specs:
- _libgcc_mutex==0.1=main
- _openmp_mutex==4.5=1_gnu
- brotlipy==0.7.0=py39h27cfd23_1003
- ca-certificates==2022.3.29=h06a4308_1
- certifi==2021.10.8=py39h06a4308_2
- cffi==1.15.0=py39hd667e15_1
- charset-normalizer==2.0.4=pyhd3eb1b0_0
- colorama==0.4.4=pyhd3eb1b0_0
- conda-content-trust==0.1.1=pyhd3eb1b0_0
- conda-package-handling==1.8.1=py39h7f8727e_0
- conda==4.12.0=py39h06a4308_0
- cryptography==36.0.0=py39h9ce1e76_0
- idna==3.3=pyhd3eb1b0_0
- ld_impl_linux-64==2.35.1=h7274673_9
- libffi==3.3=he6710b0_2
- libgcc-ng==9.3.0=h5101ec6_17
- libgomp==9.3.0=h5101ec6_17
- libstdcxx-ng==9.3.0=hd4cf53a_17
- ncurses==6.3=h7f8727e_2
- openssl==1.1.1n=h7f8727e_0
- pip==21.2.4=py39h06a4308_0
- pycosat==0.6.3=py39h27cfd23_0
- pycparser==2.21=pyhd3eb1b0_0
- pyopenssl==22.0.0=pyhd3eb1b0_0
- pysocks==1.7.1=py39h06a4308_0
- python==3.9.12=h12debd9_0
- readline==8.1.2=h7f8727e_1
- requests==2.27.1=pyhd3eb1b0_0
- ruamel_yaml==0.15.100=py39h27cfd23_0
- setuptools==61.2.0=py39h06a4308_0
- six==1.16.0=pyhd3eb1b0_1
- sqlite==3.38.2=hc218d9a_0
- tk==8.6.11=h1ccaba5_0
- tqdm==4.63.0=pyhd3eb1b0_0
- tzdata==2022a=hda174b7_0
- urllib3==1.26.8=pyhd3eb1b0_0
- wheel==0.37.1=pyhd3eb1b0_0
- xz==5.2.5=h7b6447c_0
- yaml==0.2.5=h7b6447c_0
- zlib==1.2.12=h7f8727e_1

The following NEW packages will be INSTALLED:

_libgcc_mutex pkgs/main/linux-64::_libgcc_mutex-0.1-main
_openmp_mutex pkgs/main/linux-64::_openmp_mutex-4.5-1_gnu
brotlipy pkgs/main/linux-64::brotlipy-0.7.0-py39h27cfd23_1003
ca-certificates pkgs/main/linux-64::ca-certificates-2022.3.29-h06a4308_1
certifi pkgs/main/linux-64::certifi-2021.10.8-py39h06a4308_2
cffi pkgs/main/linux-64::cffi-1.15.0-py39hd667e15_1
charset-normalizer pkgs/main/noarch::charset-normalizer-2.0.4-pyhd3eb1b0_0
colorama pkgs/main/noarch::colorama-0.4.4-pyhd3eb1b0_0
conda pkgs/main/linux-64::conda-4.12.0-py39h06a4308_0
conda-content-tru~ pkgs/main/noarch::conda-content-trust-0.1.1-pyhd3eb1b0_0
conda-package-han~ pkgs/main/linux-64::conda-package-handling-1.8.1-py39h7f8727e_0
cryptography pkgs/main/linux-64::cryptography-36.0.0-py39h9ce1e76_0
idna pkgs/main/noarch::idna-3.3-pyhd3eb1b0_0
ld_impl_linux-64 pkgs/main/linux-64::ld_impl_linux-64-2.35.1-h7274673_9
libffi pkgs/main/linux-64::libffi-3.3-he6710b0_2
libgcc-ng pkgs/main/linux-64::libgcc-ng-9.3.0-h5101ec6_17
libgomp pkgs/main/linux-64::libgomp-9.3.0-h5101ec6_17
libstdcxx-ng pkgs/main/linux-64::libstdcxx-ng-9.3.0-hd4cf53a_17
ncurses pkgs/main/linux-64::ncurses-6.3-h7f8727e_2
openssl pkgs/main/linux-64::openssl-1.1.1n-h7f8727e_0
pip pkgs/main/linux-64::pip-21.2.4-py39h06a4308_0
pycosat pkgs/main/linux-64::pycosat-0.6.3-py39h27cfd23_0
pycparser pkgs/main/noarch::pycparser-2.21-pyhd3eb1b0_0
pyopenssl pkgs/main/noarch::pyopenssl-22.0.0-pyhd3eb1b0_0
pysocks pkgs/main/linux-64::pysocks-1.7.1-py39h06a4308_0
python pkgs/main/linux-64::python-3.9.12-h12debd9_0
readline pkgs/main/linux-64::readline-8.1.2-h7f8727e_1
requests pkgs/main/noarch::requests-2.27.1-pyhd3eb1b0_0
ruamel_yaml pkgs/main/linux-64::ruamel_yaml-0.15.100-py39h27cfd23_0
setuptools pkgs/main/linux-64::setuptools-61.2.0-py39h06a4308_0
six pkgs/main/noarch::six-1.16.0-pyhd3eb1b0_1
sqlite pkgs/main/linux-64::sqlite-3.38.2-hc218d9a_0
tk pkgs/main/linux-64::tk-8.6.11-h1ccaba5_0
tqdm pkgs/main/noarch::tqdm-4.63.0-pyhd3eb1b0_0
tzdata pkgs/main/noarch::tzdata-2022a-hda174b7_0
urllib3 pkgs/main/noarch::urllib3-1.26.8-pyhd3eb1b0_0
wheel pkgs/main/noarch::wheel-0.37.1-pyhd3eb1b0_0
xz pkgs/main/linux-64::xz-5.2.5-h7b6447c_0
yaml pkgs/main/linux-64::yaml-0.2.5-h7b6447c_0
zlib pkgs/main/linux-64::zlib-1.2.12-h7f8727e_1

Preparing transaction: done
Executing transaction: done
installation finished.
WARNING:
You currently have a PYTHONPATH environment variable set. This may cause
unexpected behavior when running the Python interpreter in Miniconda3.
For best results, please verify that your PYTHONPATH only points to
directories of packages that are compatible with the Python interpreter
in Miniconda3: /usr/local
Collecting package metadata (current_repodata.json): done
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: failed

PackagesNotFoundError: The following packages are not available from current channels:

  • numba=0.56.0

Current channels:

To search for alternate channels that may provide the conda package you're
looking for, navigate to

https://anaconda.org/

and use the search bar at the top of the page.

An error occured during the installation of Habitat-sim or Habitat-Lab.


@pkachana3
Copy link

@srkiyengar Were you able to get this installation issue resolved? I am having the same problem as well.

@srkiyengar
Copy link
Author

srkiyengar commented Aug 6, 2022

Not properly @pkachana3.
"conda search -f numba" stops at numba version 0.55.1 where as the colab default seems to be 0.56.0

I harded coded numba to 0.51.2. In the install script:

conda install -S -y --prefix /usr/local -c "${CHANNEL}" -c conda-forge habitat-sim=0.2.0 headless withbullet "python=${PYTHON_VERSION}" "numpy=${NUMPY_VERSION}" "pillow=${PIL_VERSION}" "cffi=${CFFI_VERSION}" "scipy=${SCIPY_VERSION}" "numba=0.51.2"

It seems to install the habitat.

@aclegg3 aclegg3 added the bug Something isn't working label Aug 12, 2022
@aclegg3
Copy link
Contributor

aclegg3 commented Aug 12, 2022

Thanks for the temporary solutions here, folks! With these I'm able to run the colabs successfully. @Skylion007 and I will work on a real fix.

For now, you can:

  1. modify the install cell to use the installer on a branch with the change suggested by @srkiyengar:
https://raw.githubusercontent.com/facebookresearch/habitat-sim/colab-numba-version/examples/colab_utils/colab_install.sh

Edited path 10/11

  1. apply the fix suggested by @arjunmajum in this comment:

Run this one time:

!add-apt-repository ppa:ubuntu-toolchain-r/test -y
!apt-get update --fix-missing
!apt-get dist-upgrade

Then, restart the runtime

@aclegg3 aclegg3 added the solution proposed A solution or workaround is proposed but may not be merged to main. label Sep 1, 2022
@aclegg3 aclegg3 pinned this issue Dec 14, 2022
@rpartsey
Copy link
Collaborator

@aclegg3 thank you for advice!
works for me

@aclegg3 aclegg3 changed the title Habitat_sim installs failing in Colab [solution] Habitat_sim installs failing in Colab Jan 11, 2023
@aclegg3 aclegg3 added the wontfix This will not be worked on label Jan 16, 2024
@aclegg3
Copy link
Contributor

aclegg3 commented Jan 16, 2024

Note we have stopped supporting colab and converted tutorials to local Jupyter notebooks instead.

@aclegg3 aclegg3 closed this as completed Jan 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working solution proposed A solution or workaround is proposed but may not be merged to main. wontfix This will not be worked on
Projects
None yet
Development

No branches or pull requests

7 participants