"Getting started" questions

[ThePassedWind]

After I succeeded in executing those commands:

1. ./install
2. benchbot_install --addons benchbot-addons/ssu,benchbot-addons/data_collect
   Their results both ended with "Finished!"

However, I directly input the command "benchbot_run --robot carter_omni --env miniroom:1 --task semantic_slam:active:ground_truth"
and got the ERROR:

**Robot & environment types aren't consistent. Please ensure each of the following
have the same type:

	Robot 'carter_omni' has type 'sim_omni'
	Environment 'miniroom:1' has type 'sim_unreal'

ERROR: Inconsistent types selected (robot = 'miniroom:1', environment = 'miniroom:1')**

Then, I changed the robot as "carter" in order to make sure that Robot & environment types are consistent,
so I input the command "benchbot_run --robot carter --env miniroom:1 --task semantic_slam:active:ground_truth"

**################################################################################
################# CHECKING FOR BENCHBOT SOFTWARE STACK UPDATES #################
################################################################################

fatal: not a git repository (or any of the parent directories): .git
Checking BenchBot version ...				fatal: not a git repository (or any of the parent directories): .git
Outdated.
Checking BenchBot API version ...			Outdated.
Checking BenchBot Eval version ...			Outdated.
Checking BenchBot Supervisor version ...		Outdated.
Checking installed BenchBot add-ons are up-to-date ...	Up-to-date.

WARNING: Please install the latest BenchBot software stack by re-running the 
"benchbot_install" command, or run this with [-f|--force-updateless] to skip
updates.**

Three versions are outdated! I don't know why they would be outdated due to the fact that I succeeded in installing all of them just now(maybe in two days)?
And I also tried to add '-f' to the end of this command, and run it again(input the command "benchbot_run --robot carter --env miniroom:1 --task semantic_slam:active:ground_truth -f"):

**################################################################################
################# CHECKING FOR BENCHBOT SOFTWARE STACK UPDATES #################
################################################################################

Skipping ...

################################################################################
###################### CLEANING UP ALL BENCHBOT REMNANTS #######################
################################################################################


Deleted the following containers:
Total reclaimed space: 0B

Finished cleaning! (use 'benchbot_run -k' for a full clean)


################################################################################
##################### STARTING THE BENCHBOT SOFTWARE STACK #####################
################################################################################

Running the BenchBot system with the following settings:

    Selected task:        semantic_slam:active:ground_truth
    Task results format:  object_map
    Selected robot:       carter
    Selected environment: miniroom:1
    Scene/s:              miniroom:1, starting @ pose [0.7, 0, 0, -0.7, 1.2, 1.5, 0.3]
                          (map_path = '/Game/AI_vol3_03_base/Maps/AI_vol3_scene_03')
    Simulator required:   Yes (sim_unreal)

Creating shared network 'benchbot_network':

Starting persistent container for ROS core:
Skipping (already running)

Starting persistent container for BenchBot Robot Controller (sim_unreal):
**Unable to find image 'benchbot/simulator:sim_unreal' locally**
docker: Error response from daemon: pull access denied for benchbot/simulator, repository does not exist or may require 'docker login': denied: requested access to the resource is denied.
See 'docker run --help'.

################################################################################
###################### CLEANING UP ALL BENCHBOT REMNANTS #######################
################################################################################


Deleted the following containers:
Total reclaimed space: 0B

Finished cleaning! (use 'benchbot_run -k' for a full clean)**

This error is "Unable to find image 'benchbot/simulator:sim_unreal' locally", but I have no idea how to solve it...
For those abnormal problems, could you please give me some valuable suggestions again?

[david2611]

Thanks for your interest in BenchBot :) So this is a bit of a version issue at the moment. In early 2022 we moved from using Isaac Unreal to Isaac Omniverse as our underlying simulator. The data_collect add-on was designed to work with the Unreal simulator and has not yet been transferred to Omniverse (related to #73). We are no longer able to install the Unreal version of the simulator that was used in the past as that is no longer available for download from NVIDIA. I deeply apologize for this.

Is data_collect your main use for BenchBot at this point? If so I unfortunately can't offer assistance until the Omniverse version of the add-on is developed. Otherwise I would say that the rest of the functionality of BenchBot is currently available in the Omniverse version and a fresh install should hopefully help sort out some of the issues.

[ThePassedWind]

Copy that! Thanks for your reply~
I sincerely want to know "the rest of the functionality of BenchBot" you mentioned above, for the reason that I cannot successfully execute any command which starts with "benchbot_run", and the error was recorded in my last passage.
If the "benchbot_run" command is not able to be correctly executed, maybe I cannot do anything about BenchBot?
And could you please tell me when would Omniverse version of the add-on be developed?

PS: what I'd say is that the error of "benchbot_run" isn't likely to be avoided, even though I desire to ignore BenchBot add-ons. And if I cannot successfully execute "benchbot_run", the simulator window would not pop up.

Thanks again~~~

[david2611]

So what should be operating as normal are benchbot_run commands for omniverse environments allowing you to run semantic slam and scene change detection tasks (as should get installed by default). Benchbot is designed around creating tasks and submitting solutions to those tasks.

What I am guessing might have happened is that by installing the data_collect add-on you may have installed it's dependencies (aka the old Unreal environments). In this case, you would have downloaded the files for the environments but due to our new installation process, there would be no simulator actually able to render and use those environments. In which case you might have both unreal and omniverse environments present. As the unreal and omniverse environments have the same names, you might have a case where calling an environment with the benchbot_run command might be calling the unreal one just due to a quirk of ranking. Thus the conflict of version between your robot and your simulator. Everything at present should be run using omniverse simulator environments and robots.

Easy way to check this would be to do do a benchbot_run --list-envs command and see if there are two entries present for each environment (e.g. miniroom:1, miniroom:1, miniroom:2, miniroom:2, ...). If this is the case it should be as simple as running benchbot_install --remove_addons <unreal_env_addons>. You can use benchbot_install --list-addons to check the naming convention of the addon. You could also more hackily do this by deleting the envs_bear_develop_sim_unreal and/or envs_bear_challenge_sim_unreal from your add-ons folder/s within your benchbot repo.

An omniverse version of the data_collect add-on has unfortunately not been a priority as we are preparing to launch this year's robotic vision scene understanding challenge for which the add-on is not required. I can't really give you a definitive date for it unfortunately.

[ThePassedWind]

Emmm, I did the benchbot_run --list-envs command and showed that:

Either simulated or real world environments can be selected. Please see the
'--list-robots' command for the available robot platforms. Only simulated robots 
can be run in simulated environments, and only real robots in real environments 
(as you would expect).

The following environments are supported in your BenchBot installation:
    
	apartment:1
	apartment:2
	apartment:3
	apartment:4
	apartment:5
	company:1
	company:2
	company:3
	company:4
	company:5
	house:1
	house:2
	house:3
	house:4
	house:5
	miniroom:1
	miniroom:2
	miniroom:3
	miniroom:4
	miniroom:5
	office:1
	office:2
	office:3
	office:4
	office:5

See the '--show-environment ENVIRONMENT_NAME' command for specific details about
each environment, or check you have the appropriate add-on installed if you are 
missing an environment.

And I found there is only one entry present for each environment(which means apartment:1-5, ..., office1-5 have the same type). Their types are all "sim_unreal".
Maybe should I install the environment of the type "sim_omni"?
Thus I hope to re-run ./install in order to finish this problem, but there are so many "rm errors" in the second line in PART4:

################################################################################
#################### PART 4: RUNNING POST-BUILD HOST CHECKS ####################
################################################################################


Validating the build against the host system:
	CUDA / NVIDIA versions match:                                    Matches

Validating BenchBot libraries on the host system:
	BenchBot Add-ons Manager cloned:                                      No

  The BenchBot Add-ons Manager Python library is not cloned on the host system.
  Having it installed is required for using add-ons, which contain all of the
  pre-made content for BenchBot (robots, environments, task definitions,
  evaluation methods, etc.).

  The following commands can be run to try & fix the issue:

rm -rf /home/midea/qcr/benchbot-master/addons && 
git clone https://github.com/qcr/benchbot_addons /home/midea/qcr/benchbot-master/addons &&
pushd /home/midea/qcr/benchbot-master/addons && 
git fetch --all && (git checkout -t origin/$BRANCH_DEFAULT || 
git checkout $BRANCH_DEFAULT) && popd

  Would you like the above commands to be run (y/N)? y

rm: cannot remove '/home/midea/qcr/benchbot-master/addons/setup.py': Permission denied
rm: cannot remove '/home/midea/qcr/benchbot-master/addons/benchbot_addons/.local/my_addons/tasks': Permission denied
rm: cannot remove '/home/midea/qcr/benchbot-master/addons/benchbot_addons/.local/my_addons/batches': Permission denied
rm: cannot remove '/home/midea/qcr/benchbot-master/addons/benchbot_addons/.local/my_addons/formats': Permission denied
rm: cannot remove '/home/midea/qcr/benchbot-master/addons/benchbot_addons/.local/my_addons/robots': Permission denied
rm: cannot remove '/home/midea/qcr/benchbot-master/addons/benchbot_addons/.local/my_addons/examples': Permission denied
rm: cannot remove '/home/midea/qcr/benchbot-master/addons/benchbot_addons/.local/my_addons/ground_truths': Permission denied
rm: cannot remove '/home/midea/qcr/benchbot-master/addons/benchbot_addons/.local/my_addons/environments': Permission denied
rm: cannot remove '/home/midea/qcr/benchbot-master/addons/benchbot_addons/.local/my_addons/evaluation_methods': Permission denied
rm: cannot remove '/home/midea/qcr/benchbot-master/addons/benchbot_addons/benchbot-addons/eval_omq/.git/FETCH_HEAD': Permission denied
rm: cannot remove '/home/midea/qcr/benchbot-master/addons/benchbot_addons/benchbot-addons/eval_omq/.git/refs/remotes/origin/HEAD': Permission denied
rm: cannot remove '/home/midea/qcr/benchbot-master/addons/benchbot_addons/benchbot-addons/eval_omq/.git/refs/tags': Permission denied
rm: cannot remove '/home/midea/qcr/benchbot-master/addons/benchbot_addons/benchbot-addons/eval_omq/.git/refs/heads/master': Permission denied
rm: cannot remove '/home/midea/qcr/benchbot-master/addons/benchbot_addons/benchbot-addons/eval_omq/.git/objects/eb/6e0d6f029785f7b6ce47b2a53f63c268b3f401': Permission denied
rm: cannot remove '/home/midea/qcr/benchbot-master/addons/benchbot_addons/benchbot-addons/eval_omq/.git/objects/61/824f9785e93b6da7944a22bca9741dda760e23': Permission denied
rm: cannot remove '/home/midea/qcr/benchbot-master/addons/benchbot_addons/benchbot-addons/eval_omq/.git/objects/b8/8b7070c841e8f9410cad21171d93877080e1cc': Permission denied
rm: cannot remove '/home/midea/qcr/benchbot-master/addons/benchbot_addons/benchbot-addons/eval_omq/.git/objects/94/f65842092954e17963e0f68ec3581204f97d9b': Permission denied
rm: cannot remove '/home/midea/qcr/benchbot-master/addons/benchbot_addons/benchbot-addons/eval_omq/.git/objects/b1/6b236b01afc2d062e7076a271c7251a51d239b': Permission denied

Could you please give me a hand again?
Thanks you!

[david2611]

Hmmm unusual. Perhaps you installed something originally with the sudo command and so created files could not be removed without root permission? That should not be happening under a standard install.

Though not ideal, one option perhaps would be to do a full uninstall and reinstall of the system. You can do this with benchbot_install -f. You can also try manually removing the problematic files while having root (sudo) access.

[ThePassedWind]

OK, now I understand it.
Can I just do the command "benchbot_install --uninstall"
And then re-install it correctly, but do not install BenchBot add-ons.
Can I run "getting started" successfully later?

[david2611]

Yes you can do the uninstall and reinstall manually. the benchbot_install -f should simply do that as a single command. My big thing would be to not install any additional add-ons. I think the core ones that are available should be all you need. The installation of the data_collect add-on should be the only thing that caused headaches for you in the past.
Once your system has been installed fresh you should be able to do the steps in the "getting started" documentation of the main page. If not just let us know.

[ThePassedWind]

Okk, Thanks for your help!!!

[ThePassedWind]

Unluckily, I have uninstalled and then reinstalled manually, but something is wrong:
Do the command benchbot_run --robot carter_omni --env miniroom:1 --task semantic_slam:active:ground_truth

Environment 'miniroom:1' is not a supported environment. Please check '--list-environments'.

ERROR: Invalid environment selected (environment = 'miniroom:1')

And I found there is not environment there, do the command benchbot_run --list-environments:

Either simulated or real world environments can be selected. Please see the
'--list-robots' command for the available robot platforms. Only simulated robots 
can be run in simulated environments, and only real robots in real environments 
(as you would expect).

The following environments are supported in your BenchBot installation:
    
	NONE!

See the '--show-environment ENVIRONMENT_NAME' command for specific details about
each environment, or check you have the appropriate add-on installed if you are 
missing an environment.

Facing this situation, could you please tell me how can I do?
PS: I have executed ./install twice to make sure the command would be executed correctly.

[david2611]

Can I confirm that your benchbot add-ons folder has the development environments present within it?
e.g. /path/to/benchbot/addons/benchbot_addons/benchbot-addons/envs_bear_develop_sim_omni
This folder should then contain an environments folder with several environment.yaml files contained therein.

[ThePassedWind]

Ohh, there is nothing but "LICENSE.txt README.md"
But my installation seems to be right, as same as before.
Could you please share your own email? Due to the fact that I really desire to solve my problems as soon as possible~

[david2611]

Hmmm that is very unusual to me. Are you able to uninstall and re-install the add-on itself? or remove the folder and attempt an install again and print the full log output? The entire install process for environment add-ons is that we take download the files found in the .remote file which should also be in that main environment folder (envs_bear_develop_sim_omni) and extract the zip folder located at the URL provided to an environments folder (which your install seems to lack). If all else fails that is a manual way you could go about "installing" the environment add-ons.

I shan't be sharing an e-mail here but if you wish another line of communication then you can get in contact via our robotic vision challenge slack workspace where you can find and contact me

[david2611]

Pinging this issue again. If not commented on in 3 days issue shall be closed due to inactivity

[david2611]

Closed due to inactivity