Improvement: [CI-self] Add jobs to verify pylint features

[130s]

Issue

There are a few pylint features in ICI but they are not set to be verified on CI of this repo so it's unclear whether it's working.

Approach to solution

Add pipeline on this repo's CI.

[130s]

Not sure why this job fails.

• pylint is only receiving ./industrial_ci/python/industrial_ci/travis.py but I see other .py files in the repo are passed.
• Also I chose errors-only option but

https://github.com/ros-industrial/industrial_ci/actions/runs/3832848710/jobs/6523638429#step:3:431

run_pylint

$ ( source /root/target_ws/install/setup.bash && cd /root/target_ws && pylint /root/target_ws/src/industrial_ci/industrial_ci/mockups/testpkg_broken_install/script/sample_talker.py /root/target_ws/src/industrial_ci/industrial_ci/setup.py /root/target_ws/src/industrial_ci/industrial_ci/python/industrial_ci/__init__.py /root/target_ws/src/industrial_ci/industrial_ci/python/industrial_ci/travis.py; )

Locally the same file doesn't fail `pylint`.

$ git log -1
commit 2337c12d25a227bc9f2701a15b1d6b684d081896 (HEAD -> feature-pylint-unittest, plusone-robotics/feature-pylint-unittest)
Author: Isaac I.Y. Saito <[email protected]>
Date:   Tue Jan 3 15:40:44 2023 -0500

    Improvement: [CI-self] Add jobs to verify pylint features
$ pylint --errors-only ./industrial_ci/python/industrial_ci/travis.py
$
$ pip3 --version pylint
pip 20.0.2 from /usr/lib/python3/dist-packages/pip (python 3.8)

[130s]

Hm, ok, specifying a module in PYLINT_ARGS seems to be out of scope, as I found ICI tries to find all .py in a repo. Will update the PR with a new tactics.

industrial_ci/industrial_ci/src/tests/source_tests.sh

Line 138 in 6a8f546

local find_pattern=(-type f -iname '*.py')

[130s]

I'm trying to exclude `sample_talker.py`, but I see `pylint` still looks into it.

Exit code 30: 0001 1110 (according to stackoverflow.com#71315164), error + warning + refactor + convention. So which modules are emitting the errors?

Lines that contains ": E" (e.g. : E0401) are only found in Module sample_talker section. So sample_talker.py

TRACE:tests/source_tests.sh:146 ici_find_nonhidden /root/target_ws/src -type f -iname *.py -not -path *./industrial_ci/mockups/testpkg_broken_install/script/sample_talker.py*

Ah, I found passing a path of the file to be excluded to PYLINT_EXCLUDE doesn't seem to exclude. Just giving a pattern of the file name seems to work (added an updated to the doc in this PR).

Yet I still see the job fails with pylint's code 28 (0001 1100) meaning warning, refactor, and convention. No more error. Now the question is how to make this pass, when I don't care those non-error output. Maybe I should try the config file (although I reported an issue about using a conf #803).

[130s]

I think what I'm stuck at might be a limitation of current ICI, so I ticketed it #805 for the confirmation, and further discussion.

[130s]

There are still 2 jobs that failed but don't seem to be relevant (not sure what I should do for this PR). So I removed WIP status. Please review @mathias-luedtke

https://github.com/ros-industrial/industrial_ci/actions/runs/3847511406/jobs/6554073265#step:3:1528

  Crawling for packages in workspace '/tmp/ws/src'
  Found the following packages:
    - control_msgs
    - industrial_ci
  Package maintainer emails: [email protected] [email protected] [email protected] [email protected]
  Traceback (most recent call last):
    File "/tmp/ros_buildfarm/ros_buildfarm/scripts/devel/create_devel_task_generator.py", line 287, in resolve_names
      resolved_names = resolve_for_os(
    File "/usr/lib/python3/dist-packages/rosdep2/catkin_support.py", line 87, in resolve_for_os
      d = view.lookup(rosdep_key)
    File "/usr/lib/python3/dist-packages/rosdep2/lookup.py", line 206, in lookup
      return self.rosdep_defs[rosdep_name]
  KeyError: 'ros_workspace'
  
  During handling of the above exception, another exception occurred:
  
  Traceback (most recent call last):
    File "/tmp/ros_buildfarm/scripts/devel/create_devel_task_generator.py", line 21, in <module>
      run_module('ros_buildfarm.scripts.devel.create_devel_task_generator', run_name='__main__')
    File "/usr/lib/python3.8/runpy.py", line 210, in run_module
      return _run_code(code, {}, init_globals, run_name, mod_spec)
    File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
      exec(code, run_globals)
    File "/tmp/ros_buildfarm/ros_buildfarm/scripts/devel/create_devel_task_generator.py", line 304, in <module>
      sys.exit(main())
    File "/tmp/ros_buildfarm/ros_buildfarm/scripts/devel/create_devel_task_generator.py", line 126, in main
      debian_pkg_names += resolve_names(['ros_workspace'], **context)
    File "/tmp/ros_buildfarm/ros_buildfarm/scripts/devel/create_devel_task_generator.py", line 290, in resolve_names
      raise RuntimeError(
  RuntimeError: Could not resolve the rosdep key 'ros_workspace'
  'sudo -EH -u ci sh -c cd '/tmp/tmp.ijGRuYvzye' && exec ./prerelease.sh -y' returned with 1
'run_prerelease_script' returned with code '1' after 1 min 40 sec

https://github.com/ros-industrial/industrial_ci/actions/runs/3847511406/jobs/6554062024#step:4:1595

  Add distro "rolling"
  updated cache in /home/buildfarm/.ros/rosdep/sources.cache
  Crawling for packages in workspace '/tmp/ws/src'
  Found the following packages:
    - control_msgs
  Package maintainer emails: [email protected]
  Traceback (most recent call last):
    File "/tmp/ros_buildfarm/ros_buildfarm/scripts/devel/create_devel_task_generator.py", line 287, in resolve_names
      resolved_names = resolve_for_os(
    File "/usr/lib/python3/dist-packages/rosdep2/catkin_support.py", line 87, in resolve_for_os
      d = view.lookup(rosdep_key)
    File "/usr/lib/python3/dist-packages/rosdep2/lookup.py", line 206, in lookup
      return self.rosdep_defs[rosdep_name]
  KeyError: 'ros_workspace'
  
  During handling of the above exception, another exception occurred:
  
  Traceback (most recent call last):
    File "/tmp/ros_buildfarm/scripts/devel/create_devel_task_generator.py", line 21, in <module>
      run_module('ros_buildfarm.scripts.devel.create_devel_task_generator', run_name='__main__')
    File "/usr/lib/python3.8/runpy.py", line 210, in run_module
      return _run_code(code, {}, init_globals, run_name, mod_spec)
    File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
      exec(code, run_globals)
    File "/tmp/ros_buildfarm/ros_buildfarm/scripts/devel/create_devel_task_generator.py", line 304, in <module>
      sys.exit(main())
    File "/tmp/ros_buildfarm/ros_buildfarm/scripts/devel/create_devel_task_generator.py", line 126, in main
      debian_pkg_names += resolve_names(['ros_workspace'], **context)
    File "/tmp/ros_buildfarm/ros_buildfarm/scripts/devel/create_devel_task_generator.py", line 290, in resolve_names
      raise RuntimeError(
  RuntimeError: Could not resolve the rosdep key 'ros_workspace'
  'sudo -EH -u ci sh -c . /opt/ros/noetic/setup.sh && cd '/tmp/tmp.KdtTOtpk2P' && exec ./prerelease.sh -y' returned with 1
'run_prerelease_script' returned with code '1' after 1 min 42 sec