Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

unittest generator and scheduler based on FMF metadata #161

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

jscotka
Copy link
Collaborator

@jscotka jscotka commented Jul 17, 2018

unittest dynamic class generator

you can use this class file as dynamic generator for another scheduler like nosetest

$ TARGET=tests/data/Dockerfile nosetests unittest-run.py 
...
----------------------------------------------------------------------
Ran 3 tests in 0.003s

OK

scheduler possibilities

./unittest-run.py --help
Usage: unittest-run.py [OPTIONS] TARGET

Options:
  -c, --checks-paths DIRECTORY  Path to directory containing checks (default
                                ['/home/jscotka/git/colin/colin/checks']).
  -r, --ruleset-dir PATH        Path to a directory with rulesets
  -n, --name TEXT               Select cases by key names
  -f, --filter TEXT             Filter cases based on FMF filter rules
  -v, --verbose INTEGER         change verbosity of unittest scheduler
  --debug                       Enable debugging mode (debugging logs, full
                                tracebacks).
  --help                        Show this message and exit.

usage like:

 $ ./unittest-run.py -v 2 tests/data/Dockerfile 
test (__main__.from_tag_not_latest) ... ok
test (__main__.maintainer_deprecated) ... ok
test (__main__.maintainer_label) ... ok

----------------------------------------------------------------------
Ran 3 tests in 0.002s

OK

@jscotka jscotka mentioned this pull request Jul 17, 2018
4 tasks
"""
reference_nodes = self.prune(whole=whole, names=["@"])
for node in reference_nodes:
node.data = node.original_data
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

node.data is not used in function.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is intentional, this allows us to modify FMF to allow using rulesets with full power of FMF. in this case + sign for nodes.

@phracek
Copy link
Member

phracek commented Jul 18, 2018

Basically, FMF removes python code into fmf (YAML) format. I don't see any advantage for colin alone.
Colin functionality, backwards compatibility has to remain. Do not remove python files at all.

I would like to see the other projects which will use FMF. I see only colin uses FMF, like PoC for using FMF, what about to wait till the other project will use/implement it.

Copy link
Member

@phracek phracek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please fix my issues.

logger.debug("get FMF metadata for test (path:%s name=%s)", fmfpath, name)
items = [x for x in fmf_tree.climb(
) if name in x.name and "@" not in x.name]
if object_list:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if bool is set return all items? Why, I don't understand this part of code.

if len(items) == 1:
output = items[0]
elif len(items) > 1:
raise Exception("There is more FMF test metadata for item by name:{}({}) {}".format(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

naming would be nice like n=name, l=len(items).

@classmethod
def get_metadata(cls, name, path=None):
"""
Very important method what returns tuple for object initialization
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Get rid of documentation like 'Very important method'. It is enough to say. 'Method returns ..'

fmf_scheduler.py Outdated
return test


def make_base_fmf_class_abstract(node, target, base_class=DynamicClassBase, fmf_class=FMFAbstractCheck):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

docstring is missing.

fmf_scheduler.py Outdated
return outclass


def make_wrapped_fmf_class(node, target):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function is a bit complicated. What about to add an example into docstring for better understanding.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll add, good idea. whole idea of this class is to create wrapper class like (in short):

class testClass(unittest):
    backendclass = abstract_check_inherited_class
    def test():
           return backendclass().check(TARGET)

is this sufficient comment in doc string

fmf_scheduler.py Outdated
return test_classes


def scheduler_opts(target_name=None, checks=None, ruleset_path=None,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

colin functionality has to remain as it is. scheduler_opts has to be valid only for FMF.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure, thats why it is directly in FMF scheduler and not inside colin, but finally we, remove colin and use this and make it backward complatible, so finally there will be no changes for current colin users

@phracek
Copy link
Member

phracek commented Jul 18, 2018

I am afraid of using FMF in colin.
Once we check Dockerfile, image etc. we have rulesets and we want to check all things.

@user-cont user-cont deleted a comment Jul 25, 2018
@user-cont user-cont deleted a comment Jul 25, 2018
@user-cont user-cont deleted a comment Jul 25, 2018
@user-cont user-cont deleted a comment Jul 25, 2018
@user-cont user-cont deleted a comment Jul 25, 2018
@user-cont user-cont deleted a comment Jul 25, 2018
@user-cont user-cont deleted a comment Jul 25, 2018
@user-cont user-cont deleted a comment Jul 25, 2018
@user-cont user-cont deleted a comment Jul 25, 2018
@user-cont user-cont deleted a comment Jul 25, 2018
@user-cont user-cont deleted a comment Jul 25, 2018
@user-cont user-cont deleted a comment Jul 25, 2018
@user-cont user-cont deleted a comment Jul 25, 2018
@user-cont user-cont deleted a comment Jul 25, 2018
@user-cont user-cont deleted a comment Jul 25, 2018
@user-cont user-cont deleted a comment Jul 25, 2018
@user-cont user-cont deleted a comment Jul 25, 2018
@user-cont user-cont deleted a comment Jul 25, 2018
@user-cont user-cont deleted a comment Jul 25, 2018
@user-cont user-cont deleted a comment Jul 25, 2018
@user-cont user-cont deleted a comment Jul 25, 2018
@user-cont user-cont deleted a comment Jul 25, 2018
@user-cont user-cont deleted a comment Jul 25, 2018
@user-cont user-cont deleted a comment Jul 25, 2018
@user-cont user-cont deleted a comment Aug 14, 2018
@user-cont user-cont deleted a comment Aug 14, 2018
@user-cont user-cont deleted a comment Aug 14, 2018
@user-cont user-cont deleted a comment Aug 14, 2018
@user-cont user-cont deleted a comment Aug 14, 2018
@user-cont user-cont deleted a comment Aug 14, 2018
@user-cont user-cont deleted a comment Aug 14, 2018
@user-cont user-cont deleted a comment Aug 14, 2018
@user-cont user-cont deleted a comment Aug 14, 2018
@user-cont user-cont deleted a comment Aug 14, 2018
@user-cont user-cont deleted a comment Aug 14, 2018
@user-cont user-cont deleted a comment Aug 14, 2018
@user-cont user-cont deleted a comment Aug 14, 2018
@user-cont user-cont deleted a comment Aug 14, 2018
@user-cont user-cont deleted a comment Aug 14, 2018
@user-cont user-cont deleted a comment Aug 14, 2018
@user-cont user-cont deleted a comment Aug 14, 2018
@user-cont user-cont deleted a comment Aug 14, 2018
@user-cont user-cont deleted a comment Aug 14, 2018
@user-cont user-cont deleted a comment Aug 14, 2018
@user-cont user-cont deleted a comment Aug 14, 2018
@user-cont user-cont deleted a comment Aug 14, 2018
@user-cont user-cont deleted a comment Aug 14, 2018
@user-cont user-cont deleted a comment Aug 14, 2018
@user-cont user-cont deleted a comment Aug 14, 2018
@user-cont user-cont deleted a comment Aug 14, 2018
@user-cont user-cont deleted a comment Aug 14, 2018
@user-cont user-cont deleted a comment Aug 14, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants