Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve context storage to share data through whole run #396

Merged
Merged
Show file tree
Hide file tree
Changes from 27 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
8671248
Improve storage in context
josemiguel-rodrigueznaranjo Jun 26, 2024
aaf21ec
Fix in changelog
josemiguel-rodrigueznaranjo Jun 26, 2024
2c126b6
add run storage
legnadev Jun 26, 2024
4f93209
Fix in changelog
josemiguel-rodrigueznaranjo Jun 26, 2024
f3b8268
Update CHANGELOG.rst
josemiguel-rodrigueznaranjo Jun 26, 2024
87eefd3
Merge branch 'feat/improve_storage' of github.com:josemiguel-rodrigue…
josemiguel-rodrigueznaranjo Jun 26, 2024
0d7db1d
fix linter
legnadev Jun 27, 2024
87791bd
fix linter
legnadev Jun 27, 2024
8935bc3
Improve _get_initial_value_from_context
josemiguel-rodrigueznaranjo Jun 27, 2024
13e0e14
Merge branch 'feat/improve_storage' of github.com:josemiguel-rodrigue…
josemiguel-rodrigueznaranjo Jun 27, 2024
555822f
Improve _get_initial_value_from_context
josemiguel-rodrigueznaranjo Jun 27, 2024
8f528fd
Rename store_key_in_storage
josemiguel-rodrigueznaranjo Jun 27, 2024
ca23ff6
Improve store_key_in_storage
josemiguel-rodrigueznaranjo Jun 27, 2024
6ed93a7
Added test to check storage capabilities
josemiguel-rodrigueznaranjo Jun 27, 2024
dcfc351
Fix in test
josemiguel-rodrigueznaranjo Jun 27, 2024
90d4fe5
Added test to check storage capabilities
josemiguel-rodrigueznaranjo Jun 27, 2024
a5385a0
Added test to check storage capabilities
josemiguel-rodrigueznaranjo Jun 27, 2024
f407411
Added test to check storage capabilities
josemiguel-rodrigueznaranjo Jun 27, 2024
289c4a5
Improved managing of storages using ChainMap
josemiguel-rodrigueznaranjo Aug 1, 2024
cc0e689
Update version and changelog
josemiguel-rodrigueznaranjo Aug 1, 2024
01b4b7a
Merge branch 'master' into feat/improve_storage
josemiguel-rodrigueznaranjo Aug 1, 2024
34f6c7f
Debugging
josemiguel-rodrigueznaranjo Aug 2, 2024
93ea8a9
Improved managing of storages when dynamic env are not used
josemiguel-rodrigueznaranjo Aug 2, 2024
0adb553
Fix lint warning
josemiguel-rodrigueznaranjo Aug 2, 2024
6a2090e
Fix lint warning
josemiguel-rodrigueznaranjo Aug 2, 2024
bd6fd9e
Fix lint warning
josemiguel-rodrigueznaranjo Aug 2, 2024
49d6d26
Fix lint warning
josemiguel-rodrigueznaranjo Aug 2, 2024
71f8905
Updated Changelog
josemiguel-rodrigueznaranjo Aug 5, 2024
8c9c7b1
Remove dict() from a ChainMap in before_feature
josemiguel-rodrigueznaranjo Sep 10, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,10 @@ v3.1.6

*Release date: In development*

- Added `run_storage` to store information during the whole test execution
- Merge storages (context.storage, context.feature_storage and context.run_storage)
- In steps, be able to store values into desire storage by using [key], [FEATURE:key] and [RUN:key]

v3.1.5
------

Expand Down
3 changes: 2 additions & 1 deletion VERSION
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
3.1.6.dev0
3.1.6.dev1

15 changes: 11 additions & 4 deletions toolium/behave/environment.py
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Redesigned again with Chainmap, this time making sure it works, just to give more information:
ChainMap unifies, or links, the 3 context dictionaries into one

Considering we have a mapchain( Dict1, Dict2, Dict3) where dicts are storage, feature and run:

  • If i get a value with context.storage["key"] the first key found in the 3 dictionaries will be retrieved, accesing them by order from left to right. So if we have a duplicated key in storate and run storage, it will get the storage one. -> To avoid this, the programmer should be responsible of where the key is added, but we handle it if you use the store_in_context function.
  • If i set a value into context.storage, it will be stored in the first dictionary always, so it will always be stored in context.storage so this leaves the context.storage to work as the previous behavior as desired and expected.
  • context.storage and run_storage are linked in before all, so instead of a chainmap, any change you do to storage is stored in run, which is an expected behaviour since the only existant context in before all is the run_storage.

Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
import logging
import os
import re
import collections

from toolium.utils import dataset
from toolium.config_files import ConfigFiles
Expand Down Expand Up @@ -50,6 +51,13 @@ def before_all(context):
context.global_status = {'test_passed': True}
create_and_configure_wrapper(context)

# Dictionary to store information during the whole test execution
context.run_storage = dict()
context.storage = context.run_storage

# Method in context to store values in context.storage, context.feature_storage or context.run_storage from steps
context.store_key_in_storage = dataset.store_key_in_storage

# Behave dynamic environment
context.dyn_env = DynamicEnvironment(logger=context.logger)

Expand All @@ -72,10 +80,9 @@ def before_feature(context, feature):
no_driver = 'no_driver' in feature.tags
start_driver(context, no_driver)

# Dictionary to store information between steps
context.storage = dict()
# Dictionary to store information between features
context.feature_storage = dict()
context.storage = collections.ChainMap(dict(), context.feature_storage, context.run_storage)
pabloge marked this conversation as resolved.
Show resolved Hide resolved

# Behave dynamic environment
context.dyn_env.get_steps_from_feature_description(feature.description)
Expand Down Expand Up @@ -129,8 +136,8 @@ def before_scenario(context, scenario):

context.logger.info("Running new scenario: %s", scenario.name)

# Make sure storage dict are empty in each scenario
context.storage = dict()
# Make sure context storage dict is empty in each scenario and merge with the rest of storages
context.storage = collections.ChainMap(dict(), context.feature_storage, context.run_storage)

# Behave dynamic environment
context.dyn_env.execute_before_scenario_steps(context)
Expand Down
73 changes: 73 additions & 0 deletions toolium/test/utils/test_dataset_map_param_context.py
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,79 @@ class Context(object):
assert expected_st == result_st


def test_a_context_param_storage_and_run_storage():
"""
Verification of a mapped parameter as CONTEXT saved in storage and run storage
"""
class Context(object):
pass
context = Context()
context.attribute = "attribute value"
context.storage = {"storage_key": "storage entry value"}
context.run_storage = {"storage_key": "run storage entry value"}
dataset.behave_context = context

result_st = map_param("[CONTEXT:storage_key]")
expected_st = "storage entry value"
assert expected_st == result_st


def test_store_key_in_feature_storage():
"""
Verification of method store_key_in_storage with a mapped parameter as FEATURE saved in feature storage
"""
class Context(object):
pass
context = Context()
context.attribute = "attribute value"
context.storage = {"storage_key": "storage entry value"}
context.feature_storage = {}
dataset.store_key_in_storage(context, "[FEATURE:storage_key]", "feature storage entry value")
dataset.behave_context = context

result_st = map_param("[CONTEXT:storage_key]")
expected_st = "feature storage entry value"
assert expected_st == result_st


def test_store_key_in_run_storage():
"""
Verification of method store_key_in_storage with a mapped parameter as RUN saved in run storage
"""
class Context(object):
pass
context = Context()
context.attribute = "attribute value"
context.storage = {"storage_key": "storage entry value"}
context.run_storage = {}
context.feature_storage = {}
dataset.store_key_in_storage(context, "[RUN:storage_key]", "run storage entry value")
dataset.behave_context = context

result_st = map_param("[CONTEXT:storage_key]")
expected_st = "run storage entry value"
assert expected_st == result_st


def test_a_context_param_using_store_key_in_storage():
"""
Verification of a mapped parameter as CONTEXT saved in storage and run storage
"""
class Context(object):
pass
context = Context()
context.attribute = "attribute value"
context.feature_storage = {}
context.storage = {"storage_key": "previous storage entry value"}
dataset.store_key_in_storage(context, "[FEATURE:storage_key]", "feature storage entry value")
dataset.store_key_in_storage(context, "[storage_key]", "storage entry value")
dataset.behave_context = context

result_st = map_param("[CONTEXT:storage_key]")
expected_st = "storage entry value"
assert expected_st == result_st


def test_a_context_param_without_storage_and_feature_storage():
"""
Verification of a mapped parameter as CONTEXT when before_feature and before_scenario have not been executed, so
Expand Down
59 changes: 47 additions & 12 deletions toolium/utils/dataset.py
Original file line number Diff line number Diff line change
Expand Up @@ -682,19 +682,24 @@ def _get_initial_value_from_context(initial_key, context):
:param context: behave context
:return: mapped value
"""
context_storage = context.storage if hasattr(context, 'storage') else {}
if hasattr(context, 'feature_storage'):
# context.feature_storage is initialized only when before_feature method is called
context_storage = collections.ChainMap(context.storage, context.feature_storage)
# If dynamic env is not initialized, the storages are initialized if needed

context_storage = getattr(context, 'storage', {})
run_storage = getattr(context, 'run_storage', {})
feature_storage = getattr(context, 'feature_storage', {})

if not isinstance(context_storage, collections.ChainMap):
context_storage = collections.ChainMap(context_storage, run_storage, feature_storage)

if initial_key in context_storage:
value = context_storage[initial_key]
elif hasattr(context, initial_key):
value = getattr(context, initial_key)
else:
msg = f"'{initial_key}' key not found in context"
logger.error(msg)
raise Exception(msg)
return value
return context_storage[initial_key]

if hasattr(context, initial_key):
return getattr(context, initial_key)

msg = f"'{initial_key}' key not found in context"
logger.error(msg)
raise Exception(msg)


def get_message_property(param, language_terms, language_key):
Expand Down Expand Up @@ -813,3 +818,33 @@ def convert_file_to_base64(file_path):
except Exception as e:
raise Exception(f' ERROR - converting the "{file_path}" file to Base64...: {e}')
return file_content


def store_key_in_storage(context, key, value):
"""
Store values in context.storage, context.feature_storage or context.run_storage,
using [key], [FEATURE:key] OR [RUN:key] from steps.
context.storage is also updated with given key,value
By default, values are stored in context.storage.

:param key: key to store the value in proper storage
:param value: value to store in key
:param context: behave context
:return:
"""
clean_key = re.sub(r'[\[\]]', '', key)
if ":" in clean_key:
context_type = clean_key.split(":")[0]
context_key = clean_key.split(":")[1]
acccepted_context_types = ["FEATURE", "RUN"]
assert context_type in acccepted_context_types, (f"Invalid key: {context_key}. "
f"Accepted keys: {acccepted_context_types}")
if context_type == "RUN":
context.run_storage[context_key] = value
elif context_type == "FEATURE":
context.feature_storage[context_key] = value
# If dynamic env is not initialized linked or key exists in context.storage, the value is updated in it
if hasattr(context.storage, context_key) or not isinstance(context.storage, collections.ChainMap):
context.storage[context_key] = value
else:
context.storage[clean_key] = value
pabloge marked this conversation as resolved.
Show resolved Hide resolved
Loading