Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

simulation tags are not converted to int/float as appropriate #1658

Open
ckirkman-IDM opened this issue Nov 5, 2021 · 4 comments
Open

simulation tags are not converted to int/float as appropriate #1658

ckirkman-IDM opened this issue Nov 5, 2021 · 4 comments
Assignees
Labels
Analyzers bug Something isn't working Core Discuss Discuss with group

Comments

@ckirkman-IDM
Copy link
Contributor

ckirkman-IDM commented Nov 5, 2021

dtk-tools used to do this, and is critical for all calibration analyzers, otherwise their sample index values are string-sorted (0, 1, 10, 11, 12, 2, 3, 4, 5, 6, 7, 8, 9) instead of int sorted (0, 1, 2, 3, 4, 5, ...). For calibration, this could lead to inappropriate matching between analyzer scores and the parameter sample it is supposed to be linked to.

I have diagnosed this as a non-feature parity issue between idmtools <-> dtk-tools that does not depend on pyComps or pandas.

We will need to decide if numerical conversion of tag values is up to the user (in analyzers, etc) or is reliably done by idmtools, or have calibra convert just-this-tag, as every ported dtk-tools -> idmtools calibration analyzer will be exposed to this error.

Discussion and details from teams:
MicrosoftTeams-image (1)

@ckirkman-IDM ckirkman-IDM added bug Something isn't working and removed COMPS labels Nov 5, 2021
@ckirkman-IDM
Copy link
Contributor Author

@issuelabeler issuelabeler bot added COMPS Discuss Discuss with group labels Nov 5, 2021
@ckirkman-IDM ckirkman-IDM removed the COMPS label Nov 5, 2021
@ckirkman-IDM
Copy link
Contributor Author

Is there a way to make the issuelabeler bot less ... AGGRESSIVE in adding labels over and over again?

@shchen-idmod
Copy link
Collaborator

To give the more history context, we used to fix Prashanth and Jillian's calibration with 2 lines of code at right before return result in analyzer's reduce function to first convert result index(which is sample_index tag) to int then sort by index. But this solution basically needs each user to remember this 2 lines of code. other wise, calibration result will be screwed up.
If we can change in idmtools or some where not ask user to do it, it would be idea solution.

@devclinton
Copy link
Member

I think maybe we could add a function the the IEntity(has the tags object on it) to make it easier for users to easily convert the columns in tags to specific datatypes. My first instinct is to imitate libraries that users are familiar with, so looking at Dataframe constructor as inspiration(see https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.html), we could add something like

def convert_tag_types(dtypes):
       .......
       
# and example
a = Simulation()
a.convert_tag_types([("seed", int), ("s", float)])
# or
a.convert_tag_types({"seed": int, "s": float})

We could then enable automatic conversion with a new option in the iplatform later possible on fetch as a post load option?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Analyzers bug Something isn't working Core Discuss Discuss with group
Projects
None yet
Development

No branches or pull requests

4 participants