Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flatten GetResponse/SubscribeResponse #62

Closed
wants to merge 10 commits into from
Closed

Flatten GetResponse/SubscribeResponse #62

wants to merge 10 commits into from

Conversation

remingtonc
Copy link
Contributor

@remingtonc remingtonc commented Jun 17, 2020

Responses to Get and Subscribe can be deeply nested, this will provide a flattened view.

e.g.

DEBUG:root:SubscribeResponse received.
INFO:root:{
    "/Cisco-IOS-XR-ethernet-lldp-oper:lldp/nodes/node[node-name=0/RP0/CPU0]/statistics/aged-out-entries": 0,
    "/Cisco-IOS-XR-ethernet-lldp-oper:lldp/nodes/node[node-name=0/RP0/CPU0]/statistics/bad-packets": 0,
    "/Cisco-IOS-XR-ethernet-lldp-oper:lldp/nodes/node[node-name=0/RP0/CPU0]/statistics/discarded-packets": 0,
    "/Cisco-IOS-XR-ethernet-lldp-oper:lldp/nodes/node[node-name=0/RP0/CPU0]/statistics/discarded-tl-vs": 0,
    "/Cisco-IOS-XR-ethernet-lldp-oper:lldp/nodes/node[node-name=0/RP0/CPU0]/statistics/encapsulation-errors": 0,
    "/Cisco-IOS-XR-ethernet-lldp-oper:lldp/nodes/node[node-name=0/RP0/CPU0]/statistics/out-of-memory-errors": 0,
    "/Cisco-IOS-XR-ethernet-lldp-oper:lldp/nodes/node[node-name=0/RP0/CPU0]/statistics/queue-overflow-errors": 0,
    "/Cisco-IOS-XR-ethernet-lldp-oper:lldp/nodes/node[node-name=0/RP0/CPU0]/statistics/received-packets": 0,
    "/Cisco-IOS-XR-ethernet-lldp-oper:lldp/nodes/node[node-name=0/RP0/CPU0]/statistics/table-overflow-errors": 0,
    "/Cisco-IOS-XR-ethernet-lldp-oper:lldp/nodes/node[node-name=0/RP0/CPU0]/statistics/transmitted-packets": 0,
    "/Cisco-IOS-XR-ethernet-lldp-oper:lldp/nodes/node[node-name=0/RP0/CPU0]/statistics/unrecognized-tl-vs": 0
}

Fixes #60

@remingtonc remingtonc added enhancement New feature or request feature labels Jun 17, 2020
@remingtonc remingtonc self-assigned this Jun 17, 2020
@remingtonc remingtonc marked this pull request as ready for review June 22, 2020 16:36
@remingtonc remingtonc requested review from a team, brh55, skkumaravel and miott and removed request for a team June 22, 2020 16:36
Copy link
Collaborator

@miott miott left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you think of any general way to make this conversion so it is not message type specific? Since you can specify any message structure you want in a proto file and compile it, this could be never-ending. I have now seen 3 gRPC type proto's that Cisco supports and the latest "gNOI" is the perfect example of, basically, endless custom proto's. Seems like "MessageToJson" is exactly that type of general parser. Might be better to dig into that library function and subclass or enhance it.

raise Exception("Unhandled element type!")
key_string = ""
for key, value in keys.items():
key_string += "[{key}={value}]".format(key=key, value=value)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Values to keys may have illegal url characters so you should put them in quotes (I think that is good enough)
"[{key}=\"{value}\"]"

for elem in path.elem:
xpath += "/{}".format(elem.name)
for key in elem.key:
xpath += "[{}={}]".format(key, elem.key[key])
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here:
"[{}=\"{}\"]"

@remingtonc
Copy link
Contributor Author

remingtonc commented Jun 23, 2020

Good callouts @miott. The main point here is purely to flatten the data itself within the GetResponse and SubscribeResponse methods - I thought about including EVERYTHING such as timestamping etc. but it pollutes what I really care about here which is simply the data itself being flattened. I agree there are probably better abstractions - this flatten() function does drop some meta aspects of the *Responses today. I believe in a previous pull request you had some commits including effectively flattened XPaths and values and ops, is this a format of some kind specified? Could potentially adopt if it fits the use cases. e.g. 5e8bafd#diff-95ed23ef189c186c1f56b0492d78e067R85

@miott
Copy link
Collaborator

miott commented Jun 24, 2020

I have some decode functions in the gnmi class I use to talk to your library. After going through that, I then saw gRPC MDT protobufs and was introduced to gNOI protobufs. That is what got me trying to think broader on this subject. I would like to try figuring out what the services are and the expected input/output in a general infrastructure by parsing the protobuf files rather than coming up with a new test tool every time someone creates a new protobuf. Just thinking out loud.

@remingtonc
Copy link
Contributor Author

Moving this functionality to a separate library.

@remingtonc remingtonc closed this Jun 24, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Flatten responses to XPaths
2 participants