Get number of dims at EP GetCapability() time

[eralmual]

Hi all!

We are trying to implement new features into our EP but for them we need to know either the tensor dimensions or number of dimensions when the GetCapability method gets called.
We understand that knowing the specific shapes of a dynamic model at that time is pretty much impossible, but we where wondering if there was a way to at least access the number of dimensions as this information can be found in the ONNX model.

For example, we've seen that dynamic models have input shapes like [batch_size, width, height] in that case we would like to be able to know that num_dims=3.

Currently is there any way to do this?

[edgchen1]

Yes, given a Node, you can check the available shape information in its input NodeArgs (from Node::InputDefs). Here's an example:

onnxruntime/onnxruntime/core/providers/coreml/builders/impl/depthtospace_op_builder.cc

Lines 63 to 73 in d1e8d4a

	const auto& input_defs = node.InputDefs();
	std::vector<int64_t> input_shape;
	if (!GetShape(*input_defs[0], input_shape, logger)) {
	return false;
	}
	const auto input_rank = input_shape.size();
	if (input_rank < 4) {
	LOGS(logger, VERBOSE) << "DepthToSpace does not support input shape of " << input_rank << "d shape.";
	}

onnxruntime/onnxruntime/core/providers/coreml/builders/helper.cc

Lines 22 to 35 in d1e8d4a

	bool GetShape(const NodeArg& node_arg, std::vector<int64_t>& shape, const logging::Logger& logger) {
	const auto* shape_proto = node_arg.Shape();
	if (!shape_proto) {
	LOGS(logger, WARNING) << "NodeArg [" << node_arg.Name() << "] has no shape info";
	return false;
	}
	// We already checked the shape has no dynamic dimension
	for (const auto& dim : shape_proto->dim()) {
	shape.push_back(dim.dim_value());
	}
	return true;
	}

[eralmual]

Is the result of node_arg.Shape() always nullptr when we have dynamic shapes? Is there no way to know the number of dims of a dynamic tensor? It's in this case we are particularly interested, because we can extract the shapes from constant tensors but we also need the number of dims in the dynamic ones to optimize the op