Coordinate Orientation

[jramborger78]

Hey guys,

Possibly a silly questions but been peaking through the github and paper but can't find anything on the (0,0) orientation point relative to the image. Searching online it seems to be varying as well. I assumed bottom left corner of the video is the orientation, making the video be represented in the 1st quadrant of a cartesian graph, where increases in Y coords are up and x to the right, but ChatGPT seems to think you guys and computer vision often does form the top left corner, or the 4th quadrant of a cartesian graph, where increases in Y go down with increases in X goes right. Thanks for all the help!

• Jarryd

[talmo]

Hi @jramborger78,

Good question! We assume that the center of the pixel is the integer coordinate, with the edges being a half-pixel off. There's some info on that in the docstring for a relevant function in our data generation pipeline:

sleap/sleap/nn/data/instance_cropping.py

Lines 124 to 157 in 2d24296

	def make_centered_bboxes(
	centroids: tf.Tensor, box_height: int, box_width: int
	) -> tf.Tensor:
	"""Generate bounding boxes centered on a set of centroid coordinates.
	Args:
	centroids: A tensor of shape (n_centroids, 2) and dtype tf.float32, where the
	last axis corresponds to the (x, y) coordinates of each centroid.
	box_height: Scalar integer indicating the height of the bounding boxes.
	box_width: Scalar integer indicating the width of the bounding boxes.
	Returns:
	Tensor of shape (n_centroids, 4) and dtype tf.float32, where the last axis
	corresponds to (y1, x1, y2, x2) coordinates of the bounding boxes in absolute
	image coordinates.
	Notes:
	The bounding box coordinates are calculated such that the centroid coordinates
	map onto the center of the pixel. For example:
	For a single row image of shape (1, 4) with values: `[[a, b, c, d]]`, the x
	coordinates can be visualized in the diagram below:
	_______________________
	| a | b | c | d |
	| | | | | | | | |
	-0.5 | 0.5 | 1.5 | 2.5 | 3.5
	0 1 2 3
	To get a (1, 3) patch centered at c, the centroid would be at (x, y) = (2, 0)
	with box height of 1 and box width of 3, to yield `[[b, c, d]]`.
	For even sized bounding boxes, e.g., to get the center 2 elements, the centroid
	would be at (x, y) = (1.5, 0) with box width of 2, to yield `[[b, c]]`.
	"""

So the center of the of top-left pixel of the image is at (0, 0) while the top-left corner of that same top-left pixel is at (-0.5, -0.5). Here's a diagram:

Note that, as you pointed out, image coordinates are a little different than standard cartesian coordinates in that the origin (0, 0) is close to the top-left, and it increases in the y-coordinate as you move vertically down.

Let us know if you have any questions!

Talmo

[jramborger78]

Oh interesting, i see. So a 800x600 image would have (0,600) in the bottom left corner and the center of the image would be (400,300), and say if there was a node in the top right portion of the image itd be something like (700,200). ``` (0,0)———————————(800,0)   |               |    |               |    |               |    |               |    |               |    |               |  (600,0)——————————(800,600) ``` Sincerely, Jarryd Ramborger (951) 541-3922 On Dec 15, 2023, at 12:11 PM, Talmo Pereira ***@***.***> wrote: Hi @jramborger78, Good question! We assume that the center of the pixel is the integer coordinate, with the edges being a half-pixel off. There's some info on that in the docstring for a relevant function in our data generation pipeline: https://github.com/talmolab/sleap/blob/2d24296c2149402c7ee07601e4d0272fdd308441/sleap/nn/data/instance_cropping.py#L124-L157 So the center of the of top-left pixel of the image is at (0, 0) while the top-left corner of that same top-left pixel is at (-0.5, -0.5). Here's a diagram: image.png (view on web) Note that, as you pointed out, image coordinates are a little different than standard cartesian coordinates in that the origin (0, 0) is close to the top-left, and it increases in the y-coordinate as you move vertically down. Let us know if you have any questions! Talmo —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>

[talmo]

Correct!