[Cloudflare Calls] Undocumented API endpoint request fields for creating tracks

[oxcabe]

I'm opening this issue here since it's where the code with the relevant fields are. Consider moving it to a more appropiate repository if necessary.

The following code creates a track to communicate with the OpenAI Realtime models.
Notice the params used in the NewTracks method, especially those with comments on top of them (i.e. bidirectionalMediaStream and kind):

orange/app/durableObjects/ChatRoom.server.ts

Lines 394 to 406 in 61608e1

	const openAiTracksResponse = await openAiSession.NewTracks({
	// No offer is provided so Calls will generate one for us
	tracks: [
	{
	location: 'local',
	trackName: 'ai-generated-voice',
	// Let it know a sendrecv transceiver is wanted to receive this track instead of a recvonly one
	bidirectionalMediaStream: true,
	// Needed to create an appropriate response
	kind: 'audio',
	},
	],
	})

NewTracks is defined here. The method sends the body argument directly as the request body:

orange/app/utils/openai.server.ts

Lines 44 to 57 in 61608e1

	async NewTracks(body: any): Promise<NewTracksResponse> {
	const newTracksURL = new URL(
	`${this.endpoint}/sessions/${this.sessionId}/tracks/new?${this.params.toString()}`
	)
	const newTracksResponse = (await fetch(newTracksURL.href, {
	method: 'POST',
	headers: this.headers,
	body: JSON.stringify(body),
	}).then(async (res) => {
	console.log(await res.clone().text())
	return res.json()
	})) as NewTracksResponse
	return newTracksResponse
	}

Developers find the Cloudflare Calls API reference here, which is where the OpenAPI schema for the endpoint used in NewTracks can be found.

We can find that the schema for this endpoint is defined as:

TracksRequest:
      type: object
      properties:
        sessionDescription:
          $ref: "#/components/schemas/SessionDescription"
        tracks:
          type: array
          items:
            $ref: "#/components/schemas/TrackObject"

The tracks array is composed of TrackObject elements, which looks like:

TrackObject:
      type: object
      properties:
        location:
          type: string
          enum:
            - local
            - remote
          description: If you want to share a track, it should be local. If you want to play a track shared by a remote agent, it should be remote
        mid:
          type: string
          description: mid associated to track's transceiver. It should be set with local tracks only
        sessionId:
          type: string
          description: Session ID of the track owner. It should be set for remote tracks only
        trackName:
          type: string
          description: Given name for the track

The TrackObject schema doesn't reference neither bidirectionalMediaStream nor kind fields in it. If these parameters do actually change the behavior of the API call (I haven't tested this), it might happen that they're undocumented.

If that's the case, could we have an update for the schema?