ai-proxy nil error with bedrock misconfiguration

[t-yuki]

Is there an existing issue for this?

• I have searched the existing issues

Kong version ($ kong version)

Kong 3.8.0.0

Current Behavior

nil error happen when aws_region is missing.
[kong] init.lua:427 [ai-proxy] /usr/local/share/lua/5.1/kong/llm/proxy/handler.lua:471: attempt to call field 'set_response_transformer_skipped' (a nil value)

Expected Behavior

Error detail on AWS auth config such as error authenticating with cloud-provider, AWS region not specified anywhere

Steps To Reproduce

1. in docker environment

2. with config of a route like plugins:
   - name: ai-proxy
   config:
   route_type: llm/v1/chat
   auth:
   aws_access_key_id: MYKEY
   aws_secret_access_key: MYSECRET
   model:
   provider: bedrock
   name: anthropic.claude-3-haiku-20240307-v1:0
   #options: # NO AWS_REGION CONFIG
   # bedrock: {"aws_region": "ap-northeast-1"}

3. call the route

Anything else?

I think

kong/kong/llm/proxy/handler.lua

Line 461 in c983384

llm_state.set_response_transformer_skipped()

should use disable_ai_proxy_response_transform

[chobits]

hi @t-yuki thanks for report, it seems a function typo here. I think your advice might be right.

• not find the function definition after searching the source code:

xc kong $ grep 'set_response_transformer_skipped' -R kong
kong/llm/proxy/handler.lua:    llm_state.set_response_transformer_skipped()

Could you open a pull request for this? We warmly welcome community users to submit PRs to us, and we are also happy to help with reviews.