[firebase_vertexai]: Multi tool generative AI calls fail with server error "Request contains an invalid argument"

[MrCsabaToth]

Is there an existing issue for this?

• I have searched the existing issues.

Which plugins are affected?

Other

Which platforms are affected?

Android

Description

My Generative AI application contains several functions grouped into multiple tools. Whenever when I try to perform a call on the Gemini 1.5 or 2.0 model (regardless of Flash or Pro) equipped with those tools the call fails with server error "Request contains an invalid argument". When I refactor the Tool list to "flatten" all the functions into one unified mixed tool, the call succeeds.

How it should work:

Tool A:
  FunctionDeclaration A1
  FunctionDeclaration A2
Tool B (completely unrelated to A):
  FunctionDeclaration B1
  FunctionDeclaration B2
...

The only way it works now:

Tool (mixed salad of unrelated functions):
  FunctionDeclaration A1
  FunctionDeclaration A2
  FunctionDeclaration B1
  FunctionDeclaration B2
...

I supply the tools during the model isntantiation:

    final model = FirebaseVertexAI.instance.generativeModel(
      model: 'gemini-1.5-flash',
      systemInstruction: Content.text(systemInstruction),
      [getFLattenedFunctionDeclarations(preferences)],
    );
    final chatSession = model.startChat();
    await chatSession.sendMessage(message)

Reproducing the issue

1. Take a simple use case when you have at last two tools each with at least one function.
2. Instantiate a model with these tools.
3. Instantiate a chat session.
4. Send a user message via the chat session.
   The call only survives the call without the server side error if the tools' functions are flattened into a list.

Firebase Core version

3.8.1

Flutter Version

3.27.0

Relevant Log Output

No response

Flutter dependencies

Expand Flutter dependencies snippet

Dart SDK 3.6.0
Flutter SDK 3.27.0
inspector_gadget 1.0.2+1

dependencies:
- adaptive_dialog 2.3.0 [animations collection dynamic_color flutter intersperse macos_ui meta]
- assorted_layout_widgets 10.0.4 [clock flutter matrix4_transform]
- camera 0.11.0+2 [camera_android_camerax camera_avfoundation camera_platform_interface camera_web flutter flutter_plugin_android_lifecycle]
- cloud_functions 5.2.0 [cloud_functions_platform_interface cloud_functions_web firebase_core firebase_core_platform_interface flutter]
- collection 1.19.1
- country_flags 3.1.0 [collection flutter jovial_svg]
- dart_helper_utils 3.3.0 [intl mime meta]
- daylight 3.1.0 [equatable]
- file_picker 8.1.6 [flutter flutter_web_plugins flutter_plugin_android_lifecycle plugin_platform_interface ffi path win32 cross_file web]
- firebase_app_check 0.3.1+7 [firebase_app_check_platform_interface firebase_app_check_web firebase_core firebase_core_platform_interface flutter]
- firebase_auth 5.3.4 [firebase_auth_platform_interface firebase_auth_web firebase_core firebase_core_platform_interface flutter meta]
- firebase_core 3.8.1 [firebase_core_platform_interface firebase_core_web flutter meta]
- firebase_storage 12.3.7 [firebase_core firebase_core_platform_interface firebase_storage_platform_interface firebase_storage_web flutter]
- firebase_ui_storage 2.1.0 [flutter firebase_storage firebase_ui_localizations firebase_ui_shared path uuid file_selector flutter_blurhash]
- firebase_vertexai 1.0.4 [firebase_app_check firebase_auth firebase_core firebase_core_platform_interface flutter http]
- fl_location 5.0.0 [flutter fl_location_platform_interface fl_location_web]
- flex_color_scheme 8.0.2 [flex_seed_scheme flutter meta]
- flutter 0.0.0 [characters collection material_color_utilities meta vector_math sky_engine]
- flutter_easy_animations 0.0.2 [flutter]
- flutter_layout_grid 2.0.7 [flutter collection quiver]
- flutter_localizations 0.0.0 [flutter intl characters clock collection material_color_utilities meta path vector_math]
- flutter_localized_locales 2.0.5 [flutter intl]
- flutter_tts 4.2.0 [flutter flutter_web_plugins]
- gap 3.0.1 [flutter]
- get_it 8.0.3 [async collection meta]
- heart_rate_flutter 0.0.2 [flutter plugin_platform_interface]
- http 1.2.2 [async http_parser meta web]
- intl 0.19.0 [clock meta path]
- listview_utils_plus 0.0.3 [flutter http]
- media_kit 1.1.11 [meta path image uri_parser collection synchronized uuid http safe_local_storage universal_platform js]
- media_kit_libs_audio 1.0.5 [media_kit_native_event_loop media_kit_libs_android_audio media_kit_libs_ios_audio media_kit_libs_macos_audio media_kit_libs_windows_audio media_kit_libs_linux]
- mime 2.0.0
- number_selector 1.0.7 [flutter]
- objectbox 4.0.3 [collection flat_buffers ffi meta path]
- objectbox_flutter_libs 4.0.3 [flutter objectbox path_provider]
- path 1.9.1
- path_provider 2.1.5 [flutter path_provider_android path_provider_foundation path_provider_linux path_provider_platform_interface path_provider_windows]
- pref 2.8.0 [flutter logging shared_preferences]
- record 5.2.0 [flutter uuid record_platform_interface record_web record_windows record_linux record_android record_darwin]
- smooth_page_indicator 1.2.0+3 [flutter]
- speech_to_text 7.0.0 [flutter speech_to_text_platform_interface json_annotation clock pedantic flutter_web_plugins meta web]
- statemachine 3.3.1 [meta]
- strings 3.1.2 [characters unicode]
- translator 1.0.3+1 [http]
- tuple 2.0.2
- video_player 2.9.2 [flutter html video_player_android video_player_avfoundation video_player_platform_interface video_player_web]
- watch_it 1.6.1 [get_it flutter functional_listener]

dev dependencies:
- build_runner 2.4.13 [analyzer args async build build_config build_daemon build_resolvers build_runner_core code_builder collection crypto dart_style frontend_server_client glob graphs http_multi_server io js logging meta mime package_config path pool pub_semver pubspec_parse shelf shelf_web_socket stack_trace stream_transform timing watcher web_socket_channel yaml]
- flutter_test 0.0.0 [flutter test_api matcher path fake_async clock stack_trace vector_math leak_tracker_flutter_testing async boolean_selector characters collection leak_tracker leak_tracker_testing material_color_utilities meta source_span stream_channel string_scanner term_glyph vm_service]
- mocktail 1.0.4 [collection matcher test_api]
- objectbox_generator 4.0.3 [objectbox analyzer build collection dart_style glob path source_gen pubspec_parse yaml http pointycastle]
- very_good_analysis 6.0.0

dependency overrides:
- collection 1.19.1
- mime 2.0.0
- path 1.9.1

transitive dependencies:
- _fe_analyzer_shared 76.0.0 [meta]
- _flutterfire_internals 1.3.47 [collection firebase_core firebase_core_platform_interface flutter meta]
- _macros 0.3.3
- analyzer 6.11.0 [_fe_analyzer_shared collection convert crypto glob macros meta package_config path pub_semver source_span watcher yaml]
- animations 2.0.11 [flutter]
- appkit_ui_element_colors 1.0.0 [equatable flutter plugin_platform_interface]
- archive 4.0.1 [crypto path posix]
- args 2.6.0
- async 2.11.0 [collection meta]
- boolean_selector 2.1.1 [source_span string_scanner]
- build 2.4.1 [analyzer async convert crypto glob logging meta package_config path]
- build_config 1.1.1 [checked_yaml json_annotation path pubspec_parse yaml]
- build_daemon 4.0.2 [built_collection built_value crypto http_multi_server logging path pool shelf shelf_web_socket stream_transform watcher web_socket_channel]
- build_resolvers 2.4.2 [analyzer async build collection convert crypto graphs logging package_config path pool pub_semver stream_transform yaml]
- build_runner_core 7.3.2 [async build build_config build_resolvers collection convert crypto glob graphs json_annotation logging meta package_config path pool timing watcher yaml]
- built_collection 5.1.1
- built_value 8.9.2 [built_collection collection fixnum meta]
- camera_android_camerax 0.6.10 [async camera_platform_interface flutter meta stream_transform]
- camera_avfoundation 0.9.17+5 [camera_platform_interface flutter stream_transform]
- camera_platform_interface 2.8.0 [cross_file flutter plugin_platform_interface stream_transform]
- camera_web 0.3.5 [camera_platform_interface flutter flutter_web_plugins stream_transform web]
- characters 1.3.0
- checked_yaml 2.0.3 [json_annotation source_span yaml]
- clock 1.1.1
- cloud_functions_platform_interface 5.5.40 [firebase_core flutter meta plugin_platform_interface]
- cloud_functions_web 4.10.5 [cloud_functions_platform_interface firebase_core firebase_core_web flutter flutter_web_plugins]
- code_builder 4.10.1 [built_collection built_value collection matcher meta]
- convert 3.1.2 [typed_data]
- cross_file 0.3.4+2 [meta web]
- crypto 3.0.6 [typed_data]
- csslib 1.0.2 [source_span]
- dart_style 2.3.7 [analyzer args collection package_config path pub_semver source_span]
- dynamic_color 1.7.0 [flutter flutter_test material_color_utilities]
- equatable 2.0.7 [collection meta]
- fake_async 1.3.1 [clock collection]
- ffi 2.1.3
- file 7.0.1 [meta path]
- file_selector 1.0.3 [file_selector_android file_selector_ios file_selector_linux file_selector_macos file_selector_platform_interface file_selector_web file_selector_windows flutter]
- file_selector_android 0.5.1+12 [file_selector_platform_interface flutter plugin_platform_interface]
- file_selector_ios 0.5.3+1 [file_selector_platform_interface flutter]
- file_selector_linux 0.9.3+2 [cross_file file_selector_platform_interface flutter]
- file_selector_macos 0.9.4+2 [cross_file file_selector_platform_interface flutter]
- file_selector_platform_interface 2.6.2 [cross_file flutter http plugin_platform_interface]
- file_selector_web 0.9.4+2 [file_selector_platform_interface flutter flutter_web_plugins web]
- file_selector_windows 0.9.3+3 [cross_file file_selector_platform_interface flutter]
- firebase_app_check_platform_interface 0.1.0+41 [_flutterfire_internals firebase_core flutter meta plugin_platform_interface]
- firebase_app_check_web 0.2.0+3 [_flutterfire_internals firebase_app_check_platform_interface firebase_core firebase_core_web flutter flutter_web_plugins web]
- firebase_auth_platform_interface 7.4.10 [_flutterfire_internals collection firebase_core flutter meta plugin_platform_interface]
- firebase_auth_web 5.13.5 [firebase_auth_platform_interface firebase_core firebase_core_web flutter flutter_web_plugins http_parser meta web]
- firebase_core_platform_interface 5.3.1 [collection flutter flutter_test meta plugin_platform_interface]
- firebase_core_web 2.18.2 [firebase_core_platform_interface flutter flutter_web_plugins meta web]
- firebase_storage_platform_interface 5.1.34 [_flutterfire_internals collection firebase_core flutter meta plugin_platform_interface]
- firebase_storage_web 3.10.6 [_flutterfire_internals async firebase_core firebase_core_web firebase_storage_platform_interface flutter flutter_web_plugins http meta web]
- firebase_ui_localizations 1.13.0 [flutter flutter_localizations path]
- firebase_ui_shared 1.4.1 [flutter]
- fixnum 1.1.1
- fl_location_platform_interface 5.1.0 [flutter meta vector_math plugin_platform_interface]
- fl_location_web 5.0.0 [flutter flutter_web_plugins fl_location_platform_interface web]
- flat_buffers 23.5.26
- flex_seed_scheme 3.4.1 [collection flutter meta]
- flutter_blurhash 0.8.2 [flutter]
- flutter_plugin_android_lifecycle 2.0.23 [flutter]
- flutter_web_plugins 0.0.0 [flutter characters collection material_color_utilities meta vector_math]
- frontend_server_client 4.0.0 [async path]
- functional_listener 4.1.0 [flutter]
- glob 2.1.2 [async collection file path string_scanner]
- gradient_borders 1.0.1 [flutter]
- graphs 2.3.2 [collection]
- html 0.15.5 [csslib source_span]
- http_multi_server 3.2.1 [async]
- http_parser 4.1.1 [collection source_span string_scanner typed_data]
- image 4.4.0 [archive meta xml]
- intersperse 2.0.0
- io 1.0.5 [meta path string_scanner]
- jovial_misc 0.9.0 [async collection convert meta]
- jovial_svg 1.1.24 [xml collection args vector_math jovial_misc http meta flutter]
- js 0.6.7 [meta]
- json_annotation 4.9.0 [meta]
- leak_tracker 10.0.7 [clock collection meta path vm_service]
- leak_tracker_flutter_testing 3.0.8 [flutter leak_tracker leak_tracker_testing matcher meta]
- leak_tracker_testing 3.0.1 [leak_tracker matcher meta]
- lists 1.0.1 [meta]
- logging 1.3.0
- macos_ui 2.0.9 [flutter macos_window_utils gradient_borders appkit_ui_element_colors equatable]
- macos_window_utils 1.6.1 [flutter]
- macros 0.1.3-main.0 [_macros]
- matcher 0.12.16+1 [async meta stack_trace term_glyph test_api]
- material_color_utilities 0.11.1 [collection]
- matrix4_transform 4.0.1 [flutter]
- media_kit_libs_android_audio 1.3.6 [flutter plugin_platform_interface]
- media_kit_libs_ios_audio 1.1.4 [flutter plugin_platform_interface]
- media_kit_libs_linux 1.1.3 [flutter]
- media_kit_libs_macos_audio 1.1.4 [flutter plugin_platform_interface]
- media_kit_libs_windows_audio 1.0.9 [flutter]
- media_kit_native_event_loop 1.0.9 [flutter]
- meta 1.15.0
- package_config 2.1.1 [path]
- path_provider_android 2.2.15 [flutter path_provider_platform_interface]
- path_provider_foundation 2.4.1 [flutter path_provider_platform_interface]
- path_provider_linux 2.2.1 [ffi flutter path path_provider_platform_interface xdg_directories]
- path_provider_platform_interface 2.1.2 [flutter platform plugin_platform_interface]
- path_provider_windows 2.3.0 [ffi flutter path path_provider_platform_interface]
- pedantic 1.11.1
- petitparser 6.0.2 [meta]
- platform 3.1.6
- plugin_platform_interface 2.1.8 [meta]
- pointycastle 3.9.1 [collection convert js]
- pool 1.5.1 [async stack_trace]
- posix 6.0.1 [ffi meta path]
- pub_semver 2.1.5 [collection meta]
- pubspec_parse 1.3.0 [checked_yaml collection json_annotation pub_semver yaml]
- quiver 3.2.2 [matcher]
- record_android 1.3.0 [flutter record_platform_interface]
- record_darwin 1.2.2 [flutter record_platform_interface]
- record_linux 0.7.2 [flutter record_platform_interface]
- record_platform_interface 1.2.0 [flutter plugin_platform_interface]
- record_web 1.1.4 [flutter flutter_web_plugins web record_platform_interface]
- record_windows 1.0.4 [flutter record_platform_interface]
- safe_local_storage 1.0.2 [path synchronized]
- shared_preferences 2.3.3 [flutter shared_preferences_android shared_preferences_foundation shared_preferences_linux shared_preferences_platform_interface shared_preferences_web shared_preferences_windows]
- shared_preferences_android 2.3.4 [flutter shared_preferences_platform_interface]
- shared_preferences_foundation 2.5.3 [flutter shared_preferences_platform_interface]
- shared_preferences_linux 2.4.1 [file flutter path path_provider_linux path_provider_platform_interface shared_preferences_platform_interface]
- shared_preferences_platform_interface 2.4.1 [flutter plugin_platform_interface]
- shared_preferences_web 2.4.2 [flutter flutter_web_plugins shared_preferences_platform_interface web]
- shared_preferences_windows 2.4.1 [file flutter path path_provider_platform_interface path_provider_windows shared_preferences_platform_interface]
- shelf 1.4.2 [async collection http_parser path stack_trace stream_channel]
- shelf_web_socket 2.0.1 [shelf stream_channel web_socket_channel]
- sky_engine 0.0.0
- source_gen 1.5.0 [analyzer async build dart_style glob path source_span yaml]
- source_span 1.10.0 [collection path term_glyph]
- speech_to_text_platform_interface 2.3.0 [flutter meta plugin_platform_interface]
- sprintf 7.0.0
- stack_trace 1.12.0 [path]
- stream_channel 2.1.2 [async]
- stream_transform 2.1.0
- string_scanner 1.3.0 [source_span]
- synchronized 3.3.0+3
- term_glyph 1.2.1
- test_api 0.7.3 [async boolean_selector collection meta source_span stack_trace stream_channel string_scanner term_glyph]
- timing 1.0.2 [json_annotation]
- typed_data 1.4.0 [collection]
- unicode 0.3.1 [lists]
- universal_platform 1.1.0
- uri_parser 2.0.2 [path safe_local_storage]
- uuid 4.5.1 [crypto sprintf meta fixnum]
- vector_math 2.1.4
- video_player_android 2.7.16 [flutter video_player_platform_interface]
- video_player_avfoundation 2.6.5 [flutter video_player_platform_interface]
- video_player_platform_interface 6.2.3 [flutter plugin_platform_interface]
- video_player_web 2.3.3 [flutter flutter_web_plugins video_player_platform_interface web]
- vm_service 14.3.0
- watcher 1.1.0 [async path]
- web 1.1.0
- web_socket 0.1.6 [web]
- web_socket_channel 3.0.1 [async crypto stream_channel web web_socket]
- win32 5.9.0 [ffi]
- xdg_directories 1.1.0 [meta path]
- xml 6.5.0 [collection meta petitparser]
- yaml 3.1.2 [collection source_span string_scanner]

Additional context and comments

No response