[BUG] VPC flow log doesn't work with default setting

[joshuali925]

What is the bug?

1. the default config when creating vpc flow logs stores logs in text, but integrations only supports parquet. there is no error messages on the UI, so it's hard to debug
2. after specifying parquet, it is stored in this format: s3://bucket/AWSLogs/1111111111/vpcflowlogs/us-west-2/2024/09/05/xxx.log.parquet. but apparently the integrations query only supports hive format (e.g. year=2024/month=09/day=05). using the integration query directly leads to no results in the response
   1. this can be worked around using OPTIONS (recursiveFileLookup='true') at the end of the create table query, but this way there's no partition
3. with recursiveFileLookup, the query returns this error

   {
       "data": {
           "ok": true,
           "resp": {
               "status": "FAILED",
               "error": "{\"Message\":\"Fail to run query. Cause: null\"}"
           }
       }
   }

   the actual error says Column: [protocol], Expected: bigint, Found: INT32. so protocol column in create table query should be changed to bigint instead of int

How can one reproduce the bug?
Steps to reproduce the behavior:

1. create datasource s3 using vpc flow logs bucket
2. go to dashboards saved objects and import ndjson from https://github.com/opensearch-project/opensearch-catalog/releases/tag/amazon_vpc_flow_1.1.0 with overwrite=true
3. go to dashboards integrations, add vpc integration
4. see these issues

What is the expected behavior?
A clear and concise description of what you expected to happen.

What is your host/environment?

• OS: [e.g. iOS]
• Version [e.g. 22] 2.13
• Plugins

Do you have any screenshots?
If applicable, add screenshots to help explain your problem.

Do you have any additional context?
Add any other context about the problem.

[andrross]

[Catch All Triage - 1, 2, 3]