Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Casting a DATE literal without time part to TIME datatype fails #1657

Open
wants to merge 43 commits into
base: BABEL_3_X_DEV
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
3113bd6
Casting a DATE literal without time part to TIME datatype fails
Suthapalli-Ramya-satya-vasavi-srija Jul 19, 2023
94c338e
Added hook and modified the tests
Suthapalli-Ramya-satya-vasavi-srija Aug 7, 2023
15364ef
Merge branch 'BABEL_3_X_DEV' into jira-babel-1528-vasavi
Suthapalli-Ramya-satya-vasavi-srija Aug 7, 2023
63994c6
Merge branch 'babelfish-for-postgresql:BABEL_3_X_DEV' into jira-babel…
Suthapalli-Ramya-satya-vasavi-srija Aug 14, 2023
5970208
Add support for 'dd mon yyyy' syntax and add tests
Suthapalli-Ramya-satya-vasavi-srija Aug 14, 2023
5abb533
Fix build failure
Suthapalli-Ramya-satya-vasavi-srija Aug 14, 2023
ef3b0a5
Empty commit
Suthapalli-Ramya-satya-vasavi-srija Aug 14, 2023
aeaaf58
Merge branch 'babelfish-for-postgresql:BABEL_3_X_DEV' into jira-babel…
Suthapalli-Ramya-satya-vasavi-srija Aug 16, 2023
425667a
Merge branch 'babelfish-for-postgresql:BABEL_3_X_DEV' into jira-babel…
Suthapalli-Ramya-satya-vasavi-srija Aug 22, 2023
45d14ee
Rename hooks, add hook for handling month case and add tests
Suthapalli-Ramya-satya-vasavi-srija Aug 22, 2023
8d0cead
Merge branch 'BABEL_3_X_DEV' into jira-babel-1528-vasavi
Suthapalli-Ramya-satya-vasavi-srija Sep 7, 2023
5256372
Merge branch 'babelfish-for-postgresql:BABEL_3_X_DEV' into jira-babel…
Suthapalli-Ramya-satya-vasavi-srija Sep 14, 2023
2b52253
Removed all the logic and called a separate function to handle time d…
Suthapalli-Ramya-satya-vasavi-srija Sep 19, 2023
165e5bf
Merge branch 'babelfish-for-postgresql:BABEL_3_X_DEV' into jira-babel…
Suthapalli-Ramya-satya-vasavi-srija Sep 19, 2023
8438f2a
Implemented to-do parts and formatting
Suthapalli-Ramya-satya-vasavi-srija Sep 28, 2023
d129287
Merge branch 'BABEL_3_X_DEV' into jira-babel-1528-vasavi
Suthapalli-Ramya-satya-vasavi-srija Sep 28, 2023
95f935e
Empty commit
Suthapalli-Ramya-satya-vasavi-srija Sep 28, 2023
83c5c9b
Remove additional changes came when merging
Suthapalli-Ramya-satya-vasavi-srija Sep 28, 2023
9d9e3bc
Modified the formatting and comments
Suthapalli-Ramya-satya-vasavi-srija Oct 4, 2023
a545dee
Merge branch 'babelfish-for-postgresql:BABEL_3_X_DEV' into jira-babel…
Suthapalli-Ramya-satya-vasavi-srija Oct 4, 2023
1be81a1
Removed unused variables
Suthapalli-Ramya-satya-vasavi-srija Oct 4, 2023
4e77880
Added comments for code
Suthapalli-Ramya-satya-vasavi-srija Oct 5, 2023
897c4fa
Merge branch 'babelfish-for-postgresql:BABEL_3_X_DEV' into jira-babel…
Suthapalli-Ramya-satya-vasavi-srija Oct 11, 2023
a317305
Formatting fixes and new fixes implementation for offset
Suthapalli-Ramya-satya-vasavi-srija Oct 11, 2023
f7c6a26
Include correct header files
Suthapalli-Ramya-satya-vasavi-srija Oct 11, 2023
aa54d49
Remove unnecessary changes from tsqlIface.cpp file
Suthapalli-Ramya-satya-vasavi-srija Oct 11, 2023
ae5430d
Implemented validation of dates like 02000-2-2 and added tests for it
Suthapalli-Ramya-satya-vasavi-srija Oct 12, 2023
e9457a6
Resolve failures
Suthapalli-Ramya-satya-vasavi-srija Oct 12, 2023
b5f22b9
Merge branch 'babelfish-for-postgresql:BABEL_3_X_DEV' into jira-babel…
Suthapalli-Ramya-satya-vasavi-srija Oct 12, 2023
a59fa96
Accept 0's infront of year and date only if text month is present
Suthapalli-Ramya-satya-vasavi-srija Oct 12, 2023
3ce386a
Merge branch 'BABEL_3_X_DEV' into jira-babel-1528-vasavi
Suthapalli-Ramya-satya-vasavi-srija Oct 18, 2023
edf89fb
Moved the logic to babelfish_common and fixed indentation and added m…
Suthapalli-Ramya-satya-vasavi-srija Oct 18, 2023
c143d3f
Removed unwanted changes and added the test to all schedule files
Suthapalli-Ramya-satya-vasavi-srija Oct 19, 2023
58061ea
Fix upgrade test failures and also covered positive case for hh:mm:ss…
Suthapalli-Ramya-satya-vasavi-srija Oct 20, 2023
a222981
Merge branch 'babelfish-for-postgresql:BABEL_3_X_DEV' into jira-babel…
Suthapalli-Ramya-satya-vasavi-srija Oct 20, 2023
71aa6b5
Empty commit to check if the tests are failing againa
Suthapalli-Ramya-satya-vasavi-srija Oct 20, 2023
9201ca5
Support '' for casting to time and addressed comments in the pr
Suthapalli-Ramya-satya-vasavi-srija Oct 23, 2023
4e9b59b
Merge branch 'babelfish-for-postgresql:BABEL_3_X_DEV' into jira-babel…
Suthapalli-Ramya-satya-vasavi-srija Oct 24, 2023
4c1c577
Empty commit
Suthapalli-Ramya-satya-vasavi-srija Oct 24, 2023
1f342f3
Merge branch 'babelfish-for-postgresql:BABEL_3_X_DEV' into jira-babel…
Suthapalli-Ramya-satya-vasavi-srija Jan 3, 2024
47150ec
Merge branch 'babelfish-for-postgresql:BABEL_3_X_DEV' into jira-babel…
Suthapalli-Ramya-satya-vasavi-srija Jan 23, 2024
35617aa
Added more cases related to tim(HH:MM:SS:nnn) and restricted the date…
Suthapalli-Ramya-satya-vasavi-srija Jan 23, 2024
eae2cc4
Increased sla for parallel-query-mode, and added a case where code co…
Suthapalli-Ramya-satya-vasavi-srija Jan 24, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
260 changes: 260 additions & 0 deletions contrib/babelfishpg_tsql/src/hooks.c
Original file line number Diff line number Diff line change
Expand Up @@ -41,10 +41,13 @@
#include "parser/parser.h"
#include "parser/scanner.h"
#include "parser/scansup.h"
#include "regex/regex.h"
#include "replication/logical.h"
#include "rewrite/rewriteHandler.h"
#include "tcop/utility.h"
#include "utils/builtins.h"
#include "utils/date.h"
#include "utils/datetime.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
Expand Down Expand Up @@ -139,6 +142,9 @@ static bool pltsql_bbfCustomProcessUtility(ParseState *pstate,
const char *queryString,
ProcessUtilityContext context,
ParamListInfo params, QueryCompletion *qc);
static bool isTextMonthPresent(char* field);
static bool containsInTextMonthFormat(int *ftype, char **field);
static Datum pltsql_time_in(const char *str, int32 typmod);
static void pltsql_bbfSelectIntoAddIdentity(IntoClause *into, List *tableElts);
extern void pltsql_bbfSelectIntoUtility(ParseState *pstate, PlannedStmt *pstmt, const char *queryString,
QueryEnvironment *queryEnv, ParamListInfo params, QueryCompletion *qc);
Expand Down Expand Up @@ -224,6 +230,7 @@ static table_variable_satisfies_update_hook_type prev_table_variable_satisfies_u
static table_variable_satisfies_vacuum_hook_type prev_table_variable_satisfies_vacuum = NULL;
static table_variable_satisfies_vacuum_horizon_hook_type prev_table_variable_satisfies_vacuum_horizon = NULL;
static drop_relation_refcnt_hook_type prev_drop_relation_refcnt_hook = NULL;
static tsql_time_in_hook_type prev_tsql_time_in_hook = NULL;

/*****************************************
* Install / Uninstall
Expand Down Expand Up @@ -384,6 +391,9 @@ InstallExtendedHooks(void)

prev_drop_relation_refcnt_hook = drop_relation_refcnt_hook;
drop_relation_refcnt_hook = pltsql_drop_relation_refcnt_hook;

prev_tsql_time_in_hook = tsql_time_in_hook;
tsql_time_in_hook = pltsql_time_in;
Suthapalli-Ramya-satya-vasavi-srija marked this conversation as resolved.
Show resolved Hide resolved
}

void
Expand Down Expand Up @@ -446,6 +456,7 @@ UninstallExtendedHooks(void)
IsToastRelationHook = PrevIsToastRelationHook;
IsToastClassHook = PrevIsToastClassHook;
drop_relation_refcnt_hook = prev_drop_relation_refcnt_hook;
tsql_time_in_hook = prev_tsql_time_in_hook;
}

/*****************************************
Expand Down Expand Up @@ -2822,6 +2833,255 @@ pltsql_detect_numeric_overflow(int weight, int dscale, int first_block, int nume
return (total_digit_count > TDS_NUMERIC_MAX_PRECISION);
}

/* Checks whether the field is valid text month */
static bool isTextMonthPresent(char* field)
{
char* months[] = {"january", "february", "march", "april", "may",
"june", "july", "august", "september", "october",
"november", "december", "jan", "feb", "mar", "apr",
"may", "jun", "jul", "aug", "sep", "oct", "nov", "dec"};
for(int i = 0; i < 24; i++)
Suthapalli-Ramya-satya-vasavi-srija marked this conversation as resolved.
Show resolved Hide resolved
if(pg_strcasecmp(field, months[i]) == 0)
return true;
return false;
}

/*
* This function will check whether the first 3 inputs are in any of the format
* `DD MON YYY`, `DD YYYY MON`, `MON DD YYYY`, `MON YYYY DD`, `YYYY MM DD`, `YYYY DD MM`
Suthapalli-Ramya-satya-vasavi-srija marked this conversation as resolved.
Show resolved Hide resolved
* where MON is month in text format then returns true if present.
*/
static bool containsInTextMonthFormat(int *ftype, char **field){
int count_number = 0, count_string = 0;
for(int i = 0; i < 3; i++)
Suthapalli-Ramya-satya-vasavi-srija marked this conversation as resolved.
Show resolved Hide resolved
{
if(ftype[i] == DTK_NUMBER)
count_number++;
else if(ftype[i] == DTK_STRING)
{
/* Check whether the string is valid text month */
if(!isTextMonthPresent(field[i]))
return false;
count_string++;
}
else
return false;
}
if(count_number == 2 && count_string == 1)
{
/*
* If the first field is an string then swap with the second field
* as when the date is given separatly then all different forms of
* dates is supported. To avoid the conversion failure from `isTextMonthPresent`
* later we are swapping earlier.
*/
if(ftype[0] == DTK_STRING)
{
char* temp_field;
int temp_ftype;
temp_field = field[0];
temp_ftype = ftype[0];
field[0] = field[1];
field[1] = temp_field;
ftype[0] = ftype[1];
ftype[1] = temp_ftype;
}
return true;
}
return false;
}

Datum pltsql_time_in(const char *str, int32 typmod)
{
Suthapalli-Ramya-satya-vasavi-srija marked this conversation as resolved.
Show resolved Hide resolved
TimeADT result;
fsec_t fsec;
struct pg_tm tt,
*tm = &tt;
int tz;
int nf;
int dterr;
char workbuf[MAXDATELEN + 1];
char *field[MAXDATEFIELDS];
int dtype;
int ftype[MAXDATEFIELDS];
StringInfo res ;
int status;

/* Throw a common error message while casting to time datatype */
#define TIME_IN_ERROR() \
ereport(ERROR, \
(errcode(ERRCODE_INVALID_DATETIME_FORMAT), \
errmsg("Conversion failed when converting date and/or time from character string."))); \

dterr = ParseDateTime(str, workbuf, sizeof(workbuf),
field, ftype, MAXDATEFIELDS, &nf);
if (dterr == 0)
{
if(nf >= 3)
Suthapalli-Ramya-satya-vasavi-srija marked this conversation as resolved.
Show resolved Hide resolved
{
/*
* If the input is of format "YYYY MON DD", then convert
* to format of YYYY-MON-DD and ftype changes to "DTK_DATE"
*/
if(containsInTextMonthFormat(ftype, field))
{
/*
* For example if input is "2000 nov 23" will be converted
* to "2000-nov-23".
*/
res = makeStringInfo();
appendStringInfo(res, "%s-%s-%s", field[0], field[1], field[2]);
field[0] = NULL;
Suthapalli-Ramya-satya-vasavi-srija marked this conversation as resolved.
Show resolved Hide resolved
field[0] = res->data;
Suthapalli-Ramya-satya-vasavi-srija marked this conversation as resolved.
Show resolved Hide resolved
ftype[0] = DTK_DATE;
nf = nf - 2;
/*
* Since the first 3 fields converted to 1,
* skip the attached fields.
*/
for(int i = 1; i < nf; i++)
{
field[i] = field[i+2];
ftype[i] = ftype[i+2];
}
}
}

for(int i = 0; i < nf; i++)
{
char *temp_field;
int len;
char* different_date_formats[] = {"/", ".", "-"};
temp_field = pstrdup(field[i]);
len = strlen(temp_field);
switch(ftype[i])
{
case DTK_NUMBER:
switch(len)
{
case 1:
case 2:
/*
* At this point the digit number of length 1,2 should be considered as
* time only if there is an [ap]m after the number or else
* should throw an error
*/
if(i == nf - 1 || ftype[i+1] != DTK_STRING)
TIME_IN_ERROR();
status = RE_compile_and_execute(cstring_to_text("^([ap]m)$"),
Suthapalli-Ramya-satya-vasavi-srija marked this conversation as resolved.
Show resolved Hide resolved
(char *) field[i+1], strlen(field[i+1]),
REG_ADVANCED, DEFAULT_COLLATION_OID,
0, NULL);
if(!status)
TIME_IN_ERROR();
/*
* For example if the input is "1 am"
* will be converted to "1:00:00 am".
*/
res = makeStringInfo();
appendStringInfo(res, "%s%s", field[i], ":00:00");
field[i] = NULL;
field[i] = res->data;
Suthapalli-Ramya-satya-vasavi-srija marked this conversation as resolved.
Show resolved Hide resolved
ftype[i] = DTK_TIME;
break;

case 4:
/*
* If the numeric input is of length 4, then convert
* to year with default format of YYYY-01-01.
* For example if input is "2000" will be converted
* to "2000-01-01".
*/
res = makeStringInfo();
appendStringInfo(res, "%s%s", field[i], "-01-01");
field[i] = NULL;
field[i] = res->data;
ftype[i] = DTK_DATE;
break;

case 6:
case 8:
res = makeStringInfo();
for(int k = 0; k < len; k++)
{
appendStringInfo(res, "%c", temp_field[k]);
Suthapalli-Ramya-satya-vasavi-srija marked this conversation as resolved.
Show resolved Hide resolved
if((len == 6 && ((k + 1) % 2 == 0 && k < 5)) ||
Suthapalli-Ramya-satya-vasavi-srija marked this conversation as resolved.
Show resolved Hide resolved
(len == 8 && (k == 3 || k == 5)))
appendStringInfo(res, "%c", '-');
}
field[i] = NULL;
field[i] = res->data;
ftype[i] = DTK_DATE;
break;
/*
* If the numeric is of length {3,5,7, >8} then an error should be thrown*/
default:
TIME_IN_ERROR();
}
break;

case DTK_DATE:
/*
* If the input is of format `Mon{/.-}yyyy{/.-}dd` or `Mon{/.-}dd{/.-}yyyy
* shouldn't be supported.
* Supported date formats are `Mon yyyy dd`, `Mon dd yyyy`, `mm{-/.}dd{-/.}yyyy`, `yyyy{-/.}mm{-/.}dd`
Suthapalli-Ramya-satya-vasavi-srija marked this conversation as resolved.
Show resolved Hide resolved
*/
for(int k = 0 ; k < 3; k++)
{
temp_field = strtok(temp_field, different_date_formats[k]);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are we not overwriting temp_field again without freeing earlier memory?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, because when tried to pfree resulting in crash.

if (pg_strcasecmp(field[i], temp_field) != 0)
Suthapalli-Ramya-satya-vasavi-srija marked this conversation as resolved.
Show resolved Hide resolved
break;
}
if(isTextMonthPresent(temp_field))
Suthapalli-Ramya-satya-vasavi-srija marked this conversation as resolved.
Show resolved Hide resolved
TIME_IN_ERROR();
break;

default:
break;
}
pfree(temp_field);
}
switch(nf)
{
case 1:
/*
* If only date is specified add an default time of
* 00:00:00
*/
if(ftype[0] == DTK_DATE)
{
ftype[1] = DTK_TIME;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we always have ftype/fields allocated irrespective of number of fields (nfs)? Here we have nf =1 but we are assuming that f*[1] is also available

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we are initializing the fields/ftype with max count at initialization. Later when we go deep into the code the field/ftype values are checked based on the nf count.

field[1] = "00:00:00";
nf = nf + 1;
}
break;
case 3:
/*
* If the given input is of format `yyyy-mm-ddThh:mm:ss`
* then conver it to `yyyy-mm-dd hh:mm:ss` by ignoring 'T'
*/
if(ftype[1] == DTK_STRING && pg_strcasecmp(field[1], "t") == 0 && ftype[2] == DTK_TIME)
Suthapalli-Ramya-satya-vasavi-srija marked this conversation as resolved.
Show resolved Hide resolved
{
ftype[1] = ftype[2];
field[1] = field[2];
nf = nf - 1;
}
Suthapalli-Ramya-satya-vasavi-srija marked this conversation as resolved.
Show resolved Hide resolved
default:
break;
}

dterr = DecodeTimeOnly(field, ftype, nf, &dtype, tm, &fsec, &tz);
}

if (dterr != 0)
TIME_IN_ERROR();

tm2time(tm, fsec, &result);
AdjustTimeForTypmod(&result, typmod);

PG_RETURN_TIMEADT(result);
}

/*
* Stores argument positions of default values of a PL/tsql function to bbf_function_ext catalog
*/
Expand Down
32 changes: 32 additions & 0 deletions contrib/babelfishpg_tsql/src/tsqlIface.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
#include <iostream>
#include <strstream>
#include <string>
#include <regex>
#include <unordered_map>

#pragma GCC diagnostic ignored "-Wattributes"
Expand Down Expand Up @@ -1990,6 +1991,37 @@ class tsqlBuilder : public tsqlCommonMutator
/* Re-write session_user to sys.session_user(). */
if (bctx->bif_no_brackets && bctx->SESSION_USER())
rewritten_query_fragment.emplace(std::make_pair(bctx->bif_no_brackets->getStartIndex(), std::make_pair(::getFullText(bctx->SESSION_USER()), "sys.session_user()")));

if(bctx->bif_cast_parse())
{
auto cast_parse = bctx->bif_cast_parse();
/* Re-write the expression for casting to time datatype. */
if(cast_parse->data_type() && pg_strncasecmp(::getFullText(cast_parse->data_type()).c_str(), "TIME", 4) == 0)
{
std::string cast_expression(pstrdup(::getFullText(cast_parse->expression()).c_str()));

/*
* Remove spaces between before or after {/:.} in the given expression.
Suthapalli-Ramya-satya-vasavi-srija marked this conversation as resolved.
Show resolved Hide resolved
* If the expression contains input of ' dd / mm/ yyyy hh: mm : ss . ss'
* will be converted to ' dd/mm/yyyy hh:mm:ss.ss'
*/
std::regex pattern("\\s*([/:.])\\s*");
std::string result_expression = std::regex_replace(cast_expression, pattern, "$1");

/*
* Remove spaces in between date in different kind of format dd {-} mm {-} yyyy.
* The {-} was not included above as the timezone can contain {+,-} which shouldn't be affected.
* For example if the expression contains ' dd - mm -yyyy hh:mm'
* will be converted to ' dd-mm-yyyy hh:mm'
*/
pattern = ("\\s*(\\d{2,4}|\\w+)\\s*(-)\\s*(\\d{1,2}|\\w+)\\s*(-)\\s*(\\d{1,4}|\\w+)\\s*");
result_expression = std::regex_replace(result_expression, pattern, "$1$2$3$4$5 ");

/* If the expression didn't change after comparing with above regex no need to change */
if ( cast_expression!= result_expression)
rewritten_query_fragment.emplace(std::make_pair(cast_parse->expression()->start->getStartIndex(), std::make_pair(::getFullText(cast_parse->expression()), result_expression)));
}
}
}

/* analyze scalar function call */
Expand Down
11 changes: 11 additions & 0 deletions test/JDBC/expected/babel_time-vu-cleanup.out
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
DROP TABLE babel_time_vu_prepare_t1
GO

DROP VIEW babel_time_vu_prepare_v1
GO

DROP PROCEDURE babel_time_vu_prepare_p1
GO

DROP FUNCTION babel_time_vu_prepare_f1
GO
21 changes: 21 additions & 0 deletions test/JDBC/expected/babel_time-vu-prepare.out
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
create table babel_time_vu_prepare_t1(a time)
Suthapalli-Ramya-satya-vasavi-srija marked this conversation as resolved.
Show resolved Hide resolved
GO

insert into babel_time_vu_prepare_t1 values ('2012-02-23')
GO
~~ROW COUNT: 1~~


create view babel_time_vu_prepare_v1 as select CAST('2012-02-23' AS time) as val
GO

create procedure babel_time_vu_prepare_p1 as select CAST('2012-02-23' AS time) as val
GO

create function babel_time_vu_prepare_f1()
returns time
as
begin
return (select CAST('2012-02-23' AS time) as val)
end
GO
Loading