Skip to content
This repository was archived by the owner on May 9, 2024. It is now read-only.

[Date Conversion] Enables conversion date to int #427

Closed
wants to merge 1 commit into from
Closed

[Date Conversion] Enables conversion date to int #427

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion omniscidb/QueryBuilder/QueryBuilder.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -828,7 +828,7 @@ BuilderExpr BuilderExpr::cast(const Type* new_type) const {
return {builder_, expr_->cast(new_type), "", true};
}
} else if (expr_->type()->isDate()) {
if (new_type->isDate() || new_type->isTimestamp()) {
if (new_type->isNumber() || new_type->isDate() || new_type->isTimestamp()) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isNumber includes decimals and FP types. Is it intentional?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I took conversion timestamp as an example, it uses 'timestamp' -> 'number, date, timestamp'. Maybe we should create some conversion map?

return {builder_, expr_->cast(new_type), "", true};
}
} else if (expr_->type()->isTime()) {
Expand Down
38 changes: 38 additions & 0 deletions omniscidb/Tests/QueryBuilderTest.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -321,6 +321,21 @@ class QueryBuilderTest : public TestSuite {
{"col_vc_10", ctx().varChar(10)},
});

createTable("test_date",
{
{"col_bi", ctx().int64()},
{"col_i", ctx().int32()},
{"col_f", ctx().fp32()},
{"col_d", ctx().fp64()},
{"col_dec", ctx().decimal64(10, 2)},
{"col_b", ctx().boolean()},
{"col_str", ctx().text()},
{"col_date", ctx().date32(hdk::ir::TimeUnit::kDay)},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to check other date sizes and time units as well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, it's initial check.

});
insertCsvValues("test_date",
"1,1,0.75,0.13444545,50.02,false,some_text,2000-01-01\n"
"2,2,30.82,0.461,7.05,true,a_text,2015-03-18\n");

createTable("sort",
{{"x", ctx().int32()}, {"y", ctx().int32()}, {"z", ctx().int32()}});
insertCsvValues("sort",
Expand Down Expand Up @@ -348,6 +363,7 @@ class QueryBuilderTest : public TestSuite {
dropTable("join1");
dropTable("join2");
dropTable("withNull");
dropTable("test_date");
}

void compare_res_fields(const ExecutionResult& res,
Expand Down Expand Up @@ -493,6 +509,28 @@ TEST_F(QueryBuilderTest, Arithmetics) {
compare_res_data(res, std::vector<int64_t>({0, NULL_BIGINT}));
}

TEST_F(QueryBuilderTest, DateToInt) {
QueryBuilder builder(ctx(), schema_mgr_, configPtr());

auto tinfo_a = builder.scan("test_date");

auto dag = tinfo_a.proj(tinfo_a.ref("col_date").cast("int32")).finalize();
auto res = runQuery(std::move(dag));
LOG(ERROR) << "res: " << res.toString();
LOG(ERROR) << "res: " << toArrow(res)->ToString();
compare_res_data(res, std::vector<int32_t>({946684800, 1426636800}));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these numbers what you would expect for used dates in kDay time unit?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I converted 2000-01-01, 2015-03-18 to int and used this values. (also checked what happens with online converter https://timestamp.online) In general it can be expanded to Hours and Milli.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This converter gives you a timestamp in seconds. I would expect different int values when casting dates in different units.


dag = tinfo_a.proj(tinfo_a.ref("col_date")).finalize();
res = runQuery(std::move(dag));
LOG(ERROR) << "res: " << res.toString();
LOG(ERROR) << "res: " << toArrow(res)->ToString();
compare_res_data(
res,
std::vector<int64_t>(
{dateTimeParse<hdk::ir::Type::kDate>("2000-01-01", TimeUnit::kDay),
dateTimeParse<hdk::ir::Type::kDate>("2015-03-18", TimeUnit::kDay)}));
}

TEST_F(QueryBuilderTest, Arithmetics2) {
QueryBuilder builder(ctx(), schema_mgr_, configPtr());

Expand Down