Skip to content

Drop dwrf and parquet dependency from HiveDataSinkTest #10

@yingsu00

Description

@yingsu00

Description

This file has the following includes:

#include "velox/dwio/dwrf/reader/DwrfReader.h"
#include "velox/dwio/dwrf/writer/FlushPolicy.h"
#include "velox/dwio/dwrf/writer/Writer.h"
#include "velox/exec/tests/utils/PlanBuilder.h"
#include "velox/exec/tests/utils/TempDirectoryPath.h"
#include "velox/vector/fuzzer/VectorFuzzer.h"

#ifdef VELOX_ENABLE_PARQUET
#include "velox/dwio/parquet/RegisterParquetReader.h"
#include "velox/dwio/parquet/RegisterParquetWriter.h"
#include "velox/dwio/parquet/reader/ParquetReader.h"
#include "velox/dwio/parquet/writer/Writer.h"
#endif

They should be removed and only include velox/dwio/common ones

The first step is to be able to create a FlushPolicyFactory that is similar to ReaderFactory and a registry flushPolicyFactories() just like readerFactories(). Make sure we can do something like this

 auto flushPolicy = flushPolicyFactories().getLambdaFactory(FileFormat::PARQUET);
 auto flushPolicy = flushPolicyFactories().getDefaultFactory(FileFormat::DWRF);

or

auto flushPolicy = flushPolicyFactories().find(std::make_pair(FileFormat::PARQUET, FlushPolicyType::Default));

Then remove all file format specific references in the test.

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions