Skip to content

Commit d7a9db3

Browse files
committed
Added dockerfile and updated README
1 parent af20b1b commit d7a9db3

File tree

3 files changed

+115
-57
lines changed

3 files changed

+115
-57
lines changed

CHANGELOG.md

Lines changed: 32 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,15 @@
11
Change Log
22
==========
33

4-
All notable changes to Fili will be documented here. Changes are accumulated as new paragraphs at the top of the current
4+
All notable changes to Fili will be documented here. Changes are accumulated as new paragraphs at the top of the current
55
major version. Each change has a link to the pull request that makes the change and to the issue that triggered the
66
pull request if there was one.
77

88
Current
99
-------
1010

1111
### Added:
12+
- [Added dockerfile and updated README](https://github.com/yahoo/fili/pull/429)
1213

1314
- [Add Table-wide Availability](https://github.com/yahoo/fili/pull/414)
1415
* Add `availableIntervals` field to tables endpoint by union the availability for the logical table without taking
@@ -290,7 +291,7 @@ Removals:
290291
* Bound and default versions of getAvailableIntervals and getAllAvailableIntervals added to PhysicalTable interface
291292
* Package-private optimize tests in `DruidQueryBuilder` moved to protected
292293
* Immutable `NoVolatileIntervalsFunction` class made final
293-
294+
294295
- [Moved UnionDataSource to support only single tables](https://github.com/yahoo/fili/pull/262)
295296
* `UnionDataSource` now accepts only single tables instead of sets of tables.
296297
* `DataSource` now supports `getDataSource()` operation
@@ -369,7 +370,7 @@ Removals:
369370
* `TableLoader` now takes an additional constructor argument (`DataSourceMetadataService`) for creating tables
370371
* `PartialDataHandler::findMissingRequestTimeGrainIntervals` now takes `DataSourceConstraint`
371372
* Renamed `buildTableGroup` method to `buildDimensionSpanningTableGroup`
372-
373+
373374
- [Restored flexibility about columns for query from DruidResponseParser](https://github.com/yahoo/fili/pull/198)
374375
* Immutable schemas prevented custom query types from changing `ResultSetSchema` columns.
375376
* Columns are now sourced from `DruidResponseParser` and default implemented on `DruidAggregationQuery`
@@ -539,7 +540,7 @@ Removals:
539540
- [Fixed `SegmentMetadataLoader` Unconfigured Dimension Bug](https://github.com/yahoo/fili/pull/197)
540541
* Immutable availability was failing when attempting to bind segment dimension columns not configured in the
541542
dimension dictionary.
542-
* Fix to filter irrelevant column names.
543+
* Fix to filter irrelevant column names.
543544

544545
- [Major refactor for availability and schemas and tables](https://github.com/yahoo/fili/pull/165)
545546
* Ordering of fields on serialization could be inconsistent if intermediate stages used `HashSet` or `HashMap`.
@@ -665,8 +666,8 @@ Changes:
665666
### Changed:
666667

667668
- [The druid query posting timer has been removed](https://github.com/yahoo/fili/pull/141)
668-
* There wasn't really a good way of stopping timing only the posting itself. Since the timer is
669-
probably not that useful, it has been removed.
669+
* There wasn't really a good way of stopping timing only the posting itself. Since the timer is
670+
probably not that useful, it has been removed.
670671

671672
- [Dimension Field Tagging and Dynamic Dimension Field Serilization](https://github.com/yahoo/fili/pull/137)
672673
* Changed `fili-core` dimension endpoint `DimensionField` serialization strategy from hard coded static attributes
@@ -712,7 +713,7 @@ Changes:
712713
* `Response.StatusType` is the interface that `Response.Status` implements.
713714
* This will have no impact on current code in Fili that uses `ResponseValidationException`, and it allows customers
714715
to inject http codes not included in `Response.Status`.
715-
716+
716717
- [Removed "provided" modifier for SLF4J and Logback dependencies in the Wikipedia example](https://github.com/yahoo/fili/pull/102)
717718

718719
- [Updated dependencies](https://github.com/yahoo/fili/pull/103)
@@ -833,7 +834,7 @@ Changes:
833834
* `AggregationAverageMaker` deprecated conversion method required by deprecated sketch library
834835

835836
- [Metric configuration deprecations](https://github.com/yahoo/fili/pull/124)
836-
* Deprecated superfluous constructor of `FilteredAggregator` with superfluous argument
837+
* Deprecated superfluous constructor of `FilteredAggregator` with superfluous argument
837838
* Deprecated MetricMaker utility method in favor of using new field accessor on Metric
838839

839840
- [Deprecated MetricMaker.getDependentQuery lookup method in favor of simpler direct access](https://github.com/yahoo/fili/pull/124)
@@ -856,7 +857,7 @@ Changes:
856857
* Discovered a bug where `user_roles` is declared but unset still reads as a list with empty string (included a temporary fix by commenting the variable declaration)
857858
* Refactored `RoleBasedAuthFilter` and `RoleBasedAuthFilterSpec` for better testing
858859

859-
- [Added missing coverage for `ThetaSketchEstimate` unwrapping in `MetricMaker.getSketchField`](https://github.com/yahoo/fili/pull/128)
860+
- [Added missing coverage for `ThetaSketchEstimate` unwrapping in `MetricMaker.getSketchField`](https://github.com/yahoo/fili/pull/128)
860861

861862
- [`DataSource::getNames` now returns Fili identifiers, not fact store identifiers](https://github.com/yahoo/fili/pull/125/files)
862863

@@ -868,7 +869,7 @@ Changes:
868869

869870
### Removed:
870871

871-
- [Removed invalid constructor from SketchRoundUpMappepr](https://github.com/yahoo/fili/pull/148)
872+
- [Removed invalid constructor from SketchRoundUpMappepr](https://github.com/yahoo/fili/pull/148)
872873

873874

874875
v0.6.29 - 2016/11/16
@@ -899,7 +900,7 @@ New Capabilities & Enhancements:
899900

900901
- [Added Dimension Value implementation for PartitionTableDefinition]
901902
* Added `DimensionIdFilter` implementation of `DataSourceFilter`
902-
* Created `DimensionListPartitionTableDefinition`
903+
* Created `DimensionListPartitionTableDefinition`
903904

904905
- [Added 'hasAnyRows' to SearchProvider interface](https://github.com/yahoo/fili/pull/259)
905906
* Has Any Rows allows implementations to optimize queries which only need to identify existence of matches
@@ -921,8 +922,8 @@ New Capabilities & Enhancements:
921922
* Slice availability can be used to debug availability issues on Physical tables
922923

923924
- [Ability to set headers for requests to Druid](https://github.com/yahoo/fili/pull/62)
924-
* The `AsyncDruidWebServiceImpl` now accepts a `Supplier<Map<String, String>>` argument which specifies the headers
925-
to add to the Druid data requests. This feature is made configurable through `SystemConfig` in the
925+
* The `AsyncDruidWebServiceImpl` now accepts a `Supplier<Map<String, String>>` argument which specifies the headers
926+
to add to the Druid data requests. This feature is made configurable through `SystemConfig` in the
926927
`AbstractBinderFactory`.
927928

928929
### Changed:
@@ -942,12 +943,12 @@ New Capabilities & Enhancements:
942943
* This change is made to allow running multi-api request with csv format using chrome browser.
943944

944945
- [Improves error messages when querying Druid goes wrong](https://github.com/yahoo/fili/pull/61)
945-
* The `ResponseException` now includes a message that prints the `ResponseException`'s internal state
946-
(i.e. the druid query and response code) using the error messages
946+
* The `ResponseException` now includes a message that prints the `ResponseException`'s internal state
947+
(i.e. the druid query and response code) using the error messages
947948
`ErrorMessageFormat::FAILED_TO_SEND_QUERY_TO_DRUID` and `ErrorMessageFormat::ERROR_FROM_DRUID`
948-
* The druid query and status code, reason and response body are now logged at the error level in the
949-
failure and error callbacks in `AsyncDruidWebServiceImpl`
950-
949+
* The druid query and status code, reason and response body are now logged at the error level in the
950+
failure and error callbacks in `AsyncDruidWebServiceImpl`
951+
951952
- [Fili now supports custom Druid query types](https://github.com/yahoo/fili/pull/57)
952953
* `QueryType` has been turned into an interface, backed by an enum `DefaultQueryType`.
953954
- The default implementations of `DruidResponseParser` `DruidQueryBuilder`, `WeightEvaluationQuery` and
@@ -974,9 +975,9 @@ New Capabilities & Enhancements:
974975

975976
- Cleaned up dependencies in pom files
976977
* Moved version management of dependencies up to the parent Pom's dependency management section
977-
* Cleaned up the parent Pom's dependency section to only be those dependencies that truly _every_ sub-project should
978+
* Cleaned up the parent Pom's dependency section to only be those dependencies that truly _every_ sub-project should
978979
depend on.
979-
* Cleaned up sub-project Pom dependency sections to handle and better use the dependencies the parent Pom provides
980+
* Cleaned up sub-project Pom dependency sections to handle and better use the dependencies the parent Pom provides
980981

981982
### Deprecated:
982983

@@ -1007,22 +1008,22 @@ New Capabilities & Enhancements:
10071008

10081009
- [Adds read locking to all attempts to read the Lucene index](https://github.com/yahoo/fili/pull/52)
10091010
* Before, if Fili attempted to read from the Lucene indices (i.e. processing a query with filters) while loading
1010-
dimension indices, the request would fail and we would get a `LuceneIndexReaderAlreadyClosedException`. Now, the
1011+
dimension indices, the request would fail and we would get a `LuceneIndexReaderAlreadyClosedException`. Now, the
10111012
read locks should ensure that the query processing will wait until indexing completes (and vice versa).
10121013

10131014
- [Fixes a bug where job metadata was being stored in the `ApiJobStore` even when the results came back synchronously](https://github.com/yahoo/fili/pull/49)
1014-
* The workflow that updates the job's metadata with `success` was running even when the query was synchronous. That
1015+
* The workflow that updates the job's metadata with `success` was running even when the query was synchronous. That
10151016
update also caused the ticket to be stored in the `ApiJobStore`.
10161017
* The delay operator didn't stop the "update" workflow from executing because it viewed an `Observable::onCompleted`
1017-
call as a message for the purpose of the delay. Since the two observables that that the metadata update gated on
1018+
call as a message for the purpose of the delay. Since the two observables that that the metadata update gated on
10181019
are empty when the query is synchronous, the "update metadata" workflow was being triggered every time.
10191020
* The delay operator was replaced by `zipWith` as a gating mechanism.
1020-
1021+
10211022
- [#45, removing sorting from weight check queries](https://github.com/yahoo/fili/pull/46)
10221023

10231024
- [`JsonSlurper` can now handle sorting lists with mixed-type entries](https://github.com/yahoo/fili/pull/58)
10241025
* even if the list starts with a string, number, or boolean
1025-
1026+
10261027
- [Broken segment metadata with Druid v0.9.1](https://github.com/yahoo/fili/issues/63)
10271028
* Made `NumberedShardSpec` ignore unexpected properties during deserialization
10281029
* Added tests to `DataSourceMetadataLoaderSpec` to test the v.0.9.1 optional field `shardSpec.partitionDimensions`
@@ -1041,7 +1042,7 @@ Jobs resource. Here are the highlights of what's in this release:
10411042
- Filtering and pagination on the Jobs resource
10421043
- A `userId` field for default Job resource representations
10431044
- Package cleanup for the jobs-related classes
1044-
1045+
10451046
### Added:
10461047

10471048
- [`always` keyword for the `asyncAfter` parameter now guarantees that a query will be asynchronous](https://github.com/yahoo/fili/pull/39)
@@ -1061,7 +1062,7 @@ Jobs resource. Here are the highlights of what's in this release:
10611062
* Added `JobRowFilter` to hold filter information
10621063

10631064
- [QueryTimeLookup Functionality Testing](https://github.com/yahoo/fili/pull/34)
1064-
* Added two tests `LookupDimensionFilteringDataServletSpec` and `LookupDimensionGroupingDataServletSpec` to test QTL
1065+
* Added two tests `LookupDimensionFilteringDataServletSpec` and `LookupDimensionGroupingDataServletSpec` to test QTL
10651066
functionality
10661067

10671068
- [Lookup Dimension Serializer](https://github.com/yahoo/fili/pull/31)
@@ -1095,7 +1096,7 @@ Jobs resource. Here are the highlights of what's in this release:
10951096
for thread-safe tests.
10961097

10971098
- [Removed `JobsApiRequest::handleBroadcastChannelNotification`](https://github.com/yahoo/fili/pull/39)
1098-
* That logic does not really belong in the `JobsApiRequest` (which is responsible for modeling a response, not
1099+
* That logic does not really belong in the `JobsApiRequest` (which is responsible for modeling a response, not
10991100
processing it), and has been consolidated into the `JobsServlet`.
11001101

11011102
- [ISSUE-17](https://github.com/yahoo/fili/issues/17) [Added pagination parameters to `PreResponse`](https://github.com/yahoo/fili/pull/19)
@@ -1105,7 +1106,7 @@ Jobs resource. Here are the highlights of what's in this release:
11051106
* The default job payload generated by `DefaultJobPayloadBuilder` now has a `userId`
11061107

11071108
- [Removed timing component in JobsApiRequestSpec](https://github.com/yahoo/fili/pull/27)
1108-
* Rather than setting an async timeout, and then sleeping, `JobsApiRequestSpec::handleBroadcastChannelNotification`
1109+
* Rather than setting an async timeout, and then sleeping, `JobsApiRequestSpec::handleBroadcastChannelNotification`
11091110
returns an empty Observable if a timeout occurs before the notification is received now verifies that the Observable
11101111
returned terminates without sending any messages.
11111112

@@ -1126,11 +1127,11 @@ Jobs resource. Here are the highlights of what's in this release:
11261127
* `AbstractBinderFactory` now uses `TypeAwareDimensionLoader` instead of `KeyValueStoreDimensionLoader`
11271128

11281129
- [Fix Dimension Serialization Problem with Nested Queries](https://github.com/yahoo/fili/pull/15)
1129-
* Modified `DimensionToDefaultDimensionSpec` serializer to serialize Dimension to apiName if it's not in the
1130+
* Modified `DimensionToDefaultDimensionSpec` serializer to serialize Dimension to apiName if it's not in the
11301131
inner-most query
11311132
* Added `Util::hasInnerQuery` helper in serializer package to determine if query is the inner most query or not
11321133
* Added tests for `DimensionToDefaultDimensionSpec`
1133-
1134+
11341135
#### General:
11351136

11361137
- [Preserve collection order of dimensions, dimension fields and metrics](https://github.com/yahoo/fili/pull/25)

fili-generic-example/Dockerfile

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
FROM druidio/example-cluster:latest
2+
3+
# fili github information
4+
ENV GITHUB_OWNER=yahoo BRANCH=master
5+
6+
RUN apt-get update && apt-get install -y git
7+
8+
RUN wget -q -O - http://archive.apache.org/dist/maven/maven-3/3.2.5/binaries/apache-maven-3.2.5-bin.tar.gz | tar -xzf - -C /usr/local \
9+
&& ln -s /usr/local/apache-maven-3.2.5 /usr/local/apache-maven \
10+
&& ln -s -f /usr/local/apache-maven/bin/mvn /usr/local/bin/mvn
11+
12+
# fili code goes here
13+
RUN mkdir -p /usr/local/fili
14+
15+
# update in case of new commits
16+
ADD https://api.github.com/repos/$GITHUB_OWNER/fili/git/refs/heads/$BRANCH \
17+
fili-branch.json
18+
RUN git clone -q --branch $BRANCH --depth 1 \
19+
https://github.com/$GITHUB_OWNER/fili.git /usr/local/fili
20+
21+
WORKDIR /usr/local/fili
22+
23+
# install Fili in the container
24+
RUN mvn -U -B install -DskipTests=true -Dmaven.javadoc.skip=true -Dcheckstyle.skip
25+
26+
# run with run --rm -i -p 3000:8082 -p 3001:8081 !!
27+
EXPOSE 9998
28+
29+
ENTRYPOINT export HOSTIP="$(resolveip -s $HOSTNAME)" \
30+
&& exec /usr/bin/supervisord -c /etc/supervisor/conf.d/supervisord.conf \
31+
& mvn -pl fili-generic-example exec:java -Dbard__fili_port=9998 \
32+
-Dbard__druid_coord=http://localhost:8081/druid/coordinator/v1 \
33+
-Dbard__non_ui_druid_broker=http://localhost:8082/druid/v2 \
34+
-Dbard__ui_druid_broker=http://localhost:8082/druid/v2
35+
36+
# mvn clean install? load own data? new druid image? documentation?

0 commit comments

Comments
 (0)