Skip to content

Commit 9488b41

Browse files
m-manum-manu
authored
Moved sections around a bit
1 parent a6576c6 commit 9488b41

File tree

1 file changed

+80
-79
lines changed

1 file changed

+80
-79
lines changed

‎README.md‎

Lines changed: 80 additions & 79 deletions
Original file line numberDiff line numberDiff line change
@@ -99,85 +99,6 @@ public class Citizen implements HBRecord<CitizenKey> {
9999
* The default codec class `BestSuitCodec` takes a flag `BestSuitCodec.SERIALIZE_AS_STRING`, whose value is "serializeAsString" (as in the above `Citizen` class example). When this flag is set to `true` on a field, the default codec serializes that field (even numerical fields) as strings.
100100
* Your custom codec may take other such flags to customize serialization/deserialization behavior at a **class field level**.
101101

102-
## Using this library in MapReduce jobs
103-
104-
### Mapper
105-
If your MapReduce job is reading from an HBase table, in your `map()` method, HBase's `Result` object can be converted to object of your bean-like class using below method:
106-
107-
```java
108-
T readValue(ImmutableBytesWritable rowKey, Result result, Class<T> clazz)
109-
```
110-
111-
For example:
112-
113-
```java
114-
Citizen e = hbObjectMapper.readValue(key, value, Citizen.class);
115-
```
116-
117-
### Reducer
118-
If your MapReduce job is writing to an HBase table, in your `reduce()` method, object of your bean-like class can be converted to HBase's `Put` (for row contents) and `ImmutableBytesWritable` (for row key) using below methods:
119-
120-
```java
121-
ImmutableBytesWritable getRowKey(HBRecord<R> obj)
122-
```
123-
```java
124-
Put writeValueAsPut(HBRecord<R> obj)
125-
```
126-
For example, below code in Reducer writes your object as one HBase row with appropriate column families and columns:
127-
128-
```java
129-
Citizen citizen = new Citizen(/*details*/);
130-
context.write(hbObjectMapper.getRowKey(citizen), hbObjectMapper.writeValueAsPut(citizen));
131-
```
132-
### Unit-test for Mapper
133-
If your MapReduce job is reading from an HBase table, you would want to unit-test your `map()` method as below.
134-
135-
Object of your bean-like class can be converted to HBase's `Result` (for row contents) and `ImmutableBytesWritable` (for row key) using below methods:
136-
137-
```java
138-
ImmutableBytesWritable getRowKey(HBRecord<R> obj)
139-
```
140-
```java
141-
Result writeValueAsResult(HBRecord<R> obj)
142-
```
143-
Below is an example of unit-test of a Mapper using [MRUnit](https://attic.apache.org/projects/mrunit.html):
144-
145-
```java
146-
Citizen citizen = new Citizen(/*params*/);
147-
citizenMapDriver
148-
.withInput(
149-
hbObjectMapper.getRowKey(citizen),
150-
hbObjectMapper.writeValueAsResult(citizen)
151-
)
152-
.withOutput(
153-
hbObjectMapper.toIbw("key"),
154-
new IntWritable(citizen.getAge())
155-
)
156-
.runTest();
157-
```
158-
159-
### Unit-test for Reducer
160-
If your MapReduce job is writing to an HBase table, you would want to unit-test your `reduce()` method as below.
161-
162-
HBase's `Put` object can be converted to your object of you bean-like class using below method:
163-
164-
```java
165-
T readValue(ImmutableBytesWritable rowKey, Put put, Class<T> clazz)
166-
```
167-
168-
Below is an example of unit-test of a Reducer using [MRUnit](https://attic.apache.org/projects/mrunit.html):
169-
170-
```java
171-
Pair<ImmutableBytesWritable, Mutation> reducerResult = citizenReduceDriver
172-
.withInput(hbObjectMapper.toIbw("key"), inputList)
173-
.run()
174-
.get(0);
175-
CitizenSummary citizenSummary = hbObjectMapper.readValue(
176-
reducerResult.getFirst(),
177-
(Put) reducerResult.getSecond(),
178-
CitizenSummary.class
179-
);
180-
```
181102
## Using this library for database access (DAO)
182103
This library provides an abstract class to define your own [data access object](https://en.wikipedia.org/wiki/Data_access_object). For example, you can create one for `Citizen` class in the above example as follows:
183104

@@ -292,6 +213,86 @@ citizenDao.getHBaseTable() // returns HTable instance (in case you want to direc
292213

293214
**Please note:** Since we're dealing with HBase (and not an OLTP data store), fitting a classical (Hibernate-like) ORM paradigm may not make sense. So this library doesn't intend to evolve as a full-fledged ORM. However, if that's your intent, I suggest you use [Apache Phoenix](https://phoenix.apache.org/).
294215

216+
## Using this library in MapReduce jobs
217+
218+
### Mapper
219+
If your MapReduce job is reading from an HBase table, in your `map()` method, HBase's `Result` object can be converted to object of your bean-like class using below method:
220+
221+
```java
222+
T readValue(ImmutableBytesWritable rowKey, Result result, Class<T> clazz)
223+
```
224+
225+
For example:
226+
227+
```java
228+
Citizen e = hbObjectMapper.readValue(key, value, Citizen.class);
229+
```
230+
231+
### Reducer
232+
If your MapReduce job is writing to an HBase table, in your `reduce()` method, object of your bean-like class can be converted to HBase's `Put` (for row contents) and `ImmutableBytesWritable` (for row key) using below methods:
233+
234+
```java
235+
ImmutableBytesWritable getRowKey(HBRecord<R> obj)
236+
```
237+
```java
238+
Put writeValueAsPut(HBRecord<R> obj)
239+
```
240+
For example, below code in Reducer writes your object as one HBase row with appropriate column families and columns:
241+
242+
```java
243+
Citizen citizen = new Citizen(/*details*/);
244+
context.write(hbObjectMapper.getRowKey(citizen), hbObjectMapper.writeValueAsPut(citizen));
245+
```
246+
### Unit-test for Mapper
247+
If your MapReduce job is reading from an HBase table, you would want to unit-test your `map()` method as below.
248+
249+
Object of your bean-like class can be converted to HBase's `Result` (for row contents) and `ImmutableBytesWritable` (for row key) using below methods:
250+
251+
```java
252+
ImmutableBytesWritable getRowKey(HBRecord<R> obj)
253+
```
254+
```java
255+
Result writeValueAsResult(HBRecord<R> obj)
256+
```
257+
Below is an example of unit-test of a Mapper using [MRUnit](https://attic.apache.org/projects/mrunit.html):
258+
259+
```java
260+
Citizen citizen = new Citizen(/*params*/);
261+
citizenMapDriver
262+
.withInput(
263+
hbObjectMapper.getRowKey(citizen),
264+
hbObjectMapper.writeValueAsResult(citizen)
265+
)
266+
.withOutput(
267+
hbObjectMapper.toIbw("key"),
268+
new IntWritable(citizen.getAge())
269+
)
270+
.runTest();
271+
```
272+
273+
### Unit-test for Reducer
274+
If your MapReduce job is writing to an HBase table, you would want to unit-test your `reduce()` method as below.
275+
276+
HBase's `Put` object can be converted to your object of you bean-like class using below method:
277+
278+
```java
279+
T readValue(ImmutableBytesWritable rowKey, Put put, Class<T> clazz)
280+
```
281+
282+
Below is an example of unit-test of a Reducer using [MRUnit](https://attic.apache.org/projects/mrunit.html):
283+
284+
```java
285+
Pair<ImmutableBytesWritable, Mutation> reducerResult = citizenReduceDriver
286+
.withInput(hbObjectMapper.toIbw("key"), inputList)
287+
.run()
288+
.get(0);
289+
CitizenSummary citizenSummary = hbObjectMapper.readValue(
290+
reducerResult.getFirst(),
291+
(Put) reducerResult.getSecond(),
292+
CitizenSummary.class
293+
);
294+
```
295+
295296
## Advantages
296297
* Your application code will be clean and minimal.
297298
* Your code need not worry about HBase methods or serialization/deserialization at all, thereby helping you maintain clear [separation of concerns](https://en.wikipedia.org/wiki/Separation_of_concerns).

0 commit comments

Comments
 (0)