Skip to content

Commit 4ee16ed

Browse files
author
caijieming
committed
issue #934: multithread compactiong support
1 parent b66c065 commit 4ee16ed

File tree

101 files changed

+4426
-4429
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

101 files changed

+4426
-4429
lines changed

README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -30,8 +30,8 @@ map<RowKey, map<ColummnFamily:Qualifier, map<Timestamp, Value> > >
3030
![架构图](resources/images/arch.png)
3131

3232
#系统依赖
33-
* 使用分布式文件系统([BFS](https://github.com/baidu/bfs)、HDFS等)持久化数据与元信息
34-
* 使用分布式协调服务([Nexus](https://github.com/baidu/ins/)或者Zookeeper)选主与协调
33+
* 使用分布式文件系统(HDFS、NFS等)持久化数据与元信息
34+
* 使用分布式协调服务([Nexus](https://github.com/baidu/ins/)或者[zookeeper](http://zookeeper.apache.org/))选主与协调
3535
* 使用[Sofa-pbrpc](https://github.com/baidu/sofa-pbrpc/)实现跨进程通信
3636

3737
#系统构建

doc/new_sdk_dev.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@
2929
* 按接口使用频率重新排序
3030
* 接口分类
3131
* 默认为已发布接口
32-
* DEVELOPING 开发中接口,鼓励完善
33-
* EXPERIMENTAL 正在测试中,鼓励试用
34-
* DEPRECATED 未实现或不建议用户使用,保证兼容性
32+
* pre-release 待发布,正在测试中,鼓励试用
33+
* developing 开发中接口,鼓励完善
34+
* do-not-use 未实现或不建议用户使用,保证兼容性
3535
* 重写接口注释

doc/sdk_dev_guide_for_python.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
# 使用tera的Python Sdk
22

33
1. 下载[TeraSdk.py](https://github.com/baidu/tera/blob/master/src/sdk/python/TeraSdk.py)
4-
2. 编译(或从其它途径获取)得到libtera_c.so;将.so与TeraSdk.py置于同一目录下,或者通过`LD_LIBRARY_PATH`环境变量等方法指定libtera_c.so的查找路径
4+
2. 编译(或从其它途径获取)得到libtera_c.so,与TeraSdk.py置于同一目录下
55
3. 编写应用程序
66
1. 示例[sample](https://github.com/baidu/tera/blob/master/src/sdk/python/sample.py)
77

doc/sdk_reference/mutation.md

Lines changed: 0 additions & 108 deletions
This file was deleted.

include/tera.h

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,5 @@
1919
#include "tera/table.h"
2020
#include "tera/table_descriptor.h"
2121
#include "tera/transaction.h"
22-
#include "tera/utils.h"
2322

2423
#endif // TERA_TERA_H_

include/tera/client.h

Lines changed: 53 additions & 56 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,10 @@
55
#ifndef TERA_CLIENT_H_
66
#define TERA_CLIENT_H_
77

8+
#include <stdint.h>
9+
#include <list>
10+
#include <map>
11+
#include <set>
812
#include <string>
913
#include <vector>
1014

@@ -17,88 +21,81 @@ namespace tera {
1721

1822
class Client {
1923
public:
20-
// Create a new client
21-
// User should delete Client* if it is no longer needed.
22-
// A Client can only be deleted if ALL the tables it opened have been deleted.
24+
/// 使用glog的用户必须调用此接口,避免glog被重复初始化
25+
static void SetGlogIsInitialized();
26+
2327
static Client* NewClient(const std::string& confpath,
2428
const std::string& log_prefix,
2529
ErrorCode* err = NULL);
30+
2631
static Client* NewClient(const std::string& confpath,
2732
ErrorCode* err = NULL);
28-
static Client* NewClient();
2933

30-
// Open a table by name.
31-
// This operation could fail due to zookeeper down, meta not avaliable, table not exists, etc.
32-
virtual Table* OpenTable(const std::string& table_name, ErrorCode* err) = 0;
34+
static Client* NewClient();
3335

34-
// Create a new table with specified descriptor.
36+
/// 创建表格
3537
virtual bool CreateTable(const TableDescriptor& desc, ErrorCode* err) = 0;
36-
// Create a new table with multiple tablets pre-assigned by tablet_delim.
3738
virtual bool CreateTable(const TableDescriptor& desc,
3839
const std::vector<std::string>& tablet_delim,
3940
ErrorCode* err) = 0;
40-
41-
// Update table schema. User should call UpdateCheck to check if the update operation is complete.
42-
virtual bool UpdateTableSchema(const TableDescriptor& desc, ErrorCode* err) = 0;
41+
/// 更新表格Schema
42+
virtual bool UpdateTable(const TableDescriptor& desc, ErrorCode* err) = 0;
4343
virtual bool UpdateCheck(const std::string& table_name, bool* done, ErrorCode* err) = 0;
44-
// Disable a table by name. A disabled table will not provide any service.
44+
/// 删除表格
45+
virtual bool DeleteTable(const std::string& name, ErrorCode* err) = 0;
46+
/// 停止表格服务
4547
virtual bool DisableTable(const std::string& name, ErrorCode* err) = 0;
46-
// Drop a table by name. Only a disabled table can be dropped.
47-
virtual bool DropTable(const std::string& name, ErrorCode* err) = 0;
48-
// Revive a disabled table.
48+
/// 恢复表格服务
4949
virtual bool EnableTable(const std::string& name, ErrorCode* err) = 0;
5050

51-
// Get the descriptor of the table
52-
virtual TableDescriptor* GetTableDescriptor(const std::string& table_name, ErrorCode* err) = 0;
53-
// List all tables.
54-
virtual bool List(std::vector<TableInfo>* table_list, ErrorCode* err) = 0;
55-
// Get table & tablet(s) info for a specified table.
56-
virtual bool List(const std::string& table_name, TableInfo* table_info,
57-
std::vector<TabletInfo>* tablet_list, ErrorCode* err) = 0;
58-
59-
// Check the table status by name.
60-
virtual bool IsTableExist(const std::string& table_name, ErrorCode* err) = 0;
61-
virtual bool IsTableEnabled(const std::string& table_name, ErrorCode* err) = 0;
62-
virtual bool IsTableEmpty(const std::string& table_name, ErrorCode* err) = 0;
63-
64-
// Send command to to server, like meta, tablet, etc.
65-
virtual bool CmdCtrl(const std::string& command, const std::vector<std::string>& arg_list,
66-
bool* bool_result, std::string* str_result, ErrorCode* err) = 0;
67-
// User who use glog besides tera should call this method to prevent conflict.
68-
static void SetGlogIsInitialized();
69-
70-
// User management.
71-
virtual bool CreateUser(const std::string& user, const std::string& password, ErrorCode* err) = 0;
51+
/// acl
52+
virtual bool CreateUser(const std::string& user,
53+
const std::string& password, ErrorCode* err) = 0;
7254
virtual bool DeleteUser(const std::string& user, ErrorCode* err) = 0;
73-
virtual bool ChangePwd(const std::string& user, const std::string& password, ErrorCode* err) = 0;
55+
virtual bool ChangePwd(const std::string& user,
56+
const std::string& password, ErrorCode* err) = 0;
7457
virtual bool ShowUser(const std::string& user, std::vector<std::string>& user_groups,
7558
ErrorCode* err) = 0;
76-
virtual bool AddUserToGroup(const std::string& user, const std::string& group, ErrorCode* err) = 0;
59+
virtual bool AddUserToGroup(const std::string& user,
60+
const std::string& group, ErrorCode* err) = 0;
7761
virtual bool DeleteUserFromGroup(const std::string& user,
7862
const std::string& group, ErrorCode* err) = 0;
63+
/// 打开表格, 失败返回NULL
64+
virtual Table* OpenTable(const std::string& table_name, ErrorCode* err) = 0;
65+
/// 获取表格分布信息
66+
virtual bool GetTabletLocation(const std::string& table_name,
67+
std::vector<TabletInfo>* tablets,
68+
ErrorCode* err) = 0;
69+
/// 获取表格Schema
70+
virtual TableDescriptor* GetTableDescriptor(const std::string& table_name,
71+
ErrorCode* err) = 0;
72+
73+
virtual bool List(std::vector<TableInfo>* table_list,
74+
ErrorCode* err) = 0;
7975

80-
// EXPERIMENTAL
76+
virtual bool List(const std::string& table_name,
77+
TableInfo* table_info,
78+
std::vector<TabletInfo>* tablet_list,
79+
ErrorCode* err) = 0;
80+
81+
virtual bool IsTableExist(const std::string& table_name, ErrorCode* err) = 0;
82+
83+
virtual bool IsTableEnabled(const std::string& table_name, ErrorCode* err) = 0;
84+
85+
virtual bool IsTableEmpty(const std::string& table_name, ErrorCode* err) = 0;
8186

82-
// Create a snapshot for the table.
8387
virtual bool GetSnapshot(const std::string& name, uint64_t* snapshot, ErrorCode* err) = 0;
84-
// Delete a specified snapshot.
85-
virtual bool DelSnapshot(const std::string& name, uint64_t snapshot, ErrorCode* err) = 0;
86-
// Perform a rollback operation to a specified snapshot
88+
virtual bool DelSnapshot(const std::string& name, uint64_t snapshot,ErrorCode* err) = 0;
8789
virtual bool Rollback(const std::string& name, uint64_t snapshot,
8890
const std::string& rollback_name, ErrorCode* err) = 0;
8991

90-
// DEPRECATED
91-
92-
// Use DropTable instead.
93-
virtual bool DeleteTable(const std::string& name, ErrorCode* err) = 0;
94-
// Use UpdateTableSchema instead.
95-
virtual bool UpdateTable(const TableDescriptor& desc, ErrorCode* err) = 0;
96-
// Use List instead.
97-
virtual bool GetTabletLocation(const std::string& table_name, std::vector<TabletInfo>* tablets,
98-
ErrorCode* err) = 0;
99-
100-
// Rename a table.
101-
virtual bool Rename(const std::string& old_table_name, const std::string& new_table_name,
92+
virtual bool CmdCtrl(const std::string& command,
93+
const std::vector<std::string>& arg_list,
94+
bool* bool_result,
95+
std::string* str_result,
96+
ErrorCode* err) = 0;
97+
virtual bool Rename(const std::string& old_table_name,
98+
const std::string& new_table_name,
10299
ErrorCode* err) = 0 ;
103100
Client() {}
104101
virtual ~Client() {}

include/tera/error_code.h

Lines changed: 22 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -1,54 +1,50 @@
11
// Copyright (c) 2015, Baidu.com, Inc. All Rights Reserved
22
// Use of this source code is governed by a BSD-style license that can be
33
// found in the LICENSE file.
4-
//
5-
// A ErrorCode encapsulates the result of an operation.
64

7-
#ifndef TERA_ERROR_CODE_H_
8-
#define TERA_ERROR_CODE_H_
5+
#ifndef TERA_ERROR_CODE_
6+
#define TERA_ERROR_CODE_
97

8+
#include <stdint.h>
9+
#include <list>
10+
#include <map>
11+
#include <set>
1012
#include <string>
13+
#include <vector>
1114

1215
#pragma GCC visibility push(default)
13-
1416
namespace tera {
1517

1618
class ErrorCode {
1719
public:
1820
enum ErrorCodeType {
19-
kOK = 0,
20-
kNotFound = 1,
21-
kBadParam = 2,
22-
kSystem = 3,
23-
kTimeout = 4,
24-
kBusy = 5,
25-
kNoQuota = 6,
26-
kNoAuth = 7,
27-
kUnknown = 8,
28-
kNotImpl = 9,
29-
kTxnFail = 10
21+
kOK = 0,
22+
kNotFound,
23+
kBadParam,
24+
kSystem,
25+
kTimeout,
26+
kBusy,
27+
kNoQuota,
28+
kNoAuth,
29+
kUnknown,
30+
kNotImpl,
31+
kTxnFail
3032
};
31-
32-
public:
33-
// Returns a string includes type&reason
34-
// Format: "type [kOK], reason [success]"
33+
ErrorCode();
3534
std::string ToString() const;
3635

37-
ErrorCodeType GetType() const;
3836
std::string GetReason() const;
39-
40-
// Internal funcion, do not use
41-
ErrorCode();
37+
ErrorCodeType GetType() const;
4238
void SetFailed(ErrorCodeType err, const std::string& reason = "");
4339

4440
private:
4541
ErrorCodeType _err;
4642
std::string _reason;
4743
};
4844

49-
// DEPRECATED. Use error_code.ToString() instead.
5045
const char* strerr(ErrorCode error_code);
5146

5247
} // namespace tera
5348
#pragma GCC visibility pop
54-
#endif // TERA_ERROR_CODE_H_
49+
50+
#endif // TERA_ERROR_CODE_

0 commit comments

Comments
 (0)