forked from chadeLin/chubaodb
-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Signed-off-by: ansjsun <[email protected]>
- Loading branch information
Showing
49 changed files
with
780 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,32 @@ | ||
name: Book | ||
|
||
on: | ||
push: | ||
branches: | ||
- release | ||
paths: | ||
- 'docs/**' | ||
|
||
jobs: | ||
deploy_en: | ||
name: Deploy book on gh-pages | ||
runs-on: ubuntu-latest | ||
steps: | ||
- name: Checkout | ||
uses: actions/checkout@v2 | ||
- name: Install mdBook | ||
uses: peaceiris/actions-mdbook@v1 | ||
- name: Render book | ||
run: | | ||
mdbook build -d gh-pages docs/zh-CN | ||
mkdir gh-pages | ||
mv docs/zh-CN/gh-pages gh-pages/zh-CN | ||
- name: Deploy | ||
uses: peaceiris/[email protected] | ||
with: | ||
emptyCommits: true | ||
keepFiles: false | ||
env: | ||
ACTIONS_DEPLOY_KEY: ${{ secrets.ACTIONS_DEPLOY_KEY }} | ||
PUBLISH_BRANCH: gh-pages | ||
PUBLISH_DIR: gh-pages |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,34 @@ | ||
name: Book | ||
|
||
on: | ||
push: | ||
branches: | ||
- release | ||
paths: | ||
- 'docs/**' | ||
|
||
jobs: | ||
deploy_en: | ||
name: Deploy book on gh-pages | ||
runs-on: ubuntu-latest | ||
steps: | ||
- name: Checkout | ||
uses: actions/checkout@v2 | ||
- name: Install mdBook | ||
uses: peaceiris/actions-mdbook@v1 | ||
- name: Render book | ||
run: | | ||
mdbook build -d gh-pages docs/zh-CN | ||
mkdir gh-pages | ||
mv docs/zh-CN/gh-pages gh-pages/zh-CN | ||
mv docs/index.html gh-pages | ||
- name: Deploy | ||
uses: peaceiris/[email protected] | ||
with: | ||
emptyCommits: true | ||
keepFiles: false | ||
env: | ||
ACTIONS_DEPLOY_KEY: ${{ secrets.ACTIONS_DEPLOY_KEY }} | ||
PUBLISH_BRANCH: gh-pages | ||
PUBLISH_DIR: gh-pages |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,39 @@ | ||
|
||
|
||
on: | ||
push: | ||
tags: | ||
- 'v*' | ||
|
||
name: release | ||
|
||
jobs: | ||
build: | ||
name: Upload Release Asset | ||
runs-on: ubuntu-latest | ||
steps: | ||
- name: Checkout code | ||
uses: actions/checkout@v2 | ||
- name: Build project # This would actually build your project, using zip for an example artifact | ||
run: | | ||
cargo build --release | ||
- name: Create Release | ||
id: create_release | ||
uses: actions/create-release@v1 | ||
env: | ||
GITHUB_TOKEN: ${{ secrets.ACTIONS_DEPLOY_KEY }} | ||
with: | ||
tag_name: ${{ github.ref }} | ||
release_name: Release ${{ github.ref }} | ||
draft: false | ||
prerelease: false | ||
- name: Upload Release Asset | ||
id: upload-release-asset | ||
uses: actions/upload-release-asset@v1 | ||
env: | ||
GITHUB_TOKEN: ${{ secrets.ACTIONS_DEPLOY_KEY }} | ||
with: | ||
upload_url: ${{ steps.create_release.outputs.upload_url }} # This pulls from the CREATE RELEASE step above, referencing it's ID to get its outputs object, which include a `upload_url`. See this blog post for more info: https://jasonet.co/posts/new-features-of-github-actions/#passing-data-to-future-steps | ||
asset_path: ./target/release/chubaodb | ||
asset_name: chubaodb-linux | ||
asset_content_type: application/zip |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,22 @@ | ||
name: Rust | ||
|
||
on: | ||
push: | ||
branches: [ master ] | ||
pull_request: | ||
branches: [ master ] | ||
|
||
env: | ||
CARGO_TERM_COLOR: always | ||
|
||
jobs: | ||
build: | ||
|
||
runs-on: ubuntu-latest | ||
|
||
steps: | ||
- uses: actions/checkout@v2 | ||
- name: Build | ||
run: cargo build --verbose | ||
- name: Run tests | ||
run: cargo test --verbose |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
<a href="zh-CN/"> 中文文档 </a> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
book |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
[book] | ||
authors = ["AnsjSun"] | ||
language = "cn" | ||
multilingual = false | ||
src = "src" | ||
title = "chubaodb 白皮书" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
# Summary | ||
|
||
- [介绍](./introduction.md) | ||
- [编译与安装](./install.md) | ||
- [配置文件](./config.md) | ||
- [集群模式](./cluster.md) | ||
- [元数据管理](./master.md) | ||
- [库表管理](./collection.md) | ||
- [数据操作](./document.md) | ||
- [CRUD](./crud.md) | ||
- [搜索](./search.md) | ||
- [聚合](./aggregation.md) | ||
- [向量](./vector.md) | ||
- [设计](./design.md) | ||
- [架构](./architecture.md) | ||
- [计划](./plan.md) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,127 @@ | ||
# 聚合那些事儿 | ||
|
||
下面我们来介绍一下chubaodb的聚合功能,在这之前,你确信已经通过[库表管理](*./collection.md*)创建了表。 | ||
|
||
我们先插入一些测试数据吧,先创建5个人 | ||
|
||
```` | ||
curl -H "Content-Type: application/json" -XPOST -d' | ||
{ | ||
"name": "张三", | ||
"age": 20, | ||
"birthday": "2000-02-02", | ||
"description": "zhangsan can use java good at php, but python not good at", | ||
"skills": ["java", "php", "python"] | ||
} | ||
' "http://127.0.0.1:8080/put/person/1" | ||
curl -H "Content-Type: application/json" -XPOST -d' | ||
{ | ||
"name": "李四", | ||
"age": 30, | ||
"birthday": "1990-02-02", | ||
"description": "lisi can use java ,only java", | ||
"skills": ["java"] | ||
} | ||
' "http://127.0.0.1:8080/put/person/2" | ||
curl -H "Content-Type: application/json" -XPOST -d' | ||
{ | ||
"name": "王五", | ||
"age": 20, | ||
"birthday": "2000-03-20", | ||
"description": "wangwu is c++ rust good at!", | ||
"skills": ["c++", "rust"] | ||
} | ||
' "http://127.0.0.1:8080/put/person/3" | ||
curl -H "Content-Type: application/json" -XPOST -d' | ||
{ | ||
"name": "牛六", | ||
"age": 35, | ||
"birthday": "1985-12-02", | ||
"description": "niuliu age too old", | ||
"skills": ["java", "golang", "python", "rust"] | ||
} | ||
' "http://127.0.0.1:8080/put/person/4" | ||
curl -H "Content-Type: application/json" -XPOST -d' | ||
{ | ||
"name": "赵七", | ||
"age": 18, | ||
"birthday": "2002-03-12", | ||
"description": "is a web user, he can use ruby and php", | ||
"skills": ["php", "ruby"] | ||
} | ||
' "http://127.0.0.1:8080/put/person/5" | ||
```` | ||
|
||
|
||
|
||
插入完成后,我们通过search接口可以查看到这五个人 `http://127.0.0.1:8080/search/person` | ||
|
||
|
||
|
||
我们先了解下聚合的真谛。 我们从一句大家都熟悉的sql 入手吧 | ||
|
||
`select age,count(age) from person group by age` | ||
|
||
这个聚合函数可以堪称两个部分。分别是 `group by ` 分组,和 `count(*)` 指标连个部分。所以聚合份两种,一个是分组,一个和统计。 | ||
|
||
ok,现在让我们看看在`chubaodb`中这两部分是如何完成的。我们通过一个例子入手 | ||
|
||
`http://127.0.0.1:8080/agg/person?group=term(age)&fun=stats(age)` | ||
|
||
* agg 是方法的操作 | ||
* group= 是聚合方法。例子中我们用的term 方式进行分组。 | ||
* fun = 是指标方法。例子中我们统计 max ,min 等 | ||
|
||
![image-20200715135803824](image/image-20200715135803824.png) | ||
|
||
|
||
|
||
好的恭喜你已经看到结果了。默认是按照value中的count 进行排序 | ||
|
||
|
||
|
||
### 目前group 支持的方法有 | ||
|
||
`term(name)` 按照字段进行聚合 | ||
|
||
`date(name,format)` example: date(birthday, yyyy-MM) 按照时间格式进行聚合 | ||
|
||
`range(name, range_str)` example:*range(age,-0,0-20,20-30,30-40,40-)* | ||
|
||
|
||
|
||
### Fun 支持的方法有 | ||
|
||
`hits(size)`:每个分组中返回size个文档,无排序功能 | ||
|
||
`stats(name)`: 每个分组中,count max, min , sum missing 的个数 | ||
|
||
* fun 可以为空,如果为空则默认为count方式 | ||
|
||
|
||
|
||
现在让我们自举一些例子: | ||
|
||
### 按照年龄分组,分别显示每个年龄中的人 | ||
|
||
![image-20200715140655966](image/image-20200715140655966.png) | ||
|
||
### 照技能分组,显示每个技能的人数 | ||
|
||
`http://127.0.0.1:8080/agg/person?group=term(skills)` | ||
|
||
![image-20200715140812082](image/image-20200715140812082.png) | ||
|
||
### 按照技能分组,显示每个技能的人数,按照技能名称排序 | ||
|
||
`http://127.0.0.1:8080/agg/person?group=term(skills)&sort=key:asc` | ||
|
||
![image-20200715140902639](image/image-20200715140902639.png) | ||
|
||
### 按照年龄和技能分组查看人数 | ||
|
||
![image-20200715141018443](image/image-20200715141018443.png) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
# 关于架构 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
# 集群模式 | ||
|
||
编造ing...... |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,60 @@ | ||
# 库表管理 | ||
|
||
下面我们会对`创建表`,`查询表`,`删除表` 进行演示。目前表结构一旦创建不支持修改。 | ||
打开master的管理地址 http://127.0.0.1:7070 | ||
|
||
### 创建表 | ||
|
||
通过 `collectionCreate`接口进行创建 | ||
|
||
![image-20200715112907536](image/image-20200715112907536.png) | ||
|
||
* Name 是表名称, | ||
* partitionNum 是这个表分多少个分片。分片多会提高插入的并发能力,但是会降低搜索效率,并非越多或者越少越好 | ||
* partitionReplicaNum 是每个分片多少个副本。建议要么1,要么3+ 。在传统分布式系统环境,可以设置为3,单机版智能设置1.partitionReplicaNum 必须小于等于你的机器个数 | ||
* Fields 是这个表里面的字段。我们提供了 `int`, `float`, `string`, `text`, `vector`, `date` 几种字段格式,注意 text 和string的区别是。text是全文检索,比如 `中国银行` 搜索`中国`是会被召回的, `string`的话必须输入完整的 匹配。 | ||
|
||
|
||
|
||
下面我们创建一个人物表,包含 `ID`, `姓名`, `年龄` ,`生日` , `简介` ,`技能` 几个字段, | ||
|
||
```` | ||
mutation{ | ||
collectionCreate( | ||
name:"person", | ||
partitionNum:1, | ||
partitionReplicaNum:1 | ||
fields:{ | ||
int:[{name:"age", value:true }] | ||
string:[{name:"name" }, {name:"skills", value:true, array:true}] | ||
date:[{name:"birthday", value:true }] | ||
text:[{name:"description"}] | ||
} | ||
) | ||
} | ||
```` | ||
|
||
![image-20200715115437856](image/image-20200715115437856.png) | ||
|
||
|
||
|
||
出现如下结构意味着创建表成功了。每种类型有自己的参数大家可以参阅iql的文档。 | ||
|
||
|
||
|
||
### 查询表 | ||
|
||
![image-20200715115617185](image/image-20200715115617185.png) | ||
|
||
|
||
|
||
|
||
|
||
删除表 | ||
|
||
![image-20200715115703796](image/image-20200715115703796.png) | ||
|
||
|
||
|
||
ok 你已经具备了元数据管理的基本技能。 |
Oops, something went wrong.