Skip to content

Feat/q3 ik custom dict lzh#1053

Open
zhanhongliu2001 wants to merge 136 commits into
oceanbase:vldb_2026from
zhanhongliu2001:feat/q3-ik-custom-dict-lzh
Open

Feat/q3 ik custom dict lzh#1053
zhanhongliu2001 wants to merge 136 commits into
oceanbase:vldb_2026from
zhanhongliu2001:feat/q3-ik-custom-dict-lzh

Conversation

@zhanhongliu2001

Copy link
Copy Markdown

Task Description

Solution Description

Passed Regressions

Upgrade Compatibility

Other Information

Release Note

wangzelin.wzl and others added 30 commits November 4, 2025 19:10
[TABLELOCK] fix the bug of mysql lock table in observer-lite
[MV.BUGFIX] fail to drop index for mv table with renamed index
fix ivf async task generate_aux_table_info_map when schema_version not change
Optimize the bootstrap time of the observer
Fixed oceanbase#21, [Enhancement]: remove old python sdk demo in the Readme
Fixed oceanbase#24, [Enhancement]: Refine Key Features in README.md
Fixed oceanbase#27, [Bug]: update build and run observer part in README.md
refine product comparison table
Added full-text and vector indexes to the articles table for improved search capabilities.
Added full-text and vector indexing to articles table for improved search capabilities.
Signed-off-by: SamYuan1990 <[email protected]>
hnwyllmm and others added 27 commits June 16, 2026 14:03
Conflicts resolved by accepting master (theirs) version.
# Conflicts:
#	src/share/ob_errno.cpp
#	src/share/ob_errno.h
#	tools/ob_error/src/ob_error.cpp
…nbase#1022)

Read ai_funcs_score.txt out of the comment artifact and emit it as a
"Document AI & IK Custom Dictionary Score" section above the FTS sections.

Each section is now emitted only when its input file exists, and the job no
longer skips the comment when the FTS benchmark produced no report but the
ai_funcs score is present.

This workflow is triggered by workflow_run, so GitHub always runs the copy on
the default branch. The change has to land here for PRs targeting vldb_2026 to
show the scores.
@CLAassistant

CLAassistant commented Jul 14, 2026

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 3 committers have signed the CLA.

✅ cao1629
❌ wangzelin.wzl
❌ zhanhongliu2001


wangzelin.wzl seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

@LINxiansheng

Copy link
Copy Markdown
Member

Document AI & IK Custom Dictionary Score

Document AI Functions Score
===========================
score: 0.00 / 100
load_file: 0 / 50
ai_split_document: 0 / 50

IK Custom Dictionary Score
==========================
score: 100.00 / 100
ik_custom_dict: 100 / 100

FTS Large Benchmark Score

FTS Large Benchmark Score
=========================
score: 1.35 / 100
mean_improvement: 0.68%
full_score_improvement: 50.00%

build_improvement: -0.18%
  build_ik_all_sec: baseline=35.2836, current=34.545, improvement=2.09%
  build_ik_content_sec: baseline=28.3764, current=28.232, improvement=0.51%
  build_beng_en_sec: baseline=14.7578, current=15.222, improvement=-3.15%
tokenize_improvement: 0.94%
  tokenize_ik_avg_ms: baseline=0.76478, current=0.7705, improvement=-0.75%
  tokenize_beng_avg_ms: baseline=0.42262, current=0.4115, improvement=2.63%
query_improvement: 1.27%
  query_cn_avg_ms: baseline=16.6628, current=16.442, improvement=1.33%
  query_beng_avg_ms: baseline=24.3042, current=23.9165, improvement=1.60%
  query_mixed_avg_ms: baseline=17.5593, current=17.2837, improvement=1.57%
  query_limit_avg_ms: baseline=16.2334, current=16.138, improvement=0.59%

FTS Large Benchmark Report

========================================
FTS Large Benchmark Report
========================================
timestamp:              2026-07-14 07:32:27
label:                  vldb-ci-29312826514-1
git_head:               52ac060
git_dirty:              0
rows:                   20000
batch:                  500
rounds:                 3000
query_rounds:           200
samples:                3
warmup:                 30
skip_load:              0
----------------------------------------
select1_avg_ms:         0.2017
select1_stdev_ms:       0.0006
raw_load_sec:           1.474
raw_load_rows_per_sec:  13568.5
build_ik_all_sec:       34.545
build_ik_content_sec:   28.232
build_beng_en_sec:      15.222
build_total_sec:        78.010
----------------------------------------
tokenize_ik_avg_ms:     0.7705
tokenize_ik_median_ms:  0.7703
tokenize_ik_stdev_ms:   0.0011
tokenize_beng_avg_ms:   0.4115
tokenize_beng_median_ms:0.4110
tokenize_beng_stdev_ms: 0.0008
----------------------------------------
query_cn_hits:          8001
query_cn_avg_ms:        16.4420
query_cn_stdev_ms:      0.0324
query_beng_hits:        11000
query_beng_avg_ms:      23.9165
query_beng_stdev_ms:    0.0252
query_mixed_hits:       7332
query_mixed_avg_ms:     17.2837
query_mixed_stdev_ms:   0.0076
query_limit_hits:       20
query_limit_avg_ms:     16.1380
query_limit_stdev_ms:   0.0016
========================================

Workflow run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.