Feat/index

[XiangyunDu]

========================================
FTS Large Benchmark Report

timestamp: 2026-07-15 11:02:07
label: after_45246
git_head: a52e956
git_dirty: 1
rows: 20000
batch: 500
rounds: 3000
query_rounds: 200
samples: 3
warmup: 30
skip_load: 0

select1_avg_ms: 0.1819
select1_stdev_ms: 0.0004
raw_load_sec: 1.325
raw_load_rows_per_sec: 15094.3
build_ik_all_sec: 29.548
build_ik_content_sec: 23.600
build_beng_en_sec: 12.261
build_total_sec: 65.420

tokenize_ik_avg_ms: 0.6502
tokenize_ik_median_ms: 0.6498
tokenize_ik_stdev_ms: 0.0047
tokenize_beng_avg_ms: 0.3569
tokenize_beng_median_ms:0.3563
tokenize_beng_stdev_ms: 0.0020

query_cn_hits: 8001
query_cn_avg_ms: 13.7243
query_cn_stdev_ms: 0.0059
query_beng_hits: 11000
query_beng_avg_ms: 19.9312
query_beng_stdev_ms: 0.0149
query_mixed_hits: 7332
query_mixed_avg_ms: 14.4125
query_mixed_stdev_ms: 0.0073
query_limit_hits: 20
query_limit_avg_ms: 13.6504
query_limit_stdev_ms: 0.0150

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ XiangyunDu
❌ likayan
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[LINxiansheng]

Document AI & IK Custom Dictionary Score

Document AI Functions Score
===========================
score: 0.00 / 100
load_file: 0 / 50
ai_split_document: 0 / 50

IK Custom Dictionary Score
==========================
score: 100.00 / 100
ik_custom_dict: 100 / 100

FTS Large Benchmark Score

FTS Large Benchmark Score
=========================
score: 17.29 / 100
mean_improvement: 8.65%
full_score_improvement: 50.00%

build_improvement: 24.27%
  build_ik_all_sec: baseline=35.2836, current=27.116, improvement=23.15%
  build_ik_content_sec: baseline=28.3764, current=21.829, improvement=23.07%
  build_beng_en_sec: baseline=14.7578, current=10.835, improvement=26.58%
tokenize_improvement: 1.40%
  tokenize_ik_avg_ms: baseline=0.76478, current=0.7566, improvement=1.07%
  tokenize_beng_avg_ms: baseline=0.42262, current=0.4153, improvement=1.73%
query_improvement: 0.27%
  query_cn_avg_ms: baseline=16.6628, current=16.6451, improvement=0.11%
  query_beng_avg_ms: baseline=24.3042, current=24.2766, improvement=0.11%
  query_mixed_avg_ms: baseline=17.5593, current=17.4999, improvement=0.34%
  query_limit_avg_ms: baseline=16.2334, current=16.1477, improvement=0.53%

FTS Large Benchmark Report

========================================
FTS Large Benchmark Report
========================================
timestamp:              2026-07-15 15:46:07
label:                  vldb-ci-29427580984-1
git_head:               496dcf0
git_dirty:              0
rows:                   20000
batch:                  500
rounds:                 3000
query_rounds:           200
samples:                3
warmup:                 30
skip_load:              0
----------------------------------------
select1_avg_ms:         0.2273
select1_stdev_ms:       0.0166
raw_load_sec:           1.489
raw_load_rows_per_sec:  13431.8
build_ik_all_sec:       27.116
build_ik_content_sec:   21.829
build_beng_en_sec:      10.835
build_total_sec:        59.793
----------------------------------------
tokenize_ik_avg_ms:     0.7566
tokenize_ik_median_ms:  0.7554
tokenize_ik_stdev_ms:   0.0019
tokenize_beng_avg_ms:   0.4153
tokenize_beng_median_ms:0.4149
tokenize_beng_stdev_ms: 0.0011
----------------------------------------
query_cn_hits:          8001
query_cn_avg_ms:        16.6451
query_cn_stdev_ms:      0.0144
query_beng_hits:        11000
query_beng_avg_ms:      24.2766
query_beng_stdev_ms:    0.0016
query_mixed_hits:       7332
query_mixed_avg_ms:     17.4999
query_mixed_stdev_ms:   0.0054
query_limit_hits:       20
query_limit_avg_ms:     16.1477
query_limit_stdev_ms:   0.0212
========================================

Workflow run

[XiangyunDu]

========================================
FTS Benchmark Comparison
========================================
baseline: fts_large_bench_after_45246.txt
candidate:fts_large_bench_after_sort_fts_20260715_newsort.txt
----------------------------------------
select1_avg_ms: base=0.1819 new=0.1833 worse -0.77%
raw_load_sec: base=1.3250 new=1.3440 worse -1.43%
raw_load_rows_per_sec: base=15094.3000 new=14881.0000 worse -1.41%
build_ik_all_sec: base=29.5480 new=23.4850 better +20.52%
build_ik_content_sec: base=23.6000 new=19.0170 better +19.42%
build_beng_en_sec: base=12.2610 new=9.3270 better +23.93%
build_total_sec: base=65.4200 new=51.8400 better +20.76%
tokenize_ik_avg_ms: base=0.6502 new=0.6512 worse -0.15%
tokenize_beng_avg_ms: base=0.3569 new=0.3593 worse -0.67%
query_cn_avg_ms: base=13.7243 new=13.9256 worse -1.47%
query_beng_avg_ms: base=19.9312 new=20.1272 worse -0.98%
query_mixed_avg_ms: base=14.4125 new=14.5584 worse -1.01%
query_limit_avg_ms: base=13.6504 new=13.7054 worse -0.40%
----------------------------------------
summary:
build_total_sec delta=+20.76%
build_ik_all_sec delta=+20.52%
build_ik_content_sec delta=+19.42%
build_beng_en_sec delta=+23.93%