Skip to content

Implement LOAD_FILE function#1052

Open
ouyiz wants to merge 5 commits into
oceanbase:vldb_2026from
ouyiz:doc-ai-functions
Open

Implement LOAD_FILE function#1052
ouyiz wants to merge 5 commits into
oceanbase:vldb_2026from
ouyiz:doc-ai-functions

Conversation

@ouyiz

@ouyiz ouyiz commented Jul 14, 2026

Copy link
Copy Markdown

Task Description

Implement LOAD_FILE(location, filename).

Solution Description

Add a new SQL scalar function load_file that resolves a LOCATION, reads a local file:// file, and returns the file content as a binary string/BLOB.

Passed Regressions

Local macOS build is blocked by toolchain compatibility. Submitted to GitHub Action for official validation.

@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


冯轩 seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

@LINxiansheng

LINxiansheng commented Jul 15, 2026

Copy link
Copy Markdown
Member

Document AI & IK Custom Dictionary Score

Document AI Functions Score
===========================
score: 100.00 / 100
load_file: 50 / 50
ai_split_document: 50 / 50

IK Custom Dictionary Score
==========================
score: 100.00 / 100
ik_custom_dict: 100 / 100

FTS Large Benchmark Score

FTS Large Benchmark Score
=========================
score: 0.00 / 100
mean_improvement: -74.07%
full_score_improvement: 50.00%

build_improvement: -9.94%
  build_ik_all_sec: baseline=35.2836, current=40.276, improvement=-14.15%
  build_ik_content_sec: baseline=28.3764, current=33.31, improvement=-17.39%
  build_beng_en_sec: baseline=14.7578, current=14.504, improvement=1.72%
tokenize_improvement: 6.32%
  tokenize_ik_avg_ms: baseline=0.76478, current=0.6945, improvement=9.19%
  tokenize_beng_avg_ms: baseline=0.42262, current=0.408, improvement=3.46%
query_improvement: -218.59%
  query_cn_avg_ms: baseline=16.6628, current=130.518, improvement=-683.29%
  query_beng_avg_ms: baseline=24.3042, current=24.0929, improvement=0.87%
  query_mixed_avg_ms: baseline=17.5593, current=31.3596, improvement=-78.59%
  query_limit_avg_ms: baseline=16.2334, current=34.6347, improvement=-113.35%

FTS Large Benchmark Report

========================================
FTS Large Benchmark Report
========================================
timestamp:              2026-07-15 11:13:06
label:                  vldb-ci-29409571705-1
git_head:               60dbd40
git_dirty:              0
rows:                   20000
batch:                  500
rounds:                 3000
query_rounds:           200
samples:                3
warmup:                 30
skip_load:              0
----------------------------------------
select1_avg_ms:         0.2036
select1_stdev_ms:       0.0012
raw_load_sec:           1.465
raw_load_rows_per_sec:  13651.9
build_ik_all_sec:       40.276
build_ik_content_sec:   33.310
build_beng_en_sec:      14.504
build_total_sec:        88.103
----------------------------------------
tokenize_ik_avg_ms:     0.6945
tokenize_ik_median_ms:  0.6954
tokenize_ik_stdev_ms:   0.0012
tokenize_beng_avg_ms:   0.4080
tokenize_beng_median_ms:0.4084
tokenize_beng_stdev_ms: 0.0009
----------------------------------------
query_cn_hits:          20000
query_cn_avg_ms:        130.5182
query_cn_stdev_ms:      0.0514
query_beng_hits:        11000
query_beng_avg_ms:      24.0929
query_beng_stdev_ms:    0.0063
query_mixed_hits:       9998
query_mixed_avg_ms:     31.3596
query_mixed_stdev_ms:   0.0127
query_limit_hits:       20
query_limit_avg_ms:     34.6347
query_limit_stdev_ms:   0.0241
========================================

Workflow run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants