patch 9.1.1627: fuzzy matching can be improved

Problem:  fuzzy-matching can be improved
Solution: Implement a better fuzzy matching algorithm
          (Girish Palya)

Replace fuzzy matching algorithm with improved fzy-based implementation

The
[current](https://www.forrestthewoods.com/blog/reverse_engineering_sublime_texts_fuzzy_match/)
fuzzy matching algorithm has several accuracy issues:

* It struggles with CamelCase
* It fails to prioritize matches at the beginning of strings, often
  ranking middle matches higher.

After evaluating alternatives (see my comments
[here](vim/vim#17531 (comment))
and
[here](vim/vim#17531 (comment))),
I chose to adopt the [fzy](https://github.com/jhawthorn/fzy) algorithm,
which:

* Resolves the aforementioned issues.
* Performs better.

Implementation details

This version is based on the original fzy
[algorithm](https://github.com/jhawthorn/fzy/blob/master/src/match.c),
with one key enhancement: **multibyte character support**.

* The original implementation supports only ASCII.
* This patch replaces ascii lookup tables with function calls, making it
  compatible with multibyte character sets.
* Core logic (`match_row()` and `match_positions()`) remains faithful to
  the original, but now operates on codepoints rather than single-byte
  characters.

Performance

Tested against a dataset of **90,000 Linux kernel filenames**. Results
(in milliseconds) show a **\~2x performance improvement** over the
current fuzzy matching algorithm.

```
Search String            Current Algo    FZY Algo
-------------------------------------------------
init                          131.759    66.916
main                          83.688     40.861
sig                           98.348     39.699
index                         109.222    30.738
ab                            72.222     44.357
cd                            83.036     54.739
a                             58.94      62.242
b                             43.612     43.442
c                             64.39      67.442
k                             40.585     36.371
z                             34.708     22.781
w                             38.033     30.109
cpa                           82.596     38.116
arz                           84.251     23.964
zzzz                          35.823     22.75
dimag                         110.686    29.646
xa                            43.188     29.199
nha                           73.953     31.001
nedax                         94.775     29.568
dbue                          79.846     25.902
fp                            46.826     31.641
tr                            90.951     55.883
kw                            38.875     23.194
rp                            101.575    55.775
kkkkkkkkkkkkkkkkkkkkkkkkkkkkk 48.519     30.921
```

```vim
vim9script

var haystack = readfile('/Users/gp/linux.files')

var needles = ['init', 'main', 'sig', 'index', 'ab', 'cd', 'a', 'b',
'c', 'k',
    'z', 'w', 'cpa', 'arz', 'zzzz', 'dimag', 'xa', 'nha', 'nedax',
'dbue',
    'fp', 'tr', 'kw', 'rp', 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkk']
for needle in needles
    var start = reltime()
    var tmp = matchfuzzy(haystack, needle)
    echom $'{needle}' (start->reltime()->reltimefloat() * 1000)
endfor
```

Additional changes

* Removed the "camelcase" option from both matchfuzzy() and
  matchfuzzypos(), as it's now obsolete with the improved algorithm.

related: neovim/neovim#34101
fixes #17531
closes: #17900

Signed-off-by: Girish Palya <[email protected]>
Signed-off-by: Christian Brabandt <[email protected]>

Author: girishji

Filelist

@@ -82,6 +82,7 @@ SRC_ALL =	\
 		src/findfile.c \
 		src/float.c \
 		src/fold.c \
+		src/fuzzy.c \
 		src/getchar.c \
 		src/gc.c \
 		src/globals.h \
@@ -291,6 +292,7 @@ SRC_ALL =	\
 		src/proto/findfile.pro \
 		src/proto/float.pro \
 		src/proto/fold.pro \
+		src/proto/fuzzy.pro \
 		src/proto/getchar.pro \
 		src/proto/gc.pro \
 		src/proto/gui.pro \

runtime/doc/builtin.txt

@@ -1,4 +1,4 @@
-*builtin.txt*	For Vim version 9.1.  Last change: 2025 Aug 10
+*builtin.txt*	For Vim version 9.1.  Last change: 2025 Aug 12
 
 
 		  VIM REFERENCE MANUAL	  by Bram Moolenaar
@@ -7421,9 +7421,6 @@ matchfuzzy({list}, {str} [, {dict}])			*matchfuzzy()*
 				given sequence.
 		    limit	Maximum number of matches in {list} to be
 				returned.  Zero means no limit.
-		    camelcase	Use enhanced camel case scoring making results
-				better suited for completion related to
-				programming languages.  Defaults to v:true.
 
 		If {list} is a list of dictionaries, then the optional {dict}
 		argument supports the following additional items:

runtime/doc/pattern.txt

@@ -1,4 +1,4 @@
-*pattern.txt*   For Vim version 9.1.  Last change: 2025 Aug 06
+*pattern.txt*   For Vim version 9.1.  Last change: 2025 Aug 12
 
 
 		  VIM REFERENCE MANUAL    by Bram Moolenaar
@@ -1509,6 +1509,9 @@ characters in the search string.  If the search string has multiple words, then
 each word is matched separately.  So the words in the search string can be
 present in any order in a string.
 
+Vim uses the same improved algorithm as the fzy project:
+https://github.com/jhawthorn/fzy
+
 Fuzzy matching assigns a score for each matched string based on the following
 criteria:
     - The number of sequentially matching characters.

runtime/doc/version9.txt

@@ -41723,6 +41723,8 @@ Functions: ~
 - Add the optional {opts} |Dict| argument to |getchar()| to control: cursor
   behaviour, return type and whether or not to simplify the returned key
 - |chdir()| allows to optionally specify a scope argument
+- |matchfuzzy()| and |matchfuzzypos()| use an improved fuzzy matching
+  algorithm (same as fzy).
 
 Others: ~
 - the regex engines match correctly case-insensitive multi-byte characters

src/Make_ami.mak

@@ -113,6 +113,7 @@ SRC += \
 	findfile.c \
 	float.c \
 	fold.c \
+	fuzzy.c \
 	getchar.c \
 	gc.c \
 	hardcopy.c \

src/Make_cyg_ming.mak

@@ -823,6 +823,7 @@ OBJ = \
 	$(OUTDIR)/findfile.o \
 	$(OUTDIR)/float.o \
 	$(OUTDIR)/fold.o \
+	$(OUTDIR)/fuzzy.o \
 	$(OUTDIR)/getchar.o \
 	$(OUTDIR)/gc.o \
 	$(OUTDIR)/gui_xim.o \

src/Make_mvc.mak

@@ -732,6 +732,7 @@ OBJ = \
 	$(OUTDIR)\findfile.obj \
 	$(OUTDIR)\float.obj \
 	$(OUTDIR)\fold.obj \
+	$(OUTDIR)\fuzzy.obj \
 	$(OUTDIR)\getchar.obj \
 	$(OUTDIR)\gc.obj \
 	$(OUTDIR)\gui_xim.obj \
@@ -1616,6 +1617,8 @@ $(OUTDIR)/float.obj:	$(OUTDIR) float.c  $(INCL)
 
 $(OUTDIR)/fold.obj:	$(OUTDIR) fold.c  $(INCL)
 
+$(OUTDIR)/fuzzy.obj:	$(OUTDIR) fuzzy.c  $(INCL)
+
 $(OUTDIR)/getchar.obj:	$(OUTDIR) getchar.c  $(INCL)
 
 $(OUTDIR)/gc.obj:	$(OUTDIR) gc.c  $(INCL)
@@ -1961,6 +1964,7 @@ proto.h: \
 	proto/filepath.pro \
 	proto/findfile.pro \
 	proto/float.pro \
+	proto/fuzzy.pro \
 	proto/getchar.pro \
 	proto/gc.pro \
 	proto/gui_xim.pro \

src/Make_vms.mms

@@ -529,6 +529,7 @@ SRC = \
  findfile.c \
  float.c \
  fold.c \
+ fuzzy.c \
  getchar.c \
  gc.c \
  gui_xim.c \
@@ -665,6 +666,7 @@ OBJ = \
  [.$(DEST)]findfile.obj \
  [.$(DEST)]float.obj \
  [.$(DEST)]fold.obj \
+ [.$(DEST)]fuzzy.obj \
  [.$(DEST)]getchar.obj \
  [.$(DEST)]gc.obj \
  [.$(DEST)]gui_xim.obj \
@@ -1141,6 +1143,10 @@ lua_env :
 [.$(DEST)]fold.obj : fold.c vim.h [.$(DEST)]config.h feature.h os_unix.h   \
  ascii.h keymap.h termdefs.h macros.h structs.h regexp.h gui.h beval.h \
  [.proto]gui_beval.pro option.h ex_cmds.h proto.h errors.h globals.h
+[.$(DEST)]fuzzy.obj : fuzzy.c vim.h [.$(DEST)]config.h feature.h os_unix.h \
+ ascii.h keymap.h termdefs.h macros.h structs.h regexp.h \
+ gui.h beval.h [.proto]gui_beval.pro option.h ex_cmds.h proto.h \
+ errors.h globals.h
 [.$(DEST)]getchar.obj : getchar.c vim.h [.$(DEST)]config.h feature.h os_unix.h \
  ascii.h keymap.h termdefs.h macros.h structs.h regexp.h \
  gui.h beval.h [.proto]gui_beval.pro option.h ex_cmds.h proto.h \

src/Makefile

@@ -1523,6 +1523,7 @@ BASIC_SRC = \
 	findfile.c \
 	float.c \
 	fold.c \
+	fuzzy.c \
 	getchar.c \
 	gc.c \
 	gui_xim.c \
@@ -1701,6 +1702,7 @@ OBJ_COMMON = \
 	objects/findfile.o \
 	objects/float.o \
 	objects/fold.o \
+	objects/fuzzy.o \
 	objects/getchar.o \
 	objects/gc.o \
 	objects/gui_xim.o \
@@ -1886,6 +1888,7 @@ PRO_AUTO = \
 	findfile.pro \
 	float.pro \
 	fold.pro \
+	fuzzy.pro \
 	getchar.pro \
 	gc.pro \
 	gui_xim.pro \
@@ -3309,6 +3312,9 @@ objects/float.o: float.c
 objects/fold.o: fold.c
 	$(CCC) -o $@ fold.c
 
+objects/fuzzy.o: fuzzy.c
+	$(CCC) -o $@ fuzzy.c
+
 objects/getchar.o: getchar.c
 	$(CCC) -o $@ getchar.c
 
@@ -3988,6 +3994,11 @@ objects/fold.o: fold.c vim.h protodef.h auto/config.h feature.h os_unix.h \
  proto/gui_beval.pro structs.h regexp.h gui.h libvterm/include/vterm.h \
  libvterm/include/vterm_keycodes.h alloc.h ex_cmds.h spell.h proto.h \
  globals.h errors.h
+objects/fuzzy.o: fuzzy.c vim.h protodef.h auto/config.h feature.h os_unix.h \
+ auto/osdef.h ascii.h keymap.h termdefs.h macros.h option.h beval.h \
+ proto/gui_beval.pro structs.h regexp.h gui.h libvterm/include/vterm.h \
+ libvterm/include/vterm_keycodes.h alloc.h ex_cmds.h spell.h proto.h \
+ globals.h errors.h
 objects/getchar.o: getchar.c vim.h protodef.h auto/config.h feature.h os_unix.h \
  auto/osdef.h ascii.h keymap.h termdefs.h macros.h option.h beval.h \
  proto/gui_beval.pro structs.h regexp.h gui.h libvterm/include/vterm.h \

src/README.md

@@ -48,6 +48,7 @@ fileio.c	| reading and writing files
 filepath.c	| dealing with file names and paths
 findfile.c	| search for files in 'path'
 fold.c		| folding
+fuzzy.c	    | fuzzy matching
 getchar.c	| getting characters and key mapping
 gc.c	    | garbage collection
 help.c		| vim help related functions