Skip to content

Commit f0c6d0d

Browse files
committed
tests: Remove MacVim-specific flaky tests guard
A while ago, flaky Vim tests would routinely fail in MacVim's repo and an intermediate hack was added to essentially pass all flaky tests and these failed tests would be added to a "pending" list for review. The issue is that this essentially gives a free pass for tests to fail and no one really has time to look at these "pending" tests when they see a green checkmark. It's much better to fail early and fast so we are forced to address them as they come up. Waiting till later tends to make these test failures harder to isolate and diagnose, leading to us just ignoring them. Even worse, flaky tests are retried multiple times, so they end up making MacVim CI run longer. Simply revert the changes that added these flaky tests and we should fix broken tests as they come up just like any other non-flaky tests.
1 parent 9c0d779 commit f0c6d0d

1 file changed

Lines changed: 1 addition & 19 deletions

File tree

src/testdir/runtest.vim

Lines changed: 1 addition & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -457,15 +457,6 @@ endfunc
457457

458458
func AfterTheTest(func_name)
459459
if len(v:errors) > 0
460-
if has('gui_macvim') && g:test_is_flaky
461-
" MacVim's currently doesn't always pass these tests. Make these
462-
" tests pending for now before a more proper fix is implemented.
463-
call add(s:messages, 'MacVim marked ' .. a:func_name .. 'as pending')
464-
let s:pending += 1
465-
call add(s:errors_pending, 'Found errors in ' . g:testfunc . ':')
466-
call extend(s:errors_pending, v:errors)
467-
call add(s:errors_pending, 'PENDING ' .. a:func_name)
468-
else
469460
if match(s:may_fail_list, '^' .. a:func_name) >= 0
470461
let s:fail_expected += 1
471462
call add(s:errors_expected, 'Found errors in ' . g:testfunc . ':')
@@ -475,7 +466,6 @@ func AfterTheTest(func_name)
475466
call add(s:errors, 'Found errors in ' . g:testfunc . ':')
476467
call extend(s:errors, v:errors)
477468
endif
478-
endif
479469
let v:errors = []
480470
endif
481471
endfunc
@@ -498,7 +488,7 @@ func FinishTesting()
498488
" Don't write viminfo on exit.
499489
set viminfo=
500490

501-
if s:fail == 0 && s:fail_expected == 0 && s:pending == 0
491+
if s:fail == 0 && s:fail_expected == 0
502492
" Success, create the .res file so that make knows it's done.
503493
exe 'split ' . fnamemodify(g:testname, ':r') . '.res'
504494
write
@@ -548,12 +538,6 @@ func FinishTesting()
548538
call add(s:messages, message)
549539
call extend(s:messages, s:errors_expected)
550540
endif
551-
if s:pending > 0
552-
let message = s:pending . ' FAILED (pending):'
553-
echo message
554-
call add(s:messages, message)
555-
call extend(s:messages, s:errors_pending)
556-
endif
557541

558542
" Add SKIPPED messages
559543
call extend(s:messages, s:skipped)
@@ -578,8 +562,6 @@ let s:errors = []
578562
let s:errors_expected = []
579563
let s:messages = []
580564
let s:skipped = []
581-
let s:pending = 0
582-
let s:errors_pending = []
583565
if expand('%') =~ 'test_vimscript.vim'
584566
" this test has intentional errors, don't use try/catch.
585567
source %

0 commit comments

Comments
 (0)