Remove fade-to-black animation for native/custom fullscreen

For native fullscreen, simply remove the custom animation as it doesn't
look good and doesn't look native. This was added back when resizing the
window rapidly would result in artifacts, but they should have all been
fixed. As such, no need to hide everything under a black fade.

For custom fullscreen, make it so that if `MMFullScreenFadeTime` is set
to 0 (which is now the default), the fading animation will simply not be
drawn as it simply flickers. Since it now defaults to 0, the user will
have to manually set this back to 0.25 to get the old functionality,
which is ok as it's somewhat a legacy feature anyway and there shouldn't
be any flickering now during the transition.

Also, update documentation on full-screen to provide more info on native
vs custom, and clean up old outdated texts.

Misc driveby fixes related to full screen:
- Robustify the Touch Bar full screen button to listen to `OptionSet`
  event which is more robust than relying on `VimResized`, which also
  didn't work when 'fuoptions' is set to nothing.
- Make MacVim handle the DidFailToEnter/ExitFullScreen events better.
  These events could fire when using native full screen while the user
  is e.g. switching space. We need to make sure to properly update the
  states when these two events happen and let Vim know what the current
  `'fullscreen'` state is so both sides are synced.
- Fix up the preferences pane's full screen buttons to not trigger
  fontPropertiesChanged which was a mistake due to copy-and-paste in
  Interface Builder.

See:
- #292 / #289 which implemented the fade effect.

Author: ychin

runtime/doc/gui_mac.txt

@@ -314,6 +314,8 @@ this behaviour set MMLoginShellArgument to "--".
 ==============================================================================
 4. MacVim appearance					*macvim-appearance*
 
+MacVim can be used in full screen mode, see 'fullscreen'.
+
 				*macvim-appearance-mode* *macvim-dark-mode*
 MacVim will by default use the system apperance mode (light or dark). However,
 you can manually force MacVim to use either light or dark mode in the

runtime/doc/options.txt

@@ -3580,27 +3580,30 @@ A jump table for the options with a short description can be found at |Q_op|.
 	(e.g. toolbar, title bar).  The tab bar and scroll bars remain visible.
 	Updates to the window position are ignored in fullscreen mode.
 
+	By default, this will use macOS's native full screen feature, which
+	will put the MacVim window into another Space under Mission Control.
+	MacVim also provides a custom full screen solution which you can
+	select by setting |MMNativeFullScreen| to NO, or toggle it under
+	Preferences → Appearance. Under custom full-screen, the window will
+	not be put in another space, which makes it easier to Cmd-Tab to other
+	windows.
+
+	Custom / non-native full screen configuration:~
+
 	See 'fuoptions' for how Vim resizes and colors the background when 
 	entering and leaving fullscreen mode.
 
-	You can use the hidden preference MMFullScreenFadeTime to adjust how
-	long the animation takes to fade in and out.  The default is 0.25
-	seconds.  See |macvim-preferences|for how to set hidden preferences.
-
-	Note: Setting 'fullscreen' usually changes the size of the Vim
-	control. However, for technical reasons, 'lines' and 'columns' will
-	currently only be updated when Vim runs its event loop. As a
-	consequence, if you set 'fullscreen' and 'lines' or 'columns' in a
-	Vim script file, you should always set 'fullscreen' after setting
-	'lines' and 'columns', else 'lines' and 'columns' will be overwritten
-	with the values 'fullscreen' sets after the script has been executed
-	and the event loop is ran again.
-
-	XXX: Add fuenter/fuleave autocommands? You might want to display
-	a NERDTree or a Tlist only in fullscreen for example. Then again, this
-	could probably be in a sizechanged autocommand that triggers if the
-	size is above a certain threshold.
-	XXX: Think about how 'fullscreen' and 'transparency' should interact. 
+	There is an optional fade-to-black effect while transitioning that
+	could be turned on by using the hidden preference
+	|MMFullScreenFadeTime| (specified in seconds). It defaults to 0,
+	meaning this effect is turned off.  Setting it to a positive value
+	(e.g. 0.25) will create an effect that fades to black during the full
+	screen transition to make it less jarring.
+
+	Note: While in 'fullscreen', you cannot set 'lines' or 'columns', as
+	they are determined by the size of the window.  'fuoptions' allows you
+	to override part of that behavior if using custom full screen.
+
 
 						*'fuoptions'* *'fuopt'*
 'fuoptions' 'fuopt'	string (default "maxvert,maxhorz")

runtime/menu.vim

@@ -1383,7 +1383,8 @@ if has("touchbar")
   endfunc
   aug FullScreenTouchBar
     au!
-    au VimEnter,VimResized * call <SID>SetupFullScreenTouchBar()
+    au VimEnter * call <SID>SetupFullScreenTouchBar()
+    au OptionSet fullscreen call <SID>SetupFullScreenTouchBar()
   aug END
 
   " 2. Character (i.e. emojis) picker. Only in modes where user is actively

src/MacVim/Base.lproj/Preferences.xib

@@ -342,11 +342,10 @@
                                 <font key="font" metaFont="system"/>
                             </buttonCell>
                             <connections>
-                                <action selector="fontPropertiesChanged:" target="-2" id="uaN-zX-Lvq"/>
                                 <binding destination="58" name="value" keyPath="values.MMNativeFullScreen" id="Y8t-au-n4b"/>
                             </connections>
                         </button>
-                        <textField verticalHuggingPriority="750" horizontalCompressionResistancePriority="250" id="VAj-Yx-2uZ" userLabel="Full Sc">
+                        <textField verticalHuggingPriority="750" horizontalCompressionResistancePriority="250" id="VAj-Yx-2uZ">
                             <rect key="frame" x="-2" y="20" width="187" height="17"/>
                             <autoresizingMask key="autoresizingMask" flexibleMinY="YES"/>
                             <textFieldCell key="cell" sendsActionOnEndEditing="YES" alignment="right" title="Full Screen:" id="bMQ-uG-iDR">
@@ -362,15 +361,14 @@
                                 <behavior key="behavior" changeContents="YES" doesNotDimImage="YES" lightByContents="YES"/>
                                 <font key="font" metaFont="system"/>
                                 <connections>
-                                    <binding destination="58" name="enabled" keyPath="values.MMNativeFullScreen" id="Uw8-jM-9g1">
+                                    <binding destination="58" name="enabled" keyPath="values.MMNativeFullScreen" id="YVo-6f-uND">
                                         <dictionary key="options">
                                             <string key="NSValueTransformerName">NSNegateBoolean</string>
                                         </dictionary>
                                     </binding>
                                 </connections>
                             </buttonCell>
                             <connections>
-                                <action selector="fontPropertiesChanged:" target="-2" id="1RM-UT-GNp"/>
                                 <binding destination="58" name="value" keyPath="values.MMNonNativeFullScreenShowMenu" id="5wX-jg-QPo"/>
                             </connections>
                         </button>

src/MacVim/MMAppController.m

@@ -248,7 +248,7 @@ + (void)initialize
 #endif // INCLUDE_OLD_IM_CODE
         [NSNumber numberWithBool:NO],     MMSuppressTerminationAlertKey,
         [NSNumber numberWithBool:YES],    MMNativeFullScreenKey,
-        [NSNumber numberWithDouble:0.25], MMFullScreenFadeTimeKey,
+        [NSNumber numberWithDouble:0.0],  MMFullScreenFadeTimeKey,
         [NSNumber numberWithBool:NO],     MMNonNativeFullScreenShowMenuKey,
         [NSNumber numberWithBool:YES],    MMShareFindPboardKey,
         nil];

src/MacVim/MMFullScreenWindow.m

@@ -150,10 +150,12 @@ - (void)enterFullScreen
     // fade to black
     Boolean didBlend = NO;
     CGDisplayFadeReservationToken token;
-    if (CGAcquireDisplayFadeReservation(fadeReservationTime, &token) == kCGErrorSuccess) {
-        CGDisplayFade(token, fadeTime, kCGDisplayBlendNormal,
-            kCGDisplayBlendSolidColor, .0, .0, .0, true);
-        didBlend = YES;
+    if (fadeTime > 0) {
+        if (CGAcquireDisplayFadeReservation(fadeReservationTime, &token) == kCGErrorSuccess) {
+            CGDisplayFade(token, fadeTime, kCGDisplayBlendNormal,
+                kCGDisplayBlendSolidColor, .0, .0, .0, true);
+            didBlend = YES;
+        }
     }
 
     // NOTE: The window may have moved to another screen in between init.. and
@@ -238,10 +240,12 @@ - (void)leaveFullScreen
     // fade to black
     Boolean didBlend = NO;
     CGDisplayFadeReservationToken token;
-    if (CGAcquireDisplayFadeReservation(fadeReservationTime, &token) == kCGErrorSuccess) {
-        CGDisplayFade(token, fadeTime, kCGDisplayBlendNormal,
-            kCGDisplayBlendSolidColor, .0, .0, .0, true);
-        didBlend = YES;
+    if (fadeTime > 0) {
+        if (CGAcquireDisplayFadeReservation(fadeReservationTime, &token) == kCGErrorSuccess) {
+            CGDisplayFade(token, fadeTime, kCGDisplayBlendNormal,
+                kCGDisplayBlendSolidColor, .0, .0, .0, true);
+            didBlend = YES;
+        }
     }
 
     // restore old vim view size

src/MacVim/MMWindowController.m

@@ -292,13 +292,6 @@ - (void)cleanup
 
     [[NSNotificationCenter defaultCenter] removeObserver:self];
 
-    if (fullScreenEnabled) {
-        // If we are closed while still in full-screen, end full-screen mode,
-        // release ourselves (because this won't happen in MMWindowController)
-        // and perform close operation on the original window.
-        [self leaveFullScreen];
-    }
-
     vimController = nil;
 
     [vimView removeFromSuperviewWithoutNeedingDisplay];
@@ -309,7 +302,7 @@ - (void)cleanup
     // dialog is displayed.
     [decoratedWindow setDocumentEdited:NO];
 
-    [[self window] orderOut:self];
+    [[self window] close];
 }
 
 - (void)openWindow
@@ -344,14 +337,6 @@ - (BOOL)presentWindow:(id)unused
 
     [decoratedWindow makeKeyAndOrderFront:self];
 
-    // HACK! Calling makeKeyAndOrderFront: may cause Cocoa to force the window
-    // into native full-screen mode (this happens e.g. if a new window is
-    // opened when MacVim is already in full-screen).  In this case we don't
-    // want the decorated window to pop up before the animation into
-    // full-screen, so set its alpha to 0.
-    if (fullScreenEnabled && !fullScreenWindow)
-        [decoratedWindow setAlphaValue:0];
-
     [decoratedWindow setBlurRadius:blurRadius];
 
     // Flag that the window is now placed on screen.  From now on it is OK for
@@ -368,9 +353,6 @@ - (BOOL)presentWindow:(id)unused
         fullScreenEnabled = YES;
         shouldResizeVimView = YES;
     } else if (delayEnterFullScreen) {
-        // Set alpha to zero so that the decorated window doesn't pop up
-        // before we enter full-screen.
-        [decoratedWindow setAlphaValue:0];
         [self enterNativeFullScreen];
     }
 
@@ -932,14 +914,14 @@ - (void)enterFullScreen:(int)fuoptions backgroundColor:(NSColor *)back
 
     fullScreenOptions = fuoptions;
     if (useNativeFullScreen) {
-        // Enter native full-screen mode.  Only supported on Mac OS X 10.7+.
+        // Enter native full-screen mode.
         if (windowPresented) {
             [self enterNativeFullScreen];
         } else {
             delayEnterFullScreen = YES;
         }
     } else {
-        // Enter custom full-screen mode.  Always supported.
+        // Enter custom full-screen mode.
         ASLogInfo(@"Enter custom full-screen");
 
         // fullScreenWindow could be non-nil here if this is called multiple
@@ -1380,46 +1362,6 @@ - (NSApplicationPresentationOptions)window:(NSWindow *)window
     return opt | NSApplicationPresentationAutoHideToolbar;
 }
 
-- (NSArray *)customWindowsToEnterFullScreenForWindow:(NSWindow *)window
-{
-    return [NSArray arrayWithObject:decoratedWindow];
-}
-
-- (void)window:(NSWindow *)window
-    startCustomAnimationToEnterFullScreenWithDuration:(NSTimeInterval)duration
-{
-    // Fade out window, remove title bar and maximize, then fade back in.
-    // (There is a small delay before window is maximized but usually this is
-    // not noticeable on a relatively modern Mac.)
-
-    // Fade out
-    [NSAnimationContext runAnimationGroup:^(NSAnimationContext *context) {
-        [context setDuration:0.5*duration];
-        [[window animator] setAlphaValue:0];
-    } completionHandler:^{
-        [window setStyleMask:([window styleMask] | NSWindowStyleMaskFullScreen)];
-        NSString *tabBarStyle = [[self class] tabBarStyleForUnified];
-        [[vimView tabBarControl] setStyleNamed:tabBarStyle];
-        [self updateTablineSeparator];
-
-        // Stay dark for some time to wait for things to sync, then do the full screen operation
-        [NSAnimationContext runAnimationGroup:^(NSAnimationContext *context) {
-            [context setDuration:0.5*duration];
-            [[window animator] setAlphaValue:0];
-        } completionHandler:^{
-            [self maximizeWindow:fullScreenOptions];
-
-            // Fade in
-            [NSAnimationContext runAnimationGroup:^(NSAnimationContext *context) {
-                [context setDuration:0.5*duration];
-                [[window animator] setAlphaValue:1];
-            } completionHandler:^{
-                // Do nothing
-            }];
-        }];
-    }];
-}
-
 - (void)windowWillEnterFullScreen:(NSNotification *)notification
 {
     // Store window frame and use it when exiting full-screen.
@@ -1455,67 +1397,32 @@ - (void)windowDidEnterFullScreen:(NSNotification *)notification
     // when titlebar is configured as hidden. Simply re-assert it to make sure
     // text is still focused.
     [decoratedWindow makeFirstResponder:[vimView textView]];
+
+    if (!fullScreenEnabled) {
+        // In case for some odd sequence of events (e.g. getting a
+        // windowDidFailToEnterFullScreen, then this call), if we have
+        // mismatched state, just reset it back to the correct one.
+        fullScreenEnabled = YES;
+        [vimController addVimInput:@"<C-\\><C-N>:set fu<CR>"];
+    }
 }
 
 - (void)windowDidFailToEnterFullScreen:(NSWindow *)window
 {
-    // NOTE: This message can be called without
-    // window:startCustomAnimationToEnterFullScreenWithDuration: ever having
-    // been called so any state to store before entering full-screen must be
-    // stored in windowWillEnterFullScreen: which always gets called.
     ASLogNotice(@"Failed to ENTER full-screen, restoring window frame...");
 
     fullScreenEnabled = NO;
-    [window setAlphaValue:1];
-    [window setStyleMask:([window styleMask] & ~NSWindowStyleMaskFullScreen)];
-    NSString *tabBarStyle = [[self class] tabBarStyleForMetal];
-    [[vimView tabBarControl] setStyleNamed:tabBarStyle];
-    [self updateTablineSeparator];
     [window setFrame:preFullScreenFrame display:YES];
 
     // Sometimes full screen will de-focus the text view. This seems to happen
     // when titlebar is configured as hidden. Simply re-assert it to make sure
     // text is still focused.
     [decoratedWindow makeFirstResponder:[vimView textView]];
-}
-
-- (NSArray *)customWindowsToExitFullScreenForWindow:(NSWindow *)window
-{
-    return [NSArray arrayWithObject:decoratedWindow];
-}
-
-- (void)window:(NSWindow *)window
-    startCustomAnimationToExitFullScreenWithDuration:(NSTimeInterval)duration
-{
-    if (!setupDone) {
-        // HACK! The window has closed but Cocoa still brings it back to life
-        // and shows a grey box the size of the window unless we explicitly
-        // hide it by setting its alpha to 0 here.
-        [window setAlphaValue:0];
-        return;
-    }
 
-    // Fade out window, add back title bar and restore window frame, then fade
-    // back in.  (There is a small delay before window contents is drawn after
-    // the window frame is set but usually this is not noticeable on a
-    // relatively modern Mac.)
-    [NSAnimationContext runAnimationGroup:^(NSAnimationContext *context) {
-        [context setDuration:0.5*duration];
-        [[window animator] setAlphaValue:0];
-    } completionHandler:^{
-        [window setStyleMask:([window styleMask] & ~NSWindowStyleMaskFullScreen)];
-        NSString *tabBarStyle = [[self class] tabBarStyleForMetal];
-        [[vimView tabBarControl] setStyleNamed:tabBarStyle];
-        [self updateTablineSeparator];
-        [window setFrame:preFullScreenFrame display:YES];
-
-        [NSAnimationContext runAnimationGroup:^(NSAnimationContext *context) {
-            [context setDuration:0.5*duration];
-            [[window animator] setAlphaValue:1];
-        } completionHandler:^{
-            // Do nothing
-        }];
-    }];
+    // Vim needs to be told that it's no longer in full screen. Because we
+    // already set fullScreenEnabled=NO, this won't do anything other than
+    // updating Vim's state.
+    [vimController addVimInput:@"<C-\\><C-N>:set nofu<CR>"];
 }
 
 - (void)windowWillExitFullScreen:(NSNotification *)notification
@@ -1546,26 +1453,33 @@ - (void)windowDidExitFullScreen:(NSNotification *)notification
     // when titlebar is configured as hidden. Simply re-assert it to make sure
     // text is still focused.
     [decoratedWindow makeFirstResponder:[vimView textView]];
+
+    if (fullScreenEnabled) {
+        // Sometimes macOS will first send a windowDidFailToExitFullScreen
+        // notification (e.g. if user is in the middle of switching spaces)
+        // before actually sending windowDidExitFullScreen. Just to be safe, if
+        // we are actually confused here, simply reset the state back.
+        fullScreenEnabled = NO;
+        [vimController addVimInput:@"<C-\\><C-N>:set nofu<CR>"];
+    }
 }
 
 - (void)windowDidFailToExitFullScreen:(NSWindow *)window
 {
-    // TODO: Is this the correct way to deal with this message?  Are we still
-    // in full-screen at this point?
     ASLogNotice(@"Failed to EXIT full-screen, maximizing window...");
 
     fullScreenEnabled = YES;
-    [window setAlphaValue:1];
-    [window setStyleMask:([window styleMask] | NSWindowStyleMaskFullScreen)];
-    NSString *tabBarStyle = [[self class] tabBarStyleForUnified];
-    [[vimView tabBarControl] setStyleNamed:tabBarStyle];
-    [self updateTablineSeparator];
     [self maximizeWindow:fullScreenOptions];
 
     // Sometimes full screen will de-focus the text view. This seems to happen
     // when titlebar is configured as hidden. Simply re-assert it to make sure
     // text is still focused.
     [decoratedWindow makeFirstResponder:[vimView textView]];
+
+    // Vim needs to be told that it's still in full screen. Because we already
+    // set fullScreenEnabled=YES, this won't do anything other than updating
+    // Vim's state.
+    [vimController addVimInput:@"<C-\\><C-N>:set fu<CR>"];
 }
 
 #endif // (MAC_OS_X_VERSION_MAX_ALLOWED >= MAC_OS_X_VERSION_10_7)
@@ -1868,6 +1782,10 @@ - (void)updateToolbar
 
 - (BOOL)maximizeWindow:(int)options
 {
+    // Note:
+    // This is deprecated code and will be removed later. 'fuopt' should be
+    // handled in processInputQueueDidFinish instead.
+
     if (floor(NSAppKitVersionNumber) > NSAppKitVersionNumber10_10_Max) {
         // NOTE: Prevent to resize the window in Split View on El Capitan or
         // later.