Custom submodules without default values

[lucasshiva]

In Configure Aspects, we have an example on how to setup a custom submodule for an user aspect, but is it possible to make it work without setting any default values?

For context, I have a den.aspects.fonts module that exposes an option to set the default sans-serif font:

{ ... }:
{
  den.aspects.fonts =
    { config, lib, ... }:
    {

      imports = [
        {
          options = with lib.types; {
            sansSerif = lib.mkOption { type = str; };
          };
        }
      ];

      nixos =
        { pkgs, ... }:
        {
          fonts.fontconfig.defaultFonts = {
            sansSerif = [ config.sansSerif ];
          };
        };
    };
}

Notice the lack of default values. My goal is to force hosts/users to set these values, but I also want to reference the values in other submodules, such as den.aspects.gnome.

So, in the host file:

den.aspects.myHost = {
  includes = [ 
    den.aspects.fonts
    den.aspects.gnome
  ];
  
  # Set default fonts here.
  fonts.sansSerif = "Ubuntu";
  
  # Or maybe we need the whole namespace?
  den.aspects.fonts.sansSerif = "Ubuntu";
  
  # Host config
  nixos = { pkgs, ... }: {};
};

Then, inside my den.aspects.gnome module, I want to reference the font using either den.aspects.fonts.sansSerif or config.den.fonts.sansSerif.

{ den, ... }:
{
  # Simplified for the example.
  den.aspects.gnome.homeManager.dconf = {
    enable = true;
    settings = {
      "org/gnome/desktop/interface" = {
        font-name = "${den.aspects.fonts.sans} 11";
        # Or using `config.den.aspects.fonts.sans`.
      };
    };
  };
}

I've tried everything, but I could only make it work by setting a default value in the option. Without the default value, it says the option was accessed but has no value defined, even with me configuring the option in the host. It could be some kind of evaluation problem, where den.aspects.gnome runs before the font options are set, meaning they're still null/empty.

So, I'm either doing something wrong, forgetting about something important, or Den simply doesn't support this in the way that I'm approaching it. I know an alternative would be to set these options via den.schema, but I only want this to apply to host/users that import the den.aspects.fonts module. Additionally, it might be possible to expose a submodule for den.aspects.gnome and pass the fonts to it directly in the host config, but I'm not a big fan of this approach.

Any tips?

[theutz]

If it's something you want set per-user, you might be better served by creating a new option on the user schema.

[lucasshiva]

It's something that I want per aspect/host, actually. The idea is simple: include and configure den.aspects.fonts in the host. Then, other aspects would reference the values set by the host. I could probably achieve this with a schema, but options would be preferable to keep everything contained in the aspect itself.

[sini]

I'll put together a gist of how I do aspect settings in my config tomorrow.

What you're describing is either our quirks system, or my custom settings extension. Aspects provide behavior; they aren't data. Users/hosts provide data/context -- so in my configuration I define options on aspects, and assign values to context (generally the host).

[lucasshiva]

Just took a look at it, injection like you showed in the example would be perfect:

nixos = { settings, host, lib, ... }: {
  services.nginx.config = ''
    worker_processes ${toString settings.nginx.performance.workers};
  '';
};

But personally I'd prefer writing values on host, user or home instead of putting overrides in scopeSettings. And I'd still keep option types instead of settings schemas, at least for the initial release, just to simplify things.

Either way, you're doing a great job on this, I appreaciate you taking your time to do that. Hope to see more on this in the future.

[sini]

Don't worry, host.settings/user.setting/environment.settings will still be available even if they're just compat shims. :)

[sini]

Also we already shipped gen-schema into den; hosts already use schema options. :-D

[amanako]

I took a look at the gist and the idea seems amazing, @sini. One thing though, would it be possible to obtain host or user context within settings or is there some solution or workaround for this specifically? I was thinking of something like this:

  den.aspects.core.impermanence = {
    settings = {
      enableUsers = let
        hasUsers =
          host.users         # error: undefined variable `host`
          |> lib.attrValues
          |> builtins.any (val: true);
      in
        lib.mkEnableOption ""
        // {
          default = hasUsers;
          description = ''
            Whether to enable impermanence module for users, that is impermanence for `/home` directory.
          '';
        };
    };

Wrapping whole aspect in { host, ... } makes it parametric so settings get ignored, but also wrapping settings alone or parts of it (as a module declaration) make it a lambda so once again error.

[lucasshiva]

I took a look at the gist and the idea seems amazing, @sini. One thing though, would it be possible to obtain host or user context within settings or is there some solution or workaround for this specifically? I was thinking of something like this:

Wrapping whole aspect in { host, ... } makes it parametric so settings get ignored, but also wrapping settings alone or parts of it (as a module declaration) make it a lambda so once again error.

That's one of the reasons why I went back to using normal NixOS options - more flexibity. I've never used impermanence, but maybe you could write a system module like this:

den.aspects.core.impermanence = {
  nixos = { host, lib, ... }: {
    options.my.impermanence = {
      enableUsers = lib.mkEnableOption ""; # Now you should have access to host.users
    };

    config = {
      # manage system config for impermanence
    };
  };
};

Alternatively, if you're using home-manager, maybe let users opt-in into the module by exposing a home module instead, so now users would have to call my.impermanence.enable = true; directly in their homeManager class module.

# impermanence.nix
den.aspects.core.impermanence = {
  homeManager =
    { lib, config, ... }:
    let
      cfg = config.my.impermanence;
    in
    {
      options.my.impermanence = {
        enable = lib.mkEnableOption "Enable impermanence";
      };

      config = lib.mkIf cfg.enable {
        # Only runs if enabled.
      };
    };
};

# <user>.nix
{ den, ... }:
{
  den.aspects.username = {
    includes = [ den.aspects.core.impermanence ];
    homeManager = { ... }: {
      my.impermanence.enable = true; # Enable for this user. No need for `host.users.`
    };
  };
}

But if you want every user in a given host to use impermanence, then you could use provides.to-users instead:

den.aspects.myHost = {
  includes = [ den.aspects.core.impermanence ];
  provides.to-users = { user, ... }: {
    homeManager = { ... }: {
      my.impermanence.enable = true; # Enable for all users on this host.
    };
  };
};

Note that the above might require including den.batteries.host-aspects in your user aspect. I haven't tested without it.

[lucasshiva]

Just wanted to mention that I got this working nicely. The gist was very helpful, thanks again @sini. I implemented Option 1 as shown, but then created a policy to automatically inject settings from either host.settings on NixOS or home.settings on standalone home-manager:

# den/policies/settings-injection.nix
{ den, ... }:
{
  den.policies.settings-injection =
    {
      host ? null,
      home ? null,
      ...
    }:
    [
      (den.lib.policy.resolve {
        settings =
          if host ? settings then
            host.settings
          else if home ? settings then
            home.settings
          else
            throw "No settings found on host or home";
      })
    ];
}

# den/default.nix
{ den, ... }: 
{
  den.default.includes = [
    den.policies.settings-injection
  ];
  
  # Other settings..
}

Also, an important change from the gist is that I went from den.schema.host to den.schema.conf to also support standalone home-manager:

# den/settings.nix
{ den, lib, ... }:
let
  inherit (lib) mkOption types;
  
  # Dynamic settings type. Read the gist for details.
  settingsType = ...
in
{
  den.reservedKeys = [ "settings" ];
  
  # Applies to hosts, homes, and users, but I tend to configure everything in the host/home side.
  den.schema.conf = {
    imports = [
      {
        options.settings =
          mkOption {
            type = settingsType;
            default = { };
            description = "Per-aspect typed settings";
          }
          # Exclude settings from entity identity hashing
          // {
            identity = false;
          };
      }
    ];
  };
}

Now, the only thing "missing" would be the ability to reference values from inside the settings block somehow, like so:

den.hosts.x86_64-linux."${host}" = {
  users.lucas = { };

  settings = rec {
    fonts = {
      sans = "Adwaita Sans";
    };
    gnome = {
      interface-font = fonts.sans; # Defined above, `rec` lets us use this.
      document-font = ???; # Use the default from `den.aspects.fonts.settings.serif`.
    };
  };
};

But the above seems to be an unusual pattern in NixOS (even though it's common in enterprise development), so that's not really an issue for me.

Besides that, I might run into issues if I ever need something more than what NixOS options pattern provides, but for now this is working nicely as a way for aspects to expose configurable options that can be defined in a per-host (or home) basis.

EDIT: I think we're also missing the ability to set values inside another aspect, like the gnome aspect itself setting the value for fonts.sans. This would be nice to have as well. Only way I can see to support this is a top-level settings e.g. {den, settings, ... }: ... instead of scoping it to hosts, homes, and users. Or scoping it to den itself, via den.settings.

[sini]

Glad you found something that works for you. I'd love if you could expand more on your thoughts around inter-aspect side-effects/shared settings and how you feel that should work.

I'm hesitant to make things 'the way you do it in den', as a framework my general policy is to try to avoid opinionated implementations and instead provide the tools for users to express their own opinions which is why my personal settings pattern isn't already in den. The new settings pattern that described as part 2 in my gist that I have demo'd in gen-aspects is more about graph based composition, layering, provenance, etc as pipeline layering/resolution tooling than it is about providing a settings solution. The primitives will be exposed for users to change or implement their own variants should they choose to.

However, understanding alternative user patterns/desires can help inform me as to any gaps in my designs that would prevent user expression.

Note: I want to avoid boilerplate wherever possible, the common path should be easy/magic/effortless without constraining users to it. :)

[lucasshiva]

Glad you found something that works for you. I'd love if you could expand more on your thoughts around inter-aspect side-effects/shared settings and how you feel that should work.

I'd prefer something closer to what we have in NixOS. This means making settings available at a higher-level than injecting it into homeManager or nixos. For example:

{ lib, ... }:
{
  den.aspects.myModule =
    { config, settings, ... }:
    let
      cfg = settings.myModule;
    in
    {
      options = {
        enable = lib.mkEnableOption "Enable something";
        path = lib.mkOption { type = lib.types.str; };
      };

      nixos =
        { pkgs, ... }:
        {
          services.xxx.enable = true;
          services.yyy = lib.mkIf cfg.enable {
            baseDirectory = cfg.path;
          };
        };

      homeManager = lib.mkIf settings.anotherModule.enable {
        # Only runs if `anotherModule` is enabled somewhere else.
      };
    };
}

As a side note, I'd rename (or alias it, if possible) config to aspect or self, just to make it clearer we're referencing the aspect itself. As for the settings, we could manage them inside a host's or user's aspect:

{ den, ... }:
{
  den.hosts.x86_64-linux.myHost = {
    users.lucas = { };
  };

  den.aspects.myHost =
    { settings, ... }:
    {
      includes = [
        den.aspects.myModule
        den.aspects.anotherModule
      ];

      nixos =
        { user, ... }:
        {
          settings.myModule.enable = true;
          settings.myModule.path = "/home/${user.userName}/someRandomPath";
          settings.anotherModule.enable = false; # Or just don't include it.
        };
    };
}

The above is closer to what we currently have in pure NixOS. I understand that we can achieve something similar without defining any options, just based on how the modules are structured and how we include them, but having options would be nice for the ones that do want or need it. It could even be behind a battery in case you want it opt-in.

Also, perhaps it would be possible to move settings even higher, making it available essentially anywhere:

{ lib, settings, ... }:
{
  den.aspects.someModule = {
  
    options.someOption = lib.mkOption { type = lib.types.str; };
    
    nixos =
    { pkgs, ...}:
    {
      programs.xxx.enable = true;
      # If we use `config` to reference the aspect iself, perhaps this could become `config.someOption` by default.
      programs.xxx.someValue = settings.someModule.someOption; 
    };
  };
}

Lastly, notice how I didn't use the settings keyword to define the options themselves, only to reference them. This means that for anything more advanced than what NixOS offers, like what you described in Part 2, it could be available only inside a custom block. Since you called it "Settings schemas" in the gist, I'm going to use schema as the name for the block, just to avoid conflicts with a top-level settings:

{ settings, ... }:
let
  cfg = settings.web;
in
{
  den.aspects.web = {
    schema = {
      performance.workers = {
        default = 4;
      };
      upstream.servers = {
        default = [ ];
        merge = "append";
      };
    };

    nixos =
      { ... }:
      {
        services.nginx = {
          enable = true;
          workers = cfg.performance.workers;
          servers = cfg.upstream.servers;
        };
      };
  };
}

That was the best I could manage, but I know nothing about the internals of Nix or Den, so it might not even be possible from a technical standpoint. All I can say is that I'd love if settings worked like shown above — it would make Den even better for me, especially since I tend to create a lot of options.

Also, I haven't tried it yet, but it might be possible to follow the usual options/config pattern by defining the options inside the nixos or homeManager blocks. Then, we could probably use a scoped config (e.g. homeManager = { config, pkgs, ... }: {}; to access the options. No shared options, but that's the default behavior in NixOS anyway.

Curious about your thoughts on this.