[Improvement] Support multi-language setups

[msujew]

See also the discussion in here for hints on how we could implement this.

[spoenemann]

IMO we should aim for a less obtrusive solution than what we have in Langium. Ideally, we can keep the complexity largely unchanged for single-language projects and only increase it when you configure your project for multiple grammars.

[msujew]

I'm currently thinking about an approach very similar to how one uses esbuild with a non-trivial setup:

import "typefox.dev/fastbelt/cmd"

func main() {
  context := &cmd.BuildContext{
    Input: "some-dir/grammar-dir" // <-- points at a directory, will collect all .fb files within
    Languages: []cmd.Language{
      cmd.Language{
        Id: "some-language",
        Ext: []string{".some"},
        Entry: "SomeRule",
      },
      cmd.Language{
        Id: "other-language",
        Ext: []string{".other"},
        Entry: "OtherRule",
      }
    },
    // Maybe more available properties like:
    Plugins: []cmd.Plugin{ ... }
  }
  context.Build()
}

The cmd will then generate a LanguageSelector (interface defined in the framework) default implementation that uses the Ext property to differentiate languages. Returns indices to the given Languages array in order to select the language. Adopters can override this to adjust it. The build will validate that the expected rule are actually available.

The current cmd will be kept as it is for single language projects.

[Lotes]

1. I find it somehow attractive to define the entry rules in such a file. Also because you can have one grammar and N languages using N different entry rules from the same grammar. (On the other hand you do not have any validation at edit time that the rule really exists.)

2. Shall we enforce a acyclic import structure then? I think in Langium it was possible to import everything from everywhere, which made me wondering. It felt unnatural, to me at least. Having no cycles in the import feels good from a architectural perspective (you made enough thoughts about the concepts, but I can also see that it could look attractive to import again everything from everywhere...).

3. Another question that arises for me is: Can we create a .go-Script as entry point for the Fastbelt CLI? Or is this file calling Fastbelt?
   After a small research that seem actually unusual in Go to follow this Vitest-like path. I would expect a Yaml or Json file only instead.

[msujew]

On the other hand you do not have any validation at edit time that the rule really exists

People will probably quick to identify issues if their build fails to do anything with an error message and exit code 1 ;)

Shall we enforce a acyclic import structure then?

Currently, our plan is to use a similar structure as Go. I.e. make everything from the current package (read: directory) available without any imports. This still makes everything available from anywhere, within the constraints of the current package. See also #83.

Or is this file calling Fastbelt?

Yes 👍

After a small research that seem actually unusual in Go to follow this Vitest-like path. I would expect a Yaml or Json file only instead.

Vitest? That uses a different style of API. Anyway, my research actually said the opposite. ESBuild has a Go API that works exactly this way. There's also Mage which is designed for this use case (but more as a library to write scripts in Go).

The issue with using a simple config file is that you cannot execute any non-trivial logic. If you want to build a compiler plugin for Fastbelt (e.g. to generate textMate grammars), you can only fall back to stuff that is already built into the Fastbelt CLI. I was always unhappy that we had to maintain so many integrations with different syntax-highlighting providers in the langium-cli. This eliminates the need for that completely (or at least makes this more flexible - it's not just all or nothing anymore; We can provide some integrations, and the adopter can also write their own).