Repeated fragment spreading causes perf issues with near-operation-files + typescript-operations

[vhfmag]

Which packages are impacted by your issue?

@graphql-codegen/near-operation-file-preset

Describe the bug

While investigating an issue with graphql codegen taking too long in the pipeline of one of my company's projects, I came across a performance issue caused when spreading a deeply nested fragment multiple times in the same file. The issue happens when near-operation-file and typescript-operations are used together, and is ultimately caused due to the first supplying repeated input to the latter.

Here's how it works:

1. near-operation-file resolves document imports here and then generates a list of externalFragments for each given file here. That list includes all nested fragments, and if a fragment is spread N times in the same file, it and of its nested fragments will be included N times. externalFragments is then passed as a config to plugins here.
2. typescript-operations includes externalFragments into allFragments here and then passes it to TypeScriptDocumentsVisitor here, which passes it to SelectionSetToObject here, which passes it to getFieldNames here.
3. getFieldNames iterates over its first argument (selections) and when it finds a fragment spread, it calls itself passing loadedFragments.filter(def => def.name === selection.name.value).flatMap(...) as the selections argument here. That's where the issue happens: because the same fragment is included multiple times in loadedFragments, the recursive function call will iterate multiple times over the same fragment. If that fragment has a nested one, the same will happen to it, and so on.
4. All that means that if a fragment is spread M times and has N levels of fragment spreads (e.g. if fragment A spreads fragment B, which spreads fragment C, which doesn't spread anything, N = 3), getFieldNames will be called at least M^N times.

Your Example Website or App

https://github.com/vhfmag/codegen-perf-issue-repro

Steps to Reproduce the Bug or Issue

1. Clone the repo
2. Run yarn codegen
3. Add a new fragment to frags.ts and spread it into the last one
4. Run yarn codegen — it should take 4 times as long as it did before because we spread fragment A 4 times in the ops.ts file
5. Steps 3 & 4 can be repeated indefinitely

Example change:

Before	After
// ... const J = gql` fragment J on User { j ...K } `; const K = gql` fragment K on User { k } `;	// ... const J = gql` fragment J on User { j ...K } `; const K = gql` fragment K on User { k ...L } `; const L = gql` fragment L on User { l } `;

Expected behavior

Time complexity should increase linearly with M & N

Screenshots or Videos

No response

Platform

• OS: macOS & linux
• NodeJS: 18.18.2
• graphql version: 16.2.0
• @graphql-codegen/near-operation-file-preset version(s): 3.0.0

Codegen Config File

{
  schema: "schema.graphql",
  documents: "src/**/*.ts",
  generates: {
    "types.ts": {
      preset: "near-operation-file",
      presetConfig: {
        baseTypesPath: "types.ts",
        folder: "__generated__",
      },
      config: {
        avoidOptionals: {
          field: true,
          inputValue: false,
          object: false,
          defaultValue: false,
        },
        arrayInputCoercion: false,
        extractAllFieldsToTypes: true,
        printFieldsOnNewLines: true,
        omitOperationSuffix: true,
        namingConvention: "keep",
      },
      plugins: ["typescript", "typescript-operations"],
    },
  },
}

Additional context

No response

[corpooo]

Any updates here?

[vhfmag]

if that's helpful for anyone else, our workaround was to patch the @graphql-codegen/typescript-operations package like this:

diff --git a/cjs/index.js b/cjs/index.js
index 1924b61..de5c014 100644
--- a/cjs/index.js
+++ b/cjs/index.js
@@ -13,7 +13,7 @@ const plugin = (schema, rawDocuments, config) => {
         })
         : rawDocuments;
     const allAst = (0, graphql_1.concatAST)(documents.map(v => v.document));
-    const allFragments = [
+    const allFragments = require("lodash").uniqWith([
         ...allAst.definitions.filter(d => d.kind === graphql_1.Kind.FRAGMENT_DEFINITION).map(fragmentDef => ({
             node: fragmentDef,
             name: fragmentDef.name.value,
@@ -21,7 +21,7 @@ const plugin = (schema, rawDocuments, config) => {
             isExternal: false,
         })),
         ...(config.externalFragments || []),
-    ];
+    ], require("fast-deep-equal"));
     const visitor = new visitor_js_1.TypeScriptDocumentsVisitor(schema, config, allFragments);
     const visitorResult = (0, plugin_helpers_1.oldVisit)(allAst, {
         leave: visitor,

[Nurou]

I reproduced the same issue downstream with near-operation-file + typescript-operations and traced it to duplicated externalFragments being forwarded into typescript-operations.

In our case, assistant codegen went from about 248s to about 2s after deduping externalFragments by name before passing them to TypeScriptDocumentsVisitor.

I opened a PR for that here: dotansimha/graphql-code-generator#10728