[BUG] Core dumped when running gcg-operation-location-migration codemod

[MarlonVCendron]

Describe the bug
I tried running the gcg-operation-location-migration on a big codebase and I keep getting core dumped

✔ Parse Configuration
❯ Generate outputs
  ❯ Generate to src/
    ✔ Load GraphQL schemas
    ✔ Load GraphQL documents
    ⠸ Generate


#
# Fatal error in , line 0
# Check failed: (result.ptr) != nullptr.
#
#
#
#FailureMessage Object: 0x7ffc75a49fe0
----- Native stack trace -----

 1: 0x101c6e1  [node]
 2: 0x29ddbcb V8_Fatal(char const*, ...) [node]
 3: 0x14c8a55 heap::base::SlotCallbackResult v8::internal::Scavenger::ScavengeObject<v8::internal::FullHeapObjectSlot>(v8::internal::FullHeapObjectSlot, v8::internal::Tagged<v8::internal::HeapObject>) [node]
 4: 0x14ca05c void v8::internal::BodyDescriptorBase::IteratePointers<v8::internal::IterateAndScavengePromotedObjectsVisitor>(v8::internal::Tagged<v8::internal::HeapObject>, int, int, v8::internal::IterateAndScavengePromotedObjectsVisitor*) [node]
 5: 0x14d2c52 v8::internal::Scavenger::IterateAndScavengePromotedObject(v8::internal::Tagged<v8::internal::HeapObject>, v8::internal::Tagged<v8::internal::Map>, int) [node]
 6: 0x14d323b v8::internal::Scavenger::Process(v8::JobDelegate*) [node]
 7: 0x14d524f v8::internal::ScavengerCollector::JobTask::ProcessItems(v8::JobDelegate*, v8::internal::Scavenger*) [node]
 8: 0x14d535c v8::internal::ScavengerCollector::JobTask::Run(v8::JobDelegate*) [node]
 9: 0x2343edd v8::platform::DefaultJobState::Join() [node]
10: 0x2344073 v8::platform::DefaultJobHandle::Join() [node]
11: 0x14d6adc v8::internal::ScavengerCollector::CollectGarbage() [node]
12: 0x14312d2 v8::internal::Heap::Scavenge() [node]
13: 0x144f4ba v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::internal::GarbageCollectionReason, char const*) [node]
14: 0x1450f99  [node]
15: 0x1454428  [node]
16: 0x1cba291  [node]
Illegal instruction (core dumped)
 ELIFECYCLE  Command failed with exit code 132.
 ELIFECYCLE  Command failed with exit code 132.

[eddeee888]

Hello @MarlonVCendron , sorry that this happened!

Could you please let me know how big the codebase is? e.g. how many .ts files, how many .graphql files files
As a workaround, you could try splitting up your code into multiple runs e.g.

// codegen.ts
const config = {
  schema: 'your-schema.graphql',
  documents: [
    'src/app1', // <-- try running with just one part of your codebase
    // 'src/app2',
  ];
  generates: {
    // ... the rest
  }
}

[MarlonVCendron]

Thanks for the tip, that'll probably be enough! Just FYI, we have 8404 .ts/.tsx files and 530 .gql files

[MarlonVCendron]

yeah, not even with a single graphql file did it work. Adding --verbose didn't reveal much

[STARTED] Parse Configuration
[SUCCESS] Parse Configuration
[STARTED] Generate outputs
[STARTED] Generate to src/
[STARTED] Load GraphQL schemas
[SUCCESS] Load GraphQL schemas
[STARTED] Load GraphQL documents
[SUCCESS] Load GraphQL documents
[STARTED] Generate

<--- Last few GCs --->

[140223:0x36b84000]   113988 ms: Scavenge 3925.7 (4125.2) -> 3915.6 (4127.4) MB, pooled: 2 MB, 12.81 / 0.00 ms  (average mu = 0.244, current mu = 0.223) allocation failure; 
[140223:0x36b84000]   115633 ms: Mark-Compact 3929.5 (4128.4) -> 3887.3 (4129.7) MB, pooled: 0 MB, 1621.27 / 0.00 ms  (average mu = 0.231, current mu = 0.218) allocation failure; scavenge might not succeed


<--- JS stacktrace --->

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
----- Native stack trace -----

 1: 0xe38a0e node::OOMErrorHandler(char const*, v8::OOMDetails const&) [node]
 2: 0x120a5a0 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [node]
 3: 0x120a877 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [node]
 4: 0x1438155  [node]
 5: 0x1438183  [node]
 6: 0x145125a  [node]
 7: 0x1454428  [node]
 8: 0x1cba291  [node]

This is my codegen config, the schema is fetched from the API:

const config: CodegenConfig = {
  overwrite: true,
  schema: process.env.CODEGEN_SCHEMA_PATH || 'schema.json',
  documents: ['src/**/*.graphql', 'src/**/*.gql'],
  generates: {
    'src/': defineConfig({
      tsConfigFilePath: 'tsconfig.json',
      gqlTag: {
        name: 'graphql',
        importFrom: './gql',
        importType: 'relative',
      },
      hooksImportFrom: '@apollo/client', // Use @apollo/client for v3
    }),
  },
  config: {
    inputValueDeprecation: true,
  },
}

[eddeee888]

Hi @MarlonVCendron ,

This may happen because there are lots of files being included by tsconfig.json. In a recent repo I looked at, we excluded unrelated files to avoid reading too many files at once, causing OOM e.g.

// tsconfig.json
"compilerOptions": {
  },
  "include": [
    "./src", // Try to narrow this down to sub folders 
  ],
  // Try ignoring test files, assuming they won't call the hooks
  "exclude": [
    "./src/**/*.spec.ts", 
    "./src/**/*.spec.tsx",
    "./src/**/*.test.ts",
    "./src/**/*.test.tsx",
  ]

[MarlonVCendron]

Thank you! Yeah, I was able to run it limiting to very few files