gatsby-source-shopify throws EPERM: operation not permitted in gatsby develop and gatsby build

[unimprobable]

Description

I am running into issues with gatsby-source-shopify on multiple projects and using different stores. See repro repo here.

Steps to reproduce

1. gatsby new bug-repro https://github.com/gatsbyjs/gatsby-starter-hello-world
2. npm install gatsby-source-shopify
3. added shopify details to gatsby-config.js from gatsby-shopify-starter store
4. gatsby develop

Expected result

Gatsby develop should run completely.

Actual result

Gatsby develop exits after an error occurs sourcing shopify data, and throws another error with gatsby-source-filesystem

PS D:\websites\gatsby\bug-repro> gatsby develop
success open and validate gatsby-configs - 0.013s
success load plugins - 0.127s
success onPreInit - 0.020s
info One or more of your plugins have changed since the last time you ran Gatsby. As
a precaution, we're deleting your site's cache to ensure there's no stale data.
success initialize cache - 0.014s
success copy gatsby files - 0.054s
success onPreBootstrap - 0.014s
success createSchemaCustomization - 0.003s

gatsby-source-shopify/graphql starting to fetch data from Shopify

gatsby-source-shopify/graphql fetched and processed Shop nodes: 279.885ms

gatsby-source-shopify/graphql fetched and processed Blog nodes: 283.298ms

gatsby-source-shopify/graphql fetched and processed ShopPolicy nodes: 304.754ms

gatsby-source-shopify/graphql fetched and processed Page nodes: 310.893ms

gatsby-source-shopify/graphql fetched and processed Article nodes: 336.872ms

 ERROR 


error an error occurred while sourcing data

success Downloading remote files - 30.276s - 118/133 4.39/s

 ERROR #11321  PLUGIN

"gatsby-source-shopify" threw an error while running the sourceNodes lifecycle:

failed to process https://cdn.shopify.com/s/files/1/1312/0893/collections/parker-burchfield-224850.jpg
Error: EPERM: operation not permitted, open 'D:\websites\gatsby\bug-repro\.cache\caches\gatsby-source-filesystem\tmp-2fe79fb7728cde2af1f2c12e16aa19c9.jpg'

success Checking for changed pages - 0.006s
success source and transform nodes - 30.654s
warn Multiple node fields resolve to the same GraphQL field `ShopifyArticle.blog` - [`blog`, `blog___NODE`]. Gatsby will use `blog___NODE`.
success building schema - 0.307s
info Total nodes: 391, SitePage nodes: 1 (use --verbose for breakdown)
success createPages - 0.004s
success Checking for changed pages - 0.001s
success createPagesStatefully - 0.035s
success update schema - 0.014s
success write out redirect data - 0.002s
success onPostBootstrap - 0.006s
info bootstrap finished - 33.226s
success onPreExtractQueries - 0.001s
D:\websites\gatsby\bug-repro\node_modules\yoga-layout-prebuilt\yoga-layout\build\Release\nbind.js:53
        throw ex;
        ^

Error: EPERM: operation not permitted, lstat 'D:\websites\gatsby\bug-repro\.cache\caches\gatsby-source-filesystem\tmp-9583dfddf6ff828cf12e0ec18599388c.jpg'
    at Object.lstatSync (fs.js:1007:3)
    at Object.lstatSync (D:\websites\gatsby\bug-repro\node_modules\graceful-fs\polyfills.js:308:16)
    at Object.rimrafSync (D:\websites\gatsby\bug-repro\node_modules\fs-extra\lib\remove\rimraf.js:237:18)
    at WriteStream.<anonymous> (D:\websites\gatsby\bug-repro\node_modules\gatsby-source-filesystem\create-remote-file-node.js:209:12)
    at WriteStream.emit (events.js:326:22)
    at _stream_writable.js:616:5
    at WriteStream._final (internal/fs/streams.js:352:3)
    at callFinal (_stream_writable.js:609:10)
    at processTicksAndRejections (internal/process/task_queues.js:84:21) {
  errno: -4048,
  syscall: 'lstat',
  code: 'EPERM',
  path: 'D:\\websites\\gatsby\\bug-repro\\.cache\\caches\\gatsby-source-filesystem\\tmp-9583dfddf6ff828cf12e0ec18599388c.jpg'
}

gatsby-config.js

/**
 * Configure your Gatsby site with this file.
 *
 * See: https://www.gatsbyjs.com/docs/gatsby-config/
 */

module.exports = {
  /* Your site config here */
  plugins: [
    {
      resolve: "gatsby-source-shopify",
      options: {
        shopName: "graphql",
        accessToken: "dd4d4dc146542ba7763305d71d1b3d38",
      }
    }
  ],
}

package.json

{
  "name": "gatsby-starter-hello-world",
  "private": true,
  "description": "A simplified bare-bones starter for Gatsby",
  "version": "0.1.0",
  "license": "0BSD",
  "scripts": {
    "build": "gatsby build",
    "develop": "gatsby develop",
    "format": "prettier --write \"**/*.{js,jsx,ts,tsx,json,md}\"",
    "start": "npm run develop",
    "serve": "gatsby serve",
    "clean": "gatsby clean",
    "test": "echo \"Write tests! -> https://gatsby.dev/unit-testing\" && exit 1"
  },
  "dependencies": {
    "gatsby": "^2.26.1",
    "gatsby-source-shopify": "^3.10.0",
    "react": "^16.12.0",
    "react-dom": "^16.12.0"
  },
  "devDependencies": {
    "prettier": "2.2.1"
  },
  "repository": {
    "type": "git",
    "url": "https://github.com/gatsbyjs/gatsby-starter-hello-world"
  },
  "bugs": {
    "url": "https://github.com/gatsbyjs/gatsby/issues"
  }
}

Environment

System:
OS: Windows 10 10.0.19041
CPU: (8) x64 Intel(R) Core(TM) i7-9700 CPU @ 3.00GHz
Binaries:
Node: 12.20.2 - C:\Program Files\nodejs\node.EXE
Yarn: 1.22.10 - C:\Program Files\nodejs\yarn.CMD
npm: 6.14.11 - C:\Program Files\nodejs\npm.CMD
Languages:
Python: 3.9.1
Browsers:
Chrome: 88.0.4324.182
Edge: Spartan (44.19041.423.0), Chromium (88.0.705.74)
npmPackages:
gatsby: ^2.26.1 => 2.26.1
gatsby-source-shopify: ^3.10.0 => 3.10.0
npmGlobalPackages:
gatsby-cli: 2.19.1

[LekoArts]

It tries to download files and write them to your hard-drive. Make sure that your user/the process has enough permissions to do that.

[unimprobable]

Hi @LekoArts . I have seen similar posts that had this error, but I have tried all the solutions and cannot fix the issue.

I have tried

1. running VS Code as an administrator
2. tried running in powershell as administrator
3. tried clearing node_modules and npm ci
4. tried adding the project folder to Windows' Microsoft Defender Antivirus Exclusion list
5. tried switching to yarn
6. tried switching node versions

I am at a loss as how to proceed. Any help would be appreciated.

Thanks.

[LekoArts]

Can you try it on another drive? (In case D: is not your default one)

[unimprobable]

@LekoArts I just tried it on my C:\ drive, and got the same errors:

PS C:\Users\obrie\Documents\websites\bug-repro> gatsby develop
success open and validate gatsby-configs - 0.014s
success load plugins - 0.148s
success onPreInit - 0.021s
success initialize cache - 0.007s
success copy gatsby files - 0.517s
success onPreBootstrap - 0.021s
success createSchemaCustomization - 0.002s

gatsby-source-shopify/graphql starting to fetch data from Shopify

gatsby-source-shopify/graphql fetched and processed ShopPolicy nodes: 338.505ms

gatsby-source-shopify/graphql fetched and processed Blog nodes: 346.378ms

gatsby-source-shopify/graphql fetched and processed Page nodes: 362.387ms

gatsby-source-shopify/graphql fetched and processed Shop nodes: 371.695ms

gatsby-source-shopify/graphql fetched and processed Article nodes: 427.773ms

 ERROR 


error an error occurred while sourcing data

success Downloading remote files - 26.132s - 39/133 5.09/s

 ERROR #11321  PLUGIN

"gatsby-source-shopify" threw an error while running the sourceNodes lifecycle:

failed to process https://cdn.shopify.com/s/files/1/1312/0893/products/RUNWAY_COLOR_BLOCKED_DRESS_3.jpg
RequestError: socket hang up

success Checking for changed pages - 0.001s
success source and transform nodes - 26.679s
warn Multiple node fields resolve to the same GraphQL field `ShopifyArticle.blog` - [`blog`, `blog___NODE`]. Gatsby will use `blog___NODE`.
success building schema - 0.369s
info Total nodes: 81, SitePage nodes: 1 (use --verbose for breakdown)
success createPages - 0.004s
success Checking for changed pages - 0.001s
success createPagesStatefully - 0.042s
success update schema - 0.017s
success write out redirect data - 0.002s
success onPostBootstrap - 0.001s
info bootstrap finished - 30.001s
success onPreExtractQueries - 0.002s
success extract queries from components - 0.127s
success write out requires - 0.023s
success run page queries - 0.038s - 2/2 52.19/s
warn Multiple node fields resolve to the same GraphQL field `ShopifyArticle.blog` - [`blog`, `blog___NODE`]. Gatsby will use `blog___NODE`.
success building schema - 0.282s
info Total nodes: 84, SitePage nodes: 2 (use --verbose for breakdown)
success createPages - 0.003s
success Checking for changed pages - 0.001s
success update schema - 0.013s
success onPreExtractQueries - 0.001s
success extract queries from components - 0.018s
success write out requires - 0.005s
⠹ Building development bundle
C:\Users\obrie\Documents\websites\bug-repro\node_modules\yoga-layout-prebuilt\yoga-layout\build\Release\nbind.js:53
        throw ex;
        ^

Error: EPERM: operation not permitted, lstat 'C:\Users\obrie\Documents\websites\bug-repro\.cache\caches\gatsby-source-filesystem\tmp-9155649d690ef3fba6ad1058d847a43d.jpg'
    at Object.lstatSync (fs.js:1007:3)
    at Object.lstatSync (C:\Users\obrie\Documents\websites\bug-repro\node_modules\graceful-fs\polyfills.js:308:16)
    at Object.rimrafSync (C:\Users\obrie\Documents\websites\bug-repro\node_modules\fs-extra\lib\remove\rimraf.js:237:18)
    at WriteStream.<anonymous> (C:\Users\obrie\Documents\websites\bug-repro\node_modules\gatsby-source-filesystem\create-remote-file-node.js:209:12)
    at WriteStream.emit (events.js:326:22)
    at finishMaybe (_stream_writable.js:639:14)
    at _stream_writable.js:616:5
    at WriteStream._final (internal/fs/streams.js:352:3)
    at callFinal (_stream_writable.js:609:10)
    at processTicksAndRejections (internal/process/task_queues.js:84:21)
    at runNextTicks (internal/process/task_queues.js:66:3)
    at listOnTimeout (internal/timers.js:523:9)
    at processTimers (internal/timers.js:497:7) {
  errno: -4048,
  syscall: 'lstat',
  code: 'EPERM',
  path: 'C:\\Users\\obrie\\Documents\\websites\\bug-repro\\.cache\\caches\\gatsby-source-filesystem\\tmp-9155649d690ef3fba6ad1058d847a43d.jpg'
}
PS C:\Users\obrie\Documents\websites\bug-repro>

[unimprobable]

@LekoArts I just tried it on my laptop and got the same results. Have you tried the reproduction repo?

PS C:\Users\paul\Documents\websites\gatsby\bug-repro> gatsby develop
success open and validate gatsby-configs - 0.021s
success load plugins - 0.410s
success onPreInit - 0.058s
success initialize cache - 0.028s
success copy gatsby files - 0.261s
success onPreBootstrap - 0.067s
success createSchemaCustomization - 0.004s

gatsby-source-shopify/graphql starting to fetch data from Shopify

gatsby-source-shopify/graphql fetched and processed Shop nodes: 774.040ms

gatsby-source-shopify/graphql fetched and processed ShopPolicy nodes: 845.350ms

gatsby-source-shopify/graphql fetched and processed Page nodes: 890.396ms

gatsby-source-shopify/graphql fetched and processed Blog nodes: 915.092ms

gatsby-source-shopify/graphql fetched and processed Article nodes: 992.431ms

gatsby-source-shopify/graphql fetched and processed Collection nodes: 17594.246ms
error 
error an error occurred while sourcing data
success Downloading remote files - 22.745s - 60/133 5.85/s
error "gatsby-source-shopify" threw an error while running the sourceNodes lifecycle:

failed to process https://cdn.shopify.com/s/files/1/1312/0893/products/STRAPLESS_DRESS_WITH_FLARE_SKIRT_3.jpg
RequestError: connect ETIMEDOUT 23.197.180.162:443
success Checking for changed pages - 0.001s
success source and transform nodes - 23.856s
warning Multiple node fields resolve to the same GraphQL field `ShopifyArticle.blog` - [`blog`, `blog___NODE`]. Gatsby will use `blog___NODE`.
success building schema - 1.010s
success createPages - 0.007s
success Checking for changed pages - 0.023s
success createPagesStatefully - 0.152s
success update schema - 0.042s
success write out redirect data - 0.004s
success onPostBootstrap - 0.003s
info bootstrap finished - 31.531s
success onPreExtractQueries - 0.016s
success extract queries from components - 0.400s
success write out requires - 0.027s
success run page queries - 0.100s - 2/2 19.91/s
error EPERM: operation not permitted, open 'C:\Users\paul\Documents\websites\gatsby\bug-repro\.cache\caches\gatsby-source-filesystem\tmp-d32523c9f42656800fc9b7527dd4b007.jpg'       


  Error: EPERM: operation not permitted, open 'C:\Users\paul\Documents\websites\gatsby\bug-repro\.cache\caches\gatsby-source-filesystem\tmp-d32523c9f42656800fc9b7527dd4b007.jpg'    

not finished Building development bundle - 6.378s
PS C:\Users\paul\Documents\websites\gatsby\bug-repro> gatsby develop
success open and validate gatsby-configs - 0.017s
success load plugins - 0.220s
success onPreInit - 0.051s
success initialize cache - 0.135s
success copy gatsby files - 0.208s
success onPreBootstrap - 0.033s
success createSchemaCustomization - 0.001s

gatsby-source-shopify/graphql starting to fetch data from Shopify

gatsby-source-shopify/graphql fetched and processed Blog nodes: 331.818ms

gatsby-source-shopify/graphql fetched and processed Shop nodes: 337.120ms

gatsby-source-shopify/graphql fetched and processed Page nodes: 344.456ms

gatsby-source-shopify/graphql fetched and processed ShopPolicy nodes: 369.235ms

gatsby-source-shopify/graphql fetched and processed Article nodes: 426.647ms
error 
error an error occurred while sourcing data
success Downloading remote files - 30.489s - 83/133 4.36/s
error "gatsby-source-shopify" threw an error while running the sourceNodes lifecycle:

failed to process https://cdn.shopify.com/s/files/1/1312/0893/collections/rawpixel-com-196464.jpg
Error: EPERM: operation not permitted, open 'C:\Users\paul\Documents\websites\gatsby\bug-repro\.cache\caches\gatsby-source-filesystem\tmp-b1b40786111a5756e9f1778aa127e423.jpg'      
success Checking for changed pages - 0.001s
success source and transform nodes - 31.080s
warning Multiple node fields resolve to the same GraphQL field `ShopifyArticle.blog` - [`blog`, `blog___NODE`]. Gatsby will use `blog___NODE`.
error EPERM: operation not permitted, open 'C:\Users\paul\Documents\websites\gatsby\bug-repro\.cache\caches\gatsby-source-filesystem\tmp-088f9d72874ce13963c4e21d306a200a.jpg'       


  Error: EPERM: operation not permitted, open 'C:\Users\paul\Documents\websites\gatsby\bug-repro\.cache\caches\gatsby-source-filesystem\tmp-088f9d72874ce13963c4e21d306a200a.jpg'    

not finished building schema - 0.670s
error 
   ╭───────────────────────────────────────╮
   │                                       │
   │   Update available 2.15.1 → 2.19.1    │
   │   Run npm i -g gatsby-cli to update   │
   │                                       │
   ╰───────────────────────────────────────╯

PS C:\Users\paul\Documents\websites\gatsby\bug-repro> 

system info

  System:
    OS: Windows 10 10.0.18363
    CPU: (4) x64 Intel(R) Core(TM) i7-6600U CPU @ 2.60GHz
  Binaries:
    Node: 12.18.4 - C:\Program Files\nodejs\node.EXE
    npm: 6.14.6 - C:\Program Files\nodejs\npm.CMD
  Languages:
    Python: 3.8.5
  Browsers:
    Chrome: 88.0.4324.182
    Edge: Spartan (44.18362.449.0)
  npmPackages:
    gatsby: ^2.26.1 => 2.26.1 
    gatsby-source-shopify: ^3.10.0 => 3.10.0 
  npmGlobalPackages:
    gatsby-cli: 2.15.1

[unimprobable]

@LekoArts I was able to get Gatsby develop working again after downgrading gatsby-cli to 2.17.0. I tried each major release from 2.19.2 backwards to 2.17.0. Do you know of any issues with the gatsby-cli recent releases?

[unimprobable]

@LekoArts Unfortunately this issue persists after going away for a bit. I started a new site and proceeded to build up again, and still run into this issue. I tried downgrading node to 10.22.1 to see if that is the problem. No luck, still have EPERM issues.