Computing Seal Proof Failed: storage call error 0: %!w(Bad file descriptor (os error 9)

[superxii]

Hi, I am the beginner to the lotus.

I don't know why all the sectors are retrying between PC1 and PC2.
Right after the PC2, there is an error on

event;sealing.SectorComputeProofFailed]
computing seal proof failed(2): storage call error 0: %!w(Bad file descriptor (os error 9)

---

Event Log:
0. 2022-06-27 15:17:28 +0800 HKT: [event;sealing.SectorStartCC] {"User":{"ID":17,"SectorType":8}}

1. 2022-06-27 15:55:06 +0800 HKT: [event;sealing.SectorPacked] {"User":{"FillerPieces":[{"Size":34359738368,"PieceCID":{"/":"baga6ea4seaqao7s73y24kcutaosvacpdjgfe5pw76ooefnyqw4ynr3d2y6x2mpq"}}]}}
2. 2022-06-27 15:55:06 +0800 HKT: [event;sealing.SectorTicket] {"User":{"TicketValue":"HTcRtZQJkHGSZDmRhLIz8D8rWzfSkl69o1KptiB1NIY=","TicketEpoch":1932770}}
3. 2022-06-27 20:50:31 +0800 HKT: [event;sealing.SectorPreCommit1] {"User":{"PreCommit1Out":"eyJfbG90dXNfU2VhbFJhbmRvbW5lc3MiOiJIVGNSdFpRSmtIR1NaRG1SaExJejhEOHJXemZTa2w2OW8xS3B0aUIxTklZPSIsImNvbW1fZCI6WzcsMTI2LDk1LDIyMiw1MywxOTcsMTAsMTQ3LDMsMTY1LDgwLDksMjI3LDczLDEzOCw3OCwxOTAsMjIzLDI0MywxNTYsNjYsMTgzLDE2LDE4Myw0OCwyMTYsMjM2LDEyMiwxOTksMTc1LDE2Niw2Ml0sImNvbmZpZyI6eyJpZCI6InRyZWUtZCIsInBhdGgiOiIvaG9tZS91c2VyL0Rlc2t0b3AvbG90dXNfZGF0YV9zdG9yYWdlL2NhY2hlL3MtdDAxODY1NDk0LTE3Iiwicm93c190b19kaXNjYXJkIjo3LCJzaXplIjoyMTQ3NDgzNjQ3fSwibGFiZWxzIjp7IlN0YWNrZWREcmczMkdpQlYxIjp7Il9oIjpudWxsLCJsYWJlbHMiOlt7ImlkIjoibGF5ZXItMSIsInBhdGgiOiIvaG9tZS91c2VyL0Rlc2t0b3AvbG90dXNfZGF0YV9zdG9yYWdlL2NhY2hlL3MtdDAxODY1NDk0LTE3Iiwicm93c190b19kaXNjYXJkIjo3LCJzaXplIjoxMDczNzQxODI0fSx7ImlkIjoibGF5ZXItMiIsInBhdGgiOiIvaG9tZS91c2VyL0Rlc2t0b3AvbG90dXNfZGF0YV9zdG9yYWdlL2NhY2hlL3MtdDAxODY1NDk0LTE3Iiwicm93c190b19kaXNjYXJkIjo3LCJzaXplIjoxMDczNzQxODI0fSx7ImlkIjoibGF5ZXItMyIsInBhdGgiOiIvaG9tZS91c2VyL0Rlc2t0b3AvbG90dXNfZGF0YV9zdG9yYWdlL2NhY2hlL3MtdDAxODY1NDk0LTE3Iiwicm93c190b19kaXNjYXJkIjo3LCJzaXplIjoxMDczNzQxODI0fSx7ImlkIjoibGF5ZXItNCIsInBhdGgiOiIvaG9tZS91c2VyL0Rlc2t0b3AvbG90dXNfZGF0YV9zdG9yYWdlL2NhY2hlL3MtdDAxODY1NDk0LTE3Iiwicm93c190b19kaXNjYXJkIjo3LCJzaXplIjoxMDczNzQxODI0fSx7ImlkIjoibGF5ZXItNSIsInBhdGgiOiIvaG9tZS91c2VyL0Rlc2t0b3AvbG90dXNfZGF0YV9zdG9yYWdlL2NhY2hlL3MtdDAxODY1NDk0LTE3Iiwicm93c190b19kaXNjYXJkIjo3LCJzaXplIjoxMDczNzQxODI0fSx7ImlkIjoibGF5ZXItNiIsInBhdGgiOiIvaG9tZS91c2VyL0Rlc2t0b3AvbG90dXNfZGF0YV9zdG9yYWdlL2NhY2hlL3MtdDAxODY1NDk0LTE3Iiwicm93c190b19kaXNjYXJkIjo3LCJzaXplIjoxMDczNzQxODI0fSx7ImlkIjoibGF5ZXItNyIsInBhdGgiOiIvaG9tZS91c2VyL0Rlc2t0b3AvbG90dXNfZGF0YV9zdG9yYWdlL2NhY2hlL3MtdDAxODY1NDk0LTE3Iiwicm93c190b19kaXNjYXJkIjo3LCJzaXplIjoxMDczNzQxODI0fSx7ImlkIjoibGF5ZXItOCIsInBhdGgiOiIvaG9tZS91c2VyL0Rlc2t0b3AvbG90dXNfZGF0YV9zdG9yYWdlL2NhY2hlL3MtdDAxODY1NDk0LTE3Iiwicm93c190b19kaXNjYXJkIjo3LCJzaXplIjoxMDczNzQxODI0fSx7ImlkIjoibGF5ZXItOSIsInBhdGgiOiIvaG9tZS91c2VyL0Rlc2t0b3AvbG90dXNfZGF0YV9zdG9yYWdlL2NhY2hlL3MtdDAxODY1NDk0LTE3Iiwicm93c190b19kaXNjYXJkIjo3LCJzaXplIjoxMDczNzQxODI0fSx7ImlkIjoibGF5ZXItMTAiLCJwYXRoIjoiL2hvbWUvdXNlci9EZXNrdG9wL2xvdHVzX2RhdGFfc3RvcmFnZS9jYWNoZS9zLXQwMTg2NTQ5NC0xNyIsInJvd3NfdG9fZGlzY2FyZCI6Nywic2l6ZSI6MTA3Mzc0MTgyNH0seyJpZCI6ImxheWVyLTExIiwicGF0aCI6Ii9ob21lL3VzZXIvRGVza3RvcC9sb3R1c19kYXRhX3N0b3JhZ2UvY2FjaGUvcy10MDE4NjU0OTQtMTciLCJyb3dzX3RvX2Rpc2NhcmQiOjcsInNpemUiOjEwNzM3NDE4MjR9XX19LCJyZWdpc3RlcmVkX3Byb29mIjoiU3RhY2tlZERyZzMyR2lCVjFfMSJ9"}}
4. 2022-06-27 23:19:42 +0800 HKT: [event;sealing.SectorPreCommit2] {"User":{"Sealed":{"/":"bagboea4b5abcalqzsyswbhdnaimnhrpqje4tbleqphmy7z3ojw3iunx6xctfevk7"},"Unsealed":{"/":"baga6ea4seaqao7s73y24kcutaosvacpdjgfe5pw76ooefnyqw4ynr3d2y6x2mpq"}}}
5. 2022-06-27 23:19:42 +0800 HKT: [event;sealing.SectorPreCommitBatch] {"User":{}}
6. 2022-06-28 11:07:35 +0800 HKT: [event;sealing.SectorPreCommitBatchSent]{"User":{"Message":{"/":"bafy2bzacebjxiivukt42ilksza3v44z44m7ytpag4lkl2s22h5dkjn7bdbaw4"}}}
7. 2022-06-28 11:35:30 +0800 HKT: [event;sealing.SectorPreCommitLanded] {"User":{"TipSet":"AXGg5AIgmYKMykrMN2SvQ6q5kK9CdKO3PPJQPHjr7X4VS9MPcaABcaDkAiApciPKB5+CgrEHPaA5/2U5u2hkq6Va3C42YJruPuOAPgFxoOQCIGtQH7tyJ5EYk51Jr0kjEhnquV8/6onP4CseWeonX1F1AXGg5AIgtAgFjcWMRN2h7GzOgB9lSFEK+N4MYt7NOqbj3skLzbMBcaDkAiCEQ9plD4WIGc6DzYPcVorefVyY0NfHOfUr2roaa3BwXgFxoOQCIDTX2Gzi/HIWQjBuZtzrcM1CpGSoPeZDINfhgVS8FM4zAXGg5AIgnxQN1Oyv/ZcmlhijLO991tlKJas0jlVVHEtmNV8u/bYBcaDkAiBQgyOUbMr5UT8XSaIcrSuhrTler0hIBoTTcw492NwJxQFxoOQCIOTf/zbAUKgDXG1QzbJBVrOOIN+PWyDZkiVCT+tL+5r/"}}
8. 2022-06-28 12:50:29 +0800 HKT: [event;sealing.SectorSeedReady] {"User":{"SeedValue":"Kc3wzqhmNatMyKKu11eCzIZAXM0dVPnAzDIFOpJ7jSE=","SeedEpoch":1936175}}
9. 2022-06-28 12:50:47 +0800 HKT: [event;sealing.SectorComputeProofFailed]{"User":{}}
   computing seal proof failed(2): storage call error 0: %!w(Bad file descriptor (os error 9)

Stack backtrace:
0: anyhow::error::<impl core::convert::From for anyhow::Error>::from
1: storage_proofs_core::parameter_cache::read_cached_params
2: storage_proofs_core::parameter_cache::CacheableParameters::get_groth_params
3: filecoin_proofs::caches::get_stacked_params
4: filecoin_proofs::api::seal::seal_commit_phase2
5: filecoin_proofs_api::seal::seal_commit_phase2_inner
6: filecoin_proofs_api::seal::seal_commit_phase2
7: <core::panic::unwind_safe::AssertUnwindSafe as core::ops::function::FnOnce<()>>::call_once
8: ffi_toolkit::catch_panic_response
9: fil_seal_commit_phase2
10: _cgo_b4f6fc9f483f_Cfunc_fil_seal_commit_phase2
at /tmp/go-build/cgo-gcc-prolog:1641:11
11: runtime.asmcgocall
at /snap/go/9848/src/runtime/asm_amd64.s:821 [Hostname: 6400])
10. 2022-06-28 12:51:47 +0800 HKT: [event;sealing.SectorRetryComputeProof]{"User":{}}
11. 2022-06-28 12:51:50 +0800 HKT: [event;sealing.SectorComputeProofFailed]{"User":{}}
computing seal proof failed(2): storage call error 0: %!w(Bad file descriptor (os error 9)

May I know why this happening and how I can fix this issue? I am having one miner and one worker,both hosted in different PC.The sealed data are stored on a mounted disk from other PC.

One more thing, my company network is very unstable. The network may out of the services with few minutes time to time. Will this be the cause to this error?

Solution:
In my case, my disk is mounted as a network drive to the miner and worker. But due to an unstable connection, the miner and worker can't locate the drive which creates this problem. Leave this as a hint for someone encounter this problem.

[LexLuthr]

This is a little tough to answer without debugging. But error is very clear that file descriptor is closed when program tries to access it. It could be due to network issue or disk issues. You need to check where these sealing files are and on which node C2 task is running (failing). Then try to figure out what could be causing the file access to get interrupted.

[superxii]

Thank you for your answer. I believe that disk should be fine.Since the disks are new and turned into RAID-0.I may deploy the worker,miner and storage somewhere else to see if it is the network issue.

[TippyFlitsUK]

Hi @superxii

Are you still experiencing this issue?

[superxii]

No anymore, thanks. Just the disk access problem.

[TippyFlitsUK]

many thanks @superxii!! 🙏