SPX v1.17.0-rc3 - Testing & Call Outs #9057

rjan90 · 2022-07-19T06:59:49Z

rjan90
Jul 19, 2022
Maintainer

⚠️ This is for SPX testing purposes - features here may not be production ready!

Hey! After a lot of dependency fixes in the v1.17.0 release train, v1.17.0-rc3 is finally out and ready for broader testing. This version includes a lot of goodies and we will focus on some markets (legacy) testing, sealing- and new scheduler features in this assignment.

We know a lot of the people in SPX-group are using Boost on mainnet, but we currently do not have a great overview if you are using it on calibnet. Please flag if you are not able to do the legacy deals-related testing on calibnet due to using Boost.

If you aren't able to join this test assignment, please reach out to @jennijuju or me on Slack!

General assignment:

🔁 If lotus daemon stays in sync
🦭If you have successfully sealed some new sectors.
🤝If you have successfully made some storage deals.
🔥 If you have successfully made some snap deals.
📂 If you have successfully had some retrieval deals.
💻 If you passed your first WindowPoSt.

Disable the [IndexProvider] in the lotus-miner config.toml-file and restart the lotus-miner and check that:

✅ the lotus-miner is able to run with the IndexProvider disabled.

New Features assignment:

Please add a maximum task limit for any sealing task on a lotus-worker, and check that the limits are enforced. Documentation about how to limit the amount of tasks run in parallel on a lotus-workercan be found here: https://lotus.filecoin.io/storage-providers/seal-workers/seal-workers/#limit-tasks-run-in-parallel
✅ If your lotus-worker task limits where enforced.
If you have multiple workers, please test out the new spread assigner logic: https://lotus.filecoin.io/storage-providers/advanced-configurations/sealing/#worker-assigning-logic.

Please elaborate if you feel there are some issues with it, or if it feels like it assigns the workers correctly (assigning tasks to as many distinct workers as possible)

Questions are welcomed in this discussion, the team will be monitoring this discussion and check-in regularly.

Issues can be reported in https://github.com/filecoin-project/lotus/issues.

❗ If you think the v1.17.0-rc3 branch is not working well, keep all the logs for describing the issue and revert back to the stable v1.16.0 or v1.16.1 release.

rjan90 · 2022-07-19T07:06:51Z

rjan90
Jul 19, 2022
Maintainer Author

Phi-rjan´s report card:

General assignment:

🔁 If lotus daemon stays in sync
🦭If you have successfully sealed some new sectors.
🤝If you have successfully made some storage deals.
🔥 If you have successfully made some snap deals.
📂 If you have successfully had some retrieval deals.
💻 If you passed your first WindowPoSt.

Disable the [IndexProvider] in the lotus-miner config.toml-file and restart the lotus-miner and check that:

✅ the lotus-miner is able to run with the IndexProvider disabled.

New Features assignment:

Please add a maximum task limit for any sealing task on a lotus-worker, and check that the limits are enforced. Documentation about how to limit the amount of tasks run in parallel on a lotus-workercan be found here: https://lotus.filecoin.io/storage-providers/seal-workers/seal-workers/#limit-tasks-run-in-parallel

✅ If your lotus-worker task limits where enforced.

Limited PC1 to 1 and pledged two sectors:

root@Ubuntu-2004-focal-64-minimal ~ # lotus-miner sealing workers
Worker d66c30c3-0927-433c-bd29-55a202bcf78b, host Ubuntu-2004-focal-64-minimal
        CPU:  [                                                                ] 0/12 core(s) in use
        RAM:  [||                                                              ] 3% 1.995 GiB/62.79 GiB
        VMEM: [|                                                               ] 1% 2.814 GiB/190.8 GiB
Worker d90bc9d5-2b02-4a02-9647-56cc70536236, host Ubuntu-2004-focal-64-minimal
        TASK: PC1(1/1) 
        CPU:  [|||||                                                           ] 1/12 core(s) in use
        RAM:  [|||                                                             ] 3% 3.042 GiB/62.79 GiB
        VMEM: [|                                                               ] 1% 3.034 GiB/190.8 GiB

In the lotus-miner sealing jobs the second one is listed as prepared:

lotus-miner sealing jobs
ID        Sector  Worker    Hostname                      Task  State     Time
74568537  11      d90bc9d5  Ubuntu-2004-focal-64-minimal  PC1   running   10.9s
00000000  12      d90bc9d5  Ubuntu-2004-focal-64-minimal  PC1   prepared  4.8s

If you have multiple workers, please test out the new spread assigner logic: https://lotus.filecoin.io/storage-providers/advanced-configurations/sealing/#worker-assigning-logic.

Please elaborate if you feel there are some issues with it, or if it feels like it assigns the workers correctly (assigning tasks to as many distinct workers as possible)

0 replies

tmyuu · 2022-07-19T09:26:24Z

tmyuu
Jul 19, 2022

livspect´s report card:

General assignment:

🔁 If lotus daemon stays in sync
🦭If you have successfully sealed some new sectors.
🤝If you have successfully made some storage deals.
🔥 If you have successfully made some snap deals.
📂 If you have successfully had some retrieval deals.
💻 If you passed your first WindowPoSt.
✅ the lotus-miner is able to run with the IndexProvider disabled.

New Features assignment:

✅ If your lotus-worker task limits where enforced.

env

PC1_32G_MAX_CONCURRENT=3

Limited to 3 tasks.

ID        Sector  Worker    Hostname  Task  State    Time
1e534f59  2065    4879ace0  nodeman1  PC1   running  52m15.6s
6195da34  2068    4879ace0  nodeman1  PC1   running  52m15.6s
69fb7dec  2069    4879ace0  nodeman1  PC1   running  52m15.6s

getting an error with boost v1.2.0:

2022/07/19 09:14:31 proto: duplicate proto type registered: pb.NoiseHandshakePayload
Error: getting full node api: Remote API version didn't match (expected 2.2.0, remote 2.3.0)

✅ spread assigner logic

$ lotus-miner sealing jobs
ID        Sector  Worker    Hostname  Task  State    Time
83349f6a  2070    3037eefa  nodeman1  AP    running  11.4s
0f0febd9  2071    7a5bd3d5  nodeman2  AP    running  10.1s
cffdc4d9  2072    3037eefa  nodeman1  AP    running  9.5s
75714d1a  2073    7a5bd3d5  nodeman2  AP    running  8.4s
e85a3860  2074    3037eefa  nodeman1  AP    running  6.2s
7e4d6d61  2075    7a5bd3d5  nodeman2  AP    running  800ms

3 replies

rjan90 Jul 20, 2022
Maintainer Author

Hey @tmyuu!

This Boost-branch should work with v1.17.0-rc3 and fix the error you are getting on Boost: https://github.com/filecoin-project/boost/tree/feat/lotus-v1.17.0-rc3

tmyuu Jul 21, 2022

Hi @rjan90
Thank you for teaching me!

$ boostd -v
boostd version 1.2.0+git.e4a20a3

rjan90 Jul 21, 2022
Maintainer Author

Thanks for reporting the issue 😄

benjaminh83 · 2022-07-20T12:28:21Z

benjaminh83
Jul 20, 2022

BenjaminH's report card:

General assignment:

🔁 If lotus daemon stays in sync
🦭If you have successfully sealed some new sectors.
🤝If you have successfully made some storage deals.
🔥 If you have successfully made some snap deals.
📂 If you have successfully had some retrieval deals.
💻 If you passed your first WindowPoSt.
✅ the lotus-miner is able to run with the IndexProvider disabled.

New Features assignment:

✅ If your lotus-worker task limits where enforced.

1 reply

benjaminh83 Jul 23, 2022

$ boostd --version
boostd version 1.3.0-rc1+git.b85d1b9

cryptowhizzard · 2022-07-20T15:06:57Z

cryptowhizzard
Jul 20, 2022

Cryptowhizzard's report card:

General assignment:

🔁 If lotus daemon stays in sync
🦭If you have successfully sealed some new sectors.
🤝If you have successfully made some storage deals.
🔥 If you have successfully made some snap deals.
📂 If you have successfully had some retrieval deals.
💻 If you passed your first WindowPoSt.
✅ the lotus-miner is able to run with the IndexProvider disabled.

New Features assignment:

✅ If your lotus-worker task limits where enforced.

0 replies

Reiers · 2022-07-21T08:57:52Z

Reiers
Jul 21, 2022

Reiers´s report card:

General assignment:

🔁 If lotus daemon stays in sync
🦭If you have successfully sealed some new sectors.
🤝If you have successfully made some storage deals.
🔥 If you have successfully made some snap deals.
📂 If you have successfully had some retrieval deals.
💻 If you passed your first WindowPoSt.

Disable the [IndexProvider] in the lotus-miner config.toml-file and restart the lotus-miner and check that:

✅ the lotus-miner is able to run with the IndexProvider disabled.

0 replies

neondragon · 2022-07-21T19:28:28Z

neondragon
Jul 21, 2022

Neonix's report card:

General assignment:

🔁 If lotus daemon stays in sync
🦭If you have successfully sealed some new sectors.
🤝If you have successfully made some storage deals.
🔥 If you have successfully made some snap deals.
📂 If you have successfully had some retrieval deals.
💻 If you passed your first WindowPoSt.
✅ the lotus-miner is able to run with the IndexProvider disabled.

New Features assignment:

✅ If your lotus-worker task limits where enforced.

1 reply

neondragon Jul 21, 2022

PC1_64G_MAX_CONCURRENT=3

# lotus-miner sealing jobs
ID        Sector  Worker    Hostname                               Task  State     Time
0d7440c7  1569    1055b194  gb-wlv1-filecoin-seal2.corp.neonix.io  PC1   running   19m21.2s
7eb898bf  1570    1055b194  gb-wlv1-filecoin-seal2.corp.neonix.io  PC1   running   6m37.6s
5074d354  1572    1055b194  gb-wlv1-filecoin-seal2.corp.neonix.io  PC1   running   6m35s
00000000  1571    1055b194  gb-wlv1-filecoin-seal2.corp.neonix.io  PC1   prepared  5m59.7s

# lotus-miner sealing workers
Worker 1055b194-51c9-425c-a623-28587fa792ef, host gb-wlv1-filecoin-seal2.corp.neonix.io
	TASK: PC1(3/3)
	CPU:  [||||||||||||                                                    ] 12/64 core(s) in use
	RAM:  [|||||||||||||||||||||||||||||||||||||||||||                     ] 66% 362.3 GiB/503.8 GiB
	VMEM: [|||||||||||||||||||||||||||||||||||||||||||||||||               ] 76% 384 GiB/503.8 GiB

SBudo · 2022-07-21T22:49:29Z

SBudo
Jul 21, 2022

Stephane's report card:

General assignment:

🔁 If lotus daemon stays in sync
🦭If you have successfully sealed some new sectors.
🤝If you have successfully made some storage deals.
🔥 If you have successfully made some snap deals.
📂 If you have successfully had some retrieval deals.
💻 If you passed your first WindowPoSt.
✅ the lotus-miner is able to run with the IndexProvider disabled.

New Features assignment:

✅ If your lotus-worker task limits where enforced.

0 replies

Angelo-gh3990 · 2022-07-25T08:36:35Z

Angelo-gh3990
Jul 25, 2022

Angelo´s report card:

General assignment:

🔁 If lotus daemon stays in sync
🦭If you have successfully sealed some new sectors.
🤝If you have successfully made some storage deals.
🔥 If you have successfully made some snap deals.
📂 If you have successfully had some retrieval deals.
💻 If you passed your first WindowPoSt.

Disable the [IndexProvider] in the lotus-miner config.toml-file and restart the lotus-miner and check that:

✅ the lotus-miner is able to run with the IndexProvider disabled.

0 replies

namoo1 · 2022-08-17T04:08:27Z

namoo1
Aug 17, 2022

Meest NAM

🔁 If lotus daemon stays in sync
🦭If you have successfully sealed some new sectors.
🤝If you have successfully made some storage deals.
🔥 If you have successfully made some snap deals.
📂 If you have successfully had some retrieval deals.
💻 If you passed your first WindowPoSt.
✅ the lotus-miner is able to run with the IndexProvider disabled.
New Features assignment:
✅ If your lotus-worker task limits where enforced.

Daemon:  1.17.0+mainnet+git.2830429ad+api1.5.0
Local: lotus-miner version 1.17.0+mainnet+git.2830429ad

 boost --version
boost version 1.3.0+git.d9e9ab7

0 replies

SPX v1.17.0-rc3 - Testing & Call Outs #9057

Uh oh!

rjan90 Jul 19, 2022 Maintainer

Replies: 9 comments · 5 replies

Uh oh!

Uh oh!

rjan90 Jul 19, 2022 Maintainer Author

Phi-rjan´s report card:

Uh oh!

Uh oh!

livspect´s report card:

Uh oh!

rjan90 Jul 20, 2022 Maintainer Author

Uh oh!

Uh oh!

rjan90 Jul 21, 2022 Maintainer Author

Uh oh!

Uh oh!

BenjaminH's report card:

Uh oh!

Uh oh!

Uh oh!

Cryptowhizzard's report card:

Uh oh!

Reiers´s report card:

Uh oh!

Uh oh!

Neonix's report card:

Uh oh!

Uh oh!

Uh oh!

Stephane's report card:

Uh oh!

Uh oh!

Angelo´s report card:

Uh oh!

rjan90
Jul 19, 2022
Maintainer

Replies: 9 comments 5 replies

rjan90
Jul 19, 2022
Maintainer Author

rjan90 Jul 20, 2022
Maintainer Author

rjan90 Jul 21, 2022
Maintainer Author