feat: allow overriding withCDN via env-var

[SgtPooki]

> npm run build && WITH_CDN=true npx filecoin-pin add ~/Downloads/images/IMG_1341.png


> [email protected] build
> tsc

┌  Filecoin Pin Add
│
◇  ✓ File validated (17.8 MB)
│
◇  ✓ Connected to calibration
│
◇  ✓ Payment capacity verified
│
◇  ✓ File packed with root CID: bafybeierfvvhvb73ntvschtheysvpscpynjuxgsi7jtst4wg3n3jittuca
│
◇  ✓ IPFS content loaded (17.8 MB)
│
◇  ✓ Payment capacity verified
│
◇  ✓ Storage context ready
│
◇  ━━━ Add Complete ━━━
│
│  Network: calibration
│  
│  Add Details
│    File: /Users/sgtpooki/Downloads/js-colo-media/IMG_1341.png
│    Size: 17.8 MB
│    Root CID: bafybeierfvvhvb73ntvschtheysvpscpynjuxgsi7jtst4wg3n3jittuca
│  
│  Filecoin Storage
│    Piece CID: bafkzcibfqh67ubquflgzvzb76hayv6o4yqer2wpa7susa6xna3yfktvb5zxv37nbvqbq
│    Piece ID: 0
│    Data Set ID: 244
│  
│  Storage Provider
│    Provider ID: 3
│    Name: ezpdpz-calib
│    Direct Download URL: https://calib.ezpdpz.net/piece/bafkzcibfqh67ubquflgzvzb76hayv6o4yqer2wpa7susa6xna3yfktvb5zxv37nbvqbq
│  
│  Transaction
│    Hash: 0x88314b2d3f576fb0a248f82be0971cc77a4b68ad67acc3df4db013058ff6538e
│
└  Add completed successfully

> npx filecoin-pin data-set 244

┌  Filecoin Onchain Cloud Data Sets
│
◇  ━━━ Data Sets ━━━
│
│  Data Set #244 • live
│    Managed by Warm Storage: yes
│    CDN add-on: enabled
│    Pieces stored: 1
│    Leaf count: 588804
│    Total size: 18.0 MB
│    Client data set ID: 3
│    PDP rail ID: 485
│    CDN rail ID: 487
│    Cache-miss rail ID: 486
│    Payer: 0x44f08D1beFe61255b3C3A349C392C560FA333759
│    Payee: 0xa3971A7234a3379A1813d9867B531e7EeB20ae07
│    Service provider: 0xa3971A7234a3379A1813d9867B531e7EeB20ae07
│    Provider: ezpdpz-calib (ID 3)
│    Commission: 0.00%
│  
│  Provider Service
│    Service URL: https://calib.ezpdpz.net
│    Min piece size: 1.0 KB
│    Max piece size: 32.0 GB
│    Storage price: < 0.0001 USDFC/TiB/month
│    Min proving period: 30 epochs
│    Location: unknown
│    Payment token: USDFC (native)
│  
│  Metadata
│      source: filecoin-pin
│      withCDN: (empty)
│      withIPFSIndexing: (empty)
│  
│  
│  Pieces
│    Total pieces: 1
│    Unique CommPs: 1
│    Unique root CIDs: 1
│  
│    #0
│      CommP: bafkzcibfqh67ubquflgzvzb76hayv6o4yqer2wpa7susa6xna3yfktvb5zxv37nbvqbq
│      Root CID: bafybeierfvvhvb73ntvschtheysvpscpynjuxgsi7jtst4wg3n3jittuca
│
└  Data set inspection complete

[SgtPooki]

self review

[jennijuju]

if we were going to add this. 2 TODOs needs to be considered before this is merged is:

• the cost of with cdn add on is different, we will need to account for it filecoin-pin payment deposit. I won’t suggest doing this change now because filcdn is changing how they are billing (transit from a fixed fee to an egress based fee)
• filecoin-pin get should fetch from filcdn link first if cdn service is purchased

[SgtPooki]

@jennijuju I could add a warning in here during add/import to show users that withCDN isn't fully priced throughout the codebase and link to a tracking issue. Would you like me to do that so we don't miss those follow-up actions?

[bajtos]

The first version looks great to me! I would like filecoin-pin to print the FilBeam retrieval URL when CDN is enabled, but that can be added later.

I did not realise there are implications for filecoin-pin payment deposit. Showing a warning to users seems reasonable enough to me. 👍🏻

[SgtPooki]

opened up todos: #61 and have a warning with e687b57 (#55):

> npm run build && WITH_CDN=true npx filecoin-pin add ~/Downloads/js-colo-media/71713366785__2CBE4B4C-4404-470F-AEAD-DFF56D011409.png 

> [email protected] build
> tsc

┌  Filecoin Pin Add
│
▲  CDN Pricing Notice
│
│  
│  Filecoin-pin currently does not support CDN pricing in payment calculations.
│  
│  This means:
│    • Deposit calculations may not be accurate for CDN storage
│    • You may need additional USDFC deposits for CDN-enabled uploads
│    • Filcdn is transitioning to egress-based billing (from fixed fees)
│  
│
◇  Do you want to proceed with CDN-enabled upload?
│  No
└  Add cancelled

Add failed: CDN pricing limitations warning cancelled   

and for car import:

> [email protected] build
> tsc

┌  Filecoin Pin CAR Import
│
▲  CDN Pricing Notice
│
│  
│  Filecoin-pin currently does not support CDN pricing in payment calculations.
│  
│  This means:
│    • Deposit calculations may not be accurate for CDN storage
│    • You may need additional USDFC deposits for CDN-enabled uploads
│    • Filcdn is transitioning to egress-based billing (from fixed fees)
│  
│
◇  Do you want to proceed with CDN-enabled upload?
│  No
└  Import cancelled

Import failed: CDN pricing limitations warning cancelled

[BigLep]

I think this environment variable option for now is fine. It unblocks being able to test Filecoin Beam with filecoin-pin data.

What we really need though is a followup issue about adding "withCdn" support in general. This means exposing i through the CLI as an config option, spitting out filecoinbeam URLs, etc. plus handling the payment issues highlighted in #61.

I will get that issue created later today.

[bajtos]

Awesome, thank you @SgtPooki! ❤️

[rvagg]

Quick mention in here (without looking at this), that withCDN changes the pricing. So far we've been avoiding the complexity of different prices for storage by just avoiding withCDN entirely. Now that it's in scope we need to think about how to present the pricing info, the optionality and how to adjust to user request to "upgrade".
Ideally this would be a simple add-on, perhaps even a default so you automatically sign up for your own custom IPFS gateway, but we need to prove this out first.

[rvagg]

I see I'm kind of duplicating Jen's comment about pricing already .. so my drive-by probably isn't very helpful here!

[SgtPooki]

@rvagg so are you good with us merging this so @bajtos can use and test (likely along with provider override)?

[rvagg]

ok, I don't love the optionality in envToBool but whatever