Max Record LifeTime Set To 36 Hours, Bug?

[bonedaddy]

While working on a go-libp2p-kad-dht fork I noticed this line of code within the records.go file that says the max record age is 36 hours.

Throughout other parts of the codebase, such as the the providers.go file, provide validity is set to 24 hours. Additionally from conversations over irc and discord, I was under the assumption that the max record age is 24 hours.

For example in handlers.go we perform a check against MaxRecordAge to determine whether or not we should delete old records. With the current MaxRecordAge, instead of invalidating records after 24 hours, we will do it after 36 hours.

Is this a bug, or an undocumented edge-case?

[aschmahmann]

Good question, I'm not sure what the reason is behind the different durations. However, it's worth noting that provider records are different than regular DHT records and so it's not implausible that they would have different max record times.

For example, a DHT server node may have many provider records for a given entry, but only a single regular record for a given entry.

[bonedaddy]

Good point, one of the things that confused me was the comment in records.go.

With IPNS, records are published with a default eol of 24 hours. However as per the comment linked above, if the IPNS record is around in local storage, it wont be removed from the dht storage until 36 hours after. This also means that if we publish an IPNS record with an eol of 72 hours, it will be removed from dht storage after 36 hours, not 72 hours.

[aarshkshah1992]

@postables Just some background into this. We use Kad-Dht to store two "types" of records:

1. Provider Records: These are (Hash, List of Providers for that hash) kind of records which are added to the Dht via the dht.Provide RPC call. They are essentially used for file sharing & my suspicion is that we actively expire them after 24 hours because the Kad-Dht paper mentions an expiry of 24 hours for file sharing records.

2. PutValue records: These are (Key, Value) kind of records which are added to the Dht via the PutValue RPC call. I think IPNS uses this to store name records. Though we have a MaxAge of 36 hours for this, note that we don't "actively" remove these records after 36 hours. They are removed only if someone asks for them after their expiry. We have a PR in the works to fix this.

[Stebalien]

#397 (comment)

This is correct.

Basically, there are two concepts here:

• max record age -- All DHT records (go-libp2p-record, not provider "records") are (supposed to be) expunged 36 hours after they're received. This is to ensure that the DHT doesn't fill up with useless records that nobody cares about. If someone cares about the record, they need to rebroadcast it.
• validity -- IPNS records are invalid after their EOL. That is, the record itself is no longer valid and can't be rebroadcast.

Note: There are currently two supported record types. The other is the "public key" record type. Public keys:

1. Have the same max record age as IPNS records. If you want peers to be able to find your public key, you have to rebroadcast it every 36 hours (likely more frequently).
2. Are always valid.

[Stebalien]

@postables can you think of a good way to document this? I agree it's a confusing distinction.

[bonedaddy]

@Stebalien Not sure to be honest, I think maybe just documenting the two different record types in th readme, and then in the code comments telling people to look at the readme?

[simonbonnefoy]

How does it affect the query of a file on IPFS? My plan is to use a private IPFS network to share files. However, I need the files to be available for download anytime. Does it mean that with this lifetime limit, after 24 hours, the files will not be referenced anymore by the DHT, and that nobody can retrieve them?

Is there a solution that can be used on the node side to gurarantee the availability of the data?