How to fetch/scrape all the post of a particular channel?

[monk1337]

Hi, I am trying to fetch/scrape all the posts of a particular channel for research and paper writing. I searched but couldn't find any way to fetch all messages from a particular channel.
Please let me know how to proceed with this!

Thank you!

[SebbyLaw]

You can use the history method of a channel object to fetch messages in a channel as shown in the example.

Keep in mind that this will make 1 API call for every 100 messages that you fetch.

[monk1337]

@SebbyLaw Thank you for the reply. So is there any API limit? For example, if I want to fetch from a channel operating for a long time, and every day there are more than 5k+ messages, is it possible to fetch all messages in a few days, weeks or months? What is the best way possible?

[SebbyLaw]

There's no real way to know how long it will take since discord's ratelimits are dynamic. discord.py handles ratelimits automatically and will sleep in between requests when necessary.

For your particular use-case I would recommend starting from the oldest message and periodically storing the ID of the latest message fetched in case you need to re-run your script if something goes wrong. You will be able to pass that ID into the after= argument to continue where you left off.