Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8018 commits
Select commit Hold shift + click to select a range
9fd732a
[config] support setting default file type via GDL_CONFIG_TYPE
mikf Jan 31, 2026
1a8ccbe
[common] use f-string in BaseExtractor.update()
mikf Feb 1, 2026
b37acd1
[artstation] fix embedded videos (#8972)
mikf Feb 1, 2026
343981a
[common] add 'utils()' method
mikf Jan 31, 2026
7692d31
[twitter] move transaction_id.py
mikf Jan 31, 2026
20ef39b
[tsumino] export 'jsurl' code
mikf Jan 31, 2026
3d114db
[deviantart] export 'tiptap' functions
mikf Jan 31, 2026
1c2e2d5
[deviantart] export journal templates
mikf Jan 31, 2026
51d9fd2
[behance] export GraphQL queries
mikf Jan 31, 2026
40a4ff9
[500px] export GraphQL queries
mikf Jan 31, 2026
0c24955
[mangapark] export GraphQL queries
mikf Jan 31, 2026
cc64598
[luscious] export GraphQL queries
mikf Oct 21, 2025
73bf996
[scrolller] move GraphQL queries
mikf Oct 21, 2025
eed46f8
[build] update PyInstaller hiddenimports and py2exe modules
mikf Oct 25, 2025
451ed23
[discord:server-search] use 'max_id' for pagination
mikf Feb 1, 2026
efa301f
Merge branch 'res'
mikf Feb 1, 2026
56dcd00
[job] add 'init' option
mikf Feb 1, 2026
c42a5dc
[pp] add 'child' & 'child-after' events
mikf Feb 1, 2026
44e18f9
[tsumino] remove module
mikf Feb 1, 2026
f99c8f5
[xhamster] fix user profile extraction (#8974)
mikf Feb 2, 2026
17e1d25
[scrolller] add 'user' extractor (#8961)
mikf Feb 2, 2026
a2c79ba
[8chan] skip 'TOS' cookie name lookup if already present
mikf Feb 2, 2026
5c8bea3
[8chan] fail downloads of 'POW' images (#8975)
mikf Feb 2, 2026
39fb51d
[mangafire] export 'VRF' functions
mikf Feb 2, 2026
a95cf92
[twitter] add & use '_tweetid_to_datetime' method
mikf Feb 2, 2026
153e589
[twitter] implement '"search-pagination": "date"'
mikf Feb 2, 2026
87323fd
rename 'image-*' & 'chapter-*' options to 'file-*' & 'child-*'
mikf Feb 2, 2026
e37fe05
[docs/configuration] use anchors for alternative/deprecated names
mikf Feb 2, 2026
cbb9264
use tempfile when updating input files (#8981)
mikf Feb 2, 2026
a7cc447
[tiktok] fix outdated error message (#8979)
CasualYT31 Feb 2, 2026
7c8fbb6
[tiktok] do not exit early when rolling back cursor (#8968)
CasualYT31 Feb 2, 2026
e44c5bd
[tiktok] identify when user accounts do not exist (#8977)
CasualYT31 Feb 2, 2026
0c04090
[util] set flags in range predicates
mikf Feb 2, 2026
25d4547
[docs/configuration] document postprocessor /MODE and @EVENT (#8980)
mikf Feb 2, 2026
d232f0d
[gofile][pixeldrain] enable 'recursive' by default
mikf Feb 3, 2026
5ddee42
[exhentai][schalenetwork] enable 'tags' categories by default
mikf Feb 3, 2026
0be3383
[formatter] add 'q' & 'Q' conversions - URL-en/decode values
mikf Feb 3, 2026
f0f9575
[job] fix 'AttributeError' when enabling 'init' for non-DownloadJob
mikf Feb 3, 2026
2cbd974
enable 'metadata-url/-path/-parent/-extractor]' by default
mikf Feb 3, 2026
614e1a5
[twitter] implement 'metadata-user' option (#8984)
mikf Feb 3, 2026
0b49712
[tiktok] use time cursor for story requests (#8991)
CasualYT31 Feb 4, 2026
cbf8ed1
[tiktok] always try to resolve JS challenges even if retries is set t…
CasualYT31 Feb 4, 2026
f688c59
[common] add __str__ method for Extractor class
mikf Feb 4, 2026
49b2fa4
[job] fix overwriting '_extractor' (#8958)
mikf Feb 4, 2026
b4351b8
[job] more 'is None' checks
mikf Feb 4, 2026
9379397
[simpcity] extract 'tiktok' media embeds (#8994)
mikf Feb 4, 2026
9958678
[simpcity] extract 'reddit' media embeds (#8994)
mikf Feb 4, 2026
b329e6e
[xenforo] add 'attachments' & 'embeds' options
mikf Feb 4, 2026
947085c
[common] add '--xff' / 'geo-bypass' option
mikf Feb 4, 2026
3d36ee0
[common] support multiple codes/blocks for '--xff'
mikf Feb 4, 2026
42407af
[xenforo] implement '"order-posts": "reaction"' (#8997)
mikf Feb 4, 2026
09fbb3a
[imagefap] use self.groups, remove __init__
mikf Feb 5, 2026
df8aaa7
[options] add '--restrict-filenames' & '--windows-filenames'
mikf Feb 5, 2026
2ac55f4
[instagram] cache '/users/web_profile_info' results on disk (#8978)
mikf Feb 5, 2026
fdc59ef
[pixiv] fix errors when using metadata options for avatar/background
mikf Feb 5, 2026
04442e2
[artstation] download '/8k/' images (#9003)
mikf Feb 5, 2026
d3adfd6
[artstation] fix & update 'challenge' extractor
mikf Feb 5, 2026
cf6dc2e
[config] improve --config-status
mikf Feb 5, 2026
f046529
[tests:job] add tests for DataJob 'resolve'
mikf Feb 5, 2026
22b12a1
[tests:job] test 'parent-metadata' / '_extractor' handling
mikf Feb 5, 2026
71680fe
[actions] implement 'flag … = skip' (#8960)
mikf Feb 6, 2026
da2a6a8
[imhentai] use alternate strategy for galleries without image data (#…
mikf Feb 6, 2026
06da53f
[cookies] add support for Floorp (#9005)
dimomarg Feb 6, 2026
da88772
[instagram] use '/topsearch/' to fetch user information (#8978)
mikf Feb 6, 2026
40e4cc6
[common] pass job status to 'finalize()'
mikf Feb 6, 2026
7a98a93
[common] only call 'skip()' & 'finalize()' when defined
mikf Feb 6, 2026
98ef34a
[twitter] support 'article' media (#8995)
mikf Feb 6, 2026
c978fe1
[text] add 'extract_urls()' helper
mikf Feb 6, 2026
2d64e76
[job] implement 'follow' option (#8752)
mikf Feb 7, 2026
b174bb4
[job] improve 'metadata_url' handling
mikf Feb 7, 2026
c16892a
[reddit:user] provide 'user' metadata field (#8228)
mikf Feb 7, 2026
935bdb6
[reddit:user] implement 'only' option (#8228)
mikf Feb 7, 2026
8eafa15
[reddit] try to improve comment metadata (#8228)
mikf Feb 7, 2026
56f2790
[options] add '--config-type' command-line option
mikf Feb 7, 2026
5d9b607
[fikfap] allow for dash in usernames (#9019)
wise-immersion Feb 8, 2026
a8636e7
[fikfap] add 'hashtag' extractor (#9018)
wise-immersion Feb 8, 2026
d3c4328
[imagefap:user] support multiple pages (#9016)
mikf Feb 8, 2026
53747c6
[instagram] fix 'avatar' & 'info' extractors (#8978)
mikf Feb 8, 2026
e628f4c
[cookies] support Firefox 147+ profile paths (#8803)
mikf Feb 8, 2026
3cf8813
[instagram] fix errors for missing user profiles
mikf Feb 8, 2026
d77078d
[fikfap] support main page post URLs (#9026)
wise-immersion Feb 9, 2026
b769dc7
[pornpics] fix 'search' extractor pagination (#9022)
mikf Feb 8, 2026
52a5e39
[reddit:user] fix user lookup when using sub view (#8228 #9032)
mikf Feb 9, 2026
640d5f1
[fikfap] improve URL patterns
mikf Feb 10, 2026
ce8d61d
[imagefap] don't return anything for empty profiles (#9034)
mikf Feb 10, 2026
ace8c50
[imagefap] handle '/galleries?folderid=0' URLs (#9034)
mikf Feb 10, 2026
a8376f2
[instagram] add 'user-cache' option (#8978 #9025)
mikf Feb 10, 2026
d491564
[instagram] add 'user-strategy' option (#8978 #9025)
mikf Feb 10, 2026
f67b99a
[reddit] fix "KeyError: 'children'" when expanding comments (#9037)
mikf Feb 10, 2026
102f8da
[reddit] fix '/external-preview' embed downloads (#9037)
mikf Feb 10, 2026
448ec12
[tests/extractor] test 'extractor.find()' results
mikf Feb 10, 2026
04905ff
[weebdex] fix 'chapter-reverse' (#9041)
mikf Feb 11, 2026
136b7d4
[pp:ugoira] fix processing '.gif' frames
mikf Feb 11, 2026
e3868fb
[options] use 'yuv420p' output pixel format for ugoira presets
mikf Feb 11, 2026
09dbdee
[options] use '"ugoira": "original"' by default
mikf Feb 11, 2026
fc589e9
[simpcity] fix 'gofile' links (#9042)
mikf Feb 11, 2026
12f5e24
use sets for ' in { ... }' checks
mikf Feb 11, 2026
f7f2584
[foolfuuka] use filter to skip posts missing media
NecRaul Feb 12, 2026
1744eb0
[foolfuuka] early resolution for wsg/gif boards
NecRaul Feb 12, 2026
0fb5ce6
[xenforo] fix 'IndexError' when extracting attachments (#9046)
mikf Feb 12, 2026
eb4e444
[util] implement 'build_duration_func_ex()'
mikf Feb 12, 2026
0f41f34
implement linear/exponential backoff for 'sleep-429'
mikf Feb 12, 2026
84b5b1e
[koofer] include '{hash}' in default filenames
mikf Feb 12, 2026
df7642e
[foolfuuka] simplify
mikf Feb 12, 2026
866e6df
merge #9047: [foolfuuka] improve media link resolution
mikf Feb 12, 2026
34e402a
[koofr] improve subdirectory handling - re-add 'num' & 'count'
mikf Feb 12, 2026
e6f2e31
[pp:ugoira] disable 'mkvmerge' output, add 'mkvmerge-output' option
mikf Feb 12, 2026
d2477a9
[options] add 'sleep-retries' option
mikf Feb 13, 2026
f1da162
[common] include duration in 'wait()' output
mikf Feb 13, 2026
d99c8c1
[manganelo] fix 'manga' extractor (#9059)
mikf Feb 14, 2026
98eb857
[pp:exec] use non-UNC path replacements (#8879)
mikf Feb 14, 2026
cc52986
[config] fix 'JSONDecodeError' when using non-JSON config types (#9063)
mikf Feb 14, 2026
e9989ff
[options] add '--config-json' command-line option
mikf Feb 14, 2026
a33b9e5
[chevereto] extract 'title' metadata (#9061)
mikf Feb 14, 2026
1c8d078
[chevereto] improve password requirement detection
mikf Feb 14, 2026
41998cb
[chevereto] combine 'image' & 'video' extractors into a 'file' extractor
mikf Feb 14, 2026
01cb378
[imagepond] support '/i/' URLs, update root domain
mikf Feb 14, 2026
b552cdb
[pholder] add support (#2568 #9067)
Amar1729 Feb 15, 2026
53cdfaa
[common] add reference to 'exception' module to Extractor class
mikf Feb 14, 2026
5548cff
[eporner] include 'www' subdomain in 'root' domain (#9064)
mikf Feb 14, 2026
a51cd7d
[eporner] detect 'Age Verification' pages (#9064)
mikf Feb 14, 2026
7602f2b
[pholder] restore 'subredditTitle' (#9067)
mikf Feb 15, 2026
1cf284e
[xenforo] fix attachments extraction (#9070)
mikf Feb 16, 2026
68facec
[xenforo] add 'profile' extractor (#9046)
mikf Feb 16, 2026
e8cb9e3
[pholder] extend test results
mikf Feb 16, 2026
98c5922
[reddit:user] disable 'only' for '/saved' results (#9071)
mikf Feb 16, 2026
1b47483
fix accessing 'self.exc' in …API methods
mikf Feb 16, 2026
be62b2c
[mangadex:covers] fix '400 Bad Request' when using 'lang' option
mikf Feb 17, 2026
4862d7d
[deviantart] use private token for 'tier' locked deviations (#9072)
mikf Feb 17, 2026
2878a5f
[mangadex] add 'covers' option
mikf Feb 17, 2026
979ad71
[job] support 'skip_posts' & 'skip_children' extractor functions
mikf Feb 17, 2026
1cca8f7
[pixiv] rename 'follow' to 'followed' subcategory
mikf Feb 18, 2026
0d5127a
[common] integrate 'cache' method
mikf Feb 18, 2026
2eaf93a
[cache] update database file access
mikf Feb 18, 2026
f109235
[common] implement 'cache_update()'
mikf Feb 18, 2026
61e1291
[common] fix condition for cached value usage
mikf Feb 18, 2026
4840504
[bluesky] remove cache decorators
mikf Feb 18, 2026
5b5e1a1
[civitai] remove cache decorators
mikf Feb 18, 2026
5a43757
[deviantart] remove cache decorators
mikf Feb 18, 2026
03e732a
[urlgalleries] update extractor (#9090)
mikf Feb 19, 2026
a842d68
[8chan] remove cache decorators
mikf Feb 19, 2026
572a863
[reddit] remove cache decorators
mikf Feb 19, 2026
8d22fbe
[pixiv] remove cache decorators
mikf Feb 19, 2026
015440c
[oauth] update token cache handling
mikf Feb 19, 2026
c390f3e
[twitter] remove cache decorators
mikf Feb 19, 2026
3d28964
remove other memcache decorators
mikf Feb 19, 2026
cd10069
[mangadex] remove cache decorators
mikf Feb 19, 2026
843f4d4
[kemono] remove cache decorators
mikf Feb 19, 2026
629957e
[iwara] update API domain
mikf Feb 20, 2026
a1345db
[iwara] improve video format selection
mikf Feb 20, 2026
30f11b5
[imagepond] update extractors
mikf Feb 20, 2026
9320eaa
[mangataro] fix 'chapter' extractor
mikf Feb 20, 2026
db2a276
replace remaining memcache decorators
mikf Feb 20, 2026
246a992
[kemono] fix default coomer file order (#9097)
AngeredBacterium Feb 21, 2026
cf4e7dd
replace remaining db cache decorators
mikf Feb 20, 2026
a857c5e
[cache] remove decorator classes
mikf Feb 21, 2026
a6d5ff8
[oauth] fix 'cache' attribute overwriting function
mikf Feb 21, 2026
f7e9a77
[instagram] fix user caching behavior (#9025)
mikf Feb 21, 2026
acea70c
[iwara] update 'sha_postfix' (#9096)
mikf Feb 21, 2026
4141c80
[tests/cache] update to 'Extractor.cache()'
mikf Feb 22, 2026
56aa089
[cache] handle ':memory:' as database path
mikf Feb 22, 2026
8582412
merge branch 'cache'
mikf Feb 22, 2026
d538567
[paheal] fix 'md5' metadata (#9110)
mikf Feb 22, 2026
734d32e
[dl:http] check 'download' flag before connecting to server (#9109)
mikf Feb 22, 2026
8b36c5f
[dl:ytdl] process 'download' flag
mikf Feb 22, 2026
ce9ce24
[common] remove unused helper methods
mikf Feb 22, 2026
78228da
[kemono] add 'archives-format' option (#9104)
mikf Feb 22, 2026
d1e1ccb
[mangadex] preserve 'api-parameters' order (#9111)
mikf Feb 23, 2026
90a8ce8
[formatter] fix exceptions for non-default 'fmt' (#9113)
mikf Feb 23, 2026
0f273c9
[tumblr] retry without OAuth when blocked by other user (#8803)
mikf Feb 23, 2026
aab137a
[iwara] add 'format' option
mikf Feb 23, 2026
5ae2ff3
[instagram] add 'static-videos' option (#9103)
mikf Feb 23, 2026
b3cc505
[instagram] extend 'user-strategy' option (#9039 #9069)
mikf Feb 23, 2026
f48fa9b
[iwara] fix 'search' extractor
mikf Feb 23, 2026
f676978
[postimg:gallery] support multiple pages (#9119)
mikf Feb 24, 2026
de0d239
[postimg] fix 'filename' metadata (#9119)
mikf Feb 24, 2026
2c899dc
[weebdex] fix "KeyError: 'chapter'" for oneshots (#9123)
mikf Feb 24, 2026
7f7dfd2
[reddit:user] disable 'only' for 'upvoted' & 'downvoted' results (#9124)
mikf Feb 24, 2026
50d1894
[xenforo] support 'imgur' media embeds (#9127)
mikf Feb 25, 2026
69f5d01
[path] add "windows+" as a special 'path-restrict' value
mikf Feb 25, 2026
daad1d9
[fanbox] add 'tag' extractor (#9128)
mikf Feb 25, 2026
4184455
[e621] update 'tags' format
mikf Feb 25, 2026
37d0630
[pp:metadata] improve '_write_tags()' code
mikf Feb 26, 2026
4d3ebbc
[util] implement 'predicate_tags()'
mikf Feb 26, 2026
d0ca98e
implement 'tags-blacklist/-whitelist' options
mikf Feb 26, 2026
d911286
[mixdrop] add 'file' extractor (#9130)
enduser420 Feb 27, 2026
dd2097f
[sankaku] include 'parent:…' posts (#9102)
LooseEthics Feb 27, 2026
883b8ae
[danbooru] add 'parent:…' test URL
mikf Feb 27, 2026
6a31d0a
[bellazon] fix 'content' for posts with quote (#9140)
mikf Feb 27, 2026
7299899
[smugmug] improve error for deleted files (#9140)
mikf Feb 27, 2026
00bc530
[mixdrop] '.groups(N)' -> '[N]'
mikf Feb 27, 2026
9fab763
[pp:ugoira] add more 'mkvmerge' option (#8803)
mikf Feb 27, 2026
8baba07
[options] add '--blacklist' & '--whitelist' command-line options
mikf Feb 27, 2026
7b33b71
[util] implement 'predicate_date()'
mikf Feb 27, 2026
2d5dbb0
implement 'date-before/-after' options (#1167 #1389 #2308)
mikf Feb 28, 2026
fc01f87
[reddit] support 'www.old.reddit.com' URLs (#9137)
mikf Feb 28, 2026
e6021f6
[whyp] fix downloading private content (#9133)
mikf Feb 28, 2026
c00bd3a
[tumblr] support media from 'answer'/'asks' posts (#4509)
mikf Feb 28, 2026
f4d3d61
[job] support 'skip_date' extractor utility functions
mikf Feb 28, 2026
8cb1adb
[tumblr] support 'date-before' skip
mikf Feb 28, 2026
267ffeb
remove 'date-format' option
mikf Feb 28, 2026
7b7c295
[util] remove support for paths as lists
mikf Feb 28, 2026
3eb049c
[facebook:photo] improve metadata availability (#9147)
mikf Feb 28, 2026
8e101ab
[tiktok] use more lenient regex for secondary user IDs (#9094)
mikf Feb 28, 2026
f132a11
[fansly] prevent "KeyError: 'data'" (#9149)
mikf Feb 28, 2026
c5cc83a
[patreon:user] implement 'date-before' skip
mikf Feb 28, 2026
8078cb3
[vipr] detect missing images (#8415)
mikf Feb 28, 2026
7fd0e60
[bellazon] ignore forum signatures (#9140)
mikf Mar 1, 2026
635d602
[path] implement 'generate_path()'
mikf Mar 1, 2026
3f9736d
implement path-restricted archive paths (#3359 #3361 #4100 #8027)
mikf Mar 1, 2026
459a2be
[docs/configuration] improve 'output.colors' & 'output.ansi' (#9155)
mikf Mar 1, 2026
592a766
[bellazon] improve quote processing (#9140)
mikf Mar 2, 2026
505d003
[path] support more 'generate_path()' ':' replacements
mikf Mar 2, 2026
aea83ec
report errors when using removed/deprecated options (#9163)
mikf Mar 2, 2026
7be6faa
[common] exclude 'alt' URLs from 'include=all'
mikf Mar 2, 2026
dac0df1
[twitter] rename 'replies' subcategory to 'with-replies' (#9162 #8107)
mikf Mar 2, 2026
eab369c
[patreon] update default 'User-Agent' headers (#9165)
mikf Mar 2, 2026
c8d508f
[discord] fix "AttributeError: 'exc'" (#9169)
mikf Mar 3, 2026
583023a
[mastodon] fix "AttributeError: 'cache'" (#9170)
mikf Mar 3, 2026
c09e58d
[luscious] fix empty file URLs (#9172)
mikf Mar 3, 2026
4f9fc2e
[e621] support shared post URLs (#9168)
mikf Mar 3, 2026
9348210
[path] support ':~' replacement & use ':$' for envvars
mikf Mar 3, 2026
7e07528
[simplyhentai] rewrite (#9158)
mikf Mar 3, 2026
aa20fa6
[tests/results] implement testing exception messages
mikf Mar 3, 2026
0dede42
[reddit:user] improve error message for suspended accounts (#9177)
mikf Mar 3, 2026
f44e6f2
[simplyhentai] use '"browser": "firefox"' by default (#9158)
mikf Mar 4, 2026
3e5dbe5
[pp:actions] add 'actions' postprocessor
mikf Mar 4, 2026
2ced793
update old 'image-…' & 'chapter-…' names
mikf Mar 4, 2026
1495103
[bellazon] ignore '/public/style_emoticons/' URLs (#9140)
mikf Mar 4, 2026
7f1d80e
[pp:exec] implement 'success' & 'error' options
mikf Mar 4, 2026
0cbce22
[job] process 'DOWNLOAD' flag after running 'file' postprocessors
mikf Mar 4, 2026
9816734
[deviantart] fix 'TypeError' when 'auto-unwatch' is enabled (#9182)
mikf Mar 5, 2026
4d44e0d
[dl:http] release connections on '416 Range Not Satisfiable' responses
mikf Mar 5, 2026
50e443a
[pixiv] extract 'hash' metadata when available
mikf Mar 5, 2026
713a6f5
[paheal] fix tag searches with only 1 result (#9186)
mikf Mar 5, 2026
e11f23b
[twitter] update 'directory_fmt' for 'info' extractor
mikf Mar 5, 2026
ae8fa5e
[exception] introduce 'StopDownload'
mikf Mar 5, 2026
148d535
[readcomiconline] add 'tag' extractor (#9190)
mikf Mar 5, 2026
144232d
[kemono:discord] fix archive data overwriting general data (#9184)
mikf Mar 5, 2026
b3fe2dd
[pp:hash] enable using 'mode' to select hashes
mikf Mar 5, 2026
5262f16
[simplyhentai] fix typo (#9158)
mikf Mar 6, 2026
805b789
[filester] add support (#9196)
mikf Mar 6, 2026
30b53fb
[gofile] fix extractors (#9183)
mikf Mar 6, 2026
3f26b57
[options] implement various 'cache' command-line options
mikf Mar 6, 2026
e381943
[twitter] use 'max_id' as default 'search-pagination' (#9193 #9194)
mikf Mar 6, 2026
cef9713
[bellazon] handle '/main/index.php' attachments (#9140)
mikf Mar 7, 2026
63b14e8
[build] fix 'charset_normalizer' for PyInstaller builds (#9200 #9202)
mikf Mar 7, 2026
6cfb1e9
[reddit] enable 'previews' for 'comment' submissions (#9206)
mikf Mar 8, 2026
2d27911
[imgur] fix "AttributeError: 'exc'"
mikf Mar 8, 2026
c3d65be
[common] add optional Byparr challenge solver
Dragonatorul Mar 15, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
15 changes: 15 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
root = true

[*]
charset = utf-8
end_of_line = lf
insert_final_newline = true
trim_trailing_whitespace = true

[**.py]
indent_size = 4
indent_style = space
max_line_length = 79

[Makefile]
indent_style = tab
2 changes: 2 additions & 0 deletions .github/FUNDING.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
ko_fi: mikefaehrmann
custom: https://www.paypal.me/mikefaehrmann
69 changes: 69 additions & 0 deletions .github/workflows/docker.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
name: Docker Images

on:
workflow_dispatch:
push:
branches:
- master
tags:
- v[0-9]+.[0-9]+.[0-9]+


permissions:
packages: write

concurrency:
group: docker
cancel-in-progress: false

jobs:
build:

runs-on: ubuntu-latest

# on release commits, run only for tag event
if: ${{
github.repository == 'mikf/gallery-dl' &&
( ! startsWith( github.event.head_commit.message , 'release version ' ) ||
startsWith( github.ref , 'refs/tags/v' ) )
}}

steps:
- uses: actions/checkout@v5

- uses: docker/metadata-action@v5
id: metadata
with:
images: |
mikf123/gallery-dl
ghcr.io/mikf/gallery-dl
tags: |
type=ref,event=tag
type=raw,value=dev
type=sha,format=long,prefix=
type=raw,priority=500,value={{date 'YYYY.MM.DD'}}
- uses: docker/setup-qemu-action@v3

- uses: docker/setup-buildx-action@v3

- name: Login to DockerHub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: Login to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.GHCR_TOKEN }}

- uses: docker/build-push-action@v5
with:
context: .
push: true
tags: ${{ steps.metadata.outputs.tags }}
labels: ${{ steps.metadata.outputs.labels }}
platforms: linux/amd64,linux/arm64
108 changes: 108 additions & 0 deletions .github/workflows/executables.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
name: Executables

on:
workflow_dispatch:
push:
branches:
- master
tags-ignore:
- "*"

env:
DATE_FORMAT: "%Y.%m.%d"

jobs:
build:

if: github.repository == 'mikf/gallery-dl'
runs-on: ${{ matrix.os }}
defaults:
run:
shell: bash
strategy:
fail-fast: false
matrix:
os: ["windows-latest", "macOS-latest"]
architecture: ["x64"]
python-version: ["3.13"]
python-packages: [""]
include:
- os: "ubuntu-latest"
architecture: "x64"
python-version: "3.13"
python-packages: "secretstorage"
- os: "windows-2022"
architecture: "x86"
python-version: "3.8"
python-packages: "toml"

steps:
- uses: actions/checkout@v5

- name: Set up Python ${{ matrix.python-version }} ${{ matrix.architecture }}
uses: actions/setup-python@v6
with:
python-version: ${{ matrix.python-version }}
architecture: ${{ matrix.architecture }}

- name: Environment Variables
run: |
echo "DATE=$(date '+${{ env.DATE_FORMAT }}')" >> "$GITHUB_ENV"
echo "LABEL=$(python ./scripts/pyinstaller.py --print --os '${{ matrix.os }}' --arch '${{ matrix.architecture }}')" >> "$GITHUB_ENV"

- name: Update Version
# use Python since its behavior is consistent across operating systems
shell: python
run: |
import re
path = "./gallery_dl/version.py"
with open(path) as fp:
content = fp.read()
content = re.sub(
r'\b(__version__ = "[^"]+)',
r"\1:${{ env.DATE }}",
content)
content = re.sub(
r'\b(__variant__ =).+',
r'\1 "dev/${{ env.LABEL }}"',
content)
with open(path, "w") as fp:
fp.write(content)

- name: Build executable
run: |
pip install requests requests[socks] yt-dlp[default] pyyaml ${{ matrix.python-packages }} pyinstaller
pip install truststore || true
python ./scripts/pyinstaller.py --label '${{ env.LABEL }}'

- uses: actions/upload-artifact@v4
with:
name: executable-${{ matrix.os }}-${{ matrix.architecture }}-${{ matrix.python-version }}
path: dist/*
retention-days: 1
compression-level: 0

release:

needs: build
runs-on: ubuntu-latest

steps:
- uses: actions/download-artifact@v4

- name: Environment Variables
run: echo "DATE=$(date '+${{ env.DATE_FORMAT }}')" >> "$GITHUB_ENV"

- name: Body
run: printf 'https://github.com/%s/commit/%s' '${{ github.repository }}' '${{ github.sha }}' > body.md

- uses: ncipollo/release-action@v1
with:
owner: gdl-org
repo: builds
tag: ${{ env.DATE }}
bodyFile: body.md
artifacts: "executable-*/*"
allowUpdates: true
makeLatest: true
token: ${{ secrets.REPO_TOKEN }}
58 changes: 58 additions & 0 deletions .github/workflows/pages.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
name: GitHub Pages

on:
workflow_dispatch:
push:
branches:
- master
paths:
- docs/**

permissions:
contents: read
pages: write
id-token: write

concurrency:
group: pages
cancel-in-progress: false

jobs:
dispatch:

if: github.repository == 'mikf/gallery-dl'
runs-on: ubuntu-latest

steps:
- name: Dispatch to gdl-org/docs
run: >
curl -L
-X POST
-H "Accept: application/vnd.github+json"
-H "Authorization: Bearer ${{ secrets.REPO_TOKEN }}"
-H "X-GitHub-Api-Version: 2022-11-28"
https://api.github.com/repos/gdl-org/docs/actions/workflows/pages.yml/dispatches
-d '{"ref":"master"}'

deploy:

if: github.repository == 'mikf/gallery-dl'
runs-on: ubuntu-latest

environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}

steps:
- uses: actions/checkout@v5
- uses: actions/configure-pages@v4

- name: Copy static files
run: |
mkdir --parents -- ./_site
cp --archive --target-directory=./_site -- \
./docs/oauth-redirect.html

- uses: actions/upload-pages-artifact@v3
- uses: actions/deploy-pages@v4
id: deployment
70 changes: 70 additions & 0 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
name: Tests

on:
workflow_dispatch:
push:
branches:
- master
pull_request:
branches:
- master

jobs:
test:

runs-on: ubuntu-22.04
strategy:
fail-fast: false
matrix:
python-version:
- "3.8"
- "3.9"
- "3.10"
- "3.11"
- "3.12"
- "3.13"
- "3.14"
- "pypy3.9"
- "pypy3.11"

steps:
- uses: actions/checkout@v5

- name: Check file permissions
run: |
if [[ "$(find ./gallery_dl -type f -not -perm 644)" ]]; then exit 1; fi

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v6
with:
python-version: ${{ matrix.python-version }}

- name: Install dependencies
run: |
pip install -r requirements.txt
pip install flake8 youtube-dl

- name: Install yt-dlp
run: |
case "${{ matrix.python-version }}" in
3.8|3.9|pypy3.9)
# install from PyPI
pip install yt-dlp
;;
*)
# install from master
pip install https://github.com/yt-dlp/yt-dlp/archive/refs/heads/master.tar.gz
;;
esac

- name: Lint with flake8
run: |
flake8 .

- name: Run tests
run: |
make test

- name: Test autogeneration of man pages, bash/zsh/fish completion, etc
run: |
make
70 changes: 70 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
archive/

# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]

# C extensions
*.so

# Distribution / packaging
.Python
env/
data/
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
*.egg-info/
.installed.cfg
*.egg

# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec

# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*,cover

# Translations
*.mo
*.pot

# Django stuff:
*.log

# Sphinx documentation
docs/_build/

# PyBuilder
target/

# Snap packaging specific
/snap/.snapcraft/
/parts/
/stage/
/prime/

/*.snap
/*_source.tar.bz2
/gallery-dl
Loading