Merge pull request #313 from Ljzd-PRO/devel

Bump to v0.19.0

Author: Ljzd-PRO

CHANGELOG.md

@@ -1,24 +1,129 @@
 ## Changes
 
-![Downloads](https://img.shields.io/github/downloads/Ljzd-PRO/KToolBox/v0.18.2/total)
+![Downloads](https://img.shields.io/github/downloads/Ljzd-PRO/KToolBox/v0.19.0/total)
+
+### 💡 Feature
+
+- Add **`--keywords-exclude`** parameter for post filtering - #309
+  ```shell
+  # Method 1: Include only specific character posts
+  ktoolbox sync_creator --url="https://kemono.cr/fanbox/user/32165989" --keywords="release"
+
+  # Method 2: Exclude unwanted character posts (OR logic)
+  ktoolbox sync_creator --url="https://kemono.cr/fanbox/user/32165989" --keywords_exclude="announcement,vote,share"
+  
+  # Method 3: Combined filtering (most flexible)
+  ktoolbox sync_creator --url="https://kemono.cr/fanbox/user/32165989" --keywords="ブルアカ" --keywords_exclude="全体公開,結果発表"
+  ```
+- The `--keywords` and `--keywords-exclude` features for keyword filtering and exclusion can now also be set in the configuration
+  - New configuration options:
+    - `job.keywords`: Keyword filtering (default is empty)
+    - `job.keywords_exclude`: Keyword exclusion (default is empty)
+  - You can edit these configurations by running `ktoolbox config-editor` (`Job -> ...`)
+  - Or manually edit them in the `.env` file or environment variables
+    ```dotenv
+    KTOOLBOX_JOB__KEYWORDS='["expression", "sound effect variation"]'
+    KTOOLBOX_JOB__KEYWORDS_EXCLUDE='["public", "result announcement"]'
+    ```
+  - 📖More information: [Configuration-Reference-JobConfiguration](https://ktoolbox.readthedocs.io/latest/configuration/reference/#ktoolbox.configuration.JobConfiguration)
+- Add **year/month** **grouping** functionality for post organization - #306
+  - You can group downloaded posts by year and month with customizable directory naming formats
+  - New configuration options:
+    - `job.group_by_year`: Enable grouping by year (Disabled by default)
+    - `job.group_by_month`: Enable grouping by month (Disabled by default)
+    - `job.year_month_format`: Customize the directory naming format for year grouping (Defaults to `{year}`)
+    - `job.month_format`: Customize the directory naming format for month grouping (Defaults to `{year}-{month:02d}`)
+  - Run `ktoolbox config-editor` to edit these configurations (`Job -> ...`)
+  - Or manually edit them in `.env` file or environment variables
+    ```dotenv
+    # Environment variables (Defaults to False)
+    KTOOLBOX_JOB__GROUP_BY_YEAR=True
+    KTOOLBOX_JOB__GROUP_BY_MONTH=True
+  
+    # Custom style naming
+    KTOOLBOX_JOB__YEAR_DIRNAME_FORMAT="Year {year}"
+    KTOOLBOX_JOB__MONTH_DIRNAME_FORMAT="Month {month:02d}"
+    ```
+    Resulting directory structure:
+    ```
+    creator/
+    ├── Year 2020/
+    │   ├── Month 01/
+    │   │   └── post_title/
+    │   └── Month 12/
+    │       └── another_post/
+    └── Year 2021/
+        └── Month 03/
+            └── latest_post/
+    ```
+  - 📖More information: [Configuration-Reference-JobConfiguration](https://ktoolbox.readthedocs.io/latest/configuration/reference/#ktoolbox.configuration.JobConfiguration)
 
-[//]: # (### 💡 Feature)
 
 ### 🪲 Fix
 
-- Fixed the issue where **warning messages** were displayed regardless of whether **`job.include_revisions`** was enabled or not.
-- Fixed the issue where extracted external links contained extra characters (v0.18.0)
-  - Related configuration options: `job.extract_external_links`, `job.external_link_patterns`
+- Fixed the issue where the `--keywords` parameter could not be parsed correctly in the `sync-creator` command
 
 - - -
 
-[//]: # (### 💡 新特性)
+### 💡 新特性
+
+- 新增 **`--keywords-exclude`** 参数用于帖子筛选 - #309
+  ```shell
+  # 方法1：仅包含特定关键词的帖子
+  ktoolbox sync_creator --url="https://kemono.cr/fanbox/user/32165989" --keywords="发布"
+
+  # 方法2：排除不需要的关键词帖子（或逻辑）
+  ktoolbox sync_creator --url="https://kemono.cr/fanbox/user/32165989" --keywords_exclude="公告,投票,分享"
+  
+  # 方法3：组合筛选（最灵活）
+  ktoolbox sync_creator --url="https://kemono.cr/fanbox/user/32165989" --keywords="ブルアカ" --keywords_exclude="全体公開,結果発表"
+  ```
+- 关键词筛选和关键词排除的 `--keywords` 和 `--keywords-exclude` 功能现在也可以在配置中设置
+  - 新配置项：
+    - `job.keywords`：关键词筛选（默认为空）
+    - `job.keywords_exclude`：关键词排除（默认为空）
+  - 可通过运行 `ktoolbox config-editor` 编辑这些配置（`Job -> ...`）
+  - 或手动在 `.env` 文件或环境变量中编辑
+    ```dotenv
+    KTOOLBOX_JOB__KEYWORDS='["表情", "効果音差分"]'
+    KTOOLBOX_JOB__KEYWORDS_EXCLUDE='["全体公開", "結果発表"]'
+    ```
+  - 📖更多信息：[配置参考-JobConfiguration](https://ktoolbox.readthedocs.io/latest/configuration/reference/#ktoolbox.configuration.JobConfiguration)
+- 新增按**年份/月**分组功能用于帖子整理 - #306
+  - 可按年份和月份分组下载的帖子，支持自定义目录命名格式
+  - 新配置项：
+    - `job.group_by_year`：启用按年份分组（默认关闭）
+    - `job.group_by_month`：启用按月份分组（默认关闭）
+    - `job.year_month_format`：自定义年份分组目录命名格式（默认为 `{year}`）
+    - `job.month_format`：自定义月份分组目录命名格式（默认为 `{year}-{month:02d}`）
+  - 可通过运行 `ktoolbox config-editor` 编辑这些配置（`Job -> ...`）
+  - 或手动在 `.env` 文件或环境变量中编辑
+    ```dotenv
+    # 是否启用（默认 False）
+    KTOOLBOX_JOB__GROUP_BY_YEAR=True
+    KTOOLBOX_JOB__GROUP_BY_MONTH=True
+  
+    # 自定义目录命名
+    KTOOLBOX_JOB__YEAR_DIRNAME_FORMAT="{year}年"
+    KTOOLBOX_JOB__MONTH_DIRNAME_FORMAT="{month:02d}月"
+    ```
+    目录结构示例：
+    ```
+    creator/
+    ├── 2020年/
+    │   ├── 01月/
+    │   │   └── post_title/
+    │   └── 12月/
+    │       └── another_post/
+    └── 2021年/
+        └── 03月/
+            └── latest_post/
+    ```
+  - 📖更多信息：[配置参考-JobConfiguration](https://ktoolbox.readthedocs.io/latest/configuration/reference/#ktoolbox.configuration.JobConfiguration)
 
 ### 🪲 修复
 
-- 修复了无论是否开启 **`job.include_revisions`** 都会提示**警告信息**的问题
-- 修复了程序提取的外部链接（external links）包含多余字符的问题 (v0.18.0)
-  - 相关配置选项：`job.extract_external_links`, `job.external_link_patterns`
+- 修复 `--keywords` 参数在 `sync-creator` 命令中无法正确解析的问题
 
 ## Upgrade
 
@@ -27,4 +132,4 @@ Use this command to upgrade if you are using **pipx**:
 pipx upgrade ktoolbox
 ```
 
-**Full Changelog**: https://github.com/Ljzd-PRO/KToolBox/compare/v0.18.1...v0.18.2
+**Full Changelog**: https://github.com/Ljzd-PRO/KToolBox/compare/v0.18.2...v0.19.0

ktoolbox/__init__.py

@@ -1,4 +1,4 @@
 __title__ = "KToolBox"
 # noinspection SpellCheckingInspection
 __description__ = "A useful CLI tool for downloading posts in Kemono.cr / .su / .party"
-__version__ = "v0.18.2"
+__version__ = "v0.19.0"

ktoolbox/_cli_zh.py

@@ -173,7 +173,8 @@ async def sync_creator(
             mix_posts: bool = None,
             start_time: str = None,
             end_time: str = None,
-            keywords: str = None
+            keywords: str = None,
+            keywords_exclude: str = None
     ):
         """
         同步创作者所有帖子（通过 URL）
@@ -185,6 +186,7 @@ async def sync_creator(
         :param start_time: 帖子发布时间范围起始
         :param end_time: 帖子发布时间范围结束
         :param keywords: 按标题过滤帖子，逗号分隔关键词
+        :param keywords_exclude: 按标题排除帖子，逗号分隔关键词
         """
         ...
 
@@ -199,7 +201,8 @@ async def sync_creator(
             mix_posts: bool = None,
             start_time: str = None,
             end_time: str = None,
-            keywords: str = None
+            keywords: str = None,
+            keywords_exclude: str = None
     ):
         """
         同步创作者所有帖子（通过参数）
@@ -212,6 +215,7 @@ async def sync_creator(
         :param start_time: 帖子发布时间范围起始
         :param end_time: 帖子发布时间范围结束
         :param keywords: 按标题过滤帖子，逗号分隔关键词
+        :param keywords_exclude: 按标题排除帖子，逗号分隔关键词
         """
         ...
 
@@ -228,7 +232,8 @@ async def sync_creator(
             end_time: str = None,
             offset: int = 0,
             length: int = None,
-            keywords: str = None
+            keywords: str = None,
+            keywords_exclude: str = None
     ):
         """
         同步创作者所有帖子
@@ -248,6 +253,7 @@ async def sync_creator(
         :param offset: 结果偏移量
         :param length: 获取帖子数量，默认为全部
         :param keywords: 按标题过滤帖子，逗号分隔关键词
+        :param keywords_exclude: 按标题排除帖子，逗号分隔关键词
         """
         return await super().sync_creator(
             url=url,
@@ -260,5 +266,6 @@ async def sync_creator(
             end_time=end_time,
             offset=offset,
             length=length,
-            keywords=keywords
+            keywords=keywords,
+            keywords_exclude=keywords_exclude
         )

ktoolbox/_configuration_zh.py

@@ -101,6 +101,13 @@ class JobConfiguration(ktoolbox.configuration.JobConfiguration):
         | ``published``| 日期   |
         | ``edited``   | 日期   |
 
+    - ``year_dirname_format`` 和 ``month_dirname_format`` 可用属性
+
+        | 属性         | 类型   |
+        |--------------|--------|
+        | ``year``     | 字符串 |
+        | ``month``    | 字符串 |
+
     :ivar count: 并发下载的协程数量
     :ivar include_revisions: 下载时包含修订帖子
     :ivar post_dirname_format: 自定义帖子目录名格式，可使用 [属性][ktoolbox.configuration.JobConfiguration]。例如：``[{published}]{id}`` > ``[2024-1-1]123123``，``{user}_{published}_{title}`` > ``234234_2024-1-1_TheTitle``
@@ -113,6 +120,10 @@ class JobConfiguration(ktoolbox.configuration.JobConfiguration):
     :ivar block_list: 不下载匹配这些模式（Unix shell 风格）的文件，如 ``["*.psd","*.zip"]``
     :ivar extract_external_links: 从帖子内容中提取外部文件分享链接并保存到单独文件
     :ivar external_link_patterns: 用于提取外部链接的正则表达式模式
+    :ivar group_by_year: 根据发布日期按年分组到不同目录
+    :ivar group_by_month: 根据发布日期按月分组到不同目录（需要启用 group_by_year）
+    :ivar year_dirname_format: 自定义年份目录名格式。可用属性：``year``。例如：``{year}`` > ``2024``，``Year_{year}`` > ``Year_2024``
+    :ivar month_dirname_format: 自定义月份目录名格式。可用属性：``year``、``month``。例如：``{year}-{month}`` > ``2024-01``，``{year}_{month}`` > ``2024_01``
     """
     ...
 

ktoolbox/action/job.py

@@ -11,7 +11,7 @@
 
 from ktoolbox._enum import PostFileTypeEnum, DataStorageNameEnum
 from ktoolbox.action import ActionRet, fetch_creator_posts, FetchInterruptError
-from ktoolbox.action.utils import generate_post_path_name, filter_posts_by_date, generate_filename, filter_posts_by_keywords
+from ktoolbox.action.utils import generate_post_path_name, filter_posts_by_date, generate_filename, filter_posts_by_keywords, filter_posts_by_keywords_exclude, generate_grouped_post_path
 from ktoolbox.api.model import Post, Attachment
 from ktoolbox.api.posts import get_post_revisions as get_post_revisions_api
 from ktoolbox.configuration import config, PostStructureConfiguration
@@ -155,7 +155,8 @@ async def create_job_from_creator(
         mix_posts: bool = None,
         start_time: Optional[datetime],
         end_time: Optional[datetime],
-        keywords: Optional[Set[str]] = None
+        keywords: Optional[Set[str]] = None,
+        keywords_exclude: Optional[Set[str]] = None
 ) -> ActionRet[List[Job]]:
     """
     Create a list of download job from a creator
@@ -172,6 +173,7 @@ async def create_job_from_creator(
     :param start_time: Start time of the time range
     :param end_time: End time of the time range
     :param keywords: Set of keywords to filter posts by title (case-insensitive)
+    :param keywords_exclude: Set of keywords to exclude posts by title (case-insensitive)
     """
     mix_posts = config.job.mix_posts if mix_posts is None else mix_posts
 
@@ -207,16 +209,26 @@ async def create_job_from_creator(
     if keywords:
         post_list = list(filter_posts_by_keywords(post_list, keywords))
 
+    # Filter out posts by exclude keywords
+    if keywords_exclude:
+        post_list = list(filter_posts_by_keywords_exclude(post_list, keywords_exclude))
+        
     logger.info(f"Get {len(post_list)} posts after filtering, start creating jobs")
 
     # Filter posts and generate ``CreatorIndices``
     if not mix_posts:
         if save_creator_indices:
+            # Generate posts_path with year/month grouping if enabled
+            posts_path = {}
+            for post in post_list:
+                grouped_base_path = generate_grouped_post_path(post, path)
+                posts_path[post.id] = grouped_base_path / sanitize_filename(post.title)
+            
             indices = CreatorIndices(
                 creator_id=creator_id,
                 service=service,
                 posts={post.id: post for post in post_list},
-                posts_path={post.id: path / sanitize_filename(post.title) for post in post_list}
+                posts_path=posts_path
             )
             async with aiofiles.open(
                     path / DataStorageNameEnum.CreatorIndicesData.value,
@@ -232,7 +244,12 @@ async def create_job_from_creator(
     job_list: List[Job] = []
     for post in post_list:
         # Get post path
-        post_path = path if mix_posts else path / generate_post_path_name(post)
+        if mix_posts:
+            post_path = path
+        else:
+            # Apply year/month grouping if enabled
+            grouped_base_path = generate_grouped_post_path(post, path)
+            post_path = grouped_base_path / generate_post_path_name(post)
 
         # Generate jobs for the main post
         job_list += await create_job_from_post(