Skip to content

Commit fa11c4c

Browse files
committed
Add support for GitHub
1 parent 53d27a7 commit fa11c4c

File tree

12 files changed

+525
-25
lines changed

12 files changed

+525
-25
lines changed

README.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Upvote RSS
22

3-
Generate rich RSS feeds for popular posts from social aggregation websites Reddit, Lemmy, Hacker News, Lobsters, PieFed, and Mbin.
3+
Generate rich RSS feeds for popular posts from social aggregation websites Reddit, Lemmy, Hacker News, Lobsters, PieFed, Mbin, and git forge GitHub.
44

55
![Application Screenshot](img/screenshot.png)
66

@@ -19,7 +19,7 @@ Generate rich RSS feeds for popular posts from social aggregation websites Reddi
1919

2020
## Features
2121

22-
- Supports subreddits, Hacker News, Lemmy communities, Lobste.rs, PieFed communities, and Mbin magazines
22+
- Supports subreddits, Hacker News, Lemmy communities, Lobste.rs, PieFed communities, Mbin magazines, and GitHub
2323
- Configurable filtering to dial in the right number of posts per day in your feed reader
2424
- Embedded post media (videos, galleries, and images)
2525
- Parsers to extract clean content and add featured images
@@ -95,15 +95,15 @@ docker build -f docker/Dockerfile .
9595
## Feed options
9696
| Option | Description |
9797
|-|-|
98-
| **Platform** | Upvote RSS currently supports Reddit, Hacker News, Lemmy instances, Lobste.rs, PieFed communities, and Mbin magazines. |
98+
| **Platform** | Upvote RSS currently supports Reddit, Hacker News, Lemmy instances, Lobste.rs, PieFed communities, Mbin magazines, and GitHub. |
9999
| **Instance** | The fully qualified domain name to a Lemmy, Mbin, or PieFed instance (shown when Lemmy, Mbin, or PieFed is selected as the platform). |
100-
| **Subreddit/Community/Type** | `Subreddit` field is available when Reddit is selected as the platform and your Reddit API credentials are set through environment variable. Available subreddits should populate in a datalist as you type.<br><br>`Community` field is available when Lemmy, PieFed, or Mbin is selected as the platform.<br><br>`Type` field is available when Hacker News or Lobsters is selected as the platform. Available options for Hacker News are [Front Page](https://news.ycombinator.com), [Best](https://news.ycombinator.com/best), [New](https://news.ycombinator.com/newest), [Ask](https://news.ycombinator.com/ask), and [Show](https://news.ycombinator.com/show). Available options for Lobsters are All posts, Category, and Tag. When either Category or Tag is chosen for Lobsters, the corresponding field is available. The list of categories and tags on the main Lobsters instance can be found here: [https://lobste.rs/tags](https://lobste.rs/tags). |
100+
| **Subreddit/Community/Type** | `Subreddit` field is available when Reddit is selected as the platform and your Reddit API credentials are set through environment variable. Available subreddits should populate in a datalist as you type.<br><br>`Community` field is available when Lemmy, PieFed, or Mbin is selected as the platform.<br><br>`Type` field is available when Hacker News or Lobsters is selected as the platform. Available options for Hacker News are [Front Page](https://news.ycombinator.com), [Best](https://news.ycombinator.com/best), [New](https://news.ycombinator.com/newest), [Ask](https://news.ycombinator.com/ask), and [Show](https://news.ycombinator.com/show). Available options for Lobsters are All posts, Category, and Tag.<br><br>When either Category or Tag is chosen for Lobsters, the corresponding `Category` or `Tag` field is available. The list of categories and tags on the main Lobsters instance can be found here: [https://lobste.rs/tags](https://lobste.rs/tags).<br><br>When GitHub is selected as the platform, a `Language` and `Topic` field will be available. Multiple languages and topics can be specified in each field; a `+` between search terms denotes an `and` operator, while a `comma` denotes an `or` operator. For example, entering `python+typescript` in the `Language` field will return repositories that use both Python and TypeScript, while entering `python,typescript` will return repositories that use either Python or TypeScript. The same logic applies to the `Topic` field.<br>|
101101
| **Filter type** | `Score`: Items below the desired score will be filtered out.<br><br>`Threshold`: This parameter will get the average score for the past month's hot posts and will filter out items that fall below this percentage. This is helpful for volatile communities when more people are using the service and causing posts to be scored higher and higher. Since this is a percentage, the number of items in the outputted feed should be more consistent than when using the `score` parameter. Not available for Lobsters.<br><br>`Posts Per Day`: Upvote RSS will attempt to output an average number of posts per day by looking at a community's recent history to determine the score below which posts will be filtered out. This is the filter I find most useful most of the time. |
102102
| **Use custom Reddit domain** | Override the base domain that Reddit posts will link to from the RSS feeds, e.g. `old.reddit.com` instead of `www.reddit.com`, or a self-hosted Reddit front-end. |
103103
| **Show score in feed** | Includes the score of the post in the feed. |
104104
| **Include article content**| Includes the parsed content of the article in the feed. |
105105
| **Include summary** | Includes a summary of the article in the feed. Only available an AI summarizer is set through [environment variables](#environment-variables). |
106-
| **Include comments** | Includes top-voted comments in the feed. When checked, you can specify the number of comments to include at the end of each post. |
106+
| **Include comments** | Includes top-voted comments in the feed. When checked, you can specify the number of comments to include at the end of each post. Not available for GitHub. |
107107
| **Filter pinned comments** | Filter out pinned moderator comments (available for Lemmy, PieFed, and Reddit). |
108108
| **Filter old posts** | Filters out old posts from the feed. You can specify the cutoff in days. This is helpful for communities that don't have a lot of posts or engagement since older posts can show up in the feed when the monthly average scores drop. |
109109
| **Filter NSFW posts** | Filters out NSFW posts from the feed. Only available for Reddit. |

ajax.php

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -114,6 +114,8 @@
114114
$instance = $data['instance'] ?? null;
115115
$community = $data['community'] ?? null;
116116
$community_type = $data['communityType'] ?? null;
117+
$language = $data['language'] ?? null;
118+
$topic = $data['topic'] ?? null;
117119
$filter_nsfw = $data['filterNSFW'] ?? false;
118120
$blur_nsfw = $data['blurNSFW'] ?? false;
119121
$filter_old_posts = $data['filterOldPosts'] ?? false;
@@ -134,6 +136,10 @@
134136
case 'reddit':
135137
$community = new Community\Reddit($subreddit);
136138
break;
139+
case 'github':
140+
$community = new Community\GitHub($community, $language, $topic);
141+
$instance = 'github.com';
142+
break;
137143
case 'hacker-news':
138144
$community = new Community\HackerNews($community);
139145
$instance = 'news.ycombinator.com';

classes/communities/github.php

Lines changed: 240 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,240 @@
1+
<?php
2+
3+
namespace Community;
4+
5+
class GitHub extends Community
6+
{
7+
8+
// Properties
9+
public $platform = 'github';
10+
public $instance = 'github.com';
11+
public $is_instance_valid = true;
12+
public $platform_icon = UPVOTE_RSS_URI . 'img/platforms/github.png';
13+
public $language = null;
14+
public $topic = null;
15+
public $max_items_per_request = 100;
16+
17+
18+
// Constructor
19+
function __construct(
20+
$slug = null,
21+
$language = null,
22+
$topic = null
23+
) {
24+
$this->language = $language ? trim($language) : LANGUAGE;
25+
$this->topic = $topic ? trim($topic) : TOPIC;
26+
$this->getCommunityInfo();
27+
}
28+
29+
30+
// Enable loading from cache
31+
static function __set_state($array) {
32+
$community = new self();
33+
foreach ($array as $key => $value) {
34+
$community->{$key} = $value;
35+
}
36+
return $community;
37+
}
38+
39+
40+
protected function getInstanceInfo() {}
41+
42+
43+
protected function getCommunityInfo(): void {
44+
// Name
45+
$name = 'GitHub';
46+
// Title
47+
$title = $name;
48+
if ($this->language || $this->topic) {
49+
$title .= ' - ';
50+
if ($this->language) {
51+
$title .= $this->language;
52+
}
53+
if ($this->language && $this->topic) {
54+
$title .= ' / ';
55+
}
56+
if ($this->topic) {
57+
$title .= $this->topic;
58+
}
59+
} else {
60+
$title .= ' - Trending new repositories';
61+
}
62+
$language_text = (strpbrk($this->language, ',+') !== false) ? 'languages' : 'language';
63+
$topic_text = (strpbrk($this->topic, ',+') !== false) ? 'topics' : 'topic';
64+
// Slug
65+
$slug = 'github';
66+
if ($this->language) {
67+
$slug .= '/language/' . strtolower(str_replace([' ', '+'], '-', $this->language));
68+
}
69+
if ($this->topic) {
70+
$slug .= '/topic/' . strtolower(str_replace([' ', '+'], '-', $this->topic));
71+
}
72+
$this->slug = $slug;
73+
// Description
74+
$description = "New repositories";
75+
if ($this->language || $this->topic) {
76+
if ($this->language) {
77+
$description .= " in $language_text \"" . $this->language . "\"";
78+
}
79+
if ($this->language && $this->topic) {
80+
$description .= " and in $topic_text \"" . $this->topic . "\"";
81+
} else if ($this->topic) {
82+
$description .= " with $topic_text \"" . $this->topic . "\"";
83+
}
84+
}
85+
$description .= " on " . $name . ", a platform for code hosting, version control, and collaboration among developers.";
86+
// Feed description
87+
$feed_description = "New repositories on " . $name;
88+
if ($this->language) {
89+
$feed_description .= " with $language_text \"" . $this->language . "\"";
90+
}
91+
if ($this->language && $this->topic) {
92+
$feed_description .= " and";
93+
$feed_description .= " $topic_text \"" . $this->topic . "\"";
94+
} else if ($this->topic) {
95+
$feed_description .= " with $topic_text \"" . $this->topic . "\"";
96+
}
97+
$this->platform = "github";
98+
$this->name = $name;
99+
$this->title = $title;
100+
$this->description = $description;
101+
$this->url = 'https://github.com/';
102+
$this->icon = $this->platform_icon;
103+
$this->created = '2008-02-08T00:00:00.000Z';
104+
$this->nsfw = false;
105+
$this->feed_title = $this->title;
106+
$this->feed_description = $feed_description;
107+
$this->needs_authentication = false;
108+
$this->is_community_valid = true;
109+
}
110+
111+
112+
// Get top posts
113+
public function getTopPosts($limit, $period = null): array {
114+
$limit = $limit ?? $this->max_items_per_request;
115+
$cache_object_key = "top_repositories_";
116+
if ($this->language || $this->topic) {
117+
if ($this->language) {
118+
$cache_object_key .= "lang_" . strtolower($this->language) . "_";
119+
}
120+
if ($this->topic) {
121+
$cache_object_key .= "topic_" . strtolower($this->topic) . "_";
122+
}
123+
} else {
124+
$cache_object_key .= "all_";
125+
}
126+
$cache_object_key .= "limit_" . $limit;
127+
$cache_directory = "communities/github/hot_posts";
128+
if ($cache = cache()->get($cache_object_key, $cache_directory)) {
129+
return $cache;
130+
}
131+
$cache_expiration = HOT_POSTS_EXPIRATION;
132+
133+
$top_posts = [];
134+
135+
$created_since = date('Y-m-d', strtotime('-30 days'));
136+
$language = $this->language;
137+
$language_query = $language ? "%20language:$language" : '';
138+
$topic = $this->topic;
139+
$topic_query = $topic ? "%20topic:$topic" : '';
140+
$url = "https://api.github.com/search/repositories?sort=stars&order=desc&per_page=$limit&q=created:>$created_since$language_query$topic_query";
141+
$message = 'Fetching GitHub top repositories';
142+
if ($this->language) {
143+
$message .= ' with language "' . $this->language . '"';
144+
}
145+
if ($this->language && $this->topic) {
146+
$message .= ' and';
147+
} else if ($this->topic) {
148+
$message .= ' with';
149+
}
150+
if ($this->topic) {
151+
$message .= ' topic "' . $this->topic . '"';
152+
}
153+
$message .= ' from URL: ' . $url;
154+
logger()->info($message);
155+
156+
$curl_response = curlURL($url);
157+
if (empty($curl_response)) {
158+
$message = 'Empty response when trying to get repositories for GitHub at ' . $url;
159+
logger()->error($message);
160+
return ['error' => $message];
161+
}
162+
163+
$curl_data = json_decode($curl_response, true);
164+
if (empty($curl_data) || !empty($curl_data['error'])) {
165+
$message = 'There was an error communicating with GitHub: ' . ($curl_data['error'] ?? 'Unknown error');
166+
logger()->error($message);
167+
return ['error' => $message];
168+
}
169+
170+
if (empty($curl_data['items'])) {
171+
$message = 'No repositories found in response when trying to get repositories for GitHub at ' . $url;
172+
logger()->error($message);
173+
return [];
174+
}
175+
176+
foreach ($curl_data['items'] as $post_data) {
177+
$post = [
178+
'id' => $post_data['id'] ?? 0,
179+
'title' => $post_data['full_name'] ?? '',
180+
'score' => $post_data['stargazers_count'] ?? 0,
181+
'url' => $post_data['html_url'] ?? '',
182+
'created_at' => $post_data['created_at'] ?? 0,
183+
'description' => $post_data['description'] ?? '',
184+
'avatar_url' => $post_data['owner']['avatar_url'] ?? '',
185+
];
186+
$top_posts[] = $post;
187+
}
188+
189+
cache()->set($cache_object_key, $top_posts, $cache_directory, $cache_expiration);
190+
return $top_posts;
191+
}
192+
193+
194+
// Get hot posts
195+
public function getHotPosts($limit, $filter_nsfw = FILTER_NSFW, $blur_nsfw = BLUR_NSFW): array {
196+
$limit = $limit ?? $this->max_items_per_request;
197+
$cache_object_key = "hot_repositories_";
198+
if ($this->language || $this->topic) {
199+
if ($this->language) {
200+
$cache_object_key .= "lang_" . strtolower($this->language) . "_";
201+
}
202+
if ($this->topic) {
203+
$cache_object_key .= "topic_" . strtolower($this->topic) . "_";
204+
}
205+
} else {
206+
$cache_object_key .= "all_";
207+
}
208+
$cache_object_key .= "limit_" . $limit;
209+
$cache_directory = "communities/github/hot_posts";
210+
if ($cache = cache()->get($cache_object_key, $cache_directory)) {
211+
return $cache;
212+
}
213+
$cache_expiration = HOT_POSTS_EXPIRATION;
214+
215+
$hot_posts = $this->getTopPosts($limit);
216+
217+
$hot_posts_min = [];
218+
foreach ($hot_posts as $post) {
219+
$post = new \Post\GitHub($post);
220+
$hot_posts_min[] = $post;
221+
}
222+
223+
cache()->set($cache_object_key, $hot_posts_min, $cache_directory, $cache_expiration);
224+
return $hot_posts_min;
225+
}
226+
227+
228+
// Get monthly average top score
229+
public function getMonthlyAverageTopScore(): float {
230+
$top_posts = $this->getTopPosts($this->max_items_per_request);
231+
if (empty($top_posts)) {
232+
logger()->info('No top posts found for monthly average score calculation');
233+
return 0;
234+
}
235+
$total_score = array_sum(array_column($top_posts, 'score'));
236+
$average_score = $total_score / count($top_posts);
237+
$average_score = round($average_score, 1);
238+
return $average_score;
239+
}
240+
}

0 commit comments

Comments
 (0)