feat: Add rich link previews on posts to Bluesky, LinkedIn, and Facebook #831
Conversation
…ook. Uses OpenGraph scraping for LinkedIn, API link parameter for Facebook, and app.bsky.embed.external for Bluesky. Adds PostizBot user agents to metadata fetching to make it distinguishable to firewalls.
|
@jbenton is attempting to deploy a commit to the Listinai Team on Vercel. A member of the Team first needs to authorize it. |
WalkthroughThe changes introduce OpenGraph URL preview support for Bluesky, Facebook, and LinkedIn providers. New utility functions extract URLs from post text and fetch OpenGraph metadata. Posting logic is updated to detect URLs and, if no media is attached, generate and attach rich link previews using fetched metadata. Error handling is incorporated throughout. Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant Provider
participant OpenGraphFetcher
participant MediaUploader
participant SocialAPI
User->>Provider: Submit post with text (may include URL)
Provider->>Provider: Extract URLs from text
alt No media attached and URL found
Provider->>OpenGraphFetcher: fetchOpenGraphData(url)
OpenGraphFetcher-->>Provider: Return metadata (title, desc, image)
alt Image present in metadata
Provider->>MediaUploader: Upload image as thumbnail
MediaUploader-->>Provider: Return image reference
end
Provider->>SocialAPI: Post content with external embed/link preview
else Media attached or no URL
Provider->>SocialAPI: Post content with media or plain text
end
SocialAPI-->>Provider: Return post response
Provider-->>User: Return post result
Assessment against linked issues
Assessment against linked issues: Out-of-scope changes
Suggested reviewers
Poem
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 ESLint
libraries/nestjs-libraries/src/integrations/social/bluesky.provider.tsOops! Something went wrong! :( ESLint: 8.57.0 Error [ERR_MODULE_NOT_FOUND]: Cannot find package '@eslint/eslintrc' imported from /eslint.config.mjs libraries/nestjs-libraries/src/integrations/social/facebook.provider.tsOops! Something went wrong! :( ESLint: 8.57.0 Error [ERR_MODULE_NOT_FOUND]: Cannot find package '@eslint/eslintrc' imported from /eslint.config.mjs libraries/nestjs-libraries/src/integrations/social/linkedin.provider.tsOops! Something went wrong! :( ESLint: 8.57.0 Error [ERR_MODULE_NOT_FOUND]: Cannot find package '@eslint/eslintrc' imported from /eslint.config.mjs
✨ Finishing Touches
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
♻️ Duplicate comments (2)
libraries/nestjs-libraries/src/integrations/social/linkedin.page.provider.ts (1)
18-69: Code duplication - extract to shared utilities.This is another instance of the duplicated OpenGraph functions.
Please see the previous comment about extracting these functions to a shared utility module.
libraries/nestjs-libraries/src/integrations/social/linkedin.provider.ts (1)
20-127: Code duplication - extract to shared utilities.This is the fourth instance of the duplicated OpenGraph functions across the providers.
Please see the previous comment about extracting these functions to a shared utility module.
🧹 Nitpick comments (5)
libraries/nestjs-libraries/src/integrations/social/bluesky.provider.ts (1)
380-390: Consider adding debug logging for URL processing.The implementation correctly handles the priority of embeds (video > images > URLs). Consider adding debug logging when URLs are found but not used due to media presence.
} else { // If no media, check for URLs to create external embeds const urls = extractUrls(post.message); if (urls.length > 0) { + console.log(`Creating external embed for URL: ${urls[0]}`); // Use the first URL found for the external embed const externalEmbed = await createExternalEmbed(agent, urls[0]); if (externalEmbed) { embed = externalEmbed; } + } else { + console.log('No URLs found in post message for external embed'); } }libraries/nestjs-libraries/src/integrations/social/facebook.provider.ts (2)
67-67: Add missing newline before class declaration.} + export class FacebookProvider extends SocialAbstract implements SocialProvider {
310-314: Consider improving URL removal logic to avoid awkward message formatting.Simply removing the URL with
replace()might leave awkward spacing or punctuation. For example, "Check out this link: https://example.com!" becomes "Check out this link: !".- // Remove URL from message text since we're putting it in the link field - const messageWithoutUrl = firstPost.message.replace(urls[0], '').trim(); + // Remove URL from message text since we're putting it in the link field + let messageWithoutUrl = firstPost.message.replace(urls[0], '').trim(); + // Clean up any awkward punctuation or spacing left behind + messageWithoutUrl = messageWithoutUrl + .replace(/:\s*$/, '') // Remove trailing colons + .replace(/\s+/g, ' ') // Normalize whitespace + .trim();libraries/nestjs-libraries/src/integrations/social/linkedin.page.provider.ts (1)
71-71: Consider using union type for better type safety.The function signature uses string literal for type but only 'company' is valid here.
-async function createLinkedInContentEntities(url: string, accessToken: string, personId: string, type: 'company', uploadPictureMethod: any): Promise<any> { +async function createLinkedInContentEntities( + url: string, + accessToken: string, + personId: string, + type: 'company' | 'personal', + uploadPictureMethod: any +): Promise<any> {libraries/nestjs-libraries/src/integrations/social/linkedin.provider.ts (1)
129-129: Add missing newline before class declaration.} + export class LinkedinProvider extends SocialAbstract implements SocialProvider {
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (4)
libraries/nestjs-libraries/src/integrations/social/bluesky.provider.ts(5 hunks)libraries/nestjs-libraries/src/integrations/social/facebook.provider.ts(8 hunks)libraries/nestjs-libraries/src/integrations/social/linkedin.page.provider.ts(2 hunks)libraries/nestjs-libraries/src/integrations/social/linkedin.provider.ts(4 hunks)
⏰ Context from checks skipped due to timeout of 90000ms (1)
- GitHub Check: build-and-publish
🔇 Additional comments (5)
libraries/nestjs-libraries/src/integrations/social/bluesky.provider.ts (2)
221-224: LGTM!The URL extraction implementation is clean and handles the edge case of no matches properly.
136-176: [web_search]Does axios 1.7.7 support the maxContentLength configuration option to limit HTTP response size?libraries/nestjs-libraries/src/integrations/social/facebook.provider.ts (1)
389-407: Good implementation of Facebook-specific optimization.The prewarm scraper method is well-implemented with proper error handling that doesn't fail the main post operation.
libraries/nestjs-libraries/src/integrations/social/linkedin.page.provider.ts (1)
359-369: Excellent use of inheritance and delegation pattern.The override properly delegates to the parent class's enhanced functionality while specifying the 'company' type. This maintains code reuse and consistency.
libraries/nestjs-libraries/src/integrations/social/linkedin.provider.ts (1)
657-669: Well-implemented URL detection and link preview support.The implementation correctly checks for URLs only when no media is present and properly integrates with the existing post creation flow.
| const imageResponse = await axios.get(ogData.image, { | ||
| responseType: 'arraybuffer', | ||
| timeout: 10000, | ||
| headers: { | ||
| 'User-Agent': 'Mozilla/5.0 (compatible; PostizBot/1.0; +https://postiz.com/)' | ||
| } | ||
| }); | ||
| const imageBuffer = Buffer.from(imageResponse.data); | ||
|
|
||
| // Reduce image size if needed (Bluesky has limits) | ||
| const processedImage = await reduceImageBySize(ogData.image, 976); | ||
|
|
||
| const blob = await agent.uploadBlob(new Blob([processedImage])); |
There was a problem hiding this comment.
Fix inefficient double download of images.
The image is downloaded twice - once in this function and again in reduceImageBySize. Pass the buffer instead of the URL to avoid redundant network requests.
- const processedImage = await reduceImageBySize(ogData.image, 976);
+ const processedImage = imageBuffer.length / 1024 > 976
+ ? await reduceImageBySize(ogData.image, 976)
+ : imageBuffer;Consider refactoring reduceImageBySize to accept a buffer as an alternative to URL to improve efficiency.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| const imageResponse = await axios.get(ogData.image, { | |
| responseType: 'arraybuffer', | |
| timeout: 10000, | |
| headers: { | |
| 'User-Agent': 'Mozilla/5.0 (compatible; PostizBot/1.0; +https://postiz.com/)' | |
| } | |
| }); | |
| const imageBuffer = Buffer.from(imageResponse.data); | |
| // Reduce image size if needed (Bluesky has limits) | |
| const processedImage = await reduceImageBySize(ogData.image, 976); | |
| const blob = await agent.uploadBlob(new Blob([processedImage])); | |
| const imageResponse = await axios.get(ogData.image, { | |
| responseType: 'arraybuffer', | |
| timeout: 10000, | |
| headers: { | |
| 'User-Agent': 'Mozilla/5.0 (compatible; PostizBot/1.0; +https://postiz.com/)' | |
| } | |
| }); | |
| const imageBuffer = Buffer.from(imageResponse.data); | |
| // Reduce image size if needed (Bluesky has limits) | |
| - const processedImage = await reduceImageBySize(ogData.image, 976); | |
| + const processedImage = imageBuffer.length / 1024 > 976 | |
| + ? await reduceImageBySize(ogData.image, 976) | |
| + : imageBuffer; | |
| const blob = await agent.uploadBlob(new Blob([processedImage])); |
🤖 Prompt for AI Agents
In libraries/nestjs-libraries/src/integrations/social/bluesky.provider.ts
between lines 191 and 203, the image is downloaded twice: once directly with
axios and again inside reduceImageBySize using the URL. To fix this
inefficiency, refactor reduceImageBySize to accept an image buffer as input in
addition to a URL. Then, pass the already downloaded imageBuffer to
reduceImageBySize instead of the URL to avoid redundant network requests.
| interface OpenGraphData { | ||
| title?: string; | ||
| description?: string; | ||
| image?: string; | ||
| } | ||
|
|
||
| async function fetchOpenGraphData(url: string): Promise<OpenGraphData> { | ||
| try { | ||
| const response = await axios.get(url, { | ||
| timeout: 10000, | ||
| headers: { | ||
| 'User-Agent': 'Mozilla/5.0 (compatible; PostizBot/1.0; +https://postiz.com/)' | ||
| } | ||
| }); | ||
| const html = response.data; | ||
| const dom = new JSDOM(html); | ||
| const document = dom.window.document; | ||
|
|
||
| const getMetaContent = (property: string) => { | ||
| const element = document.querySelector(`meta[property="${property}"]`) || | ||
| document.querySelector(`meta[name="${property}"]`); | ||
| return element?.getAttribute('content') || ''; | ||
| }; | ||
|
|
||
| const ogImage = getMetaContent('og:image'); | ||
| let imageUrl = ogImage; | ||
|
|
||
| // Handle relative URLs for images | ||
| if (ogImage && !ogImage.startsWith('http')) { | ||
| try { | ||
| imageUrl = new URL(ogImage, url).href; | ||
| } catch { | ||
| imageUrl = ogImage; // Fallback to original if URL parsing fails | ||
| } | ||
| } | ||
|
|
||
| return { | ||
| title: getMetaContent('og:title') || getMetaContent('title') || | ||
| document.querySelector('title')?.textContent || '', | ||
| description: getMetaContent('og:description') || getMetaContent('description') || '', | ||
| image: imageUrl || '' | ||
| }; | ||
| } catch (error) { | ||
| console.error('Error fetching OpenGraph data:', error); | ||
| return {}; | ||
| } | ||
| } | ||
|
|
||
| function extractUrls(text: string): string[] { | ||
| const urlRegex = /https?:\/\/[^\s/$.?#].[^\s]*/g; | ||
| return text.match(urlRegex) || []; | ||
| } | ||
|
|
There was a problem hiding this comment.
🛠️ Refactor suggestion
Extract duplicated OpenGraph functions to a shared utility module.
The fetchOpenGraphData and extractUrls functions are duplicated across multiple providers. This violates the DRY principle.
Consider creating a shared utility file:
// libraries/nestjs-libraries/src/integrations/social/utils/opengraph.utils.ts
export interface OpenGraphData {
title?: string;
description?: string;
image?: string;
}
export async function fetchOpenGraphData(url: string): Promise<OpenGraphData> {
// ... implementation
}
export function extractUrls(text: string): string[] {
// ... implementation
}Then import and use these utilities in each provider instead of duplicating the code.
🤖 Prompt for AI Agents
In libraries/nestjs-libraries/src/integrations/social/facebook.provider.ts lines
14 to 66, the functions fetchOpenGraphData and extractUrls are duplicated across
multiple providers, violating the DRY principle. To fix this, create a new
shared utility module at
libraries/nestjs-libraries/src/integrations/social/utils/opengraph.utils.ts
containing the OpenGraphData interface and the implementations of
fetchOpenGraphData and extractUrls. Then, remove these functions from
facebook.provider.ts and import them from the new utility module to reuse the
code.
| return text.match(urlRegex) || []; | ||
| } | ||
|
|
||
| async function createLinkedInContentEntities(url: string, accessToken: string, personId: string, type: 'personal' | 'company', uploadPictureMethod: any): Promise<any> { |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Use union type and consider extracting shared LinkedIn utilities.
The type parameter should use a union type for better type safety. Also, this function is duplicated in linkedin.page.provider.ts.
-async function createLinkedInContentEntities(url: string, accessToken: string, personId: string, type: 'personal' | 'company', uploadPictureMethod: any): Promise<any> {
+// Consider moving to a shared LinkedIn utilities file
+async function createLinkedInContentEntities(
+ url: string,
+ accessToken: string,
+ personId: string,
+ type: 'personal' | 'company',
+ uploadPictureMethod: (filename: string, token: string, id: string, buffer: Buffer, type: 'personal' | 'company') => Promise<string>
+): Promise<any> {📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| async function createLinkedInContentEntities(url: string, accessToken: string, personId: string, type: 'personal' | 'company', uploadPictureMethod: any): Promise<any> { | |
| // Consider moving to a shared LinkedIn utilities file | |
| async function createLinkedInContentEntities( | |
| url: string, | |
| accessToken: string, | |
| personId: string, | |
| type: 'personal' | 'company', | |
| uploadPictureMethod: (filename: string, token: string, id: string, buffer: Buffer, type: 'personal' | 'company') => Promise<string> | |
| ): Promise<any> { |
🤖 Prompt for AI Agents
In libraries/nestjs-libraries/src/integrations/social/linkedin.provider.ts at
line 73, update the type parameter from a string literal union to a proper
TypeScript union type for better type safety. Additionally, refactor by
extracting this function into a shared LinkedIn utilities module to avoid
duplication with linkedin.page.provider.ts, then import and use the shared
function in both places.
|
Thanks! |
|
Definitely worth reviewing and merging into the main. @nevo-david |
Uses OpenGraph scraping for LinkedIn, API link parameter for Facebook, and app.bsky.embed.external for Bluesky. Adds PostizBot user agents to metadata fetching to make it distinguishable to firewalls.
What kind of change does this PR introduce?
Before, Postiz posts to Bluesky, LinkedIn, and Facebook would appear without dedicated link previews. They'd appear as plain URLs, without the headline/description/thumbnail you'd get if you posted to the platforms directly. This PR adds custom code for those three platforms to deliver those URLs as distinct objects in the ways their various APIs prefer. Now there are link previews!
Why was this change needed?
I was frustrated seeing these subpar posts appear on social networks important to my website, and it was preventing me from pushing to adopt Postiz for our organization — knowing that the output wasn't as good in this particular way as Buffer or other competitors. (This PR also fixes #667.)
Other information:
Checklist:
Put a "X" in the boxes below to indicate you have followed the checklist;
Summary by CodeRabbit
New Features
Bug Fixes