Skip to content

CLOUDP-308303: size of postman file #600

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 3 commits into from
Closed

CLOUDP-308303: size of postman file #600

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
67b68ee
CLOUDP-308303: size of postman file
wtrocki Mar 25, 2025
1ce77d4
fix: reduce size to 4mb
wtrocki Mar 25, 2025
97e3887
fix: remove whitespaces from the collection
wtrocki Mar 25, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions .github/workflows/release-postman.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,8 +48,7 @@ jobs:
BASE_URL: ${{ inputs.atlas_prod_base_url }}
working-directory: ./tools/postman
run: |
make transform_collection
make transform_collection_test
make transform_collection_js

- name: Upload Collection to Postman
env:
Expand Down
15 changes: 7 additions & 8 deletions tools/postman/scripts/transform-for-api.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
#!/usr/bin/env bash
set -euo pipefail

## NOTE: Use JS script instead.
## Script is kept only for backwards compatibilty

#########################################################
# Prepare collection for Postman API
# Environment variables:
Expand All @@ -27,7 +30,6 @@ DESCRIPTION_FILE=${DESCRIPTION_FILE:-"../collection-description.md"}

TOGGLE_INCLUDE_BODY=${TOGGLE_INCLUDE_BODY:-true}
TOGGLE_ADD_DOCS_LINKS=${TOGGLE_ADD_DOCS_LINKS:-true}
TOKEN_URL_ENV=${TOKEN_URL_ENV:-""}

current_api_revision=$(<"$OPENAPI_FOLDER/$VERSION_FILE_NAME")

Expand Down Expand Up @@ -117,13 +119,10 @@ else
cp intermediateCollectionWithLinks.json intermediateCollectionPostBody.json
fi

if [ "$TOKEN_URL_ENV" != "" ]; then
echo "Adding client credentials auth url variable $TOKEN_URL_ENV"
jq --arg token_url "$TOKEN_URL_ENV" '.collection.variable += [{"key": "clientCredentialsTokenUrl", "value": $token_url}]' \
intermediateCollectionPostBody.json > "$COLLECTION_TRANSFORMED_FILE_NAME"
else
cp intermediateCollectionPostBody.json "$COLLECTION_TRANSFORMED_FILE_NAME"
fi
# Remove trailing whitespaces by reformatting the JSON
echo "Removing trailing whitespaces from the final JSON file"
jq '.' "$COLLECTION_TRANSFORMED_FILE_NAME" > tmp.json && mv tmp.json "$COLLECTION_TRANSFORMED_FILE_NAME"


# Clean up temporary files
echo "Removing temporary files"
Expand Down
35 changes: 32 additions & 3 deletions tools/postman/scripts/transform-postman.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,8 @@ const OPENAPI_FOLDER = process.env.OPENAPI_FOLDER || './openapi';
const TMP_FOLDER = process.env.TMP_FOLDER || './tmp';
const VERSION_FILE_NAME = process.env.VERSION_FILE_NAME || 'version.txt';
const DESCRIPTION_FILE = process.env.DESCRIPTION_FILE || './collection-description.md';
const TOGGLE_INCLUDE_BODY = process.env.TOGGLE_INCLUDE_BODY !== 'false';
const TOGGLE_ADD_DOCS_LINKS = process.env.TOGGLE_ADD_DOCS_LINKS === 'true';
const TOGGLE_INCLUDE_BODY = false;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have access but is this currently enabled in env vars?
I take it by disabling this we lose the request body template, which seems like a decent value add for customers using the API for the first time

image

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. This PR explores options to see what elements add size. I commited those changes to evaluate what is the lowest we can go and then we come back enable what is broken and required.

const TOGGLE_ADD_DOCS_LINKS = false;
const TOKEN_URL_ENV = process.env.TOKEN_URL_ENV || '';
const BASE_URL = process.env.BASE_URL || '';

Expand Down Expand Up @@ -67,6 +67,35 @@ const transform = () => {
console.log(`Adding baseUrl property ${BASE_URL}`);
collection.collection.variable.push({ key: 'baseUrl', value: BASE_URL });

console.log('Removing redundant fields to reduce size');
_.forEach(collection.collection.item, (item) => {
if (item.request) {
delete item.request.description; // Remove request descriptions
delete item.request.auth; // Remove unused auth fields

if (item.request.body && item.request.body.mode === 'raw') {
delete item.request.body.raw; // Remove raw fields from body
}
}
if (item.response) {
item.response.forEach((response) => {
delete response.originalRequest; // Remove original request from responses
delete response._postman_previewlanguage; // Remove preview language metadata
delete response.header; // Remove headers from responses
response.body = ''; // Clear response body
});
}
});

console.log('Removing empty arrays and objects');
collection = JSON.parse(
JSON.stringify(collection, (key, value) => {
if (Array.isArray(value) && value.length === 0) return undefined;
if (value && typeof value === 'object' && Object.keys(value).length === 0) return undefined;
return value;
})
);

if (TOGGLE_ADD_DOCS_LINKS) {
console.log('Adding links to docs for each request');

Expand Down Expand Up @@ -119,7 +148,7 @@ function loadJsonFile(filePath) {
}

function saveJsonFile(filePath, json) {
fs.writeFileSync(filePath, JSON.stringify(json, null, 2), 'utf8');
fs.writeFileSync(filePath, JSON.stringify(json, null, 0), 'utf8');
}

// hack
Expand Down