BUG - generate changelog does not work with linux sort command (does not sort correctly)

[reactive-firewall]

Describe the bug

A clear and concise description of what the bug is.

generate_changelog.sh expects BSD sort for sorting what is called dictionary order (e.g. -d) however on ubuntu it seems that the sort command (unknown version for now) does not sort the change-log entries correctly or even consistently at all.

To Reproduce

Steps to reproduce the behavior:

1. use BSD sort environment (e.g. macOS) to run generate_changelog.sh HEAD...stable > /tmp/bsd_changelog.txt
2. use Ubuntu sort environment to run generate_changelog.sh HEAD...stable > /tmp/linux_changelog.txt
3. collect both bsd_changelog.txt and linux_changelog.txt into a single environment and check with git diff --no-index bsd_changelog.txt linux_changelog.txt
4. See error in sorting

Expected behavior

A clear and concise description of what you expected to happen.

There should not be such a difference between the generated change-logs that the majority of the contents are sorted differently (a few differences in very similar lines would be fine, but not un-sorted)

should probably add function to sort, check sorted, and loop until working or fail at timeout

  graph TD;
    input-->BUFFER;
    BUFFER-->sort;
    sort-->BUFFER;
    BUFFER-->output;

Loading

sequenceDiagram
  participant script as "generate_changelog.sh"
  participant func as Sort Function

  script->>func: call function with INPUT
  create participant BUFFER as string buffer
  func-->>BUFFER: Initialize buffer with INPUT
  loop: while unsorted
    create participant Sort as sort command
    func-->>Sort: sort buffer
    BUFFER->>Sort: does sort stuff
    Sort-->>BUFFER: update BUFFER with sorted results
    Sort->>func: done
    func-->>Sort: check buffer is indeed sorted
    BUFFER->>Sort: does sort check stuff
    alt if sorted
      Sort->>func: buffer is sorted
      func-->>func: Exit Loop
    else not sorted
      destroy Sort
      Sort->>func: buffer is Unsorted
      func-->>func: Continue Loop
    end
  end
  destroy BUFFER
  BUFFER->>func: drain buffer as output
  func->>script: return sorted output

Loading

Additional context (optional)

• see generate_changelog.sh

Namely:

multicast/generate_changelog.sh

Line 341 in bd23f96

' <"${CHANGELOG_BUFFER}" | sort -id | uniq | sort -rd

[reactive-firewall]

AI assisted (subject to further review as per our AI Policy):

# USAGE:
#   ~$ robust_sort_file INPUT_FILE OUTPUT_FILE [MAX_RETRIES] [SORT_ARGS...]
# Arguments:
#   INPUT_FILE   -- File to sort (required)
#   OUTPUT_FILE  -- Destination for sorted output (required)
#   MAX_RETRIES  -- Optional: Maximum attempts (default: 5)
#   SORT_ARGS    -- Optional: Additional arguments to sort (e.g., -d, -k2, etc.)
# Results:
#   Returns 0 on success, nonzero otherwise.
#   Writes sorted output to OUTPUT_FILE, or fails with message.

function robust_sort_file() {
    local INPUT_FILE="$1"
    local OUTPUT_FILE="$2"
    local MAX_RETRIES="${3:-5}"
    shift 3 || shift $#
    local SORT_ARGS=("$@")

    test -f "$INPUT_FILE" || { printf "Error: INPUT_FILE not found\n" >&2; return 65; }
    test -r "$INPUT_FILE" || { printf "Error: INPUT_FILE not readable\n" >&2; return 77; }

    local attempt=1
    local TMP_SORTED
    TMP_SORTED="$(mktemp "${TMPDIR:-/tmp}/robust_sort_XXXXXX")"

    while (( attempt <= MAX_RETRIES )); do
        # Try with current locale
        LC_ALL="${LC_ALL:-en_US.utf-8}" sort "${SORT_ARGS[@]}" "$INPUT_FILE" > "$TMP_SORTED" 2>/dev/null

        # Check if sorted
        if cmp -s <(sort "${SORT_ARGS[@]}" "$TMP_SORTED") "$TMP_SORTED"; then
            mv "$TMP_SORTED" "$OUTPUT_FILE"
            return 0
        fi

        # Try with LC_ALL=C as fallback on next attempt
        if (( attempt == MAX_RETRIES )); then
            LC_ALL=C sort "${SORT_ARGS[@]}" "$INPUT_FILE" > "$TMP_SORTED" 2>/dev/null
            if cmp -s <(LC_ALL=C sort "${SORT_ARGS[@]}" "$TMP_SORTED") "$TMP_SORTED"; then
                mv "$TMP_SORTED" "$OUTPUT_FILE"
                return 0
            fi
        fi

        sleep 0.5
        (( attempt++ ))
    done

    rm -f "$TMP_SORTED"
    printf "Error: Could not reliably sort %s after %d attempts\n" "$INPUT_FILE" "$MAX_RETRIES" >&2
    return 1
}
# export for subshells
export -f robust_sort_file