Skip to content

Tutorial timestamps #2876

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 7 commits into from
Closed
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .jenkins/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,9 @@ if [[ "${JOB_TYPE}" == "worker" ]]; then
# Files to run must be accessible to subprocessed (at least to `download_data.py`)
export FILES_TO_RUN

# Step 2.1: Add timestamps to .py and .rst files in source directories
bash $DIR/update_timestamps_batch.sh .

# Step 3: Run `make docs` to generate HTML files and static files for these tutorialis
pip3 install -e git+https://github.com/pytorch/pytorch_sphinx_theme.git#egg=pytorch_sphinx_theme
make docs
Expand Down Expand Up @@ -118,6 +121,10 @@ if [[ "${JOB_TYPE}" == "worker" ]]; then
7z a worker_${WORKER_ID}.7z docs
awsv2 s3 cp worker_${WORKER_ID}.7z s3://${BUCKET_NAME}/${COMMIT_ID}/worker_${WORKER_ID}.7z
elif [[ "${JOB_TYPE}" == "manager" ]]; then

# Step 0.9: Add timestamps to .py and .rst files in source directories
bash $DIR/update_timestamps_batch.sh .

# Step 1: Generate no-plot HTML pages for all tutorials
pip3 install -e git+https://github.com/pytorch/pytorch_sphinx_theme.git#egg=pytorch_sphinx_theme
make html-noplot
Expand Down
57 changes: 57 additions & 0 deletions .jenkins/update_timestamps.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
import re
import sys
import os
import subprocess

def get_last_commit_timestamp_for_file(file_path: str) -> str:
"""Get the last commit timestamp for a file.

Args:
file_path (str): Path to file

Returns:
str: Last committed timestamp string
"""
git_command = ["git", "log", "-1", "--format=%at", "--", file_path]
timestamp = subprocess.check_output(git_command).decode().strip()

if not timestamp:
# If there is no git commit history, use last modified date
timestamp = str(int(os.path.getmtime(file_path)))

date_command = ["date", "-d", "@" + timestamp, "+%I:%M %p, %B %d, %Y"]
return subprocess.check_output(date_command).decode().strip()

def update_timestamp(file_path: str):
"""Adds a timestamp of the most recent time the file was edited.

Args:
file_path (str): Path to file
"""
with open(file_path, 'r') as file:
lines = file.readlines()

# Get current timestamp
timestamp = get_last_commit_timestamp_for_file(file_path)
timestamp_line = f'{"# " if file_path.endswith("py") else ""}**Updated:** *{timestamp}*\n'
timestamp_pattern = r'\*\*Updated:\*\*\s\**\d{1,2}:\d{2} [AP]M, \w+ \d{1,2}, \d{4}\*'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would rather try to avoid regexes like this

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I updated it! For my own knowledge, may I ask why? Is it because it's too specific? I was trying to be specific just so I didn't grab any lines unexpectedly from the tutorial files.


if not lines:
lines = [timestamp_line]
else:
i = len(lines) - 1
while i > 0 and not lines[i].strip():
i -= 1

if re.search(timestamp_pattern, lines[i]):
lines[i] = timestamp_line
else:
lines.append('\n\n' + timestamp_line)

# Write updated lines back to file
with open(file_path, 'w') as file:
file.writelines(lines)


file_path = sys.argv[1]
update_timestamp(file_path)
16 changes: 16 additions & 0 deletions .jenkins/update_timestamps_batch.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
#!/bin/bash

SOURCEDIR=$1

DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null && pwd )"

directories=("$SOURCEDIR/beginner_source" "$SOURCEDIR/intermediate_source" "$SOURCEDIR/advanced_source")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have other directories as well and what if we add a new gallery directory, this won't work anymore? You could try perhaps to get them from the conf.py sphinx-gallery settings?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I updated it to get the files from the python script get_sphinx_filenames. This is what is used in conf.py, so can I assume that is a complete set?


for dir in "${directories[@]}"; do
# Process .py and .rst files in the current directory
for file in "$dir"/*.{py,rst}; do
if [ -f "$file" ]; then
python "$DIR/update_timestamps.py" "$file"
fi
done
done