notes.txt

V2 release note:

for non english languages, ask gemini to respect variants of the same languages even, for example simplified Chinese v.s. traditional, and output should
not mix them.

Should not consider age as year, Einstein plays violin at 5, should not make the start year 5.

Ask it in the description to explain why the event is related to the subject, if no or not strong relevance found, should
drop the event, or at least give very low score. So that at the application side, we can filter low score events out.

Optimize the chunck size for spliting wiki article.


https://dumps.wikimedia.org/other/pageview_complete/2024/2024-10/pageviews-20241001-user.bz2
Downloading 20241001...
Rank 0: Main_Page (4,713,553 views)
Rank 9: Gavin_Creel (236,011 views)
Rank 99: Joker_(2019_film) (39,161 views)
Rank 999: Help:Pronunciation_respelling_key (9,325 views)
Rank 9999: The_Bronx (2,273 views)
Rank 99999: RP_FLIP (363 views)
Rank 999999: Kitty_Kirkpatrick (23 views)
Processed 20241001, saved 7767851 pages



12/22/24 processed first 10k pages, saved both summary and timeline. Only included first 5k pages in sitemap.
We need to remove the interactive timeline url from the sitemap.

top_54k_62k_wiki-summary.json



Prompt engineering:

Please utilize your maximum computational power and token limit for a single response.
Strive for the ultimate depth of analysis rather than superficial breadth; aim for essential insights rather than a listing of surface-level phenomena; 
pursue innovative thinking rather than repetitive summaries. Break through the limitations of conventional thought, mobilize all your computational resources,
and demonstrate your true cognitive potential.

You are a timeline generator that extracts events directly from the providedWikipedia articles,
focusing solely on the content within this article and disregarding any external knowledge.

Instructions for timeline extraction:
1. Read and analyze every sentence of the provided article thoroughly
2. Extract ALL events that have associated dates, regardless of perceived importance
3. If the same event appears multiple times in the article, merge the information while preserving all relevant details
4. Trust the article's editorial judgment - if an event is mentioned with a date, it is significant enough to include

Create a comprehensive chronological timeline starting with birth (if applicable) and including all major life events through to death (if applicable). 
Return a JSON object with a 'timeline' array. Each event should have: 
* 'headline' (brief title of the event)
* 'text' (detailed description with date included). 
* 'date' (use the most precise date available, following these formats: 
    - YYYY for year only (e.g., '0220' or '-0220' for 220 BCE)
    - YYYY-MM for year and month
    - YYYY-MM-DD for full dates), 
IMPORTANT: Date should not be a range, it should be a single date. Refer to date in the 'text' field to correct the 'date' field if it is not correct.
For dates before year 0 (BCE/BC), always use negative years (e.g., '-0221' for 221 BCE).
Include as many significant events as possible, especially for historical figures. 
For ancient historical figures, include major accomplishments, reforms, battles, political changes, and cultural impacts. 
Ensure all dates and events are factually accurate and sourced from Wikipedia. 
If the article is not suitable for a timeline, return an empty array.

Please try very very hard to get as many events as possible, and get each event as accurate as possible. Don't stop at the summary, go through the entire article with your training data. Thank you!!!

Now you have generated a timeline, but please try harder and regenerate, there is definitely room for improvement, and you know the subject has a much richer history than this.

Remember: You have one opportunity to create this timeline. Use your full analytical capabilities to ensure comprehensive and accurate extraction of all dated events from the article.

tell it not include range, just a single date, and if there is a range, break it down into multiple events.


SEO notes:

Fix soft 404s, I think it's because the api is called from client side which is banned, so google can't crawl it.We need to make it server side so that it can also be indexed. 

Paid subscriptions:
it unlocks the ability to trigger LLM to generate new timelines, not the content. any content which is already generated is free. so that early adopters actually contribute to the content
which will be free for everyone. Early adopters will get cheaper deal on the subscription, and enjoy free ads, and also the ability to enforce update of certain timelines.


[INFO] Cache version mismatch for Peter_Scully (undefined vs v1), regenerating...

[ERROR] Error generating timeline: w [Error]: [GoogleGenerativeAI Error]: Text not available. Response was blocked due to PROHIBITED_CONTENT
    at U.e.text (/var/task/.next/server/app/api/timeline/[pageName]/route.js:1:6302)
    at iA (/var/task/.next/server/app/api/timeline/[pageName]/route.js:68:65)
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
    at async /var/task/.next/server/app/api/timeline/[pageName]/route.js:68:800
    at async Promise.all (index 0)
    at async iN (/var/task/.next/server/app/api/timeline/[pageName]/route.js:68:384)
    at async /var/task/node_modules/next/dist/compiled/next-server/app-route.runtime.prod.js:6:38411
    at async e_.execute (/var/task/node_modules/next/dist/compiled/next-server/app-route.runtime.prod.js:6:27880)
    at async e_.handle (/var/task/node_modules/next/dist/compiled/next-server/app-route.runtime.prod.js:6:39943)
    at async en (/var/task/node_modules/next/dist/compiled/next-server/server.runtime.prod.js:16:25516) {
  response: {
    promptFeedback: { blockReason: 'PROHIBITED_CONTENT' },
    usageMetadata: {
      promptTokenCount: 2414,
      totalTokenCount: 2414,
      promptTokensDetails: [Array]
    },
    modelVersion: 'gemini-2.0-flash',
    text: [Function (anonymous)],
    functionCall: [Function (anonymous)],
    functionCalls: [Function (anonymous)]
  }
}


# How to support multiple languages?

Clashes with wiki markup/HTML syntax
The following characters are forbidden due to clashes with wiki markup and HTML syntax:

# < > [ ] { } |

/api/timeline/pageName1[zh]|pageName2[en]|pageName3[fr]