Merge branch 'release/0.6.0' into main

Author: djsudduth

.gitignore

@@ -2,6 +2,7 @@ settings*.cfg
 skiptest.py
 fuzzy.py
 zz*.py
+configs.txt
 mdfiles/
 backup/
 .vscode/

Dockerfile

@@ -1,11 +1,10 @@
 FROM ubuntu:22.04
 RUN apt update
 RUN apt install -y python-is-python3 python3-pip
+RUN apt-get install -y git
 RUN apt-get install -y curl
 RUN apt-get install ca-certificates
-RUN curl -L -o tmp/keep-it-markdown-0.5.4.tar.gz https://github.com/djsudduth/keep-it-markdown/archive/refs/tags/0.5.4.tar.gz
-RUN tar -zxvf tmp/keep-it-markdown-0.5.4.tar.gz
-RUN pip install -r keep-it-markdown-0.5.4/requirements.txt
-RUN pip install requests==2.23.0
-RUN pip install gpsoauth==1.0.2
-RUN pip install keyrings.alt
+RUN curl -L -o tmp/keep-it-markdown-0.6.0.tar.gz https://github.com/djsudduth/keep-it-markdown/archive/refs/tags/0.6.0.tar.gz
+RUN tar -zxvf tmp/keep-it-markdown-0.6.0.tar.gz
+RUN pip install keyrings.alt
+RUN pip install git+https://github.com/simon-weber/gpsoauth.git@8a5212481f80312e06ba6e0a29fbcfca1f210fd1

INSTALL.md

@@ -0,0 +1,16 @@
+## Installation Steps for Advanced Users (only needs to run once)
+- Python 3.10+ required (or use pyenv for a specific new version)
+- Install git 
+- Download latest version of `keep-it-markdown` from releases and unzip it to your working directory
+- Create a fresh virtual environment within the working directory 
+- Activate the virtual environment 
+- Install lastest `gpsoauth` package - `pip install git+https://github.com/simon-weber/gpsoauth.git@8a5212481f80312e06ba6e0a29fbcfca1f210fd1`
+- Follow **Second Way** instructions here to get a copy of the oauth_token cookie value - https://github.com/rukins/gpsoauth-java?tab=readme-ov-file
+- Run the script in the KIM directory - `python get_token.py`
+- Enter your Google email account name, oauth_token, and Android ID when prompted (Android ID can be anything, OAuth token expires in about 5 min)
+- Copy the Keep Token value output from `get_token.py` (do not execute the next step unless you have the token)
+- Install KIM dependencies in the venv - `pip install -r requirements.txt`
+- Linux users - you may need to install a Keyring - `pip install keyrings.alt`
+- Run KIM to save the Token in your local Keyring - `python kim.py -t <your long token value here>`
+
+Congratulations! You're done! Your Token is valid for all KIM versions now (unless Google changes it). You can now run KIM locally from this point on (see `kim.py --help` for switch options) 

README.md

@@ -6,47 +6,75 @@ Keep-it-markdown or KIM converts Google Keep notes to markdown using the unoffic
 The overall goal is to utilize Google Keep as an easy way to capture raw notes on all devices or additionally using the browser plugin. Then, notes can be queried for export to markdown files directly into notetaking apps such as Obsidian, Logseq and/or Notion, or used directly with Typora. 
 
 ## Installation
-Install assumes you have some familiarity with running scripts through a terminal or command line. KIM is a command line script that **requires Python 3.8 or greater** and utilizes the unofficial gkeepapi. (**If you have Python versions 3.10+ and have login issues you may need to review Advanced Docker Setup below**)
+**Advanced Users:** see the **INSTALL.md** to skip over these installation details for simple step-by-steps  
+
+Install assumes you have some familiarity with running scripts through a terminal or command line. KIM is a command line script that **requires Python 3.10 or greater** and utilizes the unofficial gkeepapi. (**If you have Python versions 3.10+ and have login issues you may need to review Advanced Docker Setup below**)
 
 **NOTE: Be aware that 'unofficial' implies that Google could change the API at any time that might stop the script from working!**
 
-You only need to run installation steps 1 through 4 one time.
+You only need to run installation steps 1 through 6 one time.
 
 #### Step 1: 
-Install Python (there are plenty of tutorials online for installation instructions) on you PC or Mac. Start your command prompt, shell or terminal and verify your python version by running:
+Install Python (there are plenty of tutorials online for installation instructions) on you PC or Mac. If you have an older version of Python (3.8 or 3.9) installed and you want to leave it, you can install pyenv to run multiple versions. Start your command prompt, shell or terminal and verify your python version by running:
 ```bash
 > python --version
 ```
-If you had Python 2 installed already you may need to type 'python3' instead of just 'python' for the rest of these steps to use version 3.8+.
+If you had Python 2 installed already you may need to type 'python3' instead of just 'python' for the rest of these steps to use version 3.10+.
 
 #### Step 2: 
 Download this project's zip file into any new directory of you choice. Select the most current release noted at the top right of this page and download 'Source code' using this link:  
 https://github.com/djsudduth/keep-it-markdown/releases
 
-Unzip the files within your chosen directory.  
+Unzip the files within your chosen directory.  **DO NOT RUN REQUIREMENTS.TXT YET!**  
+
 
 #### Step 3:
-Start your command prompt, shell or terminal, find your download directory and run 
+KIM requires a Google Keep authentication token in order to run. The token can only be retrieved once you have a web page OAuth cookie. To do this it is important that you have a fresh Python install or a new virtual environment (venv) setup.
+
+Once you've setup a fresh environment, install the Chrome extension called 'Cookie Tab Viewer'. Change the directory to where you installed KIM. You must also install `git` on your PC. Installation is simple here -> https://git-scm.com/downloads  
+
+One Python module is needed to get the token. Run this command on your PC:  
+```bash
+> pip install git+https://github.com/simon-weber/gpsoauth.git@8a5212481f80312e06ba6e0a29fbcfca1f210fd1
+```
+
+#### Step 4:
+Here's the tricky part - you need to get your OAuth token from a Google cookie. To get the OAuth token - follow the **"Second way"** instructions here (but get the cookie value using the Chrome extension once you've pressed "I agree" on the Google page):
+https://github.com/rukins/gpsoauth-java?tab=readme-ov-file  
+
+Copy the cookie called `oauth_token` using the Chrome Cookie Tab Viewer from the cookies in your local browser. Then, run the script
+```bash
+> python get_token.py
+```
+You will be prompted for your Google email account name, OAuth token, and Android ID
+
+The AndroidID can just be a random value like: `abcdef123`  
+
+So, when you get the prompt when running the script:  
+**Email:** your google ID  
+**OAuth Token:** oauth2_4/......rest of token  
+**Android ID:** abcdef123  
+
+The Keep token should be displayed - it should look like:  
+"aas_et/FKcp.............lots of characters.....................BjQ="  
+
+Copy that token and save it in a safe place! If it didn't work your OAuth token may have expired (takes about 5 min to expire). Run this step again until you get the token.  
+
+#### Step 5:
+Make sure you are in the KIM directory to install all needed dependencies with:  
 ```bash
 > pip install -r requirements.txt
 ```
 (you may need to use 'pip3' instead of 'pip' if you have both python versions 2 and 3 installed) This will install the additional libraries needed to run KIM. You only need to do this once. If you have Anaconda as your Python base you may need to find tutorials on how to get pip and install dependencies. Advanced users may want to setup a virtual environment for this.
 
-#### Step 4: 
-This script was written before the official Google Keep API was available. The Google Keep API is currently only available to workspace users and not individuals. So, you must manually create an application password for authentication purposes. 
-
-1) Navigate to your [Google Account Page](https://myaccount.google.com)
-2) Click on the Security option on the left
-3) Under the **Signing in to Google** header select **App passwords**
-4) Choose **Select App** and select **Other**, the name can be anything you want but I suggest something identifiable such as "**KeepItMarkdown**"
-5) Click on Generate and Copy the password that's generated
-6) Run the following command and logon with your Google email and the newly generated application password. 
+#### Step 6: 
+You now need to save your Keep token within the KIM secure keyring
 ```bash
-> python kim.py
+> python kim.py -t <your long token value here>
 ```
-If you entered your Google email and application password correctly, you should see a successful login with the statement -> "You've succesfully logged into Google Keep!"
+If you entered your Google email and token correctly, you should see a successful login with the statement -> "You've succesfully logged into Google Keep!"
 
-**If this step keeps failing see 'Key Callouts' #9 below, or, if you are using Python 3.10 or greater and have issues with the login - see Advanced Docker Setup below or read this note: https://github.com/djsudduth/keep-it-markdown/issues/72**
+**If this step keeps failing see 'Key Callouts' #9 below, and have issues with the login - see Advanced Docker Setup below or read this note: https://github.com/djsudduth/keep-it-markdown/issues/72**
 
 ## Usage
 Congrats! You can now run KIM. Simply start by running:  
@@ -178,24 +206,27 @@ Note: skip -s and overwrite -o cannot be used at the same time
 9. There seems to be login issues due to some of the authentication and security library changes with Google and Python. Take a look at this note -> https://github.com/djsudduth/keep-it-markdown/issues/72 or use the Advanced Docker Setup in the next section
 
 ## Advanced Docker Setup
-If you are having difficulty logging in to Google you can use Docker with the preconfigured OS and Python version to access KIM and save your exported notes (see alternative step 7 if you want to save the Keep token on your PC).
+If you are having difficulty logging in to Google you can use Docker with the preconfigured OS and Python version to access KIM and save your exported notes (see alternative step 12 if you want to save the Keep token on your PC).
 
 **Steps:**
 1) Install Docker on any PC (find the online instructions for your particular operating system)
 2) Startup Docker (or it will autostart on reboot depending on how you installed it)
 3) Go to the command line and run ``docker build -t kim .`` in the directory where you installed KIM (it will take about 5 min to create the image)
-4) Run the Docker image with ``docker run --mount type=bind,source=(your PC's KIM directory)/mdfiles,target=/keep-it-markdown-0.5.4/mdfiles -it kim`` (you will be automatically logged into the Docker image and your PC's directory will be mapped to the Docker image directory)
-5) Change the directory to Kim ``cd keep-it-markdown-0.5.4``
-6) Create a temporary app password on Google
+4) Run the Docker image with ``docker run --mount type=bind,source=(your PC's KIM directory)/mdfiles,target=/keep-it-markdown-0.6.0/mdfiles -it kim`` (you will be automatically logged into the Docker image and your PC's directory will be mapped to the Docker image directory)
+5) Change the directory to Kim ``cd keep-it-markdown-0.6.0``
+6) Install lastest `gpsoauth` package - `pip install git+https://github.com/simon-weber/gpsoauth.git@8a5212481f80312e06ba6e0a29fbcfca1f210fd1`
+7) Follow **Second Way** instructions here to get a copy of the oauth_token cookie value - https://github.com/rukins/gpsoauth-java?tab=readme-ov-file
+8) Run the script in the KIM directory - `python get_token.py`
+9) Enter your Google email account name, oauth_token, and Android ID when prompted (Android ID can be anything, OAuth token expires in about 5 min)
+10) Copy and save the Keep Token value output from `get_token.py` on your PC
 ---
-7) For one time or sporatic use, run KIM per the instructions above (note that in Docker python3 is aliased to python) - your exported notes will be exported to your PC. NOTE, however, that running Docker this way will not save any passwords or exported notes when you exit and you may need to recreate Google app passwords each time you use KIM with Docker this way.(Exit the Docker image with ``exit``)
+11) For one time or sporatic use, install the KIM dependencies with `pip install -r requirements.txt` and then run KIM in Docker with ``python kim.py -t <long token here>`` using the saved Token above (note that in Docker python3 is aliased to python) - your exported notes will be exported to your PC. NOTE, however, that running Docker this way will not save any passwords or exported notes when you exit and you will need use the saved Token each time you use KIM with Docker this way.(Exit the Docker image with ``exit``)
 ---
-7) Alternatively, run ``python keep-test.py -t`` in the Docker image to log in and display the Keep token (keep token will **appear be very long** - almost 2 lines)
-8) Copy the token by highlighting the entire string and hitting enter
-9) Paste and save the token somewhere safe
-10) Exit the Docker image with ``exit``
-11) Now run KIM in your current OS with the -t switch once to save it in the keystore (``python kim.py -t <long token here>`` - you may need a new Google app password to do this)
-12) You can now run KIM on any PC (once you save the token) with Python v-3.8 or higher
+12) Alternatively, exit the Docker image with ``exit``
+13) Download and install KIM in your current OS
+14) Install KIM dependencies your PC using `pip install -r requirements.txt`
+15) Run KIM with the -t switch once to save the Token in your PC keystore (``python kim.py -t <long token here>``)
+16) You can now run KIM on your PC (once you save the token) with Python v-3.10 or higher without having to run these steps again. The Token will be saved in your local PC's keystore
 
 ## Obsidian Use
 Since KIM converts Google Keep notes to markdown, you can use some of the Obsidian text markdown features in your Keep notes as you're capturing information. For example, you can begin to cross-link notes in Keep by using the Wikilink double-brackets within a note like this [[Title of another Keep note]]. Then, when you convert your notes to the Obsidian vault they will be automatically linked. This will also work for block references and other markdown notation. Most markdown types in Keep notes should convert successfully even if Keep cannot render them. **Do not try to add markdown for links/URLs in Keep**. KIM will try to map link any of Keep's URLs to markdown format for you.
@@ -232,11 +263,6 @@ Thanks for trying this markdown converter! I hope you find it useful!
 There's always room for improvement. Feel free to add issues to the issues list.
 
 
-## 0.5.1 Recent Changes
-Fixed image overwrite if note has no title or text and using -c switch  
-Fixed error of markdown note imports if there are special characters within  
-Added create and update dates of markdown files to imported notes  
-
 ## 0.5.2 Recent Changes
 Switched audio file extensions from AAC back to M4A  
 Added Joplin exports -j flag to use front matter header  
@@ -249,4 +275,10 @@ Removed captcha note in keep-test.py
 ## 0.5.4 Recent Changes
 Docker image altered to use Ubuntu:22.04 to fix Google auth issues with gkeepapi  
 Added new flag -m to move exported images to Archive folder  
-Removed python deprecated imghdr library with pillow module  
+Removed python deprecated imghdr library with pillow module  
+
+## 0.6.0 Recent Changes
+Now requires Python v-3.10+ to run KIM  
+New Docker image to get the Keep token  
+Old keep-test.py module removed for new Google authentication (get_token.py added)
+New simple INSTALL.md steps

get_token.py

@@ -0,0 +1,13 @@
+import gpsoauth as go
+
+try:
+   print ("\nEnter your Google ID, the copied oauth_token cookie, and any value for the Android ID:\n")
+   t = dict(go.exchange_token(input('Google ID: '), input('oauth_token Cookie: '), input('Android ID: ')))
+   final_token = t["Token"]
+   print ("\nCongratulations! This is your Google Keep token - you only need to extract this once. Please copy and run 'python kim.py -t <your token here>' to save the token key in your keystore. Keep-it-markdown should work on your PC from now on!\n")
+   print (final_token)
+except Exception as e:
+   print ("\nEither you entered your ID and OAuth cookie incorrectly or the OAuth cookie has expired. Try getting the OAuth cookie and run this again - the cookie expires within 5 minutes.")
+
+
+

keep-test.py

@@ -14,7 +14,7 @@
 from xmlrpc.client import boolean
 from PIL import Image
 
-
+KIM_VERSION = "0.6.0"
 KEEP_KEYRING_ID = 'google-keep-token'
 KEEP_NOTE_URL = "https://keep.google.com/#NOTE/"
 CONFIG_FILE = "settings.cfg"
@@ -723,7 +723,7 @@ def main(r, o, a, p, s, c, l, j, m, i, search_term, master_token):
 
         #m = True
         opts = Options(o, a, p, s, c, l, j, m, i)
-        click.echo("\r\nWelcome to Keep it Markdown or KIM!\r\n")
+        click.echo("\r\nWelcome to Keep it Markdown or KIM " + KIM_VERSION + "!\r\n")
 
         if i and (r or o or a or s or p or c or m):
             print ("Importing markdown notes with export options is not compatible -- please use -i only to import")
@@ -749,8 +749,6 @@ def main(r, o, a, p, s, c, l, j, m, i, search_term, master_token):
     #    raise Exception("Problem with markdown file creation: " + repr(e))
 
 
-#Version 0.5.4
-
 if __name__ == '__main__':
 
     main()  # pylint: disable=no-value-for-parameter