'NoneType' object has no attribute 'get'

[emhl]

when trying to run ols list i get the following response:

💥  Querying all projects
Error: Querying all projects failed. Please try again.

with verbose logging i get this traceback additionally

Traceback (most recent call last):
  File "/home/emil/.local/lib/python3.10/site-packages/olsync/olsync.py", line 348, in execute_action
    success = action()
  File "/home/emil/.local/lib/python3.10/site-packages/olsync/olsync.py", line 175, in <lambda>
    lambda: overleaf_client.get_project(project_name),
  File "/home/emil/.local/lib/python3.10/site-packages/olsync/olclient.py", line 104, in get_project
    BeautifulSoup(projects_page.content, 'html.parser').find('meta', {'name': 'ol-projects'}).get('content'))
AttributeError: 'NoneType' object has no attribute 'get'

i reinstalled all python dependencies and qt6-webengine and the problem still persists

[adibMosharrof]

ols used to work for me before, but recently I am facing the same issue.

[Lightup1]

ols used to work for me before, but recently I am facing the same issue.

me too

[gyy0592]

when trying to run ols list i get the following response:

💥  Querying all projects
Error: Querying all projects failed. Please try again.

with verbose logging i get this traceback additionally

Traceback (most recent call last):
  File "/home/emil/.local/lib/python3.10/site-packages/olsync/olsync.py", line 348, in execute_action
    success = action()
  File "/home/emil/.local/lib/python3.10/site-packages/olsync/olsync.py", line 175, in <lambda>
    lambda: overleaf_client.get_project(project_name),
  File "/home/emil/.local/lib/python3.10/site-packages/olsync/olclient.py", line 104, in get_project
    BeautifulSoup(projects_page.content, 'html.parser').find('meta', {'name': 'ol-projects'}).get('content'))
AttributeError: 'NoneType' object has no attribute 'get'

i reinstalled all python dependencies and qt6-webengine and the problem still persists
I seem to solve this it's because the change of overleaf I think.
There's no ol-projects anymore, but ol-prefetchedProjectsBlob
copy these python code and cover the original olclient.py
seems to have some format problem, sorry I dont know how to write md document

'''

import requests as reqs
from bs4 import BeautifulSoup
import json
import uuid
from socketIO_client import SocketIO
import time

LOGIN_URL = "https://www.overleaf.com/login"
PROJECT_URL = "https://www.overleaf.com/project" # The dashboard URL

DOWNLOAD_URL = "https://www.overleaf.com/project/{}/download/zip"
UPLOAD_URL = "https://www.overleaf.com/project/{}/upload" # The URL to upload files
FOLDER_URL = "https://www.overleaf.com/project/{}/folder" # The URL to create folders
DELETE_URL = "https://www.overleaf.com/project/{}/doc/{}" # The URL to delete files
COMPILE_URL = "https://www.overleaf.com/project/{}/compile?enable_pdf_caching=true" # The URL to compile the project
BASE_URL = "https://www.overleaf.com" # The Overleaf Base URL
PATH_SEP = "/" # Use hardcoded path separator for both windows and posix system
class OverleafClient(object):
@staticmethod
def filter_projects(json_content, more_attrs=None):
more_attrs = more_attrs or {}
for p in json_content:
if not p.get("archived") and not p.get("trashed"):
if all(p.get(k) == v for k, v in more_attrs.items()):
yield p

def __init__(self, cookie=None, csrf=None):
    self._cookie = cookie  # Store the cookie for authenticated requests
    self._csrf = csrf  # Store the CSRF token since it is needed for some requests

def login(self, username, password):

    get_login = reqs.get(LOGIN_URL)
    self._csrf = BeautifulSoup(get_login.content, 'html.parser').find(
        'input', {'name': '_csrf'}).get('value')
    login_json = {
        "_csrf": self._csrf,
        "email": username,
        "password": password
    }
    post_login = reqs.post(LOGIN_URL, json=login_json,
                           cookies=get_login.cookies)

    # On a successful authentication the Overleaf API returns a new authenticated cookie.
    # If the cookie is different than the cookie of the GET request the authentication was successful
    if post_login.status_code == 200 and get_login.cookies["overleaf_session2"] != post_login.cookies[
        "overleaf_session2"]:
        self._cookie = post_login.cookies

        # Enrich cookie with GCLB cookie from GET request above
        self._cookie['GCLB'] = get_login.cookies['GCLB']

        # CSRF changes after making the login request, new CSRF token will be on the projects page
        projects_page = reqs.get(PROJECT_URL, cookies=self._cookie)
        self._csrf = BeautifulSoup(projects_page.content, 'html.parser').find('meta', {'name': 'ol-csrfToken'}) \
            .get('content')

        return {"cookie": self._cookie, "csrf": self._csrf}

def all_projects(self):
    projects_page = reqs.get(PROJECT_URL, cookies=self._cookie)
    # print(BeautifulSoup(projects_page.content, 'html.parser'))
    # json_content = json.loads(
    #     BeautifulSoup(projects_page.content, 'html.parser').find('meta', {'name': 'ol-projects'}).get('content'))
    testname='ol-prefetchedProjectsBlob'
    # print(BeautifulSoup(projects_page.content, 'html.parser').find('meta', {'name': testname}).get('content'))
    # print(type(BeautifulSoup(projects_page.content, 'html.parser').find('meta', {'name': testname}).get('content')))
    json_content = json.loads(
        (BeautifulSoup(projects_page.content, 'html.parser').find('meta', {'name': testname}).get('content')))
    json_content=json_content.get('projects') 
    # print(((json_content.get('projects')[0])))
    # print(list(OverleafClient.filter_projects(json_content)))
    return list(OverleafClient.filter_projects(json_content))
    # test2=OverleafClient.filter_projects((json_content.get('projects')[0]))
    # return (list(test2))

def get_project(self, project_name):

    projects_page = reqs.get(PROJECT_URL, cookies=self._cookie)
    json_content = json.loads(
        BeautifulSoup(projects_page.content, 'html.parser').find('meta', {'name': 'ol-prefetchedProjectsBlob'}).get('content'))
    return next(OverleafClient.filter_projects(json_content, {"name": project_name}), None)

def download_project(self, project_id):

    r = reqs.get(DOWNLOAD_URL.format(project_id),
                 stream=True, cookies=self._cookie)
    return r.content

def create_folder(self, project_id, parent_folder_id, folder_name):


    params = {
        "parent_folder_id": parent_folder_id,
        "name": folder_name
    }
    headers = {
        "X-Csrf-Token": self._csrf
    }
    r = reqs.post(FOLDER_URL.format(project_id),
                  cookies=self._cookie, headers=headers, json=params)

    if r.ok:
        return json.loads(r.content)
    elif r.status_code == str(400):
        # Folder already exists
        return
    else:
        raise reqs.HTTPError()

def get_project_infos(self, project_id):

    project_infos = None

    # Callback function for the joinProject emitter
    def set_project_infos(a, project_infos_dict, c, d):
        # Set project_infos variable in outer scope
        nonlocal project_infos
        project_infos = project_infos_dict

    # Convert cookie from CookieJar to string
    cookie = "GCLB={}; overleaf_session2={}" \
        .format(
        self._cookie["GCLB"],
        self._cookie["overleaf_session2"]
    )

    # Connect to Overleaf Socket.IO, send a time parameter and the cookies
    socket_io = SocketIO(
        BASE_URL,
        params={'t': int(time.time())},
        headers={'Cookie': cookie}
    )

    # Wait until we connect to the socket
    socket_io.on('connect', lambda: None)
    socket_io.wait_for_callbacks()

    # Send the joinProject event and receive the project infos
    socket_io.emit('joinProject', {'project_id': project_id}, set_project_infos)
    socket_io.wait_for_callbacks()

    # Disconnect from the socket if still connected
    if socket_io.connected:
        socket_io.disconnect()

    return project_infos

def upload_file(self, project_id, project_infos, file_name, file_size, file):
    

    # Set the folder_id to the id of the root folder
    folder_id = project_infos['rootFolder'][0]['_id']

    # The file name contains path separators, check folders
    if PATH_SEP in file_name:
        local_folders = file_name.split(PATH_SEP)[:-1]  # Remove last item since this is the file name
        current_overleaf_folder = project_infos['rootFolder'][0]['folders']  # Set the current remote folder

        for local_folder in local_folders:
            exists_on_remote = False
            for remote_folder in current_overleaf_folder:
                # Check if the folder exists on remote, continue with the new folder structure
                if local_folder.lower() == remote_folder['name'].lower():
                    exists_on_remote = True
                    folder_id = remote_folder['_id']
                    current_overleaf_folder = remote_folder['folders']
                    break
            # Create the folder if it doesn't exist
            if not exists_on_remote:
                new_folder = self.create_folder(project_id, folder_id, local_folder)
                current_overleaf_folder.append(new_folder)
                folder_id = new_folder['_id']
                current_overleaf_folder = new_folder['folders']
    params = {
        "folder_id": folder_id,
        "_csrf": self._csrf,
        "qquuid": str(uuid.uuid4()),
        "qqfilename": file_name,
        "qqtotalfilesize": file_size,
    }
    files = {
        "qqfile": file
    }

    # Upload the file to the predefined folder
    r = reqs.post(UPLOAD_URL.format(project_id), cookies=self._cookie, params=params, files=files)

    return r.status_code == str(200) and json.loads(r.content)["success"]

def delete_file(self, project_id, project_infos, file_name):


    file = None

    # The file name contains path separators, check folders
    if PATH_SEP in file_name:
        local_folders = file_name.split(PATH_SEP)[:-1]  # Remove last item since this is the file name
        current_overleaf_folder = project_infos['rootFolder'][0]['folders']  # Set the current remote folder

        for local_folder in local_folders:
            for remote_folder in current_overleaf_folder:
                if local_folder.lower() == remote_folder['name'].lower():
                    file = next((v for v in remote_folder['docs'] if v['name'] == file_name.split(PATH_SEP)[-1]),
                                None)
                    current_overleaf_folder = remote_folder['folders']
                    break
    # File is in root folder
    else:
        file = next((v for v in project_infos['rootFolder'][0]['docs'] if v['name'] == file_name), None)

    # File not found!
    if file is None:
        return False

    headers = {
        "X-Csrf-Token": self._csrf
    }

    r = reqs.delete(DELETE_URL.format(project_id, file['_id']), cookies=self._cookie, headers=headers, json={})

    return r.status_code == str(204)

def download_pdf(self, project_id):

    headers = {
        "X-Csrf-Token": self._csrf
    }

    body = {
        "check": "silent",
        "draft": False,
        "incrementalCompilesEnabled": True,
        "rootDoc_id": "",
        "stopOnFirstError": False
    }

    r = reqs.post(COMPILE_URL.format(project_id), cookies=self._cookie, headers=headers, json=body)

    if not r.ok:
        raise reqs.HTTPError()

    compile_result = json.loads(r.content)

    if compile_result["status"] != "success":
        raise reqs.HTTPError()

    pdf_file = next(v for v in compile_result['outputFiles'] if v['type'] == 'pdf')

    download_req = reqs.get(BASE_URL + pdf_file['url'], cookies=self._cookie, headers=headers)

    if download_req.ok:
        return pdf_file['path'], download_req.content

    return None

'''

[zangzelin]

ols used to work for me before, but recently I am facing the same issue.

me too

me too

[DDDOH]

Change olclient.py line 92 to
BeautifulSoup(projects_page.content, 'html.parser').find('meta', {'name': 'ol-prefetchedProjectsBlob'}).get('content'))['projects'] will fix this issue. This is due to the overleaf webpage is changed, and the original code cannot find projects anymore. Maybe I will make a pull request later.

[jurray-jiajun]

Change olclient.py line 92 to BeautifulSoup(projects_page.content, 'html.parser').find('meta', {'name': 'ol-prefetchedProjectsBlob'}).get('content'))['projects'] will fix this issue. This is due to the overleaf webpage is changed, and the original code cannot find projects anymore. Maybe I will make a pull request later.

It works. That's cool!

[tianyilt]

Change olclient.py line 92 to BeautifulSoup(projects_page.content, 'html.parser').find('meta', {'name': 'ol-prefetchedProjectsBlob'}).get('content'))['projects'] will fix this issue. This is due to the overleaf webpage is changed, and the original code cannot find projects anymore. Maybe I will make a pull request later.

Thank you for your response. The command ols list executed successfully, however, the command 'ols download -n "test"' did not. It returned the following error message:

$ ols download -n test
💥  Querying project
💥  Downloading project's PDF
Error: Downloading project's PDF failed. Please try again.

I would appreciate it if you could look into this issue further. Thanks in advance.

[tianyilt]

Change olclient.py line 92 to BeautifulSoup(projects_page.content, 'html.parser').find('meta', {'name': 'ol-prefetchedProjectsBlob'}).get('content'))['projects'] will fix this issue. This is due to the overleaf webpage is changed, and the original code cannot find projects anymore. Maybe I will make a pull request later.

Thank you for your response. The command ols list executed successfully, however, the command 'ols download -n "test"' did not. It returned the following error message:

$ ols download -n test
💥  Querying project
💥  Downloading project's PDF
Error: Downloading project's PDF failed. Please try again.

I would appreciate it if you could look into this issue further. Thanks in advance.

Change olclient.py line 104 to
BeautifulSoup(projects_page.content, 'html.parser').find('meta', {'name': 'ol-prefetchedProjectsBlob'}).get('content'))['projects'] will fix this issue.

[DDDOH]

Change olclient.py line 92 to BeautifulSoup(projects_page.content, 'html.parser').find('meta', {'name': 'ol-prefetchedProjectsBlob'}).get('content'))['projects'] will fix this issue. This is due to the overleaf webpage is changed, and the original code cannot find projects anymore. Maybe I will make a pull request later.

Thank you for your response. The command ols list executed successfully, however, the command 'ols download -n "test"' did not. It returned the following error message:

$ ols download -n test
💥  Querying project
💥  Downloading project's PDF
Error: Downloading project's PDF failed. Please try again.

I would appreciate it if you could look into this issue further. Thanks in advance.

Change olclient.py line 104 to BeautifulSoup(projects_page.content, 'html.parser').find('meta', {'name': 'ol-prefetchedProjectsBlob'}).get('content'))['projects'] will fix this issue.

Sorry for the delayed reply. It is good to know this issue has been fixed!