reMarkable 2 not detected at Calibre startup

[robfreundlich]

My reMarkable 2 is connected via USB, and I can successfully ssh to it at IP address 10.11.99.1. I have the plugin configured as follows:

(the password is the correct one)

And yet, when I start Calibre, it doesn't see the device. When I run calibre-debug -g, I see this in the output, but no more details:

Error during device detection for <calibre_plugins.remarkable_plugin.RemarkablePlugin object at 0x000001FF7A2DFF70>:
Traceback (most recent call last):
  File "calibre\gui2\device.py", line 291, in detect_device
  File "calibre_plugins.remarkable_plugin.__init__", line 50, in detect_managed_devices
AttributeError: 'RemarkablePlugin' object has no attribute 'seen_device'

[naclander]

Hey @robfreundlich , thanks for opening the issue. I need to dedicate some time to figure out why this is happening. Are you using the code directly from master? or one of the packaged releases?

[robfreundlich]

Howdy! That's an excellent question! And, unfortunately, one that I can't answer for certain, because it was more than a week ago :-) But I'm gonna say that I grabbed it from master, because I don't remember and the only references to it that I can find in my browser history are the github page. Rob

…

On Fri, Apr 1, 2022 at 9:26 PM Nathan Aclander ***@***.***> wrote: Hey @robfreundlich <https://github.com/RobFreundlich> , thanks for opening the issue. I need to dedicate some time to figure out why this is happening. Are you using the code directly from master? or one of the packaged releases? — Reply to this email directly, view it on GitHub <#14 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB43AZI5COQ2LEZYBHWWRQLVC6O4NANCNFSM5P3RDR3Q> . You are receiving this because you were mentioned.Message ID: <naclander/Calibre-Remarkable-Device-Driver-Plugin/issues/14/1086457829@ github.com>

[posita]

I am using the code from master and am observing the same:

/Applications/calibre.app/Contents/MacOS/calibre-debug -g
calibre 5.40  embedded-python: True is64bit: True
macOS-10.16-x86_64-i386-64bit Darwin ('64bit', '')
('Darwin', '20.6.0', 'Darwin Kernel Version 20.6.0: Tue Feb 22 21:10:41 PST 2022; root:xnu-7195.141.26~1/RELEASE_X86_64')
Python 3.8.5
OSX: ('10.16', ('', '', ''), 'x86_64')
Interface language: None
Successfully initialized third party plugins: DeDRM (7, 2, 1) && Remarkable Plugin (1, 2, 3)
devicePixelRatio: 2.0
logicalDpi: 72.0 x 72.0
physicalDpi: 93.50000140422914 x 93.50000140422914
Using calibre Qt style: True
[0.00] Starting up...
[0.13] Showing splash screen...
[0.70] splash screen shown
[0.70] Initializing db...
[0.77] db initialized
[0.77] Constructing main UI...
Startup method for device RemarkablePlugin threw exception
Traceback (most recent call last):
  File "calibre/gui2/device.py", line 375, in run_startup
  File "calibre_plugins.remarkable_plugin.__init__", line 33, in startup
    import remarkable_fs
ModuleNotFoundError: No module named 'remarkable_fs'
Error during device detection for <calibre_plugins.remarkable_plugin.RemarkablePlugin object at 0x107b883d0>:
Traceback (most recent call last):
  File "calibre/gui2/device.py", line 290, in detect_device
  File "calibre_plugins.remarkable_plugin.__init__", line 50, in detect_managed_devices
    if self.seen_device:
AttributeError: 'RemarkablePlugin' object has no attribute 'seen_device'
[3.34] main UI initialized...
[3.34] Hiding splash screen
Starting QuickView
[4.67] splash screen hidden
[4.68] Started up in 4.68 seconds with 253 books
Error during device detection for <calibre_plugins.remarkable_plugin.RemarkablePlugin object at 0x107b883d0>:
Traceback (most recent call last):
  File "calibre/gui2/device.py", line 290, in detect_device
  File "calibre_plugins.remarkable_plugin.__init__", line 50, in detect_managed_devices
    if self.seen_device:
AttributeError: 'RemarkablePlugin' object has no attribute 'seen_device'`

[DakotaNelson]

Same error here, in Windows using the latest release zip file:

From __init__.py:

class RemarkablePlugin(DevicePlugin):
    name = "Remarkable Plugin"
    description = "Send files to Remarkable"
    author = "Nathan Aclander"
    supported_platforms = ["linux", "windows", "osx"]
    version = (1, 2, 3)  # The version number of this plugin
    minimum_calibre_version = (0, 7, 53)

Console logs:

PS C:\Users\dnelson> calibre-debug.exe -g
calibre 5.36  embedded-python: True is64bit: False
Windows-10-10.0.19041 Windows ('32bit', 'WindowsPE')
32bit process running on 64bit windows
('Windows', '10', '10.0.19041')
Python 3.8.5
Windows: ('10', '10.0.19041', '', 'Multiprocessor Free')
Interface language: None
Successfully initialized third party plugins: Remarkable Plugin (1, 2, 3)
Turning on automatic hidpi scaling
devicePixelRatio: 1.0
logicalDpi: 96.0 x 96.0
physicalDpi: 81.55183946488295 x 81.64285714285714
Using calibre Qt style: True
[0.00] Starting up...
[0.00] Showing splash screen...
[0.36] splash screen shown
[0.36] Initializing db...
[0.38] db initialized
[0.38] Constructing main UI...
Startup method for device RemarkablePlugin threw exception
Traceback (most recent call last):
  File "calibre\gui2\device.py", line 375, in run_startup
  File "calibre_plugins.remarkable_plugin.__init__", line 33, in startup
  File "zipimport.py", line 259, in load_module
  File "C:\Users\dnelson\AppData\Roaming\calibre\plugins\Remarkable Plugin.zip\remarkable_fs\__init__.py", line 1, in <module>
  File "zipimport.py", line 259, in load_module
  File "C:\Users\dnelson\AppData\Roaming\calibre\plugins\Remarkable Plugin.zip\remarkable_fs\connection.py", line 5, in <module>
  File "zipimport.py", line 259, in load_module
  File "C:\Users\dnelson\AppData\Roaming\calibre\plugins\Remarkable Plugin.zip\paramiko\__init__.py", line 22, in <module>
  File "zipimport.py", line 259, in load_module
  File "C:\Users\dnelson\AppData\Roaming\calibre\plugins\Remarkable Plugin.zip\paramiko\transport.py", line 89, in <module>
  File "zipimport.py", line 259, in load_module
  File "C:\Users\dnelson\AppData\Roaming\calibre\plugins\Remarkable Plugin.zip\paramiko\dsskey.py", line 25, in <module>
  File "zipimport.py", line 259, in load_module
  File "C:\Users\dnelson\AppData\Roaming\calibre\plugins\Remarkable Plugin.zip\cryptography\hazmat\primitives\serialization\__init__.py", line 15, in <module>
  File "zipimport.py", line 259, in load_module
  File "C:\Users\dnelson\AppData\Roaming\calibre\plugins\Remarkable Plugin.zip\cryptography\hazmat\primitives\serialization\base.py", line 11, in <module>
  File "zipimport.py", line 259, in load_module
  File "C:\Users\dnelson\AppData\Roaming\calibre\plugins\Remarkable Plugin.zip\cryptography\hazmat\primitives\asymmetric\types.py", line 7, in <module>
  File "zipimport.py", line 259, in load_module
  File "C:\Users\dnelson\AppData\Roaming\calibre\plugins\Remarkable Plugin.zip\cryptography\hazmat\primitives\asymmetric\dsa.py", line 12, in <module>
  File "zipimport.py", line 259, in load_module
  File "C:\Users\dnelson\AppData\Roaming\calibre\plugins\Remarkable Plugin.zip\cryptography\hazmat\primitives\asymmetric\utils.py", line 6, in <module>
ImportError: cannot import name 'asn1' from 'cryptography.hazmat.bindings._rust' (unknown location)
Error during device detection for <calibre_plugins.remarkable_plugin.RemarkablePlugin object at 0x07A0AAF0>:
Traceback (most recent call last):
  File "calibre\gui2\device.py", line 290, in detect_device
  File "calibre_plugins.remarkable_plugin.__init__", line 50, in detect_managed_devices
AttributeError: 'RemarkablePlugin' object has no attribute 'seen_device'
[1.22] main UI initialized...
[1.22] Hiding splash screen
Starting QuickView
[1.52] splash screen hidden
[1.52] Started up in 1.52 seconds with 316 books
Error during device detection for <calibre_plugins.remarkable_plugin.RemarkablePlugin object at 0x07A0AAF0>:
Traceback (most recent call last):
  File "calibre\gui2\device.py", line 290, in detect_device
  File "calibre_plugins.remarkable_plugin.__init__", line 50, in detect_managed_devices
AttributeError: 'RemarkablePlugin' object has no attribute 'seen_device'
Error during device detection for <calibre_plugins.remarkable_plugin.RemarkablePlugin object at 0x07A0AAF0>:
Traceback (most recent call last):
  File "calibre\gui2\device.py", line 290, in detect_device
  File "calibre_plugins.remarkable_plugin.__init__", line 50, in detect_managed_devices
AttributeError: 'RemarkablePlugin' object has no attribute 'seen_device'
Error during device detection for <calibre_plugins.remarkable_plugin.RemarkablePlugin object at 0x07A0AAF0>:
Traceback (most recent call last):
  File "calibre\gui2\device.py", line 290, in detect_device
  File "calibre_plugins.remarkable_plugin.__init__", line 50, in detect_managed_devices
AttributeError: 'RemarkablePlugin' object has no attribute 'seen_device'

I think this is the actual root of the error:
ImportError: cannot import name 'asn1' from 'cryptography.hazmat.bindings._rust' (unknown location)

I suspect the issue is that this import requires native code to be compiled on the target, which Calibre may not handle (I'm actually not sure): pyca/cryptography#4927 (comment)

[DakotaNelson]

Ok, I think the import is the root issue.

Using this patch (just adding try/excepts):

diff --git a/__init__.py b/__init__.py
index 1aa101a..ad84305 100644
--- a/__init__.py
+++ b/__init__.py
@@ -30,8 +30,16 @@ class RemarkablePlugin(DevicePlugin):
         sys.path.append(self.plugin_path)
         global remarkable_fs
         global paramiko
-        import remarkable_fs
-        import paramiko
+
+        try:
+            import remarkable_fs
+        except ImportError as e:
+            print("Failed to import module: ", e)
+
+        try:
+            import paramiko
+        except ImportError as e:
+            print("Failed to import module: ", e)

         # Currently we only support 1 device. Use this variable to remember if we've already seen it or not so as to
         # not keep detecting it. If for some reason we decide to support multiple devices, we should probably change this

Returns new and exciting issues:

PS C:\Users\dnelson> calibre-debug.exe -g
calibre 5.36  embedded-python: True is64bit: False
Windows-10-10.0.19041 Windows ('32bit', 'WindowsPE')
32bit process running on 64bit windows
('Windows', '10', '10.0.19041')
Python 3.8.5
Windows: ('10', '10.0.19041', '', 'Multiprocessor Free')
Interface language: None
Successfully initialized third party plugins: Remarkable Plugin (1, 2, 3)
Turning on automatic hidpi scaling
devicePixelRatio: 1.0
logicalDpi: 96.0 x 96.0
physicalDpi: 81.55183946488295 x 81.64285714285714
Using calibre Qt style: True
[0.00] Starting up...
[0.00] Showing splash screen...
[0.36] splash screen shown
[0.36] Initializing db...
[0.38] db initialized
[0.38] Constructing main UI...
Failed to import module:  cannot import name 'asn1' from 'cryptography.hazmat.bindings._rust' (unknown location)
Failed to import module:  cannot import name 'asn1' from 'cryptography.hazmat.bindings._rust' (unknown location)
Trying to connect to 192.168.1.8

Pinging 192.168.1.8 with 32 bytes of data:
[1.25] main UI initialized...
[1.25] Hiding splash screen
Starting QuickView
Reply from 192.168.1.8: bytes=32 time=329ms TTL=64
[1.53] splash screen hidden
[1.53] Started up in 1.53 seconds with 316 books
Reply from 192.168.1.8: bytes=32 time=10ms TTL=64
Reply from 192.168.1.8: bytes=32 time=8ms TTL=64
Reply from 192.168.1.8: bytes=32 time=8ms TTL=64

Ping statistics for 192.168.1.8:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 8ms, Maximum = 329ms, Average = 88ms
Devcie 192.168.1.8 Present
Opening device
Error while trying to open True (Driver: <calibre_plugins.remarkable_plugin.RemarkablePlugin object at 0x0725AB08>)
Traceback (most recent call last):
  File "calibre\gui2\device.py", line 297, in detect_device
  File "calibre_plugins.remarkable_plugin.__init__", line 82, in open
NameError: name 'paramiko' is not defined
Opening device
Error while trying to open True (Driver: <calibre_plugins.remarkable_plugin.RemarkablePlugin object at 0x0725AB08>)
Traceback (most recent call last):
  File "calibre\gui2\device.py", line 297, in detect_device
  File "calibre_plugins.remarkable_plugin.__init__", line 82, in open
NameError: name 'paramiko' is not defined

So the issue is in paramiko, specifically its dependence on asn1, which I think is because it relies on native code. Ignoring the problem importing lets you connect to the device successfully, buuuuut then it fails when it tries to actually go in and do stuff (using paramiko).

Not sure if there's a way to pre-install paramiko/asn1 in a way that calibre's embedded interpreter can pick it up, or... something.

[posita]

Any update on this? Anything those of us encountering this issue can do to help get it fixed? Alternatively, is there a prior working version? (I suspect no, given the nature of the issue.)

[mfouesneau]

Yes please

[xals]

The PR #16 should fix this issue.

[xals]

After further reading of this issue, my PR seems to be not related to the errors you reported, but fixing device space retrieving as I did allowed me to see the device and send documents on it.

[xals]

After further reading of this issue, my PR seems to be not related to the errors you reported, but fixing device space retrieving as I did allowed me to see the device and send documents on it.

I just realized that the behavior has been changed in master so my PR is not relevant unless you want to remove the device storage configuration parameter.

[xals]

Hello, I modified the PR to include the configuration parameter storage. And if you set it to /home/root it works fine, and should be compatible with reMarkable 1 and 2.

[Kenivia]

Hey guys, I've found a workaround for this.

In the __init__.py file, around line 32: sys.path.append(self.plugin_path)
What's being added to sys.path is the path to the zip file, as documented here, which can't be opened by python for importing.

Step by step workaround:

1. Download the repository
2. Open __init__.py, go to around line 32 where it says sys.path.append(self.plugin_path)
3. Replace self.plugin_path with the path to where you've put Calibre-Remarkable-Device-Driver-Plugin/ folder, PLUS target at the end(we will make the target folder later)
   e.g: sys.path.append("/home/kenivia/Desktop/Calibre-Remarkable-Device-Driver-Plugin/target/")
4. Make sure that your active system(or conda) Python version is 3.10(Otherwise the plugin will say that it failed to import _cffi)
5. Run create_plugin_zip.sh, this will create the target folder and a zip file
6. Keep the target/ folder somewhere safe, e.g. desktop, so that it is there whenever you run calibre. Make sure that the path in __init__.py still points to the target folder
7. Install the plugin through Calibre preference normally using the zip file generated previously. If you change __init__.py, make sure you generate the zip file again.
   Alternatively, run calibre-customize -b ./ from the folder, but you may have to first delete the zip file.

I should mention that this question & answer helped pinning down the problem

[atze007]

How exactly can I create the new ZIP file? Or more precisely, how does step 5 work?
When I compress it "normally" I get the following error
calibre-remarkable-device-driver-plugin.zip' is invalid. It does not contain a top-level __init__.py file

[Kenivia]

@atze007 The script create_plugin_zip.sh will 1: make a target folder and put the libraries in it and 2: put both the target folder, __init__.py, config.py and the other files into a zip file. It puts the zip file in the same folder as the script. The script will make the zip file for you, you shouldn't do it yourself.

[atze007]

@Kenivia
I'm sorry, but I'm at a loss at the moment....
Where can I get the scirpt? 🙈