Doesn't work on M1 Macbook

[bhirsch42]

When running with calibre-customize -b, loading the plugin on calibre startup fails at import remarkable_fs because it can't find the package.

When running with the generated zip file, it fails with:

ImportError: cannot import name 'asn1' from 'cryptography.hazmat.bindings._rust' (unknown location)

...like in #2.

When running with the pycrypt version from #2, it gets to here:

➜  Calibre-Remarkable-Device-Driver-Plugin git:(main) ✗ calibre-debug -g
calibre 5.29  embedded-python: True is64bit: True
macOS-10.16-x86_64-i386-64bit Darwin ('64bit', '')
('Darwin', '20.5.0', 'Darwin Kernel Version 20.5.0: Sat May  8 05:10:31 PDT 2021; root:xnu-7195.121.3~9/RELEASE_ARM64_T8101')
Python 3.8.5
OSX: ('10.16', ('', '', ''), 'x86_64')
Interface language: None
Successfully initialized third party plugins: Remarkable Plugin (1, 2, 1)
devicePixelRatio: 2.0
logicalDpi: 72.0 x 72.0
physicalDpi: 149.00000223775552 x 149.0000022377555
Using calibre Qt style: True
[0.00] Starting up...
[0.02] Showing splash screen...
[0.22] splash screen shown
[0.22] Initializing db...
[0.24] db initialized
[0.24] Constructing main UI...
> /Users/benhirsch/repos/Calibre-Remarkable-Device-Driver-Plugin/calibre_plugins.remarkable_plugin.__init__(34)startup()
-> import remarkable_fs
(Pdb)

...and then hangs forever.

[bhirsch42]

I suspect it has something to do with the fact that Calibre is on Python 3.8.5, which is incompatible with M1 Macbooks.

[bhirsch42]

Hitting Command-D during the (Pdb) hang gives this:

(Pdb) ^D
Startup method for device RemarkablePlugin threw exception
Traceback (most recent call last):
  File "calibre/gui2/device.py", line 378, in run_startup
  File "calibre_plugins.remarkable_plugin.__init__", line 34, in startup
    import remarkable_fs
  File "calibre_plugins.remarkable_plugin.__init__", line 34, in startup
    import remarkable_fs
  File "bdb.py", line 88, in trace_dispatch
  File "bdb.py", line 113, in dispatch_line
bdb.BdbQuit
Error during device detection for <calibre_plugins.remarkable_plugin.RemarkablePlugin object at 0x10f8c9820>:
Traceback (most recent call last):
  File "calibre/gui2/device.py", line 293, in detect_device
  File "calibre_plugins.remarkable_plugin.__init__", line 51, in detect_managed_devices
    if self.seen_device:
AttributeError: 'RemarkablePlugin' object has no attribute 'seen_device'

[bhirsch42]

Ah, I was on the wrong network. If ssh is configured properly, it doesn't hang! It says:

Successfully initialized third party plugins: Remarkable Plugin (1, 2, 1)

which is promising. I do not, however, see my device in calibre.

[bhirsch42]

I removed some trailing slashes from the config and it hangs again.

[bhirsch42]

Using remarkable-fs doesn't work properly, so I'm going to assume that's the root cause of my issue, not this plugin.

[naclander]

Hey @bhirsch42 , I probably left a test breakpoint in the code somewhere when making that test plugin. Sorry about that, glad you were able to Ctr-D out of it.

The trailing slashes shouldn't matter in the config. After the plugin loads, can you show me what the terminal prompt is after the plugin loads? I don't know why remarkable-fs isn't working for you and would be interested in knowing more.

Also, can you clarify again how you moved on from this error:
ImportError: cannot import name 'asn1' from 'cryptography.hazmat.bindings._rust' (unknown location)
That's where @barnumbirr is stuck and i would like to get him further, to where you got the plugin loaded apparently.

[RMHogervorst]

I'm not on an macbook M1 (old version of macbook) and i get the same error. using the latest version.
It smells like an import error, but i'm not sure.

(i've included the relevant part of the logs)

[49.32] Constructing main UI...
Startup method for device RemarkablePlugin threw exception
Error during device detection for <calibre_plugins.remarkable_plugin.RemarkablePlugin object at 0x110b9ba90>:
[50.76] main UI initialized...
[50.76] Hiding splash screen
Starting QuickView
Traceback (most recent call last):
  File "calibre/gui2/device.py", line 378, in run_startup
  File "calibre_plugins.remarkable_plugin.__init__", line 33, in startup
    import remarkable_fs
  File "zipimport.py", line 259, in load_module
  File "/Users/roelhogervorst/Library/Preferences/calibre/plugins/Remarkable Plugin.zip/remarkable_fs/__init__.py", line 1, in <module>
    from remarkable_fs.connection import connect
  File "zipimport.py", line 259, in load_module
  File "/Users/roelhogervorst/Library/Preferences/calibre/plugins/Remarkable Plugin.zip/remarkable_fs/connection.py", line 5, in <module>
    from paramiko.client import SSHClient, AutoAddPolicy
  File "zipimport.py", line 259, in load_module
  File "/Users/roelhogervorst/Library/Preferences/calibre/plugins/Remarkable Plugin.zip/paramiko/__init__.py", line 22, in <module>
    from paramiko.transport import SecurityOptions, Transport
  File "zipimport.py", line 259, in load_module
  File "/Users/roelhogervorst/Library/Preferences/calibre/plugins/Remarkable Plugin.zip/paramiko/transport.py", line 89, in <module>
    from paramiko.dsskey import DSSKey
  File "zipimport.py", line 259, in load_module
  File "/Users/roelhogervorst/Library/Preferences/calibre/plugins/Remarkable Plugin.zip/paramiko/dsskey.py", line 25, in <module>
    from cryptography.hazmat.primitives import hashes, serialization
  File "zipimport.py", line 259, in load_module
  File "/Users/roelhogervorst/Library/Preferences/calibre/plugins/Remarkable Plugin.zip/cryptography/hazmat/primitives/serialization/__init__.py", line 15, in <module>
    from cryptography.hazmat.primitives.serialization.base import (
  File "zipimport.py", line 259, in load_module
  File "/Users/roelhogervorst/Library/Preferences/calibre/plugins/Remarkable Plugin.zip/cryptography/hazmat/primitives/serialization/base.py", line 11, in <module>
    from cryptography.hazmat.primitives.asymmetric.types import (
  File "zipimport.py", line 259, in load_module
  File "/Users/roelhogervorst/Library/Preferences/calibre/plugins/Remarkable Plugin.zip/cryptography/hazmat/primitives/asymmetric/types.py", line 7, in <module>
    from cryptography.hazmat.primitives.asymmetric import (
  File "zipimport.py", line 259, in load_module
  File "/Users/roelhogervorst/Library/Preferences/calibre/plugins/Remarkable Plugin.zip/cryptography/hazmat/primitives/asymmetric/dsa.py", line 12, in <module>
    from cryptography.hazmat.primitives.asymmetric import (
  File "zipimport.py", line 259, in load_module
  File "/Users/roelhogervorst/Library/Preferences/calibre/plugins/Remarkable Plugin.zip/cryptography/hazmat/primitives/asymmetric/utils.py", line 6, in <module>
    from cryptography.hazmat.bindings._rust import asn1
ImportError: cannot import name 'asn1' from 'cryptography.hazmat.bindings._rust' (unknown location)

[naclander]

@RMHogervorst What version are you running. Please try the test version from the other thread.

[RMHogervorst]

I used the then latest version from github releases (1.2.3).
with 'the other thread' do you mean this: #2 (comment) (version 1.2.1)

Log from debug session (with version 1.2.1, referenced in issue 2):

calibre Debug log
calibre 5.29  embedded-python: True is64bit: True
macOS-10.16-x86_64-i386-64bit Darwin ('64bit', '')
('Darwin', '20.6.0', 'Darwin Kernel Version 20.6.0: Mon Aug 30 06:12:21 PDT 2021; root:xnu-7195.141.6~3/RELEASE_X86_64')
Python 3.8.5
OSX: ('10.16', ('', '', ''), 'x86_64')
Interface language: None
Successfully initialized third party plugins: Remarkable Plugin (1, 2, 1)
calibre 5.29  embedded-python: True is64bit: True
macOS-10.16-x86_64-i386-64bit Darwin ('64bit', '')
('Darwin', '20.6.0', 'Darwin Kernel Version 20.6.0: Mon Aug 30 06:12:21 PDT 2021; root:xnu-7195.141.6~3/RELEASE_X86_64')
Python 3.8.5
OSX: ('10.16', ('', '', ''), 'x86_64')
Interface language: None
Successfully initialized third party plugins: Remarkable Plugin (1, 2, 1)
devicePixelRatio: 2.0
logicalDpi: 72.0 x 72.0
physicalDpi: 113.50000170459899 x 113.50000170459901
Using calibre Qt style: True
[0.00] Starting up...
[0.02] Showing splash screen...
[0.28] splash screen shown
[0.30] Initializing db...
[78.22] db initialized
[78.22] Constructing main UI...
> /calibre_plugins.remarkable_plugin.__init__(34)startup()
-> import remarkable_fs
(Pdb) 
Startup method for device RemarkablePlugin threw exception
Error during device detection for <calibre_plugins.remarkable_plugin.RemarkablePlugin object at 0x1113f1a90>:
[79.57] main UI initialized...
[79.57] Hiding splash screen
Starting QuickView

[naclander]

Yes, that other thread. If you C-d out of that PDB session does the plugin start up fine?

[RMHogervorst]

I've been using calibre on a Mac and I can only start the debug session from calibre itself.

[RMHogervorst]

and so it gives me the debuglog afterwards, Maybe I can start calibre from the commandline? I'll try.
I see some more logs

23.34] main UI initialized...
[23.34] Hiding splash screen
Starting QuickView
Traceback (most recent call last):
  File "calibre/gui2/device.py", line 378, in run_startup
  File "calibre_plugins.remarkable_plugin.__init__", line 34, in startup
    import remarkable_fs
  File "calibre_plugins.remarkable_plugin.__init__", line 34, in startup
    import remarkable_fs
  File "bdb.py", line 88, in trace_dispatch
  File "bdb.py", line 113, in dispatch_line
bdb.BdbQuit
Traceback (most recent call last):
  File "calibre/gui2/device.py", line 293, in detect_device
  File "calibre_plugins.remarkable_plugin.__init__", line 51, in detect_managed_devices
    if self.seen_device:
AttributeError: 'RemarkablePlugin' object has no attribute 'seen_device'
Traceback (most recent call last):
  File "calibre/gui2/device.py", line 293, in detect_device
  File "calibre_plugins.remarkable_plugin.__init__", line 51, in detect_managed_devices
    if self.seen_device:
AttributeError: 'RemarkablePlugin' object has no attribute 'seen_device'

[RMHogervorst]

I started it from the cmdline and tried to interupt during PDB session using ctrl c and later ctrl d (I assume that is what you meant (with C-d )? But that doesn't seem to do anything.

/Applications/calibre.app/Contents/MacOS/calibre-debug --gui
calibre 5.29  embedded-python: True is64bit: True
macOS-10.16-x86_64-i386-64bit Darwin ('64bit', '')
('Darwin', '20.6.0', 'Darwin Kernel Version 20.6.0: Mon Aug 30 06:12:21 PDT 2021; root:xnu-7195.141.6~3/RELEASE_X86_64')
Python 3.8.5
OSX: ('10.16', ('', '', ''), 'x86_64')
Interface language: None
Successfully initialized third party plugins: Remarkable Plugin (1, 2, 1)
devicePixelRatio: 2.0
logicalDpi: 72.0 x 72.0
physicalDpi: 113.50000170459899 x 113.50000170459901
Using calibre Qt style: True
[0.00] Starting up...
[0.02] Showing splash screen...
[0.27] splash screen shown
[0.28] Initializing db...
[22.95] db initialized
[22.95] Constructing main UI...
> /Users/roelhogervorst/calibre_plugins.remarkable_plugin.__init__(34)startup()
-> import remarkable_fs
(Pdb) ^C^D
Startup method for device RemarkablePlugin threw exception
Traceback (most recent call last):
  File "calibre/gui2/device.py", line 378, in run_startup
  File "calibre_plugins.remarkable_plugin.__init__", line 34, in startup
    import remarkable_fs
  File "calibre_plugins.remarkable_plugin.__init__", line 34, in startup
    import remarkable_fs
  File "bdb.py", line 88, in trace_dispatch
  File "bdb.py", line 113, in dispatch_line
bdb.BdbQuit
Error during device detection for <calibre_plugins.remarkable_plugin.RemarkablePlugin object at 0x110544760>:
Traceback (most recent call last):
  File "calibre/gui2/device.py", line 293, in detect_device
  File "calibre_plugins.remarkable_plugin.__init__", line 51, in detect_managed_devices
    if self.seen_device:
AttributeError: 'RemarkablePlugin' object has no attribute 'seen_device'
[104.67] main UI initialized...
[104.67] Hiding splash screen
Starting QuickView
[107.55] splash screen hidden
[107.55] Started up in 107.55 seconds with 4463 books