pix-plot on MacOS 15.2 - Apple Silicon M3?

[mikima]

Hello. Tried to install pix-plot on a MacOS Apple Silicon M3, without success.

I followed this guide: https://github.com/pleonard212/pix-plot/wiki/MacOS-12.2.1---Apple-Silicon-(M1-chip)-with-GPU-acceleration

But when i run it i get this error:

Traceback (most recent call last):
  File "/opt/anaconda3/envs/pixplot/lib/python3.9/site-packages/pkg_resources/__init__.py", line 656, in _build_master
    ws.require(__requires__)
  File "/opt/anaconda3/envs/pixplot/lib/python3.9/site-packages/pkg_resources/__init__.py", line 1065, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/opt/anaconda3/envs/pixplot/lib/python3.9/site-packages/pkg_resources/__init__.py", line 892, in resolve
    dist = self._resolve_dist(
  File "/opt/anaconda3/envs/pixplot/lib/python3.9/site-packages/pkg_resources/__init__.py", line 938, in _resolve_dist
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (numpy 2.0.2 (/opt/anaconda3/envs/pixplot/lib/python3.9/site-packages), Requirement.parse('numpy==1.23.5'), {'pixplot'})

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/anaconda3/envs/pixplot/bin/pixplot", line 33, in <module>
    sys.exit(load_entry_point('pixplot==0.0.113', 'console_scripts', 'pixplot')())
  File "/opt/anaconda3/envs/pixplot/bin/pixplot", line 25, in importlib_load_entry_point
    return next(matches).load()
  File "/opt/anaconda3/envs/pixplot/lib/python3.9/importlib/metadata.py", line 86, in load
    module = import_module(match.group('module'))
  File "/opt/anaconda3/envs/pixplot/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/opt/anaconda3/envs/pixplot/lib/python3.9/site-packages/pixplot-0.0.113-py3.9.egg/pixplot/__init__.py", line 2, in <module>
    from pixplot.pixplot import process_images, parse
  File "/opt/anaconda3/envs/pixplot/lib/python3.9/site-packages/pixplot-0.0.113-py3.9.egg/pixplot/pixplot.py", line 10, in <module>
    import pkg_resources
  File "/opt/anaconda3/envs/pixplot/lib/python3.9/site-packages/pkg_resources/__init__.py", line 3668, in <module>
    def _initialize_master_working_set() -> None:
  File "/opt/anaconda3/envs/pixplot/lib/python3.9/site-packages/pkg_resources/__init__.py", line 3651, in _call_aside
    f(*args, **kwargs)
  File "/opt/anaconda3/envs/pixplot/lib/python3.9/site-packages/pkg_resources/__init__.py", line 3680, in _initialize_master_working_set
    working_set = _declare_state('object', 'working_set', WorkingSet._build_master())
  File "/opt/anaconda3/envs/pixplot/lib/python3.9/site-packages/pkg_resources/__init__.py", line 658, in _build_master
    return cls._build_from_requirements(__requires__)
  File "/opt/anaconda3/envs/pixplot/lib/python3.9/site-packages/pkg_resources/__init__.py", line 671, in _build_from_requirements
    dists = ws.resolve(reqs, Environment())
  File "/opt/anaconda3/envs/pixplot/lib/python3.9/site-packages/pkg_resources/__init__.py", line 892, in resolve
    dist = self._resolve_dist(
  File "/opt/anaconda3/envs/pixplot/lib/python3.9/site-packages/pkg_resources/__init__.py", line 933, in _resolve_dist
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'numpy==1.23.5' distribution was not found and is required by pixplot
pixplot --images="images/*.jpg"  0.11s user 0.03s system 93% cpu 0.143 total

[nabsiddiqui]

@mikima Try

pip install numpy==1.23.5

[mikima]

Thank you @nabsiddiqui.

It fixed the installation. however, when i test it with the oslomini dataset, it returns this:

2025-01-07 11:55:01.841221: HDBSCAN not available; using sklearn KMeans
2025-01-07 11:55:03.558203: CUML not available; using umap-learn UMAP
2025-01-07 11:55:03.558600: I metal_plugin/src/device/metal_device.cc:1154] Metal device set to: Apple M3 Pro
2025-01-07 11:55:03.558619: I metal_plugin/src/device/metal_device.cc:296] systemMemory: 18.00 GB
2025-01-07 11:55:03.558626: I metal_plugin/src/device/metal_device.cc:313] maxCacheSize: 6.00 GB
2025-01-07 11:55:03.558640: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:305] Could not identify NUMA node of platform GPU ID 0, defaulting to 0. Your kernel may not have been built with NUMA support.
2025-01-07 11:55:03.558648: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:271] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 0 MB memory) -> physical PluggableDevice (device: 0, name: METAL, pci bus id: <undefined>)
2025-01-07 11:55:03.559653: I tensorflow/core/common_runtime/direct_session.cc:380] Device mapping:
/job:localhost/replica:0/task:0/device:GPU:0 -> device: 0, name: METAL, pci bus id: <undefined>

2025-01-07 11:55:06.252989: Creating atlas files
2025-01-07 11:55:09.755973: Creating Inception vectors for 1000 images
2025-01-07 11:55:09.758779: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:305] Could not identify NUMA node of platform GPU ID 0, defaulting to 0. Your kernel may not have been built with NUMA support.
2025-01-07 11:55:09.758791: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:271] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 0 MB memory) -> physical PluggableDevice (device: 0, name: METAL, pci bus id: <undefined>)
2025-01-07 11:55:11.004657: Creating image array
  0%|                                                                                                                                                 | 0/1000 [00:00<?, ?it/s]2025-01-07 11:55:11.516031: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:117] Plugin optimizer for device_type GPU is enabled.
[libprotobuf FATAL google/protobuf/message_lite.cc:353] CHECK failed: target + size == res: 
libc++abi: terminating due to uncaught exception of type google::protobuf::FatalException: CHECK failed: target + size == res: 
zsh: abort      pixplot --images "datasets/oslomini/images/images/*" --metadata 
(pixplotEnv) michelemauri@Mac pix-plot % /opt/anaconda3/envs/pixplotEnv/lib/python3.9/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '

it creates an "output" folder, but when i browse it with local server it is stuck at 0%. When i check the console it says: http://localhost:5500/output/data/manifest.json 404 (File not found)

[nabsiddiqui]

Try pip install tensorflow-macos tensorflow-metal

[nabsiddiqui]

I'm pretty sure this is more or less an abandoned project though. It works really well and I know a bunch of people that use it, but it needs to be updated.

Perhaps @wjbmattingly would know more?

[wjbmattingly]

Thanks for tagging me. I was a huge fan of PixPlot before coming to Yale (and still am!). I will do some digging to see if anyone is maintaining it. If you want something similar (but less robust), consider Bulk. It works well and is well-maintained.

[nabsiddiqui]

@wjbmattingly I think most of the code is still working. I keep an eye out for issues, but I don't work on the code. The simple issues are the installation dependencies and the various user requests. I've created a bunch of pixPlots over the year. There also seem to be some issues with blurriness and clicking images when they are in a 3d space.

Larger issue is probably that image embeddings should be used rather than CNNs.

I did a search for Bulk but couldn't find it. Can you provide a link?

[wjbmattingly]

@nabsiddiqui ahh good to know! Yeah. There are definitely better ways to do embeddings today. Here is bulk https://github.com/koaning/bulk . It uses more modern embedding abbroaches by default but you can really use any model you want on HuggingFace.