VILA-M3 Demo Feedback

[holgerroth]

Please provide any feedback you might have regarding using the VILA-M3 demo.

[amitjc]

Dear Experts,

Congratulations on the brilliant project.

Looking forward to following features:

1. Provision to run the inference on user data
2. Support for DICOM
3. User feedback for model fine-tuning if there's any inaccuracy in the AI assistant prediction.

Thanks and Regards,
Amit.

[mingxin-zheng]

Hi @amitjc , For the feedback, I agree with 1 and 2. For 3, I'm unclear what forms of user feedback may be. Can you give an example how that should work?

Thank you!

[amitjc]

any updates?

[mingxin-zheng]

Hi @amitjc , sorry for my delayed reply. I took a break to end 2024.

I couldn't replicate what you mentioned above. On my side, it's working okay. The video should explain what I saw, by starting with no user-uploaded image. Then I uploaded the 1st one (text-labeled with "User 1") and the 2nd one ("User 2) subsequently.

There is a small delay for the UI to refresh around 00:00:29. Is that what confuses you?

20250101_175118.mp4

[mingxin-zheng]

Hi @amitjc , please let me know if we can merge this feature:
#59

Thanks!

[amitjc]

Hello @mingxin-zheng,

Apologies for my delayed response.

I’m still struggling to upload multiple X-ray cases. For instance after analysing the first radiograph, uploading next image doesn’t display the new uploaded image/s, instead only earlier uploaded image is shown. I don’t know what I’m missing.

It’s definitely a desirable feature, that you have added for the local instance, to run the inference on user data.

You may consider merging the feature.

Thanks.

Best Wishes,
Amit.

[mingxin-zheng]

Thanks @amitjc . I created this issue to track.

I will merge my PR first and investigate if I can reproduce the error you met again.

Hello @mingxin-zheng,

Apologies for my delayed response.

I’m still struggling to upload multiple X-ray cases. For instance after analysing the first radiograph, uploading next image doesn’t display the new uploaded image/s, instead only earlier uploaded image is shown. I don’t know what I’m missing.

It’s definitely a desirable feature, that you have added for the local instance, to run the inference on user data.

You may consider merging the feature.

Thanks.

Best Wishes, Amit.

[michelodu]

Hello,
I'm a prof, previously with Kitware prior to coming to Old Dominion University, and I have a question regarding Vila M3. I have suggested Vila M3 to a PhD student who would like to pursue VLMs for pathology applications, which includes 2D microscopy images. This student reported not being able to get Vila M3 to work with microscopy images because of their 2D format, given that the code base emphasizes 3D. This seems to me like a trivial exercise, converting 3D to 2D, but the student claims otherwise. What would be involved in adapting Vila M3 to 2D analysis applications?
Thanks for your support. Best wishes,
Michel