Fix None issue in reorganizer queue#124
Conversation
There was a problem hiding this comment.
Pull Request Overview
This PR adds implementation files for the MMLongBench-Doc benchmark for evaluating long-context document understanding with visualizations. The PR includes model integration files for multiple LVLMs (Large Vision Language Models), evaluation utilities, and documentation.
- Adds run_api.py for API-based model execution with MOS (Memory Operating System) integration
- Implements model wrappers for three LVLM architectures: MiniCPM-Llama3, InternVL-Chat, and InternLM-XComposer2-4KHD
- Provides evaluation framework with answer extraction, scoring, and prompt templates
- Includes comprehensive README documentation with setup instructions and usage examples
Reviewed Changes
Copilot reviewed 8 out of 11 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| MMLongBench-Doc/run_api.py | Main API runner integrating MOS memory system for document processing and evaluation |
| MMLongBench-Doc/models/minicpm_llama3.py | Model wrapper for MiniCPM-Llama3-V-2.5 vision-language model |
| MMLongBench-Doc/models/internvl_chat.py | Model wrapper for InternVL-Chat with dynamic image preprocessing |
| MMLongBench-Doc/models/internlm_xc2_4khd.py | Model wrapper for InternLM-XComposer2-4KHD with high-definition image support |
| MMLongBench-Doc/eval/prompt_for_answer_extraction.md | Prompt template for extracting structured answers from model responses |
| MMLongBench-Doc/eval/extract_answer.py | OpenAI API client for answer extraction using GPT-4o |
| MMLongBench-Doc/eval/eval_score.py | Comprehensive evaluation utilities for scoring different answer types |
| MMLongBench-Doc/README.md | Project documentation with setup instructions and usage examples |
| mos.clear_messages() | ||
| response = mos.chat(messages) | ||
| is_success = True | ||
| except: |
There was a problem hiding this comment.
Using bare except clause is not recommended. Consider catching specific exceptions or at minimum use 'except Exception:' to avoid catching system-exiting exceptions like KeyboardInterrupt.
| except: | |
| except Exception: |
|
|
||
| try: | ||
| from transformers.generation.streamers import BaseStreamer | ||
| except: # noqa # pylint: disable=bare-except |
There was a problem hiding this comment.
Although there are disable comments, using bare except is still not recommended. Consider catching ImportError specifically since this appears to be handling an optional import.
| except: # noqa # pylint: disable=bare-except | |
| except ImportError: # noqa |
| reference = float(str(reference).strip().rstrip("%").strip()) | ||
| try: | ||
| prediction = float(str(prediction).strip().rstrip("%").strip()) | ||
| except: |
There was a problem hiding this comment.
Using bare except clause is not recommended. Consider catching ValueError or TypeError specifically since this is handling float conversion.
| except: | |
| except (ValueError, TypeError): |
| if answer_type == "Int": | ||
| try: | ||
| gt, pred = int(gt), int(float(pred)) | ||
| except: |
There was a problem hiding this comment.
Using bare except clause is not recommended. Consider catching ValueError or TypeError specifically since this is handling type conversions.
| except: | |
| except (ValueError, TypeError): |
| try: | ||
| gt = float(get_clean_string(str(gt))) | ||
| pred = float(get_clean_string(str(pred))) | ||
| except: |
There was a problem hiding this comment.
Using bare except clause is not recommended. Consider catching ValueError or TypeError specifically since this is handling float conversion.
| except: | |
| except (ValueError, TypeError): |
| ] | ||
| ) / len([sample for sample in evaluated_samples if sample["pred"] != "Not answerable"]) | ||
| f1 = 2 * recall * precision / (recall + precision) if (recall + precision) > 0.0 else 0.0 | ||
| except: |
There was a problem hiding this comment.
Using bare except clause is not recommended. Consider catching ZeroDivisionError specifically since this is handling division operations.
| except: | |
| except (ZeroDivisionError, ValueError): |
Description
Summary: (summary)
fix the None type in minheap of reorganizer.
Reviewer: @CaralHsi
Checklist: