[HF][5/n] Image2Text: Allow base64 inputs for images

[rossdanlm]

[HF][5/n] Image2Text: Allow base64 inputs for images

Before we didn't allow base64, only URI (either local or http or https). This is good becuase our text2Image model parser outputs into a base64 format, so this will allow us to chain model prompts!

Test Plan

Rebase and test on 0d7ae2b.

Follow the README from AIConfig Editor https://github.com/lastmile-ai/aiconfig/tree/main/python/src/aiconfig/editor#dev, then run these command

aiconfig_path=/Users/rossdancraig/Projects/aiconfig/cookbooks/Gradio/huggingface.aiconfig.json
parsers_path=/Users/rossdancraig/Projects/aiconfig/cookbooks/Gradio/hf_model_parsers.py
alias aiconfig="python3 -m 'aiconfig.scripts.aiconfig_cli'"
aiconfig edit --aiconfig-path=$aiconfig_path --server-port=8080 --server-mode=debug_servers --parsers-module-path=$parsers_path

Then in AIConfig Editor run the prompt (streaming not supported so just took screenshots)

These are the images I tested (with bear being in base64 format)

---

Stack created with Sapling. Best reviewed with ReviewStack.

• -> [HF][5/n] Image2Text: Allow base64 inputs for images #856
• [HF][streaming][4/n] Image2Text (no streaming, but lots of fixing) #855
• [HF][streaming][3/n] Text2Speech (no streaming, but updating docs on completion params) #854
• [HF][streaming][2/n] Text Translation #853
• [HF][streaming][1/n] Text Summarization #851

[saqadri]

This seems reasonable for now until we standardize input types.

Would it be better to flip it? i.e. check if the input_data is a URL or file path?

Perhaps something like:

from urllib.parse import urlparse
from pathlib import Path

def could_be_url_or_path(s):
    # First, check if it could be a URL.
    try:
        urlparse(s)
    except ValueError:
        pass  # Not a URL. Might still be a path.
    else:
        # URL parsing didn't raise an error, so s might be a URL.
        # Check that s has a scheme and a network location.
        if all([urlparse(s).scheme, urlparse(s).netloc]) or s.startswith('file://'):
            return True  # URL

    # Next, check if it could be a path
    p = Path(s)
    if p.is_absolute() or (p.is_relative_to('.') and p.suffix):
        return True  # Path

    # It's not any of them above -> Return False
    return False

[saqadri]

Merging this stack so I can build on top. Will leave the diff comments open, but nothing is blocking.

Thanks for driving this with these excellent fixes!

[saqadri]

Also, amazing test plans. Makes it very easy to replicate as well.