Merge Protections

Your pull request matches the following merge protections and will not be merged until they are valid.

🟢 Enforce conventional commit

The overall approach looks good to me. I do wonder about everything becoming async but I think it's okay.

@nrfulton, if we want to keep synchronous versions of functions around we definitely can. generate_from_context would just have to have a parameter that sets the correct generate function depending on what's desired. It just makes higher level abstractions like sampling strategies and result validation a bit more complicated with juggling the async vs sync versions.

I think, I understand the logic... but currently from the outside, .chat and .instruct act the same for streaming and non-streaming, or ?

Yes; once we introduce partial requirement checking / logging, you will be able to notice differences with streaming, but no real functional differences between the two.

I still need to make a few tweaks and push my changes to fix aloras / logging.

@nrfulton @HendrikStrobelt
I was unable to test vllm/openai aloras because there is some issue with our vllm script (will open an issue for that as well). All the other tests passed locally for me (but we'll see what the github actions say).

Still need to finish up and push the documentation, but the code is ready for review. I'll monitor for test failures, I believe I fixed all of the things failing due to merging main back into my branch.

I am trying to debug why the tests are failing; it's some out of space issue so I'm not sure if I just happened to be the unlucky one that pushed us over the edge or if some change I made is actually causing the issue.

Tested that new code works in collab by installing my specific branch of mellea. Didn't see any issues in the notebook.