Enhance pub_dev_search to include changelog

[RyanRamchandar]

This is useful for allowing coding assistants to update dependences while checking a packages changelog for breaking changes.

[jakemac53]

I agree exposing changelogs in some way is likely useful, although I think it mostly only is relevant for the packages you have actually added to your project, and only up to the specific version you have solved for?

So, maybe we should instead have a get_changelogs(List<String> packageNames) tool? And it would use your package config to resolve the those packages based on your exact current version solve, and return the changelogs it finds there.

This would also work for non-pub dependencies (local path dependencies, git dependencies, etc).

[jakemac53]

I do worry slightly that some changelogs are very long (especially if they are automated), and so they might take up quite a lot of tokens.

We may want to consider trimming them or summarizing them (via sampling if the client supports it) or something like that.

[RyanRamchandar]

I agree exposing changelogs in some way is likely useful, although I think it mostly only is relevant for the packages you have actually added to your project, and only up to the specific version you have solved for?

Yes, agreed. Useful just to help during migrations to newer versions.

So, maybe we should instead have a get_changelogs(List<String> packageNames) tool? And it would use your package config to resolve the those packages based on your exact current version solve, and return the changelogs it finds there.

I like this approach.

We may want to consider trimming them or summarizing them (via sampling if the client supports it) or something like that.

Trimming makes sense here as subtle details could be lost in summarization. If possible, trim the changelog of older entries than the users package version.

[davidmigloz]

Building on the idea of trimming the changelog, what if we made it more explicit by allowing (optional) version ranges?

The most common use case for checking a changelog is when updating a dependency from a version you currently have to a newer one. In this scenario, you're only interested in the changes between those two versions.

I propose we enhance the tool to accept from and to version parameters, like: get_changelog(packageName: 'openai_dart', from: '0.5.0', to: '0.5.4')

This has several benefits:

1. It gives the developer/agent the exact information they need for the migration task.
2. It directly solves the concern about token limits by naturally scoping the output.
3. This mirrors the behavior of tools like GitHub's Dependabot, which already shows the relevant part of the changelog in its update pull requests.

On the implementation side, since changelog formats aren't standardized, the server could use a hybrid approach. The primary strategy could be to parse the changelog to extract clean version sections (that most packages use). If that fails due to an unusual format, it could fall back to providing a diff of the changelog files between the two versions.

What do you think?

[RyanRamchandar]

I propose we enhance the tool to accept from and to version parameters, like: get_changelog(packageName: 'openai_dart', from: '0.5.0', to: '0.5.4')

Yes, and users may also not want to migrate to the latest version.

On the implementation side, since changelog formats aren't standardized, the server could use a hybrid approach. The primary strategy could be to parse the changelog to extract clean version sections (that most packages use). If that fails due to an unusual format, it could fall back to providing a diff of the changelog files between the two versions.

Such as a git diff between the changelogs at specified versions.

Another edge case to consider is pre-releases can have alternate changelogs.

[jakemac53]

We can't necessarily get the old versions of the changelogs (we could for pub packages assuming the versions actually exist on pub, but not for path dependencies or git dependencies).

I would probably just lean towards some sort of fuzzy matching for header lines containing the version numbers?

And then fall back on not trimming the file or something if we can't make sense of it.

[davidmigloz]

That's a very good point about the limitations with path and git dependencies (I was just thinking about pub.dev dependencies).

I agree that fuzzy matching for header lines is the best way forward. If that parsing fails, falling back to a generously trimmed version of the full changelog file should be good enough (the LLM can hopefully just discard the non-relevant parts).

[jakemac53]

It also occurs to me that the other advantage of reading the changelog from the actual version of the package that was resolved means the latest entry in the changelog will already be the upper bound that we want. There will not be any newer entries in there from unrelated versions, only older entries.

So, we probably only need to have a from parameter and not a to parameter. And can fall back on just returning the last X lines of the changelog.

[davidmigloz]

That logic would work for the scenario where we're analyzing the changelog from the package version already resolved in the project.

However, I believe the most useful use case for this tool is to help an agent (or a developer) decide whether to perform an update in the first place, by analyzing the changes before they are applied.

I imagine the agent's workflow would be:

1. Run dart pub outdated.
2. For each outdated dependency, identify the current version and the resolvable version.
3. Call get_changelog(packageName: 'some_package', from: {current}, to: {resolvable}) to know the changes between both versions.
4. Analyze the changelog for breaking changes to decide if it's a safe update and if migrations are needed.
5. If safe, update the pubspec.yaml, run pub get, and then apply any necessary code migrations.
6. Output a summary of the changelogs of all updated dependencies.

In this flow, the to parameter is still important for a couple of key scenarios:

• When the latest version you can resolve to is lower than the package's absolute latest version (due to other constraints). We need to tell the tool to stop at the resolvable version, not the latest.
• When a developer wants to intentionally upgrade to a specific pre-release version.

[jakemac53]

This sounds to me like really it is a request for a custom prompt update-dependencies, given it's really an entire flow we want users to be able to go through.