Why? Prior art?

[rotu]

It seems like there is already a package system used by SWI prolog:

https://www.swi-prolog.org/howto/Pack.html

1. Why is a different package manager necessary? Is there some reason it is infeasible or undesirable to adopt prolog_pack? (the long list of dependencies might be such a reason https://www.swi-prolog.org/pldoc/doc/_SWI_/library/prolog_pack.pl)
2. Would it be possible to extend the pack.pl format in a way that is compatible? (e.g. adding a new term to declare the dialect or version of the Prolog processor required, or define some package tokens to declare compatibility or incompatibility with various language features)
3. Are there any other package managers to adopt or compare/contrast against? EDIT I found a few:
   • https://github.com/wouterbeek/ppm
   • https://logtalk.org/manuals/devtools/packs.html
4. Nix is supposedly language-independent (and this repo contains a flake.nix file!). Is there some reason why it's a bad fit for Prolog packages?

[bakaq]

See also #1.

1- As far as I know both SWI's pack and LogTalk's pack depend on system specific features that would make them a bit of a pain to port. I think both of them also assume which predicates come from which packages, which is different in different systems (length/2 is builtin in SWI but comes from library(lists) in Scryer) . LogTalk's is the more interesting one because it seems that it can work with any (close to) ISO Prolog as a backend. However it's important to note that LogTalk currently doesn't support Scryer (because politics basically).

2- Compatibility with SWI's pack.pl manifest is certainly an interesting thing to consider, but I think doing it properly is a bit unfeasible. It would be basically reimplementing LogTalk, see #1 (comment).

3- Both of those have the same "system specificity" that I pointed out in 1.

4- It is a bad fit because to use it properly I think we would need to solve multiple versions of the same dependency (which happens all the time in Nix and it's actually a feature). However, poetry2nix exists, so it may actually be possible to do something there. Another reason that it's a bad fit is that Nix is not supported on Windows and I don't think it will gain support anytime soon.

[bakaq]

There's also the exploratory aspect (we may find a better way to do Prolog packages than any preexisting system, or at least document the bad ways we stumble on) and the less technical aspect that it's just a lot of fun to design things like this 😃.

[rotu]

As far as I know both SWI's pack and LogTalk's pack depend on system specific features that would make them a bit of a pain to port.

Yeah SWI's pack seems heavy and very SWI-specific.

LogTalk's is the more interesting one because it seems that it can work with any (close to) ISO Prolog as a backend. However it's important to note that LogTalk currently doesn't support Scryer (because politics basically).

That's odd. LogTalk seems to support Trealla which is a similar-seeming project.

Compatibility with SWI's pack.pl manifest is certainly an interesting thing to consider, but I think doing it properly is a bit unfeasible. It would be basically reimplementing LogTalk, see #1 (comment).

I don't think that's necessary. pack.pl is supposed to only contain ground terms, so only very limited parsing is necessary.

There's also the exploratory aspect (we may find a better way to do Prolog packages than any preexisting system, or at least document the bad ways we stumble on) and the less technical aspect that it's just a lot of fun to design things like this 😃.

Definitely. Greenfield development is FUN!

[bakaq]

pack.pl is supposed to only contain ground terms, so only very limited parsing is necessary.

I was thinking more about feature support than just the format. I don't know for sure, but some features of SWI's pack may leak to pack.pl in ways that are incompatible with what we are trying to do here, and vice versa. Still, it's something to keep in mind.

[bakaq]

That's odd. LogTalk seems to support Trealla which is a similar-seeming project.

It did also have support for Scryer Prolog for quite some time, but then, unfortunately, politics. If you want the deep lore you can start from the end and explore the links backwards.

[constraintAutomaton]

@bakaq already gave a thorough answer, but for me, the main motivation is having a convenient, ISO Prolog-oriented solution, rather than relying on system-specific managers or using general-purpose tools not designed for Prolog dependency management.

I also hope we can leverage the strengths of Prolog to do actual reasoning over dependencies, for example, license compatibility, automatic system compatibility detection, and potentially even (semi-)automatic compatibility layers.

I believe a well-designed package manager could also encourage more code discovery, reuse, and sharing. Of course, a good registry (as mentioned in #10) is probably the key piece, but a solid package manager is an a priori.

As someone new to Prolog, it's surprising to me that this kind of tooling doesn't already exist. It also feels strange that Prolog is so fragmented across systems, almost like multiple dialects (a bit of a strong word, but that’s how it feels) tied to specific implementations, rather than one language with multiple compatible implementations.

I hope a package manager could help, in some way, to address what seems to me like a issue. (We have to dream big, right? 😂)

[rotu]

for me, the main motivation is having a convenient, ISO Prolog-oriented solution, rather than relying on system-specific managers

Yes, BUT the module system of Scryer is not ISO, so I think any package management system is going to wind up being rather system-specific.

I also hope we can leverage the strengths of Prolog to do actual reasoning over dependencies, for example, license compatibility, automatic system compatibility detection, and potentially even (semi-)automatic compatibility layers.

That would be very cool.

As someone new to Prolog, it's surprising to me that this kind of tooling doesn't already exist. It also feels strange that Prolog is so fragmented across systems, almost like multiple dialects (a bit of a strong word, but that’s how it feels) tied to specific implementations, rather than one language with multiple compatible implementations.

I strongly agree. That's why I assumed that (1) this must already exist and (2) if it's not compatible, surely it must be easier to pick out the bad parts than rewrite de novo.

I think "dialects" is actually too weak a word. It seems like different Prolog systems are more isolated than I would expect. It might just be that people think "we don't need a new Prolog - let's just extend SWI" or something like that. Or it might be that something like lack of first-class strings or imperative control flow wind up being a "great filter" that prevent such projects from thriving.

[constraintAutomaton]

Yes, BUT the module system of Scryer is not ISO, so I think any package management system is going to wind up being rather system-specific.

I don't have the ISO specification to know the exact difference, but I'm not sure it's something we have to be bound by. I think we can adapt. For example, in JavaScript they have both CommonJS and ESM, and package management still works fine.

I strongly agree. That's why I assumed that (1) this must already exist and (2) if it's not compatible, surely it must be easier to pick out the bad parts than rewrite de novo.

I think it could be made compatible or even used as-is. And for sure, we should take inspiration from pack.pl.

I think "dialects" is actually too weak a word. It seems like different Prolog systems are more isolated than I would expect. It might just be that people think "we don't need a new Prolog - let's just extend SWI" or something like that. Or it might be that something like lack of first-class strings or imperative control flow wind up being a "great filter" that prevent such projects from thriving.

I don’t really have the background to speak (a wiser me probably wouldn’t try), but it does seem like the Prolog standard is lagging behind actual practice. I don’t know if the paywall around the standard contributes to that and it seems like there is no easy and open way to comment or contribute. Actually at my work the team (or maybe more the individual 😂) working with prolog did not go for SWI because it deviate too much from the standard and wanted to have something that would be more portable.

[bakaq]

I don't have the ISO specification to know the exact difference [...]

The difference is that no one actually does ISO Modules, so any package manager that actually uses the module systems of implementations as a basis is going to be implementation specific unless you do a lot of work to abstract it out like LogTalk does.

[triska]

As someone new to Prolog, it's surprising to me that this kind of tooling doesn't already exist.

Regarding this point: Please note how recent this all is. Scryer Prolog and Trealla Prolog are the first free Prolog systems that begin to approach the feature set of SICStus Prolog in essential areas such as ISO conformance, a correct implementation of dif/2, CLP(ℤ), library(pio), SICStus-compatible attributed variables and module systems etc. These are very recent developments, all happening within the last few years. The situation that—notably due to the compact internal string representation implemented in both Trealla and Scryer—already two free systems can now even tackle tasks where previously substantially more memory was needed, is totally unprecedented. Now that several Prolog implementations become so good and attract community interest, corresponding interest in tooling also arises.

[triska]

Yes, BUT the module system of Scryer is not ISO, so I think any package management system is going to wind up being rather system-specific.

I am surprised to see such statements now repeated here. The Scryer module system is compatible with SICStus. I again point to ichiban/prolog#292 (comment).

[bakaq]

My bad. I'm really trying to research about this myself but everything I find out seems to completely contradict all that you and Ulrich state about the topic. It's like we are looking at 2 completely different documents, and the more I look at it the less sense it makes. I'm really getting insane.

[rotu]

The module system is compatible in the sense of any processor-specific extension.

I think the question is more to do with Scryer itself and the sense in which it is ISO conformant. My assumption was that Scryer supports an ISO “strictly conforming mode”. But on close examination, I’m now realizing that there are 2 definitions of “strictly conforming” and they differ.

In order to have a package manager which runs in 13211-1 strictly conforming mode, it can’t use a module system. Even the module system defined in 13211-2.

You could have a package manager which uses 13211-2 modules but no processor (including Scryer) supports them!

[UWN]

(This is the curse of github-discussions. Things get spread around into unrelated subjects sans cesse.)

So once again, my answer on SO remains my view on the subject. We could go into more detail and inconsistencies in 13211-2, we could view it from a historical perspective which would show the inevitability of all of this.

My own most important conclusion was to stick to actual implementations when considering any step in codification.

[rotu]

My thinking on this has definitely changed over the last month or so. There is enough in the Prolog General Core to support a cross-system package manager, even without any module system using the standard include/1 and ensure_loaded/1, as well as the ubiquitous (though not ISO) consult/1. You gotta start somewhere.

So once again, my answer on SO remains my view on the subject. We could go into more detail and inconsistencies in 13211-2, we could view it from a historical perspective which would show the inevitability of all of this.

Surely SICStus heavily influenced the module system. And clearly 13211-2 has typos, inconsistencies, and omissions. I would eagerly consume anything about the details of 13211-2 and historical perspective if you have anything more to share on the topic.

the more I look at it the less sense it makes. I'm really getting insane.

AGREED. I want to validate on a personal level that you are not alone in your confusion. Frank discussion ISO modules feels taboo at best and like willful disinformation at worst.

I can't figure out how any Prolog can be regarded as 13211-2 conformant if it doesn't expose the colon_sets_calling_context flag nor recognize strictly conforming module text containing the import/2 directive. I also can't figure out why a product like SICStus would be considered compliant but not mention it in the documentation.

I recognize that there's still much work in getting vendors to align with existing standards, so 13211-2 might be seen as a distraction. Or maybe 13211-2 is still so incompatible or controversial that its adoption would be harmful. Either of these could rationalize why the technical details of 13211-2 seem to be treated as "mere suggestions".

[constraintAutomaton]

For the term expansion include/1, ensure_loaded/1 and consult/1 are definitely terms that we should support.