Allow custom binary/artifact repositories

[WubiCookie]

Is your feature request related to a problem? Please describe.

Our software is based on DLLs compiled by MATLAB, but due to license restrictions, only one PC can compile them. I wanted to store the compiled DLLs on a binary repository so that we could build our software on any machine.

Describe the solution you'd like

I would like to be able to host an artifact repository myself.

Describe alternatives you've considered

What we currently do is store the binary packages on svn, checkout it and declare the newly cloned folder as a local package repository. The svn cloning can not be automatized with xmake

Additional context

No response

[waruqi]

xmake supports adding custom artifacts repositories, just call add_repositories or xrepo add-repo to add it.

[WubiCookie]

add_repositories does not work, my package is not found. It seems the repository needs to be added globally, and through an env var, which is not very practical
https://github.com/xmake-io/xmake/blob/master/xmake/core/sandbox/modules/import/core/package/repository.lua#L108-L128

[WubiCookie]

note: the following packages were not found in any repository (check if they are spelled correctly):
  -> testlib
error: @programdir\core\main.lua:280: stack traceback:
    [C]: in function 'error'
    [@programdir\core\base\os.lua:883]:
    [...modules\private\action\require\impl\install_packages.lua:729]:
    [@programdir\modules\private\action\require\install.lua:85]:
    [@programdir\actions\require\main.lua:123]:
    [C]: in function 'xpcall'
    [@programdir\core\base\utils.lua:280]:
    [@programdir\core\base\task.lua:501]: in function 'run'
    [@programdir\core\main.lua:278]: in function 'cotask'
    [@programdir\core\base\scheduler.lua:388]:

stack traceback:
        [C]: in function 'error'
        @programdir\core\base\os.lua:872: in function 'os.raiselevel'
        (...tail calls...)
        @programdir\core\main.lua:280: in upvalue 'cotask'
        @programdir\core\base\scheduler.lua:388: in function <@programdir\core\base\scheduler.lua:385>

[waruqi]

no, add_repositories will work, but you need add this package to source code repositories first.

The build artifacts package is only used for accelerated binary installation of source packages, so you first need to define a source package in the source code repository. Then go to the build-artifacts repository and configure the artifact package for this package.

xmake/xmake/modules/private/action/require/impl/package.lua

Lines 819 to 824 in 7d245d6

	if from_repo and not option.get("build") and not requireinfo.build then
	local artifacts_manifest = repository.artifacts_manifest(packagename, version)
	if artifacts_manifest then
	_select_artifacts(package, artifacts_manifest)
	end
	end

[waruqi]

If you are not intending to do package binary acceleration and simply want to follow the binary libraries, then you can define them directly in the source code repository.

There is no need for the build-artifacts repository at all, the packages in xmake-repo are also supported for installing binary libraries. You just need to call os.cp in on_install to install them.

https://github.com/xmake-io/xmake-repo/blob/596ea080a6638a7bf26a32d93895a9e8c7e5b10f/packages/l/libsdl_image/xmake.lua#L40-L41

[WubiCookie]

Indeed, that is much simpler. Thanks!

[WubiCookie]

oh, but how do I handle packages with different modes and/or different options? I guess I could do like here https://github.com/xmake-io/xmake-repo/blob/596ea080a6638a7bf26a32d93895a9e8c7e5b10f/packages/l/libsdl_image/xmake.lua#L6 but it's fine as long as I only have a few variants... The manifest.txt was a good solution for that

[waruqi]

what about the version hash?

you need check sha256 by yourself if you use on_download

[WubiCookie]

ok, I'll try that, thanks

[waruqi]

or you can set urls in on_load.

function on_load(package)
    if package:config("xxx") then
        package:set("urls", "xxx")
    end
end

[WubiCookie]

but then how do I check the sha256?

[waruqi]

xmake/xmake/modules/private/action/require/impl/actions/download.lua

Line 126 in afbbc9a

if not os.isfile(packagefile) or sourcehash ~= hash.sha256(packagefile) then