I put together a proof of concept to get Docker support, what do you think?

[nickjj]

Hi,

All of my apps are running in Docker and psql is not installed locally. It's often not as easy as installing it through a package manager because your psql client needs to match the server and the server might be running different versions of Postgres depending on the project.

Neovim in this case is running on my work machine and Postgres is running in a container on that machine.

I poked around this plugin and changed references of ['psql'] to ['docker', 'compose', 'exec', 'postgres', 'psql'] where postgres is the name of the service in my compose.yaml file. If the project is up and running with Docker then dadbod is able to run psql in Docker.

That quickly failed saying /tmp/nvim.nick/abc/4 does not exist where abc is a random value which changes every time you open Neovim.

I hacked around that by volume mounting that path into postgres:

    volumes:
      - "postgres:/var/lib/postgresql/data"
      - "/tmp/nvim.nick:/tmp/nvim.nick"

Then it failed for another missing path, to which I also volume mounted:

   volumes:
      - "postgres:/var/lib/postgresql/data"
      - "/tmp/nvim.nick:/tmp/nvim.nick"
      - "/home/nick/.local/share/nvim/dadbod_ui/tmp:/home/nick/.local/share/nvim/dadbod_ui/tmp"

In both cases, if you were to try this, replace nick with your machine's user name.

With that in place, as far as I can tell it's working, complete with linting and auto-complete:

The local modifications I made to the compose.yaml file are in this project https://github.com/nickjj/docker-django-example if you want to use it as a quick example to have a full app running in Docker with Postgres. I didn't commit those up so you'll need to make local edits. The connection string is postgresql://hello:password@localhost:5432/hello.

Needless to say having to volume mount those paths isn't a good long term solution. I'm not well versed in Vimscript but I was wondering if you can do your tpope magic to make it "just work". Perhaps you could expose cmd as an option for the binaries so folks can customize that and then the plugin could be adjusted to not read and write from those tmp and nvim share directories from the same source as where the psql binary was found? Is there a way to make it use the same file system associated with where Neovim is running?

[kareem-abdul]

Ok for me I was using sqlserver, and the volume approach wont work as the volume is attached as a root folder and the sqlserver runs as mssql user. I know you can force the container to run as root, but this did not work for me when I tried a while ago.

What worked for me was to comment out the #input function

vim-dadbod/autoload/db/adapter/sqlserver.vim

Lines 55 to 57 in 9f0ca8b

	function! db#adapter#sqlserver#input(url, in) abort
	return db#adapter#sqlserver#interactive(a:url) + ['-i', a:in]
	endfunction

and put the following script as sqlcmd in my path.

#!/usr/bin/env bash

CONTAINER="<container id>"

args="$@"
FORMATED_ARG_STR=""
while [[ $# -gt 0 ]]; do
    case $1 in
        -Q|-s)
            FORMATED_ARG_STR+="$1 \"$2\" "
            shift 2
            ;;
        *)
            FORMATED_ARG_STR+="$1 "
            shift
            ;;
    esac
done

command="docker exec -i $CONTAINER env SQLCMDCOLSEP=\"|\" SQLCMDMAXFIXEDTYPEWIDTH=30 SQLCMDMAXVARTYPEWIDTH=60 /opt/mssql-tools18/bin/sqlcmd ${FORMATED_ARG_STR}"
results=$(eval "${command}")
echo "$results"

Why this works is because, the db.vim script first executes the sql commands as a file first if the #input method is present, otherwise it uses the #interactive method.

vim-dadbod/autoload/db.vim

Lines 144 to 148 in 9f0ca8b

	if db#adapter#supports(a:url, 'input') && !(prefer_filter && db#adapter#supports(a:url, 'filter'))
	return [db#adapter#dispatch(a:url, 'input', a:in), '']
	endif
	let op = db#adapter#supports(a:url, 'filter') ? 'filter' : 'interactive'
	return [db#adapter#dispatch(a:url, op), a:in]

This script also parses the arguments passed to sqlcmd and wraps values of -Q (used to pass query in command line) and -s in double quotes as without this when we pass the command to docker it will execute without double quotes and gives errors due to it.

For postgres I did the same and used the following psql script

#!/usr/bin/env bash

CONTAINER='<container-name>'

args="$@"
FORMATED_ARG_STR=""
while [[ $# -gt 0 ]]; do
    case $1 in
        -c)
            FORMATED_ARG_STR+="$1 \"$2\" "
            shift 2
            ;;
        *)
            FORMATED_ARG_STR+="$1 "
            shift
            ;;
    esac
done

command="docker exec -i $CONTAINER psql ${width} ${FORMATED_ARG_STR}"
results=$(eval "${command}")
echo "$results"

[kareem-abdul]

I think in future the approach might be to pass a custom argument in connection url like docker=true , and add a check here

vim-dadbod/autoload/db.vim

Lines 144 to 148 in 9f0ca8b

	if db#adapter#supports(a:url, 'input') && !(prefer_filter && db#adapter#supports(a:url, 'filter'))
	return [db#adapter#dispatch(a:url, 'input', a:in), '']
	endif
	let op = db#adapter#supports(a:url, 'filter') ? 'filter' : 'interactive'
	return [db#adapter#dispatch(a:url, op), a:in]

for if #docker is present to execute that function
The custom method #docker will prompt the user for the container id either using vim.ui.input or vim.ui.select and returns the command like docker exec -i <container_id> <result of #interactive function>

[tpope]

My high level recommendation would be to create a new adapter called docker that delegates to existing adapters. You could have URLs that looked something like docker:container-id:postgresql:///dbname, and then db#adapter#docker#interactive() could call db#adapter#postgresql#interactive() and prepend ['docker', 'exec', '-i', 'container-id'] to the command. You could solve the #input() problem by not implementing that method.

[nickjj]

Thanks. How would you suggest we solve the volume problem?

Ideally we wouldn't want to make application changes (adjusting a compose.yaml file) to add Vim specific volume mounts.

[tpope]

The volume problem is the #input() problem, right?

[nickjj]

If it's not implemented, then would you lose the ability to save queries since nothing would get written back to the host?

[kareem-abdul]

If it's not implemented, then would you lose the ability to save queries since nothing would get written back to the host?

I don't think so, because the approach suggested does not use volume's and all the queries and files are on host, so you can save queries without a problem.
The suggested solution just sends the contents of these files to docker along with the #interactive command for the respective adapter.

[kareem-abdul]

My high level recommendation would be to create a new adapter called docker that delegates to existing adapters. You could have URLs that looked something like docker:container-id:postgresql:///dbname, and then db#adapter#docker#interactive() could call db#adapter#postgresql#interactive() and prepend ['docker', 'exec', '-i', 'container-id'] to the command. You could solve the #input() problem by not implementing that method.

This works.
The only problem is for sqlserver container, the sqlcmd command is not in path, you need to explicitly give the /opt/mssql-tools18/sqlcmd to execute it.

not sure if other containers exhibit the same behaviour.

This could be also solved on the user side by updating the PATH in docker container (either through the image or in docker-compose) manually.

[tpope]

This could be also solved on the user side by updating the PATH in docker container (either through the image or in docker-compose) manually.

Yeah, or you could add something like env PATH=/opt/msql-tools/bin to the command you're building. Though if you want a generic adapter you'll need something more elaborate.

[ThomasCode92]

@tpope, thanks for the amazing plugin! Is there any chance the docker adapter will be implemented?