gdalUtils and gdal 3.0.2 : ERROR 1: PROJ: proj_create_from_wkt: Cannot find proj.db

[mgageo]

Hello,
I use gdaUtils with the gdal version from QGIS : https://www.qgis.org/fr/site/forusers/download.html
https://qgis.org/downloads/QGIS-OSGeo4W-3.10.0-2-Setup-x86_64.exe

On a command like

ogr2ogr('rgdal.geojson','rgdal.json',t_srs="EPSG:2154",verbose=TRUE)
Checking gdal_installation...
Scanning for GDAL installations...
Checking the gdalUtils_gdalPath option...
GDAL version 3.0.2
GDAL command being used: "C:\Program Files\QGIS 3.10\bin\ogr2ogr.exe" -t_srs "EPSG:2154" "rgdal.json" "rgdal.geojson"
[1] "ERROR 1: PROJ: proj_create_from_wkt: Cannot find proj.db" "ERROR 1: PROJ: proj_create_from_wkt: Cannot find proj.db"
[3] "ERROR 1: PROJ: proj_create_from_database: Cannot find proj.db" "ERROR 1: Failed to process SRS definition: EPSG:2154"

I have an other version of gdal from http://download.gisinternals.com/sdk/downloads/release-1911-x64-gdal-3-0-mapserver-7-4.zip in the directory d:\gdal30

I replace the system call "system(cmde, intern = TRUE)" by a small script, cf https://framagit.org/mgageo/osmdata/blob/master/scripts/ogr2ogr.R and it's good

[lbusett]

Hi @mgageo and @jgrn307 ,

I'd guess this is the same problem/related to what I just reported here:

r-spatial/discuss#31

, and that I was going to report also here.

HTH,

Lorenzo

[jgrn307]

If you run this line (from above) from Windows CMD line, what happens?

"C:\Program Files\QGIS 3.10\bin\ogr2ogr.exe" -t_srs "EPSG:2154" "rgdal.json" "rgdal.geojson"

[mgageo]

D:\tmp>"C:\Program Files\QGIS 3.10\bin\ogr2ogr.exe" -t_srs "EPSG:2154" "rgdal.json" "rgdal.geojson"
ERROR 1: PROJ: proj_create_from_wkt: Cannot find proj.db
ERROR 1: PROJ: proj_create_from_wkt: Cannot find proj.db
ERROR 1: PROJ: proj_create_from_database: Cannot find proj.db
ERROR 1: Failed to process SRS definition: EPSG:2154

D:\tmp>

[lbusett]

@mgageo @jgrn307

A way to fix the issue in r-spatial/discuss#31 is to reassign properly (or set in the first place if it is not set) the "PROJ_LIB" environment variable on the fly, before issuing any GDAL command that requires it. I implemented a possible fix (at the moment, only valid for gdalwarp), here:

lbusett@5d6a1ae

@mgageo

Could you try if it works also for you? You can do that by installing my fork from github:

remotes::install_github("lbusett/gdalUtils", ref = "feature/projlibpath")
library(gdalUtils)

and then trying commands:

src_dataset <- system.file("external/tahoe_highrez.tif", package="gdalUtils")
# Command-line gdalwarp call:
gdalwarp(src_dataset,dstfile="tahoe_highrez_utm11.tif",
		t_srs='+proj=utm +zone=11 +datum=WGS84',output_Raster=TRUE,
 		overwrite=TRUE,verbose=TRUE)

and see if it works.

If this seems ok/viable, the fix can be easily extended to other gdalUtils functions.

[jgrn307]

Ok, trying to get a TLDR here -- basically either rgdal or gdalUtils is causing the PROJ_LIB to change, which breaks some of the functionality?

1. Is this an accurate reading and, if so, which is changing the lib (rgdal, gdalUtils, something else)?
2. What is the TLDR fix you are proposing? Before the GDAL CMD is run, preserve the existing PROJ_LIB, run the gdal command,, and then set it back to the original PROJ_LIB?
3. Is this Windows specific?

If so, I think the simplest solution is to modify the GDAL command so this gets pre/post-appended to the command that is actually run (so I can universally change it in gdal_cmd_builder instead of having to tweak each command individually).

--j

[lbusett]

Hi @jgrn307 , I'll reply inline:

Ok, trying to get a TLDR here -- basically either rgdal or gdalUtils is causing the PROJ_LIB to change, which breaks some of the functionality?

1. Is this an accurate reading and, if so, which is changing the lib (rgdal, gdalUtils, something else)?

Yes, that is correct. The "culprit" here is rgdal, which namespace is imported by gdalUtils, so that as soon as the library is loaded (or even a function is called such as in gdalUtils::ogr2ogr(src_datasource_name,...), the environment variable is immediately modified (see https://github.com/cran/rgdal/blob/848067a0251323829dd776cf0789d1410cb77b3b/R/AAA.R#L14).

What is the TLDR fix you are proposing? Before the GDAL CMD is run, preserve the existing PROJ_LIB, run the gdal command,, and then set it back to the original PROJ_LIB?

The fix relies on introducing the new function set_projlibpath to do exactly that, using the on.exit mechanism to restore the previous value after each system call (to avoid breaking rgdal/sf functionality instead)

Is this Windows specific?

I encountered this only on Windows. It is due to the fact that at the moment the Windows rgdal/sf package binaries are linked to GDAL2/PROJ4 while the OSGEO installer/updater provides GDAL3/PROJ6 on new installs/updates. This leads gdalUtils to often rely on GDAL3 binaries, while rgdal forces the use of a PROJ_LIB folder related to a GDAL2 installation.
I do not work on MAC, so I do not know if it is also affected.
Consider however that the proposed solution, though currently being "applied" only on Windows OS, could be used with no problems on all OS, simply removing the Sys.info()[["sysname"]] == "Windows" in the new function.

If so, I think the simplest solution is to modify the GDAL command so this gets pre/post-appended to the command that is actually run (so I can universally change it in gdal_cmd_builder instead of having to tweak each command individually).

This is a good idea. However, consider that this would require moving also the system call within the gdal_cmd_builder function, instead than within the single gdalUtils functions (This because set_projlibpath needs to be called from within the function making the system call, in order to properly re-set the environment variable on exit - see https://yihui.org/en/2017/12/on-exit-parent/). I tested it, and it seems to work. I could prepare a PR if you wish so.

HTH,

Lorenzo

[mgageo]

[mgageo]

and after OK

[lbusett]

@mgageo

Thaks for testing. Could you try installing again from CRAN, test the following:

src_datasource_name <- system.file("external/tahoe_highrez_training.shp", package="gdalUtils")
dst_datasource_name <- paste(tempfile(),".shp",sep="")
# reproject the input to mercator
ogr2ogr(src_datasource_name,dst_datasource_name,t_srs="EPSG:3395",verbose=TRUE)

, and then re-test after reinstalling again from my fork on github?

[mgageo]

The version from CRAN

[mgageo]

After install from github

[lbusett]

Strange... it works for me...

could you please share your session info?

devtools::session_info()

[mgageo]

devtools::session_info() - Session info -------------------------------------------------------------------------------------------------------- setting value version R version 3.6.2 (2019-12-12) os Windows 10 x64 system x86_64, mingw32 ui RStudio language (EN) collate French_France.1252 ctype French_France.1252 tz Europe/Paris date 2020-01-23 - Packages ------------------------------------------------------------------------------------------------------------ package * version date lib source assertthat 0.2.1 2019-03-21 [1] CRAN (R 3.6.1) backports 1.1.5 2019-10-02 [1] CRAN (R 3.6.1) callr 3.4.0 2019-12-09 [1] CRAN (R 3.6.2) cli 2.0.1 2020-01-08 [1] CRAN (R 3.6.2) codetools 0.2-16 2018-12-24 [2] CRAN (R 3.6.2) crayon 1.3.4 2017-09-16 [1] CRAN (R 3.6.1) desc 1.2.0 2018-05-01 [1] CRAN (R 3.6.1) devtools 2.2.1 2019-09-24 [1] CRAN (R 3.6.1) digest 0.6.23 2019-11-23 [1] CRAN (R 3.6.2) ellipsis 0.3.0 2019-09-20 [1] CRAN (R 3.6.1) fansi 0.4.1 2020-01-08 [1] CRAN (R 3.6.2) foreach 1.4.7 2019-07-27 [1] CRAN (R 3.6.1) fs 1.3.1 2019-05-06 [1] CRAN (R 3.6.1) gdalUtils * 2.0.2.4 2020-01-23 [1] Github (lbusett/gdalUtils@709cc02) glue 1.3.1 2019-03-12 [1] CRAN (R 3.6.1) iterators 1.0.12 2019-07-26 [1] CRAN (R 3.6.1) lattice 0.20-38 2018-11-04 [2] CRAN (R 3.6.2) magrittr 1.5 2014-11-22 [1] CRAN (R 3.6.1) memoise 1.1.0 2017-04-21 [1] CRAN (R 3.6.1) pkgbuild 1.0.6 2019-10-09 [1] CRAN (R 3.6.2) pkgload 1.0.2 2018-10-29 [1] CRAN (R 3.6.1) prettyunits 1.1.0 2020-01-09 [1] CRAN (R 3.6.2) processx 3.4.1 2019-07-18 [1] CRAN (R 3.6.1) ps 1.3.0 2018-12-21 [1] CRAN (R 3.6.1) R.methodsS3 1.7.1 2016-02-16 [1] CRAN (R 3.6.0) R.oo 1.23.0 2019-11-03 [1] CRAN (R 3.6.1) R.utils 2.9.2 2019-12-08 [1] CRAN (R 3.6.1) R6 2.4.1 2019-11-12 [1] CRAN (R 3.6.2) raster 3.0-7 2019-09-24 [1] CRAN (R 3.6.1) Rcpp 1.0.3 2019-11-08 [1] CRAN (R 3.6.1) remotes 2.1.0 2019-06-24 [1] CRAN (R 3.6.1) rgdal 1.4-8 2019-11-27 [1] CRAN (R 3.6.1) rlang 0.4.2 2019-11-23 [1] CRAN (R 3.6.2) rprojroot 1.3-2 2018-01-03 [1] CRAN (R 3.6.1) rstudioapi 0.10 2019-03-19 [1] CRAN (R 3.6.1) sessioninfo 1.1.1 2018-11-05 [1] CRAN (R 3.6.1) sp 1.3-2 2019-11-07 [1] CRAN (R 3.6.1) testthat 2.3.1 2019-12-01 [1] CRAN (R 3.6.2) usethis 1.5.1 2019-07-04 [1] CRAN (R 3.6.1) withr 2.1.2 2018-03-15 [1] CRAN (R 3.6.1) [1] C:/Users/marcg/Documents/R/win-library/3.6 [2] C:/Program Files/R/R-3.6.2/library
--
 
| >

[lbusett]

@mgageo

seems that a reset of the r session is required. Could you retry after restarting the rsession/restarting rstudio, if you did not do that already?

[mgageo]

After a restart I have the popup but