Releases: dolthub/doltgresql
Releases · dolthub/doltgresql
0.9.0
Merged PRs
doltgresql
- 368: /.github/workflows/cd-release.yaml: fix sed command for server version
- 365: add smoke tests for perl, php and ruby
- 362: Perform function analysis once
Functions did their entire overload resolution duringEval, meaning if a function was called 1000 times, then we did overload resolution 1000 times for the exact same function. This changes it so that we only do it once. This optimization was made while investigating our performance benchmarks.
Of note, errors found during overload resolution are stashed for later. This ensures that other steps in the pipeline retain their error priority. Previously, when this was all done inEval, stashing wasn't needed. - 355: Added correct timeouts to server
Fixes #348
Idle connections were timing out in 60s before this change. - 350: Reworked config
This change reworks how the data dir, config file and related initialization values are loaded. The main effect is that it's now possible to rundoltgreswithout a config.yaml file.
The sever config is now resolved in this order- --config flag
- config.yaml file if present
- Built-in default config values
The data-dir for this config is then overridden as necessary with a similar pattern: - --data-dir flag
- The value in a config.yaml file if, if a file was loaded
- The env var
DOLTGRES_DATA_DIR - The default ($home/doltgres/databases)
- 343: support for drop table without a schema name
This PR has the tests. The fix for the behavior is in dolthub/dolt#7952 - 339: Release v0.8.0
Created by the Release workflow to update DoltgreSQL's version
Closed Issues
- 348: disabling the session after executing the request
0.8.0
Merged PRs
doltgresql
- 336: Added string literal to array interpretation
This adds functionality so that string literals such as'{1,2,3}'can be interpreted as arrays of any type. This builds off of the #333 PR, which enables this to "just work" for all contexts (includes casting, inserting into columns, automatic operator resolution, etc.). This also includes additional tests for a few bugs that weren't caught in the previous PR. - 333: Removed sql.Convert usage, moved everything to casts
This removes our reliance onsql.Convertand changed everything to make use of casts. Casts have three contexts: explicit (value::type), assignment (INSERTandUPDATEstatements), and implicit. In addition to moving to casts, I've also added assignment casts.
This move also comes with a massive change to how function overloads are handled. Overload resolution is handled through available casts, so by removing the hacky behavior that was in place, we're now able to unskip a number of tests that just could not be handled through the previous logic. There were other tests that were also incorrect, and those have been fixed.
Overall, this has resulted in a fairly substantial decrease in code throughout the codebase, in addition to allowing for types to finally have "full" support.
Companion PR: - 325: Update get-doltgres-correctness-job-json.sh
- 320: Update schema path in scripts
- 319: Added comparisons for JSONB
This adds comparisons forjsonb. In the original JSON PR, I mistakenly only checked ifjsonhad comparisons, and that type does not.jsonb, however, does have comparisons, so this implements those. In addition, I've added a massive collection of rows that it properly sorts over. Test results were lifted straight from a Postgres 15 server to ensure accuracy. - 317: /utils/doltgres_builder/run.go: fix doltgres builder paths
- 302: Added JSON types
This adds JSON types, and also fixes a bug where the text types would incorrectly persist an empty string as a NULL. - 288: Release v0.7.5
Created by the Release workflow to update DoltgreSQL's version - 287: Added SERIAL types along with remaining SEQUENCE functionality
Companion PRs:- dolthub/vitess#347
- dolthub/go-mysql-server#2504
This adds theSERIALtypes, along with the remaining functionality for sequences that was not present in the first PR. In addition, this adds our first uses of injected analyzer steps, and introduces custom nodes used for the sequence statements (instead of stored procedures).
- 269: New tests for schema support
Also changes expected errors to match an error string. Not all tests are passing yet. - 185: Update README.md
Closed Issues
0.7.5
Merged PRs
doltgresql
- 283: scripts/build_binaries.sh: Fix regression where we accidentally shipped all linux/amd64 binaries instead of platform-specific ones.
- 276: move config to its own package and update dolt servercfg package
- 273:
EXECUTEdoesn't send RowDescription
This PR fixes issue connecting to doltgres server using Postgres JDBC driver.
In Postgres docs, it says, "Executedoesn't cause ReadyForQuery or RowDescription to be issued."
Also, queries that don't return set of rows sendNoDatain response toDescribemessage. - 266: Release v0.7.4
Created by the Release workflow to update DoltgreSQL's version - 255: only some queries send
RowDescriptionmessage- Only queries that returns set of rows should send
RowDescriptionmessage. This includesSELECT,FETCH,SHOW, etc. - The
CommandTagis now set by theStatementTagfrom the parsed query. - Empty query should send
EmptyQueryResponsemessage.
- Only queries that returns set of rows should send
- 219: Adds CREATE SEQUENCE and DROP SEQUENCE
Companion PR:- dolthub/dolt#7848
This adds functionality forCREATE SEQUENCEandDROP SEQUENCE. For sequences, this is what is in the PR: - MINVALUE
- MAXVALUE
- INCREMENT
- AS DATA TYPE
- START WITH
- CYCLE
This is what is missing: - TEMPORARY
- UNLOGGED
- CACHE
- OWNED BY
- SERIAL
- Merging
- Table Querying
Of what's missing, merging andSERIALare the main additions, withOWNED BYrequired for properSERIALsupport. Table querying is fairly straightforward (thepg_sequencessystem table, etc.). The others are, hopefully, unnecessary for now and can be postponed, although perhaps we can save theCACHEvalue and ignore it since it's primarily a performance-oriented option.
With that,SERIALand merging will come in the next PR, since this is already very large. Also of note, theWriterandReaderutilities were implemented for my Convert-to-Cast PR but they were useful here, so I pulled them from that in-flight PR and added them here.
- dolthub/dolt#7848
Closed Issues
- 240: Returned result differs from Postgres
0.7.4
0.7.1
Merged PRs
doltgresql
- 261: add the minver_validation.txt file to the release commit
Does the same thing as: https://github.com/dolthub/dolt/blob/main/.github/workflows/cd-release-pgo.yaml#L79 - 260: Release v0.7.0
Created by the Release workflow to update DoltgreSQL's version
Closed Issues
0.7.0
Merged PRs
doltgresql
- 259: build parser earlier in release process
The parser needs to be built beforego run -mod=readonly ./utils/genminver_validation/ $FILE - 258: bump go version to 1.22.1
- 253: /.github/workflows/bump-dependency.yaml: use personal tokens to open prs to ensure ci runs
- 245: Fixes LIMIT
Fixes #172 - 244: Fixed replication tests
- 232: Added Doltgres-implementation of RootValue
Companion:- dolthub/dolt#7829
This is Doltgres' implementation ofRootValue. For now, it's essentially taken wholesale from Dolt as a starting point. Of note, we probably want to at least get something different for our collation values sometime soon. Also, we're using the same serial identifier for root values between Doltgres and Dolt. This simplifies a good bit of code in Dolt, so taking on the naming requirement seems well worth it.
- dolthub/dolt#7829
- 231: kill subcommands
- 226: Add better cast tests for
name, fix bool -> name cast - 221: move main
Move the main package to cmd/doltgres so that the source built binary matches the released binary. - 214: scripts/build_binaries.sh: Build releases with CGO enabled.
- 210: Implement
xidtype - 204: Implement oid type
- 202: Implement
nametype - 191: Fix typo and go test command in docs
- 188: Added website and documentation site to README
- 184: /.github/markdown-templates/dep-bump.md: add md template
- 183: use personal token so release pr runs CI
- 182: Added explicit initialization order to packages
This replaces all of theinit()functions throughout the project with a package-levelInit()function that is called from a newinitializationpackage. This now gives an explicit ordering to the calls. The biggest disadvantage of this approach is that some packages cannot have package-level tests that can access package-private variables without potentially causing import cycles if they need to initialize another package. This was also a limitation with the previousinit()approach as well, but the vast majority of our tests are done from thetestingsubdirectories (in part due to this limitation), so we've not lost anything by changing to this approach. - 181: Support for schemas
Things that work:Create schema- Explicit schema name qualifiers in statements, with and without database name qualifiers
Things that don't work: - Foreign keys
- Search path support for resolving table names
- Automatically creating a
publicschema for new databases - Revision name qualifiers (branch name DBs)
- 180: Update bump-dependency.yaml
- 179: Update bump-dependency.yaml
- 178: Revert "/.github/workflows/cd-release.yaml: revert bump of dolthub actions"
Reverts #176 as these actions have been updated and work now - 177: Release v0.6.0
Created by the Release workflow to update DoltgreSQL's version - 168: support
CREATE VIEW
Depends on:
Closed Issues
- 172: The LIMIT expression is not working
0.6.0
Merged PRs
doltgresql
- 176: /.github/workflows/cd-release.yaml: revert bump of dolthub actions
- 174: Db/bump actions
- 169: doltgresql version --dolt
- 167: Added more types, reworked cast framework
This adds several new types, and also reworks type serialization and casting, among a few other things. This is not nearly as tested as I'd want it to be, as I discovered that the implicit/explicit casting is fundamentally incompatible with how we usesql.Type.Convert, so that entire system needs to be "removed" for Doltgres (that is, it functionally should not do anything).
In addition, in an effort to be a little forward-thinking since I'm assuming we'll be breaking things a bit once the conversion system is removed, I added a form of type versions so that we can update types as we discover deficiencies. For example, let's say a type uses-1to mean "no limit", and then we later find out that-1is a valid value. We can make that distinction using type versions. In addition, this will break all existing databases due to the type serialization change, but I think it's fine as all currently known users should be able to rebuild the database since their replicas. - 166: add parsing support for
OUTandINOUTargmode for routines
Aggregate only allowsINandVARIADICargument modes. Routines including functions and procedures allowsIN,VARIADIC,OUTandINOUTmodes. The parser will allowOUTandINOUTfor aggregates, soTODOthere needs a check during ast evaluation/conversion for these modes in aggregate statements.- small fix for supporting
SETOFmodifier option forCREATE FUNCTIONstatement. - small fix for supporting
db.aggregate_namesyntax forAGGREGATEstatements.
- small fix for supporting
- 165: Removed inline VarChar implementation
This removes the inline VarChar variable. It's not used anywhere, so it's best to remove it before we use it in a location that we don't mean to. We still retain the inline characteristics for user-defined columns that define a limit below the inline max though. - 164: Reworked array serialization format
This is building off of the following PR comment:
#162 (comment)
In Postgres, there seems to be a field size limit of 1GB, so I've used that to determine that we can just useuint32s everywhere, rather than having two formats withuint16anduint64(which were attempts at conserving space). - 163: use auto_increment for SERIAL type
This PR reverts the latest change on using auto_increment for SERIAL type column definition.
Note: INSERT on SERIAL type as primary key does not work for now. - 162: All Array Types
This PR adds array types for all existing types. This is done by providing a wrapper that handles all of the array-functionality over the base type. In addition, this removes the mock array functionality that only worked to demonstrate the viability ofboolean[], replacing the functionality with fully working equivalents. - 161: remove usage of auto_increment for SERIAL type for column definition
- 159: small fix for
SETsyntax accepting multiple values as a single value - 158: support
COMMENTstatement syntax - 157: Release v0.5.0
Created by the Release workflow to update DoltgreSQL's version - 156: Use go 1.22 for release process
- 152: support
AGGREGATEstatement syntaxes
This PR adds support forCREATE AGGREGATEandDROP AGGREGATEstatements syntaxes.
Closed Issues
0.5.0
Merged PRs
doltgresql
- 152: support
AGGREGATEstatement syntaxes
This PR adds support forCREATE AGGREGATEandDROP AGGREGATEstatements syntaxes. - 151: /{go.mod,.github}: bump go version
- 150: support
TYPEandDOMAINstatements syntaxes
This PR adds support forCREATE TYPE,ALTER TYPE,CREATE DOMAIN,ALTER DOMAINANDDROP DOMAINstatement syntaxes. - 149: support
LANGUAGEsyntaxes
This PR adds support forCREATE,ALTERandDROPLANGUAGEstatements syntaxes. - 148: Replication options for config.yaml
Also fixed an issue where the --config option wasn't working (looking in the wrong directory). - 147: Added unary and binary operator functions
This adds all of the internal functions for unary and binary operators (for types that are currently implemented). - 145: support all
SEQUENCEstatements syntax
This PR adds complete support for parsingSEQUENCEsyntaxes. - 144: INT4 ORDER BY Workaround
There are two bugs here. The first is thatINTEGERandINTwere parsing asINT8. The second is thatORDER BY 1was failing for tables that had anINT4but not anINT8. For the first, our parser simply used the wrong bit width, which is fairly straightforward.
The second bug was far trickier to track down. Essentially,ORDER BY 1has been broken, however some combinations of tables and rows would result in the correct order on accident, and that's what led to the confusion on how the second bug worked. It's even more coincidental that the literal1inORDER BY 1happened to correspond to the first value in the affected tests, further masking the issue.
Fundamentally, the issue has not been fixed, and this submits a workaround. The root of the issue is that GMS expects thevitess.SQLValtype, and does not handle our case (Postgres literals). As a workaround, we're converting our Postgres literals tovitess.SQLValspecifically for this exact case. The real fix would be for GMS to defer to Doltgres when encountering types that it cannot hardcode (or to un-hardcode types, which seems relatively bad for GMS just to work better with Doltgres). We do not have a standard way of implementing this differing interface yet, so we'll punt on that and just get this workaround in. - 143: Support configuration parameters handling
This PR includes changes for:- The complete list of configuration parameters is in.
- All of the postgres configuration parameters will be added to existing system variables list in GMS. There are two parameters that have the same name as mysql system variables, which will be overwritten by the postgres parameter. This is temporary until we completely separate the use of them.
- Current implementation of
sql.SystemVariableRegistryinterface is for Dolt only, but we add all the postgres configuration parameters to theSystemVariablesglobal variable in GMS. This should be updated to handle Doltgres config parameters separately in future. - The default and only supported scope of postgres parameters is
SESSION, which is used asSESSIONscope in GMS. There is no GLOBAL scope inpostgresto be set during a session.
Depends on GMS PR: dolthub/go-mysql-server#2375
Depends on Dolt PR: dolthub/dolt#7579
Next up: - need more support for
Parametersuch as accept setting memory and time values that has unit information, etc. - need privileges and role checking for specific parameters.
- support setting parameter defaults from configuration file.
- `LOCAL scope support.
- 142: Added a plethora of function tests
This adds tests for most of the functions, along with the generation code that grabs results from a running PostgreSQL 15 to create tests with. - 141: added nil check in
Additionexpression- new Addition expression can be used without children expressions defined
Note: majority of the tests regressed because of not being able to check doltgres-specific types in GMS.
E.g: doltgrestypes.Int64Typeis not recognized asNumberTypeImpl_in GMS.
- new Addition expression can be used without children expressions defined
- 140: Tests and fixes for more replication scenarios
- 139: add
TRIGGERstatements parsing
Adds syntax parsing support for:CREATE TRIGGERALTER TRIGGERDROP TRIGGER
- 138: add
VIEWandMATERIALIZED VIEWstatements parsing
Adds syntax parsing support for:CREATE VIEWCREATE MATERIALIZED VIEWALTER VIEWALTER MATERIALIZED VIEW
- 137: More replication tests
Tests discovered and fixed several issues from first draft, mostly involving stopping and resuming replication and many races in testing and the replicator itself.
Next step: error handling tests using toxiproxy
Final step: UX and docs - 136: support
PROCEDUREstatements syntax parsing
This PR includes syntax parsing support for:CREATE PROCEDUREDROP PROCEDUREALTER PROCEDUREALTER FUNCTION
- 135: support parsing
ALTER INDEXstmt
This PR also fixes all the synopsis that had[ $argmode$ ] [ $argname$ ] $argtype$ [ , ... ]to correct format of{ [ $argmode$ ] [ $argname$ ] $argtype$ } [ , ... ]. The tests that are affected by this change is updated. - 134: Update update-perf.sh
- 133: Function Overload Rework
What this adds:- Type resolution for all extended (Doltgres) types
- Proper type casting between types (extendable to support user-defined conversion)
- Additional Types
The previous function overload framework only worked in a very limited capacity. This PR intends to remove those limitations, allowing for all current (and future) Postgres functions to be defined. In general, this is also designed to support user-defined functionality, or at least designed in such a way that it will be relatively easy to fully implement it later on.
Related PR: dolthub/dolt#7537
- 132: New test runner
Test runner for docker-based replication tests - 131: Docker test for postgres replication
Seeking some feedback on this approach. This works locally for me on my local Docker when I build the image from the package root:Next step would be to create a github action that runs this docker file and a workflow trigger for that action. Is this going to work? I looked into the examples in the Dolt repo and got confused by the paths that github actions appears to use at execution time, so could use some advice on best practices.docker build --file=testing/ReplicationTestDockerfile . - 130: support for parsing
CREATE TABLEandALTER TABLEstatement syntax
This PR adds support for parsingCREATE TABLEandALTER TABLEstatements
It also removes some parsing code for statements that are not supported in Postgres. - 129: replication POC and tests
Looking for feedback on this approach before polishing it off for submission.
Missing / next steps:- Keyless table tests
- Start up postgres during CI so the replication tests can run correctly there
- UX to configure and start up replication for the server
- 128: [auto-update-readme] by coffeegoddd
- 127: [auto-update-readme] by coffeegoddd
- 126: Release v0.4.0
Created by the Release workflow to update DoltgreSQL's version
Closed Issues
- 146: someone is trolling doltgresql stars and sending unsollicited emails
0.4.0
Merged PRs
doltgresql
- 125: Added UUID type
This adds the UUID type. Ignore the serialization portion for now, I'll figure out a different serialization scheme later that should work for all types (including user-defined types). While this works, we need to forward the type information through to the connector so it can use the correct OID. For now though, this should be enough to unblock. - 123: Fixed command docs generation
Sometimes, command docs would generate duplicate tests. This was due to the RNG selecting the same number multiple times. This fixes this issue, along with fixing all of the affected tests. There are still duplicate tests, but that is due to the synopses defining the duplicates, rather than the generation, which I think is fine. - 122: support syntax parsing for INDEX statements
This PR adds syntax parsing support for:CREATE INDEXCREATE TABLE AS
It also splitspostgres/parser/sem/tree/create.gofile intocreate_*.gofiles for each specific statements.
- 121: Referencing columns within arrays
This makes use of an addition to Vitess and GMS that allows for name resolution to occur for injected expressions. The newARRAY[...]smoke test demonstrates this.
Related PRs: - 120: Added BOOLEAN and BOOLEAN[] types
This adds rough support for bothBOOLEANandBOOLEAN[]types. This is primarily to show a working implementation of the newExtendedTypeinterface that was added to GMS to support these new types, demonstrating that we can now add standard PostgreSQL types, as well as array types (along with accompanying functionality, such asARRAY[]support).
Related PRs: - 119: support parsing create extension and function statements
Added parsing support to following statements:CREATE EXTENSIONCREATE FUNCTIONCOMMENT ON EXTENSIONDROP EXTENSIONDROP FUNCTION
- 118: [auto-update-readme] by coffeegoddd
- 117: [auto-update-readme] by coffeegoddd
- 115: Update get-doltgres-correctness-job-json.sh
- 114: /testing/logictest/harness/doltgres_server_harness.go: catch error
- 113: /README.md: changes for automated perf updates
- 112: /.github/{scripts,workflows}: add script for updating readme, fix other workflows
- 109: /.github/workflows: enable nightly correctness, add latency on pull comment
- 108: /.github/{scripts,workflows}: add correctness workflows
- 107: /.github/scripts/performance-benchmarking/get-{postgres,doltgres}-doltgres-job-json.sh: set email template
- 106: support ddl for schema and database
This PR adds missing postgres syntax for:CREATE SCHEMACREATE DATABASEDROP DATABASE
- 105: /.github/workflows: skip tpcc, add nightly benchmarks and benchmarks on release
- 104: /.github/workflows/email-report.yaml: fix email report workflow
- 102: Db/fix ci
- 100: /.github/{actions,scripts,workflows}: add benchmarking workflows to ci
- 99: Better error handling
Server no longer always forcefully closes a connection when catching a panic, which is still very common because of unsupported features that panic, as well as unintended panics due to bugs.
This change also refactors the connection handling logic to extract a new ConnectionHandler type.
Also fixes a problem where a errors during a Query execution would hang a connection while it waited for a Sync message that never came. AddedwaitForSyncflag in ConnectionHandler to manage this bit of state. - 96: Updated contribution guide
- 95: Better prepared statement support
Now handles type casts for placeholders, as well as matching a larger variety of expressions.
Companion PR:
dolthub/go-mysql-server#2272 - 93: PostgreSQL function PR feedback
- 91: Added more PostgreSQL functions
This only includes quite a few functions, but doesn't include their tests. I was planning on including those too, but it's turning into a "more work than expected" type of situation, and if I'm going to solve those problems, I'd rather do it with all of the rest of the functions too, so that I can just get all the functions done for all the tests as once. This implements every function that I think we can implement for now based on our currently-supported types (without ripping out the implementation as soon as we get proper type support).
In the meantime, we can get these functions in, and get the tests in later. - 89: add utility for building doltgres binaries
- 87: Prepared statement support
This PR implements postgres prepared statements with support for$1style parameters in query strings. If this approach is reasonable, the next step would be to add support for additional types and lots more tests.
Also missing is support for the client manually specifying the types of parameters in aParsemessage, which is required for certain statements where the type information cannot be determined based on referenced columns.
Companion PRs:
dolthub/go-mysql-server#2239
dolthub/vitess#299 - 86: support postgres syntaxes - 2
This PR includes some postgres-specific syntax support.ALTER DATABASEGRANTREVOKEALTER DEFAULT PRIVILEGES
- 85: support some postgres syntaxes - 1
This PR includes some postgres-specific syntax support.ABORTALTER AGGREGATEALTER COLLATIONALTER CONVERSION
- 84: Release v0.3.1
Created by the Release workflow to update DoltgreSQL's version - 83: starting doltgres server creates db in defined directory
Currently, doltgres uses dolt init when starting doltgres server with data dir set to empty directory. When there is no dolt or doltgres configuser.nameanduser.email, it fails to create a new database with error message fromdolt init, which suggests to setdoltuser info. Current fix will not return error; instead, it will use user name,postgresand email,postgres@somewhere.comif it's not set. - 82: Added a framework for creating PostgreSQL functions
This is the 2nd version of my attempt at implementing functions.
The first iteration hijacked*tree.FuncExprand special-cased names of functions that matched the built-in ones for Postgres. Then it returned a new node that approximated the functionality. In some cases this was as simple as returning a different*vitess.FuncExprnode that pointed to a different function, however some cases did not have direct analogues in MySQL. Originally I was going to ignore those, but since I was working on functions anyway, I decided to tackle them to get it over with, similar to my approach with the entire AST conversion. Well that's when I started running into two key issues:- No straightforward conversion for some functions
- Different behavior for similar functions
The second was born from the first, as I needed to extend my tests to make sure my massive nodes actually worked for most cases. However, extending the tests showed some issues that are somewhat fundamentally different to how MySQL approaches functions, with a big one being overloading.
The first was a major issue though. I'll post an example in another comment, but some of the nodes became almost unreadable, and they also took forever to create (multiple hours per function, hence the continual delays). The only real maintainable alternative would be to skip the AST conversion pipeline and jump straight to the GMS expressions, but that posed its own issues. Not only is there a lot of boilerplate for expressions, but some of the more sophisticated aspects (such as the origin of values, which I'm calling theirSource) are specific to PostgreSQL, So I instead decided to modify the entire set of functions by replacing all of the built-ins (Postgres doesn't want MySQL's built-ins anyway) with a custom "compiled" function structure.
So now, this mimics a bit how overloaded stored procedures work. We define a set of functions under a single name, and those functions become overloads for that name. Whenev...
0.3.1
Merged PRs
doltgresql
- 83: starting doltgres server creates db in defined directory
Currently, doltgres uses dolt init when starting doltgres server with data dir set to empty directory. When there is no dolt or doltgres configuser.nameanduser.email, it fails to create a new database with error message fromdolt init, which suggests to setdoltuser info. Current fix will not return error; instead, it will use user name,postgresand email,postgres@somewhere.comif it's not set. - 80: Release v0.3.0
Created by the Release workflow to update DoltgreSQL's version
Closed Issues
- 81: cannot run
doltgreswith data dir set to empty directory