-
Notifications
You must be signed in to change notification settings - Fork 0
SQL: Ephemeral Tables
Since replacing of original SQLite ephemeral tables with Tarantools' one is considered to be complicated enough, it was suggested to divide process into several stages.
The main goal of the first stage is to make prototype and estimate demanded efforts. In this regard, we will replace emphemeral tables only in one place and for one purpose. A good candidate is a prepared statement for
DELETE FROM tab WHERE, which uses emphemeral table to hold all entities to be deleted immediately before removing from ordinary table.
Basically we need to support insertion and selection from ephemeral tables, and creation/deletion ones.
This routine implies usage of IdxInsert, Rewind, Next and other opcodes, which call tarantoolSqlite3* functions. They are sort of layer between SQL and Tarantool and implemented using box API. However, emphemeral tables are incapable of using box API, mainly due to the lack of space_id. Thus, first task will be an implementation of tarantoolSqlite3Ephemeral* functions. In addition, SQL cursors should be adapted for ephemeral tables, since they need to hold struct space* and appropriate flag, in order to call tarantoolSqlite3Ephemeral* functions. Finally, implement OP_OpenTEphemeral opcode which would summarize routine described above.
To be more precise, an approximate plan for the first stage is:
-
Implement
tarantoolSqlite3API for ephemeral tables:tarantoolSqlite3EmphemeralCreate(),tarantoolSqlite3EphemeralInsert(),tarantoolSqlite3EphemeralDrop()
Subtasks:
- Deal with inner parts of Tarantool. Understand how to correctly create tables, how box API was implemented.
- Attempt to create Ephemeral API in the similar way, except for using
space_id.
Estimated time: 2-3 days.
- Expand SQL cursors in order to work with ephemeral tables:
they should store struct space* and should be handled a little bit
different in VDBE and
tarantoolSqlite3interface. For example, functions such ascursor_seek(),cursor_advance()use box iterators to navigate through the space. As far as we can't manipulate cursors to emphemeral tables via box API, these function should be reimplemented/extented.
Subtasks:
- Patch VDBE opcodes
IdxInsert,Rewind,Nextand other. - Rewrite
cursor_*funcions, which in its turn requiers to rewritebox_index_iterator.
Estimated time: 3+ days.
-
Implement
OP_OpenTEphemeralopcode which would create ephemeral space with given number of fields and set up cursor to it.- This part mainly consists from combining steps 1 and 2.
Estimated time: 1 day.
-
Expand all other opcodes which work with cursors to ephemeral tables. For instance, after cursor to ephemeral table closes, this table should be deleted with
space_delete().Estimated time: 1-2 days
Second stage is aimed to complete replacing of original OP_OpenEmpemeral with OP_OpenTEphemeral and removing code which is connected with rowid.
Architecture Specifications
- Server architecture
- Feature specifications
- What's in a good specification
- Functional indexes
- Space _index structure
- R tree index quick start and usage
- LuaJIT
- Vinyl
- SQL
- Testing
- Performance
How To ...?
- ... add new fuzzers
- ... build RPM or Deb package using packpack
- ... calculate memory size
- ... debug core dump of stripped tarantool
- ... debug core from different OS
- ... debug Lua state with GDB
- ... generate new bootstrap snapshot
- ... use Address Sanitizer
- ... collect a coredump
Lua modules
Useful links