Graphics protocol: Add support for having the terminal emulator assign image ids

Useful when multiple non co-operating programs want to share the screen.
Fixes #3163

Author: kovidgoyal

docs/changelog.rst

@@ -16,8 +16,10 @@ To update |kitty|, :doc:`follow the instructions <binary>`.
 - Allow specifying text formatting in :opt:`tab_title_template` (:iss:`3146`)
 
 - Graphics protocol: Add support for giving individual image placements their
-  own ids. This is a backwards compatible protocol extension, and also allow
-  suppressing responses from the terminal to commands (:iss:`3133`)
+  own ids and for asking the terminal emulator to assign ids for images. Also
+  allow suppressing responses from the terminal to commands.
+  These are backwards compatible protocol extensions. (:iss:`3133`,
+  :iss:`3163`)
 
 - Distribute extra pixels among all eight-blocks rather than adding them
   all to the last block (:iss:`3097`)

docs/graphics-protocol.rst

@@ -359,9 +359,8 @@ second one will replace the first. This can be used to resize or move
 placements around the screen, without flicker.
 
 
-.. note:: Support for specifying placement ids was added to kitty in
-   versions after 0.19.2. You can use the protocol documented in the
-   :doc:`kittens/query_terminal` to query kitty version.
+.. versionadded:: 0.19.3
+   Support for specifying placement ids (see :doc:`kittens/query_terminal` to query kitty version)
 
 
 Controlling displayed image layout
@@ -415,7 +414,9 @@ Value of ``d``       Meaning
 =================    ============
 ``a`` or ``A``       Delete all placements visible on screen
 ``i`` or ``I``       Delete all images with the specified id, specified using the ``i`` key. If you specify a ``p`` key for the placement                      id as well, then only the placement with the specified image id and placement id will be deleted.
-                     placement id
+``n`` or ``N``       Delete newest image with the specified number, specified using the ``I`` key. If you specify a ``p`` key for the
+                     placement id as well, then only the placement with the specified number and placement id will be deleted.
+``c`` or ``C``       Delete all placements that intersect with the current cursor position.
 ``c`` or ``C``       Delete all placements that intersect with the current cursor position.
 ``p`` or ``P``       Delete all placements that intersect a specific cell, the cell is specified using the ``x`` and ``y`` keys
 ``q`` or ``Q``       Delete all placements that intersect a specific cell having a specific z-index. The cell and z-index is specified using the ``x``, ``y`` and ``z`` keys.
@@ -447,7 +448,40 @@ script, it might be useful to avoid having to process responses from the
 terminal. For this, you can use the ``q`` key. Set it to ``1`` to suppress
 ``OK`` responses and to ``2`` to suppress failure responses.
 
-.. note:: This feature was implemented in kitty in versions after 0.19.2
+.. versionadded:: 0.19.3
+   The ability to suppress responses (see :doc:`kittens/query_terminal` to query kitty version)
+
+
+Requesting image ids from the terminal
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If you are writing a program that is going to share the screen with other
+programs and you still want to use image ids, it is not possible to know
+what image ids are free to use. In this case, instead of using the ``i``
+key to specify and image id use the ``I`` key to specify and image number
+instead. These numbers are not unique.
+When creating a new image, even if an existing image has the same number a new
+one is created. And the terminal will reply with the id of the newly created
+image. For example, when creating an image with ``I=13``, the terminal will
+send the response::
+
+    <ESC>_Gi=99,I=13;OK<ESC>\
+
+Here, the value of ``i`` is the id for the newly created image and the value of
+``I`` is the same as was sent in the creation command.
+
+All future commands that refer to images using the image number, such as
+creating placements or deleting images, will act on only the newest image with
+that number. This allows the client program to send a bunch of commands dealing
+with an image by image number without waiting for a response from the terminal
+with the image id. Once such a response is received, the client program should
+use the ``i`` key with the image id for all future communication.
+
+.. note:: Specifying both ``i`` and ``I`` keys in any command is an error. The
+   terminal must reply with an EINVAL error message, unless silenced.
+
+.. versionadded:: 0.19.3
+   The ability to use image numbers (see :doc:`kittens/query_terminal` to query kitty version)
 
 
 Image persistence and storage quotas

gen-apc-parsers.py

@@ -257,6 +257,7 @@ def graphics_parser() -> None:
         'f': ('format', 'uint'),
         'm': ('more', 'uint'),
         'i': ('id', 'uint'),
+        'I': ('image_number', 'uint'),
         'p': ('placement_id', 'uint'),
         'q': ('quiet', 'uint'),
         'w': ('width', 'uint'),

kitty/graphics.c

@@ -92,6 +92,16 @@ img_by_client_id(GraphicsManager *self, uint32_t id) {
     return NULL;
 }
 
+static inline Image*
+img_by_client_number(GraphicsManager *self, uint32_t number) {
+    // get the newest image with the specified number
+    for (size_t i = self->image_count; i-- > 0; ) {
+        if (self->images[i].client_number == number) return self->images + i;
+    }
+    return NULL;
+}
+
+
 static inline void
 remove_image(GraphicsManager *self, size_t idx) {
     free_image(self, self->images + idx);
@@ -306,6 +316,33 @@ find_or_create_image(GraphicsManager *self, uint32_t id, bool *existing) {
     return ans;
 }
 
+static int
+cmp_client_ids(const void* a, const void* b) {
+    const uint32_t *x = a, *y = b;
+    return *x - *y;
+}
+
+static inline uint32_t
+get_free_client_id(const GraphicsManager *self) {
+    if (!self->image_count) return 1;
+    uint32_t *client_ids = malloc(sizeof(uint32_t) * self->image_count);
+    size_t count = 0;
+    for (size_t i = 0; i < self->image_count; i++) {
+        Image *q = self->images + i;
+        if (q->client_id) client_ids[count++] = q->client_id;
+    }
+    if (!count) { free(client_ids); return 1; }
+    qsort(client_ids, count, sizeof(uint32_t), cmp_client_ids);
+    uint32_t prev_id = 0, ans = 1;
+    for (size_t i = 0; i < count; i++) {
+        if (client_ids[i] == prev_id) continue;
+        prev_id = client_ids[i];
+        if (client_ids[i] != ans) break;
+        ans = client_ids[i] + 1;
+    }
+    free(client_ids);
+    return ans;
+}
 
 static Image*
 handle_add_command(GraphicsManager *self, const GraphicsCommand *g, const uint8_t *payload, bool *is_dirty, uint32_t iid) {
@@ -333,6 +370,11 @@ handle_add_command(GraphicsManager *self, const GraphicsCommand *g, const uint8_
         } else {
             img->internal_id = internal_id_counter++;
             img->client_id = iid;
+            img->client_number = g->image_number;
+            if (!img->client_id && img->client_number) {
+                img->client_id = get_free_client_id(self);
+                self->last_init_graphics_command.id = img->client_id;
+            }
         }
         img->atime = monotonic(); img->used_storage = 0;
         img->width = g->data_width; img->height = g->data_height;
@@ -472,7 +514,7 @@ handle_add_command(GraphicsManager *self, const GraphicsCommand *g, const uint8_
 }
 
 static inline const char*
-finish_command_response(const GraphicsCommand *g, bool data_loaded, uint32_t iid, uint32_t placement_id) {
+finish_command_response(const GraphicsCommand *g, bool data_loaded, uint32_t iid, uint32_t placement_id, uint32_t image_number) {
     static char rbuf[sizeof(command_response)/sizeof(command_response[0]) + 128];
     bool is_ok_response = !command_response[0];
     if (g->quiet) {
@@ -483,9 +525,14 @@ finish_command_response(const GraphicsCommand *g, bool data_loaded, uint32_t iid
             if (!data_loaded) return NULL;
             snprintf(command_response, 10, "OK");
         }
-        if (placement_id) snprintf(rbuf, sizeof(rbuf)/sizeof(rbuf[0]) - 1, "Gi=%u,p=%u;%s", iid, placement_id, command_response);
-        else snprintf(rbuf, sizeof(rbuf)/sizeof(rbuf[0]) - 1, "Gi=%u;%s", iid, command_response);
+        size_t pos = 0;
+#define print(fmt, ...) pos += snprintf(rbuf + pos, arraysz(rbuf) - 1 - pos, fmt, __VA_ARGS__)
+        print("Gi=%u", iid);
+        if (image_number) print(",I=%u", image_number);
+        if (placement_id) print(",p=%u", placement_id);
+        print(";%s", command_response);
         return rbuf;
+#undef print
     }
     return NULL;
 }
@@ -521,11 +568,14 @@ update_dest_rect(ImageRef *ref, uint32_t num_cols, uint32_t num_rows, CellPixelS
 }
 
 
-static void
+static uint32_t
 handle_put_command(GraphicsManager *self, const GraphicsCommand *g, Cursor *c, bool *is_dirty, Image *img, CellPixelSize cell) {
-    if (img == NULL) img = img_by_client_id(self, g->id);
-    if (img == NULL) { set_command_failed_response("ENOENT", "Put command refers to non-existent image with id: %u", g->id); return; }
-    if (!img->data_loaded) { set_command_failed_response("ENOENT", "Put command refers to image with id: %u that could not load its data", g->id); return; }
+    if (img == NULL) {
+        if (g->id) img = img_by_client_id(self, g->id);
+        else if (g->image_number) img = img_by_client_number(self, g->image_number);
+        if (img == NULL) { set_command_failed_response("ENOENT", "Put command refers to non-existent image with id: %u and number: %u", g->id, g->image_number); return 0; }
+    }
+    if (!img->data_loaded) { set_command_failed_response("ENOENT", "Put command refers to image with id: %u that could not load its data", g->id); return 0; }
     ensure_space_for(img, refs, ImageRef, img->refcnt + 1, refcap, 16, true);
     *is_dirty = true;
     self->layers_dirty = true;
@@ -556,6 +606,7 @@ handle_put_command(GraphicsManager *self, const GraphicsCommand *g, Cursor *c, b
     update_dest_rect(ref, g->num_cells, g->num_lines, cell);
     // Move the cursor, the screen will take care of ensuring it is in bounds
     c->x += ref->effective_num_cols; c->y += ref->effective_num_rows - 1;
+    return img->client_id;
 }
 
 static int
@@ -649,20 +700,24 @@ grman_update_layers(GraphicsManager *self, unsigned int scrolled_by, float scree
 // Image lifetime/scrolling {{{
 
 static inline void
-filter_refs(GraphicsManager *self, const void* data, bool free_images, bool (*filter_func)(const ImageRef*, Image*, const void*, CellPixelSize), CellPixelSize cell) {
+filter_refs(GraphicsManager *self, const void* data, bool free_images, bool (*filter_func)(const ImageRef*, Image*, const void*, CellPixelSize), CellPixelSize cell, bool only_first_image) {
+    bool matched = false;
     for (size_t i = self->image_count; i-- > 0;) {
         Image *img = self->images + i;
         for (size_t j = img->refcnt; j-- > 0;) {
             ImageRef *ref = img->refs + j;
             if (filter_func(ref, img, data, cell)) {
                 remove_i_from_array(img->refs, j, img->refcnt);
                 self->layers_dirty = true;
+                matched = true;
             }
         }
         if (img->refcnt == 0 && (free_images || img->client_id == 0)) remove_image(self, i);
+        if (only_first_image && matched) break;
     }
 }
 
+
 static inline void
 modify_refs(GraphicsManager *self, const void* data, bool free_images, bool (*filter_func)(ImageRef*, Image*, const void*, CellPixelSize), CellPixelSize cell) {
     for (size_t i = self->image_count; i-- > 0;) {
@@ -743,7 +798,7 @@ clear_all_filter_func(const ImageRef *ref UNUSED, Image UNUSED *img, const void
 
 void
 grman_clear(GraphicsManager *self, bool all, CellPixelSize cell) {
-    filter_refs(self, NULL, true, all ? clear_all_filter_func : clear_filter_func, cell);
+    filter_refs(self, NULL, true, all ? clear_all_filter_func : clear_filter_func, cell, false);
 }
 
 static inline bool
@@ -753,6 +808,14 @@ id_filter_func(const ImageRef *ref, Image *img, const void *data, CellPixelSize
     return false;
 }
 
+static inline bool
+number_filter_func(const ImageRef *ref, Image *img, const void *data, CellPixelSize cell UNUSED) {
+    const GraphicsCommand *g = data;
+    if (img->client_number == g->image_number) return !g->placement_id || ref->client_id == g->placement_id;
+    return false;
+}
+
+
 static inline bool
 x_filter_func(const ImageRef *ref, Image UNUSED *img, const void *data, CellPixelSize cell UNUSED) {
     const GraphicsCommand *g = data;
@@ -786,8 +849,9 @@ point3d_filter_func(const ImageRef *ref, Image *img, const void *data, CellPixel
 static void
 handle_delete_command(GraphicsManager *self, const GraphicsCommand *g, Cursor *c, bool *is_dirty, CellPixelSize cell) {
     static GraphicsCommand d;
+    bool only_first_image = false;
     switch (g->delete_action) {
-#define I(u, data, func) filter_refs(self, data, g->delete_action == u, func, cell); *is_dirty = true; break
+#define I(u, data, func) filter_refs(self, data, g->delete_action == u, func, cell, only_first_image); *is_dirty = true; break
 #define D(l, u, data, func) case l: case u: I(u, data, func)
 #define G(l, u, func) D(l, u, g, func)
         case 0:
@@ -802,6 +866,10 @@ handle_delete_command(GraphicsManager *self, const GraphicsCommand *g, Cursor *c
         case 'C':
             d.x_offset = c->x + 1; d.y_offset = c->y + 1;
             I('C', &d, point_filter_func);
+        case 'n':
+        case 'N':
+            only_first_image = true;
+            I('N', &g, number_filter_func);
         default:
             REPORT_ERROR("Unknown graphics command delete action: %c", g->delete_action);
             break;
@@ -839,6 +907,11 @@ grman_handle_command(GraphicsManager *self, const GraphicsCommand *g, const uint
     const char *ret = NULL;
     command_response[0] = 0;
 
+    if (g->id && g->image_number) {
+        set_command_failed_response("EINVAL", "Must not specify both image id and image number");
+        return finish_command_response(g, false, g->id, g->placement_id, g->image_number);
+    }
+
     switch(g->action) {
         case 0:
         case 't':
@@ -848,21 +921,21 @@ grman_handle_command(GraphicsManager *self, const GraphicsCommand *g, const uint
             bool is_query = g->action == 'q';
             if (is_query) { iid = 0; if (!q_iid) { REPORT_ERROR("Query graphics command without image id"); break; } }
             Image *image = handle_add_command(self, g, payload, is_dirty, iid);
-            if (is_query) ret = finish_command_response(g, image != NULL,  q_iid, 0);
-            else ret = finish_command_response(g, image != NULL, self->last_init_graphics_command.id, self->last_init_graphics_command.placement_id);
+            if (is_query) ret = finish_command_response(g, image != NULL,  q_iid, 0, 0);
+            else ret = finish_command_response(g, image != NULL, self->last_init_graphics_command.id, self->last_init_graphics_command.placement_id, self->last_init_graphics_command.image_number);
             if (self->last_init_graphics_command.action == 'T' && image && image->data_loaded) handle_put_command(self, &self->last_init_graphics_command, c, is_dirty, image, cell);
             id_type added_image_id = image ? image->internal_id : 0;
             if (g->action == 'q') remove_images(self, add_trim_predicate, 0);
             if (self->used_storage > STORAGE_LIMIT) apply_storage_quota(self, STORAGE_LIMIT, added_image_id);
             break;
         }
         case 'p':
-            if (!g->id) {
-                REPORT_ERROR("Put graphics command without image id");
+            if (!g->id && !g->image_number) {
+                REPORT_ERROR("Put graphics command without image id or number");
                 break;
             }
-            handle_put_command(self, g, c, is_dirty, NULL, cell);
-            ret = finish_command_response(g, true, g->id, g->placement_id);
+            uint32_t image_id = handle_put_command(self, g, c, is_dirty, NULL, cell);
+            ret = finish_command_response(g, true, image_id, g->placement_id, g->image_number);
             break;
         case 'd':
             handle_delete_command(self, g, c, is_dirty, cell);
@@ -886,8 +959,8 @@ new(PyTypeObject UNUSED *type, PyObject UNUSED *args, PyObject UNUSED *kwds) {
 static inline PyObject*
 image_as_dict(Image *img) {
 #define U(x) #x, img->x
-    return Py_BuildValue("{sI sI sI sI sK sI sO sO sN}",
-        U(texture_id), U(client_id), U(width), U(height), U(internal_id), U(refcnt),
+    return Py_BuildValue("{sI sI sI sI sK sI sI sO sO sN}",
+        U(texture_id), U(client_id), U(width), U(height), U(internal_id), U(refcnt), U(client_number),
         "data_loaded", img->data_loaded ? Py_True : Py_False,
         "is_4byte_aligned", img->load_data.is_4byte_aligned ? Py_True : Py_False,
         "data", Py_BuildValue("y#", img->load_data.data, img->load_data.data_sz)
@@ -907,6 +980,13 @@ W(image_for_client_id) {
     return image_as_dict(img);
 }
 
+W(image_for_client_number) {
+    unsigned long num = PyLong_AsUnsignedLong(args);
+    Image *img = img_by_client_number(self, num);
+    if (!img) Py_RETURN_NONE;
+    return image_as_dict(img);
+}
+
 W(shm_write) {
     const char *name, *data;
     Py_ssize_t sz;
@@ -957,6 +1037,7 @@ W(update_layers) {
 
 static PyMethodDef methods[] = {
     M(image_for_client_id, METH_O),
+    M(image_for_client_number, METH_O),
     M(update_layers, METH_VARARGS),
     {NULL}  /* Sentinel */
 };

kitty/graphics.h

@@ -10,7 +10,7 @@
 
 typedef struct {
     unsigned char action, transmission_type, compressed, delete_action;
-    uint32_t format, more, id, data_sz, data_offset, placement_id, quiet;
+    uint32_t format, more, id, image_number, data_sz, data_offset, placement_id, quiet;
     uint32_t width, height, x_offset, y_offset, data_height, data_width, num_cells, num_lines, cell_x_offset, cell_y_offset;
     int32_t z_index;
     size_t payload_sz;
@@ -44,7 +44,7 @@ typedef struct {
 
 
 typedef struct {
-    uint32_t texture_id, client_id, width, height;
+    uint32_t texture_id, client_id, client_number, width, height;
     id_type internal_id;
 
     bool data_loaded;