Scrolling dashboard tables

[manson]

When scrolling table in dashboard the data is being loaded with new rows (next pages) and time fields for newly loaded rows are not displaying (empty), although data exists and when open the form, it is shown ok.

[kamalshkeir]

When scrolling table in dashboard the data is being loaded with new rows (next pages) and time fields for newly loaded rows are not displaying (empty), although data exists and when open the form, it is shown ok.

@manson can you try remove assets folder and make it clone it again ? , i have encountred this problem when i was changing time to unix, i remember i fix it but you should regenerate admin assets

[kamalshkeir]

@manson you deleted the assets folder ?

[manson]

@manson you deleted the assets folder ?
yes

    github.com/kamalshkeir/kago v1.2.7
github.com/kamalshkeir/kenv v1.0.0
github.com/kamalshkeir/klog v1.0.0
github.com/kamalshkeir/kmux v1.9.2
github.com/kamalshkeir/korm v1.7.1
github.com/kamalshkeir/ksbus v1.1.1

Recreated assets, hardreloaded page and gets the same

[kamalshkeir]

@manson you deleted the assets folder ?
yes

    github.com/kamalshkeir/kago v1.2.7
github.com/kamalshkeir/kenv v1.0.0
github.com/kamalshkeir/klog v1.0.0
github.com/kamalshkeir/kmux v1.9.2
github.com/kamalshkeir/korm v1.7.1
github.com/kamalshkeir/ksbus v1.1.1

Recreated assets, hardreloaded page and gets the same

clear the cache, it should be working

[kamalshkeir]

@manson i tested again

[manson]

clear the cache, it should be working

Damn, I did clear the cache twice it did not help, but now after 3d time it worked... miracle... :-( But do not go away I have another issue for you :-)

[manson]

@kamalshkeir
Another one thing found - when I do many writings to database from different parallel threads, sometimes I see exception "DATABASE BUSY" or something like this. I did not go further to investigate, but can suppose that database works in Multi-thread mode that although permits multithreading but connection to a database cannot be used from different threads (as it seems to be now), Say alone Single-thread mode when no any mutexes at all. But definitely this is not a Serialized mode that drops all restrictions. If this is like that, so the korm should mutex connections or use database in serialized mode...
Here what internet says:"The SQLITE_BUSY error occurs when the database is locked by another process or thread and cannot be accessed. This can happen when multiple threads or processes are trying to access the same database at the same time. To resolve this issue, you should use a database connection pool to manage the database connections or implement a locking or synchronization mechanism to ensure that only one thread or process can access the database at a time. You can also use the SQLite busy timeout setting to configure the amount of time SQLite will wait before it returns a busy error."

[manson]

@kamalshkeir
Meanwhile, could you please explain me, if possible, the next thing. I have used sse before in other projects, used regular EventSource at javascript side and other client libraries. It keeps connection to a server very long time. It does not reconnect every 10 seconds as it does with Kago. I have replaced bus.App.SSE to bus.App.Get and bind there the side sse-server just to check, and it still reconnects every about 10 seconds. I used Echo before, made the same thing, and connected client stands long while connection is broken (or at least several dozens of minutes, or even maybe an hour). The problem is: as opposed to long polling, sse connections doesn't drop on every data income, it is a stream where data flows to a client uninterruptedly. So if data comes more or less often, this reconnection slows down the client data acceptance for about 2-4 seconds. This is not good. By the method of elimination, I came to the conclusion that you somehow control connections and not let them last long. Is that true? In any case, could consult me, how to make this behave like in other frameworks - that connection would last long without interrupts of itself?

UPD: Tested again - just pointed the same testing page against my other project (with Echo framework) with sse route, and it stands without interrupts.

[kamalshkeir]

@manson it's because of kmux.ReadTimeout and IdleTimeout and maybe WriteTimeout , try to play with these values until i understand how this is working

[manson]

@kamalshkeir
Yea... I used this:

kmux.WriteTimeout = 5 * time.Minute
kmux.IdleTimeout = 5 * time.Minute

And it solved. Just making keepalive 1 minutes empty packets send and it lives well. Without keepalive packets some usefull packet (first) when long idle is lost. So, for now, I consider it as suites my needs.

[kamalshkeir]

@manson , some updates to SSE, here is an example i was playing with, working fine with the new updates, what i did is : instead of changing WriteTimeout and IdleTimeout globaly, i used a new go1.20 feature called http.ResponseController which can control these values for each request, so for SSE handlers this will be done automatically and set the headers, also it support param in path if the path doesn't conflict with another GET path .

package main

import (
	"fmt"
	"net/http"

	"github.com/kamalshkeir/kmux"
)

func main() {
	app := kmux.New()
	app.Get("/", func(c *kmux.Context) {
		c.Html("index.html", nil)
	})
	app.LocalTemplates("temps")

	msgChan := make(chan string)
	app.Sse("/test/:param", func(c *kmux.Context) {
		fmt.Println(c.Param("param"))
		notify := c.ResponseWriter.(http.CloseNotifier).CloseNotify()
		for {
			select {
			case v := <-msgChan:
				err := c.Stream(v)
				if err != nil {
					return
				}
			case <-notify:
				return
			}
		}
	})

	// when someone hit this endpoint we publish to SSE using msgChan
	app.Get("/pub/sse", func(c *kmux.Context) {
		go func() {
			msgChan <- "hello stream"
		}()
		c.Text("sse")
	})

	app.Run(":9313")
}

go get -u github.com/kamalshkeir/kmux@v1.10.5

[manson]

@kamalshkeir
I have checked new SSE. It works perfect for me. Way to go! I've switched back to this natural kago way to use SSE and satisfied. Thanks!