Added filewatch and cache functionality, notes can be searched by aliases

[kostabekre]

In order to implement search by alises, a more faster solution is needed than grep search.
In order to solve this, the cache of the vault is created on the startup as a JSON file and updated every time a note is changed.
The cache is used in search of the notes and their aliases.
For the current moment only telescope search is available.

Solves this issue.

[neo451]

really nice feature! would need to look into the code more in the next few days, meanwhile you can fix the style issue with make style and solve those small things I commented.

[kostabekre]

@neo451 Maybe I need to add additional configurations to stylelua, but running make style doesn't show anything (I already used the command and made changes before submitting the request)

[neo451]

@neo451 Maybe I need to add additional configurations to stylelua, but running make style doesn't show anything (I already used the command and made changes before submitting the request)

No idea why is that, but you can look at the action result, stylua wants to add a ;, because next line is a bit of a unidiomatic lua, where the start of expression is in (). Maybe you have a global stylua configuration, or there's issue with our makefile and workflow behaving differently. Will look into that.

ps: My local stylua does inform me of this error.

[kostabekre]

Well, I tried to different options, but my stylua says that ; is not right in this line.
Also my linter (luacheck) removes that semicolon.

[sotte]

Quite interesting. I was actually just playing with porting a sqlite full text search that I used and adapting it for obsidian. While doing that I thought it might be cool to use sqlite as a cache.

This goes into a similar direction.

@kostabekre it would be interesting to see how much faster this is compared to the base case which just uses rg.

[sotte]

Right now this implementation only works for telescope, right? I think we should have a picker independent way of using a cache.

[kostabekre]

@sotte I was thinking of using SQLite myself, and I have founded this repository.

But after some time and searching the implementations of other people, I decided that adding a new dependency won't be a good idea.

1. Using SQLite in our repository would demand both installing the SQLite on the user's computer and adding the plugin.
2. The plugin I mentioned above is not actively maintained. I don't know the reason but I think it's either people use more appropriate solution for their use case or they use the SQLite without the plugin.
3. There are list of repositories, that use the SQLite plugin. One of them removed the SQLite completely. Well, it's a weak reason, but when I was researching, I thought using a plugin that is not actively used by others is not a good idea.

Well, I don't think SQLite is bad or anything, it might be the ideal solution for our needs but I decided take a more simple route both for users and for developers - using JSON.

I was actually just playing with porting a sqlite full text search that I used and adapting it for obsidian.

Could you describe how that works?

[kostabekre]

Right now this implementation only works for telescope, right? I think we should have a picker independent way of using a cache.

Yes, for now I made the implementation only for telescope (and not a very good one, it needs some polishing) because I haven't used other pickers.
I made the basics, and I thought the code can be reviewed by other people.
The other pickers can be done in separate pull requests or in this PR some time later.

[kostabekre]

it would be interesting to see how much faster this is compared to the base case which just uses rg.

Okay, I can do that.

[neo451]

yes I was mistaken, I meant workspace.set

[neo451]

@kostabekre I did not catch this part last time

For now I didn't create an event handler when workspace is updated.
meaning it is not working now? or is this a question or issue?

[kostabekre]

@neo451 sorry, I meant that I didn't implement the event when the workspace was changed. I didn't have time to finish.
I wrote the following 3 days ago:

But as a concept, filewatch really belongs to a workspace, but the logic is tied to cache. I think I need to create a way, where anyone can subscribe to the workspace update event.

Maybe it's not necessary, but I thought it could be more flexible that way.

[neo451]

But as a concept, filewatch really belongs to a workspace, but the logic is tied to cache. I think I need to create a way, where anyone can subscribe to the workspace update event.

Since there's like the concept of stuff like dynamic workspaces (which I don't understand now as well), I think we can follow the current approach as it is.

[kostabekre]

Okay! Now, there are two things that are not implemented:

• Ignore folders. I use syncthing and it sometimes creates conflict files, which are picked up by filewatch and cache.
• Only one instance of neovim should create file watch handlers for a workspace. If there are several instances of neovim, each will create file watch handlers and because of that I have seen weird errors.

I will read how to do that and implement the solutions on the weekends.

[kostabekre]

@neo451 hi! Is there anything that I need to do?

[kostabekre]

The lock file is created in the root of the workspace, maybe I need to move it to other place?

[neo451]

sorry I don't have time to properly review this recently, but I had the idea the other day that we should maybe put cache in vim.fn.stdpath("state") / "obsidian", but how do they correspond to each workspace, (in case of moving and stuff) is a problem, but maybe worth looking into so that we don't put a new file in the vault.

Also, I am working on #339 for limiting jobs (will merge soon), and we don't want to rely on plenary no more on the main, so also look into that.

[GordianDziwis]

Why don't you use rg instead? It is an external dep, but it is most def fast enough for searching a few 100,000 wiki pages.

[kostabekre]

Why don't you use rg instead? It is an external dep, but it is most def fast enough for searching a few 100,000 wiki pages.

it's not just use rg aliases: ..., because the value of the field can be in various ways.
A custom logic would be needed to cover each case and it's very error prone.

I'm not advanced user of rg but I think using the nvim.obsidian's API is the better way.

[kostabekre]

@neo451 Hello! I was debugging my code, and for some reason I founded that in file init.lua, Workspace.set function, the workspace is completely different from the one I opened.

The reason is in the obsidian.setup function, we have the following line:

  obsidian.Workspace.set(Obsidian.workspaces[1])

Is that a bug?
I didn't notice that before, because it's the main workspace I write my notes.

[kostabekre]

for now I get the cwd and compare it with filename of a workspace. If there is a match, use the workspace.

[kostabekre]

@neo451 I moved the lock file to the state folder and use obsidian.async instead of plenary.