Code search needs enhancement

[ofnhkb1]

gitea v:1.3 rc2
When I set the code indexs
start gitea
Start normal
But the web did not start
Stay in the index creation phase

Suggest
First start the web
Create the index again
The current repo name is output during index creation

[ofnhkb1]

repos total size 1.5G
repos number 78

[lafriks]

Code search index takes a lot of disk space that's why it's currently disabled by default (5x repository size) and also can take a lot of time to index

[ofnhkb1]

My idea is to first start the web, and then start a new process or thread, according to repo polling index, instead of now, blocking the start of the web
If the current repo is generating an index, you can pause the push operation
I also have a new idea, set to determine which repo need to generate an index, because not all repo need to generate an index
Thank you

[lunny]

Currently, we have a global config to enable or disable code search, a better improvement is every repository has a config to enable or disable code search.

[lunny]

Even config which file types should be indexed.

[laoshaw]

using the newest gitea i still do not see how the indexing work, i can only search repositories and can not search the code at all, how can I enable code search?

[sapk]

@laoshaw You need to activate it via app.ini conf file. See https://github.com/go-gitea/gitea/blob/master/custom/conf/app.ini.sample#L211. Thoses options are currently missing from documentation https://docs.gitea.io/en-us/config-cheat-sheet/ and should be added in #3324.

Just tested repo indexing. It works well, but there are 2 problems:

• It doesn't show searched text for jsp and jrxml files, even if file lines seems to be right (follow an example):
  
• Is it possible to improve the disk size repo indexing file? I get a size of 131MB for just 1 repo.

[lafriks]

Might be worth to rewrite it not to use bleve index but Google codesearch code like Etsy/hound is using it. For codesearch index size is about 1/3 of code file size

The code of my project is about 41.6MB. So, I can confirm that indexing are occupying 3 times the space of code

[thehowl]

I think this should be closed. #3366 addresses the issue posted by the OP - it will do the initial indexing alongside the web frontend.

For the other issues posted in here:

• The space consumption problem is known and pretty clearly documented:
  
  For now there is no workaround, since it is caused by our indexer itself (bleve), so there is nothing to do about it. We might one day change the indexing lib as said by @lafriks.
• @giudon your JSP/JRXML issue seems interesting - mind creating an issue about it? possibly with a link to your repository, so we can clone and replicate, if you're able to replicate.