Merge branch 'master' of git://github.com/go-gitea/gitea

Author: aswild

CONTRIBUTING.md

@@ -11,6 +11,7 @@
   - [Translation](#translation)
   - [Code review](#code-review)
   - [Styleguide](#styleguide)
+  - [Design guideline](#design-guideline)
   - [Developer Certificate of Origin (DCO)](#developer-certificate-of-origin-dco)
   - [Release Cycle](#release-cycle)
   - [Maintainers](#maintainers)
@@ -71,13 +72,15 @@ Here's how to run the test suite:
 
 - Install the correct version of the drone-cli package.  As of this
   writing, the correct drone-cli version is
-  [1.1.0](https://docs.drone.io/cli/install/).
+  [1.2.0](https://docs.drone.io/cli/install/).
 - Ensure you have enough free disk space.  You will need at least
   15-20 Gb of free disk space to hold all of the containers drone
   creates (a default AWS or GCE disk size won't work -- see
   [#6243](https://github.com/go-gitea/gitea/issues/6243)).
 - Change into the base directory of your copy of the gitea repository,
   and run `drone exec --event pull_request`.
+- At the moment `drone exec` doesn't support the Docker Toolbox on Windows 10
+  (see [drone-cli#135](https://github.com/drone/drone-cli/issues/135))
 
 The drone version, command line, and disk requirements do change over
 time (see [#4053](https://github.com/go-gitea/gitea/issues/4053) and
@@ -118,6 +121,8 @@ An exception are the tools to build the CSS and images.
 - To build Images: ImageMagick, inkscape and zopflipng binaries must be
   available in your `PATH` to run `make generate-images`.
 
+For more details on how to generate files, build and test Gitea, see the [hacking instructions](https://docs.gitea.io/en-us/hacking-on-gitea/)
+
 ## Code review
 
 Changes to Gitea must be reviewed before they are accepted—no matter who
@@ -157,6 +162,22 @@ import (
 )
 ```
 
+## Design guideline
+
+To maintain understandable code and avoid circular dependencies it is important to have a good structure of the code. The gitea code is divided into the following parts:
+
+- **integration:** Integrations tests 
+- **models:** Contains the data structures used by xorm to construct database tables. It also contains supporting functions to query and update the database. Dependecies to other code in Gitea should be avoided although some modules might be needed (for example for logging).
+- **models/fixtures:** Sample model data used in integration tests.
+- **models/migrations:** Handling of database migrations between versions. PRs that changes a database structure shall also have a migration step.
+- **modules:** Different modules to handle specific functionality in Gitea.
+- **public:** Frontend files (javascript, images, css, etc.)
+- **routers:** Handling of server requests. As it uses other Gitea packages to serve the request, other packages (models, modules or services) shall not depend on routers
+- **services:** Support functions for common routing operations. Uses models and modules to handle the request.
+- **templates:** Golang templates for generating the html output.
+- **vendor:** External code that Gitea depends on.
+
+
 ## Developer Certificate of Origin (DCO)
 
 We consider the act of contributing to the code by submitting a Pull
@@ -283,7 +304,7 @@ be reviewed by two maintainers and must pass the automatic tests.
 * Add a tag as `git tag -s -F release.notes v$vmaj.$vmin.$`, release.notes file could be a temporary file to only include the changelog this version which you added to `CHANGELOG.md`.
 * And then push the tag as `git push origin v$vmaj.$vmin.$`. Drone CI will automatically created a release and upload all the compiled binary. (But currently it didn't add the release notes automatically. Maybe we should fix that.)
 * If needed send PR for changelog on branch `master`.
-* Send PR to [blog repository](https://github.com/go-gitea/blog) announcing the release.
+* Send PR to [blog repository](https://gitea.com/gitea/blog) announcing the release.
 
 ## Copyright
 

README.md

@@ -1,6 +1,6 @@
 [简体中文](https://github.com/go-gitea/gitea/blob/master/README_ZH.md)
 
-# Gitea - Git with a cup of tea
+<h1> <img src="https://raw.githubusercontent.com/go-gitea/gitea/master/public/img/gitea-192.png" alt="logo" width="30" height="30"> Gitea - Git with a cup of tea</h1>
 
 [![Build Status](https://drone.gitea.io/api/badges/go-gitea/gitea/status.svg)](https://drone.gitea.io/go-gitea/gitea)
 [![Join the Discord chat at https://discord.gg/NsatcWJ](https://img.shields.io/discord/322538954119184384.svg)](https://discord.gg/NsatcWJ)

cmd/hook.go

@@ -66,6 +66,7 @@ func runHookPreReceive(c *cli.Context) error {
 	reponame := os.Getenv(models.EnvRepoName)
 	userID, _ := strconv.ParseInt(os.Getenv(models.EnvPusherID), 10, 64)
 	prID, _ := strconv.ParseInt(os.Getenv(models.ProtectedBranchPRID), 10, 64)
+	isDeployKey, _ := strconv.ParseBool(os.Getenv(models.EnvIsDeployKey))
 
 	buf := bytes.NewBuffer(nil)
 	scanner := bufio.NewScanner(os.Stdin)
@@ -98,6 +99,7 @@ func runHookPreReceive(c *cli.Context) error {
 				GitObjectDirectory:              os.Getenv(private.GitObjectDirectory),
 				GitQuarantinePath:               os.Getenv(private.GitQuarantinePath),
 				ProtectedBranchID:               prID,
+				IsDeployKey:                     isDeployKey,
 			})
 			switch statusCode {
 			case http.StatusInternalServerError:

cmd/serv.go

@@ -191,6 +191,8 @@ func runServ(c *cli.Context) error {
 	os.Setenv(models.EnvPusherID, strconv.FormatInt(results.UserID, 10))
 	os.Setenv(models.ProtectedBranchRepoID, strconv.FormatInt(results.RepoID, 10))
 	os.Setenv(models.ProtectedBranchPRID, fmt.Sprintf("%d", 0))
+	os.Setenv(models.EnvIsDeployKey, fmt.Sprintf("%t", results.IsDeployKey))
+	os.Setenv(models.EnvKeyID, fmt.Sprintf("%d", results.KeyID))
 
 	//LFS token authentication
 	if verb == lfsAuthenticateVerb {

cmd/web.go

@@ -13,6 +13,7 @@ import (
 	"os"
 	"strings"
 
+	"code.gitea.io/gitea/modules/graceful"
 	"code.gitea.io/gitea/modules/log"
 	"code.gitea.io/gitea/modules/setting"
 	"code.gitea.io/gitea/routers"
@@ -226,6 +227,7 @@ func runWeb(ctx *cli.Context) error {
 		log.Critical("Failed to start server: %v", err)
 	}
 	log.Info("HTTP Listener: %s Closed", listenAddr)
+	graceful.WaitForServers()
 	log.Close()
 	return nil
 }

custom/conf/app.ini.sample

@@ -185,6 +185,8 @@ FILE_EXTENSIONS = .md,.markdown,.mdown,.mkd
 PROTOCOL = http
 DOMAIN = localhost
 ROOT_URL = %(PROTOCOL)s://%(DOMAIN)s:%(HTTP_PORT)s/
+; when STATIC_URL_PREFIX is empty it will follow APP_URL
+STATIC_URL_PREFIX = 
 ; The address to listen on. Either a IPv4/IPv6 address or the path to a unix socket.
 HTTP_ADDR = 0.0.0.0
 HTTP_PORT = 3000
@@ -317,10 +319,12 @@ LOG_SQL = true
 DB_RETRIES = 10
 ; Backoff time per DB retry (time.Duration)
 DB_RETRY_BACKOFF = 3s
-; Max idle database connections on connnection pool, default is 0
-MAX_IDLE_CONNS = 0
-; Database connection max life time, default is 3s
+; Max idle database connections on connnection pool, default is 2
+MAX_IDLE_CONNS = 2
+; Database connection max life time, default is 0 or 3s mysql (See #6804 & #7071 for reasoning)
 CONN_MAX_LIFETIME = 3s
+; Database maximum number of open connections, default is 0 meaning no maximum
+MAX_OPEN_CONNS = 0
 
 [indexer]
 ; Issue indexer type, currently support: bleve or db, default is bleve
@@ -436,6 +440,10 @@ ALLOW_ONLY_EXTERNAL_REGISTRATION = false
 REQUIRE_SIGNIN_VIEW = false
 ; Mail notification
 ENABLE_NOTIFY_MAIL = false
+; This setting enables gitea to be signed in with HTTP BASIC Authentication using the user's password
+; If you set this to false you will not be able to access the tokens endpoints on the API with your password
+; Please note that setting this to false will not disable OAuth Basic or Basic authentication using a token
+ENABLE_BASIC_AUTHENTICATION = true
 ; More detail: https://github.com/gogits/gogs/issues/165
 ENABLE_REVERSE_PROXY_AUTHENTICATION = false
 ENABLE_REVERSE_PROXY_AUTO_REGISTRATION = false
@@ -868,6 +876,6 @@ TOKEN =
 QUEUE_TYPE = channel
 ; Task queue length, available only when `QUEUE_TYPE` is `channel`.
 QUEUE_LENGTH = 1000
-; Task queue connction string, available only when `QUEUE_TYPE` is `redis`. 
+; Task queue connection string, available only when `QUEUE_TYPE` is `redis`.
 ; If there is a password of redis, use `addrs=127.0.0.1:6379 password=123 db=0`.
 QUEUE_CONN_STR = "addrs=127.0.0.1:6379 db=0"

docs/content/doc/advanced/config-cheat-sheet.en-us.md

@@ -138,6 +138,13 @@ Values containing `#` or `;` must be quoted using `` ` `` or `"""`.
 - `ROOT_URL`: **%(PROTOCOL)s://%(DOMAIN)s:%(HTTP\_PORT)s/**:
    Overwrite the automatically generated public URL.
    This is useful if the internal and the external URL don't match (e.g. in Docker).
+- `STATIC_URL_PREFIX`: **\<empty\>**:
+   Overwrite this option to request static resources from a different URL.
+   This includes CSS files, images, JS files and web fonts.
+   Avatar images are dynamic resources and still served by gitea.
+   The option can be just a different path, as in `/static`, or another domain, as in `https://cdn.example.com`.
+   Requests are then made as `%(ROOT_URL)s/static/css/index.css` and `https://cdn.example.com/css/index.css` respective.
+   The static files are located in the `public/` directory of the gitea source repository.
 - `HTTP_ADDR`: **0.0.0.0**: HTTP listen address.
    - If `PROTOCOL` is set to `fcgi`, Gitea will listen for FastCGI requests on TCP socket
      defined by `HTTP_ADDR` and `HTTP_PORT` configuration settings.
@@ -192,8 +199,12 @@ Values containing `#` or `;` must be quoted using `` ` `` or `"""`.
 - `LOG_SQL`: **true**: Log the executed SQL.
 - `DB_RETRIES`: **10**: How many ORM init / DB connect attempts allowed.
 - `DB_RETRY_BACKOFF`: **3s**: time.Duration to wait before trying another ORM init / DB connect attempt, if failure occured.
-- `MAX_IDLE_CONNS` **0**: Max idle database connections on connnection pool, default is 0
-- `CONN_MAX_LIFETIME` **3s**: Database connection max lifetime
+- `MAX_OPEN_CONNS` **0**: Database maximum open connections - default is 0, meaning there is no limit.
+- `MAX_IDLE_CONNS` **2**: Max idle database connections on connnection pool, default is 2 - this will be capped to `MAX_OPEN_CONNS`.
+- `CONN_MAX_LIFETIME` **0 or 3s**: Sets the maximum amount of time a DB connection may be reused - default is 0, meaning there is no limit (except on MySQL where it is 3s - see #6804 & #7071).
+  
+Please see #8540 & #8273 for further discussion of the appropriate values for `MAX_OPEN_CONNS`, `MAX_IDLE_CONNS` & `CONN_MAX_LIFETIME` and their
+relation to port exhaustion.
 
 ## Indexer (`indexer`)
 
@@ -265,6 +276,10 @@ Values containing `#` or `;` must be quoted using `` ` `` or `"""`.
 - `REQUIRE_SIGNIN_VIEW`: **false**: Enable this to force users to log in to view any page.
 - `ENABLE_NOTIFY_MAIL`: **false**: Enable this to send e-mail to watchers of a repository when
    something happens, like creating issues. Requires `Mailer` to be enabled.
+- `ENABLE_BASIC_AUTHENTICATION`: **true**: Disable this to disallow authenticaton using HTTP
+   BASIC and the user's password. Please note if you disable this you will not be able to access the
+   tokens API endpoints using a password. Further, this only disables BASIC authentication using the
+   password - not tokens or OAuth Basic.
 - `ENABLE_REVERSE_PROXY_AUTHENTICATION`: **false**: Enable this to allow reverse proxy authentication.
 - `ENABLE_REVERSE_PROXY_AUTO_REGISTRATION`: **false**: Enable this to allow auto-registration
    for reverse authentication.

docs/content/doc/advanced/external-renderers.en-us.md

@@ -51,7 +51,7 @@ add one `[markup.XXXXX]` section per external renderer on your custom `app.ini`:
 [markup.asciidoc]
 ENABLED = true
 FILE_EXTENSIONS = .adoc,.asciidoc
-RENDER_COMMAND = "asciidoctor --out-file=- -"
+RENDER_COMMAND = "asciidoctor -e -a leveloffset=-1 --out-file=- -"
 ; Input is not a standard input but a file
 IS_INPUT_FILE = false
 

docs/content/doc/installation/from-source.en-us.md

@@ -118,12 +118,12 @@ launched manually from command line, it can be killed by pressing `Ctrl + C`.
 ./gitea web
 ```
 
-## Changing the default CustomPath, CustomConf and AppWorkDir
+## Changing the default CustomPath, CustomConf and AppWorkPath
 
 Gitea will search for a number of things from the `CustomPath`. By default this is
 the `custom/` directory in the current working directory when running Gitea. It will also
 look for its configuration file `CustomConf` in `$CustomPath/conf/app.ini`, and will use the
-current working directory as the relative base path `AppWorkDir` for a number configurable
+current working directory as the relative base path `AppWorkPath` for a number configurable
 values.
 
 These values, although useful when developing, may conflict with downstream users preferences.
@@ -134,7 +134,7 @@ using the `LDFLAGS` environment variable for `make`. The appropriate settings ar
 
 * To set the `CustomPath` use `LDFLAGS="-X \"code.gitea.io/gitea/modules/setting.CustomPath=custom-path\""`
 * For `CustomConf` you should use `-X \"code.gitea.io/gitea/modules/setting.CustomConf=conf.ini\"`
-* For `AppWorkDir` you should use `-X \"code.gitea.io/gitea/modules/setting.AppWorkDir=working-directory\"`
+* For `AppWorkPath` you should use `-X \"code.gitea.io/gitea/modules/setting.AppWorkPath=working-path\"`
 
 Add as many of the strings with their preceding `-X` to the `LDFLAGS` variable and run `make build`
 with the appropriate `TAGS` as above.

docs/content/doc/usage/fail2ban-setup.md

@@ -26,7 +26,7 @@ on a bad authentication:
 2018/04/26 18:15:54 [I] Failed authentication attempt for user from xxx.xxx.xxx.xxx
 ```
 
-So we set our filter in `/etc/fail2ban/filter.d/gitea.conf`:
+Add our filter in `/etc/fail2ban/filter.d/gitea.conf`:
 
 ```ini
 # gitea.conf
@@ -35,12 +35,11 @@ failregex =  .*Failed authentication attempt for .* from <HOST>
 ignoreregex =
 ```
 
-And configure it in `/etc/fail2ban/jail.d/jail.local`:
+Add our jail in `/etc/fail2ban/jail.d/gitea.conf`:
 
 ```ini
 [gitea]
 enabled = true
-port = http,https
 filter = gitea
 logpath = /home/git/gitea/log/gitea.log
 maxretry = 10
@@ -49,6 +48,23 @@ bantime = 900
 action = iptables-allports
 ```
 
+If you're using Docker, you'll also need to add an additional jail to handle the **FORWARD** 
+chain in **iptables**. Configure it in `/etc/fail2ban/jail.d/gitea-docker.conf`:
+
+```ini
+[gitea-docker]
+enabled = true
+filter = gitea
+logpath = /home/git/gitea/log/gitea.log
+maxretry = 10
+findtime = 3600
+bantime = 900
+action = iptables-allports[chain="FORWARD"]
+```
+
+Then simply run `service fail2ban restart` to apply your changes. You can check to see if 
+fail2ban has accepted your configuration using `service fail2ban status`.
+
 Make sure and read up on fail2ban and configure it to your needs, this bans someone 
 for **15 minutes** (from all ports) when they fail authentication 10 times in an hour.