Merge branch 'release/2.3.0'

Author: brandonkelly

CHANGELOG.md

@@ -1,5 +1,19 @@
 # Release Notes for Webhooks for Craft CMS
 
+## 2.3.0 - 2020-07-18
+
+### Added
+- Added a settings page in the control panel.
+- Webhooks have a new “Debounce Key Format” setting, which can be used to debouncing similar webhook requests. ([#27](https://github.com/craftcms/webhooks/issues/27))
+- Webhook URLs can now be set to environment variables or Twig code. ([#18](https://github.com/craftcms/webhooks/issues/18))
+- Webhooks can now send PUT requests. ([#21](https://github.com/craftcms/webhooks/issues/21))
+- The Activity page now has a “Clear” button, which will clear out all request activity for completed requests. ([#32](https://github.com/craftcms/webhooks/issues/32))
+- Added a new `guzzleConfig` plugin setting, which can be set from `config/webhooks.php` to customize the Guzzle config for webhook requests. ([#31](https://github.com/craftcms/webhooks/issues/31))
+
+### Fixed
+- Fixed a bug where webhooks would lose their custom payload template when enabled or disabled from the Manage Webhooks page. ([#22](https://github.com/craftcms/webhooks/issues/22))
+- Fixed a bug where the Edit Webhook page wouldn’t immediately show available filters if the webhook didn’t have any preselected filters.
+
 ## 2.2.0 - 2019-07-29
 
 ### Added

README.md

@@ -42,15 +42,15 @@ To configure Webhooks, create a `config/webhooks.php` file, which returns an arr
 return [
     'maxDepth' => 10,
     'maxAttempts' => 3,
-    'attemptDelay' => 120,
+    'retryDelay' => 120,
 ];
 ```
 
 The array can define the following keys:
 
 - `maxDepth` – The maximum depth that the plugin should go into objects/arrays when converting them to arrays for event payloads. (Default is `5`.)
 - `maxAttempts` – The maximum number of attempts each webhook should have before giving up, if the requests are coming back with non 2xx responses. (Default is `1`.)
-- `attemptDelay` – The delay (in seconds) between webhook attempts. (Default is `60`.)
+- `retryDelay` – The delay (in seconds) between webhook attempts. (Default is `60`.)
 
 ## Managing Webhooks
 
@@ -70,6 +70,8 @@ Webhooks listen to [events](https://www.yiiframework.com/doc/guide/2.0/en/concep
 
 The Sender Class can be a subclass of the class that triggers the event. For example, all elements fire an [afterSave](https://docs.craftcms.com/api/v3/craft-base-element.html#event-after-save) event after they’ve been saved, courtesy of their base class, [craft\base\Element](https://docs.craftcms.com/api/v3/craft-base-element.html). However if you’re only interested in sending a webhook when an _entry_ gets saved, you can set the Sender Class to [craft\elements\Entry](https://docs.craftcms.com/api/v3/craft-elements-entry.html).
 
+Webhook URLs can be set to an environment variable (`$VARIABLE_NAME`) or Twig code. If you set it to Twig code, you can reference the triggered event via an `event` variable. 
+
 See [Integrating with Task Automation Tools](#integrating-with-task-automation-tools) for examples on how to get a Webhook URL from various task automation tools.
 
 ![Screenshot of the Edit Webhook page](./images/edit-webhook.png)
@@ -90,6 +92,8 @@ Some events can have filters applied to them, which prevent webhooks from being
 
 <img src="./images/event-filters.png" width="414" height="381" alt="Screenshot of the Event Filters setting">
 
+Ignored filters (`○`) will not have any impact. Positive filters (`✓`) will be required for a webhook to execute, and a negative filter (`×`) will prevent it.
+
 Only element class events and certain `craft\services\Elements` events have any filters out of the box, but modules and plugins can register additional filters using the `craft\webhooks\Plugin::EVENT_REGISTER_FILTER_TYPES` event.
 
 ```php
@@ -132,6 +136,14 @@ class ArticleFilter implements FilterInterface
 }
 ```
 
+#### Debouncing Webhooks
+
+You can prevent multiple similar webhooks from being sent by setting a “Debounce Key Format” on your webhook. This is a Twig template that defines a “debounce key” for the webhook. If two webhooks generate the same debounce key, only the second one will actually be sent.  
+
+An `event` variable will be available to it that references the event that was triggered.
+
+For example, if your webhook is for an entry (`craft\elements\Entry`), then you could set the Debounce Key Format to `{{ event.sender.id }}` to prevent multiple webhook requests from being queued up at the same time.
+
 #### Sending Custom Headers
 
 You can send custom headers along with webhook requests using the Custom Headers setting.

SECURITY.md

@@ -0,0 +1,65 @@
+# Security Policy
+
+## Reporting a Vulnerability
+
+If you discover a security vulnerability, please review these guidelines before submitting a report. We take security seriously and do our best to resolve security issues as quickly as possible.
+
+## Guidelines
+
+While working to identify potential security vulnerabilities, we ask that you:
+
+- Share any issues you discover with us via [Github](https://github.com/craftcms/cms/security/advisories) or [our website](https://craftcms.com/contact) as soon as possible.
+- Give us a reasonable amount of time to address any reported issues before publicizing them.
+- Only report issues that are [in scope](#scope).
+- Provide a quality report with precise explanations and concrete attack scenarios.
+- Make sure you’re aware of the versions of Craft and Commerce that are actively [receiving security fixes](https://craftcms.com/knowledge-base/supported-versions).
+
+## Scope
+
+We are only interested in vulnerabilities that affect Craft or [first party Craft plugins](https://github.com/craftcms), tested against **your own local installation of the software**. You can install a local copy of Craft by following these [installation instructions](https://craftcms.com/docs/installing). Do **not** test against any Craft installation that you don’t own, including [craftcms.com](https://craftcms.com) or [demo.craftcms.com](https://demo.craftcms.com).
+
+### Qualifying Vulnerabilities
+
+- [Cross-Site Scripting (XSS)](https://en.wikipedia.org/wiki/Cross-site_scripting)
+- [Cross-Site Request Forgery (CSRF)](https://en.wikipedia.org/wiki/Cross-site_request_forgery)
+- [Arbitrary Code Execution](https://en.wikipedia.org/wiki/Arbitrary_code_execution)
+- [Privilege Escalation](https://en.wikipedia.org/wiki/Privilege_escalation)
+- [SQL Injection](https://en.wikipedia.org/wiki/SQL_injection)
+- [Session Hijacking](https://en.wikipedia.org/wiki/Session_hijacking)
+
+### Non-Qualifying Vulnerabilities
+
+- Reports from automated tools or scanners
+- Theoretical attacks without proof of exploitability
+- Attacks that can be guarded against by following our [security recommendations](https://craftcms.com/guides/securing-craft).
+- Server configuration issues outside of Craft’s control
+- [Denial of Service](https://en.wikipedia.org/wiki/Denial-of-service_attack) attacks
+- [Brute force attacks](https://en.wikipedia.org/wiki/Brute-force_attack) (e.g. on password or token hashes)
+- Username or email address enumeration
+- Social engineering of Pixel & Tonic staff or users of Craft installations
+- Physical attacks against Craft installations
+- Attacks involving physical access to a user’s device, or involving a device or network that’s already seriously compromised (e.g. [man-in-the-middle attacks](https://en.wikipedia.org/wiki/Man-in-the-middle_attack))
+- Attacks that are the result of a third party Craft plugin should be reported to the plugin’s author
+- Attacks that are the result of a third party library should be reported to the library maintainers
+- Bugs that rely on an unlikely user interaction (i.e. the user effectively attacking themselves)
+- Disclosure of tools or libraries used by Craft and/or their versions
+- Issues that are the result of a user clearly ignoring common security best practices (like sharing their password publicly)
+- Missing security headers which do not lead directly to a vulnerability via proof of concept
+- Vulnerabilities affecting users of outdated/unsupported browsers or platforms
+- Vulnerabilities affecting outdated versions of Craft
+- Any behavior that is clearly documented
+- Issues discovered while scanning a site you don’t own without permission
+- Missing CSRF tokens on forms (unless you have a proof of concept, many forms either don’t need CSRF or are mitigated in other ways) and “logout” CSRF attacks
+- [Open redirects](https://www.owasp.org/index.php/open_redirect)
+
+## Bounties
+
+To show our appreciation for the work it can take to find and report a vulnerability, we’re happy to offer researchers a monetary reward.
+
+Reward amounts vary depending upon the severity. Our minimum reward for a qualifying vulnerability report is $50 USD and we expect to pay $500+ USD for major vulnerabilities.
+
+A report will qualify for a bounty if:
+
+- Our [Guidelines](#guidelines) have been followed in full.
+- The vulnerability was previously unknown to us, or your report provides more information or shows the vulnerability to be more extensive than we originally thought.
+- The vulnerability is non-trivial.

composer.json

@@ -1,7 +1,7 @@
 {
   "name": "craftcms/webhooks",
   "description": "Post webhooks when events are triggered in Craft CMS.",
-  "version": "2.2.0",
+  "version": "2.3.0",
   "type": "craft-plugin",
   "keywords": [
     "html",

src/Plugin.php

@@ -20,6 +20,7 @@
 use craft\webhooks\filters\RevisionFilter;
 use GuzzleHttp\Exception\RequestException;
 use GuzzleHttp\RequestOptions;
+use yii\base\Application;
 use yii\base\Arrayable;
 use yii\base\Event;
 use yii\base\Exception;
@@ -38,9 +39,6 @@
  */
 class Plugin extends \craft\base\Plugin
 {
-    // Constants
-    // =========================================================================
-
     const STATUS_PENDING = 'pending';
     const STATUS_REQUESTED = 'requested';
     const STATUS_DONE = 'done';
@@ -68,8 +66,10 @@ class Plugin extends \craft\base\Plugin
      */
     const EVENT_REGISTER_FILTER_TYPES = 'registerFilterTypes';
 
-    // Properties
-    // =========================================================================
+    /**
+     * @inheritdoc
+     */
+    public $hasCpSettings = true;
 
     /**
      * @inheritdoc
@@ -79,10 +79,14 @@ class Plugin extends \craft\base\Plugin
     /**
      * @inheritdoc
      */
-    public $schemaVersion = '2.2.0';
+    public $schemaVersion = '2.3.0';
 
-    // Public Methods
-    // =========================================================================
+    /**
+     * @var SendRequestJob[] The request jobs that should be queued up at the end of the web request.
+     * @see request()
+     * @since 2.3.0
+     */
+    private $_pendingJobs = [];
 
     /**
      * @inheritdoc
@@ -120,7 +124,7 @@ public function init()
 
                 $view = Craft::$app->getView();
 
-                if ($webhook->method === 'post') {
+                if (in_array($webhook->method, ['post', 'put'], true)) {
                     // Build out the body data
                     if ($webhook->payloadTemplate) {
                         $json = $view->renderString($webhook->payloadTemplate, [
@@ -154,11 +158,9 @@ public function init()
 
                 foreach ($webhook->headers as $header) {
                     $header['value'] = Craft::parseEnv($header['value']);
-                    if (strpos($header['value'], '{') !== false) {
-                        $header['value'] = $view->renderString($header['value'], [
-                            'event' => $e,
-                        ]);
-                    }
+                    $header['value'] = $view->renderString($header['value'], [
+                        'event' => $e,
+                    ]);
                     // Get the trimmed lines
                     $lines = array_filter(array_map('trim', preg_split('/[\r\n]+/', $header['value'])));
                     // Add to the header array one-by-one, ensuring that we don't overwrite existing values
@@ -182,7 +184,19 @@ public function init()
                 }
 
                 // Queue the send request up
-                $this->request($webhook->method, $webhook->url, $headers, $body, $webhook->id);
+                $url = Craft::parseEnv($webhook->url);
+                $url = $view->renderString($url, [
+                    'event' => $e,
+                ]);
+
+                if ($webhook->debounceKeyFormat) {
+                    $debounceKey = Craft::parseEnv($webhook->debounceKeyFormat);
+                    $debounceKey = $webhook->id . ':' . $view->renderString($debounceKey, [
+                        'event' => $e,
+                    ]);
+                }
+
+                $this->request($webhook->method, $url, $headers, $body, $webhook->id, $debounceKey ?? null);
             });
         }
 
@@ -196,6 +210,16 @@ public function init()
         });
     }
 
+    /**
+     * @inheritdoc
+     */
+    protected function settingsHtml()
+    {
+        return Craft::$app->getView()->renderTemplate('webhooks/_settings', [
+            'settings' => $this->getSettings(),
+        ]);
+    }
+
     /**
      * @inheritdoc
      */
@@ -217,14 +241,45 @@ public function getCpNavItem()
      * @param array|null $headers
      * @param string|null $body
      * @param int|null $webhookId
+     * @param string|null $debounceKey
      * @throws \yii\db\Exception
      */
-    public function request(string $method, string $url, array $headers = null, string $body = null, int $webhookId = null)
+    public function request(string $method, string $url, array $headers = null, string $body = null, int $webhookId = null, string $debounceKey = null)
     {
         $db = Craft::$app->getDb();
+
+        if ($debounceKey !== null) {
+            // See if there is an existing pending request with the same key
+            $requestId = (new Query())
+                ->select(['id'])
+                ->from(['{{%webhookrequests}}'])
+                ->where([
+                    'debounceKey' => $debounceKey,
+                    'status' => self::STATUS_PENDING,
+                ])
+                ->scalar();
+
+            // If so and we get a lock on it, update that request instead of creating a new one
+            if ($requestId && $this->_lockRequest($requestId)) {
+                $db->createCommand()
+                    ->update('{{%webhookrequests}}', [
+                        'method' => $method,
+                        'url' => $url,
+                        'requestHeaders' => $headers ? Json::encode($headers) : null,
+                        'requestBody' => $body,
+                        'dateCreated' => Db::prepareDateForDb(new \DateTime()),
+                    ], [
+                        'id' => $requestId,
+                    ], [], false);
+                $this->_unlockRequest($requestId);
+                return;
+            }
+        }
+
         $db->createCommand()
             ->insert('{{%webhookrequests}}', [
                 'webhookId' => $webhookId,
+                'debounceKey' => $debounceKey,
                 'status' => self::STATUS_PENDING,
                 'method' => $method,
                 'url' => $url,
@@ -235,10 +290,27 @@ public function request(string $method, string $url, array $headers = null, stri
             ], false)
             ->execute();
 
-        Craft::$app->getQueue()->push(new SendRequestJob([
+        $this->_pendingJobs[] = new SendRequestJob([
             'requestId' => $db->getLastInsertID('{{%webhookrequests}}'),
             'webhookId' => $webhookId,
-        ]));
+        ]);
+
+        if (count($this->_pendingJobs) === 1) {
+            Craft::$app->on(Application::EVENT_AFTER_REQUEST, [$this, 'pushPendingJobs'], null, false);
+        }
+    }
+
+    /**
+     * Pushes any pending jobs to the queue.
+     *
+     * @since 2.3.0
+     */
+    public function pushPendingJobs()
+    {
+        $queue = Craft::$app->getQueue();
+        while ($job = array_shift($this->_pendingJobs)) {
+            $queue->push($job);
+        }
     }
 
     /**
@@ -279,9 +351,7 @@ public function getRequestData(int $requestId): array
     public function sendRequest(int $requestId): bool
     {
         // Acquire a lock on the request
-        $lockName = 'webhook.' . $requestId;
-        $mutex = Craft::$app->getMutex();
-        if (!$mutex->acquire($lockName)) {
+        if (!$this->_lockRequest($requestId, 1)) {
             throw new Exception('Could not acquire a lock for the webhook request ' . $requestId);
         }
 
@@ -306,7 +376,8 @@ public function sendRequest(int $requestId): bool
 
         $startTime = microtime(true);
         try {
-            $response = Craft::createGuzzleClient()->request($data['method'], $data['url'], $options);
+            $response = Craft::createGuzzleClient($this->getSettings()->guzzleConfig)
+                ->request($data['method'], $data['url'], $options);
             $success = true;
         } catch (RequestException $e) {
             $response = $e->getResponse();
@@ -330,7 +401,7 @@ public function sendRequest(int $requestId): bool
             ->execute();
 
         // Release the lock
-        $mutex->release($lockName);
+        $this->_unlockRequest($requestId);
 
         return $success;
     }
@@ -418,4 +489,14 @@ public function getAllFilters(): array
 
         return $event->types;
     }
+
+    private function _lockRequest(int $requestId, int $timeout = 0): bool
+    {
+        return Craft::$app->getMutex()->acquire("webhook:$requestId", $timeout);
+    }
+
+    private function _unlockRequest(int $requestId): bool
+    {
+        return Craft::$app->getMutex()->release("webhook:$requestId");
+    }
 }

src/SendRequestJob.php

@@ -62,7 +62,7 @@ public function execute($queue)
             $settings = Plugin::getInstance()->getSettings();
             if ($attempts < $settings->maxAttempts) {
                 Craft::$app->getQueue()
-                    ->delay($settings->attemptDelay)
+                    ->delay($settings->retryDelay)
                     ->push(new self([
                         'requestId' => $this->requestId,
                     ]));