UrlValidator fails with Acute Accent "`" despite being a valid URL

[robjbrain]

• Laravel Version: 8.61.0
• PHP Version: 7.4.7
• Database Driver & Version: [email protected]

Description:

A URL with an acute accent "`" (Unicode "U+00B4") fails despite being a valid URL.

Steps To Reproduce:

$validator = Validator::make([
  'test_url' => "https://www.mediafire.com/file/qwcdz6hx8anbfjo/Spain_Liga_Santander_SS´21-22.rar/file"
], [
    'test_url' => 'url'
]);

$validated = $validator->validate();

// Illuminate\Validation\ValidationException with message 'The given data was invalid.'

However this is a valid URL accessible by all web browsers and able to be submitted by a user into a form requiring a URL. E.g: https://www.mediafire.com/file/qwcdz6hx8anbfjo/Spain_Liga_Santander_SS´21-22.rar/file

[robjbrain]

Link to original issue: #39441

@derekmd suggested that the URL needed to be

That part of the slug may need to go through urlencode():

Obviously we can't expect users to be doing this as they're just going to copy and paste whatever the browser shows them.

Can I please clarify the best way to sanitise the input before it ever reaches the validator? I'm guessing we just need to copy the method used by Illuminate\Foundation\Http\Middleware\TrimStrings and \Illuminate\Foundation\Http\Middleware\ConvertEmptyStringsToNull something like:

use Illuminate\Foundation\Http\Middleware\TransformsRequest;

class UnicodeReplacements extends TransformsRequest
{
    protected array $replacements = [
        '`' => '%C2%B4'
    ];

    /**
     * Handle an incoming request.
     *
     * @param  \Illuminate\Http\Request  $request
     * @param  \Closure  $next
     * @return mixed
     */
    public function handle($request, Closure $next)
    {
        return parent::handle($request, $next);
    }

    /**
     * Transform the given value.
     *
     * @param  string  $key
     * @param  mixed  $value
     * @return mixed
     */
    protected function transform($key, $value)
    {
        return is_string($value) ? str_replace(array_keys($this->replacements), $this->replacements, $value) : $value;
    }
}

However i'm not 100% sure if this is the best way to do it, as it would sanitise all inputs not just URLs. But I can't find any documentation suggesting how to sanitise only some inputs.

Can you clarify the correct Laravel way to handle this?

[derekmd]

I imagine there's a PHP Composer package to handle this but I can't find one since "multibyte", "encode", "sanitize", and "input" are all such generic computing terms. Writing a URL string fixer, it would need to avoid double-encoding / characters, ?/=/& from ?query=strings&more=parameters, url#hash, etc.

Instead of pulling apart and reconstructing the URL segments, the simplest fix may be to replace multibyte characters to make them ready for HTTP requests.

use Illuminate\Foundation\Http\Middleware\TransformsRequest;

class UrlEncodeMultibyteInput extends TransformsRequest
{
    /**
     * The request parameter names this sanitizer handles.
     */
    protected $only = [
        'url',
    ];

    protected function transform($key, $value)
    {
        return in_array($key, $this->only) && is_string($value)
            ? $this->replace($value)
            : $value;
    }

    protected function replace($value)
    {
        return collect(
            preg_split('//u', $value, -1, PREG_SPLIT_NO_EMPTY)
        )->map(function ($c) {
            if (strlen($c) > 1) {
                return urlencode($c);
            }

            return $c;
        })->join('');
    }
}

Apply this middleware and validation rule url to a form request and make users themselves adjust for any other fixes (if there are any?)

[robjbrain]

To anyone else finding this, this is what I wound up with:

Create this file: app\Http\Middleware\CleanUrlStrings.php

<?php

namespace App\Http\Middleware;

use Illuminate\Foundation\Http\Middleware\TrimStrings;
use Illuminate\Support\Str;

class CleanUrlStrings extends TrimStrings
{
    /**
     * @var array|string[]
     */
    protected array $replacements = [
        '´' => '%C2%B4',
        '`' => '%60'
    ];

    /**
     * Transform the given value.
     *
     * @param  string  $key
     * @param  mixed  $value
     * @return mixed
     */
    protected function transform($key, $value)
    {
        //https://github.com/laravel/framework/discussions/39444
        return Str::startsWith($value, 'http')
            ? str_replace(array_keys($this->replacements), $this->replacements, $value)
            : $value;
    }
}

Edit this file: app\Http\Kernel.php

protected $middleware = [
   //...
   \App\Http\Middleware\TrimStrings::class,
   // Add this line
   \App\Http\Middleware\CleanUrlStrings::class,
   //...
];