Add "Clean-param" directive in robots.txt

[ruslanyar]

Goals

Include "Clean-param" directive to resolve by resolveRobots

Non-Goals

No response

Background

I am facing with problem that when generating the robots.txt file, the "clean-param" directive is ignored.
Yandex recommends using "Clean-param" for cleaning URLs contains GET parameters because this directive allows transfer some accumulated metrics to the main URL or website.

Proposal

Add cleanParam? to the RobotsFile type and update the function that handles robots.txt rules.
I am interested in contributing.

[gormonn]

Any workarounds?

[ruslanyar]

Unfortunately, I haven't found it yet

[AhmadReshadarm]

Here is a way to work around this and it works for me. Hope this help)

{ userAgent: 'Yandex', disallow: '\nClean-param: ref /some_dir/get_book.pl', }