Compatibility and status of project

[dennismende]

Hi everyone,

we are using Laravel 5.4.* together with plastic:latest and Elasticsearch 5.6.10 but we receive errors for every single command that we execute.

What we did so far is:

1. Configuring $searchable, $documentType, $documentIndex on a model
2. Creating a mapping for this model
3. Configuring an index in the plastic conf for the model

Now if we try to execute php artisan mapping:run we get the error:

{"error":{"root_cause":[{"type":"security_exception","reason":"missing authentication token for REST request [/product/product/_mapping]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}}],"type":"security_exception","reason":"missing authen
  tication token for REST request [/product/product/_mapping]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}},"status":401}

If we try to execute php artisan plastic:populate --index=product we get the following error:

[Elasticsearch\Common\Exceptions\BadRequest400Exception (401)]

The status of the Elasticsearch cluster is the following:

epoch      timestamp cluster                                status node.total node.data shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent
1533124324 11:52:04  test-cluster yellow          1         1      5   5    0    0        5             0                  -                 50.0%

What are the current compatibilities of this package?

Thanks in advance.

[xtrasmal]

Hi there,

I do not use this package, but was looking around the web for elasticsearch Laravel packages.

When I looked at you question, I could see that you are trying to connect to an instance that is using token based security(https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-tokens.html). After looking at the code of Plastic, I can tell you that it is not possible to set authentication headers at this moment, due to missing config options. Also even if you had these config options, there is no functionality to set these connection parameters.

Let me explain where to find it and how to fix it:

Take a look at: https://github.com/sleimanx2/plastic/blob/master/src/Connection.php#L247
There you can see this code:

    private function buildClient(array $config)
    {
        $client = ClientBuilder::create()
            ->setHosts($config['hosts']);
        if (isset($config['retries'])) {
            $client->setRetries($config['retries']);
        }
        if (isset($config['logging']) and $config['logging']['enabled'] == true) {
            $logger = ClientBuilder::defaultLogger($config['logging']['path'], $config['logging']['level']);
            $client->setLogger($logger);
        }
        return $client->build();
    }

It configures the Elasticsearch/Client by setting the folllowing:

• $client->setHosts($config['hosts']);
• $client->setRetries($config['retries']);
• $client->setLogger($logger);

This is your problem; Because it needs run another method on $client....which it does not at the moment.

It should call the following as well:

• $client->setConnectionParams($config['params]);

When you look at the code of the ClientBuilder's build method:
https://github.com/elastic/elasticsearch-php/blob/master/src/Elasticsearch/ClientBuilder.php#L475
You can see that it set's the headers at some point.

        if (is_null($this->connectionFactory)) {
            if (is_null($this->connectionParams)) {
                $this->connectionParams = [];
            }
            // Make sure we are setting Content-Type and Accept (unless the user has explicitly
            // overridden it
            if (isset($this->connectionParams['client']['headers']) === false) {
                $this->connectionParams['client']['headers'] = [
                    'Content-Type' => ['application/json'],
                    'Accept' => ['application/json']
                ];
            } else {
                if (isset($this->connectionParams['client']['headers']['Content-Type']) === false) {
                    $this->connectionParams['client']['headers']['Content-Type'] = ['application/json'];
                }
                if (isset($this->connectionParams['client']['headers']['Accept']) === false) {
                    $this->connectionParams['client']['headers']['Accept'] = ['application/json'];
                }
            }
            $this->connectionFactory = new ConnectionFactory($this->handler, $this->connectionParams, $this->serializer, $this->logger, $this->tracer);
        }

So I suggest that you do the following:

First Update your config

        /*
        |--------------------------------------------------------------------------
        | Hosts
        |--------------------------------------------------------------------------
        |
        | The most common configuration is telling the client about your cluster: how many nodes, their addresses and ports.
        | If no hosts are specified, the client will attempt to connect to localhost:9200.
        |
        */
        'hosts'   => [
            env('PLASTIC_HOST', '127.0.0.1:9200'),
        ],

       'params' => [
                 'client' => [
                              'headers' => [
                                   'Authentication' => [ 'Bearer <your_token>' ]
                              ]                  
                 ]
      ]

You need to log in and obtain an acces_token, then use laravel to update the config at runtime.
So that the Bearer token gets set.
see: https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-tokens.html

Second change the code

    private function buildClient(array $config)
    {
        $client = ClientBuilder::create()
            ->setHosts($config['hosts']);
        if (isset($config['retries'])) {
            $client->setRetries($config['retries']);
        }

        // ADD THIS
        if (isset($config['params'])) {
            $client->setConnectionParams($config['params']);
        }

        if (isset($config['logging']) and $config['logging']['enabled'] == true) {
            $logger = ClientBuilder::defaultLogger($config['logging']['path'], $config['logging']['level']);
            $client->setLogger($logger);
        }
        return $client->build();
    }

Then you can run the mapping:run with headers set.

@sleimanx2 @dennismende

[xtrasmal]

I have made a pull request, which explains it: #172

The only thing you need to do is to set the config headers on runtime.
This might not be possible for existing config. Like this person experienced. https://laracasts.com/discuss/channels/laravel/change-environment-variables-on-the-fly-in-a-command

That person says that it was only possible to set something totally new. If that is the case, you should not update your config, but set it at runtime.

You should than create a new command, artisan make:command CustomRun
And add auth and config stuff:

<?php namespace App\Console\Mapping;

use Illuminate\Console\ConfirmableTrait;
use Sleimanx2\Plastic\Mappings\Mapper;
use Sleimanx2\Plastic\Console\Mapping\BaseCommand;
class CustomRun extends BaseCommand
{
    use ConfirmableTrait;

    /**
     * The console command name.
     *
     * @var string
     */
    protected $signature = 'mapping:customrun {--index=} {--database=} {--step} {--force}';

    /**
     * The console command description.
     *
     * @var string
     */
    protected $description = 'run the remaining mappings';

    /**
     * @var Mapper
     */
    private $mapper;


    private $authService;

    /**
     * Reset constructor.
     *
     * @param Mapper $mapper
     */
    public function __construct(Mapper $mapper, $authService)
    {
        parent::__construct();

        $this->mapper = $mapper;
        $this->authService = $authService;
    }

    /**
     * Execute the console command.
     */
    public function handle()
    {
        if (!$this->confirmToProceed()) {
            return;
        }

        $token = $this->authService->accesstoken();

        Config::set('plastic.connection.params', [
                'client' => [
                    'headers' => [
                        'Authentication' => [ 'Bearer '.$token ]
                    ]
                ]
        ]);

        $this->prepareDatabase();

        $path = $this->getMappingPath();

        $this->mapper->run($path, [
            'step'  => $this->option('step'),
            'index' => $this->option('index'),
        ]);

        // Once the mapper has run we will grab the note output and send it out to
        // the console screen, since the mapper itself functions without having
        // any instances of the OutputInterface contract passed into the class.
        foreach ($this->mapper->getNotes() as $note) {
            $this->output->writeln($note);
        }
    }

    protected function prepareDatabase()
    {
        $this->mapper->setConnection($this->option('database'));

        if (!$this->mapper->repositoryExists()) {
            $options = ['--database' => $this->option('database')];

            $this->call('mapping:install', $options);
        }
    }
}

[xtrasmal]

Ok...I am going on with my life. I was accidentally inspired to take a closer look.

Goodluck 😅