Add support for Ethereum "Events"

[davux]

Awesome stuff!

A very neat addition would be subscribing to specific events using the event-watching functions exposed by the JSON-RPC interface.

This way, you could litterally watch for specific events out of a given contract, it would be perfect.

[consulthys]

Thanks @davux ! Any way for you to submit a PR?

[davux]

I've never done any Ruby programming but I can definitely try, or I could help you with information on how Ethereum events work.

[consulthys]

Any information you can share is golden :-)

[davux]

So events is what gets emitted in a function in a smart contract. For example a function transferFrom() would emit at some point a Transfer(from, to, amount) to announce that a token was. transferred between 2 participants. Anything can be an event, they're very simple to write (see link in the issue).

Some arguments of an event can be marked as "indexed", which means they can be filtered on. For example, from and to would be marked as indexed, while it wouldn't make sense for amount.

An event being emitted results in the writing of a log to the transaction. There are various JSON-RPC methods (see other link in the issue) to access those. Indexed arguments are stored as topics, while non-indexed arguments are stored as simple data. You can search/filter logs by topic.

Since logs are so well optimised to be searched on, you don't even have to read every transaction and filter the logs yourself – instead, the JSON-RPC API provides a few methods to watch specific topics directly. I'm not sure how that would translate to the logstash filter's logic: a "log" granularity maybe? At any rate, the input would need an additional topics argument which would contain the topics to be filtered on. See https://ethereumbuilders.gitbooks.io/guide/content/en/ethereum_json_rpc.html#eth_newfilter

HTH, let me know if you need me to clarify anything specifically.

[consulthys]

Thanks @davux for this extensive explanation. Time permitting, I will get into this in early summer.

[mattvick]

Hi @consulthys @davux, Have you either of you made any progress with implementing support for subscribing to Ethereum events?

I would like to help with this and would be happy to discuss with you the preferred approach before starting. Below are some starter questions:

• Will it be possible to subscribe to all events or just specific events? Are there underlying restrictions in JSON-RPC API that will influence this?
• Should events be treated as separate objects or nested under transactions?
• @davux what do you mean by 'a "log" granularity'?

[davux]

No news on my side, unfortunately.

About your questions:

• You can check web3 filters and the watch() function. Those provide flexibility about which events you want to subscribe to, so the logstash input can have that bit as an option.
• In this mode, 1 logstash event == 1 transaction log. Since your base objects are logs, the transaction would be represented as a property of the log instead of the other way around. Note that nothing prevents us from providing all the information we can about the transaction as a full object, and it's up to the logstash user to use that object in the filters or to just ignore it.
• I mean for now there are two granularities that denote two possible base objects: block or transaction. A block contains many transactions so the granularity means your zoom level. Likewise, a transaction contains many logs. Block > Transaction > Log. I hope I'm making sense.

[QuentinCazelles]

Hi ! How are you ?

I am trying to do what you are talking about here !
I can not make a PR yet but my code should be working.

You can find the repo here :
https://github.com/QuentinCazelles/logstash-input-blockchain

Are you still working on this (= ?

Thank you

[davux]

Hi @QuentinCazelles, that's great! Did you push your code related to smart contract events yet? I can only see "block" and "transaction" as supported granularities but nothing about smart-contract-level events.

[QuentinCazelles]

Hello @davux , it seems I pushed it several days ago but I had not pushed the new readme ! My bad !
Anyway it is now on my repo with an updated readme to take into account the event part.

The modification is very light for now.
Being a beginner in Ruby (and in this case especially in JRuby) I have much trouble to use an ethereum library correctly.

I managed to make my code work by switching digest-sha3 (which is in C) to keccak-pure-ruby.
I had to switch at the plugin level, at the ethereum library level and at one of its dependency level ...

I use https://github.com/EthWorks/ethereum.rb
Which does not have a maintainer anymore ...
I only use it to decode event data, I have not found something to replace it yet.

(the ethereum.rb dependency using digest-sha3 is eth gem)

[QuentinCazelles]

Hi everyone

If you're still interested take a look at my plugin's repo.

It still have a trouble with digest-sha3/keccak-pure-ruby gem but it is working with a conf like

input {
  blockchain {
    protocol => "ethereum"
    host => "localhost"
    port => 8545
    granularity => "event"
    contract_path => "/home/Desktop/MyContract.abi"
    events_watched => ["MyEvent1", "MyEvent2"]"
    network_id => 1
  }
}

Thanks for your feedback