Piper/python3 support

[pipermerriam]

#132

What was wrong?

This project did not support python3

How was it fixed.

• Expanded Travis CI testing to include 2.7, 3.4 and 3.5
• Converted division operators to the floored division operator //
• Prefixed many strings with b to make them byte strings.
• Convert use of print statements to python3 compatable function calls.
• Change import statements to use absolute import syntax.
• Fix a few tests that were non-deterministic and intermittently failing.

Cute animal picture

[pipermerriam]

Depends on ethereum/serpent#102

[heikoheiko]

hi @pipermerriam thank you for this effort.
do you have an eye on the performance impacts?
i'm a bit afraid of py2/py3 logic overhead in inner loops.

[pipermerriam]

@heikoheiko I'm working on getting a fully passing test suite first. Then I'll deal with any performance problems.

[heikoheiko]

Then I'll deal with any performance problems.
ok, so comparing the performance could work by testing different branches / tags.

[pipermerriam]

This should NOT be merged. This is just here to get the python3 tests running until I can get ethereum/serpent#102 merged.

[pipermerriam]

ethereum/solidity#651 is causing some of the current test failures within the test_solidity.py tests. Doesn't seem fixable until a new solc release is made.

[pipermerriam]

@heikoheiko here is some profiling data from the ethereum/tests/profile_vm.py script.

https://gist.github.com/pipermerriam/3d12e18097c0d6be72956bd19a24eac7

[pipermerriam]

Alrighty!

The CI failures are all related to ethereum/solidity#651.

This is ready for review and theoretically merging if people are ok with it.

This depends on ethereum/serpent#102 being reviewed and merged. Alternatively I can remove the code that references my fork/branch which will cause CI to start failing in more places but will allow this to be merged independently.

[pipermerriam]

This should close #132, #238, #254, #329, and #347

[pipermerriam]

ok, All of the test failures are related to the solidity stdin bug (see ethereum/solidity#651). You can view these under the travis run for d3e61e1 which can be found https://travis-ci.org/ethereum/pyethereum/builds/138391919 (note this commit was rebased away but you can still see the test run)

I've just pushed 49e570f which reverts the serpent dependency back to the official repo. It was previously pointed at the branch represented by ethereum/serpent#102 which implements python3 compatability. These two repositories have a circular dependency so my feeling is that we should do the following.

• If the code is satisfactory, merge this branch as it currently stands.
• Update Python 3 Support serpent#102 to point to the newly merged pyethereum code.
• Get Python 3 Support serpent#102 merged.

Soliciting feedback on this plan and hoping to move it forward in a timely fashion.

[pipermerriam]

Benchmark Data

	# Function Calls	Time	% Change
Python 2 (with py3 support)	3668001	3.590	-10%
Python 3	2261856	2.113	n/a
Baseline (develop branch)	3668001	3.227

[heikoheiko]

Thank you. Looks way better than what I expected.

[ulope]

Those assignments (except uncle.number) could probably be pulled out of the loop (assuming self.delta_balance() doesn't modify self.number).

[pipermerriam]

Good call. Fixed

[ulope]

Looks very good to me (apart from the minor nit picks above).

One thing I would like to change a bit is the travis and tox configuration.
I've put the config I'd suggest into a gist.

There are a few reasons for doing it this way:

• We retain the ability to locally use tox to run only specific tests (through the use of {posargs}). This is very useful while working on a specific issue for example.
• Coverage is measured across all runs on all supported Python versions. This way we don't get any false positives due to version specific code.

[pipermerriam]

@ulope I've tried to adapt your examples in a manner that tries to maximize the ability to use tox to runt he tests locally and on travis while still getting full coverage reports. See 5ca58a6

I'm still probably gonna have to futz with it a bit to get it running smoothly but I'd be curious to get your feedback on whether this is inline with what you were thinking.

[pipermerriam]

@ulope I actually undid the bytearray_to_bytestr change. There are some places where it doesn't actually get passed an actual bytearray but rather an array of integer byte values. bytes(value) doesn't have the same behavior in this case across python2/3.

[pipermerriam]

Additional benchmark results from python ethereum/tests/stress_test.py:

• Baseline (python2, no support for python 3) - 9.238459
• Python 2 (with py3 support) - 9.285555
• Python 3 - 22.872115

[ulope]

@pipermerriam Yeah, ok in that case we don't have a choice. Bummer. Regarding the tox/travis config, I don't think it's critical to this PR. If it works ok just leave it for now and then we can have another look and possibly optimize it later.

[konradkonrad]

Baseline (python2, no support for python 3) - 9.238459
Python 2 (with py3 support) - 9.285555

thanks @pipermerriam!
also thanks @ulope for your review !
Overall this looks good to me! -- the failing solidity tests make me sad though..
Any open questions before the merge @ulope @heikoheiko?