Create fallback to using HTML5 canvas if TK isn't available

[TheNotary]

So ever since Peterc first introduced this gem to me, I thought it was really neat, but I could never get it to work on my platfrom due to the status of the tk package on windows and linux. It must work much nicer on macs? I would like to humbly propose we implement the canvas in HTML5 in the case that the TK package isn't accessible.

How about this:

• In the gemspec, we conditionally require tk, but failing that we require HTML5 deps (so bundle install and tests can run for non-tk users)
• In the Gem itself, we require 'tk' and failing that, we instantiate a 1:1 compatible interface that draws to an HTML5 canvas instead of TK (so alternative TkRoot, TkCanvas, TkcLine, TkcOval, TkTimer, TkSleep static methods are loaded)

Now you're probably wondering at this point, "How is old Notary Sojac going to draw to an HTML5 canvas in a ruby library." And that is where it gets tricky, but not too tricky.

Rendering to an HTML Canvas instead of a TK Canvas

Architecture

• When we start trtl, it will actually boot some web app stuff in child threads (assuming TK isn't available)
  ** Sinatra will begin listening on port 4567.
  ** Faye, my websocket server of choise will start listening on 9696
  ** Sinatra will serve a page containing some javascript that will connect to faye on 9696

System Flow

• From within the existing ruby code, when we call, say, TkRoot, that will actually send data to faye, such that any active javascript clients will be notified of a graphics call
• Then in javascript, any commands such as TkRoot will call the javascript-native functionality to, in this case, create a canvas element on the HTML page.
• All commands sent over faye will be cached and made available to Faye, such that when the user closes there browser to 127.0.0.1:4567 and reopens it, they will see the data they left off with.

What do you think? This proposal is minimally invasive (requires only the addition of lines of code: an extra .rb file tucked away, Gemfile, Gemspec, require conditionality, etc). If you're really excited about the idea, we can re-write the Tk* function calls to Canvas* calls and upon initialization, define what Canvas* translates to (either the actual TK library commands or the ruby commands that delegate that same functionality to javascript).

P.S.
Peter, I tried to get a hold of you over email, but looks like you're enjoying the summer holiday. I hope you're having a good time in Sweden! Maybe we can touch base when you get back? I'll gently spam you another email when you're back to your routine. Also is it possible for folks to arrange for a seat at your App Academy lectures?

[peterc]

I was on holiday, yeah, but unfortunately I don't recall getting an email from you. Feel free to resend to peter@peterc.org :) Things to my GitHub email account are more likely to go to spam considering how many people spam these addresses nowadays :( Regarding App Academy, no, I'm not currently doing anything with them.

On the idea presented here, it sounds like a good one! I wonder if there's already some sort of Canvas abstraction for Ruby out there? If so, using it would be a good idea. But otherwise it's not particularly complex and I'm keen on your overall idea for increasing the accessibility of the code (although I must admit I'd generally just jump ship to JavaScript for something like this if browser support were considered a must).

[TheNotary]

Btw, I sent you that email from wastyx@gmail.com (an old gamer handle). I'll follow up there.

"Some sort of canvas abstraction for ruby"
I think this will be the first effort in bridging ruby CLI and web canvases.

So with your comments, I'll outline my plan for adding HTML support to this gem:

• Create an html/javascript project to implement the TK rendering in js
• Add web sockets
• Build a gem, say, RubyTube for handling the sinatra server, websocket server, and rendering code

RubyTube's API might look something like

rt = RubyTube.new

# Point RubyTube to where the html public directory is
rt.www_root = "gui/"

# Possibly some routes setup for Sinatra
# rt... 


# start sinatra, faye/websockets and prints "browse to 127.0.0.1:4567"
rt.run!

Finally, RubyTube is included into the trtl and we can see it in action. I might need to do something to 'cache' commands, those 10.times {up} commands might give the websockets some trouble, but this is unknown to me. Once we can see it in action, if you think it's progress, I can put together a follow up commit to pull out the TK code alltogether (instead of writing code to support both).

[TheNotary]

I've got a working prototype on github now. Checkout the html-canvas branch: html-canvas

So to use it now:

$  irb
require 'trtl'

>  fd
>  left 90
>  fd

After you make a new turtle (or issue some commands to InteractiveTurtle), it will spin up the web/websocket servers (I haven't cleaned up the garbage output that spams the screen yet).

Now you should be able to navigate to 127.0.0.1:4567 and see a mostly blank canvas (there should be a trtl arrow). In the irb shell, you can control the trtl, and immediately, your updates should be reflected in the browser view thanks to the websocket server on port 4555.

What do you think of the overall approach?

'''Edit:''' I fixed InteractiveTurtle btw and edited out debugging comments =)

[peterc]

Unfortunately I haven't got time to check it out properly yet but from the way you describe it sounds like a reasonable approach, so yes!

[TheNotary]

I threw together a quick demo of it in action. The one problem is that the tree example is kind of hard on the rendering system. Sometimes it goes through fine, and sometimes it crashes fire fox.

Btw, reduce your volume level before starting the demo, youtube seems to have hidden the volume controls somewhere new in the video manager.

https://www.youtube.com/watch?v=P3ySQWH47Ks

[peterc]

That is really cool!!

On Mon, Aug 17, 2015 at 4:56 PM, TheNotary notifications@github.com wrote:

I threw together a quick demo of it in action. The one problem is that the
tree example is kind of hard on the rendering system. Sometimes it goes
through fine, and sometimes it crashes fire fox.

Btw, reduce your volume level before starting the demo, youtube seems to
have hidden the volume controls somewhere new in the video manager.

https://www.youtube.com/watch?v=P3ySQWH47Ks

—
Reply to this email directly or view it on GitHub
#2 (comment).

Peter Cooper
http://peterc.org/
http://twitter.com/peterc

[TheNotary]

Ok, I've made all the changes to Magic Mirror that I think I'm going to be making in the near future, and also made awesome optimizations to the javascript rendering system within trtl.

TheNotary@0582b2f

I think now we wait till you have a moment to look things over, check that you think this way is so good that it can replace the TK dependency, and add any spec changes you'd like to see. After that I'll either completely remove the old references to TK in trtl, or hook up a fallback solution if preferred. Finally I'll issue a final pull request and we call it a feature =)

[peterc]

OK, I've finally given it a go. Some observations:

• Initially you said this would be a fallback if Tk wasn't present, but it appears to be replacing Tk at this point. Is this something that'll be resolved in the cleanup?
• It's been a long while since I looked at the Trtl code but some DRYing up would probably be useful. I'm sure the classes in canvas.rb could look a lot better with some refactoring and inheritance.
• It worked, which is great, but it did seem a bit slow. The messages being sent over the wire don't seem to be particularly efficient, although I appreciate not a lot may be possible to do here without significant work, so it's not a killer.

[TheNotary]

It's great to hear back from you, Peter. In order:

• My apologies, I may have slightly misinterpreted you when you said ...I'd generally just jump ship to JavaScript... but I see what you meant now. The code is still in a good position to be set back into a fall back pattern and I'll commence that now --it's mostly a matter of a couple conditionals and uncommenting/ commit history copy/pasting.
• I'll begin a refactoring pass on the canvas class to clean up those multiple occurrences of meta-programming blocks.
• The performance can actually be improved in an elegant way. It's not so much passing the data down the virtual wire (which must be >= gigabit speed?) it's actually the time that the javascript client and websocket server waste queuing messages and overhead to send over a single command. If I buffered it to only send the websocket packet every 5 commands, the tree would finish 5x faster I suspect (the tree results in thousands of trtl movement/ paint commands, btw). The tree was SO time consuming to test, lol. The trick will be keeping the quick responsiveness of sending single turtle commands over the ruby shell, while still allowing some buffering to take place in scenarios where trees and tree-like artworks are being rendered.

I want this feature to be perfectly inline with your vision for the project, so thank you for getting the feedback to me. I'll get back to you with a new commit within the next week or so based on this input.

[TheNotary]

Hi Peter,

• The code is now in a fallback pattern, favoring TK. There's a caveat: I haven't been able to get TK on my system yet, so the TK side of things is untested. If it doesn't 'just work' on your end, let me know along with the error and I'll try to set up an environment on a windows machine. I'm hopeful the time I spent double checking the TK code was worth while and it will run fine on your end.
• I've cleaned up the code. The app will try to require 'tk' and upon a LoadError, it will instead load MagicMirror and go the MagicMirror route with things. highlighting on first lines of lib/trtl.lib
• I've increased the performance by caching when the websocket server is 'bogged down'. The tree will render in 3 seconds on my machine. That performance is on par with what html5's canvas is capable of at present time (google's is having trouble figuring out how to make concurrent thread safe writes to it I think.. probably an absolute nightmare to work on).

Btw, what is the aproximate rendering time of the tree example with TK support?

Edit: I did my best to setup an env and do gem install trtl on windows w/ ruby 1.9.3 (and 2.x) and active tlc, but got an error =( http://pastebin.com/H67XK9RS If you have tips on getting this functional on a mac (I'll try an apple store after a relocation I'm in the midst of) let me know and I can both try it out and also drop a paragraph touching on TK setup in the readme. Sorry I can't do a better job testing it, it's ironic that my impedes to add the feature was due to the fact that I could never get an operational install going on my end, heh. Anyway, I'd also like to add no rush, we're both a bit on the busy side I think.

Thanks again for that letter, btw!!! =)