Add --ext=go to bundle gem

[sue445]

What was the end-user or developer problem that led to this PR?

ref. https://github.com/orgs/rubygems/discussions/8128

Original idea is @mmasaki 's https://rubykaigi.org/2015/presentations/mmasaki/

At the time of his talk (2015), Go did not have a standard package manager.
So I think he had to create his own binding to call CRuby functions from Go.

However, there is now a standard package manager called Go module.

I auto-generated (almost) all of Go's bindings automatically from ruby.h. (about 1,100 functions)

https://github.com/ruby-go-gem/go-gem-wrapper/blob/v0.2.0/ruby/function_generated.go

Pros

Go has some excellent features that other languages do not have, such as goroutine and Channels.

I believe these would be more convenient if these features were available from Ruby.

Below is a benchmark comparing Ractor and goroutine.

In my benchmark, goroutine is about x26-28 faster than Ractor 💨

$ ruby tarai.rb
go version go1.23.2 darwin/arm64
ruby 3.3.5 (2024-09-03 revision ef084cc8f4) [arm64-darwin23]
Warming up --------------------------------------
          sequential     1.000 i/100ms
   parallel (Ractor)     1.000 i/100ms
parallel (goroutine)     1.000 i/100ms
Calculating -------------------------------------
          sequential      0.019 (± 0.0%) i/s    (51.42 s/i) -      1.000 in  51.417686s
   parallel (Ractor)      0.059 (± 0.0%) i/s    (17.02 s/i) -      1.000 in  17.022087s
parallel (goroutine)      1.677 (± 0.0%) i/s  (596.20 ms/i) -      9.000 in   5.365863s

Comparison:
parallel (goroutine):        1.7 i/s
   parallel (Ractor):        0.1 i/s - 28.55x  slower
          sequential:        0.0 i/s - 86.24x  slower

What is your fix for the problem, implemented in this PR?

Support --ext=go in bundle gem

bundle gem --ext=go creates the following files

• ext/newgem/go.mod
• ext/newgem/extconf.rb (Patched for Go-gem)
• ext/newgem/newgem.go
• ext/newgem/newgem.c (Patched for Go-gem)
• Gemfile (Add https://rubygems.org/gems/go_gem as dependency)
• .github/workflows/main.yml (Install Go in GitHub Actions)
• .gitlab-ci.yml (Install Go in GitLab CI)
• .circleci/config.yml (Install Go in CircleCI)

Please refer to spec files and .tt files for the contents of the generated file.

Skeleton generated by this patch utilizes https://github.com/ruby-go-gem/go-gem-wrapper functionality. This was all created by me.

go-gem-wrapper offers following two for Go-gem.

• https://pkg.go.dev/github.com/ruby-go-gem/go-gem-wrapper
  • Go module
  • Go's bindings automatically from ruby.h
• https://rubygems.org/gems/go_gem
  • Helpers for compiling Go extensions for ruby
  • Provide create_go_makefile for extconf.rb

go-gem-wrapper is similar to the https://github.com/oxidize-rb/rb-sys in Rust-gem.

Make sure the following tasks are checked

• Describe the problem / feature
• Write tests for features and bug fixes
• Write code to solve the problem
• Make sure you follow the current code style and write meaningful commit messages without tags

[welcome]

Thanks for opening a pull request and helping make RubyGems and Bundler better! Someone from the RubyGems team will take a look at your pull request shortly and leave any feedback. Please make sure that your pull request has tests for any changes or added functionality.

We use GitHub Actions to test and make sure your change works functionally and uses acceptable conventions, you can review the current progress of GitHub Actions in the PR status window below.

If you have any questions or concerns that you wish to ask, feel free to leave a comment in this PR or join our #rubygems or #bundler channel on Slack.

For more information about contributing to the RubyGems project feel free to review our CONTRIBUTING guide

[simi]

Hello @sue445, I tried to test locally, but I can't get gem installed. Is there anything I do wrong?

ruby spec/support/bundle.rb gem --ext=go goryba
cd goryba
# edit ext/goryba/go.mod with proper module name
cd ext/goryba
go mod tidy # get sum file
cd ../..
git add ext # add sum file
gem build --force && gem install goryba-0.1.0.gem

  Successfully built RubyGem
  Name: goryba
  Version: 0.1.0
  File: goryba-0.1.0.gem
Building native extensions. This could take a while...
ERROR:  Error installing goryba-0.1.0.gem:
        ERROR: Failed to build gem native extension.

    current directory: /home/retro/.gem/ruby/3.3.5/gems/goryba-0.1.0/ext/goryba
/home/retro/.rubies/ruby-3.3.5/bin/ruby extconf.rb
checking for whether -fvisibility=hidden is accepted as CFLAGS... yes
checking for go... yes
creating Makefile

current directory: /home/retro/.gem/ruby/3.3.5/gems/goryba-0.1.0/ext/goryba
make DESTDIR\= sitearchdir\=./.gem.20241030-3456757-1g1rg1 sitelibdir\=./.gem.20241030-3456757-1g1rg1 clean
Makefile:271: warning: overriding recipe for target 'goryba.so'
Makefile:263: warning: ignoring old recipe for target 'goryba.so'
/bin/sh: warning: setlocale: LC_ALL: cannot change locale (c): No such file or directory

current directory: /home/retro/.gem/ruby/3.3.5/gems/goryba-0.1.0/ext/goryba
make DESTDIR\= sitearchdir\=./.gem.20241030-3456757-1g1rg1 sitelibdir\=./.gem.20241030-3456757-1g1rg1
Makefile:271: warning: overriding recipe for target 'goryba.so'
Makefile:263: warning: ignoring old recipe for target 'goryba.so'
cd .;         CGO_CFLAGS='-I. -I/home/retro/.rubies/ruby-3.3.5/include/ruby-3.3.0/x86_64-linux -I/home/retro/.rubies/ruby-3.3.5/include/ruby-3.3.0/ruby/backward -I/home/retro/.rubies/ruby-3.3.5/include/ruby-3.3.0 -I.' CGO_LDFLAGS='-Wl,--unresolved-symbols=ignore-all'           go build -p 4 -buildmode=c-shared -o /home/retro/.gem/ruby/3.3.5/gems/goryba-0.1.0/ext/goryba/goryba.so
/bin/sh: warning: setlocale: LC_ALL: cannot change locale (c): No such file or directory
# github.com/simi/goryba
./goryba.go:4:10: fatal error: goryba.h: No such file or directory
    4 | #include "goryba.h"
      |          ^~~~~~~~~~
compilation terminated.
make: *** [Makefile:271: goryba.so] Error 1

make failed, exit code 2

Gem files will remain installed in /home/retro/.gem/ruby/3.3.5/gems/goryba-0.1.0 for inspection.
Results logged to /home/retro/.gem/ruby/3.3.5/extensions/x86_64-linux/3.3.0-static/goryba-0.1.0/gem_make.out

[sue445]

@simi This was my mistake.

A generated gem didn't contains header file. So I fixed this at 3536b25.

Also, I saw your log and thought it would be strange and inconvenient to manually modify files or run commands after running bundle gem.

Therefore, I made sure that all of this was done within the bundle gem. (see. a0e2f4a, 768b801)

$ ruby spec/support/bundle.rb gem --ext=go goryba
Creating gem 'goryba'...
MIT License enabled in config
Changelog enabled in config
Initializing git repo in /Users/sue445/workspace/github.com/rubygems/rubygems/bundler/goryba
hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint:
hint: 	git config --global init.defaultBranch <name>
hint:
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint:
hint: 	git branch -m <name>
      create  goryba/Gemfile
      create  goryba/lib/goryba.rb
      create  goryba/lib/goryba/version.rb
      create  goryba/sig/goryba.rbs
      create  goryba/goryba.gemspec
      create  goryba/Rakefile
      create  goryba/README.md
      create  goryba/bin/console
      create  goryba/bin/setup
      create  goryba/.gitignore
      create  goryba/.rspec
      create  goryba/spec/spec_helper.rb
      create  goryba/spec/goryba_spec.rb
      create  goryba/.github/workflows/main.yml
      create  goryba/LICENSE.txt
      create  goryba/CHANGELOG.md
      create  goryba/ext/goryba/go.mod
      create  goryba/ext/goryba/extconf.rb
      create  goryba/ext/goryba/goryba.h
      create  goryba/ext/goryba/goryba.go
      create  goryba/ext/goryba/goryba.c
/Users/sue445/workspace/github.com/rubygems/rubygems/bundler/goryba/ext/goryba/go.sum has been created with `go mod tidy`
Gem 'goryba' was successfully created. For more information on making a RubyGem visit https://bundler.io/guides/creating_gem.html

$ cd goryba/

$ gem build --force && gem install goryba-0.1.0.gem
  Successfully built RubyGem
  Name: goryba
  Version: 0.1.0
  File: goryba-0.1.0.gem
Building native extensions. This could take a while...
Successfully installed goryba-0.1.0
Parsing documentation for goryba-0.1.0
Done installing documentation for goryba after 0 seconds
1 gem installed

[simi]

@sue445 it is ok to let users run go mod tidy (or any other command) to create go.sum file. For example at rust, we have check on build time to ensure Cargo.lock file is present in gem. We can do the same for go gems. This is easier, since we have no idea how client would like to generate Cargo.lock or go.sum file, since the Rust or Go environment setup is unknown to RubyGems. Thanks for the updates, I'll try again and report back.

[sue445]

Sorry, I found a small bug and change to draft until this is fixed.

[sue445]

I fixed

[sue445]

@simi ping

[simi]

I can check again if ready.