From f26f577378c6afc8011131139bceaa32b685aeb7 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Juan=20V=C3=A1squez?= <juan@ombulabs.com>
Date: Wed, 30 Sep 2026 09:06:41 -0600
Subject: [PATCH] Step 1: Measure only the method definition in define_method
 vs module_eval

Every measured call built 10 random names, each letter from a fresh
('a'..'z').to_a, and defined them on one class that was never reset. The
garbage made the GC pause at random moments, and the class ended each 5 s
run with about 146,000 methods, so late calls did more work than early
ones. In run 36609397550 the reports had +-12 to 61% error on every CRuby
job and benchmark-ips said same-ish on all of them.

Now the names are built once, and each call defines them on a new class,
the same for both reports, so only define_method and module_eval with a
string differ. Locally on ruby_2.1 and ruby_4.0 the error dropped to about
+-4 to 8% and module_eval is 5 to 9x slower on CRuby and TruffleRuby.

The README keeps the old sample, with a note that it measured the old code,
and adds a ruby_4.0 run of the new code below it.
---
 README.md                                    | 18 ++++++++++
 code/general/define_method-vs-module-eval.rb | 36 ++++++++------------
 2 files changed, 32 insertions(+), 22 deletions(-)

diff --git a/README.md b/README.md
index 7fa8007..aa3b979 100644
--- a/README.md
+++ b/README.md
@@ -112,6 +112,24 @@ Comparison:
 module_eval with string:     5632.8 i/s - same-ish: difference falls within error
 ```
 
+The run above measured the old code, which built 10 random names on every call and defined them on one class that kept growing.
+The one below measures the current code, where every call defines the same 10 names on a new class.
+
+```
+$ ruby -v code/general/define_method-vs-module-eval.rb
+ruby 4.0.7 (2026-09-15 revision 229531a6cf) +PRISM [aarch64-linux]
+Warming up --------------------------------------
+          define_method    15.939k i/100ms
+module_eval with string     2.507k i/100ms
+Calculating -------------------------------------
+          define_method    132.661k (±12.4%) i/s    (7.54 μs/i) -    669.438k in   5.046213s
+module_eval with string     18.753k (±18.6%) i/s   (53.32 μs/i) -     95.266k in   5.079985s
+
+Comparison:
+          define_method:   132661.5 i/s
+module_eval with string:    18753.2 i/s - 7.07x  slower
+```
+
 ##### `String#constantize` vs a comparison for inflection [code](code/general/constantize-vs-comparison.rb)
 
 ActiveSupport's [String#constantize](https://guides.rubyonrails.org/active_support_core_extensions.html#constantize) "resolves the constant reference expression in its receiver".
diff --git a/code/general/define_method-vs-module-eval.rb b/code/general/define_method-vs-module-eval.rb
index 925849b..686aab9 100644
--- a/code/general/define_method-vs-module-eval.rb
+++ b/code/general/define_method-vs-module-eval.rb
@@ -1,24 +1,24 @@
 require 'benchmark/ips'
 
-def method_names(number)
-  number.times.map do
-    10.times.inject("") { |e| e << ('a'..'z').to_a.sample}
-  end
-end
+# Built once, so the benchmark measures defining the methods, not building names.
+# Random names built on every call made garbage, and the GC paused at random moments.
+METHOD_NAMES = Array.new(10) { |i| "method_#{i}" }
 
-class DefineMethod
-  def self.def_methods(_methods)
-    _methods.each do |method_name|
+# Each call defines the 10 methods on a new, empty class, so every call does the same work.
+# Adding them to one class that is never reset made late calls slower than early ones.
+def fast
+  Class.new do
+    METHOD_NAMES.each do |method_name|
       define_method method_name do
-        puts "win"
+        puts 'win'
       end
     end
   end
 end
 
-class ModuleEvalWithString
-  def self.def_methods(_methods)
-    _methods.each do |method_name|
+def slow
+  Class.new do
+    METHOD_NAMES.each do |method_name|
       module_eval %{
         def #{method_name}
           puts "win"
@@ -28,16 +28,8 @@ def #{method_name}
   end
 end
 
-def fast
-  DefineMethod.def_methods(method_names(10))
-end
-
-def slow
-  ModuleEvalWithString.def_methods(method_names(10))
-end
-
 Benchmark.ips do |x|
-  x.report("define_method")           { fast }
-  x.report("module_eval with string") { slow }
+  x.report('define_method')           { fast }
+  x.report('module_eval with string') { slow }
   x.compare!
 end