Flattening an array that contains itself

TL;DR

array[1] << array pushes the outer array into its own nested array, creating a cycle. A full flatten would have to expand that cycle forever, so Ruby detects the self-reference and raises ArgumentError with the message tried to flatten recursive array.

Step by step

array = [1, [2]]
array[1] << array

array.inspect # => "[1, [2, [...]]]"

After the push, array contains [2, array] as its second element. inspect handles the cycle gracefully: the [...] marks the point where the structure references itself.

flatten cannot be as forgiving. Flattening means replacing every nested array with its elements, recursively, until none remain. Here the recursion never bottoms out: expanding array requires expanding array[1], which contains array, which contains array[1], and so on. Instead of hanging, Ruby raises:

begin
  array = [1, [2]]
  array[1] << array
  array.flatten
rescue ArgumentError => e
  e.message # => "tried to flatten recursive array"
end

Edge cases

flatten takes an optional depth argument, and a depth-limited flatten terminates by construction, so it succeeds even on a recursive array:

array = [1, [2]]
array[1] << array

flat = array.flatten(1)
flat.inspect          # => "[1, 2, [1, [2, [...]]]]"
flat[2].equal?(array) # => true

One level of unwrapping produces [1, 2, array]: the cycle is still there, now sitting one level higher. Only the unbounded flatten (and flatten!) must refuse.

The same cycle detection shows up elsewhere: Array#join on a recursive array also raises ArgumentError (recursive array join), while inspect, ==, and hash are written to tolerate cycles.