/usr/bin/innodb_log: invalid option -- s

normal behavior:

-bash-4.1$ sudo innodb_space -s /home/y/var/mysql/data/ibdata1_symlink -T dfbench/ContestEntry space-indexes
id          name                            root        fseg        used        allocated   fill_factor
438         PRIMARY                         3           internal    1           1           100.00%
438         PRIMARY                         3           leaf        0           0           0.00%
439         userId                          4           internal    1           1           100.00%
439         userId                          4           leaf        0           0           0.00%
440         contestRank                     5           internal    1           1           100.00%
440         contestRank                     5           leaf        0           0           0.00%

normal failure when table not found in dictionary:

-bash-4.1$ sudo innodb_space -s /home/y/var/mysql/data/ibdata1_symlink -T dfbench/Garbled space-indexes
/home/y/lib64/ruby/gems/1.9/gems/innodb_ruby-0.9.13/lib/innodb/system.rb:101:in `space_by_table_name': Table dfbench/Garbled not found (RuntimeError)
    from /home/y/lib64/ruby/gems/1.9/gems/innodb_ruby-0.9.13/bin/innodb_space:1713:in`<top (required)>'
    from /home/y/bin64/innodb_space:23:in `load'
    from /home/y/bin64/innodb_space:23:in`<main>'

falls back to system tablespace rather than reporting a warning/error when table file is not found (but it exists in dictionary):

-bash-4.1$ sudo innodb_space -s /home/y/var/mysql/ibdata/ibdata1 -T dfbench/ContestEntry space-indexes
id          name                            root        fseg        used        allocated   fill_factor
11          ID_IND                          302         internal    1           1           100.00%
11          ID_IND                          302         leaf        0           0           0.00%
12          FOR_IND                         303         internal    1           1           100.00%
12          FOR_IND                         303         leaf        0           0           0.00%
13          REF_IND                         304         internal    1           1           100.00%
13          REF_IND                         304         leaf        0           0           0.00%
14          ID_IND                          305         internal    1           1           100.00%
14          ID_IND                          305         leaf        0           0           0.00%
15          SYS_TABLESPACES_SPACE           307         internal    1           1           100.00%
15          SYS_TABLESPACES_SPACE           307         leaf        0           0           0.00%
16          SYS_DATAFILES_SPACE             308         internal    1           1           100.00%
16          SYS_DATAFILES_SPACE             308         leaf        0           0           0.00%

[root@node2 ruby-2.2.10]# innodb_space
/usr/lib/ruby/gems/1.8/gems/bindata-2.4.3/lib/bindata.rb:5: BinData requires ruby >= 1.9.3. Use BinData version 1.8.x instead (RuntimeError)
from /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in gem_original_require' from /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in require'
from /usr/lib/ruby/gems/1.8/gems/innodb_ruby-0.9.15/lib/innodb/util/buffer_cursor.rb:3
from /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in gem_original_require' from /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in require'
from /usr/lib/ruby/gems/1.8/gems/innodb_ruby-0.9.15/lib/innodb.rb:20
from /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in gem_original_require' from /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in require'
from /usr/lib/ruby/gems/1.8/gems/innodb_ruby-0.9.15/bin/innodb_space:6
from /usr/bin/innodb_space:19:in `load'
from /usr/bin/innodb_space:19
[root@node2 ruby-2.2.10]#

[root@node2 ruby-2.2.10]# ruby -v
ruby 2.2.10p489 (2018-03-28 revision 63023) [x86_64-linux]
[root@node2 ruby-2.2.10]#

Hello, I tries to install and use innodb_ruby, but it does not work, this error appears all the time

$ innodb_space -s ibdata1 system-spaces
Traceback (most recent call last):
13: from /usr/local/bin/innodb_space:23:in

Reinstall of MySQL and innodb_ruby doesn't help, unfortunately
I tries it on Debian 10 and MySQL 5.7

/root/innodb_ruby-master/bin/innodb_space -f abc/t1.ibd space-indexes

/root/innodb_ruby-master/bin/innodb_space:754: syntax error, unexpected '.'
...innodb_system.data_dictionary&.each_data_dictionary_index do...
... ^
/root/innodb_ruby-master/bin/innodb_space:1645: syntax error, unexpected '.'
...(Innodb::Page::Index) && page&.root?
... ^

hostnamectl

Static hostname: vm1
Icon name: computer-vm
Chassis: vm
Machine ID: 2008ec27d9ce57718c5d875e1245333e
Boot ID: bf593b5f8b204a345d5ecc766893fcd2
Virtualization: vmware
Operating System: CentOS Linux 7 (Core)
CPE OS Name: cpe:/o:centos:centos:7
Kernel: Linux 3.10.0-1160.71.1.el7.x86_64
Architecture: x86-64

Or if u allow me to write in a branch I can push there

Patch 1

From 067b47d Mon Sep 17 00:00:00 2001
From: Tusamarco [email protected]
Date: Thu, 7 Jan 2016 20:19:39 -0500
Subject: [PATCH] Convert to string given not possible to cast and need to be
pure String not Array to be accepted by basename

bin/innodb_space | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/bin/innodb_space b/bin/innodb_space
index 1d3df64..b883dfd 100644
--- a/bin/innodb_space
+++ b/bin/innodb_space
@@ -458,6 +458,7 @@ def space_index_pages_free_plot(space, image, start_page)
end

image_file = image + "_free.png"
+

Aim for one horizontal pixel per extent, but min 1k and max 10k width.

image_width = [10000, [1000, space.pages / space.pages_per_extent].max].min

@@ -1813,8 +1814,8 @@ when "space-summary"
space_summary(space, @options.page || 0)
when "space-index-pages-summary"
space_index_pages_summary(space, @options.page || 0)
-when "space-index-pages-free-plot"

• name = File.basename(@options.space_file).sub(".ibd", "")
• when "space-index-pages-free-plot"
• name = File.basename(@options.space_file.first.to_s).sub(".ibd", "")
  space_index_pages_free_plot(space, name, @options.page || 0)
  when "space-page-type-regions"
  space_page_type_regions(space, @options.page || 0)
  --
  2.6.2

Patch 2

From 02e037e Mon Sep 17 00:00:00 2001
From: Tusamarco [email protected]
Date: Thu, 7 Jan 2016 20:22:03 -0500
Subject: [PATCH] Uncomment the data type reference or an NIL issue will be
generated any time the array value is selected

lib/innodb/data_dictionary.rb | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/lib/innodb/data_dictionary.rb b/lib/innodb/data_dictionary.rb
index e22af06..e7cad98 100644
--- a/lib/innodb/data_dictionary.rb
+++ b/lib/innodb/data_dictionary.rb
@@ -70,17 +70,17 @@ class Innodb::DataDictionary

A hash of MySQL's internal type system to the stored

values for those types, and the "external" SQL type.

MYSQL_TYPE = {

• :DECIMAL => { :value => 0, :type => :DECIMAL },

• :DECIMAL => { :value => 0, :type => :DECIMAL },
  :TINY => { :value => 1, :type => :TINYINT },
  :SHORT => { :value => 2, :type => :SMALLINT },
  :LONG => { :value => 3, :type => :INT },
  :FLOAT => { :value => 4, :type => :FLOAT },
  :DOUBLE => { :value => 5, :type => :DOUBLE },

• :NULL => { :value => 6, :type => nil },

• :NULL => { :value => 6, :type => nil },
  :TIMESTAMP => { :value => 7, :type => :TIMESTAMP },
  :LONGLONG => { :value => 8, :type => :BIGINT },
  :INT24 => { :value => 9, :type => :MEDIUMINT },

• :DATE => { :value => 10, :type => :DATE },

• :DATE => { :value => 10, :type => :DATE },
  :TIME => { :value => 11, :type => :TIME },
  :DATETIME => { :value => 12, :type => :DATETIME },
  :YEAR => { :value => 13, :type => :YEAR },
  @@ -88,13 +88,13 @@ class Innodb::DataDictionary
  :VARCHAR => { :value => 15, :type => :VARCHAR },
  :BIT => { :value => 16, :type => :BIT },
  :NEWDECIMAL => { :value => 246, :type => :CHAR },

• :ENUM => { :value => 247, :type => :ENUM },

• :SET => { :value => 248, :type => :SET },

• :ENUM => { :value => 247, :type => :ENUM },
• :SET => { :value => 248, :type => :SET },
  :TINY_BLOB => { :value => 249, :type => :TINYBLOB },
  :MEDIUM_BLOB => { :value => 250, :type => :MEDIUMBLOB },
  :LONG_BLOB => { :value => 251, :type => :LONGBLOB },
  :BLOB => { :value => 252, :type => :BLOB },

• :VAR_STRING => { :value => 253, :type => :VARCHAR },

• :VAR_STRING => { :value => 253, :type => :VARCHAR },
  :STRING => { :value => 254, :type => :CHAR },
  :GEOMETRY => { :value => 255, :type => :GEOMETRY },
  }
  --
  2.6.2

Hello, Dear @jeremycole
I'm trying the latest version on innodb_ruby, but when I test this case, meet an error, when you have time, could you please take a look?

Case: undo-history-summary

Test env: Ruby 2.6 & MySQL 5.6 & MariaDB 10.2 with sakiladb

docker run --name mysql-5.6 -v /data/my56:/var/lib/mysql -d mysql:5.6
docker run --name mariadb-10.2 -v /data/mdb2:/var/lib/mysql -d mariadb:10.2

error msg

• MySQL 5.6

$ innodb_space -s ibdata1 undo-history-summary
Traceback (most recent call last):
    9: from /usr/local/bin/innodb_space:23:in `<main>'
    8: from /usr/local/bin/innodb_space:23:in `load'
    7: from /usr/local/lib/ruby/gems/2.6.0/gems/innodb_ruby-0.11.0/bin/innodb_space:1715:in `<top (required)>'
    6: from /usr/local/lib/ruby/gems/2.6.0/gems/innodb_ruby-0.11.0/bin/innodb_space:1202:in `undo_history_summary'
    5: from /usr/local/lib/ruby/gems/2.6.0/gems/innodb_ruby-0.11.0/bin/innodb_space:1202:in `reject'
    4: from /usr/local/lib/ruby/gems/2.6.0/gems/innodb_ruby-0.11.0/bin/innodb_space:1202:in `each'
    3: from /usr/local/lib/ruby/gems/2.6.0/gems/innodb_ruby-0.11.0/lib/innodb/history.rb:25:in `each_history_list'
    2: from /usr/local/lib/ruby/gems/2.6.0/gems/innodb_ruby-0.11.0/lib/innodb/history.rb:25:in `each'
    1: from /usr/local/lib/ruby/gems/2.6.0/gems/innodb_ruby-0.11.0/lib/innodb/history.rb:26:in `block in each_history_list'
/usr/local/lib/ruby/gems/2.6.0/gems/innodb_ruby-0.11.0/bin/innodb_space:1202:in `block in undo_history_summary': undefined method `empty?' for #<Innodb::List::History:0x0000562fac3518d0> (NoMethodError)

• MariaDB 10.2

$ innodb_space -s ibdata1 undo-history-summary
Traceback (most recent call last):
    9: from /usr/local/bin/innodb_space:23:in `<main>'
    8: from /usr/local/bin/innodb_space:23:in `load'
    7: from /usr/local/lib/ruby/gems/2.6.0/gems/innodb_ruby-0.11.0/bin/innodb_space:1715:in `<top (required)>'
    6: from /usr/local/lib/ruby/gems/2.6.0/gems/innodb_ruby-0.11.0/bin/innodb_space:1202:in `undo_history_summary'
    5: from /usr/local/lib/ruby/gems/2.6.0/gems/innodb_ruby-0.11.0/bin/innodb_space:1202:in `reject'
    4: from /usr/local/lib/ruby/gems/2.6.0/gems/innodb_ruby-0.11.0/bin/innodb_space:1202:in `each'
    3: from /usr/local/lib/ruby/gems/2.6.0/gems/innodb_ruby-0.11.0/lib/innodb/history.rb:25:in `each_history_list'
    2: from /usr/local/lib/ruby/gems/2.6.0/gems/innodb_ruby-0.11.0/lib/innodb/history.rb:25:in `each'
    1: from /usr/local/lib/ruby/gems/2.6.0/gems/innodb_ruby-0.11.0/lib/innodb/history.rb:26:in `block in each_history_list'
/usr/local/lib/ruby/gems/2.6.0/gems/innodb_ruby-0.11.0/bin/innodb_space:1202:in `block in undo_history_summary': undefined method `empty?' for #<Innodb::List::History:0x000055c85a01d830> (NoMethodError)

I'm new to ruby. Can this tool be used on MacOS?

After reading the doc:https://github.com/jeremycole/innodb_ruby/wiki, I try to use this tool on MacOS, I got this error:

➜ innodb_ruby-master pwd
/Users/gonechaos/GitHubProjects/innodb_ruby-master
➜ innodb_ruby-master ruby -r rubygems -I lib bin/innodb_space -s ibdata1 system-spaces
Traceback (most recent call last):
2: from bin/innodb_space:5:in <main>' 1: from /Library/Ruby/Site/2.6.0/rubygems/core_ext/kernel_require.rb:85:in require'
/Library/Ruby/Site/2.6.0/rubygems/core_ext/kernel_require.rb:85:in `require': cannot load such file -- histogram/array (LoadError)

Hello, I tries to install and use innodb_ruby, but it does not work, this error appears all the time

$ innodb_space -s ibdata1 system-spaces
Traceback (most recent call last):
13: from /usr/local/bin/innodb_space:23:in <main>' 12: from /usr/local/bin/innodb_space:23:in load'
11: from /Library/Ruby/Gems/2.6.0/gems/innodb_ruby-0.12.0/bin/innodb_space:1599:in <top (required)>' 10: from /Library/Ruby/Gems/2.6.0/gems/innodb_ruby-0.12.0/bin/innodb_space:1599:in new'
9: from /Library/Ruby/Gems/2.6.0/gems/innodb_ruby-0.12.0/lib/innodb/system.rb:41:in initialize' 8: from /Library/Ruby/Gems/2.6.0/gems/innodb_ruby-0.12.0/lib/innodb/system.rb:60:in add_space_file'
7: from /Library/Ruby/Gems/2.6.0/gems/innodb_ruby-0.12.0/lib/innodb/system.rb:60:in new' 6: from /Library/Ruby/Gems/2.6.0/gems/innodb_ruby-0.12.0/lib/innodb/space.rb:65:in initialize'
5: from /Library/Ruby/Gems/2.6.0/gems/innodb_ruby-0.12.0/lib/innodb/space.rb:65:in each' 4: from /Library/Ruby/Gems/2.6.0/gems/innodb_ruby-0.12.0/lib/innodb/space.rb:66:in block in initialize'
3: from /Library/Ruby/Gems/2.6.0/gems/innodb_ruby-0.12.0/lib/innodb/space.rb:66:in new' 2: from /Library/Ruby/Gems/2.6.0/gems/innodb_ruby-0.12.0/lib/innodb/space.rb:45:in initialize'
1: from /Library/Ruby/Gems/2.6.0/gems/innodb_ruby-0.12.0/lib/innodb/space.rb:45:in open' /Library/Ruby/Gems/2.6.0/gems/innodb_ruby-0.12.0/lib/innodb/space.rb:45:in initialize': No such file or directory @ rb_sysopen - ibdata1 (Errno::ENOENT)

Reinstall of MySQL and innodb_ruby doesn't help, unfortunately

[root@localhost innodb]# innodb_space -s ibdata1 -T test/user_info space-page-type-regions Traceback (most recent call last): 11: from /opt/rh/rh-ruby27/root/usr/local/bin/innodb_space:23:in

Pls add execute bit to files. Thanks for great info!

parsing ibata1 for MariadB10.5/ 10.6 cause error

└──╼ #ruby -r rubygems -I lib bin/innodb_space -s /var/lib/mysql/ibdata1 system-spaces
name pages indexes
Traceback (most recent call last):
11: from bin/innodb_space:1714:in <main>' 10: from bin/innodb_space:192:in system_spaces'
9: from bin/innodb_space:188:in block in system_spaces' 8: from bin/innodb_space:188:in to_a'
7: from bin/innodb_space:188:in each' 6: from /root/innodb_ruby/lib/innodb/space.rb:341:in each_index'
5: from /root/innodb_ruby/lib/innodb/space.rb:321:in each_index_root_page_number' 4: from /root/innodb_ruby/lib/innodb/data_dictionary.rb:425:in each_index_by_space_id'
3: from /root/innodb_ruby/lib/innodb/data_dictionary.rb:355:in each_index' 2: from /root/innodb_ruby/lib/innodb/data_dictionary.rb:324:in each_record_from_data_dictionary_index'
1: from /root/innodb_ruby/lib/innodb/data_dictionary.rb:278:in data_dictionary_index' /root/innodb_ruby/lib/innodb/data_dictionary.rb:235:in data_dictionary_indexes': undefined method `data_dictionary_header' for nil:NilClass (NoMethodError)

Traceback (most recent call last):
1: from bin/innodb_space:1835:in <main>' bin/innodb_space:1835:in trap': unsupported signal `SIGPIPE' (ArgumentError)

OS: 10.15.7
Ruby: 2.6.3p62

when i execute
innodb_space -s ibdata1 -T my_test/agent space-extents-illustrate

Start Page ╭────────────────────────────────────────────────────────────────╮ Traceback (most recent call last): 24: from /Users/dan/MySQL/innodb_ruby/bin/innodb_space:1669:in

There are multiple things that need to happen to support ZLOB fully, at least support for:

• New page types (ZLOB_FIRST, ZLOB_INDEX, ZLOB_DATA, ZLOB_FRAG, ZLOB_FRAG_ENTRY).
• Navigation to ZLOB records from index records.

innodb_space does not support yet InnoDB FTS indexes.

emanuel@3laptop ~/sandboxes/rcsandbox_mysql-5_7_9/node1/data/test $ innodb_space -I test/oe_email -T test/Emails  -s ../ibdata1 index-recurse
/var/lib/gems/1.9.1/gems/innodb_ruby-0.9.13/lib/innodb/system.rb:213:in `index_by_name': undefined method `[]' for nil:NilClass (NoMethodError)
    from /var/lib/gems/1.9.1/gems/innodb_ruby-0.9.13/bin/innodb_space:1730:in `<top (required)>'
    from /usr/local/bin/innodb_space:23:in `load'
    from /usr/local/bin/innodb_space:23:in `<main>'

Thanks for such great tool! :D

If that doesn’t work, you might want to check out The RubyGems manual to try and get your installation working. Or abandon all hope. :-D

-> not funny, sorry

currently, innodb_space only supports the default directory structure, where ibdata1 is in the main data dir and each schema is a subdir of that. on installs that don't use this structure, the tool won't be able to find tables specified. you can still run it directly on a tablespace file using -f, but that loses all the functionality of using the sys tablespace as a baseline.

ideally, it would detect the locations from the current server config options. if that's a pain, an option to specify directories would solve the problem.

It would be so great if we could output the result of the *illustrate command directly to an image (that I could include in documentation/reports).

For example, I'm a big fan of space-lsn-age-illustrate but on a large table I can make an easy screenshot of the result.

I've looked for a solution using something in between likr from stdout to png but I didn't find anything working.

innodb_space ...
/usr/bin/innodb_space:19:in `load': /usr/lib/ruby/gems/1.8/gems/innodb_ruby-0.9.13/bin/innodb_space:720: syntax error, unexpected ')' (SyntaxError)
/usr/lib/ruby/gems/1.8/gems/innodb_ruby-0.9.13/bin/innodb_space:734: syntax error, unexpected kDO_BLOCK, expecting kEND
identifiers.sort.each do |identifier, description|
^
/usr/lib/ruby/gems/1.8/gems/innodb_ruby-0.9.13/bin/innodb_space:734: syntax error, unexpected '|', expecting '='
/usr/lib/ruby/gems/1.8/gems/innodb_ruby-0.9.13/bin/innodb_space:758: syntax error, unexpected kEND, expecting $end
from /usr/bin/innodb_space:19

714 block_x = 0
715 block_y += 10
716 puts svg_extent_legend(
717 graphic_x + block_x,
718 graphic_y + block_y,
719 block_size, // This line remove the semicolon normally
720 )

Hi. I am facing the following issue. Any idea what could be wrong?

$ innodb_space -s ibdata1 undo-history-summary
Page Offset Transaction Type Table
1283 771 1204 delete ipworks/ENUMDNSCHED
1318 272 1204 delete ipworks/ENUMDNSCHED
1250 1007 1204 delete ipworks/ENUMDNSCHED
1049 1745 1204 delete ipworks/ENUMDNSCHED
1251 774 1204 delete ipworks/ENUMDNSCHED
601 0
1243 1279 1204 delete ipworks/ENUMDNSCHED
1257 1281 1204 delete ipworks/ENUMDNSCHED
1051 1005 1204 delete ipworks/ENUMDNSCHED
1319 272 1204 delete ipworks/ENUMDNSCHED
1264 1012 1204 delete ipworks/ENUMDNSCHED
1284 1030 1204 delete ipworks/ENUMDNSCHED
1332 515 1204 delete ipworks/ENUMDNSCHED
1053 781 1204 delete ipworks/ENUMDNSCHED
1269 792 1204 delete ipworks/ENUMDNSCHED
1054 2516 1204 delete ipworks/ENUMDNSCHED
1055 1482 1204 delete ipworks/ENUMDNSCHED
1056 507 1204 delete ipworks/ENUMDNSCHED
1057 501 1204 delete ipworks/ENUMDNSCHED
1336 272 1204 delete ipworks/ENUMDNSCHED
1281 1443 1204 delete ipworks/ENUMDNSCHED
1289 1776 1204 delete ipworks/ENUMDNSCHED
1232 1272 1204 delete ipworks/ENUMDNSCHED
1277 0
/var/lib/gems/2.1.0/gems/innodb_ruby-0.9.12/lib/innodb/util/buffer_cursor.rb:217:in read_and_advance': undefined methodbytes' for nil:NilClass (NoMethodError)
from /var/lib/gems/2.1.0/gems/innodb_ruby-0.9.12/lib/innodb/util/buffer_cursor.rb:259:in get_uint16' from /var/lib/gems/2.1.0/gems/innodb_ruby-0.9.12/lib/innodb/undo_record.rb:70:inblock (2 levels) in header'
from /var/lib/gems/2.1.0/gems/innodb_ruby-0.9.12/lib/innodb/util/buffer_cursor.rb:136:in name' from /var/lib/gems/2.1.0/gems/innodb_ruby-0.9.12/lib/innodb/undo_record.rb:70:inblock in header'
from /var/lib/gems/2.1.0/gems/innodb_ruby-0.9.12/lib/innodb/util/buffer_cursor.rb:136:in name' from /var/lib/gems/2.1.0/gems/innodb_ruby-0.9.12/lib/innodb/undo_record.rb:68:inheader'
from /var/lib/gems/2.1.0/gems/innodb_ruby-0.9.12/lib/innodb/undo_record.rb:88:in type' from /var/lib/gems/2.1.0/gems/innodb_ruby-0.9.12/lib/innodb/undo_record.rb:98:inget'
from /var/lib/gems/2.1.0/gems/innodb_ruby-0.9.12/lib/innodb/undo_record.rb:109:in next' from /var/lib/gems/2.1.0/gems/innodb_ruby-0.9.12/lib/innodb/undo_log.rb:85:innext_undo_record'
from /var/lib/gems/2.1.0/gems/innodb_ruby-0.9.12/lib/innodb/undo_log.rb:104:in undo_record' from /var/lib/gems/2.1.0/gems/innodb_ruby-0.9.12/lib/innodb/history_list.rb:39:inundo_record'
from /var/lib/gems/2.1.0/gems/innodb_ruby-0.9.12/lib/innodb/history_list.rb:87:in each_undo_record' from /var/lib/gems/2.1.0/gems/innodb_ruby-0.9.12/lib/innodb/history_list.rb:102:ineach_undo_record'
from /var/lib/gems/2.1.0/gems/innodb_ruby-0.9.12/bin/innodb_space:1052:in block in undo_history_summary' from /var/lib/gems/2.1.0/gems/innodb_ruby-0.9.12/bin/innodb_space:1051:ineach'
from /var/lib/gems/2.1.0/gems/innodb_ruby-0.9.12/bin/innodb_space:1051:in undo_history_summary' from /var/lib/gems/2.1.0/gems/innodb_ruby-0.9.12/bin/innodb_space:1512:in<top (required)>'
from /usr/local/bin/innodb_space:23:in load' from /usr/local/bin/innodb_space:23:in

hello, I have a problem when use innodb_space -f yst.ibd space-page-type-regions
The error:
/usr/local/lib/ruby/gems/2.6.0/gems/innodb_ruby-0.12.0/lib/innodb/space.rb:272:in fsp': undefined method fsp_header' for nil:NilClass (NoMethodError)
from /usr/local/lib/ruby/gems/2.6.0/gems/innodb_ruby-0.12.0/lib/innodb/space.rb:150:in fsp_flags' from /usr/local/lib/ruby/gems/2.6.0/gems/innodb_ruby-0.12.0/lib/innodb/space.rb:73:in initialize'
from /usr/local/lib/ruby/gems/2.6.0/gems/innodb_ruby-0.12.0/bin/innodb_space:1612:in new' from /usr/local/lib/ruby/gems/2.6.0/gems/innodb_ruby-0.12.0/bin/innodb_space:1612:in <top (required)>'
from /usr/local/bin/innodb_space:23:in load' from /usr/local/bin/innodb_space:23:in

mariadb version is : 10.5.15
innodb version is: 10.5.15

Add support for MySQL 8.0. There are a number of changes needed to support MySQL 8.0, and some new data structures to implement.

• Support new headers in FSP pages
• Support for reconstructing full BLOB content
• Support new page types:
  • UNKNOWN (13)
  • COMPRESSED (14)
  • ENCRYPTED (15)
  • COMPRESSED_AND_ENCRYPTED (16)
  • ENCRYPTED_RTREE (17)
  • SDI_BLOB (18) (aka BLOB)
  • SDI_ZBLOB (19) (aka ZBLOB)
  • LEGACY_DBLWR (20)
  • RSEG_ARRAY (21)
  • LOB_INDEX (22)
  • LOB_DATA (23)
  • LOB_FIRST (24)
  • ZLOB_FIRST (25)
  • ZLOB_DATA (26)
  • ZLOB_INDEX (27)
  • ZLOB_FRAG (28)
  • ZLOB_FRAG_ENTRY (29)
  • SDI (17853) (aka INDEX)
  • RTREE (17854)
• Support for new SDI data dictionary structures:
  • Using SDI for data dictionary to make record describers
  • Using SDI for overview information (e.g. system-spaces)
• Support for LOB
• Support for Compressed LOB (ZLOB)
• Support for online-added columns?
• ???

Sub-issue list:

• #185
• #186
• #187

PR list:

• #184
• #188
• #189
• #190
• #191

See also: #41, #44.

This great tool works fine for 5.6&5.7, but "innodb_ruby/lib/innodb/index.rb:34:in page': undefined method record_describer=' for #Innodb::Page::FspHdrXdes:0x0000559b2ee52ee0 (NoMethodError)" error raises when work on 8.0.

Is there any plan to support MySQL8.0?

These are detailed stackbacks for 8.0.17 and 8.0.20

marvin@vm204:thebase/innodb_ruby$ sudo ruby -r rubygems -I lib bin/innodb_space -s /home/marvin/thebase/mysql-8.0.17/data/ibdata1 system-spaces
name pages indexes
Traceback (most recent call last):
14: from bin/innodb_space:1638:in <main>' 13: from bin/innodb_space:180:in system_spaces'
12: from bin/innodb_space:176:in block in system_spaces' 11: from bin/innodb_space:176:in to_a'
10: from bin/innodb_space:176:in each' 9: from /home/marvin/thebase/innodb_ruby/lib/innodb/space.rb:341:in each_index'
8: from /home/marvin/thebase/innodb_ruby/lib/innodb/space.rb:321:in each_index_root_page_number' 7: from /home/marvin/thebase/innodb_ruby/lib/innodb/data_dictionary.rb:427:in each_index_by_space_id'
6: from /home/marvin/thebase/innodb_ruby/lib/innodb/data_dictionary.rb:357:in each_index' 5: from /home/marvin/thebase/innodb_ruby/lib/innodb/data_dictionary.rb:324:in each_record_from_data_dictionary_index'
4: from /home/marvin/thebase/innodb_ruby/lib/innodb/data_dictionary.rb:287:in data_dictionary_index' 3: from /home/marvin/thebase/innodb_ruby/lib/innodb/space.rb:312:in index'
2: from /home/marvin/thebase/innodb_ruby/lib/innodb/space.rb:312:in new' 1: from /home/marvin/thebase/innodb_ruby/lib/innodb/index.rb:19:in initialize'
/home/marvin/thebase/innodb_ruby/lib/innodb/index.rb:34:in page': undefined method record_describer=' for #Innodb::Page::FspHdrXdes:0x0000559b2ee52ee0 (NoMethodError)
marvin@vm204:thebase/innodb_ruby$ sudo ruby -r rubygems -I lib bin/innodb_space -s /var/lib/mysql/ibdata1 system-spaces
name pages indexes
Traceback (most recent call last):
14: from bin/innodb_space:1638:in <main>' 13: from bin/innodb_space:180:in system_spaces'
12: from bin/innodb_space:176:in block in system_spaces' 11: from bin/innodb_space:176:in to_a'
10: from bin/innodb_space:176:in each' 9: from /home/marvin/thebase/innodb_ruby/lib/innodb/space.rb:341:in each_index'
8: from /home/marvin/thebase/innodb_ruby/lib/innodb/space.rb:321:in each_index_root_page_number' 7: from /home/marvin/thebase/innodb_ruby/lib/innodb/data_dictionary.rb:427:in each_index_by_space_id'
6: from /home/marvin/thebase/innodb_ruby/lib/innodb/data_dictionary.rb:357:in each_index' 5: from /home/marvin/thebase/innodb_ruby/lib/innodb/data_dictionary.rb:324:in each_record_from_data_dictionary_index'
4: from /home/marvin/thebase/innodb_ruby/lib/innodb/data_dictionary.rb:287:in data_dictionary_index' 3: from /home/marvin/thebase/innodb_ruby/lib/innodb/space.rb:312:in index'
2: from /home/marvin/thebase/innodb_ruby/lib/innodb/space.rb:312:in new' 1: from /home/marvin/thebase/innodb_ruby/lib/innodb/index.rb:19:in initialize'
/home/marvin/thebase/innodb_ruby/lib/innodb/index.rb:34:in page': undefined method record_describer=' for #Innodb::Page::FspHdrXdes:0x000055f09e3e6db0 (NoMethodError)
marvin@vm204:~/thebase/innodb_ruby$

I use innodb_space for mysql8.0 ibdata1, the following errors occured：

$ innodb_space -s ~/install/mysql8.0/data/ibdata1 system-spaces
name                            pages       indexes
/home/longhai/.gem/ruby/gems/innodb_ruby-0.9.15/lib/innodb/index.rb:33:in `page': undefined method `record_describer=' for #<Innodb::Page::FspHdrXdes:0x00000000ef85f8> (NoMethodError)
        from /home/longhai/.gem/ruby/gems/innodb_ruby-0.9.15/lib/innodb/index.rb:13:in `initialize'
        from /home/longhai/.gem/ruby/gems/innodb_ruby-0.9.15/lib/innodb/space.rb:308:in `new'
        from /home/longhai/.gem/ruby/gems/innodb_ruby-0.9.15/lib/innodb/space.rb:308:in `index'
        from /home/longhai/.gem/ruby/gems/innodb_ruby-0.9.15/lib/innodb/data_dictionary.rb:282:in `data_dictionary_index'
        from /home/longhai/.gem/ruby/gems/innodb_ruby-0.9.15/lib/innodb/data_dictionary.rb:325:in `each_record_from_data_dictionary_index'
        from /home/longhai/.gem/ruby/gems/innodb_ruby-0.9.15/lib/innodb/data_dictionary.rb:364:in `each_index'
        from /home/longhai/.gem/ruby/gems/innodb_ruby-0.9.15/lib/innodb/data_dictionary.rb:448:in `each_index_by_space_id'
        from /home/longhai/.gem/ruby/gems/innodb_ruby-0.9.15/lib/innodb/space.rb:320:in `each_index_root_page_number'
        from /home/longhai/.gem/ruby/gems/innodb_ruby-0.9.15/lib/innodb/space.rb:344:in `each_index'
        from /home/longhai/.gem/ruby/gems/innodb_ruby-0.9.15/bin/innodb_space:207:in `each'
        from /home/longhai/.gem/ruby/gems/innodb_ruby-0.9.15/bin/innodb_space:207:in `to_a'
        from /home/longhai/.gem/ruby/gems/innodb_ruby-0.9.15/bin/innodb_space:207:in `block in system_spaces'
        from /home/longhai/.gem/ruby/gems/innodb_ruby-0.9.15/bin/innodb_space:211:in `call'
        from /home/longhai/.gem/ruby/gems/innodb_ruby-0.9.15/bin/innodb_space:211:in `system_spaces'
        from /home/longhai/.gem/ruby/gems/innodb_ruby-0.9.15/bin/innodb_space:1976:in `<top (required)>'
        from /home/longhai/bin/innodb_space:23:in `load'
        from /home/longhai/bin/innodb_space:23:in `<main>'

Does innodb_ruby is still not supported for mysql8.0?(I tested mysql5.7, it's ok)
If so, do we have a plan to support mysql8.0?

There are multiple things that need to happen to support LOB fully, at least support for:

• New page types (LOB_FIRST, LOB_INDEX, LOB_DATA).
• Navigation to LOB records from index records.

I haven't looked into the code, but it appears that the space-index-pages-summary option to innodb_space errors when examining a table that has been created, but has no data.

Repro Steps:

mysql> create table my_test (id int auto_increment primary key, my_val varchar(10)) engine=innodb;
Query OK, 0 rows affected (0.01 sec)
mysql> select * from my_test;
Empty set (0.00 sec)
mysql> quit
Bye

innodb_space -f my_test.ibd space-index-pages-summary | head -n 10
/var/lib/gems/1.8/gems/innodb_ruby-0.9.12/lib/innodb/space.rb:113:in raw_fsp_header_flags': Something is very wrong; Page 0 does not seem to be type FSP_HDR; got page type 0 but expected 8 (RuntimeError) from /var/lib/gems/1.8/gems/innodb_ruby-0.9.12/lib/innodb/space.rb:139:infsp_flags'
from /var/lib/gems/1.8/gems/innodb_ruby-0.9.12/lib/innodb/space.rb:57:in initialize' from /var/lib/gems/1.8/gems/innodb_ruby-0.9.12/bin/innodb_space:1364:innew'
from /var/lib/gems/1.8/gems/innodb_ruby-0.9.12/bin/innodb_space:1364
from /usr/local/bin/innodb_space:19:in `load'
from /usr/local/bin/innodb_space:19

Mysql version: 5.6.22-72.0-log Percona Server (GPL), Release 72.0, Revision 738
Linux: 3.5.0-54-generic #81~precise1-Ubuntu SMP

I'm trying to space-index-pages-free-plot, but error occured like below.

➜  innodb_ruby git:(master) ✗ ruby -r rubygems -I lib bin/innodb_space -f spec/data/hello_world.ibd space-index-pages-free-plot
bin/innodb_space:1817:in `basename': no implicit conversion of Array into String (TypeError)
    from bin/innodb_space:1817:in `<main>'

I think this commit is affecting
d8c5a95

Would you please fix it?

I know this might be outside of the scope of the gem but I think these kind of go hand in hand and I would love to see this functionality in innodb_ruby

The idea is modeled after

• http://linux.die.net/man/3/innodbparser
• https://github.com/dgoodlad/innodbparser

➜ data innodb_space -s ibdata1 -T sakila/actor space-indexes
/System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/universal-darwin21/rbconfig.rb:230: warning: Insecure world writable dir /opt/homebrew/bin in PATH, mode 040777
/Users/qlh/.gem/ruby/2.6.0/gems/innodb_ruby-0.12.0/lib/innodb/index.rb:34:in page': undefined method record_describer=' for #Innodb::Page::FspHdrXdes:0x000000010412f368 (NoMethodError)
from /Users/qlh/.gem/ruby/2.6.0/gems/innodb_ruby-0.12.0/lib/innodb/index.rb:19:in initialize' from /Users/qlh/.gem/ruby/2.6.0/gems/innodb_ruby-0.12.0/lib/innodb/space.rb:312:in new'
from /Users/qlh/.gem/ruby/2.6.0/gems/innodb_ruby-0.12.0/lib/innodb/space.rb:312:in index' from /Users/qlh/.gem/ruby/2.6.0/gems/innodb_ruby-0.12.0/lib/innodb/data_dictionary.rb:287:in data_dictionary_index'
from /Users/qlh/.gem/ruby/2.6.0/gems/innodb_ruby-0.12.0/lib/innodb/data_dictionary.rb:324:in each_record_from_data_dictionary_index' from /Users/qlh/.gem/ruby/2.6.0/gems/innodb_ruby-0.12.0/lib/innodb/data_dictionary.rb:333:in each_table'
from /Users/qlh/.gem/ruby/2.6.0/gems/innodb_ruby-0.12.0/lib/innodb/data_dictionary.rb:376:in each' from /Users/qlh/.gem/ruby/2.6.0/gems/innodb_ruby-0.12.0/lib/innodb/data_dictionary.rb:376:in select'
from /Users/qlh/.gem/ruby/2.6.0/gems/innodb_ruby-0.12.0/lib/innodb/data_dictionary.rb:376:in object_by_field' from /Users/qlh/.gem/ruby/2.6.0/gems/innodb_ruby-0.12.0/lib/innodb/data_dictionary.rb:392:in table_by_name'
from /Users/qlh/.gem/ruby/2.6.0/gems/innodb_ruby-0.12.0/lib/innodb/system.rb:97:in space_by_table_name' from /Users/qlh/.gem/ruby/2.6.0/gems/innodb_ruby-0.12.0/bin/innodb_space:1603:in <top (required)>'
from /usr/local/bin/innodb_space:23:in load' from /usr/local/bin/innodb_space:23:in

When I run innodb_space in space-index-pages-free-plot, I encountered the following error:

$ innodb_space -f ./t.ibd space-index-pages-free-plot
Wrote _/t_free.png

gnuplot> set output "_/t_free.png"
                                  ^
         line 0:          cannot open file; output not changed
system error: No such file or directory

But when I change the parent directory, and repeat the above command, everything is ok:

$ innodb_space -f test/t.ibd space-index-pages-free-plot
Wrote test_t_free.png

Then I check the code , I found that it could not get the correct output file name when the file specified by '-f' is in the current directory.

Is this a bug? or on purpose?

Please check it. Thanks~

There are multiple things that need to happen to support SDI fully, at least support for:

• New SDI header in FSP page
• New page types (SDI, SDI_BLOB, SDI_ZBLOB).
• Usefully exposing the JSON structures stored in the SDI.
• Using the SDI to navigate tablespaces for e.g. system-spaces.
• Using the SDI to construct record describers for all indexes.

My system is 14.04 Ubuntu and the newest version of ruby.

I tried to install inndo_ruby both by sudo gem install innodb_ruby and from repo.
However, when I run innodb_space, it shows

vagrant@system-lock:/usr/local/mysql/data/test1$ innodb_space
/usr/local/bin/innodb_space:19:in `load': /var/lib/gems/1.8/gems/innodb_ruby-0.9.13/bin/innodb_space:720: syntax error, unexpected ')' (SyntaxError)
/var/lib/gems/1.8/gems/innodb_ruby-0.9.13/bin/innodb_space:734: syntax error, unexpected kDO_BLOCK, expecting kEND
identifiers.sort.each do |identifier, description|
^
/var/lib/gems/1.8/gems/innodb_ruby-0.9.13/bin/innodb_space:734: syntax error, unexpected '|', expecting '='
/var/lib/gems/1.8/gems/innodb_ruby-0.9.13/bin/innodb_space:758: syntax error, unexpected kEND, expecting $end
from /usr/local/bin/innodb_space:19

Instead it should advance the cursor through all records in the block. It should also handle partial records, i.e., those spanning block boundaries. Diagram of a log block:

https://blogs.oracle.com/mysqlinnodb/entry/redo_logging_in_innodb

Description

Some root index page are freed ones during bulk load, and they should not be considered as a real index root.

How to repeat

Firstly, prepare data with sysbench.

./sysbench --mysql-host=<host> --mysql-port=<port> --mysql-db=<db> --mysql-user=<user> --rand-type=gaussian --threads=4 --table-size=640000 --tables=1 oltp_read_write prepare

Then run innodb_space.

innodb_space -f ./sbtest1.ibd  space-indexes 

How to fix it

--- a/lib/innodb/space.rb
+++ b/lib/innodb/space.rb
@@ -327,7 +327,10 @@ module Innodb
         # for IBD files, if they haven't added indexes online.
+        xdes = xdes_for_page(3)
         (3...@pages).each do |page_number|
           page = page(page_number)
-          yield page_number if page.type == :INDEX && page.root?
+          xdes = xdes_for_page(page_number)
+                                                if (page_number % pages_per_extent).zero?
+          xdes_status = xdes.page_status(page_number)
+          yield page_number if !xdes_status[:free] && page.type == :INDEX &&
+                               page.root?
         end
       end

I made some dummy data using this rake task.
https://github.com/kysnm/innodb_ruby_test/blob/master/Rakefile

and I'm trying index-recurse with this class.
https://github.com/kysnm/innodb_ruby_test/blob/master/innodb_ruby_test_db.rb#L15-L20

but error occured like below.

➜  innodb_ruby_test git:(master) ✗ bundle exec innodb_space -f /usr/local/var/mysql/innodb_ruby_test/users.ibd -p 3 -r innodb_ruby_test_db.rb -d InnodbRubyTest_users_state index-recurse
ROOT NODE #3: 618 records, 8652 bytes
/tmp/.rbenv/versions/2.3.0-preview2/gemsets/innodb_ruby_test/gems/innodb_ruby-0.9.13/lib/innodb/page.rb:44:in `initialize': Page can't be initialized from nil space or buffer (space: #<Innodb::Space:0x007fc3ba0d01f8>, buffer: ) (RuntimeError)
    from /tmp/.rbenv/versions/2.3.0-preview2/gemsets/innodb_ruby_test/gems/innodb_ruby-0.9.13/lib/innodb/page.rb:24:in `new'
    from /tmp/.rbenv/versions/2.3.0-preview2/gemsets/innodb_ruby_test/gems/innodb_ruby-0.9.13/lib/innodb/page.rb:24:in `parse'
    from /tmp/.rbenv/versions/2.3.0-preview2/gemsets/innodb_ruby_test/gems/innodb_ruby-0.9.13/lib/innodb/space.rb:231:in `page'
    from /tmp/.rbenv/versions/2.3.0-preview2/gemsets/innodb_ruby_test/gems/innodb_ruby-0.9.13/lib/innodb/index.rb:31:in `page'
    from /tmp/.rbenv/versions/2.3.0-preview2/gemsets/innodb_ruby_test/gems/innodb_ruby-0.9.13/lib/innodb/index.rb:59:in `block in _recurse'
    from /tmp/.rbenv/versions/2.3.0-preview2/gemsets/innodb_ruby_test/gems/innodb_ruby-0.9.13/lib/innodb/page/index.rb:944:in `block in each_child_page'
    from /tmp/.rbenv/versions/2.3.0-preview2/gemsets/innodb_ruby_test/gems/innodb_ruby-0.9.13/lib/innodb/page/index.rb:908:in `each_record'
    from /tmp/.rbenv/versions/2.3.0-preview2/gemsets/innodb_ruby_test/gems/innodb_ruby-0.9.13/lib/innodb/page/index.rb:943:in `each_child_page'
    from /tmp/.rbenv/versions/2.3.0-preview2/gemsets/innodb_ruby_test/gems/innodb_ruby-0.9.13/lib/innodb/index.rb:58:in `_recurse'
    from /tmp/.rbenv/versions/2.3.0-preview2/gemsets/innodb_ruby_test/gems/innodb_ruby-0.9.13/lib/innodb/index.rb:73:in `recurse'
    from /tmp/.rbenv/versions/2.3.0-preview2/gemsets/innodb_ruby_test/gems/innodb_ruby-0.9.13/bin/innodb_space:1274:in `index_recurse'
    from /tmp/.rbenv/versions/2.3.0-preview2/gemsets/innodb_ruby_test/gems/innodb_ruby-0.9.13/bin/innodb_space:1834:in `<top (required)>'
    from /tmp/.rbenv/versions/2.3.0-preview2/gemsets/innodb_ruby_test/bin/innodb_space:23:in `load'
    from /tmp/.rbenv/versions/2.3.0-preview2/gemsets/innodb_ruby_test/bin/innodb_space:23:in `<main>'

Do I have something wrong?
Please tell me correct usage.

P.S.
The following will work correctly.

bundle exec innodb_space -s /usr/local/var/mysql/ibdata1 -T innodb_ruby_test/users -r innodb_ruby_test_db.rb -d InnodbRubyTest_users_state -I state_idx index-recurse

and index added manually.
https://gist.github.com/kysnm/44ea60a72546c9860093