artichoke / artichoke Goto Github PK

Likely as part of the existing String converter

Remove dependency on mruby-random mrbgem.

See the Ruby docs: https://ruby-doc.org/core-2.6.3/Regexp.html#method-i-3D-7E

Implement the Ruby 2.6.3 Standard Library.

This ticket tracks the scheduling and completion of the [Standard Library Milestone]. Checkmarks mean a ticket to track package implementation has been created.

• abbrev - GH-74
• base64 - GH-75
• benchmark - GH-76
• bigdecimal
• cgi
• cmath
• coverage
• csv
• date
• dbm
• debug
• delegate - completed artichoke/ferrocarril#110
• digest
• drb
• e2mmap
• English
• erb
• etc
• expect
• extmk
• fcntl
• fiddle
• fileutils
• find
• forwardable - completed artichoke/ferrocarril#108
• gdbm
• getoptlong
• io/console
• io/nonblock
• io/wait
• ipaddr
• irb
• json - pure Ruby implementation completed artichoke/ferrocarril#126
• logger
• mathn
• matrix
• mkmf
• monitor - completed for single-threaded implementation artichoke/ferrocarril#89
• mutex_m
• net/ftp
• net/http
• net/imap
• net/pop
• net/smtp
• net/telnet
• nkf
• objspace
• observer
• open-uri
• open3
• openssl
• optparse
• ostruct - completed artichoke/ferrocarril#112
• pathname
• prettyprint
• prime
• profile
• profiler
• pstore
• psych
• pty
• racc
• racc/parser
• rake
• rdoc
• readline
• resolv
• resolv-replace
• rexml
• rinda
• ripper
• rss
• rubygems
• scanf
• sdbm
• securerandom - GH-34
• set - completed artichoke/ferrocarril#111
• shell
• shellwords
• singleton
• socket
• stringio
• strscan - completed artichoke/ferrocarril#119
• sync
• syslog
• tempfile
• thwait
• time - GH-33
• timeout
• tmpdir
• tracer
• tsort
• un
• unicode_normalize
• uri - WIP in artichoke/ferrocarril#85 - completed artichoke/ferrocarril#125
• weakref
• webrick
• win32ole - GH-296
• yaml
• zlib - GH-100

A space to jot down things I've learned about the Ruby VM.

This would allow typing the block parameter in funcall_with_block.

These implementations of Ruby core methods have been mostly buggy and not spec-compliant.

Use them as a starting point to implement them natively in Rust/evaled Ruby with the goal of passing ruby/spec.

• mruby-compar-ext
• mruby-enum-ext
• mruby-numeric-ext
• mruby-array-ext
• mruby-hash-ext
• mruby-range-ext
• mruby-proc-ext
• mruby-symbol-ext
• mruby-object-ext
• mruby-kernel-ext
• mruby-class-ext

The long term plan is to pull all of mruby into artichoke with a native Rust implementation. Document how we'll get there.

Global functions are always implemented on top_self so they do not take a parent sys::RClass *. We also likely want to keep a separate registry for them to make them easier to create.

Once we implement support for proc-macros (see #38), these functions will have a separate creation pathway, too.

The focus of this repository is changing and the scope is growing. We are building a Ruby 💎!

Artichoke aims to be a source-compatible, ruby/spec compliant implementation of Ruby 2.6.3 written in safe Rust (excluding crate dependencies).

Go get there, we'll use the mruby base and continue to rip parts of it out of libmruby.a into the mruby crate with the goal of passing as many of the core specs as possible only by removing dependencies on mrbgems.

Complete the implementation of Ruby stdlib.

Implement File and IO in Ruby core.

Once the mrbgem dependencies are minimized, start reimplmenting the mruby C API in Rust one header at a time. There are 311 C functions.

Then we'll be left with the compiler, parser, GC, mrb_state, and VM. Figure out a way to migrate these into Rust.

Build a ruby CLI frontend.

Build support for concurrent Threads.

In MRI, Time is both a Core class and a Standard Library package.

The Core class is implemented in C, see time.c in CRuby. Reimplement this class in Rust with httpdate and chrono.

The Standard Library package is pure Ruby and can likely be imported as is once the Core class is in place. See GH-158 for an example of how to do this.

Remove dependency on mruby-time mrbgem.

Currently, the load path for require is hardcoded as /src/lib. This value should be exposed via the $LOAD_PATH global and Kernel#require should respect this configuration.

• $LOAD_PATH global
• read $LOAD_PATH in require Rust implementation
• support trying multiple load locations

NOTE: mruby does not support aliasing globals so we should punt on implementing the $: shorthand alias (even though we could with shared Rust state and the proposed global registry, see artichoke/ferrocarril#14).

Ideally lock-free. chashmap is one possible crate we could use.

Support the following from Rust:

module Inc
  def inc; end
end

module Mod
  extend Inc
end

class Cls
  extend Inc
end

Add a doc highlighting significant milestones and markers of progress with links to commits and PRs.

Make the example a little more interesting. This will likely require Regexp (see #48).

require 'json'

class A
  def bar(x)
    { bar: x }.to_json
  end
end

10.times do |i|
    puts A.new.bar(i)
end

Add tests once File is implemented and we can manipulate the VFS from Ruby code.

Blocked on #10.

• Wasm
• native
• cross

Implement the File class in Rust so that it is backed by the vfs on the State.

If an exception is thrown in the context of a C API call, the returned mrb_value is of type mrb_vtype::MRB_TT_EXCEPTION. Turn this exception into an Err(MrbError::Exec(_)).

Maybe as part of a funcall interface on Value?

Triggers unreachable code panics.

Currently disabled with conditional compilation because of a link error under emscripten.

Once #28 lands, Value is a valid key for a HashMap.

Implementing Convert<HashMap<Value, Value>> allows us to follow the same approach as Array and massively simplify the macro madness in hash.rs.

I think this should also unify the special-cased &str converters.

mruby is not thread safe, which means we cannot use it as a base for a multi-threaded Ruby implementation.

Milestone 4: Multi-threading is blocked on the completion of Milestone 13: VM which will eliminate Artichoke's dependency on mruby.

Implement a derive + attribute proc-macro based generator (like structopt) for artichoke_core::file::File impls to allow arbitrary Rust functions and types to be loaded into an interpreter.

See https://github.com/PyO3/pyo3 for inspiration.

There are some shenanigans going on to do is_a? computations in Rust. Extract the fully qualified class name for a value when constructing its Ruby type. This will additionally enable doing instance of comparisons for classes that haven't been defined in Rust.

https://github.com/lopopolo/ferrocarril/blob/ba0070eac0e54d793af9f7ed765c116cc194a51f/mruby/src/extn/core/regexp.rs#L257-L265

The vendored version of mruby has a bug where in some situations comments do not increment the line number in the parser. This breaks tests which test stack traces. thread.rs has one of these tests that has been disabled. Pull in the upstream bugfix and re-enable the test.

Upstream bug: mruby/mruby#4513

Disabled test:

https://github.com/lopopolo/ferrocarril/blob/329c544006b619b7b4fac9c96b6b5ec51a311a84/mruby/src/extn/core/thread.rs#L155-L194

Right now, the Artichoke VM type is an Rc<RefCell<State>>. None of these types are thread safe.

Ideally, Artichoke is Arc<State> where state is made up of thread-safe, lock-free data structures.

This work is blocked on GH-69.

Support the following from Rust:

module Inc
  def inc; end
end

module Mod
  include Inc
end

class Cls
  include Inc
end

Just make Define mutable and store the RClass. In ClassLike::rclass, short circuit if rclass is Some and resolve the RClass using the mruby C API. Don't store this result to keep the API simple and not require mutability.

Right now path canonicalization and absolutifying lives in the implementation of Kernel#require. This means that I think this code behaves unexpectedly:

interp.def_rb_source_file("../spec_helper.rb", "")

Move path canonicalization into the fs layer.

This code looks like it should work but it panics:

let metrics = interp
            .borrow()
            .module_spec::<Metrics>();
            .and_then(|spec| spec.borrow().value(&interp));

See if there is a way to fix this. Maybe by interning the symbol when we have the mutable borrow in def_module and passing it to module::Spec?

see also #21.

Run Ruby source files on a fully initialized Mrb interpreter.

https://doc.rust-lang.org/std/ptr/fn.write.html

Now that each Value has an interpreter associated with it (did not used to be the case), we can implement Hash with funcall("hash") and PartialEq with funcall("==", &[other])

This will enable storing Value in a HashMap, see artichoke/ferrocarril#160.

There is a gnarly state machine and String processing going on. Needs tests, which should help as we modify this function to be backed by a nom parser.

CRuby:

[2.6.3] > "\xFE".ord
Traceback (most recent call last):
        5: from /usr/local/var/rbenv/versions/2.6.3/bin/irb:23:in `<main>'
        4: from /usr/local/var/rbenv/versions/2.6.3/bin/irb:23:in `load'
        3: from /usr/local/var/rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/irb-1.0.0/exe/irb:11:in `<top (required)>'
        2: from (irb):2
        1: from (irb):2:in `ord'
ArgumentError (invalid byte sequence in UTF-8)

rirb:

>>> "\xFE".ord
=> nil

Right now, MrbError::Exec wraps a stringified stack trace.

The MrbExceptionHandler exposes a nice Exception struct that exposes exception class, message, optional backtrace, and the result of exc.inspect. It also implements display.

Wrap this Exception struct instead.

Currently, regexp::init defines constants with eval and format!.

Add an API that allows defining a constant that takes anything that implements FromMrb<Value> in the same way that you add a method to these specs.

The gymnastics and sad times in artichoke/ferrocarril#168 become unnecessary if we have a memory safe unwind. panic is exactly that.

Implement mrb_raise, mrb_raisef and mrb_protect in Rust in mrb-sys exported to C so libmruby can use them.

TODO: track the compliance of all core and stdlib functionality of mruby crate.

https://ruby-doc.org/core-2.6.3/IO.html

The IO class is a base class for all IO streams in Ruby.

The mruby implementation might be a good starting point.

The deliverable for this ticket should be to implement enough of IO to implement File, see GH-10.

Support the wasm32-unknown-unknown target, which would enable using a native web Wasm runtime and unlock the ability to use wasm-bindgen, stdweb, and lots of other goodies in the playground.

With our fork of rust-onig, the only thing remaining to support this is getting mruby building under wasm32-unknown-unknown.

There should be a test that a newly initialized Mrb has a strong count of 1.

There should be a test that the Mrb is deallocated on drop by downgrading the Mrb to a Weak and assert!(weak.upgrade().is_none()).

https://github.com/lopopolo/ferrocarril/blob/8910d5d451b0bfb522153c3b85176c7bb60c7dcd/mruby/src/interpreter.rs#L62-L65

This branch: https://github.com/artichoke/rust-onig/tree/wasm

I'm imagining that each part of Ruby core and stdlib we add to mruby crate should live as a dynamically loadable module.

This would eventually allow us to expose individual extensions that alter the execution environment as features for conditional compilation.

This code was written when I knew what I was doing less than I do now. There is some UB in here. Please audit this whole file and fix it.

This is the algorithm for Fixnum:

[2.6.0] > 122732743347.object_id
=> 245465486695
[2.6.0] > ((122732743347 << 1) | 0x1)
=> 245465486695
[2.6.0] > -122732743347.object_id
=> -245465486693
[2.6.0] > (0x1 << 1) - ((122732743347 << 1) | 0x1)
=> -245465486693