rdaly525 / linker

Repo for the coreir linker paper

linker

Experiments with Foreign Generator Interfaces and Staged Generator Compilation

linker's Issues

cullgraph deletes too much

Suspected that cullgraph does not look at any of the generated modules.

Magma is main, verilog is linked

I think if we got this example fleshed out with a parameterized verilog module, most other examples could be straightforward.

Lenny, could you provide an example of how this is currently done in Magma?

Proposed path (ROSS TODO)
I think there should be a non-magma/python specific way to get the declarations of the verilog modules as a function of the parameters. What this means is that I should be able to pass in some collateral into yosys which is basically a list of modules with parameter arguments. Yosys should then be able to output the materialized module declarations. This is equivalent to 'running the typegen'.

Separately, magma should be able to compile to coreir, and then coreir linker will pass yosys the unresolved parameters and yosys should be able to 'run the generators' for the unresolved symbols.
The Linker will then be able to merge all the resolved symbols back into coreir for a final compiled coreir circuit.

Literature Review

We should do a literature survey to find possibly related work, we can collect the list of papers here and include notes/summaries from reading them

Examples/problems of software linking

https://github.com/jarrett/cpphs

C++ is main, Magma is linked

Segfault in popcount example if I don't run printer pass

If I take out the printer pass:

diff --git a/experiment/cpp_magma_popcount/create_popcount.cpp b/experiment/cpp_magma_popcount/create_popcount.cpp
index b5c742a..0953d9f 100644
--- a/experiment/cpp_magma_popcount/create_popcount.cpp
+++ b/experiment/cpp_magma_popcount/create_popcount.cpp
@@ -30,7 +30,7 @@ int main() {
   top->setDef(def);
   c->setTop(top);
-  c->runPasses({"rungenerators", "cullgraph", "printer"});
+  c->runPasses({"rungenerators", "cullgraph"});
   if (!saveToFile(c,"popcount.json")) {
     cout << "Save to file Failed!!!";
     exit(1);

I'll get an occasional segfault:

❯ PYTHONPATH=. make
./create_popcount
Generator: PopCount
    Params: (width:Int)
    TypeGen: TODO
    Def? No
Generator: PopCount
    Params: (width:Int)
    TypeGen: TODO
    Def? Yes
hasdef:1
In Run Generators
Running magma generator
Done running generators
make: *** [counters.json] Segmentation fault: 11

If I run it in lldb, i get the following output

❯ PYTHONPATH=. lldb -- ./create_popcount
(lldb) target create "./create_popcount"
Current executable set to './create_popcount' (x86_64).
(lldb) r
Process 26632 launched: './create_popcount' (x86_64)
Generator: PopCount
    Params: (width:Int)
    TypeGen: TODO
    Def? No
Generator: PopCount
    Params: (width:Int)
    TypeGen: TODO
    Def? Yes
hasdef:1
In Run Generators
Running magma generator


















Done running generators
Process 26632 exited with status = 0 (0x00000000)
(lldb) r
Process 26637 launched: './create_popcount' (x86_64)
Generator: PopCount
    Params: (width:Int)
    TypeGen: TODO
    Def? No
Generator: PopCount
    Params: (width:Int)
    TypeGen: TODO
    Def? Yes
hasdef:1
In Run Generators
Running magma generator


















Done running generators
Process 26637 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=EXC_I386_GPFLT)
    frame #0: 0x000000010029de3c libcoreir.dylib`CoreIR::GlobalValue::getRefName() const + 28
libcoreir.dylib`CoreIR::GlobalValue::getRefName:
->  0x10029de3c <+28>: movq   0x20(%rsi), %rcx
    0x10029de40 <+32>: movq   %rdi, -0x170(%rbp)
    0x10029de47 <+39>: movq   %rcx, %rdi
    0x10029de4a <+42>: movq   %rax, -0x178(%rbp)
Target 0: (create_popcount) stopped.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=EXC_I386_GPFLT)
  * frame #0: 0x000000010029de3c libcoreir.dylib`CoreIR::GlobalValue::getRefName() const + 28
    frame #1: 0x00000001003c3fa4 libcoreir.dylib`(anonymous namespace)::Instances2Json(std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, CoreIR::Instance*, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, CoreIR::Instance*> > >&, int) + 1220
    frame #2: 0x00000001003bb8c5 libcoreir.dylib`(anonymous namespace)::Module2Json(CoreIR::Module*, int) + 7685
    frame #3: 0x00000001003be564 libcoreir.dylib`(anonymous namespace)::Generator2Json(CoreIR::Generator*) + 2740
    frame #4: 0x00000001003b6898 libcoreir.dylib`CoreIR::Passes::CoreIRJson::runOnNamespace(CoreIR::Namespace*) + 3080
    frame #5: 0x0000000100326217 libcoreir.dylib`CoreIR::PassManager::runNamespacePass(CoreIR::Pass*) + 439
    frame #6: 0x0000000100328e9a libcoreir.dylib`CoreIR::PassManager::runPass(CoreIR::Pass*) + 378
    frame #7: 0x000000010032c3c6 libcoreir.dylib`CoreIR::PassManager::run(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >) + 5190
    frame #8: 0x0000000100149a16 libcoreir.dylib`CoreIR::Context::runPassesOnAll(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >) + 1974
    frame #9: 0x0000000100274e34 libcoreir.dylib`CoreIR::saveToFile(CoreIR::Context*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >) + 2820
    frame #10: 0x000000010001912b create_popcount`main + 7467
    frame #11: 0x00007fff504ec015 libdyld.dylib`start + 1
    frame #12: 0x00007fff504ec015 libdyld.dylib`start + 1
(lldb)

Experiment list

Lets keep a list of experiments which to be able to handle, and an associated issue for each one.

We need the cross product of main is (c++,magma,verilog,Genesis,Chisel) and linked lib is (c++,magma,verilog,Genesis,Chisel)

For each of these languages, we need to know how to represent a blackbox module (header) within the language itself. Both Modules and Parameterized modules.

I think the following three blackbox modules would work great. as black boxes to link in.

//No parameters
Module adder(
  input [15:0] in0,
  input [15:0] in1,
  output [16:0] sum);

//Simple width parameter
Module #(parameter width) param_adder(
  input [width-1:0] in0,
  input [width-1:0] in1,
  output [width:0] sum);

//Parameter which changes the interface
Module #(parameter width, has_cin) param_adder(
  input [width-1:0] in0,
  input [width-1:0] in1,
  [input cin,]
  output [width:0] sum);

Index of issues

C++ is 'main' (Halide compiler for example), C++ is linked #4

C++ is 'main', magma is linked #5

C++ is 'main', verilog is linked

Magma is 'main', C++ is linked

Magma is 'main', Magma is linked

Magma is 'main', Verilog is linked #6

Verilog is 'main', magma/C++ is linked library.

Magma is 'main', Chisel is linked library
Chisel is 'main', magma is linked lib (peripheral to rocketchip?)

C++ is main, Linked is C++

I have already started this in linker/examples