esy / pesy

Project configuration for esy

pesy

pesy is a command line tool to assist with your native Reason/OCaml workflow using the package.json itself.

1. Quickly bootstrap a new Reason project
2. Configure your builds

Installation

npm install -g pesy@next

Why pesy?

esy is driven by a package.json to bring the familiar NPM workflow to the native world of Reason/OCaml. pesy takes this further by adopting NPM conventions to configure the build.esy's package.json first approach in creating developer sandboxes brings interesting possibilities to the table. pesy leverages on those features to make native development both convenient and familiar.

pesy provides

1. A bootstrapper script to quickly create a project template
2. An alternative JSON syntax around Dune that is NPM like
3. Built in tasks that take full advantages of esy's capabilities

Getting started

Creating a new project

pesy can quickly bootstrap a basic native Reason/OCaml project.

cd my-new-project/
pesy

Adding project dependencies

Say, we need @opam/yojson and @reason-native/console in App.re.

Place App.re in a folder (say, bin/?) and add the following to you package.json.

"buildDirs": {
  "bin": {
    "imports": [
	  "Json = require('@opam/yojson')",
	  "Console = require('@reason-native/console/lib')"
	]
  }
}

Run esy pesy (once). Run esy after that to create builds.

pesy abstracts library identifiers (in this case yojson and console.lib) in file paths conceptually. Use native dependencies from OCaml like you did with NPM!

Where pesy shines

However, pesy's is truly useful for frameworks that need a lot dependencies and configuration. Ex: Morph and Revery

pesy --template pesy/template-revery --directory my-new-project

If you are authoring a framework, create a template like pesy/template-revery and run the project on the CI with a setup that pesy creates for you.

You users get cached CI artifacts that will let can hydrate their local esy cache avoid long wait times! See Templates to see how to create such templates.

Looking for a tutorial?

Checkout A simple native Reason project with pesy to get an idea of what developing with pesy feels like.

Relation to Dune

Pesy as accepts package.json as input and producing dune files as output.

                  +--------+
package.json ---> |  pesy  | +--->  dune files
                  +--------+

Note that not all of Dune's features are supported in pesy (PRs welcome). pesy doesn't intend to duplicate Dune's efforts - it's meant to compliment it. For simple use cases, pesy wants to provide a NPM friendly interface to Dune so that newcomers can quickly get started, without confusing themselves with the library vs packages nuances.

Namespacing

Every library, as we know, exposes a namespace/module under which it's APIs are available. However, as package authors, it can hard to make sure one is not using a namespace already taken by another package (Otherwise it could lead to collisions). Pesy works around this by assigning the library the upper camelcase of the root package name and directory the library/sub-package resides in.

Example: if a package.json looks like this

{
  "name": "@myscope/foo",
  "buildDirs": {
    "library": { ... }
  }
}

Then, subpackage library takes a namespace of MyScopeFooLibrary. As a user, however, you shouldn't have to worry much about yourself, since you can specify how you'd like to import subpackages (and packages). In the above example, another subpackage would consume it as follows

{
  "name": "@myscope/foo",
  "buildDirs": {
    "library": { ... },
    "anotherLibrary": {
      "imports": [
        "ThatOtherLibrary = require('@myscope/foo/library')"
      ]
    }
  }
}

And if you were consuming this package (after having published to npm), you can import it as follows:

{
  "name": "bar",
  "buildDirs": {
    "library": {
      "imports": [
        "ThatFooLibrary = require('@myscope/foo/library')"
      ]
    }
  }

With the new NPM like conventions, pesy automatically handles the namespace for you so that we don't have to worry about the nuances of a package and a library during development.

Dune files

pesy generates dune files on fly behind the scenes. And to be able to so, it needs a static dune file that looks like the following

(* -*- tuareg -*- *)

open Jbuild_plugin.V1

let () =
  run_and_read_lines ("pesy dune-file " ^ Sys.getcwd ())
  |> String.concat "\n"
  |> send

These have to created only once - after that they never change (unless you decide to eject). Every bootstrapped project has them already and you dont have to create them. In case, you need to re-generate them, run esy pesy (or esy @mysandbox pesy if you are using esy sandboxes). A common need for running esy pesy is when you add a new folder to your project.

Ejecting:

It is always possible to eject out of pesy config by running esy pesy eject ./subpackage-path

CI & local cache Warming up

Compilation artifacts built on the CI can be downloaded by pesy warmup. At the moment, we only support Azure Pipelines can be configured as follows.

{
  "pesy": {
    "azure-project":
      "<azure-project>/<azure-pipeline-name>",
    "github":
      "<github-org>/<github-repo>"
  }
}

This will fetch appropriate artifacts compiled for the current machine. Note however, the best way to get this feature to work is to use cache publish and restore mechanism provided bootstrapped files. pesy assumes that the artifact zip file names and/or paths haven't changed.

How it works

Compilation artifacts created by esy are relocatable - since esy sandboxes isolated, all the dependencies are accounted for and each dependency is loaded from a path with a fixed-length prefix. Such artifacts can be built on one machine and used on other provided the prefixes are rewritten to reflect the updated path on the new machine. And esy provides all the low level command to do this out of the box. pesy simply provides a convenient wrapper that drives them.

When the project is bootstrapped for the first time, it is identical to a copy that is run on the server. This is where the first set of cached builds come from, which is why the azure-project in the pesy config is set to esy-dev/esy and github to esy/pesy-reason-template.

Once the project sees changes, you most probably would add more dependencies (or remove some) which could change the build sandbox state. It is recommended that you run the provided CI setup to cache builds on your own Azure Pipelines instance and update github and azure-project accordingly.

Publishing and Consuming from NPM

Publishing libraries

Easiest way to get started with distributing you library is to publish the source to NPM.

Let's take a look at an example.

Consider a base package foo that you created and distributed on NPM. And let's assume, bar is the package that consumes foo.

$ pesy --directory foo-lib

This would have bootstrapped a project with the default template with Util.re in the library/ folder.

Let's publish it

$ npm publish

Consuming foo-lib

Let quickly create a new project, bar and add foo.

$ pesy --directory bar
$ esy add foo-lib

We can now require foo

  "buildDirs": {
    "library": {
      "imports": [
+        "FooLib = require('foo-lib/library')"
      ]
    }
  }

And then edit Utils.re

let foo = () => {
  FooLib.Util.foo();
  print_endline("This is from bar");
};

$ esy
$ esy start
Hello from foo!
This is from bar

Templates [experimental]

Usage

To use a custom template run pesy --template=github:your-name/your-pesy-template

Creating your own template

This is a experimental feature that could see a lot of churn. We request you to watch the issue tracker for updates.

It works by downloading a git repo and then replacing special strings in filenames and files inside the repo. The special strings are currently these:

In file names:

In filename	Replaced with
__PACKAGE_NAME__	package_name
__PACKAGE_NAME_FULL__	package_name
__PACKAGE_NAME_UPPER_CAMEL__	PackageName

In file content:

In contents	Replaced with
<PACKAGE_NAME>	package_name
<PACKAGE_NAME_FULL>	package_name
<PACKAGE_NAME_UPPER_CAMEL>	PackageName
<VERSION>	version
<PUBLIC_LIB_NAME>	package_name/library
<TEST_LIB_NAME>	package_name/test

Best way to get started creating a new template is to download https://github.com/esy/pesy-reason-template and work on it. Any changes can be tested with pesy test-template.

Supported Dune Config

This is reference guide explaining the config pesy supports.

Binaries

Configuration that applies to subpackages that create binary executables. Note that these executables can be ocaml bytecode or native binaries (ELF/Mach/PE)

bin : string

A subpackage produces binary when it contains a bin property.

{
  "buildDirs": {
    "src": {
      "bin": "Main.re"
    }
  }
}

Here is a complete, working example

modes : array(string)

An array of advanced linking modes. Each string should be of the form "compilation-mode binary-kind" where compilation-mode is one byte, native or best and binary-kind is one of c, exe, object, shared_object.

{
  "buildDirs": {
    "src": {
	  "bin": "Foo.re",
	  "modes": [ "native", "exe"] 
	}
  }
}

Here is a complete, working example

name (override) : string

A string that maps to Dune's public_name. Usually unnecessary (as bin property takes care of it) and must only be used to override.

main (override) : string

A string that maps to Dune's name. Usually unnecessary (as bin property takes care of it) and must only be used to override the entry point.

Libraries

modes : array(string)

modes can be used to configure the compilation target - native or bytecode An array of string, any of byte, native, best

"byte"

This mode generates byte code output

"native"

This mode generates native output

"best"

Sometimes it may not be obvious if native compilation is supported on a machine. In such circumstances, use "best" and "native" will be picked for you if it's available. Else, it'll be "byte"

cNames : array(string)

When writing C stubs to FFI into a library, simply mention the file name (without the .c extension) in the cNames field.

{
  "buildDirs": {
    "src": {
      "cNames": ["my-stub1", "my-stub-2"]
	}
  }
}

foreignStubs : list(ForeignStub)

From dune version 2.0 onwards the cNames field was removed and foreignStubs field was introduced to provide the FFI functionality, foreignStubs is a list of objects, where each foreignStub object should specify language, names & flags. Incase names & flags is not specified or empty, their default value will be considered Refer this for default values for names & flags

{
  "buildDirs": {
    "src": {
      "foreignStubs": [
        {
          "language": "c",
          "names": ["my-stub1", "my-stub-2"],
          "flags": ["-verbose"]
        } 
      ]
	}
  }
}

Here is a complete, working example

Config supported by both libraries and binaries

imports : array(string)

imports can be used to import a library (from a subpackage or an external npm/opam package) and utilise the namespace of the imported library.

{
  "buildDirs": {
    "src": {
	  "imports": [
	    "Console = require('console')"
	  ]
	}
  }
}

The above config makes a namespace Console available inside the subpackage src. Now any .re file inside src can use the console library.

// src/SomeFile.re
let foo = () => Console.log("Hello, world")

We can also import a package/subpackage under a different namespace

{
  "buildDirs": {
    "src": {
	  "imports": [
	    "NotConsole = require('console')"
	  ]
	}
  }
}

And we can import (oddly confusing) NotConsole from in src

// src/SomeFile.re
let foo = () => NotConsole.log("Hello, world");

We can also import other subpackages in the project.

{
  "buildDirs": {
    "src": {
      "bin": "Main.re",
      "imports": [
        "FooConsole = require('console/lib')",
        "MyOwnLibrary = require('../my-own-lib')"
      ]
    },
    "my-own-lib": {}
  }
}

Here is a complete, working example

Compiler flags

All of type list(string)

• flags - These flags will be passed to both the bytecode compiler and native compiler
• ocamlcFlags - These will be passed to ocamlc - the bytecode compiler
• ocamloptFlags - These will be passed to ocamlopt - the native compiler
• jsooFlags - These will be passed to jsoo compiler - the javascript compiler. Note: This is unrelated to Bucklescript

Preprocessor flags : list(string)

preprocess accepts options needed to pass the source files via a preprocessor first. When using custom syntax not natively supported in the compiler, we pass the sources in a subpackage via a preprocessor first.

For example, suppose we'd like to use let%lwt syntax for our Lwt promises

let%lwt foo = Lwt.return("foo");
print_endline(foo);

// instead of 
Lwt.return >>=
  (foo => print_endline(foo); Lwt.return())

We specify lwt_ppx in pps preprocess

{
  "buildDirs": {
    "src": {
	  "bin": "Main.re",
	  "preprocess": ["pps", "lwt_ppx"]
	}
  }
}

Here is a complete, working example

includeSubdirs : string = "no"|"unqualified"

Default is "no", and changing to "unqualified" will compile modules at deeper directories.

Escape hatches for unsupported Dune config

It's always possible that there are features Dune offers are needed and the options above are not enough. Use rawBuildConfig to add options in a given library or binary. Use rawBuildConfigFooter to add config to the footer.

rawBuildConfig : list(string)

Example

{
  "src": {
    "rawBuildConfig": [ "(libraries unix)" ],
    "bin": "Main.re"
  }
}

rawBuildConfigFooter : list(string)

Example

{
  "src": {
    "rawBuildConfigFooter": [
      "(install (section share_root) (files (plaintext.txt as asset.txt)))"
    ]
  }
}

Here is a complete, working example

Tutorials

Simple native example

CLI Apps

Web Development with Morph

Development

Clone the repo and run esy on it.

e2e tests

./_build/install/default/bin would contain (after running esy) Runner.exe. Runner.exe looks for PESY_CLONE_PATH variable in the environment to find pesy source. Set it to the path where the project was cloned.

To test if simple workflows work as expected. They assume both esy and pesy are installed globally (as on user's machines).

Changelog

version 0.4.0 (12/21/2018)

• Allow buildDirs to contain deeper directories such as "path/to/my-lib": {...}".
• Added support for wrapped property on libraries.
• Added support for virtualModules and implements - properties for Dune virtual libraries. (This will only be supported if you mark your project as Dune 1.7 - not yet released).
• Stopped using ignore_subdirs in new projects, instead using (dirs (:standard \ _esy)) which only works in Dune 1.6.0+, so made new projects have a lower bound of Dune 1.6.0.
• Support new properties rawBuildConfig which will be inserted at the bottom of the target being configured (library/executable).
  • It expects an array of strings, each string being a separate line in the generated config.
• Support new properties rawBuildConfigFooter which will be inserted at the bottom of the entire Dune file for the target being configured.
  • It expects an array of strings, each string being a separate line in the generated config.
• Support new properties modes for binaries and libraries list(string).

pesy's Issues

Skip aliasing if imported name is same as exported

The new npm style require syntax works by emiting modules that alias the namespace like this

(rule
    (with-stdout-to FooCliPesyModules.re
        (run echo  "module AliasedDep = DepNamespace;\
                  \nmodule AliasedDep2 = Dep2Namespace;")))

If users, especially experienced ones, know the namespace of the library beforehand and the use same,

"imports": [
  "Re = require('@opam/re')",
  "Pastel = require('@reason-native/pastel/lib')"
]

...creates...

(rule
    (with-stdout-to WebsiteDbCliPesyModules.re
        (run echo  "module Re = Re;\
                  \nmodule Pastel = Pastel;")))

This causes unnecessary indirection, noticed especially when merlin/ocamllsp shows long aliased names in the editor autocomplete.

By skipping (therefore linking such libraries directly), we could get cleaner editor completions.

Allow optional semicolons in the imports fields

(Azure Pipelines, pesy@next) error: unable to resolve command: pesy-configure

Seems like the esy-build-steps.yml file has a typo that cause the following error on Azure Pipelines:

error: unable to resolve command: pesy-configure

pesy/assets/azure-pipeline-templates/pesy-esy-build-steps.template.yml

Lines 11 to 12 in fba1134

	- script: esy pesy-configure
	displayName: 'esy pesy'

I believe it should be just esy pesy, or am I missing something?

Do not fetch github template repo if already fetched. Cached

templates must also take into account which version they are.

~/.pesy-cache/templates/pesy-hello-reason/v1/...files

Must cache user specified template too.

Investigate the use of Pastel for Bucklescript portion of the codebase

Currently, we are using ANSI escape sequences. It would be ideal to something more readable like Pastel.

Should executables also generate a library?

This came up in our adding of rely tests. For rely tests to work well, you want a library (with -link-all flags set), and then an executable entrypoint that requires that library.

What would be the harm in each executable also generating a library?

Instead of specifying the .re file, to make the binary entrypoint you could specify the library name.

  "buildDirs": {
    "thing": {
      "namespace": "Thing",
      "flags": ["-link-all"],
      "require": [ "bos.top" ],
      "bin": {
        "MyApp.exe": "my-package.thing"
      }
    },

With node, each library is an executable and it might be cool to also have a similar capability by default with pesy.

(Note that for Rely tests, we do need different flags for the library than for the executable, so we'd need some way to specify flags only for the exe or only for the library).

Prototype warming up sandbox cache

Pesy sets up the CI for the user's project and the CI is setup such that we create zip of all dependencies so that PRs can re-use previous build artifacts for faster runs.

With a few tweaks, pesy can also implement a feature to pull the zip file and warm up local cache right after we clone the project.

What do we need?

1. Use esy export-dependencies on the CI and zip it
2. Fetch the zip and run esy import-dependencies

Tentatively, this could be implemented as pesy warmup sub command to the npm-cli.

Feedback and querries are very welcome.

Validate templates passed via --templates

Now that templates can be provided by the user, pesy should ideally check for its validity so that build errors seen later on don't confuse the user.

Support test stanza

Dune supports producing binary test runners that are automatically added to runtest alias. Since every bootstrapped pesy project creates test sub-package too, we should support the stanza natives. This also prevents those test runner binaries from unnecessarily getting exposed as public binaries - something we are forced to do right now.

create revery structure with pesy

This project is very nice.

I wanna try to use Pesy to create Revery structure project is possible?

thanks

Validation: incorrect library import paths must be reported

    "core": {
      "imports": ["SharedUtils = require('foo-bar/share/dune')"]
                                                           ^^^^
    },
    "shared/dune": {}

It should be foo-bar/share*d*/dune. Pesy can catch these

Handle npm scoped packages

Since pesy no longer operates in the build environment, scoped package names must be handled by the user (@myscop/foo => myscope--foo). This seem inconvenient and confusing. We must figure a way around this.

Expand esy command expressions in pesy config

Using the trick used to figure out library path of an imported library, we could also determine other variables like #{@opam/zarith.lib}, #{@opam/zarith.etc} etc - paths in the install tree of a given package.

If needed, paths in the source tree can be looked up in _esy/installation.json

This will allow pesy configs like

{
  "flags": ["-ccopt -L#{esy-zlib.lib} -lz"]
}

Pesy (native) alpha question.

From Discord:

One thing that came up is that it's not clear how to use the bin field. Can you specify multiple binaries per directory? The syntax appears to allow that. If so, how do you specify which "requires" accompany each binary in that directory?
Or is the convention that each directory can have one binary, but all of the binaries in a directory must have the same set of requires?

I believe Dune only really allows one executable per directory (unless you do a bunch of complex stuff). Please feel free to close this with a simple answer.

refmterr should be devDependency.

When I make a new pesy project, I usually place a couple of changes:

    "build": "dune build -p #{self.name}",
    "buildDev": "refmterr dune build -p #{self.name}",

Then I move refmterr to a devDependency. That's so that this avoids putting additional dependencies that are only needed for development time.

c_names vs. foreign stubs.

I used the cNames field in pesy, which creates the c_names field in dune, but it looks like Dune deprecated that field, and says to use foreign_stubs.

Error: 'c_names' was deleted in version 2.0 of the dune language. Use the (foreign_stubs ...) field instead.

How should we deal with this? Can we detect which version of Dune is being used?

Unused lwt/unix import

Sometime, certain APIs require that more than one libraries be added to the build system. Example lwt and lwt.unix. Currently the require syntax forces assignment of the import into a namespace Lwt_unix = require('lwt/unix'), even if Lwt_unix is not really used anywhere.

Can't use preprocessors

When adding this to my package.json I get the a error

      "preprocess": [
        "pps",
        "graphql_ppx"
      ]

File "library/dune", line 5, characters 16-33:
5 |     (preprocess (pps graphql_ppx)))
                    ^^^^^^^^^^^^^^^^^
Warning: .merlin generated is inaccurate. Cannot mix preprocessed and non
preprocessed specificiations.
Split the stanzas into different directories or silence this warning by
adding (allow_approximate_merlin) to your dune-project.

To fix it I have to add (preprocess (pps graphql_ppx)) to both the library stanzas

Modes support:

I'm not sure if this is a new form for dune, but now the modes must/may be defined as described in the Dune manual

For instance the following modes fields are all equivalent:
(modes (exe object shared_object))
(modes ((best exe)
(best object)
(best shared_object)))
And finally, you can use the special mode byte_complete for building a bytecode executable as a > native selfcontained executable.

So apparently modes may be:

(modes (M k1 k2 ..kn) (N j1 j2 ...jn))

And k may now be js, which pesy doesn't seem to support.

When I try to do this in pesy:

      "modes": [
        ["byte", "exe"],
        ["byte", "js"]
      ]

I get the error:

pesy: internal error, uncaught exception:
      Lib__FieldTypes.ConversionException("Expected string. Actual list")

And when I just specify the JS one: "modes": ["byte", "js"] I get this error:

pesy: internal error, uncaught exception:
      Lib__Executable.Mode.InvalidBinaryKind(0)

Maybe we could clean up some of the errors too, but would be nice to support the more recent form of modes.

What about this approach to many of the fields: Instead of requiring a list in one place, or a list of lists in another - we accept any form, and merely warn when the form was not what is expected, but print the most reasonable interpretation (list of lists becomes ( ( ) () ) etc.
That way if the dune spec changes at least it can still generate valid dune even if it doesn't match our expectations.

Pesy must advise on an OCaml compiler version

If someone updates their pesy (local), it's possible that it depends on a different compiler version. This might lead to conflicts.

Why not set it as '*'? In the past, this has cause minor issues like when ocaml 4.8.0 was made available it cause the wrong version/outdated version of deps which were incorrectly packaged. to be pulled. Advising a compiler version (or a resolutions field) to the user would be nice way to handle this.

Add support for custom messages from templates

When #36 is merged it might make sense to add support for a file to print some custom message.
The file should be read, printed to stdout and then removed.

Add a more descriptive message if the required package is not found

Massage the opam requires at import time.

When we do:

        "YoJson = require('@opam/yojson')",

Then dune will say:

      Fl_package_base.No_such_package("opam--yojson", "")

because opam packages weren't created with pesy.

I think we should probably special case opam package names to not assume they abide by the convention? Or is that not a good idea?

Check if directory is empty before initializing project

Implement jsooFiles flag.

I didn't document it in jordwalke/pesy, but there's a jsooFiles config option per library.

https://github.com/jordwalke/pesy/blob/7903b66a4f546ba358be47594047abe88ac2d04a/pesy#L580

Migrate inline tests to Rely

After #26 , all inline unit tests have been disabled. They have to be migrated to Rely

EPERM issue with pesy@next

On windows 10, using [email protected] I get:

{ Error: EPERM: operation not permitted, rename 'bin/__PACKAGE_NAME_UPPER_CAMEL__App.re' -> 'bin/MorphGraphqlServerApp.re'    at Object.renameSync (fs.js:589:3)
    at C:\Users\ysg4206\AppData\Roaming\npm\node_modules\pesy\src\Pesy.bs.js:169:13
  errno: -4048,
  syscall: 'rename',
  code: 'EPERM',
  path: 'bin/__PACKAGE_NAME_UPPER_CAMEL__App.re',
  dest: 'bin/MorphGraphqlServerApp.re' }

I am running in an admin console

Prettify generated dune files with Format from stdlib

• https://caml.inria.fr/pub/docs/manual-ocaml/libref/Format.html
• https://hal.archives-ouvertes.fr/hal-01503081/file/format-unraveled.pdf
• https://gist.github.com/keleshev/5af7502e6810b8986d9f5332e7e8a795

"pesy project assist" feature [Brainstorm]

Pesy would be a great utility for detecting and reporting common issues with your project setup.
This Issue is to accumulate common issues that pesy is well positioned to detect and warn about.
We don't have enough requirements gathered to actually implement something like this feature but let's discuss the possibilities.
Basically, any time someone is confused by something with native projects, and they ask about it in the Discord support, is there some way that pesy can automatically detect the scenario and preemptively warn about it?

Workflow

• pesy detects something with your project. Sometimes it can do so when running esy pesy and sometimes when running esy (if pesy is part of their build command).
• Warnings could be shown once per project config.
  -- An issue is detected and a warning is shown when running esy pesy or esy.
  -- The output is printed with the same warning/error format as refmterr.
• pesy snooze could snooze all warnings for your project.
  -- The warning is not shown again if nothing changes about your project.
  -- If you change your package.json, or file directory structure, then the warning could be shown again.
• pesy snoozethat could just snooze the most recently shown warning (recording just that one in your package.json config)
• pesy silence/pesy silencethat could be like snooze but disable warnings for the project permanently including for new checkouts (it could store them in the package.json or something).

Ideas for issues to detect

• Your esy cache includes artifacts generated by different versions of gcc, or a version that is different than the one in your esy build environment.
• Your build config refers to modules for which there are no .re files etc when it can easily be detected.
• Your package.json includes things in dependencies that should probably be listed as devDependencies.

(Please reply if you have any ideas about common pitfalls we could automatically detect)

Problem running pesy on latest preview version.

npm install -g [email protected]
mkdir pesy-tmp && cd pesy-tmp
pesy

Output:

macbook-pro-41:pesy-tmp jwalke$ which pesy
/Users/jwalke/.npm-packages/bin/pesy
macbook-pro-41:pesy-tmp jwalke$ pesy
'esy pesy' failed. Dune files could not be created. -
 Logs can be found in pesy.stdout.log and pesy.stderr.log
macbook-pro-41:pesy-tmp jwalke$ ls pesy.std
pesy.stderr.log  pesy.stdout.log
macbook-pro-41:pesy-tmp jwalke$ cat pesy.stdout.log
macbook-pro-41:pesy-tmp jwalke$ cat pesy.stderr.log
error: project is not installed, run `esy install`
esy: exiting due to errors above

Add e2e tests for the newly added workflow

Multi compiler pipeline

Pesy atleast 4.6.1000 We need a pipeline on the CI to test against all the compiler versions.

Command to list packages that can be required

Similar to esy ls-libs but in a form that can required

Provide convenient rules for Atdgen

Atdgen needs rules like these

(rule
 (targets example_j.ml
          example_j.mli)
 (deps    example.atd)
 (action  (run atdgen -j -j-std %{deps})))

(rule
 (targets example_t.ml
          example_t.mli)
 (deps    example.atd)
 (action  (run atdgen -t %{deps})))

Right now, we have to use the following,

{
      "rawBuildConfigFooter": [
        "(rule (targets example_j.ml example_j.mli) (deps   example.atd) (action  (run atdgen -j -j-std %{deps})))",
        "(rule (targets example_t.ml example_t.mli) (deps   example.atd) (action  (run atdgen -t %{deps})))"
      ]
}

Probably should tell people to use the `esy` shortcut instead of running `esy build`.

esy pesy

    Created package-json-to-opam.opam

Ready for esy build

I don't think there's a reason to use esy build instead of just esy.

Documentation support

Can we add something like esy pesy docs?

What it should do is something like this:

1. check if the package.json have "@opam/odoc": "*" as a dependency.
   a. if it's there move to 2
   b. if it's not there, add it and run esy
2. run esy dune build @doc

`pesy eject` command

This could remove

1. pesy from dependencies
2. Remove buildDirs
3. Clean up dune files

(pesy@next) no error when flags are a string

      "flags": "foo",

Doesn't error when I run esy pesy or esy build. It doesn't turn the string into Dune config because it's not an array. I'm not sure what to do here.
I think we can either just dump the string as all the flags or error on esy pesy saying it needs to be an array. I don't feel strongly, just good to let the user know something is wrong.

Create a new repo with pesy is failing at the moment

https://gist.github.com/sync/27740d5f52983b979883ba961f24be77

_/i/ocaml-4.7.1004-e5a2754f/lib/ocaml/topfind: Operation not permitted

if you change your package to this it works (downgrading ocamlfind to pre 1.8.0):

 "dependencies": {
    "@opam/dune": "*",
    "@esy-ocaml/reason": "*",
    "refmterr": "*",
    "ocaml": "<4.7.0",
    "pesy": "*",
    "@reason-native/rely": "*",
    "@opam/ocamlfind": "1.7.3"
  },

So it seems to be related to latest ocamlfind

I am using [email protected] and what i did was:

mkdir monorepo
cd monorepo
pesy

Replicate sha256 verification mechanism for bootstrapped projects

The CI pipeline for pesy includes a setup that display the checksum for the prebuilt pesy binaries. Projects bootstrapped using pesy can benefit from such a CI project too

Figure out a Module_test.re convention as seen in the JS world

Dune let's us abstract a package by using a module of the same name as the package. If Foo.re is present in a package namespaced under Foo, other modules can be hidden

pesy currently doesn't do this, but it would be really ideal if it could do this using Index.re. (idea credits jaredly)

FooBar = require('foo/bar')

... and automatically expect everything from Index.re to be imported.

In its, current state, Index.re has to be accessed as FooBar.Index

Fix Dune warnings on pesy preprocessors config.

If you specify a preprocessor, now recent Dune will complain that the generated .merlin is not accurate (that's fine).
Solution: apply the same preprocessor config to the pesy-modules library as well.

Discussed Here

The warning makes sense in the general case, but it can be fixed by making sure that even the pesy-modules has the same ppx applied to it.

Currently if you have:

"library": {
  "imports": [
    "Console = require('console/lib')",
    "React = require('static-react')"
  ],
  "preprocess": ["pps", "static-react.reactjs_jsx_ppx"]
},

The generated Dune will be:

(library (name AppLibrary) (public_name app.library)
    (modules (:standard \ AppLibraryPesyModules))
    (libraries app.library.pesy-modules)
    (flags -open AppLibraryPesyModules)
    (preprocess (pps static-react.reactjs_jsx_ppx)))
(library (public_name app.library.pesy-modules)
    (name AppLibraryPesyModules) (modules AppLibraryPesyModules)
    (libraries console.lib static-react))
(rule
    (with-stdout-to AppLibraryPesyModules.re
        (run echo  "module Console = Console;\
                  \nmodule React = StaticReact;")))

And Dune complains because it needs everything in a directory to have the same ppx config.

The solution is easy it seems: pesy should just make sure to add the same (preprocess) spec to the pesy-modules segment as well.

kebab function adds hyphen before characters which are not lowercase

Here are the sample string and the response.

kebab("Hello World-2020!")
'hello- -world--2-0-2-0-!'

bisect_ppx support

It would be cool if pesy would support bisect_ppx to get nice coverage reports.

It could be as simple as this (from the recommended dune setup):

let preprocess =
  match Sys.getenv "BISECT_ENABLE" with
  | "yes" -> "(preprocess (pps bisect_ppx))"
  | _ -> ""
  | exception Not_found -> ""

Remove bs-cmdliner from dependencies

And move it to devDependencies

pesy/npm-cli/package.json

Line 59 in c1b2bbf

"@elliottcable/bs-cmdliner": "^10.2.1",

`esy pesy` says that it created the opam file every time.

Every time I run esy pesy it says it created the opam file. I don't think that it needs to be printed unless it actually created the opam file right?

Running esy with an out of date esy pesy, should fail to build.

If you forgot to run esy pesy after changing the package.json in a way that affects the output generated files, then running esy or esy build should fail. This is lower priority, but would be nice. The way it's done is that there would be two commands.

build: "pesy refmterr dune build -p package-name",
buildDev: "dune build -p package-name"

Where pesy knows it's running in the esy build environment instead of as an esy pesy command. We discussed this and it wasn't so easy to detect which of the modes we're in, but thought I'd make a task to track this feature.

Guide users in providing the right config

Specifying certain configs can get ambiguous at times.

"modes": "best exe",
"modes": ["best", "exe"],
"modes": ["best exe", "best object"]

The last one isn't supported yet, but seems perfectly valid. While it can always be supported,
JSON can make many such configurations possible and it would be nice to guide the user to the most meaningful syntax.

Assist workflow during build steps

Creating an umbrella issue to track all use cases/issues where pesy needs to be able to run in build env

• Handle scoped packages #31
• Avoid running outdated pesy config by validating them #75

Re-enabled e2e tests on Windows

Due to space issues, e2e tests had to be disabled on Windows. This needs to be revisited