Coder Social home page Coder Social logo

chef / concrete Goto Github PK

View Code? Open in Web Editor NEW
56.0 78.0 15.0 221 KB

Concrete enhances your rebar based Erlang project by providing a common Makefile wrapper, a dialyzer make target that caches PLT analysis of your project's dependencies, and a mechanism to specify development only dependencies.

License: Apache License 2.0

Makefile 43.28% Erlang 56.72%

concrete's Introduction

concrete: enhance your rebar build experience

Concrete enhances your rebar based Erlang project by providing a common Makefile wrapper, a dialyzer make target that caches PLT analysis of your project's dependencies, and a mechanism to specify development only dependencies.

Features

Standard make targets

  • all
  • allclean
  • clean
  • compile
  • dialyzer
  • distclean
  • doc
  • eunit
  • tags
  • test

Dialyzer config

The makefile rules included in concrete will check for a ~/.concrete_dialyzer_plt_<apps hash>_<erlang version>.plt file and will create a reusable PLT for the OTP modules.

Here are some examples:

~/.concrete_dialyzer_plt_4667e0f8e4ec738d28efbc1212495b1d_17.plt
~/.concrete_dialyzer_plt_4667e0f8e4ec738d28efbc1212495b1d_R16B02.plt
~/.concrete_dialyzer_plt_4667e0f8e4ec738d28efbc1212495b1d_R16B03.plt
~/.concrete_dialyzer_plt_ce4f2cc7cbd1bdd337c6ba5d475c1290_R16B03.plt

By default, concrete will build a local PLT file containing analysis of the dependencies of your project found in the deps directory. When you run the dialyzer target via make dialyzer, the global ~/.concrete_dialyzer_plt_<apps hash>_<erlang version>.plt is combined with the project-specific deps.plt to analze your code. Including dependencies in the analysis is important to get the most out of dialyzer and precomputing a PLT for your deps saves time.

You may encounter some dependenices which do not play well with dialyzer. You can tell concrete to omit these problem dependencies by adding them to a DIALYZER_SKIP_DEPS make variable in your Makefile.

In general, concrete should be able to detect when deps.plt needs to be rebuilt. If you are encountering confusing dialyzer warnings and have recently updated your dependencies, you can remove deps.plt and rebuild.

Travis CI

Concrete will attempt to pull in cached PLTs from S3 when running on Travis CI. They're built for Travis, so they won't be downloaded locally. We got them here: ESL/erlang-plts. Thanks ESL!

Dev only dependencies

You can specify dependencies that are only needed during development using the dev_only_deps key in your rebar.config file. Example:

{dev_only_deps,
 [
  {proper, ".*", {git, "git://github.com/manopapad/proper.git", "master"}}
 ]}.

When you run make, concrete will create .concrete/DEV_MODE and use this file as an indicator to include dev_only_deps in the build. These dependencies will not be incurred by other projects that add your project as a dependency.

When in dev mode, concrete will define a macro DEV_ONLY which can be used to conditionally include test code that makes use of a dev only dependency.

Another way to think of DEV_MODE is "top level project mode". When your project is included as someone else's dependency, rebar will not pull in the dev_only_deps into their project.

Generate markdown docs via edown

By default, concrete will include [edown][] as a dev only dependency and use it to generate markdown from the edoc in your code when you run make doc. You can disable this behavior by adding the following to your rebar.config file:

 {use_edown, false}.

Custom Makefile Targets

Concrete supports a custom.mk file which will not be overwritten when concrete is upgraded. You can put any custom makefile targets or environment variables in this file. Any modifications you make to Makefile and concrete.mk will be overwritten when you run concrete update, so put anything you don't want to lose in custom.mk

Using HOOKs

The all, clean, and rel targets can be extended by setting additional dependencies via their respective HOOK variables: ALL_HOOK, CLEAN_HOOK, and REL_HOOK. For example, to extend the all target in your custom.mk, you can do the following:

ALL_HOOK = your_custom_target

your_custom_target:
   ./something_awesome

Configuring Travis.yml

you should set up a minimal travis.yml for concrete projects. If you leave out the install: line, rebar get-deps will NOT include your dev_only_deps. Even if you don't have any, your build will fail because it's still looking for edown and rebar_lock_deps_plugin.

language: erlang
install: true
script: make get-rebar all <any other target you want>

Installation

  1. Clone the concrete repo
  2. Build the project
  3. Add the concrete escript to your PATH. NOTE: the concrete escript cannot be relocated because it locates the template files in priv/templates based on the location of the executable.
git clone git://github.com/opscode/concrete.git
cd concrete
make
# now add `pwd` to your PATH

Is this thing on? Let's find out!

 concrete init infodata
 cd infodata
 make

Concrete Examples

Initialize a new project with concrete init

  1. Make sure the concrete escript is on your PATH.
  2. Run concrete init NAME, where NAME is your desired project name. A directory named NAME will be created in your current working directory with project skeleton. You will be asked if you want an active application. If you answer "yes", then the generated project will include a supervisor and the application will be startable. In this case, concrete will also generate a relx.config and you can build an OTP release via make rel.

The whole sequence should look like this:

$ concrete init apples
Creating the apples project with concrete

Would you like an active application? (y/n): y
Now try: cd apples; make

$ ls -a apples
   .concrete/  .gitignore  Makefile  README.md  concrete.mk include/  priv/  rebar.config  rebar.config.script  src/  test/

important Add the files that concrete created for you to git. Be sure that you git add the following:

git add concrete.mk
git add rebar.config
git add rebar.config.script
git add Makefile

Updating an existing project

When you run concrete update, concrete will create backup copies of your Makefile, rebar.config.script, and concrete.mk. Then it will copy the latest version of those files over.

The purpose of the update command is to make it easier to receive fixes and features when they are added to the make rules or rebar config script that are part of concrete's project templates.

You can also use the update command to help convert an existing project to use concrete.

How It Works

A project that uses concrete, will have a minimal Makefile that includes a set of standard rules in concrete.mk. The project will also contain both a rebar.config file and a rebar.config.script file. The .script config is evaluated by rebar after the standard config is read. The script looks for .concrete/DEV_MODE and decides whether or not to add dev_only_deps.

Initially, I was planning on making concrete a dependency that you would include in a project with the possibility of picking up the latest build rules by refetching the concrete dep. Keeping working builds working seems more important and in the common case it should still be easy to pickup updates when desired.

Contributing

For information on contributing to this project see https://github.com/chef/chef/blob/master/CONTRIBUTING.md

License

  • Copyright:: 2013-2016 Chef Software, Inc.
  • License:: Apache License, Version 2.0
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

concrete's People

Contributors

drobakowski avatar jkakar avatar joedevivo avatar jpuigm avatar jwilberding avatar marcparadise avatar stevendanna avatar tas50 avatar wk8 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

concrete's Issues

Building concrete for the first time errors out (but seems to work despite that)

A fresh checkout of Concrete errors out at the end. Even then,
running the concrete escript seems to work and I'm happily hacking
on my new project. I'm using it with Erlang/OTP 17 [erts-6.0] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] [dtrace] on Darwin rex 13.1.0 Darwin Kernel Version 13.1.0: Wed Apr 2 23:52:02 PDT 2014; root:xnu-2422.92.1~2/RELEASE_X86_64 x86_64.

jkakar@rex:~/src/github.com/opscode/concrete$ make
WARN:  Expected /Users/jkakar/src/github.com/opscode/concrete/deps/edown to be an app dir (containing ebin/*.app), but no .app found.
WARN:  Expected /Users/jkakar/src/github.com/opscode/concrete/deps/rebar_lock_deps_plugin to be an app dir (containing ebin/*.app), but no .app found.
WARN:  Missing plugins: [rebar_lock_deps_plugin]
==> concrete (get-deps)
WARN:  Expected /Users/jkakar/src/github.com/opscode/concrete/deps/edown to be an app dir (containing ebin/*.app), but no .app found.
WARN:  Expected /Users/jkakar/src/github.com/opscode/concrete/deps/rebar_lock_deps_plugin to be an app dir (containing ebin/*.app), but no .app found.
Pulling edown from {git,"git://github.com/seth/edown.git",{branch,"master"}}
Cloning into 'edown'...
Pulling rebar_lock_deps_plugin from {git,
                                     "git://github.com/seth/rebar_lock_deps_plugin.git",
                                     {branch,"master"}}
Cloning into 'rebar_lock_deps_plugin'...
WARN:  Missing plugins: [rebar_lock_deps_plugin]
==> edown (get-deps)
WARN:  Missing plugins: [rebar_lock_deps_plugin]
==> rebar_lock_deps_plugin (get-deps)
WARN:  Missing plugins: [rebar_lock_deps_plugin]
WARN:  Missing plugins: [rebar_lock_deps_plugin]
==> edown (compile)
Compiled src/edown_lib.erl
Compiled src/edown_doclet.erl
Compiled src/edown_make.erl
Compiled src/edown_layout.erl
Compiled src/edown_xmerl.erl
WARN:  Missing plugins: [rebar_lock_deps_plugin]
==> rebar_lock_deps_plugin (compile)
Compiled src/rldp_util.erl
Compiled src/rldp_change_log.erl
Compiled src/rebar_lock_deps_plugin.erl
==> concrete (compile)
Compiled src/concrete.erl
==> concrete (eunit)
Compiled src/concrete.erl
  There were no tests to run.
rebar escriptize
==> edown (escriptize)
==> rebar_lock_deps_plugin (escriptize)
==> concrete (escriptize)
Missing ~/.dialyzer_plt. Please wait while a new PLT is compiled.
dialyzer --build_plt --apps asn1 compiler crypto edoc erts eunit inets kernel mnesia public_key ssl stdlib syntax_tools tools xmerl
  Compiling some key modules to native code... done in 0m41.90s
  Creating PLT /Users/jkakar/.dialyzer_plt ...
eunit_test.erl:305: Call to missing or unexported function eunit_test:nonexisting_function/0
Unknown functions:
  dbg:ctp/1
  dbg:p/2
  dbg:stop/0
  dbg:tp/2
  dbg:tpl/2
  dbg:trace_client/3
  dbg:trace_port/2
  dbg:tracer/0
  dbg:tracer/2
  hipe:compile/4
  webtool:start/0
  webtool:start_tools/2
  webtool:stop/0
  webtool:stop_tools/2
 done in 3m44.05s
done (warnings were emitted)
make: *** [/Users/jkakar/.dialyzer_plt] Error 2

custom.mk is first so first target becomes default

So I just tried adding a custom target to custom.mk after upgrading concrete in a project. The result surprised me because the target I added (for running some slow tests via slow_test) would run when I typed make. Pretty sure that makes sense: we include custom.mk first, and the first target becomes the default target.

You can get around this by adding .DEFAULT_GOAL := in custom.mk to reset the default. We could also set that in concrete.mk to force the concrete default (that sort of seems right here?).

Or we could include custom.mk afterwards, but I suspect there was a reason we put it first.

ping @joedevivo

Edge case with deps.plt

sometimes deps.plt doesn't get generated. I thought for sure that we'd resolved this when we added the recursive $(MAKE) call to the all target.

Even if it didn't get generated because make dialyzer was run before get-deps, running make all should get deps and then spawn the recursive make that build deps.plt.

@metadave had to delete his deps dir and make clean to get deps.plt to generate.

this could have been a bad state left by an older version of concrete, but the only state it's really ever had was .concrete/DEV_MODE so I'm perplexed by this

If 'rebar' isn't in $PATH, then 'concrete init' fails silently

$ ../../opscode/concrete/concrete init
Initialize a new project with concrete

Project name: meh
Short Description:
Meh
Creating meh via 'rebar create template_dir=../../opscode/concrete/priv/templates template=concrete_project name=meh description="Meh"'
Now try: make
$ make
make: *** No targets specified and no makefile found. Stop.

Make it possible to alter behavior of the standard make targets

Ideally, users would be able to customize the behavior of the targets in an elisp advice sort of fashion: you should be able to specify targets that run just before a standard target, just after, or be able to replace/overwrite a standard target entirely.

It might be worth looking at the other Make-based erlang project from the cowboy folks as they may have solved this in a nice Makey way. But it also might be as simple as:

test_BEFORE
test_AFTER

Add support for relx 1.0

Make concrete compatible with relx 1.0. If possible, maintain backwards compatibility, but otherwise 1.0 is the first officially released version, so it would be ok to make that the first version concrete officially supports and not worry about previous beta releases of relx.

The Fresh Repo Conundrum

You check out a fresh repo from github. it's empty, you're psyched. You want to concrete init but that command is really against creating anything if it didn't also create the directory.

โžœ  new_rep git:(master) ../concrete/concrete init .
ERROR: concrete init wants to create '.', but it already exists

I'll get around to this eventually, but I'll forget if I don't put it here.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.