Coder Social home page Coder Social logo

gbroques / ose-workbench-platform Goto Github PK

View Code? Open in Web Editor NEW
6.0 3.0 2.0 757 KB

Common platform for developing Open Source Ecology (OSE) workbenches.

Home Page: https://ose-workbench-platform.readthedocs.io/en/latest/

License: GNU Lesser General Public License v2.1

Python 95.87% Shell 3.33% HTML 0.80%
freecad freecad-workbench opensourceecology ose osedev-workbench osedev workbench

ose-workbench-platform's Introduction

OSE Workbench Platform

PyPI version Conda version Build Status Documentation Status Coverage Status

Introduction

A platform for developing workbenches for Open Source Ecology (OSE).

OSE defines a "workbench" as a set of tools in CAD software to design and make a particular machine.

Each workbench OSE develops for one of it's machines has certain common development-time or "dev-time" needs and dependencies.

For example, running unit tests, making documentation, and generating code to streamline workbench development.

Rather than duplicate the approaches to each of these needs, ose-workbench-platform abstracts those needs into a common platform so they aren't the concern of individual OSE workbench maintainers.

Each workbench maintainer doesn't need to know or care about the particular versions and libraries we use to solve those needs, nor the particular configuration.

Having a common platform for OSE workbench development also makes it easier for developers to readily switch between workbenches by providing a common tool-set.

ose-workbench-platform provides a command-line interface (CLI), via the osewb command, containing commands for common dev-time tasks such as running all tests, making documentation, initializing new workbenches, and even generating code for common tasks.

Pre-Requisites

  1. Install Git
  2. Install Miniconda

Installation

Install the ose-workbench-platform package from the gbroques and conda-forge channel in a dedicated conda environment named osewb (short for Open Source Ecology WorkBench) and don't ask for confirmation:

conda create --name osewb --channel gbroques --channel conda-forge --yes ose-workbench-platform

Activate your new osewb environment:

conda activate osewb

Test your installation:

osewb --help

You can deactivate this environment later by running:

conda deactivate

Virtual Development Environment

We use Conda to create a reproducible virtualized OSE workbench development environment with requisite dependencies for development-time tasks like running FreeCAD, executing unit tests, and generating documentation from source-code comments.

In order to perform various development-time tasks for a workbench, you must first:

  1. Create a conda environment from the environment.yml file located in the root of the workbench repository
  2. Activate the environment with conda activate <environment name>

Note, each workbench will have it's own separate environment.

Workbench environments will be named after the base package in the workbench repository (e.g. ose3dprinter, osetractor, osepowercube, etc.).

Some common commands relating to managing environments with conda are documented in the below table.

Description Command
Creating the environment conda env create --file environment.yml
Activating the environment conda activate <environment name>
Deactivating the environment conda deactivate

Refer to the Conda CLI reference documentation for additional information.

Unit Tests

For running unit tests we use pytest.

For test coverage, we use coverage.py and pytest-cov.

Documentation

For building documentation, we use Sphinx.

For hosting documentation, we use a free service for open-source projects called Read the Docs.

For a modern and mobile-friendly look, we use Read the Docs Sphinx Theme.

Commands

The osewb command contains various sub-commands for performing common dev-time tasks of a OSE workbench.

$ osewb -h ↵
usage: osewb <command> [<args>]

A collection commands for OSE workbench development.

optional arguments:
  -h, --help            show this help message and exit
  --version             show program's version number and exit

Commands:
  {test,lint,docs,make,browse,br}
    test                Run tests in workbench
    lint                Lint code
    docs                Make documentation
    make                Commands for making new code
    browse (br)         Commands for opening documents in a web browser

Each sub-command may have flags and arguments, and additional information can be discovered via osewb <command> -h or --help.

Is osewb too many characters to type? We recommend aliasing the osewb command as ose to reduce typing and increase speed even further.

For further convenience, any command over four characters shall include a short-alias under four characters or less. For example, br is the short-alias for the five-character browse command.

test

OSE Workbench Platform includes a test command for interacting with the test-suite of a workbench.

$ osewb test -h ↵
usage: osewb test

optional arguments:
  -h, --help      show this help message and exit
  -c, --coverage  Run tests with coverage, and generate report

To run the entire unit-test suite for a workbench, run:

osewb test

For running tests with coverage and generating a coverage report, pass the -c or --coverage flag to the test command:

osewb test --coverage

lint

OSE Workbench Platform includes a lint command for linting the code of a workbench.

osewb lint

The lint command will:

  • Run flake8 with configuration located in .flake8.
  • Run mypy for static type checking with configuration located in .mypy.ini.

For automatically fixing some linter issues, pass the -f or --fix flag to the lint command:

osewb lint --fix

This will run isort and autopep8 recursively on the root of the workbench repository.

For additional information, see:

docs

OSE Workbench Platform includes a docs command for building the documentation of a workbench.

$ osewb docs -h ↵
usage: osewb docs [command]

optional arguments:
  -h, --help       show this help message and exit

Commands:
  {screenshot,ss}
    screenshot (ss)
                   Take screenshots of parts for documentation.

The docs command will:

  • Remove auto-generated files such as the _build/ and any auto-generated Sphinx sources.
  • Re-generate docs/_build/, docs/<base package>/, docs/freecad/<base package>/ by running sphinx-build . _build within docs/ using the Sphinx configuration specified in docs/conf.py

For additional information, see sphinx-build and Sphinx Configuration.


Additionally, you may pass a screenshot or ss sub-command to the docs command for taking screenshots of parts in a workbench:

osewb docs ss

This will look for parts in the <base package>/part package of the current workbench, and save thumbnail screen shots in the docs/_static/screenshot/ directory for each part.

build

OSE Workbench Platform includes a build command for building a workbench.

osewb build

The build command aggregates the following commands into one:

  1. Run tests with coverage - osewb test --coverage
  2. Lint all code - osewb lint
  3. Ensure the documentation builds - osewb docs

The build command exits with 0 or 1 to pass or fail the build depending upon whether the above commands return non-zero exit codes.

make

OSE Workbench Platform includes a make command for "making" new code.

$ ose make -h ↵
usage: osewb make <command>

optional arguments:
  -h, --help            show this help message and exit

Commands:
  {workbench,wb,part,model,part-feature,pf,command}
    workbench (wb)      Make Workbench
    part                Make Part class
    model               Make Model class
    part-feature (pf)   Make Part Feature creation function
    command             Make Command class

workbench

Navigate to where you want to create your new workbench. Then run:

osewb make workbench <machine_display_name>

Where <machine_display_name> is the name of the machine in Title Case. If this contains spaces, then surround the value in double-quotes "".

$ osewb make workbench Tractor ↵
Workbench initialized in "ose-tractor-workbench" directory.

Perform the following commands to get started:

1. Change directories and initialize the git repository:

    cd ose-tractor-workbench && git init

2. Create a conda environment and activate it:

    conda env create --file environment.yml && conda activate osetractor

3. Verify your installation:

    osewb -h

The above examples initializes a new workbench, in a ose-tractor-workbench directory, with the basic structure and files needed.

$ tree ose-tractor-workbench --dirsfirst ↵
ose-tractor-workbench
├── docs
│   ├── conf.py
│   └── index.rst
├── freecad
│   └── osetractor
│       ├── command
│       │   ├── _add_box
│       │   │   ├── add_box_command.py
│       │   │   └── __init__.py
│       │   └── __init__.py
│       ├── icon
│       │   ├── Box.svg
│       │   └── __init__.py
│       ├── init_gui.py
│       ├── __init__.py
│       └── OSE_Tractor.py
├── osetractor
│   ├── part
│   │   ├── _box
│   │   │   ├── box.py
│   │   │   └── __init__.py
│   │   └── __init__.py
│   └── __init__.py
├── tests
│   ├── box_test.py
│   └── __init__.py
├── CONTRIBUTING.md
├── environment.yml
├── LICENSE
├── MANIFEST.in
├── README.md
└── setup.py

10 directories, 22 files

For more information, see the Pattern Catalog in the docs.

OSE Tractor Workbench

part

Within the repository of a workbench, run the osewb make part command to make a new Part Class.

For example,

osewb make part Box

Makes a new Box part class.

For more information, see Part Classes in the docs.

model

Within the repository of a workbench, run the osewb make model command to make a new Model Class.

For example,

osewb make model Box

Makes a new BoxModel model class.

For more information, see Model Classes in the docs.

part-feature (pf)

Within the repository of a workbench, run the osewb make part-feature command to make a new Part Feature creation function.

For example,

osewb make part-feature my_box

Makes a new create_my_box part feature creation function using the MyBoxModel class.

For more information, see Part Feature Sub-package in the docs.

command

Within the repository of a workbench, run the osewb make command command to make a new Command Class.

For example,

osewb make command AddBox

Makes a new AddBoxCommand class.

For more information, see Command Classes in the docs.

browse

OSE Workbench Platform includes a browse covenience command for opening documentation and coverage reports in a web browser.

$ osewb browse -h ↵
usage: osewb browse <command>

optional arguments:
  -h, --help           show this help message and exit

Commands:
  {docs,coverage,cov}
    docs               Opens docs in web browser
    coverage (cov)     Opens coverage report in web browser

The docs command opens docs/_build/index.html in a web browser, while coverage opens htmlcov/index.html in a web browser.

editor-config

OSE Workbench Platform includes an editor-config command for outputting recommended VS Code configuration.

$ osewb editor-config -h ↵
usage: osewb editor-config

optional arguments:
  -h, --help            show this help message and exit
  -m, --merge-workspace-settings
                        Merge VS Code workspace settings.
  -o, --overwrite-workspace-settings
                        Overwrite VS Code workspace settings.

Simply running osewb editor-config will output the recommended VS Code configuration settings which user's can copy-paste into their VS Code user settings, or workspace settings, settings.json file(s). See User and Workspace Settings for additional information.

The -m or --merge-workspace-settings flag will merge the current VS workspace settings into the platform's recommended settings. The platform's settings will win any collisions or merge conflicts.

The -o or --overwrite-workspace-settings flag will overwrite the current VS Code workspace settings with either the minimal-set of recommended configuration or merged settings depending upon the presence of the -m flag. Before overwriting, users will see a preview of the settings and must confirm overwriting in a yes or no CLI prompt.

Contributing

See Contributing Guidelines.

License

Licensed under the GNU Lesser General Public License, version 2.1 or LGPL v2.1. See LICENSE for details.

This is the same license as FreeCAD to ensure this code could potentially be incorporated into future FreeCAD modules or FreeCAD source itself.

ose-workbench-platform's People

Contributors

gbroques avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

ose-workbench-platform's Issues

Suggest most similar command when incorrect command is typed

For example,

$ osewb lintt
osewb: error: argument command: invalid choice: 'lintt' (choose from 'test', 'lint', 'docs', 'make', 'browse', 'br', 'build', 'bld')

We could say: did you mean osewb lint?.

For example, git does this:

$ git checkoutt
git: 'checkoutt' is not a git command. See 'git --help'.

The most similar command is
        checkout

Investigate WebGL Rendering

https://forum.freecadweb.org/viewtopic.php?f=9&t=3914
https://sliptonic.com/freecad-webgl-export/
https://gist.github.com/yorikvanhavre/2286282

>>> ### Begin command Std_Export
>>> __objs__=[]
>>> __objs__.append(FreeCAD.getDocument("Unnamed").getObject("Frame"))
>>> import importWebGL
>>> importWebGL.export(__objs__,u"/home/g/Desktop/test.html")
>>> 
>>> del __objs__
>>> ### End command Std_Export

Related to #15.

I'm imagining when you click the picture, then you get a three-3d WebGL rendering in a modal or separate page.

Consider Adding Logger

We might consider adding a logger for more robust logging than print statements.

Then, we could have different log levels and verbosity.

React to Shell Command Exit Codes

Right now we're blindly printing stuff after executing commands, but this could be confusing if there was an error with one of the shell commands.

If there's an error, then we shouldn't print a bunch of other "next step" messages after the shell command output.

                coverage_report_cmd = 'docker exec -it {} coverage html'
                execute_command_in_docker_container(
                    coverage_report_cmd, base_package)
                print('Coverage report generated in htmlcov/ directory.')

Document Why and How We Use Docker

This will be probably be a point of confusion.

We need to document WHY and HOW we use Docker better, and re-organize the slim documentation that exists the README today.

Investigate creating OSE FreeCAD AppImage

From @looooo

... Openglider already uses too many libraries which are not bundled with freecad, so we cannot rely on the addon-manager. We already see the difficulties with the bundles (appimage, dmg, win). The size grows up to 800mb (appimage) because we now ship scipy, sympy, pandas,... within this bundle. In my eyes this is not the way to proceed.

So as you are talking about a lot of libraries which will be created for ose, I guess you should have a look at conda-packaging (or if no difficult [dependencies] and no compiled stuff is involved pypi is also a possibility). But with conda you shouldn't see any theoretic limitations (for sure packaging is always frustrating as dependencies are updated and incompatibilities are introduced). Once everything is available with conda it's easy to create bundles via the scripts we offer [3] (in case you want to offer a freecad-ose bundle or something like this).

Add Build Command

The build command would be an aggregate or compound command of the following:

  1. Run tests with coverage via osewb test --coverage
  2. Perform linting via osewb lint
  3. Build the docs via osewb docs

The motivation is to provide more centralized control over what constitutes a "build" that passes or fails, and simplify .travis.yml files in each workbench.

That way the osewb build aggregates multiple commands and provides a simple pass-or-fail exit code of 0 of 1.

Fix ose make when __init__ module doesn't exist

Example of error.

(oseexample) ➜  ose-example-workbench git:(master) ose make model HexNut
Traceback (most recent call last):
  File "/home/g/miniconda3/envs/oseexample/bin/osewb", line 10, in <module>
    sys.exit(main())
  File "/home/g/miniconda3/envs/oseexample/lib/python3.8/site-packages/osewb/osewb.py", line 44, in main
    handle_make_component_command(base_package,
  File "/home/g/miniconda3/envs/oseexample/lib/python3.8/site-packages/osewb/handle_make_component_command.py", line 41, in handle_make_component_command
    with component_package_init_module_path.open('r+') as f:
  File "/home/g/miniconda3/envs/oseexample/lib/python3.8/pathlib.py", line 1218, in open
    return io.open(self, mode, buffering, encoding, errors, newline,
  File "/home/g/miniconda3/envs/oseexample/lib/python3.8/pathlib.py", line 1074, in _opener
    return self._accessor.open(self, flags, mode)
FileNotFoundError: [Errno 2] No such file or directory: '/home/g/.FreeCAD/Mod/ose-example-workbench/oseexample/model/__init__.py'

Figure out if 'new workbenches' are supported by the Add Ons Manager

I have a feeling they aren't if you include external dependencies from PyPi.

If not, then we're going to need an easy way for users to download or install OSE workbenches.

Maybe a separate AppImage that people can download containing all OSE workbenches and dependencies.

Instruct User to Run git init After osewb init Command

➜  Mod osewb init
machine_display_name [CEB Brick Press]: Tractor
repo_name [ose-tractor-workbench]: 
machine_title [OSE Tractor]: 
workbench_title [OSE Tractor Workbench]: 
base_package [osetractor]: 
command_registry_filename [OSE-Tractor]: 
command_namespace [OSETractor]: 
workbench_class_filename [tractor_workbench]: 
workbench_class_name [TractorWorkbench]: 
owner_name [G Roques]: 

Workbench initialized in ose-tractor-workbench.
To initialize the git repository, run:

    cd ose-tractor-workbench && git init

Investigate Python No Arch

https://www.anaconda.com/blog/condas-new-noarch-packages

https://forum.freecadweb.org/viewtopic.php?f=8&t=45582&start=20#p411514

From @looooo

I had a brief look at the ose-packages that you uploaded. As the packages are python only (?) I think you should be able to add the "noarch python" instruction to make the build-matrix smaller. Example for a noarch python package can be found here: https://github.com/looooo/freecad.asm3- ... a.yaml#L17 If this is not the case ignore my comment.

Document Installing via Virtual Environment as Best Practice

In the case of working in-between multiple workbenches, you may need to install multiple different versions of ose-workbench-platform.

This is why it's recommended to install ose-workbench-platform from within a virtual environment.

TODO: Need to test and document this process.

Consider Externalizing Cookiecutter Template

It's worth considering externalizing the cookiecutter workbench template as it may prove useful to others in the FreeCAD community.

However, OSE will want an opinionated template, while the community will likely want a more flexible template.

Can we have a flexible base template, that we add an opinionated layer on top of?

Working External Repo Name:
cookiecutter-freecad-workbench

Add VISION

A VISION document can serve as a helpful guide to future maintainers or as a reminder to ensure the project remains true to it's original goal or vision.

Automatically take screenshots of parts

If we could screenshot parts from a pure command-line or server context, then we could automate this process when running osewb docs.

Instead of users having to manually run osewb docs screenshot.

You can apparently do this, but it's a lot more complicated.

Related Links

From wmayer circa 2016:

The pure command line version is:

import FreeCAD, Part
from pivy import coin

box=Part.makeCone(10,8,10)
iv=box.writeInventor()

# SoInput.setBuffer seems broken with pivy, so save to file and load from there
output=open("part.iv","w")
output.write(iv)
output.close()

inp=coin.SoInput()
inp.openFile(output.name)
data = coin.SoDB.readAll(inp)

base = coin.SoBaseColor()
base.rgb.setValue(0.6,0.7,1.0)
data.insertChild(base,0)

root = coin.SoSeparator()
light = coin.SoDirectionalLight()
cam = coin.SoOrthographicCamera()
root.addChild(cam)
root.addChild(light)
root.addChild(data)

axo = coin.SbRotation(-0.353553, -0.146447, -0.353553, -0.853553)
viewport=coin.SbViewportRegion(400,400)
cam.orientation.setValue(axo)
cam.viewAll(root,viewport)
off=coin.SoOffscreenRenderer(viewport)
root.ref()
ret=off.render(root)
root.unref()

if ret > 0:
    off.writeToPostScript("crystal.ps")
    if off.isWriteSupported("PNG"):
        off.writeToFile("crystal.png","PNG")

I was received an error when trying to get his code-snippet to work in FreeCAD 19.

Fix Lint Error with Generated Workbench

Executing the following command:

    flake8 --config /home/g/Projects/ose-workbench-platform/osewb/.flake8 /home/g/.FreeCAD/Mod/ose-screw-workbench

/home/g/.FreeCAD/Mod/ose-screw-workbench/tests/box_test.py:4:1: F401 'FreeCAD as App' imported but unused
/home/g/.FreeCAD/Mod/ose-screw-workbench/tests/box_test.py:5:1: I003 isort expected 1 blank line in imports, found 0
/home/g/.FreeCAD/Mod/ose-screw-workbench/freecad/osescrew/command/_add_box/add_box_command.py:3:1: I003 isort expected 1 blank line in imports, found 0
Executing the following command:

    mypy --config-file /home/g/Projects/ose-workbench-platform/osewb/.mypy.ini /home/g/.FreeCAD/Mod/ose-screw-workbench

Success: no issues found in 7 source files

Lint errors detected from the following command(s):
    flake8

Run Container as Non Root User

Sometimes, when we run builds in Docker containers, the build creates files in a folder that’s mounted into the container from the host (e.g. the source code directory). This can cause us pain, because those files will be owned by the root user. When an ordinary user tries to clean those files up when preparing for the next build (for example by using git clean), they get an error and our build fails.

https://medium.com/redbubble/running-a-docker-container-as-a-non-root-user-7d2e00f8ee15

Instruct User To Start Existing Container If Already Created

➜  ose-3d-printer-workbench git:(master) ✗ osewb test                
Multiple potential base packages starting with "ose" found:

    ose3dprinter, osecore

Choosing first "ose3dprinter" as base package.

No ose3dprinter container running.
To create a ose3dprinter container, run:

   osewb container create

➜  ose-3d-printer-workbench git:(master) ✗ osewb container create
Multiple potential base packages starting with "ose" found:

    ose3dprinter, osecore

Choosing first "ose3dprinter" as base package.

Container ose3dprinter already created.

Add osewb make command <name>

Add command to make Command classes.

Similar to osewb make part <name> or osewb make model <name>.

osewb make command <name>

Remove Dependency on Docker Compose

I think we can simplify things greatly by only making developers install Docker as opposed to Docker and Docker Compose.

We can accomplish this by only having one container, thus removing the need for Docker Compose to manage multiple containers.

Then, we can offer a osewb container command to offer convenience, build, start, stop, rm, etc. commands for interacting with it.


This also reduces the number of references to ose-workbench-platform versions in each workbench from 3 files to 2 files.

  1. .travis.yml
  2. docs/requirements.txt
  3. docker-compose.yml

Consider Opening Web Browser with Test and Docs Command

It might be nice to launch the user's web browser the first time running the osewb docs or osewb test command.

That, or maybe we add a flag like -b and --open-web-browser to each command.

import webbrowser

webbrowser.open('file:///home/g/.FreeCAD/Mod/ose-tractor-workbench/docs/_build/index.html')

Another idea is to have a browse command.

$ osewb browse coverage

Opens /ose-3d-printer-workbench/htmlcov/index.html in web-browser.

$ osewb browse docs

Opens /ose-tractor-workbench/docs/_build/index.html in web browser.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.