https://slott56.github.io/py-web-tool/src/_build/html/pyweb.html#additional-features

An anonymous chunk with some RST documentation.

@o myFile.py
@{
import math, time
@}

Some notes on the packages used.

@o myFile.py
@{
print(math.pi, time.time())
@}

Some more HTML documentation.

Tangled output received:

print(math.pi, time.time())

Named chunks are concatenated from their various pieces.

whereas documentation hints at getting something like:

import math, time

print(math.pi, time.time())

This is not (currently) hosted in PyPI. Instead of installing it with PIP,
clone the GitHub repository or download the distribution kit.

why? would u need any help regarding that?

I may have missed it, but it looks like this project does not set an explicit open-source license. The default license is that the copyright remains with you, and collaboration within the GitHub workflow becomes problematic.

If there is already an open-source license associated, can you point me to it's location?

UnicodeEncodeError: 'charmap' codec can't encode character

Summary

Running pyweb -xt hw.w (where hw.w is took from pyweb#using) (or any *.w file) results in this error:

UnicodeEncodeError: 'charmap' codec can't encode character '\uXXXX' in position 37: character maps to <undefined>

Sample & Trace

###########
Hello World
###########

This file has a small example.

@d The Body Of The Script @{
print("Hello, World!")
@}

The Python module includes a small script.

@o hw.py @{
@<The Body...@>
@}

$ pyweb -xt hw.w 
INFO:Application:load and weave 3.2 WindowsPath('hw.w')
INFO:ActionSequence:Starting load and weave
INFO:LoadAction:Starting Load
INFO:WeaveAction:Starting Weave
INFO:Weaver:Weaving ...\hw.rst using rst markup
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\...\.local\bin\pyweb.exe\__main__.py", line 7, in <module>
  File "D:\....\pipx\venvs\py-web-lp\Lib\site-packages\pyweb.py", line 1663, in config
    main(base_config=base_config.get('pyweb', {}))
  File "D:\....\pipx\venvs\py-web-lp\Lib\site-packages\pyweb.py", line 1650, in main
    a.process(config)
  File "D:\....\pipx\venvs\py-web-lp\Lib\site-packages\pyweb.py", line 1590, in process
    self.theAction(config)
  File "D:\....\pipx\venvs\py-web-lp\Lib\site-packages\pyweb.py", line 1378, in __call__
    o(self.options)
  File "D:\....\pipx\venvs\py-web-lp\Lib\site-packages\pyweb.py", line 1408, in __call__
    self.options.theWeaver.emit(self.options.web)
  File "D:\....\pipx\venvs\py-web-lp\Lib\site-packages\pyweb.py", line 874, in emit
    target_file.write(text)
  File "D:\...\Python\Python311\Lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeEncodeError: 'charmap' codec can't encode character '\u220e' in position 37: character maps to <undefined>

Fix

On looking around on web, i found this SO answer 27092833/ddg#42495690
And rightly so, it worked.

--- ...\Local\Temp\pyweb.py_changed_fmkraywx99
+++ d:\....\pipx\venvs\py-web-lp\Lib\site-packages\pyweb.py
@@ -868,7 +868,7 @@
     def emit(self, web: Web) -> None:
         self.target_path = (self.output / web.web_path.name).with_suffix(f".{self.markup}")
         self.logger.info("Weaving %s using %s markup", self.target_path, self.markup)
-        with self.target_path.open('w') as target_file:
+        with self.target_path.open('w', encoding="utf-8") as target_file:
             for text in self.generate_text(web):
                 self.linesWritten += text.count("\n")
                 target_file.write(text)

--- ...\Local\Temp\impl.w_changed_1xvxx7cl7n
+++ d:\...\py-web-tool\src\impl.w
@@ -1122,7 +1122,7 @@
     def emit(self, web: Web) -> None:
         self.target_path = (self.output / web.web_path.name).with_suffix(f".{self.markup}")
         self.logger.info("Weaving %s using %s markup", self.target_path, self.markup)
-        with self.target_path.open('w') as target_file:
+        with self.target_path.open('w', encoding="utf-8") as target_file:
             for text in self.generate_text(web):
                 self.linesWritten += text.count("\n")
                 target_file.write(text)

Commit message

fix(Weaver): Use utf-8 encoding

Problem: UnicodeEncodeError on trying to weave
Solution: Specify `encoding="utf-8"` while emitting 
        the woven target file

Fix https://github.com/slott56/py-web-tool/issues/18

Output samples

###########
Hello World
###########

This file has a small example.


..  _`The Body Of The Script (1)`:
..  rubric:: The Body Of The Script (1) =
..  parsed-literal::
    :class: code

    
    print("Hello, World!")

###########
Hello World
###########

This file has a small example.


..  _`The Body Of The Script (1)`:
..  rubric:: The Body Of The Script (1) =
..  parsed-literal::
    :class: code

    
    print("Hello, World!")

..

..  container:: small

    ∎ *The Body Of The Script (1)*.
    Used by     → `hw.py (2)`_.



The Python module includes a small script.


..  _`hw.py (2)`:
..  rubric:: hw.py (2) =
..  parsed-literal::
    :class: code

    
    → `The Body Of The Script (1)`_    

..

..  container:: small

    ∎ *hw.py (2)*.
    

Environment

$ pyweb -V
py-web-lp pyweb.py 3.2

$ pipx list | grep -iB1 pyweb
   package py-web-lp 3.2, installed using Python 3.11.2
    - pyweb.exe

$ systeminfo | grep -iA1 "os name"
OS Name:                   Microsoft Windows 10 Home Single Language
OS Version:                10.0.19045 N/A Build 19045

During the weave phase, it would be powerful if you could make use of PanFlute to natively render to PanDoc's internal AST. This would leverage the power of PanDoc to make exporting to languages other than RST much simpler.

• the @f (index of @o files) ones are faulty here,
• the @m (index of @d macros) ones are correct.

Actual:

:tests/correct_data.yaml:
    → `tests/correct_data.yaml (2)`_:tests/faulty_data.yaml:
    → `tests/faulty_data.yaml (3)`_:inventory/datafile.yaml:
    → `inventory/datafile.yaml (4)`_:src/basic_test/attempt.py:
    → `src/basic_test/attempt.py (10)`_:tests/test_load_db.py:
    → `tests/test_load_db.py (19)`_

Expected:

(obtained after: s/_:/_\n\n:/ on it)

:tests/correct_data.yaml:
    → `tests/correct_data.yaml (2)`_

:tests/faulty_data.yaml:
    → `tests/faulty_data.yaml (3)`_

:inventory/datafile.yaml:
    → `inventory/datafile.yaml (4)`_

:src/basic_test/attempt.py:
    → `src/basic_test/attempt.py (10)`_

:tests/test_load_db.py:
    → `tests/test_load_db.py (19)`_

hi again!

summary & titles

pyweb only lists a sinlge "used by" entry even when there are lots & lots of it.

details

it seems that it only retains the last entry.

Alternative? Add a .nojekyll and index.html to redirect to the pyweb.html page.

this seems to be only majorly crucial thing missing from here.

i know that for tangling purpose, this ain't very useful, but otherwise it can come in handy various places in weaving and elsewhere.

it's only the state of today, that rst or the other markup toolchains may not have ability to parse as well as syntax highlight, but we shouldn't constrain our language due to that.

one syntactic sugar can be that the language is specified only in the "file" @o chunks, and all the other chunks used in that file get the same language tag. but this would cause problems wherein same chunk's used in multiple @o chunks.

anyhow.. just came here to say this for the time being. 'would love to discuss more about this in future.

mwe:

0. spaces (correct)

input (spaces)

@o src/attempt.py @{
@< att load data @>
@}

@d data schema@{
data_schema = sml.Seq(
    sml.Map({
        "ques": sml.Str(),
    })
)
@}

@d att load data @{
def load_data():
    @< data schema @>
@}

output (spaces)

def load_data():
        
    data_schema = sml.Seq(
        sml.Map({
            "ques": sml.Str(),
        })
    )

faulty ones

1. with tabs:

input (tabs):

@o test.py @{
@< att load data @>
@}

@d data schema@{
data_schema = sml.Seq(
	sml.Map({
		"ques": sml.Str(),
	})
)
@}

@d att load data @{
def load_data():
	@< data schema @>
@}

output (tabs):

notice the space before def

def load_data():
	 
 data_schema = sml.Seq(
 	sml.Map({
 		"ques": sml.Str(),
 	})
 )

Others too (with tabs):

2. -indent

@o test.py @{
@< att load data @>
@}

@d -indent data schema@{
data_schema = sml.Seq(
	sml.Map({
		"ques": sml.Str(),
	})
)
@}

@d att load data @{
def load_data():
	@< data schema @>
@}

notice the space before data_scheme & its corresponding closing parenthesis ) at same indentation

def load_data():
	 
 data_schema = sml.Seq(
 	sml.Map({
 		"ques": sml.Str(),
 	})
 )

3. -noindent

@o test.py @{
@< att load data @>
@}

@d -noindent data schema@{
data_schema = sml.Seq(
	sml.Map({
		"ques": sml.Str(),
	})
)
@}

@d att load data @{
def load_data():
	@< data schema @>
@}

$ pdm run pyweb -xw test.w 
    ...
ERROR:TextCommand:attempt to tangle a text block ('test.w', 6) 'data_schema = sml.Seq( [...]'
ERROR:TangleAction:Problems tangling outputs from WindowsPath('test.w') (tangle files are faulty).
INFO:Application:Load 0 lines from 1 files in 0.000 sec., Tangle 0 lines in 0.000 sec.

hi again! as always, thank for such a wonderful tool.

just today i stumbled upon a use case where it might make sense to be able to use a chunk i.e. @< ... @> but in a single line form, i.e. such that all the newlines in the chunk's definition are removed.

this can be given name of:

• --as-single-line: direct, obvious, understandable
• --super-trimmed: as it's just trimming the terminal spaces: leading, trailing, and newlines
• --minify: might be a bit misleading, as minify involves removing spaces within lines too, which is NOT the scope of this issue

example use case:

These chunks contain filters in human-readable (i.e. pretty printed) form

@d filter1 @{
reduce(
  .packages[]
  | select (.id | test("^Microsoft\\.VisualStudio\\.Component\\.Windows1[01]SDK\\..*") )
  | {
      ( .id 
        | split(".")[-1] 
        | select(tonumber?) 
      ): .id
    }
) as $item ({}; . + $item)
@}

@d filter2 @{ ... @}
@d filter3 @{ ... @}

----

Save filters to ``.jq`` files for standalone use with ``jq`` CLI application

@o filter1.jq @{< filter1 @>@}
@o filter2.jq @{< filter2 @>@}
@o filter3.jq @{< filter3 @>@}

-----

Use the filter in a ``.jqpg`` (jq playground) file for quick interactive use

@o my_jq_based_application.jqpg @{

jq @< --as-single-line filter1 @>
> output1.json
$ cat input.json

jq @< --as-single-line filter2 @>
> output2.json
$ cat output1.json

jq @< --as-single-line filter3 @>
> final_output.json
$ cat output2.json

@}

Versions:

$ pyweb -V
py-web-lp pyweb.py 3.2

$ systeminfo | grep -iA1 "os name"
OS Name:                   Microsoft Windows 10 Home Single Language
OS Version:                10.0.19045 N/A Build 19045

Scenario:

I want to keep things:

• separate: the web files (root source), the src files (output of tangling), the docs files (output of weaving).
• foolproof, straightforward & lean
• so, the single invokation is imagined to be: pyweb lit/impl.w -

directory structure:

lit/
|__ impl.w
src/
|__ main.py
docs/
|__ docs.rst

the lit/impl.w file looks like this:

@o src/main.py @{@}

• The -o or --output flag changes output dir of both the tangled & weaved outputs.
• The output of tangled can still be controlled by title of the @o <path> @{ ... @}
• but there's no way to restrict the scope of -o to apply only to output of weaving

I did the download and install on CentOS 7 with Python 3.7.4

I'm running the test suite:

python ../pyweb.py pyweb_test.w
INFO:Application:Weaver RST
INFO:Application:load, tangle and weave 'pyweb_test.w'
INFO:LoadAction:Starting Load
INFO:WebReader:Including 'intro.w'
INFO:WebReader:Including 'unit.w'
INFO:WebReader:Including 'func.w'
INFO:WebReader:Including 'combined.w'
ERROR:WebReader:Failure to process 'time.asctime()': result is TypeError("'types.SimpleNamespace' object is not subscriptable")
ERROR:WebReader:Failure to process 'time.ctime(os.path.getmtime(theFile))': result is TypeError("'types.SimpleNamespace' object is not subscriptable")
ERROR:WebReader:Failure to process 'os.getcwd()': result is AttributeError("'types.SimpleNamespace' object has no attribute 'getcwd'")
ERROR:LoadAction:Problems with source file 'pyweb_test.w', no output produced.
ERROR:LoadAction:Problems with source file 'pyweb_test.w', no output produced.
Traceback (most recent call last):
File "../pyweb.py", line 1913, in
main( )
File "../pyweb.py", line 1909, in main
a.process(config)
File "../pyweb.py", line 1849, in process
self.theAction()
File "../pyweb.py", line 1622, in call
o()
File "../pyweb.py", line 1721, in call
raise Error( "Syntax Errors in the Web" )
main.Error: Syntax Errors in the Web
(flask_env) [webvieww@vmcent test]$