algorithm error input/output file about pykit

[drmingdrmer]

Oh... I finally got what you mean. Sorry for that I didn't noticed infile and outfile is FileType.
I'd fix this soon.

Really appreciate for you explanation and apology for my incaution.

[4383]

@drmingdrmer no problem :)

My pleasure to help you :)

[4383]

Feel free to ask for review if you want :)

[drmingdrmer]

It does not seem to happen because it reads the entire input file before writing into it:

    with infile:
        try:
            if sort_keys:
                obj = json.load(infile)
            else:
                obj = json.load(infile,
                                object_pairs_hook=collections.OrderedDict)
        except ValueError as e:
            raise SystemExit(e)
    with outfile:
        json.dump(obj, outfile, sort_keys=sort_keys, indent=4)
        outfile.write('\n')

https://github.com/bsc-s2/pykit/blob/master/p3json/tool.py#L38

If you do have such an issue with it, do you have a snippet of code to reproduce this problem?

[4383]

There is an issue to use two FileType arguments though, they open the file when parsing command line arguments (https://github.com/python/cpython/blob/master/Lib/argparse.py#L1198).

This means that when infile and outfile refer to the same file, it will get open for writing before you read from it and its content will be lost before the program even begin:

➜  ~ echo foo >test.txt
➜  ~ python3
Python 3.7.2 (default, Jan 13 2019, 12:50:01)
[Clang 10.0.0 (clang-1000.11.45.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> open('test.txt')
<_io.TextIOWrapper name='test.txt' mode='r' encoding='UTF-8'>
>>> open('test.txt', 'w')
<_io.TextIOWrapper name='test.txt' mode='w' encoding='UTF-8'>
>>>
➜  ~ cat test.txt
➜  ~

[4383]

It's not your fault is the default argparse behavior when you use FileType on the same file.

Python itself is impacted by the same issue => python/cpython#7865

[drmingdrmer]

Your codes do not actually do what you described.

so the file will be replaced before you read it.

This wouldn't happen with my code. You just simply re-opened a file. My code does this:

• open(fn, 'r'); read it; close it.
• open(fn, 'w'); write it; close it.

[4383]

Since you use FileType for infile and outfile and since your users can pass the same file as infile and outfile you potentially facing this error due to argparse that return to you a file descriptor opened on the same file in the same time and that the second file descriptor is open on writing mode.

[4383]

https://bugs.python.org/issue33927

[4383]

Feel free to close this issue if you want I'm not impacted by this it's more to advertise you.

[drmingdrmer]

I've fixed this problem in my latest PR. I'd be glad if you got time having a look at my patch.☝️ ☝️

Thanks again :DDD