Comments (11)
Oh... I finally got what you mean. Sorry for that I didn't noticed infile and outfile is FileType
.
I'd fix this soon.
Really appreciate for you explanation and apology for my incaution.
from pykit.
@drmingdrmer no problem :)
My pleasure to help you :)
from pykit.
Feel free to ask for review if you want :)
from pykit.
It does not seem to happen because it reads the entire input file before writing into it:
with infile:
try:
if sort_keys:
obj = json.load(infile)
else:
obj = json.load(infile,
object_pairs_hook=collections.OrderedDict)
except ValueError as e:
raise SystemExit(e)
with outfile:
json.dump(obj, outfile, sort_keys=sort_keys, indent=4)
outfile.write('\n')
https://github.com/bsc-s2/pykit/blob/master/p3json/tool.py#L38
If you do have such an issue with it, do you have a snippet of code to reproduce this problem?
from pykit.
There is an issue to use two FileType
arguments though, they open the file when parsing command line arguments (https://github.com/python/cpython/blob/master/Lib/argparse.py#L1198).
This means that when infile
and outfile
refer to the same file, it will get open for writing before you read from it and its content will be lost before the program even begin:
➜ ~ echo foo >test.txt
➜ ~ python3
Python 3.7.2 (default, Jan 13 2019, 12:50:01)
[Clang 10.0.0 (clang-1000.11.45.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> open('test.txt')
<_io.TextIOWrapper name='test.txt' mode='r' encoding='UTF-8'>
>>> open('test.txt', 'w')
<_io.TextIOWrapper name='test.txt' mode='w' encoding='UTF-8'>
>>>
➜ ~ cat test.txt
➜ ~
from pykit.
It's not your fault is the default argparse behavior when you use FileType
on the same file.
Python itself is impacted by the same issue => python/cpython#7865
from pykit.
Your codes do not actually do what you described.
so the file will be replaced before you read it.
This wouldn't happen with my code. You just simply re-opened a file. My code does this:
- open(fn, 'r'); read it; close it.
- open(fn, 'w'); write it; close it.
from pykit.
Since you use FileType
for infile
and outfile
and since your users can pass the same file as infile
and outfile
you potentially facing this error due to argparse that return to you a file descriptor opened on the same file in the same time and that the second file descriptor is open on writing
mode.
from pykit.
https://bugs.python.org/issue33927
from pykit.
Feel free to close this issue if you want I'm not impacted by this it's more to advertise you.
from pykit.
I've fixed this problem in my latest PR. I'd be glad if you got time having a look at my patch.☝️ ☝️
Thanks again :DDD
from pykit.
Related Issues (20)
- utfjson.dump(obj, encoding='utf-8')中添加indent参数
- pykit.strutil中添加对于不可见字符的处理
- ectypes.ServerID 修改结构: 增加 IDC信息 HOT 9
- zkutil.CachedReader: 实现一个可以本地缓存zookeeper 数据的reader, 用于实现消费存储在zk中的配置信息的功能. HOT 2
- 增加python2.6.5 的测试
- http module 增加个util.py 添加http相关常用操作
- http module添加基于FixedKeysDict的request的结构定义 HOT 5
- fsutil.cat stat file 特别耗ioutil HOT 1
- 考虑needle_source中needle_id为None的情况 HOT 2
- httpmultipart README中的例子需要更新
- ectypes.DriveID 增加1个属性port, 整数类型, 表示这个drive所在blockserver的port HOT 2
- ectypes缺少文档, 需要补充: NeedleIdElt, Referrer, NeedleSource
- mysqlutil.create_table(conn_argkw, jinja2=None) HOT 1
- `mysqlutil.setup_user(conn_kwargs, users)`
- ectypes 下添加error模块,统一定义通用的error类型
- 导入一个新函数到strutil里, from s2/src/mgrcmd/cmd_bucket.py: _str_to_dict()
- rangeset 不支持unicode的比较? HOT 3
- zktx: refine dealing with connection state changes
- Dependabot couldn't authenticate with https://pypi.python.org/simple/
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pykit.