Comments (7)
- What OS are you using?
- What hardware is it?
- On the OS is the /tmp directory mapped to system memory?
- Are you seeing a "leak" where the application doesn't return memory to the OS after it completes?
- Or are you seeing a situation where the application just consumes memory as it runs?
If it is the last one and your /tmp is mapped to memory then the application is consuming disk space but that disk space has been mapped to memory.
If this is the case then libxlsxwriter has an option to change the location of the temp files it used:
http://libxlsxwriter.github.io/workbook_8h.html#a7329be90faba9e9ee65f58cb221aa4f1
Check if php-ext-xlsxwriter has the same opinion.
Note, libxlsxwriter has hundreds of tests for memory leaks that run in the CI so it is unlikely that it is an actual leak.
from libxlsxwriter.
@jmcnamara, thansk for the tips, there is no option to change the location of the temp files in the php-ext-xlsxwriter , and the problem it not the tmp files, I run the code on MacOS, Ubuntu , Alpine linux, all have the issue.
I think it can reproduced by this link, you can observe continious growth of memory when constant memory enabled and insert 1 million rows with a datetime. sorry it is PHP code.
from libxlsxwriter.
Ok. Sorry I can't help you other than the advice above.
from libxlsxwriter.
@jmcnamara after some experiments, I found this line caused the memory leak. does the code below remind you something please?
the below are from the datetime_writer
method above.
// it might this line. because I inserted 1 millon datetime object, the value_format will be created that much,
// I think somehow, this object not released in const memory.
lxw_format *value_format = workbook_add_format(res->workbook);
if (format_handle != NULL) {
format_copy(value_format, format_handle);
}
from libxlsxwriter.
That definitely looks like an issue. It seems to be creating a new format object for each call to datetime_writer()
. These objects are cleaned up on exit so it isn't a leak. However, it will cause a memory growth. Especially, if you writing a million cells.
You should contact the author and let them know.
You may be able to work around it by writing a number instead of a date. In Excel a date is a number with a format. You will need to covert the dates yourself but it will avoid this issue (unless the number writer is doing the same thing).
See: https://libxlsxwriter.github.io/working_with_dates.html
from libxlsxwriter.
@jmcnamara I see, thanks a lot. Finally, I understand what the real problem is. The php-ext-xlswriter
almost calls the workbook_add_format
method for each cell. that issue existed for 2 years, still not fixed. Some design issues might exist , not easy to fix. There might be two ways to solve the problem:
- Reuse existing from upper level code.
- Or don't create format for each cell, but use the format set by
set_column
method instead.
except strings, the number
has the same issue as datetime
, if no format passed for the worksheet_write_string
, worksheet_write_number
, worksheet_write_datetime
, it will use the format set by theset_column
method , right?
from libxlsxwriter.
Closed as solved by php-ext-xlswriter, @jmcnamara thanks for help,
from libxlsxwriter.
Related Issues (20)
- Linux(CentOS6) Build Error.
- LXW_ERROR_CREATING_XLSX_FILE: error creating output xlsx file. Usually a permissions error. HOT 1
- cannot run on x64 Release in Visual Studio 2022 HOT 5
- Issue: Corrupt file compiled using VS2008 on Windows HOT 9
- Issue when using constant memory with merge range HOT 2
- password HOT 1
- Still Tagging Releases? HOT 2
- 1.1.6 release did not get marked as latest release HOT 4
- How to insert image data of type CV:: Mat HOT 2
- test/functional/test_image.py::TestCompareXLSXFiles::test_image58 failure on 32bit architectures HOT 10
- Bug: workbook_validate_sheet_name buffer-overflow HOT 4
- Bug: workbook_add_worksheet does not return a reasonable worksheet HOT 1
- workbook_define_name stack-buffer-underflow HOT 2
- workbook_close stack-buffer-overflow HOT 4
- _store_array_formula heap-buffer-overflow HOT 5
- worksheet_set_selection stack-buffer-overflow HOT 1
- Question: Are people still using the libxlsxwriter cocoapod?
- Chinese in workbook_add_worksheet cause crash
- add support for xlsb HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from libxlsxwriter.