Comments (8)
It is a worthy expansion.
When we do that, the complexity to watch for obscure hang states and performance suck is between the Worker thread, the Worker queue, and the source of the stream. On some devices we only use 2 Worker threads so locking a Worker for an indeterminate amount of time might be expensive.
Several approaches come to mind:
- reverse logic a bit and the stream source drives the process. Some more thought needed here
- create a dedicated thread for the duration of the streaming operation
- instead of stream, break the stream of data into a series of fixed-size byte[] buffer chunks, similar to iphone media framework and others. Here the risk is heap memory growth from queing, but proper dance between threads can keep that down.
from tantalum.
We should specify "writeReady" so that it should not block beyond what it takes to write to the OutputStream (i.e. to the http connection). That way, there is no more blocking than there is with the variant that takes byte[].
from tantalum.
Assigning to @jnynas as he has a working implementation is his own app already
from tantalum.
I've shifted the default number of worker threads from 4 to 2 in large part because flash writes seem to suck CPU cycles on some phones. For algorithm reasons we can't really run with less that 2 workers. A nice implementation thus might need to add a Thread per stream and Task.fork(Task.SERIAL_PRIORITY) to drop ready chunks of data into flash memory.
from tantalum.
On not blocking when reading; yeah, correct user code will have to read all of the stream and enqueue the data for background writing. The advantage of stream would mainly be to only store the data that needs to be stored; e.g. in case of base64 encoded content, we would stream in the data as one decoded memory block that then gets queued for writing
... or indeed manually fork a new Task to handle that stream, for each request
from tantalum.
Streaming large objects to flash memory file system could be quite useful, for example storing songs and videos on the memory card.
Let's keep this in the backlog
from tantalum.
Reopened, javadocs need to be added before closing the issue.
from tantalum.
Did some JavaDoc and minor thread hardening.
A unit test or example to validate is still needed
from tantalum.
Related Issues (20)
- Fix Linux build HOT 4
- JavaDoc has useless stubs showing HOT 1
- Study: Explicit LWUIT support HOT 1
- PlatformAdapter for JSE HOT 3
- Tantalum Gradle build fails with latest version 1.5 of Gradle HOT 2
- Gradle findbugs output is in XML, could be in HTML HOT 2
- Remove "android stubs" from the main project HOT 5
- AndroidPlatformAdapter missing default ctor, doesn't work
- Agree on "full name" of project HOT 4
- init() call missing from setProgram for Android HOT 1
- Geocoder examples need to be updated
- JsonGetter::exec() should return "null" on error HOT 6
- JsonGetter doesn't survive if Content-Length is missing HOT 5
- Constructor with default priority should be provided HOT 4
- Put initialization in platform specific classses. HOT 8
- Continuous integration for Tantalum HOT 1
- CI for Tantalum HOT 7
- Preprocessing disabled HOT 1
- Promises (research) HOT 1
- Generics (research)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tantalum.