Comments (4)
Note that widgets are in a separate package: https://github.com/yiisoft/yii-dataview
from data.
This is simple way of scanning from database. There are problematic cases that require special planning:
not has unique key or unique key is not sort key
Records:
date | color |
---|---|
2019-01-01 |
red |
2019-01-02 |
green |
2019-01-02 |
orange |
2019-01-02 |
black |
2019-01-03 |
white |
Scan pages:
SELECT * FROM ... WHERE 1 ORDER BY date LIMIT 2
// result: red, green
SELECT * FROM ... WHERE date > '2019-01-02' ORDER BY date LIMIT 2
// result: white
// missed records: orange, black
For proper operation, you need an optional key that is unique with the sort key.
date | number | color |
---|---|---|
2019-01-01 |
1 |
red |
2019-01-02 |
1 |
green |
2019-01-02 |
2 |
orange |
2019-01-02 |
3 |
black |
2019-01-03 |
1 |
white |
Scan pages:
SELECT * FROM ... WHERE 1 ORDER BY date, number LIMIT 2
// result: red, green
SELECT * FROM ... WHERE date > '2019-01-02' or (date = '2019-01-02' and number > 1) ORDER BY date, number LIMIT 2
// result: orange, black
SELECT * FROM ... WHERE date > '2019-01-02' or (date = '2019-01-02' and number > 3) ORDER BY date, number LIMIT 2
// result: white
// missed records: -
Scanning is slow (too many records, complicated filtering conditions)
Records
id | large_text | created_at |
---|---|---|
1 |
apple |
2019-01-01 00:00:00 |
2 |
apple |
2019-01-01 00:00:01 |
3 |
peach |
2019-01-01 00:00:02 |
... 1000000000 |
not apple |
2019-01-31 23:59:59 |
1000000001 |
apple |
2019-02-01 00:00:00 |
... |
Scan pages:
SELECT * FROM ... WHERE larege_text like `%apple%` order by created_at LIMIT 3
// after 10 mins
// result: 1, 2, 1000000001
With date restriction:
SELECT * FROM ... WHERE larege_text like '%apple%' and date <= '2019-01-01' order by created_at LIMIT 3
// result: 1, 2
SELECT * FROM ... WHERE larege_text like '%apple%' and date <= '2019-01-01' and id > 2 order by created_at LIMIT 3
// result: -
SELECT * FROM ... WHERE larege_text like '%apple%' and date >= '2019-01-02' and date < '2019-01-03' order by created_at LIMIT 3
// result: -
...
SELECT * FROM ... WHERE larege_text like '%apple%' and date >= '2019-02-01' and date < '2019-02-02' order by created_at LIMIT 3
// result: 1000000001
In this case, the web request may return to the user and display currently scanning: xxxx-xx-xx date
. Because the user knows he is only interested in the previous week, he can interrupt the process because he no longer makes sense.
from data.
Here's draft implementation btw. (not finished): https://github.com/yiisoft/data/pull/3/files#diff-05676246c23f4c54406cfb8f18b388cbR1
from data.
Implemented.
from data.
Related Issues (20)
- Add `iterable` support in `DataProcessorInterface` HOT 1
- Paginator refactoring concept HOT 2
- Remove `PaginatorInterface` HOT 2
- Increase test coverage to 100%
- Rename filter processor to handler
- Improve namespacing HOT 1
- Move `FilterDataValidationHelper::assertFilterHandlerIsIterable()` method to `Iterable` namespace HOT 2
- Allow pass criteria array to `Group` (reader) filter constructor HOT 1
- Throws exception when pass to `IterableDataReader` non-`IterableFilterHandlerInterface` filter handlers HOT 2
- Make accessing item methods explicit or not? HOT 2
- Add suffix `Handler` for all filter handlers HOT 1
- Finalize `IterableDataReader` HOT 4
- Rename `withFilterHandlers()` to `withAddedFilterHandlers()` into `FilterableDataInterface`
- Remove scrutinizer
- Class Yiisoft\Yii\Cycle\Data\Reader\FilterHandler\AllHandler contains 1 abstract method and must therefore be declared abstract or implement the remaining methods (Yiisoft\Data\Reader\FilterHandlerInterface::getFilterClass) HOT 3
- update irc link
- Reuse tests for all data packages
- Add `ILike` filter
- Find a test case with multibyte strings for `Like` and `ILike` filter handlers
- Do not preserve keys of filtered items in iterable data reader HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from data.