Is there any idea for pagination? about laravel-dynamodb

[baopham]

Pagination isn't supported right now. But try chunk - it's the closest thing to pagination depending on your needs?

[baopham]

Oops, closing to soon. I will mark this as a feature request.

Pagination isn't supported but I would love to have it implemented. PR is welcome of course.

Related issue: #15

[asantibanez]

Pagination is supported in DynamoDB in the form of LastEvaluatedKey in the result set and ExclusiveStartKey when making a query or scan. See http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/QueryAndScan.html#Pagination

[baopham]

Yep. We just need to get it in here :)

[zoul0813]

A step forward for pagination?

https://github.com/baopham/laravel-dynamodb/compare/master...zoul0813:feature/skip-offset?expand=1

I added support for skip/offset and introduced setLastEvaluatedKey to the QueryBuilder. I also refactored the getCompositeKey method and added getCompositeKeyName to match the Eloquent getKey and getKeyName methods.

This doesn't introduce pagination, but allows you to use the primaryKey/compositeKey of a model to "skip".

For example, when calling chunk to walk through large datasets ... in the event of an error (ie; provisioned capacity issues) you can use the primaryKey/compositeKey of the last model evaluated as the lastEvaluatedKey and call chunk again, skipping anything you've already gone through.

I'm using this in a custom artisan command to reindex my DynamoDB Items in ElasticSearch. I occasionally run into provisioned capacity errors, and introducing this skip/offset modification has allowed me to "resume" after an error without having to go through all the items I've already processed.

Example of my "elasticsearch:reindex" command, taking advantage of "skip". I output the current models primaryKey/compositeKey values to the console so I can copy/paste them and pass them to the "--skip" parameter of the command to "resume".

<?php
namespace App\Console\Commands;

use Illuminate\Support\Facades\App;
use Illuminate\Console\Command;

use Exception;

class ElasticSearchReindex extends Command
{
  protected $signature = 'elasticsearch:reindex {model} {--size=100} {--rebuild} {--skip=*}';
  protected $description = 'DynamoDB Table Clear';

  public function handle()
  {
    $model = 'App\\' . $this->argument('model');
    $pageSize = (int)$this->option('size');
    $skip = $this->option('skip', null);

    $rebuild = filter_var($this->option('rebuild', false), FILTER_VALIDATE_BOOLEAN);

    if(!class_exists($model)) {
      $this->error('Invalid Class: ' . $model);
      die();
    }

    if($rebuild) {
      try {
        $this->warn('Deleting and rebuilding index');
        $model::deleteIndex();
        $model::createIndex();
        $model::putMapping();
        $this->info("Index Rebuilt (delete, create, put)");
      } catch(Exception $e) {
        $this->error($e->getMessage());
        die();
      }
    }

    // if $skip is empty, then lastEvaluatedKey is set to null
    $query = $model::skip($skip);

    $this->warn('pageSize: ' . $pageSize . ', skipping: ' . print_r($skip, true) . ', lastEvaluatedKey: ' . print_r($query->getLastEvaluatedKey(), true));

    try {
      $query->chunk($pageSize, function($models) {
        static $total = 0;
        $this->info('Total: ' . $total . ', Found: ' . count($models));
        foreach($models as $item) {
          try {
            $item->addToIndex();
            $this->line('Indexed: ' . $item->getKey() . ', ' . json_encode($item->getCompositeKey()) . ', ' . json_encode($item->getCompositeKeyValue()));
          } catch(Exception $e) {
            $this->error("EXCEPTION: " . $item->getKey() . ', ' . $e->getMessage() . ")");
          }
        }
        $total += count($models);
      });
    } catch(Exception $e) {
      $this->error("EXCEPTION: " . $e->getMessage());
    }
  }
}

@baopham I can create a PR if you'd like ...

[baopham]

yep please go ahead.

[Saracevas]

Any progress on this?

[baopham]

No, no update yet. During the holidays, I should have more time to look into it.

[baopham]

@Saracevas pagination is now added. You can try v4.0.0. Please check README, since it's DynamoDB so we cannot match exactly with the same concept as relational DB.