Beta Bulk Upload API support about pyairtable

[yun-wang]

I believe bulk API is GA now.

[xoelop]

Yes! It's ready now
https://twitter.com/airtable/status/1174753477211213824

[walterdefoor]

It is not available to everyone yet, but here is the sign-up page for those interested in accessing it:
https://airtable.com/shr2Hk8bNAA6wp1zM?utm_source=newsletter&utm_campaign=may2019&utm_content=batchapibeta

Here is a look at my API documentation. The same "records" wrapper can be applied for updates as well. It still allows uploading single records, so it's backwards compatible.

[InfiniteLooper]

I am interested in this feature as well. Hi, @gtalarico , to @walterdefoor 's point, are you working on this already or should the community get it started?

[gtalarico]

Can you include a link? Is it available to everyone?
I didn't see anything in the main docs.

[gtalarico]

Cool! Thanks for sharing this.
Probably not a good idea to implement this until it's fully rolled out, but we will keep this on our radar

[gtalarico]

Happy to consider it if someone wants to start working on this

[jws325]

I can take a crack at this tomorrow if nobody else is doing it.
The changes would be pretty simple.

For example, right now insert is:

def insert(self, fields, typecast=False):
  return self._post(
    self.url_table, json_data={"fields": fields, "typecast": typecast}
  )

I would make it something like this:

def insert(self, fields_or_records, typecast=False):
  # if fields_or_records is a dict, assume it's the fields for a single record
  if isinstance(fields_or_records, dict):
      # Changed back to fields for backwards compatibility
      return self._post(
        self.url_table, json_data={"fields": fields_or_records, "typecast": typecast}
      )
  # assume an iterable of records was passed in.
  # pass to list constructor to make sure json serializer doesn't choke on it
  records = list(fields_or_records)

  # max records per call is 10. this should be defined as a constant in the Airtable class
  if len(records) > self.MAX_RECORDS_PER_CALL:
     raise RuntimeError(
          'Only {} records can be inserted per call'.format(self.MAX_RECORDS_PER_CALL)
     )

   return self._post(
    self.url_table, json_data={"records": records, "typecast": typecast}
    )

[InfiniteLooper]

insert() has a batch_insert() counterpart that you can utilize so that it simplifies out the conditional detection of the param type.

[jws325]

I feel like the normal insert() call should be able to insert 10 records if that's only one network request.

batch_insert() would now be mostly for inserting more than 10 records.

_batch_request() could break up the iterable into sets of 10 for maximum speed.

Also I noticed that _batch_request() does not accept the typecast param.

So right now typecast does nothing for batch requests.

[gtalarico]

This should all be implemented by now