Coder Social home page Coder Social logo

gtalarico / pyairtable Goto Github PK

View Code? Open in Web Editor NEW
720.0 21.0 137.0 7.71 MB

Python Api Client for Airtable

Home Page: https://pyairtable.readthedocs.io

License: MIT License

Python 99.55% Makefile 0.11% Shell 0.34%
python airtable api client api-wrapper

pyairtable's Introduction

pyAirtable (formerly airtable-python-wrapper)

CI PyPI PyPI Downloads Documentation Status codecov

Python client for the Airtable API.

Installing

pip install pyairtable

Documentation

Read the full documentation on pyairtable.readthedocs.io.

If you're still using airtable-python-wrapper and want to upgrade, read the migration guide.

Contributing

Everyone who has an idea or suggestion is welcome to contribute! As maintainers, we expect our community of users and contributors to adhere to the guidelines and expectations set forth in the Contributor Covenant. Be kind and empathetic, respect differing opinions, and stay focused on what is best for the community.

Getting started

If it's your first time working on this library, clone the repo, set up pre-commit hooks, and make sure you can run tests (and they pass). If that doesn't work out of the box, please check your local development environment before filing an issue.

% make setup
% make test

Reporting a bug

We encourage anyone to submit an issue to let us know about bugs, as long as you've followed these steps:

  1. Confirm you're on the latest version of the library and you can run the test suite locally.
  2. Check open issues to see if someone else has already reported it.
  3. Provide as much context as possible, i.e. expected vs. actual behavior, steps to reproduce, and runtime environment.
  4. If possible, reproduce the problem in a small example that you can share in the issue summary.

We ask that you never report security vulnerabilities to the GitHub issue tracker. Sensitive issues of this nature must be sent directly to the maintainers via email.

Submitting a patch

Anyone who uses this library is welcome to submit a pull request for a bug fix or a new feature. We do ask that all pull requests adhere to the following guidelines:

  1. Public functions/methods have docstrings and type annotations.
  2. New functionality is accompanied by clear, descriptive unit tests.
  3. You can run make test && make docs successfully.
  4. You have signed your commits.

If you want to discuss an idea you're working on but haven't yet finished all of the above, please open a draft pull request. That will be a clear signal that you're not asking to merge your code (yet) and are just looking for discussion or feedback.

Thanks in advance for sharing your ideas!

pyairtable's People

Contributors

a5r0n avatar ahmedhindi avatar bapcon avatar bpeterso2000 avatar daneasterman avatar dargueta avatar dependabot[bot] avatar fmorato avatar gtalarico avatar k-dal avatar larsakerson avatar lemonez avatar madindustries avatar marks avatar meli-lewis avatar mesozoic avatar michalkacprzak99 avatar nicohood avatar nikiljos avatar owfm avatar ozanhalisilter avatar richie78321 avatar rjmoggach avatar robbieaverill avatar rwdzero avatar smyja avatar thomasopsomer avatar valentinbourgoin avatar xl0 avatar yun-wang avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

pyairtable's Issues

JSONDecodeError does not exist in Python 2

_process_response uses JSONDecodeError, which is not in Python 2.
This causes airtable to crash without printing the error correctly.

We can remedy this by doing something like:

try:
from json.decoder import JSONDecodeError
except ImportError:
JSONDecodeError = ValueError

Then in _process_response use JSONDecodeError instead of json.decoder.JSONDecodeError

How to search in view ?

airtable = Airtable('BaseName', 'TableName', api_key='dfldndljndl')
records = airtable.search('Email', email, view='Authenticator')

@gtalarico i would like to perform a search in a Table View, but this code doesn't work, what's the best syntax ?

Thanks a lot for this lib !

Character character limit on "Long Text"

This is related to #16 but for Long Text type fields there is a (barely documented) limit of 100,000 characters.

I've got around this by just truncating the things I'm uploading (as it works for this use case).

new_log = str(new_log + log_notes)[-80000:]

Don't know if the best action would be to just document it somewhere, or possibly provide an option to automatically trim things so they fit.

(Great module by the way, really appreciate it!)

Typos in the documentation

issues from http://airtable-python-wrapper.readthedocs.io/en/master/api.html#module-airtable.airtable

  1. search

`airtable.search('ColumnA', 'SeachValue')

Should say 'SearchValue'`

  1. batch_delete(record_ids)

Calls delete repetitively, following set API Rate Limit (5/sec) To change the rate limit set value of airtable.API_LIMIT to the time in seconds it should sleep before calling the funciton again.

  • function is misspelled

record_ids = ['recwPQIfs4wKPyc9D', 'recwDxIfs3wDPyc3F']
airtable.batch_delete(records)

Shouldn't "record_ids" be passed instead of just "records" ?

GitHub/PyPi Release Incongruence / Missing HISTORY.MD in 11.2

Hi there -

Just wondering - the latest release on here (GitHub) is 10.0.1. The latest release on PyPi is 11.2.

Just noticed this because HISTORY.md is missing from 11.2, but is found here. The lack of it caused my conda skeleton pypi airtable-python-wrapper to fail.

Is 11.2 maintained somewhere else?

update entire table with new records information

It would be great if you could just provide all the records and update the entire table.

records = airtable.get_all()
records[1]['fields']['Status']) = 'Updated'
airtable.update_all(records)

This is particularly useful if you say have a csv file you've loaded as a dataframe from an external database and you want to populate/update the Airtable.

Deprecate "mirror" method

Does anyone out there use the mirror method? It was on the to-do list to be deprecated. It's kind of dangerous in that it'll wipe out all your existing data in a table.

There's currently no test for it -- if someone uses it I don't mind writing a test, but otherwise, we can deprecate it.

Implement append_by_field

This wrapper is amazing, fantastic, incredibly useful. Thank you.

Is it possible to append an update to not overwrite the content in a table cell, but rather add it to the beginning or end of a cell?

Cheers.

Typo

def _batch_request(self, iterable, func):

responses = []

for item in iterable:

responses.append(func(item))

time.sleept(self.API_LIMIT)

return responses

Search Param

How would I get the search result to be query-able?

In other words, let's say I have
airtable.search('Name', str(List[i])), where i in that list is some name equivalent to some name in the airtable. I then would like to pull the information from that specific item in the airtable. I understand that search returns that, but how would I then capture specific portions, say, data[i]['fields']['Location'] or similar?

Cheers!

Cannot insert record with "Link" type field

I tried to insert the following record but it threw an error. It works when I remove the User ID field. The User ID is referencing another table.

payload = { 
            'Website ID': str(website.get_wid()),
            'Title': str(website.get_title()),
            'URL': str(website.get_url()),
            'User ID': [str(user.get_uid())]
}

The official documentation suggests I should post an array of string:

EXAMPLE VALUE ["rec8116cdd76088af", "rec245db9343f55e8", "rec4f3bade67ff565"]

Handling HTTP errors

I have following function to get a specific table from Airtable or get it from local database in case of API limit / connection issues .. etc :

def get_airtable(table_name, base_key='xxx' , api_key='xxx'):
    try:
        print('try !')
        table = Airtable(base_key, table_name, api_key='xxx')
    except: #Offline ? Auth ? Need more data.
        print('except !')
        table = get_table(table_name)
    finally:
        return table

The issues is if something wrong like I mentioned above, the try except will never work and I get this Error message for example if the api key is wrong.

HTTPError                                 Traceback (most recent call last)
~\Anaconda3\lib\site-packages\airtable\airtable.py in _process_response(self, response)
    141         try:
--> 142             response.raise_for_status()
    143         except requests.exceptions.HTTPError as exc:

~\Anaconda3\lib\site-packages\requests\models.py in raise_for_status(self)
    939         if http_error_msg:
--> 940             raise HTTPError(http_error_msg, response=self)
    941 

HTTPError: 401 Client Error: Unauthorized for url: https://api.airtable.com/v0/xxx/categories

During handling of the above exception, another exception occurred:

HTTPError                                 Traceback (most recent call last)
<ipython-input-24-e600241f16c9> in <module>
     21     print(type(categories))
     22     rows = categories.get_iter()
---> 23     for row in rows:
     24         for record in row:
     25             ls.append(record['fields']['name'])

~\Anaconda3\lib\site-packages\airtable\airtable.py in get_iter(self, **options)
    235         offset = None
    236         while True:
--> 237             data = self._get(self.url_table, offset=offset, **options)
    238             records = data.get("records", [])
    239             time.sleep(self.API_LIMIT)

~\Anaconda3\lib\site-packages\airtable\airtable.py in _get(self, url, **params)
    173     def _get(self, url, **params):
    174         processed_params = self._process_params(params)
--> 175         return self._request("get", url, params=processed_params)
    176 
    177     def _post(self, url, json_data):

~\Anaconda3\lib\site-packages\airtable\airtable.py in _request(self, method, url, params, json_data)
    169     def _request(self, method, url, params=None, json_data=None):
    170         response = self.session.request(method, url, params=params, json=json_data)
--> 171         return self._process_response(response)
    172 
    173     def _get(self, url, **params):

~\Anaconda3\lib\site-packages\airtable\airtable.py in _process_response(self, response)
    159                 if "error" in error_dict:
    160                     err_msg += " [Error: {}]".format(error_dict["error"])
--> 161             raise requests.exceptions.HTTPError(err_msg)
    162         else:
    163             return response.json()

HTTPError: 401 Client Error: Unauthorized for url: https://api.airtable.com/v0/xxx/categories [Error: {'type': 'AUTHENTICATION_REQUIRED', 'message': 'Authentication required'}]

As far as I understand the issue is because at the _process_response(self, response) function it never return if HTTP error happened ,so my try except is useless.
My work around is to:
1- adjust the file itself to give return
2- Open for suggestion to catch the errors.

Search to find linked value

I'm trying to grab the value of the 'Full Student Name' from the 'Eligible Students' table by grabbing the 'id' found in the 'Monthly Registration' table and searching for it in the 'Eligible Students' table. The 'Eligible Student' is a linked field found in the 'Monthly Registration' table.

eltable = 'Eligible Students'
monthtable = 'Monthly Registration'
regtable = 'Registrations'
registrations = Airtable(key, regtable, api_key='key...Ep')
monthlies = Airtable(key, monthtable, api_key='key...Ep')
students = Airtable(key, eltable, api_key='key...Ep')

for month in monthlies.get_iter(view='Attended any'):
    for m in month:
        count = count + 1

        el_id = m['fields']['Eligible Student']
        flname = students.search('id',el_id,fields='Full Student Name')

        cur.execute("SELECT id FROM registrations WHERE first || ' ' || last = ?", (flname[0],))

        data = cur.fetchone()[0]

        cur.execute('''INSERT OR IGNORE INTO attendance (reg_id, workshop_id)
        VALUES ( ?, ? )''', (data, m['fields']['Monthly Workshop'][0]))
        conn.commit()

I'm not sure what I've done wrong, but here's the error I get when I try the above code:

Traceback (most recent call last):
  File "stats.py", line 295, in <module>
    flname = students.search('id',el_id,fields='Full Student Name')
  File "/Library/Python/2.7/site-packages/airtable/airtable.py", line 338, in search
    records = self.get_all(**options)
  File "/Library/Python/2.7/site-packages/airtable/airtable.py", line 273, in get_all
    for records in self.get_iter(**options):
  File "/Library/Python/2.7/site-packages/airtable/airtable.py", line 237, in get_iter
    data = self._get(self.url_table, offset=offset, **options)
  File "/Library/Python/2.7/site-packages/airtable/airtable.py", line 175, in _get
    return self._request("get", url, params=processed_params)
  File "/Library/Python/2.7/site-packages/airtable/airtable.py", line 171, in _request
    return self._process_response(response)
  File "/Library/Python/2.7/site-packages/airtable/airtable.py", line 161, in _process_response
    raise requests.exceptions.HTTPError(err_msg)
requests.exceptions.HTTPError: 422 Client Error: Unprocessable Entity for url: https://api.airtable.com/v0/app.../Eligible Students?fields[]=Full+Student+Name&filterByFormula={id}=[u'recyaAPrk3b74OnGz'] (Decoded URL) [Error: {u'message': u'The formula for filtering records is invalid: Invalid formula. Please check your formula text.', u'type': u'INVALID_FILTER_BY_FORMULA'}]

update_by_field error

I am logging my s3 bucket activity with Airtable and AWS Lambda.

I used the following import:

from airtable.airtable import Airtable

[ERROR] AttributeError: 'Airtable' object has no attribute 'update_by_field'
Traceback (most recent call last):
File "/var/task/function.py", line 31, in lambda_handler
airtable.update_by_field('field1 field2')

Not sure what should I do next

Allow get_all without fields specified

My use case is to retrieve a row count for a particular filter. Ideally, we want to send as lightweight as possible of a request. I currently include a single field with the smallest data size. The result set includes the record id, my requested field, and the create time.

To improve this query performance further, it'd be helpful to send a request with no fields and return just the record id. For example, I'd like to call:
airtbl_ppl.get_all(sort=['Name'], formula="{Import Status}=''", fields='')

API rate limit documentation

I had to submit a support request to Airtable in order to discover that the API rate limit applies per base, not per API KEY. After that discovery I proposed documentation updates for them to clarify that.

Would you consider updating your docs as well to clarify that point for users of your wrapper? This affects teams with multiple collaborators on one base, all using the API concurrently, raising the potential for interfering with each other's API use and running afoul of the API rate limit even though each user thinks they are complying on their own.

To be precise: if API_KEY A and API_KEY B both access the same Base, A may attempt to make 5 requests in a second, and B may attempt to make 5 requests in the same second, and that would overflow the Base rate limit.

422 client error

I am getting a 422 error when trying to insert a dictionary into Airtable. I get the same error is I use json.dumps to convert the dict into a json document.

The error goes away if I change the numbers in the dict into strings, but I don't think that should be necessary for json formatting.

{u'RowPointer': '17B42935-5002-4E97-B86C-35275F278E8F',
 u'base_style': u'11BAGK',
 u'country_origin': u'CN',
 u'description': u'ASST KIDS SUNGLASSES IN BAG12 PER INNER9',
 u'family_code': u'ACC',
 u'fourth_category': u'Z MISC',
 u'inventory_level': 110.0,
 u'item': u'11BAGK-K-WE-1',
 u'item_size': u'K',
 u'licensee_mark': None,
 u'plan_code': u'KID',
 u'product_code': u'TG-700-08',
 u'stat': u'A',
 u'unit_weight': 0.6,
 u'upc': u'072264110276',
 u'weight_units': u'KG'}

Any ideas or resources I can review?

Usage for insert attachment.

I want to upload attachment like image file using this wrapper directly rather than through official API, but seems it didn't provide this function, so does it offer this feature? Thanks!

image

Beta Bulk Upload API support

I was wondering if there were plans to support the new beta Bulk Upload API allowing for 10 records uploaded at a time rather than 1 by 1. I can take a swing at it, but was wondering if you were already planning on implementing.

Table name can't take special characters

While using the method

airtable = Airtable(base_key, table_name, api_key=api_key)

where table_name is specified to be a string 'Imported%20table', the url generator adds additional characters rendering the url invalid.

ValueError                                Traceback (most recent call last)
<ipython-input-13-3a7330261202> in <module>
----> 1 airtable = Airtable(base_key, table_name, api_key=api_key)

/usr/local/lib/python3.7/site-packages/airtable/airtable.py in __init__(self, base_key, table_name, api_key)
    127         self.url_table = posixpath.join(self.API_URL, base_key,
    128                                         url_safe_table_name)
--> 129         self.is_authenticated = self.validate_session(self.url_table)
    130 
    131     def validate_session(self, url):

/usr/local/lib/python3.7/site-packages/airtable/airtable.py in validate_session(self, url)
    134             return True
    135         elif response.status_code == 404:
--> 136             raise ValueError('Invalid base or table name: {}'.format(url))
    137         else:
    138             raise ValueError(

ValueError: Invalid base or table name: https://api.airtable.com/v0/dummy_base_key/Imported%2520table

Notice how the table name now reads 'Imported%2520table'

Search based on multiple columns

Hi!

In the functions for match and search, it seems that we can only search by one column at a time. Is there a way to search or filter by multiple columns?

Thanks!

'Airtable' object has no attribute 'get_all'

I am receiving the error in the Title when trying to run my Python within a cron job. I have been having trouble getting the module to even be recognized, I finally came up with
from airtable import airtable
and then instantiating objects with
airtable.Airtable(...
Not sure why I had to change that for the cron job, it used to just say
from airtable import Airtable
but anyway, now I can instantiate the object, but I'm getting the title error when I run

monitor_table = airtable.Airtable(MY_BASE_KEY, 'monitor')
monitor_recs = monitor_table.get_all()

I've tried reinstalling airtable-python-wrapper. What is going on? This all worked fine before trying to run from the cron job.

Bug in search & match functions

I'm trying to add new contacts if the email isn't already in Airtable.

outreach = Airtable(key, teachtable, api_key='MINE')
airtable = outreach.get_all(view='All Teachers', fields=['Email'])

if airtable.match('Email', email):
        print("Found in database ",email)
    else:
        airtable.insert({'First': first, 'Last': last, 'Email': email, 'Phone': phone, 'Title': title, 'School': school_id})
        print(first,last, 'added')

But I keep getting this error:

Traceback (most recent call last):
  File "rrisd2019-air.py", line 51, in <module>
    if airtable.match('Email', email):
AttributeError: 'list' object has no attribute 'match'

I tried using search instead ...

if outreach.match('Email', email):
        print("Found in database ",email)
    else:
        airtable.insert({'First': first, 'Last': last, 'Email': email, 'Phone': phone, 'Title': title, 'School': school_id})
        print(first,last, 'added')

but got the following:

Traceback (most recent call last):
  File "rrisd2019-air.py", line 51, in <module>
    if outreach.match('Email', email):
  File "/Library/Python/2.7/site-packages/airtable/airtable.py", line 304, in match
    for record in self.get_all(**options):
  File "/Library/Python/2.7/site-packages/airtable/airtable.py", line 273, in get_all
    for records in self.get_iter(**options):
  File "/Library/Python/2.7/site-packages/airtable/airtable.py", line 237, in get_iter
    data = self._get(self.url_table, offset=offset, **options)
  File "/Library/Python/2.7/site-packages/airtable/airtable.py", line 175, in _get
    return self._request("get", url, params=processed_params)
  File "/Library/Python/2.7/site-packages/airtable/airtable.py", line 171, in _request
    return self._process_response(response)
  File "/Library/Python/2.7/site-packages/airtable/airtable.py", line 161, in _process_response
    raise requests.exceptions.HTTPError(err_msg)
requests.exceptions.HTTPError: 422 Client Error: Unprocessable Entity for url: https://api.airtable.com/v0/MINETeacher Outreach?filterByFormula={Email}[email protected] (Decoded URL) [Error: {u'message': u'The formula for filtering records is invalid: Invalid formula. Please check your formula text.', u'type': u'INVALID_FILTER_BY_FORMULA'}]

do not limit an airtable instance to a single table_name

if we have multiple tables in a base_key, it's kind of frustrating to have to create a different Airtable instance for each table, what I propose is to make table_name a variable in each insert, update, delete call, the changes are quite minim:

def __init__(self, base_key, api_key=None):
        """
        If api_key is not provided, :any:`AirtableAuth` will attempt
        to use ``os.environ['AIRTABLE_API_KEY']``
        """
        session = requests.Session()
        session.auth = AirtableAuth(api_key=api_key)
        self.session = session
        self.base_url = posixpath.join(self.API_URL, base_key)
def record_url(self, table_name, record_id):
        """ Builds URL with record id """
        url_safe_table_name = quote(table_name, safe="")
        return posixpath.join(self.base_url, url_safe_table_name, record_id)

and a method could be updated as so :

def get(self, table_name, record_id):
        """
        Retrieves a record by its id
        >>> record = airtable.get('recwPQIfs4wKPyc9D')
        Args:
            record_id(``str``): Airtable record id
        Returns:
            record (``dict``): Record
        """
        record_url = self.record_url(table_name, record_id)
        return self._get(record_url)

if you are okay with the changes I can make a PR

serach or get_all case insensitive lookup

Hello,

I wonder if there's case insensitive lookup for get_all or search method.

 AT.get_all(formula="FIND('John', {Name})")

This query returns record that has name John but not john. How can I get john, John, jOhn, .... johN.

How to query a block?

For instance, our base has a dashboard set up with various pivot tables. How could I query them by dashboard name & pivot table name?

Configurable Timeout in Requests

Hello,

I have an unsteady internet connection, and I had to add timeout=(3,10) to line 170 to get it to kick up and not get paralysed.
If you read https://realpython.com/python-requests/#timeouts, you will say they say :
"When you make an inline request to an external service, your system will need to wait upon the response before moving on. If your application waits too long for that response, requests to your service could back up, your user experience could suffer, or your background jobs could hang.

By default, requests will wait indefinitely on the response, so you should almost always specify a timeout duration to prevent these things from happening. To set the request’s timeout, use the timeout parameter. timeout can be an integer or float representing the number of seconds to wait on a response before timing out:"

Please make this standard, it will save a lot of time to others.

Create Airtable class that receives table name on each request

Context #47 (comment)

AirtableBase():
    def get(self, table_name, record_id):
        record_url = self.record_url(record_id)
        return self._get(record_url)

Airtable():
    def get(self, record_id):
        return super().get(self.table_name, record_id)

table = Airtable(x, y, z )
table.get(record_id)

base = AirtableBase(x, y)
base.get(table_name, record_id)

Authentication 404 Error

This is my first submission, so apologies if I am just missing something obvious.

I believe the API_BASE_URL (https://api.airtable.com) has been changed recently to https://airtable.com/api, this is where I am redirected when typing in the url.

The only way I seem to be able to create and successfully authenticate an Airtable class is by removing the version variable from the url_table and by switching the API_BASE_URL to the one aforementioned.

Am I doing something wrong or have there been changes to the airtable API?

SystemError: The remote server returned an error: (422) Unprocessable Entity

File "C:\Users\sjha\Documents\Python Scripts\india-dt-pyPlugins\WWI Tools.extension\WWI Tools.tab\lib\airtable\airtable.py", line 395, in insert
File "C:\Users\sjha\Documents\Python Scripts\india-dt-pyPlugins\WWI Tools.extension\WWI Tools.tab\lib\airtable\airtable.py", line 239, in _post
File "C:\Users\sjha\Documents\Python Scripts\india-dt-pyPlugins\WWI Tools.extension\WWI Tools.tab\lib\airtable\airtable.py", line 221, in _request
SystemError: The remote server returned an error: (422) Unprocessable Entity.

Fields without values are missing entirely when calling get_all()

Imagine I have a table with two columns named id and name. There is only one row in the table. The value of id is 1. The value of name is completely blank.

I make a call to get_all(). The call is successful. The return value looks like this:

>>> airtable.get_all()                                                             
[{'fields': {'id': 1}, 'createdTime': '2019-04-04T18:18:23.000Z', 'id': '***'}]

I expect the return value to look like this:

>>> airtable.get_all()                                                             
[{'fields': {'id': 1, 'name': None}, 'createdTime': '2019-04-04T18:18:23.000Z', 'id': '***'}]

Every single column should be present in every single row returned, whether that column has a value or not.
I am aware that this could be a flaw in the airtable API, and not a bug in this Python module. If that is the case, feel free to close this bug and let me know.

Formular Filter fails with boolean value

Given this base, I should be able to:

records = airtable.search('On Display?', True)

Instead I get

HTTPError: 422 Client Error: Unprocessable Entity for url: https://api.airtable.com/v0/appNPoAJOyctb4MId/Artists?filterByFormula=%7BOn+Display%3F%7D%3DTrue

While you are at it, records = airtable.search('On Display?', "1") returns all records, should return []

Porting to pandas df

Would you guys be interested in loading the data directly into pandas and then pushing it out back to Airtable

Thoughts?

Exceptions on insert could be more helpful

If you use insert a string to an integer field in Airtable, or insert to a field which doesn't exist, an error will be thrown:

requests.exceptions.HTTPError: 422 Client Error: Unprocessable Entity for url: https://api.airtable.com/v0/<your base id>/<your table name>

I notice that Airtable provides a more helpful message in the response body, which could be returned:

{
  "error": {
    "type": "INVALID_VALUE_FOR_COLUMN",
    "message": "Field Integer_Field can not accept value 113034809"
  }
}

One way of handling this could be to catch HTTP errors, and if it converts to JSON, get the error and print this. I've implemented it like this for now:

try:
    result = at.insert({'Integer_Field': '113034809'})
except requests.exceptions.HTTPError as err:
    import json
    try:
        error_object = err.response.json().get('error', {})
        msg = (error_object.get('type', 'UNKNOWN TYPE') + ": " +
               error_object.get('message', err.response.text))
        raise ValueError(msg)
    except json.decoder.JSONDecodeError:
        raise Exception(err)

I think it would be nice if the error type+message could be raised airtable-python-wrapper.

Is Travis set up?

I made a fork and opened a PR but I don't see a Travis build despite the .travis.yml file. Is this repo not set up for it or something?

Error while attempting install

Just tried to install and received the following:

ERROR: Source distribution not available for airtable-python-wrapper: 0.11.0

Timeout/connection issues with `get_iter`

I'm running into an issue where using get_iter on large tables (eg 40k+rows) ends up having connection / timeout issues after 30m-1hr of looping through records. This happens consistently. Right after it errors, I can restart the script and it resumes from the same spot without error for another ~30min.

Is there any special error handling you recommend implementing to re-authenticate or prevent this? I'm attaching the python error below -

Traceback (most recent call last):
  File "C:\Users\mkirz\Miniconda3\lib\site-packages\urllib3\connection.py", line 171, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw)
  File "C:\Users\mkirz\Miniconda3\lib\site-packages\urllib3\util\connection.py", line 56, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
  File "C:\Users\mkirz\Miniconda3\lib\socket.py", line 743, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno 11001] getaddrinfo failed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\mkirz\Miniconda3\lib\site-packages\urllib3\connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "C:\Users\mkirz\Miniconda3\lib\site-packages\urllib3\connectionpool.py", line 343, in _make_request
    self._validate_conn(conn)
  File "C:\Users\mkirz\Miniconda3\lib\site-packages\urllib3\connectionpool.py", line 849, in _validate_conn
    conn.connect()
  File "C:\Users\mkirz\Miniconda3\lib\site-packages\urllib3\connection.py", line 314, in connect
    conn = self._new_conn()
  File "C:\Users\mkirz\Miniconda3\lib\site-packages\urllib3\connection.py", line 180, in _new_conn
    self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.VerifiedHTTPSConnection object at 0x000001FE27A98D30>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\mkirz\Miniconda3\lib\site-packages\requests\adapters.py", line 445, in send
    timeout=timeout
  File "C:\Users\mkirz\Miniconda3\lib\site-packages\urllib3\connectionpool.py", line 638, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "C:\Users\mkirz\Miniconda3\lib\site-packages\urllib3\util\retry.py", line 398, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='api.airtable.com', port=443): Max retries exceeded with url: /v0/hidingAPIkey/People/hidingkey (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x000001FE27A98D30>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed',))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:/Users/mkirz/Documents/Development Work/Github/diffbot-queries/Main_dev.py", line 90, in <module>
    record_mod = airtbl_ppl.update(airtbl_ppl_id, new_record, typecast=False)
  File "C:\Users\mkirz\Miniconda3\lib\site-packages\airtable\airtable.py", line 418, in update
    return self._patch(record_url, json_data={"fields": fields, "typecast": typecast})
  File "C:\Users\mkirz\Miniconda3\lib\site-packages\airtable\airtable.py", line 197, in _patch
    return self._request('patch', url, json_data=json_data)
  File "C:\Users\mkirz\Miniconda3\lib\site-packages\airtable\airtable.py", line 183, in _request
    json=json_data)
  File "C:\Users\mkirz\Miniconda3\lib\site-packages\requests\sessions.py", line 512, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Users\mkirz\Miniconda3\lib\site-packages\requests\sessions.py", line 622, in send
    r = adapter.send(request, **kwargs)
  File "C:\Users\mkirz\Miniconda3\lib\site-packages\requests\adapters.py", line 513, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='api.airtable.com', port=443): Max retries exceeded with url: /v0/hidingAPIkey/People/hidingkey (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x000001FE27A98D30>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed',))

Process finished with exit code 1

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.