crccheck / atx-bandc Goto Github PK

Scraping "Urban Transportation Commission" id #50
 Scraped "Urban Transportation Commission"
Scraping "Low Income Consumer Advisory Task Force" id #130
Traceback (most recent call last):
  File "/app/.venv/lib/python3.10/site-packages/django/db/models/query.py", line 581, in get_or_create
    return self.get(**kwargs), False
  File "/app/.venv/lib/python3.10/site-packages/django/db/models/query.py", line 435, in get
    raise self.model.DoesNotExist(
bandc.apps.agenda.models.Document.DoesNotExist: Document matching query does not exist.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/app/.venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/app/.venv/lib/python3.10/site-packages/django/db/backends/sqlite3/base.py", line 423, in execute
    return Database.Cursor.execute(self, query, params)
sqlite3.IntegrityError: UNIQUE constraint failed: agenda_document.edims_id

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/app/manage.py", line 21, in <module>
    main()
  File "/app/manage.py", line 17, in main
    execute_from_command_line(sys.argv)
  File "/app/.venv/lib/python3.10/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
    utility.execute()
  File "/app/.venv/lib/python3.10/site-packages/django/core/management/__init__.py", line 413, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/app/.venv/lib/python3.10/site-packages/django/core/management/base.py", line 354, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/app/.venv/lib/python3.10/site-packages/django/core/management/base.py", line 398, in execute
    output = self.handle(*args, **options)
  File "/app/bandc/apps/agenda/management/commands/scrape.py", line 48, in handle
    bandc.pull()
  File "/app/bandc/apps/agenda/models.py", line 92, in pull
    return pull_bandc(self)
  File "/app/bandc/apps/agenda/utils.py", line 200, in pull_bandc
    should_process_next = _save_page(meeting_data, doc_data, bandc=bandc)
  File "/app/bandc/apps/agenda/utils.py", line 140, in _save_page
    doc, created = Document.objects.get_or_create(
  File "/app/.venv/lib/python3.10/site-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/app/.venv/lib/python3.10/site-packages/django/db/models/query.py", line 588, in get_or_create
    return self.create(**params), True
  File "/app/.venv/lib/python3.10/site-packages/django/db/models/query.py", line 453, in create
    obj.save(force_insert=True, using=self.db)
  File "/app/.venv/lib/python3.10/site-packages/django/db/models/base.py", line 739, in save
    self.save_base(using=using, force_insert=force_insert,
  File "/app/.venv/lib/python3.10/site-packages/django/db/models/base.py", line 776, in save_base
    updated = self._save_table(
  File "/app/.venv/lib/python3.10/site-packages/django/db/models/base.py", line 881, in _save_table
    results = self._do_insert(cls._base_manager, using, fields, returning_fields, raw)
  File "/app/.venv/lib/python3.10/site-packages/django/db/models/base.py", line 919, in _do_insert
    return manager._insert(
  File "/app/.venv/lib/python3.10/site-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/app/.venv/lib/python3.10/site-packages/django/db/models/query.py", line 1270, in _insert
    return query.get_compiler(using=using).execute_sql(returning_fields)
  File "/app/.venv/lib/python3.10/site-packages/django/db/models/sql/compiler.py", line 1416, in execute_sql
    cursor.execute(sql, params)
  File "/app/.venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 66, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/app/.venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/app/.venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 79, in _execute
    with self.db.wrap_database_errors:
  File "/app/.venv/lib/python3.10/site-packages/django/db/utils.py", line 90, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/app/.venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/app/.venv/lib/python3.10/site-packages/django/db/backends/sqlite3/base.py", line 423, in execute
    return Database.Cursor.execute(self, query, params)
django.db.utils.IntegrityError: UNIQUE constraint failed: agenda_document.edims_id