ZooKeeper Container should not rely on the default configuration.

The logging file handler is missing a size limit. I would like to add one.

Tasks that are created inside an "initialize" lifecycle methods of Application (1) and/or Service (2) objects escape the initialization phase. Examples below have been tried with Python 3.6.7

1) In this example with Application object, do_sleep() runs out of the scope of the init phase if called using ensure_future():

import asab
import asyncio

class Application(asab.Application):

	def __init__(self):
		super().__init__()

	async def initialize(self):
		await self.do_sleep() # do_sleep happens during the init phase
		asyncio.ensure_future(self.do_sleep()) # do_sleep escapes the init phase

	async def do_sleep(self):
		print("Application: before sleep")
		await asyncio.sleep(2)
		print("Application: after sleep")

	async def main(self):
		print("MAIN!")
		self.stop()

def main():
	app = Application()
	app.run()

if __name__ == '__main__':
	main()

2) In this example with initialization of Service object it is so that do_sleep() doesn't run within the init phase in both cases - when called using await and/or when called using ensure_future()

import asab
import asyncio

class MyService(asab.Service):

	async def initialize(self, app):
		await self.do_sleep() # do_sleep() AND self.initialize() escape the init phase
		asyncio.ensure_future(self.do_sleep()) # do_sleep() escapes init phase

	async def do_sleep(self):
		print("MyService: before sleep")
		await asyncio.sleep(2)
		print("MyService: after sleep")


class Application(asab.Application):

	def __init__(self):
		super().__init__()
		MyService(self, "MyService")

	async def main(self):
		print("MAIN!")
		self.stop()

def main():
	app = Application()
	app.run()

if __name__ == '__main__':
	main()

To preserve the timezone-aware attribute in datetime object, it is necessary to use the tz_aware=True option in MongoDBClient.

More info:
https://pymongo.readthedocs.io/en/stable/examples/datetimes.html

Got this error outside of lmio-testing:

Traceback (most recent call last):
  File "/opt/remote-control/remote_control.py", line 5, in <module>
    app = remote_control.RemoteControlApp()
  File "/usr/lib/python3.8/site-packages/asab/abc/singleton.py", line 14, in __call__
    cls._instances[cls] = super(Singleton, cls).__call__(*args, **kwargs)
  File "/opt/remote-control/remote_control/app.py", line 27, in __init__
    self.ZooKeeperContainer = asab.zookeeper.ZooKeeperContainer(self.ZooKeeperService, config_section_name='zookeeper')
  File "/usr/lib/python3.8/site-packages/asab/zookeeper/container.py", line 47, in __init__
    self.ZooKeeper = KazooWrapper(zksvc.App, self.Config, z_path)
  File "/usr/lib/python3.8/site-packages/asab/zookeeper/wrapper.py", line 81, in __init__
    self.Client = kazoo.client.KazooClient(hosts=url_netloc)
  File "/usr/lib/python3.8/site-packages/kazoo/client.py", line 223, in __init__
    self.set_hosts(hosts)
  File "/usr/lib/python3.8/site-packages/kazoo/client.py", line 461, in set_hosts
    self.hosts, chroot = collect_hosts(hosts)
  File "/usr/lib/python3.8/site-packages/kazoo/hosts.py", line 26, in collect_hosts
    raise ValueError("bad hostname")
ValueError: bad hostname
29-Jun-2022 14:17:16.071764 WARNING kazoo.client Cannot resolve zookeeper-1: [Errno -3] Try again

There is TODO in a log.py:29

As an alternative to calling OpenID Connect /userinfo endpoint, the app should support authentication and authorization using JWT ID token.

• asab/asab/api/log.py

  Line 34 in f63baa8

  "t": datetime.datetime.utcnow().isoformat() + 'Z',

  this seem to OK *

• asab/asab/api/log.py

  Line 124 in f63baa8

  "t": datetime.datetime.utcnow().isoformat() + 'Z',

  this seem to OK *

• asab/asab/api/service.py

  Line 82 in c690e06

  att["_c"] = datetime.datetime.utcnow().isoformat() + 'Z'

  this seem to OK *

• asab/asab-manifest.py

  Line 24 in 14de778

  'created_at': datetime.datetime.utcnow().isoformat() + 'Z',

  this seem to OK *

• asab/asab/storage/upsertor.py

  Line 19 in f66dcd7

  now = datetime.datetime.utcnow()

*) datetime.datetime.utcnow().isoformat() + 'Z' construct is OK, add a comment that it is approved.

See https://docs.python.org/3/library/datetime.html#datetime.datetime.utcnow

The InfluxDB API has been obviously evolved, so we need to reflect that as well.

https://docs.influxdata.com/influxdb/v2.1/api/#tag/Write

• New configuration options should be added:

• org
• bucket
• orgID

User can specify the db or org + bucket + (optionally) orgId.

• Configuration to support also token

If provided, then it should be used as header: Authorization: Token YOUR_INFLUX_TOKEN

https://docs.influxdata.com/influxdb/v2.1/api/#tag/Authentication

Example

[asab.metrics]
target=influxdb
org=your_org
bucket=your_bucket
token=YOUR_INFLUX_TOKEN

When app.stop is called, before app initialization is done, the stop event is not handeled properly.

-v can quickly become quite messy.
We propose a possibility to specify additional filter e.g. based on the module name (in logging) that enables easy filtering of debug messages for specific cases.

If we have this in our config file

[myapp:web]
listen=0.0.0.0 8081

and we create a web container like this

asab.web.WebContainer(websvc, 'myapp:web', config={listen: "0.0.0.0 8090"})

the web container will still listen at 0.0.0.0:8081

ASAB now blocks in exit time b/c of existing HTTP client connections (asab.web) - it causes an emergency abort eventually.

It can be fixed by implementing finalize() method in ServiceWebApp

In the following code of a WebContainer, an AccessLogger is provided to aiohttp.web.AppRunner:

self.WebAppRunner = aiohttp.web.AppRunner(
	self.WebApp,
	handle_signals=False,
	access_log=logging.getLogger(__name__[:__name__.rfind('.')] + '.al'),
	access_log_class=AccessLogger,
)

As a developer I want to be able to extend the AccessLogger so that I can enrich structured data in the access log (for example with an identityId).

inmemory is the default storage engine. The following code in asab/storage/__init__.py:

		if sttype == 'inmemory':
			from .inmemory import StorageService
			self.Service = StorageService(app, "asab.StorageService")

fails with TypeError: Can't instantiate abstract class StorageService with abstract methods get_by.

get_by must be implemented in asab/storage/inmemory.py in StorageService

@PremyslCerny

Issues:

• It is a "get" method but it builds self.ServerUrls internally ... and then returns self.ESURL
• there is no check for failure in the "else (missing)" branch

Currently, when ASAB runs in the docker, it disables logging to the console b/c it is non-interactive.
This is incompatible with docker logs ... feature.

So the proposal is to extend asab.docker service to handle this and enable logging to console when asab detects that it runs within docker & no log file is configured.

https://github.com/TeskaLabs/asab/blob/master/asab/docker/service.py

asab.LogObsolete.warning("Use of the obsolete function", struct_data={'eol':'2022-31-12'})

Log example:

21-Jul-2022 14:32:40.983884 WARNING OBSOLETE Using obsolete config section [asab:web]. Preferred section name is [web].

https://github.com/TeskaLabs/asab/blob/master/asab/web/tenant/service.py#L110

b/c some handlers are added later.

https://docs.influxdata.com/influxdb/v2.3/reference/syntax/line-protocol/#special-characters

asab.web.rest should provide a convenient function for validating incoming JSON REST requests using JSON schema. https://json-schema.org

Example of use:

@asab.web.rest.json_schema_handler(my_schema)
async def login(self, request, *, json):
	...

Remarks:

• https://horejsek.github.io/python-fastjsonschema/ (done by @horejsek from CZ.NIC)
• schema is a Python class that contain JSON (in form of Python dict() ).

The asab.mom should be taken in consideration as well.

When providing a command line argument for users to pass "listen" option in a custom application, the value must be wrapped in quotation marks, otherwise the value is not correctly parsed:

# Won't work:
app.py -l 0.0.0.0 8080
# Works:
app.py -l "0.0.0.0 8080"

By supporting the format with semicolon, users can omit the quotation marks:

app.py -l 0.0.0.0:8080
app.py -l [2001:db8:a::123]:8080

Shelve is cool !
https://docs.python.org/3/library/shelve.html

I can see that console logging is disabled when it is redirected. This is convenient for a service, but a problem when working in PyCharm. I would like to add a configuration entry or/and an option to force logging to console.

This becomes handy when multiple ASAB-based microservices are deployed in the "network: host" mode e.g. in a Docker.
The API port is to be assigned by OS and eventually advertised via e.g. Zookeeper into a common directory.

Zookeeper part is optional for now.

Notation:

0.0.0.0:auto (the auto part is important)

When operating ASAB microservice in the docker, the docker container id (from a hostname) is used for logging and metrics.
It will be much more readable when container name is used.

ASAB metrics should be extended to provide ability to integrate with Prometheus, using OpenMetrics standard.

Specifically it means that ASAB API should:

• add the new endpoint /asab/v1/metrics that communicates using OpenMetrics standard
• this new endpoint should locate metrics service (it is optional!), iterate over .Metrics and prepare OpenMetrics output line for each metrics object

https://openmetrics.io

Example of the output:

# TYPE acme_http_router_request_seconds summary
# UNIT acme_http_router_request_seconds seconds
# HELP acme_http_router_request_seconds Latency though all of ACME's HTTP request router.
acme_http_router_request_seconds_sum{path="/api/v1",method="GET"} 9036.32
acme_http_router_request_seconds_count{path="/api/v1",method="GET"} 807283.0
acme_http_router_request_seconds_created{path="/api/v1",method="GET"} 1605281325.0
acme_http_router_request_seconds_sum{path="/api/v2",method="POST"} 479.3
acme_http_router_request_seconds_count{path="/api/v2",method="POST"} 34.0
acme_http_router_request_seconds_created{path="/api/v2",method="POST"} 1605281325.0
# TYPE go_goroutines gauge
# HELP go_goroutines Number of goroutines that currently exist.
go_goroutines 69
# TYPE process_cpu_seconds counter
# UNIT process_cpu_seconds seconds
# HELP process_cpu_seconds Total user and system CPU time spent in seconds.
process_cpu_seconds_total 4.20072246e+06
# EOF

These error messages are very general and do not indicate what is the problem even though they catch specifically NoNode Error. It would be friendly to say that there is no node and pass the path string also into the message.

Clarify on how ASAB handles change dir after daemonization b/c it differs from a command-line (foreground) mode - and it can lead to confusion.

Refactor this:

/Users/ateska/Workspace/splang/sp-lang/asab/application.py:121: DeprecationWarning: The loop argument is deprecated since Python 3.8, and scheduled for removal in Python 3.10.

https://docs.python.org/3/library/asyncio-sync.html#asyncio.Event

Deprecated since version 3.8, removed in version 3.10: The loop parameter. This class has been implicitly getting the current running loop since 3.7. See What’s New in 3.10’s Removed section for more information.

• Remove f'xxx' and replace them with safer .format()
• Don't output "raw" objects from python