AAPS+Nightscout for selfhosting (SSL) about androidaps

[MilosKozak]

this is my nginx setting

[MilosKozak]

current libraries doesn't allow http only. that's it

[CaptainBalou]

@swissalpine This would have saved me around 200 restarts of my nightscout during the first import of hundreds of data sets. 😂 But as scripting solved that for me all went fine. Since the initial upload of historical data out of AAPS I never saw this issue again.

I'm using Apache and searched for a similar configuration to Milos' ngix configuration and found this article which I want to share to possibly help someone fixing this in an Apache environment.

Be aware that it's not verified by me nor tested because I don't see this error anymore during normal day to day usage. I though inserted that two rules and uploading is still working.

RewriteCond %{HTTP:Connection} Upgrade [NC]
RewriteCond %{HTTP:Upgrade} websocket [NC]

[old-square-eyes]

#!/bin/bash

# This bash script generates, places and tests self signed certs for self hosted sites.
# Be sure you are aware of potential risks involved with allowing self signed certs, and customising AAPS. No warranty provided.


# Configurable Parameters
local_domain="nightscout.home"
cert_dir="/etc/ssl/certs"
key_dir="/etc/ssl/private"
home_dir="$HOME"
country="NZ"   # Update as needed
state="Auckland"   # Update as needed
locality="Auckland"   # Update as needed

# Check if directories exist
[ -d "${cert_dir}" ] || { echo "Certificate directory not found: ${cert_dir}"; exit 1; }
[ -d "${key_dir}" ] || { echo "Key directory not found: ${key_dir}"; exit 1; }

# Generate a new private key and save it to the NGINX dir
if ! openssl genrsa -out "${key_dir}/${local_domain}.key" 2048; then
    echo "Failed to generate private key."
    exit 1
fi

# Generate a new certificate and save it to the NGINX dir
if ! openssl req -new -x509 -key "${key_dir}/${local_domain}.key" \
  -out "${cert_dir}/${local_domain}.crt" -days 365 \
  -subj "/C=${country}/ST=${state}/L=${locality}/O=Nightscout/CN=${local_domain}" \
  -reqexts v3_req -reqexts SAN -extensions SAN \
  -config \
  <(echo -e "
    [req]
    distinguished_name=req_distinguished_name
    [req_distinguished_name]
    [SAN]
    subjectKeyIdentifier=hash
    authorityKeyIdentifier=keyid:always,issuer:always
    basicConstraints=CA:TRUE
    subjectAltName=DNS:${local_domain}
  "); then
    echo "Failed to generate certificate."
    exit 1
fi

# Convert the certificate to DER format and save it to the home directory for use in your device
if ! openssl x509 -in "${cert_dir}/${local_domain}.crt" -outform DER -out "${home_dir}/${local_domain}.der.crt"; then
    echo "Failed to convert certificate to DER format."
    exit 1
fi

# Output and Instructions
echo "Certificates and key generated:"
echo "  PEM Certificate: ${cert_dir}/${local_domain}.crt"
echo "  Private Key: ${key_dir}/${local_domain}.key"
echo "  DER Certificate: ${home_dir}/${local_domain}.der.crt"
echo "Optional:"
echo "  Inspect and validate your cert with: openssl x509 -in ${cert_dir}/${local_domain}.crt -text -noout" 
echo "  Review/edit NGINX config with:  nano /etc/nginx/sites-available/nightscout (where \"nightscout\" is your site)"
echo "  Test the NGINX configuration for syntax errors: nginx -t"
echo "  Copy the .der cert to your local machine for installing on your phone e.g. scp nightscout:~/${local_domain}.der.crt ~/"
echo "  Original ssl tutorial: https://medium.com/@noumaan/ssl-app-dev-a2923d5113c6"
echo "  Original NS self host tutorial: https://gist.github.com/johnmales/1b3c927f2a56aae640b4b2cd0298b1e7"

# Restarting NGINX
echo "Restarting NGINX"
if service nginx restart; then
    echo "NGINX restarted successfully."
else
    echo "Failed to restart NGINX. Run optional checks above."
    exit 1  # Exit the script with an error status
fi

[Tornado-Tim]

Have you tried Lets encrypt. Open a basic webserver for auth then copy the key over to your private NS front end proxy or NS server directly.

[alfureu]

Hi @Tornado-Tim, thanks for the follow-up. I am using NS from docker, and I allow access to my docker containers through Caddy with my domain. Caddy provides https by default with Let's Encrypt. My issue is that even if I have https+domain+NS, whoever knows the address (whichever robot scans the web), the NS page is accessible to the world.

Now, Caddy provides a basic auth solution: https://caddyserver.com/docs/caddyfile/directives/basicauth. Do you refer to this option? Do you think that the API secret can go through this authentication regardless?

[Tornado-Tim]

In that case, you can use a certbot for wildcard cert and DNS auth which will allow you to get your certs for NS. Since its wildcard, your NS will not need to be exposed.

[alfureu]

Yeah, this might work. Unfortunately I have a domain with GoDaddy, and the DNS auth module is not ready yet (no experience with the Go language, so I cannot write one myself). I was wondering, will AndroidAPS accept self-signed certificates? Or it must be from signed by entities like LE?

[MilosKozak]

i'm not familiar with caddy but nginx has allow and deny directives for access filtering

[rICTx-T1D]

When you like use a selfhosted webservice NS with private selfsigned certificate, you must import the public key in android trust store. See android setting in security selection.

[alfureu]

Yeah, these are all way too complex solutions compared to the single and simple one: there should be a login page for NS, after entering the credentials it should allow the user to the next page. Contacting the API with the API_SECRET is fine, I am not sure how it works in the background, and whether it should happen on another port once opened to the Internet.

By this simple solution one can then expose the NS server with a reverse proxy solution - with https.

[swissalpine]

I have a similar problem. AAPS can receive and send encrypted data to my selfhosted nightscout-docker, but AAPS cannot retain the connection. Instead of the "ping" I get an XHR poll error ... I have a Let's encrypt certificate for this subdomain and use a reverse proxy.

[alfureu]

OK, I made it working with the latest NS version (I used niepi/cgm-remote-monitor:14.0.7), there one needs to authenticate even for viewing the web frontend. With this I was able to open the site to the Internet.

FYI, I use Caddy for securing the web with https by default (I have a domain), if this helps, I am pasting the Caddyfile config, which is super simple:

ns.{$MY_DOMAIN} {
   reverse_proxy 192.168.0.40:1337
}

[swissalpine]

@MilosKozak Thanks. Your settings led me on the right path. I needed the following slightly modified header:
proxy_set_header Connection $connection_upgrade

[wootmasterslick]

Honestly, I find it quite frustrating that it has to be so complicated to setup a local NS server to work with AAPS. Ideally there would be an option to push the data to a not SSL server on a closed off LAN and leave al the certificates out of it.

[wootmasterslick]

Its proving to be very hard to get up and running. I tried nightscout with https settings enabled and self signed certificates. get XHR poll error. Now have it setup nginx with same certificates, still the same problem. I imported the cert to the phone, still same problem. Also tried disabling the https enforcement with engineering mode, didnt work. It looks like now I need to open up my server to get letsencrypt certbot to start working. AAPS is not giving me any info except this poll error, which it also gives when I change the IP or passphrase. I am getting lost here. Im in so deep now that is MUST get it to work or I'll go mad. Basically It looks like you effectively made it impossible to use AAPS with a self hosted local ns server on LAN, which is basically the safest option there is..

[robertrub]

To add to this discussion. NS and Mongo db in a Docker container on my QNAP NAS. External 4443 routed to internal 4443 on the router. Letsencrypt certificate for the NAS. Reverse proxy from external https 4443 to internal http 1337.
I can access locally in http via nasIP:1337 in xDrip and navigators. I can access from the Internet in https via mynasdns:4443 from xDrip and navigators. Only AAPS gives XHR poll errors even though the NS terminal in the Docker container shows the connection to AAPS (no data is received by NS from AAPS).
Same setting in xDrip sends the data to NS correctly.

I wonder if it is the :port after the address that is causing the problem(wild guess, not tested as I can't direct 443 to 4443 on the router).

[swissalpine]

No, I have nearly the same setup on my Synology NAS and saw the same behaviour: xDrip has no problems, AAPS throws errors.
With Milos hints (at the end of the thread) I was capable to fix this.

[wootmasterslick]

I managed to replace part of the nsclient in AAPS so as it accepts self signed certificates.

[petervanrijt]

@wootmasterslick promising, have you got a pull request for this to review and possible implement in AAPS? :)

[TiagoPRSilva]

I managed to replace part of the nsclient in AAPS so as it accepts self signed certificates.

Hey there, can you please share what was the fix on aaps?

i am having a similar issue for quite some time and still didnt find a solution for this.

I have setup nightscout on an ubuntu home server.

My reverse-proxy is swag container from linuxserverio.

Everything works ok, only the nsclient on aaps cant communicate with nightscout, returning the xhr poll error indefinitely.

thanks :)

[MilosKozak]

like mentioned here. create selfsigned cert and add it to android

[wootmasterslick]

I managed to replace part of the nsclient in AAPS so as it accepts self signed certificates.

Hey there, can you please share what was the fix on aaps?

i am having a similar issue for quite some time and still didnt find a solution for this.

I have setup nightscout on an ubuntu home server.

My reverse-proxy is swag container from linuxserverio.

Everything works ok, only the nsclient on aaps cant communicate with nightscout, returning the xhr poll error indefinitely.

thanks :)

Will have a look this week. I believe I changed some value in the AAPS code and that enabled it again to run using http. So no more certificates. I didn't get it to accept self signed certificates as those are checked by AAPS and found to be invalid.

Update:
I just recalled that after enabling engineering_mode it still wasn't possible to enter server url starting with "http:" since it's hardcoded in the app to be "https:". So I went to the part of code where you enter that url and removed the https enforcement. After this I could enter link to unencrypted server and I worked.

[setonez]

like mentioned here. create selfsigned cert and add it to android

Since Android 7.0 user certificates are not trusted by default (https://developer.android.com/privacy-and-security/security-config#network-security-config). What I did is:

1. Install my self-singed certificate as a trusted user certificate in my phone
2. Configure AAPS to trust on user certificates as described in the previous link (add app/src/main/res/xml/network-security-config.xml file and modify AndroidManifest.xml with android:networkSecurityConfig="@xml/network_security_config")

Hope it helps.

[old-square-eyes]

@setonez Any chance you can put in a bit more detail? I tried both adding my cert directly to the build in /res/raw per the linked instructions, and also allowing all user certs installed on Android per your suggestion (with <certificates src="user"/>). Neither worked and had the same result in the screenshot below.

My cert is uploaded to Android as a .crt in pem format. When adding to the AAPS build I tried with and without file extension (only without linked properly). I tried with and without headers.

[petervanrijt]

@old-square-eyes I suggest to use a Let's Encrypt certificate (if that is not the case), works like a charm on our set-up.

[wootmasterslick]

@setonez Any chance you can put in a bit more detail? I tried both adding my cert directly to the build in /res/raw per the linked instructions, and also allowing all user certs installed on Android per your suggestion (with <certificates src="user"/>). Neither worked and had the same result in the screenshot below.

My cert is uploaded to Android as a .crt in pem format. When adding to the AAPS build I tried with and without file extension (only without linked properly). I tried with and without headers.

I also tried getting selfsigned certs to work but see its still being validated against database even when it should be disabled via engineering_mode.
You can also modify the AAPS app code to accept a http server url in the nsclient fields (currently its locked to a https only url) then run unencrypted. Thats what I'm doing on my local network.

[old-square-eyes]

Unfortunately I have CGNAT, so home server is not an option. Though, current ISP is rubbish so might be time to find a better one.

…

On Thu, 4 Jan 2024, 10:51 petervanrijt, ***@***.***> wrote: @old-square-eyes <https://github.com/old-square-eyes> I suggest to use a Let's Encrypt certificate (if that is not the case), works like a charm on our set-up. — Reply to this email directly, view it on GitHub <#75 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGERF26W4O3Z63YEUXLGY73YMXHHBAVCNFSM4T4T4J32U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOBXGYYDCNZZGE3A> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[old-square-eyes]

I also tried getting selfsigned certs to work but see its still being validated against database even when it should be disabled via engineering_mode.

I don't have engineering mode enabled. Should I? I've just made the stated changes to network_security_config.xml

[wootmasterslick]

I also tried getting selfsigned certs to work but see its still being validated against database even when it should be disabled via engineering_mode.

I don't have engineering mode enabled. Should I? I've just made the stated changes to network_security_config.xml

Give it a try

[old-square-eyes]

I also tried getting selfsigned certs to work but see its still being validated against database even when it should be disabled via engineering_mode.

I don't have engineering mode enabled. Should I? I've just made the stated changes to network_security_config.xml

Give it a try

Because it's going to work? I don't particularly want to have to enable engineering mode if I don't have to (safety and all that).

[setonez]

@setonez Any chance you can put in a bit more detail? I tried both adding my cert directly to the build in /res/raw per the linked instructions, and also allowing all user certs installed on Android per your suggestion (with <certificates src="user"/>). Neither worked and had the same result in the screenshot below.
My cert is uploaded to Android as a .crt in pem format. When adding to the AAPS build I tried with and without file extension (only without linked properly). I tried with and without headers.

I also tried getting selfsigned certs to work but see its still being validated against database even when it should be disabled via engineering_mode. You can also modify the AAPS app code to accept a http server url in the nsclient fields (currently its locked to a https only url) then run unencrypted. Thats what I'm doing on my local network.

Hi @old-square-eyes, sorry for late reply.

These are all the changes I've made in AndroidAPS:

1. Modify AndroidManifest.xml

    diff --git a/app/src/main/AndroidManifest.xml b/app/src/main/AndroidManifest.xml
    index 6baa3a5a34..a898e90f3f 100644
    --- a/app/src/main/AndroidManifest.xml
    +++ b/app/src/main/AndroidManifest.xml
    @@ -43,6 +43,7 @@
             android:restoreAnyVersion="true"
             android:roundIcon="${appIconRound}"
             android:supportsRtl="true"
    +        android:networkSecurityConfig="@xml/network_security_config"
             android:theme="@style/AppTheme.Launcher">
    
             <meta-data
   

2. Add a new file app/src/main/res/xml/network_security_config.xml

   <?xml version="1.0" encoding="utf-8"?>
   <network-security-config>
       <base-config>
           <trust-anchors>
               <certificates src="system"/>
               <certificates src="user" />
           </trust-anchors>
       </base-config>
   </network-security-config>
   

3. Build the apk and install in my phone
4. Install the root certificate in my phone under Security Settings --> More Security Settings --> Install certificates from storage

If you problem persists I guess it might be related with your certificates. There are many guides describing how to do it, adding here the one I use, maybe it helps you: https://www.digitalocean.com/community/tutorials/how-to-set-up-and-configure-a-certificate-authority-ca-on-ubuntu-20-04

[old-square-eyes]

ere the one I use, maybe it helps you:

Thanks for the reply. I have done everything you have. I used a different tutorial to create the cert. The cert itself works fine when browsing (after allowing the exception). I'm stumped. Do you have engineering mode enabled?

[setonez]

eering mode enabled?

No, I don't.
Quick google search points to configure Subject Alternative Name (SAN) in the certificate... I configured it for my local IP...

[old-square-eyes]

Subject Alternative Name

Finally got it to work. Thanks for the pointer. Seems SAN (subjectAltName) is required. I tried with an IP first but found it also worked with DNS too. e.g. see the last line (I have a local DNS record for nightscout.home on my router)...

# Generate a new certificate without a config file openssl req -new -x509 -key "${key_dir}/${cert_name}.key" \ -out "${cert_dir}/${cert_name}.crt" -days 365 \ -subj "/C=NZ/ST=Auckland/L=Auckland/O=Nightscout/CN=nightscout.home" \ -reqexts v3_req -reqexts SAN -extensions SAN \ -config \ <(echo -e ' [req]\n distinguished_name=req_distinguished_name\n [req_distinguished_name]\n [SAN]\n subjectKeyIdentifier=hash\n authorityKeyIdentifier=keyid:always,issuer:always\n basicConstraints=CA:TRUE\n subjectAltName=DNS:nightscout.home ')

[old-square-eyes]

I created a bash script to provision the cert, distribute it for NGINX to reference. Also pump out the der format for installing on the phone. Came in handy trial and error tweaking things to get it to work. Will also use the same for adding additional SAN (such as SDLAN/ZeroTier) so I can access it outside of my house without serving it to the public internet.

[sircsaba]

wootmasterslick, Can you please tell me exactly where I can modify the AAPS code to accept HTTP for my self hosted NS ? (Which file and where and what etc ...) NS is installed on a RPI4 with docker. NS Version is 14.2.6