Server ecosystem for Standard Notes; fully self-hostable.
Home Page: https://standardnotes.com/help/self-hosting/docker
License: GNU General Public License v3.0
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
Hi,
I have successfully installed the standard notes docker containers and can verify both desktop app and web app can use my own server to sync notes. I had activated the offline plan, however when I tested the upload function, I got the following error on my web browser:
The masked part is my file server public address and matched the PUBLIC_FILES_SERVER_URL variable in the .env file.
I am using a nginx reverse proxy to listen on the 443 port of my public facing IP and redirect it to the 3125 port on my server which host the standard note container.
Can someone help me figure out where I might do wrong? Thanks!
I am trying to self host on a Raspberry Pi 4 8GB. There is no WebUI access on other clients but using curl to access on the Pi curl localhost:3000 returns a welcome message in HTML. I have allowed the ports for the server and file server through UFW. I have checked the logs for the server. I am following the tutorial here @ https://standardnotes.com/help/self-hosting/docker.
Here is my docker-compose.yml:
services:
server:
image: standardnotes/server
env_file: .env
container_name: server_self_hosted
restart: unless-stopped
ports:
- 3000:3000
- 3125:3104
volumes:
- ./logs:/var/lib/server/logs
- ./uploads:/opt/server/packages/files/dist/uploads
networks:
- standardnotes_self_hosted
localstack:
image: localstack/localstack:3.0
container_name: localstack_self_hosted
expose:
- 4566
restart: unless-stopped
environment:
- SERVICES=sns,sqs
- HOSTNAME_EXTERNAL=localstack
- LS_LOG=warn
volumes:
- ./localstack_bootstrap.sh:/etc/localstack/init/ready.d/localstack_bootstrap.sh
networks:
- standardnotes_self_hosted
db:
image: mysql:8
container_name: db_self_hosted
environment:
- MYSQL_DATABASE=standard_notes_db
- MYSQL_USER=std_notes_user
- MYSQL_ROOT_PASSWORD=REDACTED
- MYSQL_PASSWORD=REDACTED
expose:
- 3306
restart: unless-stopped
command: --default-authentication-plugin=mysql_native_password --character-set-server=utf8mb4 --collation-server=utf8mb4_general_ci
volumes:
- ./data/mysql:/var/lib/mysql
- ./data/import:/docker-entrypoint-initdb.d
networks:
- standardnotes_self_hosted
cache:
image: redis:6.0-alpine
container_name: cache_self_hosted
volumes:
- ./data/redis/:/data
expose:
- 6379
restart: unless-stopped
networks:
- standardnotes_self_hosted
networks:
standardnotes_self_hosted:
name: standardnotes_self_hosted
Contents of .env:
######
# DB #
######
DB_HOST=db
DB_PORT=3306
DB_USERNAME=std_notes_user
DB_PASSWORD=REDACTED
DB_DATABASE=standard_notes_db
DB_TYPE=mysql
#########
# CACHE #
#########
REDIS_PORT=6379
REDIS_HOST=cache
CACHE_TYPE=redis
########
# KEYS #
########
AUTH_JWT_SECRET=REDACTED
AUTH_SERVER_ENCRYPTION_SERVER_KEY=REDACTED
VALET_TOKEN_SECRET=REDACTED
PUBLIC_FILES_SERVER_URL=http://localhost:3125
Output of tail -f logs/*.log
==> logs/syncing-server-worker.log <==
{"level":"error","message":"Error occured while handling SQS message: SQSError: SQS receive message failed: The specified queue does not exist.\n at toSQSError (/opt/server/.yarn/__virtual__/sqs-consumer-virtual-a292e446e4/0/cache/sqs-consumer-npm-8.1.0-acaed020b6-7fe1c17b9f.zip/node_modules/sqs-consumer/dist/errors.js:47:22)\n at Consumer.receiveMessage (/opt/server/.yarn/__virtual__/sqs-consumer-virtual-a292e446e4/0/cache/sqs-consumer-npm-8.1.0-acaed020b6-7fe1c17b9f.zip/node_modules/sqs-consumer/dist/consumer.js:189:43)\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {\n code: 'QueueDoesNotExist',\n statusCode: 400,\n retryable: undefined,\n service: undefined,\n fault: 'client',\n time: 2023-12-17T20:40:44.970Z\n}","service":"syncing-server:worker"}
==> logs/auth-worker.log <==
{"level":"error","message":"Error occured while handling SQS message: SQSError: SQS receive message failed: The specified queue does not exist.\n at toSQSError (/opt/server/.yarn/__virtual__/sqs-consumer-virtual-a292e446e4/0/cache/sqs-consumer-npm-8.1.0-acaed020b6-7fe1c17b9f.zip/node_modules/sqs-consumer/dist/errors.js:47:22)\n at Consumer.receiveMessage (/opt/server/.yarn/__virtual__/sqs-consumer-virtual-a292e446e4/0/cache/sqs-consumer-npm-8.1.0-acaed020b6-7fe1c17b9f.zip/node_modules/sqs-consumer/dist/consumer.js:189:43)\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {\n code: 'QueueDoesNotExist',\n statusCode: 400,\n retryable: undefined,\n service: undefined,\n fault: 'client',\n time: 2023-12-17T20:40:44.974Z\n}","service":"auth:worker"}
==> logs/files-worker.log <==
{"level":"error","message":"Error occured while handling SQS message: SQSError: SQS receive message failed: The specified queue does not exist.\n at toSQSError (/opt/server/.yarn/__virtual__/sqs-consumer-virtual-a292e446e4/0/cache/sqs-consumer-npm-8.1.0-acaed020b6-7fe1c17b9f.zip/node_modules/sqs-consumer/dist/errors.js:47:22)\n at Consumer.receiveMessage (/opt/server/.yarn/__virtual__/sqs-consumer-virtual-a292e446e4/0/cache/sqs-consumer-npm-8.1.0-acaed020b6-7fe1c17b9f.zip/node_modules/sqs-consumer/dist/consumer.js:189:43)\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {\n code: 'QueueDoesNotExist',\n statusCode: 400,\n retryable: undefined,\n service: undefined,\n fault: 'client',\n time: 2023-12-17T20:40:44.974Z\n}","service":"files:worker"}
==> logs/revisions-worker.log <==
{"level":"error","message":"Error occured while handling SQS message: SQSError: SQS receive message failed: The specified queue does not exist.\n at toSQSError (/opt/server/.yarn/__virtual__/sqs-consumer-virtual-a292e446e4/0/cache/sqs-consumer-npm-8.1.0-acaed020b6-7fe1c17b9f.zip/node_modules/sqs-consumer/dist/errors.js:47:22)\n at Consumer.receiveMessage (/opt/server/.yarn/__virtual__/sqs-consumer-virtual-a292e446e4/0/cache/sqs-consumer-npm-8.1.0-acaed020b6-7fe1c17b9f.zip/node_modules/sqs-consumer/dist/consumer.js:189:43)\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {\n code: 'QueueDoesNotExist',\n statusCode: 400,\n retryable: undefined,\n service: undefined,\n fault: 'client',\n time: 2023-12-17T20:40:44.998Z\n}","service":"revisions:worker"}
==> logs/files-worker.log <==
{"level":"error","message":"Error occured while handling SQS message: SQSError: SQS receive message failed: The specified queue does not exist.\n at toSQSError (/opt/server/.yarn/__virtual__/sqs-consumer-virtual-a292e446e4/0/cache/sqs-consumer-npm-8.1.0-acaed020b6-7fe1c17b9f.zip/node_modules/sqs-consumer/dist/errors.js:47:22)\n at Consumer.receiveMessage (/opt/server/.yarn/__virtual__/sqs-consumer-virtual-a292e446e4/0/cache/sqs-consumer-npm-8.1.0-acaed020b6-7fe1c17b9f.zip/node_modules/sqs-consumer/dist/consumer.js:189:43)\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {\n code: 'QueueDoesNotExist',\n statusCode: 400,\n retryable: undefined,\n service: undefined,\n fault: 'client',\n time: 2023-12-17T20:40:45.038Z\n}","service":"files:worker"}
==> logs/syncing-server-worker.log <==
{"level":"error","message":"Error occured while handling SQS message: SQSError: SQS receive message failed: The specified queue does not exist.\n at toSQSError (/opt/server/.yarn/__virtual__/sqs-consumer-virtual-a292e446e4/0/cache/sqs-consumer-npm-8.1.0-acaed020b6-7fe1c17b9f.zip/node_modules/sqs-consumer/dist/errors.js:47:22)\n at Consumer.receiveMessage (/opt/server/.yarn/__virtual__/sqs-consumer-virtual-a292e446e4/0/cache/sqs-consumer-npm-8.1.0-acaed020b6-7fe1c17b9f.zip/node_modules/sqs-consumer/dist/consumer.js:189:43)\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {\n code: 'QueueDoesNotExist',\n statusCode: 400,\n retryable: undefined,\n service: undefined,\n fault: 'client',\n time: 2023-12-17T20:40:45.038Z\n}","service":"syncing-server:worker"}
==> logs/auth-worker.log <==
{"level":"error","message":"Error occured while handling SQS message: SQSError: SQS receive message failed: The specified queue does not exist.\n at toSQSError (/opt/server/.yarn/__virtual__/sqs-consumer-virtual-a292e446e4/0/cache/sqs-consumer-npm-8.1.0-acaed020b6-7fe1c17b9f.zip/node_modules/sqs-consumer/dist/errors.js:47:22)\n at Consumer.receiveMessage (/opt/server/.yarn/__virtual__/sqs-consumer-virtual-a292e446e4/0/cache/sqs-consumer-npm-8.1.0-acaed020b6-7fe1c17b9f.zip/node_modules/sqs-consumer/dist/consumer.js:189:43)\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {\n code: 'QueueDoesNotExist',\n statusCode: 400,\n retryable: undefined,\n service: undefined,\n fault: 'client',\n time: 2023-12-17T20:40:45.042Z\n}","service":"auth:worker"}
==> logs/revisions-worker.log <==
{"level":"error","message":"Error occured while handling SQS message: SQSError: SQS receive message failed: The specified queue does not exist.\n at toSQSError (/opt/server/.yarn/__virtual__/sqs-consumer-virtual-a292e446e4/0/cache/sqs-consumer-npm-8.1.0-acaed020b6-7fe1c17b9f.zip/node_modules/sqs-consumer/dist/errors.js:47:22)\n at Consumer.receiveMessage (/opt/server/.yarn/__virtual__/sqs-consumer-virtual-a292e446e4/0/cache/sqs-consumer-npm-8.1.0-acaed020b6-7fe1c17b9f.zip/node_modules/sqs-consumer/dist/consumer.js:189:43)\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {\n code: 'QueueDoesNotExist',\n statusCode: 400,\n retryable: undefined,\n service: undefined,\n fault: 'client',\n time: 2023-12-17T20:40:45.050Z\n}","service":"revisions:worker"}
Output of tail -f logs/*.err:
==> logs/api-gateway.err <==
localhost ([::1]:3103) open
localhost ([::1]:3104) open
localhost ([::1]:3105) open
localhost ([::1]:3101) open
localhost ([::1]:3103) open
localhost ([::1]:3104) open
localhost ([::1]:3105) open
localhost ([::1]:3101) open
==> logs/auth.err <==
localhost ([::1]:3101) open
localhost ([::1]:3101) open
localhost ([::1]:3101) open
==> logs/auth-worker.err <==
localhost ([::1]:3103) open
localhost ([::1]:3103) open
==> logs/files.err <==
db (127.0.0.1:3306) open
cache (127.0.0.1:6379) open
db (127.0.0.1:3306) open
cache (127.0.0.1:6379) open
db (127.0.0.1:3306) open
cache (127.0.0.1:6379) open
==> logs/files-worker.err <==
localhost ([::1]:3101) open
localhost ([::1]:3101) open
localhost ([::1]:3101) open
==> logs/revisions.err <==
localhost ([::1]:3101) open
localhost ([::1]:3101) open
localhost ([::1]:3101) open
==> logs/revisions-worker.err <==
localhost ([::1]:3101) open
localhost ([::1]:3101) open
localhost ([::1]:3101) open
==> logs/syncing-server.err <==
db (127.0.0.1:3306) open
cache (127.0.0.1:6379) open
db (127.0.0.1:3306) open
cache (127.0.0.1:6379) open
db (127.0.0.1:3306) open
cache (127.0.0.1:6379) open
==> logs/syncing-server-worker.err <==
localhost ([::1]:3101) open
localhost ([::1]:3101) open
localhost ([::1]:3101) open
I am using podman-compose to start the stack. There are no errors reported using podman.
Hello,
is there a way to only allow sign ups with a link or some sort of invite token that the server administrator has to generate? Or if not, is it possible to disable signups alltogether and insert users directly into the database?
Thanks!
Since updating to the latest Docker image (a9cc00a or 5446f3c) I cannot setup 2FA for my account. I get this message, each time I try to set it up: "Incorrect authentication code, please try again." Any thoughts on why this is happening? My 2FA app works just fine with every other application/website I use it with.
this error make auth not to work
'emergency_access_invitations' already exists
at Socket.emit (node:events:513:28)
at Socket.emit (node:domain:489:12)
at addChunk (node:internal/streams/readable:324:12)
at readableAddChunk (node:internal/streams/readable:297:9) {
code: 'ER_TABLE_EXISTS_ERROR',
errno: 1050,
sqlState: '42S01',
sqlMessage: "Table 'emergency_access_invitations' already exists",
sql: 'CREATE TABLE emergency_access_invitations (uuid varchar(36) NOT NULL, grantor_uuid varchar(36) NOT NULL, grantee_uuid varchar(36) NOT NULL, status varchar(36) NOT NULL, expires_at datetime NOT NULL, created_at datetime NOT NULL, updated_at datetime NOT NULL, PRIMARY KEY (uuid)) ENGINE=InnoDB'
}
Migration "emergencyAccessInvitations1680597887475" failed, error: Table 'emergency_access_invitations' already exists
query: ROLLBACK
localhost:3101 is up. Proceeding to startup.
Migrated everything from the previous version to this new version, everything was working well but when I restarted the docker containers and then tried to log again, I get the following error:
{"level":"error","message":"Could not pass the request to http://localhost:3103/auth/pkce_params on underlying service: {\"error\":{\"message\":\"Unfortunately, we couldn't handle your request. Please try again or contact our support if the error persists.\"}}"}
This seems to only happen if 2FA is enabled for the account. If 2FA is disabled, everything still works.
Running v 3.41.3 via Docker
Have successfully got the server working as well as the web app, and the files interface works well too. However, when trying to create a Listed Author Profile under the Account Settings menu, I can see that it requests the account creation by calling
https://<my-api-url>/v1/users/<uid here>/integrations/listed
This response is returned:
"message": "Listed account creation requested successfully."
It then polls this endpoint waiting for a result:
https://<my-api-url>/v1/users/<uid here>/settings/listed_author_secrets
And continually it receives a message indicating that no Listed secrets can be found, indicating that either the account creation doesn't work, or that the server doesn't receive a success message from the Listed service, and thus no secret is stored.
I'm not an expert in TypeScript or tracking through all the code to figure out where this is failing, but I'd welcome any assistance to resolve. I'm sure it's something obvious.
As the title says what is the difference between this image and the self-hosted one? There is not clear documentation about all the changes in images and containers for self-host enthusiastic!
Everytime there is a new name for a image/container and no clear instructions what has changed and which one to use.
==> logs/api-gateway.log <==
localhost:3105 is unavailable yet - waiting for it to start
==> logs/revisions.log <==
query: SELECT * FROM `INFORMATION_SCHEMA`.`COLUMNS` WHERE `TABLE_SCHEMA` = 'standard_notes_db' AND `TABLE_NAME` = 'migrations'
query: SELECT * FROM `standard_notes_db`.`migrations` `migrations` ORDER BY `id` DESC
110 migrations are already loaded in the database.
5 migrations were found in the source code.
AddVaultsUser1694157482134 is the last executed migration. It was executed on Fri Sep 08 2023 07:18:02 GMT+0000 (Coordinated Universal Time).
2 migrations are new migrations must be executed.
query: START TRANSACTION
query: ALTER TABLE `revisions_revisions` ADD `edited_by` varchar(36) NULL
query failed: ALTER TABLE `revisions_revisions` ADD `edited_by` varchar(36) NULL
error: Error: Table 'standard_notes_db.revisions_revisions' doesn't exist
query: ROLLBACK
==> logs/supervisord.log <==
2023-09-15 15:03:30,038 WARN exited: revisions (exit status 1; not expected)
2023-09-15 15:03:31,041 INFO spawned: 'revisions' with pid 1181
==> logs/revisions.log <==
localhost:3101 is up. Proceeding to startup.
The stack wont start and I think it's because of this error. For some reason it is looking for revisions_revisions?
Database unable to start
db_self_hosted | 2023-04-04T13:52:48.778797Z 1 [ERROR] [MY-012960] [InnoDB] Cannot create redo log files because data files are corrupt or the database was not shut down cleanly after creating the data files.
db_self_hosted | 2023-04-04T13:52:49.267310Z 1 [ERROR] [MY-010334] [Server] Failed to initialize DD Storage Engine
db_self_hosted | 2023-04-04T13:52:49.267572Z 0 [ERROR] [MY-010020] [Server] Data Dictionary initialization failed.
db_self_hosted | 2023-04-04T13:52:49.267606Z 0 [ERROR] [MY-010119] [Server] Aborting
db_self_hosted | 2023-04-04T13:52:49.268547Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.32) MySQL Community Server - GPL.
Clean install with the simple .env
tried remove the image and pull new image and still having the same error
Long story short, my migration to V2 wasn't a success, so figured i'd export a backup from the application and then import it into the new v2 instance.
Although this worked for the notes, the files didn't come across..
Could I please get some assistance in manually decrypting the files from my old instance with a secret?
Thanks!
I selfhost (and am a paying subscriber) but since a recent update my apps don't log in anymore. Can you help me understand what I need to do to fix this?
Thank you.
Log output:
db_self_hosted | 2023-09-18T17:10:24.412219Z 15 [Warning] [MY-013360] [Server] Plugin mysql_native_password reported: ''mysql_native_password' is deprecated and will be removed in a future release. Please use caching_sha2_password instead'
server_self_hosted | 2023-09-18 17:10:24,533 WARN exited: revisions (exit status 1; not expected)
server_self_hosted | 2023-09-18 17:10:25,536 INFO spawned: 'revisions' with pid 177
db_self_hosted | 2023-09-18T17:10:26.076519Z 16 [Warning] [MY-013360] [Server] Plugin mysql_native_password reported: ''mysql_native_password' is deprecated and will be removed in a future release. Please use caching_sha2_password instead'
server_self_hosted | 2023-09-18 17:10:27,381 INFO success: revisions entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
db_self_hosted | 2023-09-18T17:10:29.778258Z 17 [Warning] [MY-013360] [Server] Plugin mysql_native_password reported: ''mysql_native_password' is deprecated and will be removed in a future release. Please use caching_sha2_password instead'
server_self_hosted | 2023-09-18 17:10:29,925 WARN exited: revisions (exit status 1; not expected)
server_self_hosted | 2023-09-18 17:10:30,929 INFO spawned: 'revisions' with pid 195
server_self_hosted | 2023-09-18 17:10:32,035 INFO success: revisions entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
db_self_hosted | 2023-09-18T17:10:35.701972Z 18 [Warning] [MY-013360] [Server] Plugin mysql_native_password reported: ''mysql_native_password' is deprecated and will be removed in a future release. Please use caching_sha2_password instead'
server_self_hosted | 2023-09-18 17:10:35,909 WARN exited: revisions (exit status 1; not expected)
server_self_hosted | 2023-09-18 17:10:36,913 INFO spawned: 'revisions' with pid 211
server_self_hosted | 2023-09-18 17:10:37,927 INFO success: revisions entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
db_self_hosted | 2023-09-18T17:10:40.381546Z 19 [Warning] [MY-013360] [Server] Plugin mysql_native_password reported: ''mysql_native_password' is deprecated and will be removed in a future release. Please use caching_sha2_password instead'
server_self_hosted | 2023-09-18 17:10:40,513 WARN exited: revisions (exit status 1; not expected)
server_self_hosted | 2023-09-18 17:10:41,040 INFO spawned: 'revisions' with pid 227
server_self_hosted | 2023-09-18 17:10:42,051 INFO success: revisions entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
db_self_hosted | 2023-09-18T17:10:44.541190Z 20 [Warning] [MY-013360] [Server] Plugin mysql_native_password reported: ''mysql_native_password' is deprecated and will be removed in a future release. Please use caching_sha2_password instead'
server_self_hosted | 2023-09-18 17:10:44,665 WARN exited: revisions (exit status 1; not expected)
server_self_hosted | 2023-09-18 17:10:45,669 INFO spawned: 'revisions' with pid 241
server_self_hosted | 2023-09-18 17:10:46,681 INFO success: revisions entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
db_self_hosted | 2023-09-18T17:10:49.157444Z 21 [Warning] [MY-013360] [Server] Plugin mysql_native_password reported: ''mysql_native_password' is deprecated and will be removed in a future release. Please use caching_sha2_password instead'
server_self_hosted | 2023-09-18 17:10:49,273 WARN exited: revisions (exit status 1; not expected)
server_self_hosted | 2023-09-18 17:10:50,277 INFO spawned: 'revisions' with pid 255
How do I enable daily email backups for self-host
Here are some of the logs..
==> logs/auth.log <==
query: SELECT VERSION() AS version
info: All classes found using provided glob pattern "dist/migrations/mysql/*.js" : "dist/migrations/mysql/1606470249553-init_database.js,dist/migrations/mysql/1610015065194-add_revoked_sessions.js,dist/migrations/mysql/1610025371088-add_foreign_key_to_revoked_sessions.js,dist/migrations/mysql/1612191669523-add_roles_and_permissions.js,dist/migrations/mysql/1612255683992-add_roles_and_permissions_data.js,dist/migrations/mysql/1612433739754-add_more_roles_and_permissions.js,dist/migrations/mysql/1614678016791-add_user_settings.js,dist/migrations/mysql/1614771815912-add_encrypted_version.js,dist/migrations/mysql/1614775877590-add_encrypted_version_for_user.js,dist/migrations/mysql/1624434102642-update_roles_and_permissions.js,dist/migrations/mysql/1625164984414-change_setting_timestamps.js,dist/migrations/mysql/1625227894975-change_setting_value_size.js,dist/migrations/mysql/1625767770284-change_setting_value_to_nullable.js,dist/migrations/mysql/1625807999951-add_unique_setting_index.js,dist/migrations/mysql/1626268390207-remove_unique_setting_index.js,dist/migrations/mysql/1626689139110-add_user_subscriptions.js,dist/migrations/mysql/1626717016896-fix_subscription_foreign_key.js,dist/migrations/mysql/1627638504691-move_mfa_items_to_user_settings.js,dist/migrations/mysql/1629215600192-generate_user_server_key.js,dist/migrations/mysql/1629217630132-encrypt_encoded_mfa_settings.js,dist/migrations/mysql/1629223072059-flatten_mfa_setting_and_encrypt.js,dist/migrations/mysql/1629703896382-add_markdown_math.js,dist/migrations/mysql/1629705289178-add_sensitive_flag.js,dist/migrations/mysql/1629972294975-fix_basic_and_core_user.js,dist/migrations/mysql/1630661830850-fix_encryption_version_on_mfa_settings.js,dist/migrations/mysql/1630905831679-add-cancelled-flag.js,dist/migrations/mysql/1634064348750-add_offline_settings.js,dist/migrations/mysql/1634102065310-add_offline_subscriptions.js,dist/migrations/mysql/1634102764797-add_offline_user_roles.js,dist/migrations/mysql/1635167238332-remove_spreadsheets_from_plus_plan.js,dist/migrations/mysql/1635172524403-add_subscription_id.js,dist/migrations/mysql/1635344737460-add_missing_permissions.js,dist/migrations/mysql/1635860707639-add_mandatory_subscription_id.js,dist/migrations/mysql/1638388151083-add_tags_and_focus_permissions.js,dist/migrations/mysql/1639998097029-add_role_version.js,dist/migrations/mysql/1640701224273-add-smart-tags.js,dist/migrations/mysql/1640862425427-remove_no_distraction_theme.js,dist/migrations/mysql/1645094434931-add_sign_in_alerts_permission.js,dist/migrations/mysql/1646817642385-add_markdown_visual_editor_permissions.js,dist/migrations/mysql/1647253634773-fix_storage_quota_on_plans.js,dist/migrations/mysql/1647421277767-remove_user_agent.js,dist/migrations/mysql/1647862631224-add_readonly_sessions.js,dist/migrations/mysql/1648112718114-add_tag_nesting_permission.js,dist/migrations/mysql/1648458841415-add_shared_subscription_invitations.js,dist/migrations/mysql/1648550676786-add_subscription_types.js,dist/migrations/mysql/1648629732139-add_beta_files_user_role.js,dist/migrations/mysql/1649660400536-add_subscription_settings.js,dist/migrations/mysql/1649679945386-remove_files_settings_from_user_settings.js,dist/migrations/mysql/1650890853447-change_upload_quota_tiers.js,dist/migrations/mysql/1651046286472-add_2fa.js,dist/migrations/mysql/1651064332146-remove_2fa_manager.js,dist/migrations/mysql/1652258146238-add_missing_plus_permissions.js,dist/migrations/mysql/1652786070920-remove_basic_user_role.js,dist/migrations/mysql/1652880249670-add_analytics_entities.js,dist/migrations/mysql/1654760926952-add_email_backup_permission.js,dist/migrations/mysql/1654877423147-add_advanced_checklist_editor_permission.js,dist/migrations/mysql/1661771230400-revoked_session_data.js,dist/migrations/mysql/1663073954000-add_subcription_sharing_permission.js,dist/migrations/mysql/1663321030000-add_renewed_at_column.js,dist/migrations/mysql/1664971834974-groups.js,dist/migrations/mysql/1665047863774-remove-groups.js,dist/migrations/mysql/1667818539829-remove_analytics.js,dist/migrations/mysql/1671448907955-add_session_traces.js,dist/migrations/mysql/1671561748264-add_session_traces_compound_index.js,dist/migrations/mysql/1672223738686-add_authenticators.js,dist/migrations/mysql/1672227471677-add_authenticator_challenges.js,dist/migrations/mysql/1672232035280-fix_authenticator_data_types.js,dist/migrations/mysql/1672299743840-add_unique_index_on_challenges.js,dist/migrations/mysql/1672307975117-remove_compound_index.js,dist/migrations/mysql/1672317378817-add_authenticator_name.js,dist/migrations/mysql/1673951291148-add_super_editor.js,dist/migrations/mysql/1674739193816-fix-client-id-type-in-authenticators.js,dist/migrations/mysql/1678110075698-remove-sign-in-emails-on-free-acounts.js,dist/migrations/mysql/1678266947362-add-internal-team-user-role.js,dist/migrations/mysql/1678340701766-remove-authenticator-names-from-server.js,dist/migrations/mysql/1680597887475-emergency-access-invitations.js,dist/migrations/mysql/1681984540867-enable-u2f.js,dist/migrations/mysql/1682926032072-cache-entries.js,dist/migrations/mysql/1683017908845-change-cache-table-name.js"
query: SELECT * FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_SCHEMA = 'standard_notes_db' AND TABLE_NAME = 'migrations'
query: SELECT * FROM standard_notes_db.migrations migrations ORDER BY id DESC
63 migrations are already loaded in the database.
79 migrations were found in the source code.
makeUserUuidNullable1669735585016 is the last executed migration. It was executed on Tue Nov 29 2022 15:26:25 GMT+0000 (Coordinated Universal Time).
33 migrations are new migrations must be executed.
query: START TRANSACTION
query: CREATE TABLE subscription_settings (uuid varchar(36) NOT NULL, name varchar(255) NOT NULL, value text NULL, server_encryption_version tinyint NOT NULL DEFAULT 0, created_at bigint NOT NULL, updated_at bigint NOT NULL, sensitive tinyint(1) NOT NULL DEFAULT 0, user_subscription_uuid varchar(36) NOT NULL, INDEX index_subcsription_settings_on_updated_at (updated_at), INDEX index_settings_on_name_and_user_subscription_uuid (name, user_subscription_uuid), PRIMARY KEY (uuid)) ENGINE=InnoDB
query failed: CREATE TABLE subscription_settings (uuid varchar(36) NOT NULL, name varchar(255) NOT NULL, value text NULL, server_encryption_version tinyint NOT NULL DEFAULT 0, created_at bigint NOT NULL, updated_at bigint NOT NULL, sensitive tinyint(1) NOT NULL DEFAULT 0, user_subscription_uuid varchar(36) NOT NULL, INDEX index_subcsription_settings_on_updated_at (updated_at), INDEX index_settings_on_name_and_user_subscription_uuid (name, user_subscription_uuid), PRIMARY KEY (uuid)) ENGINE=InnoDB
error: Error: Table 'subscription_settings' already exists
query: ROLLBACK
==> logs/supervisord.log <==
2023-05-15 16:12:53,425 WARN exited: auth (exit status 1; not expected)
2023-05-15 16:12:54,429 INFO spawned: 'auth' with pid 234
==> logs/auth.log <==
localhost:3101 is up. Proceeding to startup.
==> logs/supervisord.log <==
2023-05-15 16:12:55,448 INFO success: auth entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
==> logs/auth-worker.log <==
localhost:3103 is unavailable yet - waiting for it to start
I followed the directions here to migrate my existing self-hosted setup to V2, however when I check the logs, I see that the syncing-server and revisions keep exiting:
==> logs/supervisord.log <==
2023-03-30 18:49:37,793 INFO spawned: 'revisions' with pid 1721
2023-03-30 18:49:37,793 INFO success: syncing-server entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2023-03-30 18:49:38,802 INFO success: revisions entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2023-03-30 18:49:39,950 WARN exited: syncing-server (exit status 1; not expected)
2023-03-30 18:49:40,104 INFO spawned: 'syncing-server' with pid 1741
2023-03-30 18:49:40,118 WARN exited: revisions (exit status 1; not expected)
When I then check the error logs, I see that access is denied to the database user std_notes_user using the password that I set in my .env file:
==> logs/syncing-server.err <==
/opt/bundled/syncing-server/.yarn/cache/mysql2-npm-3.0.1-ceda50bb4d-6bbee1ee05.zip/node_modules/mysql2/lib/packets/packet.js:728
const err = new Error(message);
^
Error: Access denied for user 'std_notes_user'@'172.21.0.4' (using password: YES)
at Packet.asError (/opt/bundled/syncing-server/.yarn/cache/mysql2-npm-3.0.1-ceda50bb4d-6bbee1ee05.zip/node_modules/mysql2/lib/packets/packet.js:728:17)
at ClientHandshake.execute (/opt/bundled/syncing-server/.yarn/cache/mysql2-npm-3.0.1-ceda50bb4d-6bbee1ee05.zip/node_modules/mysql2/lib/commands/command.js:29:26)
at PoolConnection.handlePacket (/opt/bundled/syncing-server/.yarn/cache/mysql2-npm-3.0.1-ceda50bb4d-6bbee1ee05.zip/node_modules/mysql2/lib/connection.js:487:32)
at PacketParser.onPacket (/opt/bundled/syncing-server/.yarn/cache/mysql2-npm-3.0.1-ceda50bb4d-6bbee1ee05.zip/node_modules/mysql2/lib/connection.js:94:12)
at PacketParser.executeStart (/opt/bundled/syncing-server/.yarn/cache/mysql2-npm-3.0.1-ceda50bb4d-6bbee1ee05.zip/node_modules/mysql2/lib/packet_parser.js:75:16)
at Socket.<anonymous> (/opt/bundled/syncing-server/.yarn/cache/mysql2-npm-3.0.1-ceda50bb4d-6bbee1ee05.zip/node_modules/mysql2/lib/connection.js:101:25)
at Socket.emit (node:events:513:28)
at Socket.emit (node:domain:489:12)
at addChunk (node:internal/streams/readable:324:12)
at readableAddChunk (node:internal/streams/readable:297:9) {
code: 'ER_ACCESS_DENIED_ERROR',
errno: 1045,
sqlState: '28000',
sqlMessage: "Access denied for user 'std_notes_user'@'172.21.0.4' (using password: YES)",
sql: undefined
}
Node.js v18.15.0
db (172.21.0.3:3306) open
cache (172.21.0.2:6379) open
==> logs/revisions.err <==
/opt/bundled/revisions/.yarn/cache/mysql2-npm-3.0.1-ceda50bb4d-6bbee1ee05.zip/node_modules/mysql2/lib/packets/packet.js:728
const err = new Error(message);
^
Error: Access denied for user 'std_notes_user'@'172.21.0.4' (using password: YES)
at Packet.asError (/opt/bundled/revisions/.yarn/cache/mysql2-npm-3.0.1-ceda50bb4d-6bbee1ee05.zip/node_modules/mysql2/lib/packets/packet.js:728:17)
at ClientHandshake.execute (/opt/bundled/revisions/.yarn/cache/mysql2-npm-3.0.1-ceda50bb4d-6bbee1ee05.zip/node_modules/mysql2/lib/commands/command.js:29:26)
at PoolConnection.handlePacket (/opt/bundled/revisions/.yarn/cache/mysql2-npm-3.0.1-ceda50bb4d-6bbee1ee05.zip/node_modules/mysql2/lib/connection.js:487:32)
at PacketParser.onPacket (/opt/bundled/revisions/.yarn/cache/mysql2-npm-3.0.1-ceda50bb4d-6bbee1ee05.zip/node_modules/mysql2/lib/connection.js:94:12)
at PacketParser.executeStart (/opt/bundled/revisions/.yarn/cache/mysql2-npm-3.0.1-ceda50bb4d-6bbee1ee05.zip/node_modules/mysql2/lib/packet_parser.js:75:16)
at Socket.<anonymous> (/opt/bundled/revisions/.yarn/cache/mysql2-npm-3.0.1-ceda50bb4d-6bbee1ee05.zip/node_modules/mysql2/lib/connection.js:101:25)
at Socket.emit (node:events:513:28)
at Socket.emit (node:domain:489:12)
at addChunk (node:internal/streams/readable:324:12)
at readableAddChunk (node:internal/streams/readable:297:9) {
code: 'ER_ACCESS_DENIED_ERROR',
errno: 1045,
sqlState: '28000',
sqlMessage: "Access denied for user 'std_notes_user'@'172.21.0.4' (using password: YES)",
sql: undefined
}
Node.js v18.15.0
db (172.21.0.3:3306) open
cache (172.21.0.2:6379) open
The only thing I can think is that the access is being denied because my previous self-hosted server used a distinct revisions database with a unique username and password, as defined by the variables in my previous .env file:
# Revisions Database
REVISIONS_DB_HOST=revisions-db
REVISIONS_DB_PORT=3306
REVISIONS_DB_USERNAME=revisions
REVISIONS_DB_PASSWORD=******************
REVISIONS_DB_DATABASE=revisions
REVISIONS_DB_DEBUG_LEVEL=all # "all" | "query" | "schema" | "error" | "warn" | "info" | "log" | "migration"
REVISIONS_DB_MIGRATIONS_PATH=dist/migrations/*.js
The new self-hosted server .env does not seem to include these revisions db variables any more.
Am I missing something? Is this while access is being denied to std_notes_user and why both syncing-server and revisions keep exiting?
I followed directions here: https://github.com/standardnotes/app?tab=readme-ov-file#docker-setup but when I try to load the web content, it times out (refused to connect).
my docker-compose.yml
services:
web:
image: standardnotes/web:stable
env_file: .env
ports:
- 3001:3001
restart: unless-stopped
the server is running at https://notes-api.example.com and yields
Welcome to the Standard Notes server infrastructure. Learn more at https://docs.standardnotes.com when I visit the URL.
the .env file has
DEFAULT_SYNC_SERVER=https://notes-api.example.com
When running the self-hosted web (2 yrs old on docker!!) it shows this:
# docker compose up
[+] Running 2/2
✔ Network notes-app_default Created 0.1s
✔ Container notes-app-web-1 Created 5.4s
Attaching to web-1
web-1 | Prestart Step 1/1 - Removing server lock
web-1 | Starting Server...
web-1 | lerna-lite notice cli v1.6.0
web-1 | lerna-lite info versioning independent
web-1 | lerna-lite notice filter including "@standardnotes/web-server"
web-1 | lerna-lite info filter ["@standardnotes/web-server"]
web-1 | lerna-lite info Executing command in 1 package: "yarn run start"
Any further assistance would be appreciated! Thanks
I have a self-hosted server instance of standardnotes, and a self-hosted professional license. I set up the env variable PUBLIC_FILES_SERVER_URL in the .env file. The Standard Notes Desktop App (Mac M1) is logged into my self-hosted Standard Notes Server via the Custom sync server url. My Self-hosted instance is behind a reverse proxy available, the ports for the Server and the Files Server have their own unique DNS names.
Server: https://standardnotes.vps4.hochguertel.work/
Files Server: https://files-standardnotes.vps4.hochguertel.work/
When I try to upload a File, Image etc. I received a Error Message: Unable to start upload session, followed by the message: There was an error while uploading the file.
Can someone help me to get the file upload working?
Is there a Debug Mode for the Desktop app, to see a more detailed error message?
I use the latest version of standardnotes/server container. Just set up standardnotes today (22. July.2023) so everything is fresh and latest.
Files Logoutput:
❯ tail -f logs/files*.log
==> logs/files.log <==
db:3306 is unavailable yet - waiting for it to start
db:3306 is up. Proceeding to startup.
cache:6379 is up. Proceeding to startup.
{"level":"info","message":"Server started on port 3104","service":"files"}
db:3306 is up. Proceeding to startup.
cache:6379 is up. Proceeding to startup.
{"level":"info","message":"Server started on port 3104","service":"files"}
db:3306 is up. Proceeding to startup.
cache:6379 is up. Proceeding to startup.
{"level":"info","message":"Server started on port 3104","service":"files"}
==> logs/files-worker.log <==
localhost:3101 is unavailable yet - waiting for it to start
localhost:3101 is up. Proceeding to startup.
{"level":"info","message":"Starting worker...","service":"files"}
localhost:3101 is unavailable yet - waiting for it to start
localhost:3101 is up. Proceeding to startup.
{"level":"info","message":"Starting worker...","service":"files"}
{"level":"info","message":"Alive and kicking!","service":"files"}
localhost:3101 is unavailable yet - waiting for it to start
localhost:3101 is up. Proceeding to startup.
{"level":"info","message":"Starting worker...","service":"files"}I tried to setup standard notes according to: https://standardnotes.com/help/self-hosting/docker
When tailing the error logs I get:
==> logs/files.err <==
nc: bad address 'db'
==> logs/files-worker.err <==
==> logs/revisions.err <==
==> logs/revisions-worker.err <==
==> logs/syncing-server.err <==
nc: bad address 'db'
In the .env file I did set the port to 3307 and the same in the docker-compose.yml file for mysql.
I did so because I have another mysql running on port 3306 elsewhere.
Also, when checking http://device-ipv4-address-on-local-network:3000 I get nothing
Any suggestions?
Tailing the .log files says:
waiting for it to start on every file.
The docker container localstack_self_hosted (latest, pulled today) throws countless errors (all the same):
localstack_self_hosted | 2023-12-19T08:52:03.668 ERROR --- [ asgi_gw_2] l.aws.handlers.logging : exception during call chain: Operation detection failed. Missing Action in request for query-protocol service ServiceModel(sqs).
localstack_self_hosted | 2023-12-19T08:52:03.671 ERROR --- [ asgi_gw_1] l.aws.handlers.logging : exception during call chain: Operation detection failed. Missing Action in request for query-protocol service ServiceModel(sqs).
localstack_self_hosted | 2023-12-19T08:52:03.736 ERROR --- [ asgi_gw_5] l.aws.handlers.logging : exception during call chain: Operation detection failed. Missing Action in request for query-protocol service ServiceModel(sqs).
localstack_self_hosted | 2023-12-19T08:52:03.744 ERROR --- [ asgi_gw_0] l.aws.handlers.logging : exception during call chain: Operation detection failed. Missing Action in request for query-protocol service ServiceModel(sqs).
localstack_self_hosted | 2023-12-19T08:52:03.748 ERROR --- [ asgi_gw_0] l.aws.handlers.logging : exception during call chain: Operation detection failed. Missing Action in request for query-protocol service ServiceModel(sqs).
localstack_self_hosted | 2023-12-19T08:52:03.752 ERROR --- [ asgi_gw_2] l.aws.handlers.logging : exception during call chain: Operation detection failed. Missing Action in request for query-protocol service ServiceModel(sqs).
localstack_self_hosted | 2023-12-19T08:52:03.796 ERROR --- [ asgi_gw_5] l.aws.handlers.logging : exception during call chain: Operation detection failed. Missing Action in request for query-protocol service ServiceModel(sqs).
localstack_self_hosted | 2023-12-19T08:52:03.799 ERROR --- [ asgi_gw_1] l.aws.handlers.logging : exception during call chain: Operation detection failed. Missing Action in request for query-protocol service ServiceModel(sqs).
localstack_self_hosted | 2023-12-19T08:52:03.811 ERROR --- [ asgi_gw_0] l.aws.handlers.logging : exception during call chain: Operation detection failed. Missing Action in request for query-protocol service ServiceModel(sqs).
localstack_self_hosted | 2023-12-19T08:52:03.812 ERROR --- [ asgi_gw_2] l.aws.handlers.logging : exception during call chain: Operation detection failed. Missing Action in request for query-protocol service ServiceModel(sqs).
localstack_self_hosted | 2023-12-19T08:52:03.856 ERROR --- [ asgi_gw_3] l.aws.handlers.logging : exception during call chain: Operation detection failed. Missing Action in request for query-protocol service ServiceModel(sqs).
localstack_self_hosted | 2023-12-19T08:52:03.859 ERROR --- [ asgi_gw_5] l.aws.handlers.logging : exception during call chain: Operation detection failed. Missing Action in request for query-protocol service ServiceModel(sqs).
localstack_self_hosted | 2023-12-19T08:52:03.863 ERROR --- [ asgi_gw_1] l.aws.handlers.logging : exception during call chain: Operation detection failed. Missing Action in request for query-protocol service ServiceModel(sqs).
localstack_self_hosted | 2023-12-19T08:52:03.870 ERROR --- [ asgi_gw_0] l.aws.handlers.logging : exception during call chain: Operation detection failed. Missing Action in request for query-protocol service ServiceModel(sqs).
localstack_self_hosted | 2023-12-19T08:52:03.916 ERROR --- [ asgi_gw_4] l.aws.handlers.logging : exception during call chain: Operation detection failed. Missing Action in request for query-protocol service ServiceModel(sqs).
localstack_self_hosted | 2023-12-19T08:52:03.917 ERROR --- [ asgi_gw_3] l.aws.handlers.logging : exception during call chain: Operation detection failed. Missing Action in request for query-protocol service ServiceModel(sqs).
localstack_self_hosted | 2023-12-19T08:52:03.924 ERROR --- [ asgi_gw_5] l.aws.handlers.logging : exception during call chain: Operation detection failed. Missing Action in request for query-protocol service ServiceModel(sqs).
localstack_self_hosted | 2023-12-19T08:52:03.926 ERROR --- [ asgi_gw_1] l.aws.handlers.logging : exception during call chain: Operation detection failed. Missing Action in request for query-protocol service ServiceModel(sqs).
localstack_self_hosted | 2023-12-19T08:52:03.991 ERROR --- [ asgi_gw_3] l.aws.handlers.logging : exception during call chain: Operation detection failed. Missing Action in request for query-protocol service ServiceModel(sqs).
My docker compose file:
---
version: "3.8"
networks:
standardnotes_self_hosted:
name: standardnotes_self_hosted
proxy:
external: true
intranet:
external: true
services:
webclient:
image: standardnotes/web:stable
env_file: webclient.env
# ports:
# - 3001:3001
depends_on:
- server
networks:
- standardnotes_self_hosted
- proxy
- intranet
labels:
- "traefik.enable=true"
- "traefik.docker.network=proxy"
# https://standardnotes.vps4.hochguertel.work
- "traefik.http.routers.standardnotes.entrypoints=http"
- "traefik.http.routers.standardnotes.rule=Host(`standardnotes.vps4.hochguertel.work`)"
- "traefik.http.routers.standardnotes.service=standardnotes"
- "traefik.http.services.standardnotes.loadbalancer.server.port=3001"
- "traefik.http.middlewares.standardnotes-https-redirect.redirectscheme.scheme=https"
- "traefik.http.routers.standardnotes.middlewares=standardnotes-https-redirect"
- "traefik.http.routers.standardnotes-secure.entrypoints=https"
- "traefik.http.routers.standardnotes-secure.rule=Host(`standardnotes.vps4.hochguertel.work`)"
- "traefik.http.routers.standardnotes-secure.service=standardnotes"
- "traefik.http.routers.standardnotes-secure.tls=true"
- "traefik.http.routers.standardnotes-secure.tls.certresolver=http"
server:
image: standardnotes/server
env_file: .env
container_name: server_self_hosted
# ports:
# - 3000:3000
# - 3125:3104
depends_on:
- db
volumes:
- ./logs:/var/lib/server/logs
- ./uploads:/opt/server/packages/files/dist/uploads:Z
networks:
- standardnotes_self_hosted
- proxy
- intranet
labels:
- "traefik.enable=true"
- "traefik.docker.network=proxy"
# https://api.standardnotes.vps4.hochguertel.work
- "traefik.http.routers.api-standardnotes.entrypoints=http"
- "traefik.http.routers.api-standardnotes.rule=Host(`api.standardnotes.vps4.hochguertel.work`)"
- "traefik.http.routers.api-standardnotes.service=api-standardnotes"
- "traefik.http.services.api-standardnotes.loadbalancer.server.port=3000"
- "traefik.http.middlewares.api-standardnotes-https-redirect.redirectscheme.scheme=https"
- "traefik.http.routers.api-standardnotes.middlewares=standardnotes-https-redirect"
- "traefik.http.routers.api-standardnotes-secure.entrypoints=https"
- "traefik.http.routers.api-standardnotes-secure.rule=Host(`api.standardnotes.vps4.hochguertel.work`)"
- "traefik.http.routers.api-standardnotes-secure.service=api-standardnotes"
- "traefik.http.routers.api-standardnotes-secure.tls=true"
- "traefik.http.routers.api-standardnotes-secure.tls.certresolver=http"
# https://files.standardnotes.vps4.hochguertel.work
- "traefik.http.routers.files-standardnotes.entrypoints=http"
- "traefik.http.routers.files-standardnotes.rule=Host(`files.standardnotes.vps4.hochguertel.work`)"
- "traefik.http.routers.files-standardnotes.service=files-standardnotes"
- "traefik.http.services.files-standardnotes.loadbalancer.server.port=3104"
- "traefik.http.middlewares.files-standardnotes-https-redirect.redirectscheme.scheme=https"
- "traefik.http.routers.files-standardnotes.middlewares=files-standardnotes-https-redirect"
- "traefik.http.routers.files-standardnotes-secure.entrypoints=https"
- "traefik.http.routers.files-standardnotes-secure.rule=Host(`files.standardnotes.vps4.hochguertel.work`)"
- "traefik.http.routers.files-standardnotes-secure.service=files-standardnotes"
- "traefik.http.routers.files-standardnotes-secure.tls=true"
- "traefik.http.routers.files-standardnotes-secure.tls.certresolver=http"
localstack:
image: localstack/localstack:1.3
container_name: localstack_self_hosted
# expose:
# - 4566
depends_on:
- server
restart: unless-stopped
environment:
- SERVICES=sns,sqs
- HOSTNAME_EXTERNAL=localstack
- LS_LOG=warn
volumes:
- ./localstack_bootstrap.sh:/etc/localstack/init/ready.d/localstack_bootstrap.sh
networks:
- standardnotes_self_hosted
db:
image: mysql:8
container_name: db_self_hosted
environment:
- MYSQL_DATABASE=***
- MYSQL_USER=***
- MYSQL_ROOT_PASSWORD=***
- MYSQL_PASSWORD=***
# expose:
# - 3306
restart: unless-stopped
command: --default-authentication-plugin=mysql_native_password --character-set-server=utf8mb4 --collation-server=utf8mb4_general_ci
volumes:
- ./data/mysql:/var/lib/mysql
- ./data/import:/docker-entrypoint-initdb.d
networks:
- standardnotes_self_hosted
- intranet
cache:
image: redis:6.0-alpine
container_name: cache_self_hosted
# expose:
# - 6379
depends_on:
- db
restart: unless-stopped
volumes:
- ./data/redis/:/data
networks:
- standardnotes_self_hosted
How can I solve this?
Recently I deployed standardnotes on my home server and everything seems work fine, but I had some trouble setting up my reverse proxy facing this warning:
[WARNING] (3182) : config : Server standardnotes_ipvANY/web is DOWN, changed from server-state after a reload. 2 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
According to HAproxy documentation, they encourage the use of OPTIONS instead of GET to perform health check.
OPTIONS is the method usually best to perform server checks, HEAD and GET can also be used. If the server gets marked as down in the stats page then changing this to GET usually has the biggest chance of working, but might cause more processing overhead on the websever and is less easy to filter out of its logs.
Is there any guide or helm chart to deploy this application to a Kubernetes cluster?
I am getting the following error when trying to set this up on Raspberry Pi 4:
no matching manifest for linux/arm/v7 in the manifest list entries
After running:
docker compose pull && docker compose up -d
How about add "version" attribute in docker-compose.yml default sample file like:
version: "3"
at least on my server, I have to add this line to make docker-compose work well
Possibly related to #460
I have an existing StandardNotes paid account, that I've been using with the SN's own server. I have 2FA set up on that account for logging in.
I'm testing the StandardNotes self-hosted server, and I've set it up on Docker on a server on my network, using the instructions here.
It works fine, and I can connect to it with a "virgin" SN free account I set up for initial testing purposes, from my desktop app, on a laptop on the same network. (Different email address from my existing SN paid account.)
I then tried setting up a separate Workspace in the desktop app, to sign into my self-hosted server using my paid account credentials. (If I'm ultimately going to move to my self-hosted server, I still want to be able to access my paid account functionality when using it).
However, on attempting to log in to my self-hosted server, Standard Notes tells me my 2FA credentials are incorrect. I'm not sure how the self-hosted server "knows" that it should request my 2FA code, since it's literally brand new and I've never logged into it with 2FA activated.
So I assume 2FA is requested as part of the app's log-in process and tried the following:
Then:
So I'm not clear at this point what's going on. I can log into the SN-hosted server with my paid account just fine, with 2FA either enabled or disabled; but the app won't log in my paid account into my self-hosted server because it asks me for a TOTP regardless of whether 2FA is activated for the account or not; and tells me that my 2FA credentials are incorrect whenever it does.
What does a subscription do on its server? All that happens to me is a notification with congratulations about the subscription, in the end it is not there? what does this function do?
use windows client
Testing selfhosted legacy and want to update for v2.
Following the guide from https://standardnotes.com/help/self-hosting/legacy-migration the first command
./server.sh dump-db
gives error "Unknown command", and that seems there is no dump-db section in my server.sh.
How to proceed?
Hello, I launched the docker container according to the instructions, but I can't access the web browser
root@NAS:/standardnotes# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
6534156dbcff localstack/localstack:3.0 "docker-entrypoint.sh" 29 seconds ago Up 8 seconds (health: starting) 4510-4559/tcp, 4566/tcp, 5678/tcp localstack_self_hosted
8c43dd005037 standardnotes/server "docker-entrypoint.sh" 29 seconds ago Up 10 seconds 0.0.0.0:3000->32401/tcp, :::3000->32401/tcp server_self_hosted
8daa7a1588c2 mysql:8 "docker-entrypoint.s…" 29 seconds ago Up 13 seconds 3306/tcp, 33060/tcp db_self_hosted
438fefd463bb redis:6.0-alpine "docker-entrypoint.s…" 29 seconds ago Up 10 seconds 6379/tcp cache_self_hosted
root@NAS:/standardnotes#I go to localhost:32401 or 192.168.1.49:32401(this is my server on the local network) and get the error This site can't be reached(ERR_CONNECTION_REFUSED) why so, please help me, my docker started without problems in docker-compose.yaml I changed only the port of standardnotes/server
Hi, great project! I starting using it and I saw the tag of the server is based on hash and not on semantic version, any chance to start using tag on hub docker with the same version?
A plus will be using MAJOR and MINOR to make possible to upgrade only with non-breaking changes, like:
docker pull standardnotes/server:1+home-server
docker pull standardnotes/server:1.12+home-server
docker pull standardnotes/server:1.12.2+home-serverMaybe remove the server from the package name to make more clean:
docker pull standardnotes/server:1+home
docker pull standardnotes/server:1.12+home
docker pull standardnotes/server:1.12.2+homeI think this can be made in automatic way on workflow here.
Hello, I recently switchted from the self hosted legacy setup to this new server structure. When I was on the legacy service everything worked flawlessly, but with the new server setup (I appreciate the new architechture with fewer containers!) the synchronization seems to have "hiccups". The basic sync functionality works somewhat, but I often get sync request errors. For example after logging in, the first sync request (when I change a note) always comes back with an error but any subsequent sync reqests go through. Then if I wait a while, the first request is an error again and any subsequent requests go through. So basically I can sync my notes, yes, but other functionalities like for example deleting a note or multiple notes or creating a note are barely possible. The action is so often interrupted by these unable to sync messages and I sometimes get a ton of "conflicting" note copies due to the errors.
Never had any of these issues on the legacy setup. Any help is greatly appreciated!
These are error messages I could dig up:
App says
Failed to load resource: the server responded with a status of 500 ()
From api-gateway.log:
{"level":"error","message":"Could not pass the request to http://localhost:3101/items/sync on underlying service: {"error":{"message":"Unfortunately, we couldn't handle your request. Please try again or contact our support if the error persists."}}"}
From all the worker logs (same message in auth-worker.log, files-worker.log, revisions-worker.log, syncing-server-worker.log):
{"level":"error","message":"Error occured while handling SQS message: SQSError: SQS receive message failed: getaddrinfo EAI_AGAIN localstack\n at toSQSError (/opt/bundled/auth/.yarn/virtual/sqs-consumer-virtual-603a7c5831/0/cache/sqs-consumer-npm-6.2.1-857abd3d30-eba3c37353.zip/node_modules/sqs-consumer/dist/errors.js:40:22)\n at Consumer.receiveMessage (/opt/bundled/auth/.yarn/virtual/sqs-consumer-virtual-603a7c5831/0/cache/sqs-consumer-npm-6.2.1-857abd3d30-eba3c37353.zip/node_modules/sqs-consumer/dist/consumer.js:149:43)\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {\n code: 'Error',\n statusCode: undefined,\n retryable: undefined,\n service: undefined,\n fault: undefined,\n time: 2023-03-06T13:57:17.140Z\n}"}
when try download exist file its show error while downloading file
and the files.log have this
i check and the file is exist in same place in disk where is already
{"level":"error","message":"Could not get file metadata for resource: 2334354d-0328-4fab-b586-42401c7e2a0c/77b14b47-3409-45f6-a70d-1bd47b4b9d25","service":"files"}
also when i debug this request when try to download i se my file server url reply like this
Could not get file metadata
When updating the Docker containers today, the syncing-server could not be started because the environment variable DB_TYPE is not set.
The (undocumented?) environment variable DB_TYPE did not have to be set by the administrator until now. Apparently mysql was used as default value.
This has apparently changed in at least one place recently :
https://github.com/standardnotes/server/blob/f9183b4c623e16e4c6bd2ab1e77760b8eeabbcc3/packages/syncing-server/src/Bootstrap/DataSource.ts#LL10C60-L10C60
On a self hosted standalone server the environment variable DB_TYPE defaulted to mysql. It is not documented anywhere as far as I know.
Where do we have to set that variable?
Is mysql the correct value?
Error message:
{"log":"[Docker] Starting Web...\n","stream":"stdout","time":"2023-05-29T21:05:42.386538834Z"}
{"log":"/workspace/packages/syncing-server/dist/src/Bootstrap/Env.js:16\n","stream":"stderr","time":"2023-05-29T21:05:44.469639055Z"}
{"log":" throw new Error(`Environment variable ${key} not set`);\n","stream":"stderr","time":"2023-05-29T21:05:44.469680423Z"}
{"log":" ^\n","stream":"stderr","time":"2023-05-29T21:05:44.469685332Z"}
{"log":"\n","stream":"stderr","time":"2023-05-29T21:05:44.469688839Z"}
{"log":"Error: Environment variable DB_TYPE not set\n","stream":"stderr","time":"2023-05-29T21:05:44.469692365Z"}
{"log":" at Env.get (/workspace/packages/syncing-server/dist/src/Bootstrap/Env.js:16:19)\n","stream":"stderr","time":"2023-05-29T21:05:44.469696323Z"}
{"log":" at Object.\u003canonymous\u003e (/workspace/packages/syncing-server/dist/src/Bootstrap/DataSource.js:9:34)\n","stream":"stderr","time":"2023-05-29T21:05:44.469699979Z"}
{"log":" at Module._compile (node:internal/modules/cjs/loader:1267:14)\n","stream":"stderr","time":"2023-05-29T21:05:44.469710388Z"}
{"log":" at Module._extensions..js (node:internal/modules/cjs/loader:1321:10)\n","stream":"stderr","time":"2023-05-29T21:05:44.469714536Z"}
{"log":" at require$$0.Module._extensions..js (/workspace/.pnp.cjs:10987:33)\n","stream":"stderr","time":"2023-05-29T21:05:44.469718914Z"}
{"log":" at Module.load (node:internal/modules/cjs/loader:1125:32)\n","stream":"stderr","time":"2023-05-29T21:05:44.469722451Z"}
{"log":" at require$$0.Module._load (/workspace/.pnp.cjs:10825:14)\n","stream":"stderr","time":"2023-05-29T21:05:44.469725667Z"}
{"log":" at Module.require (node:internal/modules/cjs/loader:1149:19)\n","stream":"stderr","time":"2023-05-29T21:05:44.469728191Z"}
{"log":" at require (node:internal/modules/helpers:121:18)\n","stream":"stderr","time":"2023-05-29T21:05:44.469730977Z"}
{"log":" at Object.\u003canonymous\u003e (/workspace/packages/syncing-server/dist/src/Bootstrap/Container.js:8:22)\n","stream":"stderr","time":"2023-05-29T21:05:44.469733471Z"}
{"log":"\n","stream":"stderr","time":"2023-05-29T21:05:44.469736266Z"}
{"log":"Node.js v20.1.0\n","stream":"stderr","time":"2023-05-29T21:05:44.469739893Z"}
After updating my self-hosted docker instance using the latest ("v2") paradigm, my containers entered an infinite loop trying to start up and crashing.
I was able to track it down to a pair of initial errors in the auth.log
emergency_access_invitations as a new table per the 1680597887475 migration. This then fails at the ALTER TABLE step to add the grantor foreign key.emergency_access_invitations which already exists.Note: For brevity, I truncated the list of migrations to just the last few starting with the one my logs report as being the most recent completed migration; but the rest of the log context is in tact.
localhost:3101 is up. Proceeding to startup.
query: SELECT VERSION() AS `version`
info: All classes found using provided glob pattern "...TRUNCATED...,dist/migrations/mysql/1678340701766-remove-authenticator-names-from-server.js,dist/migrations/mysql/1680597887475-emergency-access-invitations.js,dist/migrations/mysql/1681984540867-enable-u2f.js,dist/migrations/mysql/1682926032072-cache-entries.js,dist/migrations/mysql/1683017908845-change-cache-table-name.js"
query: SELECT * FROM `INFORMATION_SCHEMA`.`COLUMNS` WHERE `TABLE_SCHEMA` = 'standard_notes_db' AND `TABLE_NAME` = 'migrations'
query: SELECT * FROM `standard_notes_db`.`migrations` `migrations` ORDER BY `id` DESC
92 migrations are already loaded in the database.
79 migrations were found in the source code.
removeAuthenticatorNamesFromServer1678340701766 is the last executed migration. It was executed on Thu Mar 09 2023 05:45:01 GMT+0000 (Coordinated Universal Time).
4 migrations are new migrations must be executed.
query: START TRANSACTION
query: CREATE TABLE `emergency_access_invitations` (`uuid` varchar(36) NOT NULL, `grantor_uuid` varchar(36) NOT NULL, `grantee_uuid` varchar(36) NOT NULL, `status` varchar(36) NOT NULL, `expires_at` datetime NOT NULL, `created_at` datetime NOT NULL, `updated_at` datetime NOT NULL, PRIMARY KEY (`uuid`)) ENGINE=InnoDB
query: ALTER TABLE `emergency_access_invitations` ADD CONSTRAINT `grantor_uuid_fk` FOREIGN KEY (`grantor_uuid`) REFERENCES `users`(`uuid`) ON DELETE CASCADE ON UPDATE NO ACTION
query failed: ALTER TABLE `emergency_access_invitations` ADD CONSTRAINT `grantor_uuid_fk` FOREIGN KEY (`grantor_uuid`) REFERENCES `users`(`uuid`) ON DELETE CASCADE ON UPDATE NO ACTION
error: Error: Referencing column 'grantor_uuid' and referenced column 'uuid' in foreign key constraint 'grantor_uuid_fk' are incompatible.
query: ROLLBACK
localhost:3101 is up. Proceeding to startup.
query: SELECT VERSION() AS `version`
info: All classes found using provided glob pattern "dist/migrations/mysql/*.js" : "...TRUNCATED...,dist/migrations/mysql/1678340701766-remove-authenticator-names-from-server.js,dist/migrations/mysql/1680597887475-emergency-access-invitations.js,dist/migrations/mysql/1681984540867-enable-u2f.js,dist/migrations/mysql/1682926032072-cache-entries.js,dist/migrations/mysql/1683017908845-change-cache-table-name.js"
query: SELECT * FROM `INFORMATION_SCHEMA`.`COLUMNS` WHERE `TABLE_SCHEMA` = 'standard_notes_db' AND `TABLE_NAME` = 'migrations'
query: SELECT * FROM `standard_notes_db`.`migrations` `migrations` ORDER BY `id` DESC
92 migrations are already loaded in the database.
79 migrations were found in the source code.
removeAuthenticatorNamesFromServer1678340701766 is the last executed migration. It was executed on Thu Mar 09 2023 05:45:01 GMT+0000 (Coordinated Universal Time).
4 migrations are new migrations must be executed.
query: START TRANSACTION
query: CREATE TABLE `emergency_access_invitations` (`uuid` varchar(36) NOT NULL, `grantor_uuid` varchar(36) NOT NULL, `grantee_uuid` varchar(36) NOT NULL, `status` varchar(36) NOT NULL, `expires_at` datetime NOT NULL, `created_at` datetime NOT NULL, `updated_at` datetime NOT NULL, PRIMARY KEY (`uuid`)) ENGINE=InnoDB
query failed: CREATE TABLE `emergency_access_invitations` (`uuid` varchar(36) NOT NULL, `grantor_uuid` varchar(36) NOT NULL, `grantee_uuid` varchar(36) NOT NULL, `status` varchar(36) NOT NULL, `expires_at` datetime NOT NULL, `created_at` datetime NOT NULL, `updated_at` datetime NOT NULL, PRIMARY KEY (`uuid`)) ENGINE=InnoDB
error: Error: Table 'emergency_access_invitations' already exists
query: ROLLBACK
First of all , I'm self hosting. However i think this might affect production env. So i decide to submit an issue here.
I 've get a 500 error in /check-integrity request and the log shows
{"level":"error","message":"ReplyError: ERR bit offset is not an integer or out of range\n at parseError (/workspace/.yarn/cache/redis-parser-npm-3.0.0-7ebe40abcb-02c1783a0c.zip/node_modules/redis-parser/lib/parser.js:179:12)\n at parseType (/workspace/.yarn/cache/redis-parser-npm-3.0.0-7ebe40abcb-02c1783a0c.zip/node_modules/redis-parser/lib/parser.js:302:14)"}
After some dig , i found that it's caused in analyticsStore's functions (wasActivityDone , markActivity also unmarkActivity ) in saveNotesCountStatistics when analyticsId is undefined.
If a user is registered before standardnotes/auth@7e18652#diff-036501ae7a88668f7055c26d219ba27a0d89fb99e24224f84c152cb86499b05dR74 , analyticsId will be undefined. ( i can not found other code using analyticsEntityRepository.save , so i assume the analyticsId only generated here.) and as a self hosting user , there're no chance to set analyticsId (without touching db) if registered before that commit.
As you defined in
or ,wasActivityDone , markActivity unmarkActivity etc.I have a self hosted V2 and it seems that i am not able to log in anymore. Already logged in clients have no issues.
Only when i try to log in i get an error on the web client that states:
"Unfortunately, we couldn't handle your request. Please try again or contact our support if the error persists."
On the desktop client it says:
"Invalid authentication, please try again."
the only place where somethig is stated is in the log file:
==> logs/auth.log <==
query: SELECT `session`.`uuid` AS `session_uuid`, `session`.`user_uuid` AS `session_user_uuid`, `session`.`hashed_access_token` AS `session_hashed_access_token`, `session`.`hashed_refresh_token` AS `session_hashed_refresh_token`, `session`.`access_expiration` AS `session_access_expiration`, `session`.`refresh_expiration` AS `session_refresh_expiration`, `session`.`api_version` AS `session_api_version`, `session`.`user_agent` AS `session_user_agent`, `session`.`created_at` AS `session_created_at`, `session`.`updated_at` AS `session_updated_at`, `session`.`readonly_access` AS `session_readonly_access` FROM `sessions` `session` WHERE `session`.`uuid` = ? -- PARAMETERS: ["beb28ebe-6c64-4d3e-a6ec-7140ff3c0286"]
query: SELECT `user`.`uuid` AS `user_uuid`, `user`.`version` AS `user_version`, `user`.`email` AS `user_email`, `user`.`pw_nonce` AS `user_pw_nonce`, `user`.`encrypted_server_key` AS `user_encrypted_server_key`, `user`.`server_encryption_version` AS `user_server_encryption_version`, `user`.`kp_created` AS `user_kp_created`, `user`.`kp_origination` AS `user_kp_origination`, `user`.`pw_cost` AS `user_pw_cost`, `user`.`pw_key_size` AS `user_pw_key_size`, `user`.`pw_salt` AS `user_pw_salt`, `user`.`pw_alg` AS `user_pw_alg`, `user`.`pw_func` AS `user_pw_func`, `user`.`encrypted_password` AS `user_encrypted_password`, `user`.`created_at` AS `user_created_at`, `user`.`updated_at` AS `user_updated_at`, `user`.`locked_until` AS `user_locked_until`, `user`.`num_failed_attempts` AS `user_num_failed_attempts` FROM `users` `user` WHERE `user`.`uuid` = ? -- PARAMETERS: ["098fc60c-828b-4d0c-9f35-6c550e28462a"]
query: SELECT `user`.`uuid` AS `user_uuid`, `user`.`version` AS `user_version`, `user`.`email` AS `user_email`, `user`.`pw_nonce` AS `user_pw_nonce`, `user`.`encrypted_server_key` AS `user_encrypted_server_key`, `user`.`server_encryption_version` AS `user_server_encryption_version`, `user`.`kp_created` AS `user_kp_created`, `user`.`kp_origination` AS `user_kp_origination`, `user`.`pw_cost` AS `user_pw_cost`, `user`.`pw_key_size` AS `user_pw_key_size`, `user`.`pw_salt` AS `user_pw_salt`, `user`.`pw_alg` AS `user_pw_alg`, `user`.`pw_func` AS `user_pw_func`, `user`.`encrypted_password` AS `user_encrypted_password`, `user`.`created_at` AS `user_created_at`, `user`.`updated_at` AS `user_updated_at`, `user`.`locked_until` AS `user_locked_until`, `user`.`num_failed_attempts` AS `user_num_failed_attempts` FROM `users` `user` WHERE `user`.`email` = ? -- PARAMETERS: ["[email protected]"]
query: SELECT `authenticator`.`uuid` AS `authenticator_uuid`, `authenticator`.`user_uuid` AS `authenticator_user_uuid`, `authenticator`.`credential_id` AS `authenticator_credential_id`, `authenticator`.`credential_public_key` AS `authenticator_credential_public_key`, `authenticator`.`counter` AS `authenticator_counter`, `authenticator`.`credential_device_type` AS `authenticator_credential_device_type`, `authenticator`.`credential_backed_up` AS `authenticator_credential_backed_up`, `authenticator`.`transports` AS `authenticator_transports`, `authenticator`.`created_at` AS `authenticator_created_at`, `authenticator`.`updated_at` AS `authenticator_updated_at` FROM `authenticators` `authenticator` WHERE `authenticator`.`user_uuid` = ? -- PARAMETERS: ["098fc60c-828b-4d0c-9f35-6c550e28462a"]
query: SELECT `setting`.`uuid` AS `setting_uuid`, `setting`.`name` AS `setting_name`, `setting`.`value` AS `setting_value`, `setting`.`server_encryption_version` AS `setting_server_encryption_version`, `setting`.`created_at` AS `setting_created_at`, `setting`.`updated_at` AS `setting_updated_at`, `setting`.`sensitive` AS `setting_sensitive`, `setting`.`user_uuid` AS `setting_user_uuid` FROM `settings` `setting` WHERE `setting`.`name` = ? AND `setting`.`user_uuid` = ? ORDER BY updated_at DESC LIMIT 1 -- PARAMETERS: ["MFA_SECRET","098fc60c-828b-4d0c-9f35-6c550e28462a"]
query: SELECT `user`.`uuid` AS `user_uuid`, `user`.`version` AS `user_version`, `user`.`email` AS `user_email`, `user`.`pw_nonce` AS `user_pw_nonce`, `user`.`encrypted_server_key` AS `user_encrypted_server_key`, `user`.`server_encryption_version` AS `user_server_encryption_version`, `user`.`kp_created` AS `user_kp_created`, `user`.`kp_origination` AS `user_kp_origination`, `user`.`pw_cost` AS `user_pw_cost`, `user`.`pw_key_size` AS `user_pw_key_size`, `user`.`pw_salt` AS `user_pw_salt`, `user`.`pw_alg` AS `user_pw_alg`, `user`.`pw_func` AS `user_pw_func`, `user`.`encrypted_password` AS `user_encrypted_password`, `user`.`created_at` AS `user_created_at`, `user`.`updated_at` AS `user_updated_at`, `user`.`locked_until` AS `user_locked_until`, `user`.`num_failed_attempts` AS `user_num_failed_attempts` FROM `users` `user` WHERE `user`.`uuid` = ? -- PARAMETERS: ["098fc60c-828b-4d0c-9f35-6c550e28462a"]
{"level":"error","message":"Error: Unsupported state or unable to authenticate data\n at Decipheriv.final (node:internal/crypto/cipher:199:29)\n at CryptoNode.aes256GcmDecrypt (/opt/bundled/auth/packages/sncrypto-node/dist/src/CryptoNode.js:23:103)\n at CrypterNode.decryptUserServerKey (/opt/bundled/auth/packages/auth/dist/src/Domain/Encryption/CrypterNode.js:71:32)\n at CrypterNode.decryptForUser (/opt/bundled/auth/packages/auth/dist/src/Domain/Encryption/CrypterNode.js:53:51)\n at SettingDecrypter.decryptSettingValue (/opt/bundled/auth/packages/auth/dist/src/Domain/Setting/SettingDecrypter.js:32:33)\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n at async SettingService.findSettingWithDecryptedValue (/opt/bundled/auth/packages/auth/dist/src/Domain/Setting/SettingService.js:65:25)\n at async VerifyMFA.execute (/opt/bundled/auth/packages/auth/dist/src/Domain/UseCase/VerifyMFA.js:100:31)\n at async InversifyExpressAuthController.pkceParams (/opt/bundled/auth/packages/auth/dist/src/Infra/InversifyExpressUtils/HomeServer/HomeServerAuthController.js:136:35)","service":"auth"}
==> logs/api-gateway.log <==
{"level":"error","message":"Could not pass the request to http://localhost:3103/auth/pkce_params on underlying service: {\"error\":{\"message\":\"Unfortunately, we couldn't handle your request. Please try again or contact our support if the error persists.\"}}","service":"api-gateway"}
there is nothing in the error log and docker compose logs does not help either.
would be able to tell me what to do?
Problem Description:
In the process of migrating services from Pi 4 to Pi 5 and I had to start with a fresh new install of Standard Notes. Followed their page's V2 instructions and the db does not seem to fully startup and initialize for some reason.
Troubleshooting Attempted
I have tried removing all related volumes and folders several times, switching folders and modifying permissions, etc. with no luck. I would love to console into the container but I cannot while it's restarting and it never finishes unfortunately.
I believe the crux of the issue is in the "--initialize specified but the data directory has files in it. Aborting." line but I cannot find anywhere where that flag is present. But the issue could be anything, I am not very confident. Any help is greatly appreciated!
Documentation
Error Output:
==> logs/syncing-server.err <==
nc: bad address 'db'
==> logs/files.err <==
nc: bad address 'db'
==> logs/syncing-server.err <==
nc: bad address 'db'
nc: bad address 'db'
==> logs/files.err <==
nc: bad address 'db'
Container Log Output:
2023-12-20 22:22:49+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.2.0-1.el8 started.
2023-12-20 22:22:49+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
2023-12-20 22:22:49+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.2.0-1.el8 started.
2023-12-20 22:22:50+00:00 [Note] [Entrypoint]: Initializing database files
2023-12-20T22:22:50.163260Z 0 [System] [MY-015017] [Server] MySQL Server Initialization - start.
2023-12-20T22:22:50.164994Z 0 [Warning] [MY-011068] [Server] The syntax '--skip-host-cache' is deprecated and will be removed in a future release. Please use SET GLOBAL host_cache_size=0 instead.
2023-12-20T22:22:50.165104Z 0 [System] [MY-013169] [Server] /usr/sbin/mysqld (mysqld 8.2.0) initializing of server in progress as process 81
2023-12-20T22:22:50.166985Z 0 [ERROR] [MY-010457] [Server] --initialize specified but the data directory has files in it. Aborting.
2023-12-20T22:22:50.166991Z 0 [ERROR] [MY-013236] [Server] The designated data directory /var/lib/mysql/ is unusable. You can remove all files that the server added to it.
2023-12-20T22:22:50.167174Z 0 [ERROR] [MY-010119] [Server] Aborting
2023-12-20T22:22:50.167380Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.2.0) MySQL Community Server - GPL.
2023-12-20T22:22:50.167709Z 0 [System] [MY-015018] [Server] MySQL Server Initialization - end
.env File Input:
######
# DB #
######
DB_HOST=db
DB_PORT=3306
DB_USERNAME=std_notes_user
DB_PASSWORD=4676cb18dfcb075b754783d5
DB_DATABASE=standard_notes_db
DB_TYPE=mysql
#########
# CACHE #
#########
REDIS_PORT=6379
REDIS_HOST=cache
CACHE_TYPE=redis
########
# KEYS #
########
AUTH_JWT_SECRET=219a5454ede149a785b801a6065eabebf0c4ab2d021c8dc9800ee60d24c38dea
AUTH_SERVER_ENCRYPTION_SERVER_KEY=770cd531474884ec18e295fcdd08dfcd026e56b61478b1fb6f71ae0b46c0f600
VALET_TOKEN_SECRET=ab0c0de58282aff01279ab62b19dd857adc68790d635cdcf2e21c0026c63069f
PUBLIC_FILES_SERVER_URL=https://10.0.0.117:3125
Docker-Compose Contents:
services:
server:
image: standardnotes/server
env_file: .env
container_name: server_self_hosted
restart: unless-stopped
ports:
- 2999:3000
- 3125:3104
volumes:
- ./logs:/var/lib/server/logs
- ./uploads:/opt/server/packages/files/dist/uploads
networks:
- standardnotes_self_hosted
localstack:
image: localstack/localstack:3.0
container_name: localstack_self_hosted
expose:
- 4566
restart: unless-stopped
environment:
- SERVICES=sns,sqs
- HOSTNAME_EXTERNAL=localstack
- LS_LOG=warn
volumes:
- ./localstack_bootstrap.sh:/etc/localstack/init/ready.d/localstack_bootstrap.sh
networks:
- standardnotes_self_hosted
db:
image: mysql:8
container_name: db_self_hosted
environment:
- MYSQL_DATABASE=standard_notes_db
- MYSQL_USER=std_notes_user
- MYSQL_ROOT_PASSWORD=4676cb18dfcb075b754783d5
- MYSQL_PASSWORD=4676cb18dfcb075b754783d5
expose:
- 3306
restart: unless-stopped
command: --authentication-policy=mysql_native_password --character-set-server=utf8mb4 --collation-server=utf8mb4_general_ci
volumes:
- ./data/mysql:/var/lib/mysql
- ./data/import:/docker-entrypoint-initdb.d
networks:
- standardnotes_self_hosted
cache:
image: redis:6.0-alpine
container_name: cache_self_hosted
volumes:
- ./data/redis/:/data
expose:
- 6379
restart: unless-stopped
networks:
- standardnotes_self_hosted
networks:
standardnotes_self_hosted:
name: standardnotes_self_hosted
I have tested ubuntu 18.04, 22.04 versions and the following error occurs in all versions and does not sync and register.
{"level":"error","message":"Error occured while handling SQS message: SQSError: SQS receive message failed: The specified queue does not exist for this wsdl version.\n at toSQSError (/opt/bundled/syncing-server/.yarn/virtual/sqs-consumer-virtual-603a7c5831/0/cache/sqs-consumer-npm-6.2.1-857abd3d30-eba3c37353.zip/node_modules/sqs-consumer/dist/errors.js:40:22)\n at Consumer.receiveMessage (/opt/bundled/syncing-server/.yarn/virtual/sqs-consumer-virtual-603a7c5831/0/cache/sqs-consumer-npm-6.2.1-857abd3d30-eba3c37353.zip/node_modules/sqs-consumer/dist/consumer.js:149:43)\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {\n code: 'AWS.SimpleQueueService.NonExistentQueue',\n statusCode: 400,\n retryable: undefined,\n service: undefined,\n fault: 'client',\n time: 2023-03-25T21:33:28.951Z\n}"}
Not sure what prompted this, but I see a bunch of error lines repeating this in auth-worker.log:
"Error occured while handling SQS message: SQSError: SQS receive message failed: Unexpected token '<', \"<?xml vers\"... is not valid JSON\n Deserialization error: to see the raw response, inspect the hidden field {error}.$response on this object.\n at toSQSError
I am not sure how to get the full object that is failing to be deserialized. auth-worker.err doesn't have anything except dns errors on the cache, I'd imagine from bootup.
http://localhost:3000/ is inaccessible, logs have been stuck on the following for the past 20 mins:
==> logs/syncing-server-worker.log <==
localhost:3101 is unavailable yet - waiting for it to start
==> logs/revisions-worker.log <==
localhost:3101 is unavailable yet - waiting for it to start
==> logs/revisions.log <==
db:3306 is unavailable yet - waiting for it to start
==> logs/syncing-server.log <==
db:3306 is unavailable yet - waiting for it to start
==> logs/files.log <==
db:3306 is unavailable yet - waiting for it to start
self host show error when try login on new devices
Unfortunately, we couldn't handle your request. Please try again or contact our support if the error persists.
but in old devices thats already signed in all normal
also healt check show OK
Please can you provide me a guide to run and build the application on my local environment?
As the title implies, the e-mail address entered during login is expected to be correct on a case sensitive basis, which IMO is a bit of a UX issue since e-mail addresses have never historically been sensitive to case variations.
I’d be happy to submit a PR if we agree on this.
I went to download some files I uploaded a few weeks ago on another device (different from the device I uploaded the file on) and I was getting the error There was an error while downloading the file.
I checked the uploads directory on my self-hosted instance and noticed that there were no new files added to the directory for the files I was having issues with downloading. I tested uploading a new file and the interface said everything was successful, yet there was no new file in the uploads directory.
The issue ended up being that I had this path in my docker-compose from when I performed the initial install: - ./uploads:/opt/bundled/files/packages/files/dist/upload. I checked the recent docker-compose example and the path is now - ./uploads:/opt/server/packages/files/dist/uploads.
When I try to delete the file, I get the error Unable to Delete.
Maybe the file was temporarily written to the wrong directory, so the app thought it was successfully uploaded and wrote the record to the database? Another check to verify the uploaded files are actually uploaded would be useful to have, so the frontend interface can throw an error before uploading and losing more files.
I also understand it was my issue for not checking for updates to docker-compose before updating. I think it would be helpful to update this page - https://standardnotes.com/help/self-hosting/docker under "Updating your server:
" to note that localstack_bootstrap.sh and docker-compose.yml should be checked for updates before performing a pull.
Describe the issue
Hi,
The standardnotes documentation here lists self-hosting the client notes app as supported, but links to the ostensibly-defunct docker image standardnotes/web.
I can see in other forum discussion that the web docker image has been replaced by something else. Is it possible to self-host the replacement notes client app, or is the documentation on this page no longer applicable?
WARN exited: syncing-server (exit status 1; not expected)
INFO spawned: 'syncing-server' with pid 1664
INFO success: revisions entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
INFO success: syncing-server entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
WARN exited: revisions (exit status 1; not expected)
i get this error inside sync server container
I have followed the instructions exactly and the containers are all running but cant even get to port 3000 in web browser
==> logs/syncing-server.log <==
db:3306 is up. Proceeding to startup.
cache:6379 is up. Proceeding to startup.
==> logs/supervisord.log <==
2023-04-12 01:36:23,375 INFO success: revisions entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2023-04-12 01:36:23,376 INFO success: syncing-server entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2023-04-12 01:36:25,521 WARN exited: revisions (exit status 1; not expected)
2023-04-12 01:36:26,411 INFO spawned: 'revisions' with pid 4551
2023-04-12 01:36:26,426 WARN exited: syncing-server (exit status 1; not expected)
2023-04-12 01:36:26,430 INFO spawned: 'syncing-server' with pid 4554
i get the above when i run tail -f logs/*.log
on another note is there way to use DB URL so i can point it to another bare metal db instances instead of using mysql docker container?
It seems the /auth/login endpoint now always returns status 410 Please update your client application.
This is despite using the same API version (20200115) and no changes in the auth version (4).
The [https://docs.standardnotes.com/specification/auth] also does not indicate any changes.
The root cause seems to be the !performingCodeChallengedSignIn check in SignIn.ts which was added on Oct 3 in this commit.
The request only suceeds if the previous request to /params contains a code_challenge and the /login request contains a matching code_verifier.
(I guess the only reason why I only encountered the problem now, is that I reused the same session since october...)
The class InversifyExpressAuthController defines different sets of endpoints for querying params+login (/params + /login) and (/pkce_params + /pkce_login).
Only the second variant passes the required parameters through and are thus usable (if I see it correctly the /params, /login combination will always fail).
For whatever reason I don't seem to be able to call https://sync.standardnotes.org/auth/pkce_params (or .../pkce_login) from my application.
I always get the response "Cannot POST /auth/pkce_params".
The web-app does not use the routes as described in the documentation, but uses a completely different domain and API (https://api.standardnotes.com/v2/...), which then gets redirected to the real auth-server via the class ActionsControllerV2 (which then uses the "new" pkce_xxx endpoints)
I think the documentation at https://docs.standardnotes.com/specification/auth/ should be updated, because the currently described workflow does not work (at least I think so?).
And there should be some way to use the API without having to fall back to the (undocumented?) api.standardnotes.com endpoints.
Also it would be nice if such breaking changes were announced somewhere and implemented in a new api-version to ensure existing applications keep running.
PS:
It's pretty late down here, I hope there is at least some useful information in between my late night rambling :D
Log: https://paste.debian.net/hidden/c8acd1ba/
Was working fine. Not sure what changed. I tried rebooting the server but no dice. Any help?
Note: I've never tried adding the Files functionality.
Also, not sure if it matters, but I am a paying subscriber.
Running docker container list shows:
https://paste.debian.net/hidden/f4b2ea1
Would it be possible for you all to package the self-hosted standard notes on: https://marketplace.start9.com/marketplace
While trying to build the server images for ARM Architectures (like Apple M1, M2, but also ARM Servers which will probably play a big role in the future) the node dependency @sentry/profiling-node will run into issues.
This is due to the fact that the Package is still in experimental stage and does not support all common architectures.
I was able to build & run all parts by revering following changes:
packages/auth/bin/server.ts from commits:
packages/syncing-server/bin/server.ts from commits:
This will let you run the server and i couldn't find errors running it yet.
Is there a change the dependency will get removed again?
Thanks!
build_steps_syncing_server.txt
syncing_server_docker_logs.txt
No idea on how to debug this, so even pointing me to a direction to investgate this further would be helpful.
My auth service keeps restarting for some reason I can’t seem to be able to figure out:
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.