Wrong "Server doesn't CardDAV well-known URI or DAV: current-user-principal" message about flock

[rhodey]

Hey @rfc2822 first I gotta say I love your DavDroid project :)

Unfortunately right now Flock is doing pretty poorly when it comes to support for non Darwin Calendar Server WebDAV server support, see issues #19, #21, #22, #23. I plan to dedicate a whole day to improving our WebDAV server support and closing these issues but right now I've got to de-prioritize these behind things like #45 and #13

Sorry to all for the lack of progress on the above mentioned issues, they are definitely on my mind and I'll get to them as soon as possible.

[grinapo]

I'm happy to see that #45 and #13 was fixed, maybe we should expect some progress on this (these)? ;-)

[rhodey]

I just created branch issue-42 and pushed commit 8a358a2. This commit includes patches that make Flock less picky about http status response codes and also reduces some code duplication. With these patches Radicale server fails only one test...

When executing a PROPFIND request on a non-existent WebDAV collection Radicale pretends that a WebDAV collection is actually present and returns a multi-status response with default properties (displayname set to uri, default color, etc). Creating a new calendar, updating the displayname property, and then deleting the calendar does seem to kinda delete the calendar, as in all of the properties that have been set are reset. However executing a PROPFINDafter the successful DELETE still returns the multi-status response of a default calendar.

This is a problem for Flock because a big part of determining state is checking whether or not the key collection exists and Radicale cannot be counted on for this. The key collection is a CalDAV Calendar with some special properties and resources to allow, well, all the encryption stuff.

I'll continue testing Radicale and see if there are any other client-side patches that need to be made and then merge whatever I come up with into master, but I'm not yet sure that I want to include a workaround for this (IMO) Radicale server bug.

[patcon]

@rhodey Sorry, why "wontfix" tag? Can you recommend a set of webdav servers that flock supports, if so? Thanks! :)

EDIT: I'll assume this server is working for now: http://calendarserver.org/

[patcon]

Ack. even darwin calendar server doesn't seem to follow spec in 12.04, and 14.04 seems broken somehow...

Any pointers on how you guys have configured your server would be greatly appreciated. It really seems that it's not as simple as any webdav server can be used for self-hosting. I'll keep trying other server tools though

[patcon]

Main questions: What operating system? Standard packages? Using postgres backend, I assume? What version of postgres? Thanks!

[patcon]

OK, so using radicale, seems to be getting me the furthest along. With the recent stock flock apk (0.9.0), I'm getting this error during the tests:

"Server does not support creating and deleting calendar collections"

(I'm using the default 0.8 of radicale, that's in the ubuntu 14.04 repo.)

Here are the debug logs (and config): https://gist.github.com/91c247e5c9f093978045

The "207 Unknown" response seems related. A /patcon/delete-me.prop file gets created and left sitting around, fwiw. Seems like MKCOL and DELETE are working fine (201 and 200 responses), but perhaps the the FINDPROP's 207 Unknown responses are confusing the flock tests?

I also tested with the issue-42 branch (not merged into master) and got stuck at the same error.

[patcon]

For whoever it may concern, I tried SabreDAV as well, and no luck. (Sorry, no logs on that one, but wanted to leave a comment for the next person to search.)

[patcon]

In case anyone is interested, rhodey shared the cookbook for the production webdav system. This is pure copy paste at the moment, with lots of caveats, but I plan to work on it later this week:
https://github.com/patcon/chef-darwin-calendar-server

[marciomr]

I tried with owncloud and it stops in DAV:current-user-principal and with radicale it stops in DAV:create and delete collection. Is this cookbook for Mac? I would like to serve in Linux.

[marciomr]

This worked for me: https://wiki.debian.org/HowTo/CalendarServer

[patcon]

I'll be working on this today, but @rhodey mentioned that the SVN release on the prod server is this one:
https://trac.calendarserver.org/changeset/11995

This corresponds to this git commit if you'd prefer a repo that went through the git-svn treeatment:
https://github.com/trevor/calendarserver/tree/581a2cfd30ac52f4d1afdc8cd0296fe7e23c753f

Also, the prod server is Ubuntu 12.04, using postgres backend and the LDAP directory service.

Here's a gist of the raw diff directly from the appropriate commit (it's not all relevant). It's the stuff from the darwin-patch/ directory that you'll see in the chef cookbook linked above.
https://gist.github.com/patcon/dfd1542f09d694bb76b1

[marciomr]

Thank you @rhodey. This will make things much easier.

[grinapo]

Got a reply on the referenced radicale issue report, but I'm not involved deeply enough in carddav to take part in a debate. 😃

[patcon]

@grinapo Cool! Can you share the link or message text?

[grinapo]

@patcon github was pretty smart and automagically referenced the issue in question above my comment, but here you are for simplicity: Kozea/Radicale#240
By the way the radicale people were very constructive and promised us a switch to turn autocerate off. 👍

[patcon]

Unrelated to you post @grinapo, but: wahoo! Got a stripped down test server running the darwin-calendar-server cookbook! Will get the last bits in code and push the vagrant box today or tomorrow. Still need to get the postgres directory service working and maybe get S3 photos working too, but it's enough to get working on the encryption proxy

[patcon]

Haven't tested extensively, but this should be working if anyone wants to get a simple webdav VM up and running:
https://github.com/patcon/flock-webdav-vagrant

[patcon]

OK, so it doesn't totally reflect the production server, but it works as a test server. Follow the instructions at https://github.com/patcon/flock-webdav-vagrant#readme

It should be as simple as vagrant up after the initial install of the pre-reqs for Vagrant.

(It will automatically expose an internet-accessible endpoint via ngrok.)

I might update it later to get the postgres server working with accounts, and making optional S3 profile image support, but for now, going to leave it as-is and start working on the encryption proxy :)

[patcon]

FYI starting to experiment with approaches for encryption proxy here: https://github.com/patcon/flock-encryption-proxy

Not much to look at for the moment :)

[rhodey]

Awesome!

I have some sync protocol documentation to push to this repo, I'll be flushing that out and sending that upstream this morning.

[patcon]

A little distracted with the jobhunt in Toronto, but I'm aiming to get a PoC workable before I get back to the grind :)

[monolithonadmin]

@marciomr
Thank you for debian calendarserver tip, i will test it. Can you recommend an easy option for contacts? (carddav)

Or what's the easiest way to use flock with self hosted backup? Which cms/calender etc. is using sabredav 2.x today?

[patcon]

@Muvuk This will spin up a local virtual machine server for you and make a web-accessible endpoint available at xxxxx.ngrok.com:
https://github.com/patcon/flock-webdav-vagrant#readme

You can then point the flock app at this endpoint.

I created that for box for testing, so it's only living on your computer when you have the VM running. But you could use the same Chef provisioning scripts (with a vagrant cloud provider plugin, like digital ocean) to spin up a cloud VM. That will require some manual tweaking to get it deploying, but the actual webdav software should be pre-configured to work.

EDIT: You'll want to skim through and change hardcoded passwords that are definitely NOT suitable for production. But honestly, I wouldn't self-host unless you know what you're doing. Whisper Systems service is cheap, and they never see any of your sensitive data in unencrypted form (that's the whole point of Flock's design). The value-add of self-hosting is negligible against the risk of managing it yourself.

[monolithonadmin]

Okay, do you have any recommendation for arm platform (raspberry pi)? If i am right, chefdk is x86_64 only.

[patcon]

Hm. If you can ssh into your pi, then I would investigate this vagrant plugin:
https://github.com/tknerr/vagrant-managed-servers

Not sure if all the other packages are available though. Good luck! Ping me on Twitter or irc if you hit and snags

[monolithonadmin]

I tried https://github.com/patcon/flock-webdav-vagrant on koding.com vm, but i got this error:

An error occurred while installing json (1.8.2), and Bundl
er cannot continue.
Make sure that gem install json -v '1.8.2' succeeds befo
re bundling.

i cheked version:
gem -v
1.8.23

I will try on my rpi this:
https://github.com/tknerr/vagrant-managed-servers

I hope it will be ok.

[patcon]

@Muvuk definitely open up an issue in that repo's issue queue and I'm happy to help, but this issue isn't the place for us to troubleshoot :)