Adapters#
An adapter maps one protocol onto the model in Design. Adding an adapter doesn’t change the core. The adapters in this package each sit behind their own optional extra, so the core never imports a protocol’s client library, and an adapter can also live in another package. Each adapter’s section cites the specifications behind its rules, and Server compatibility records where tested servers depart from them. The Glossary defines the sync terms this page uses.
Why every adapter provides the same things#
The core never branches on protocol, so it asks every adapter for the same things:
a version per object, if the protocol has one; without it, every reported object is fetched and compared by fingerprint
changes since a sync token, if the protocol has them; without them, every sync is a full listing
the content’s media type, and its own fingerprint function and fingerprint scheme name if the exact text isn’t stable
each object’s containers and logical key, where the protocol has them, and whether that key is unique
change sets built from the protocol’s answers: incremental or a full listing, complete or one truncated page
each push’s outcome (applied, rejected, gone, refused for its logical key, or unknown), and which errors mean the scope must be reset
An adapter’s synchronous and asynchronous variants share all of its translation and parsing, and differ only in the client library calls they make.
CalDAV#
CalDAV keeps calendars on a web server, using the WebDAV extensions to HTTP. Each calendar is a calendar collection, a folder at its own web address. Each event or task in a calendar is a member: one iCalendar file, the standard text format for calendar data, at its own address, called its href. The server gives every member an ETag, a short string that changes when the member changes. To learn what changed, a client sends a sync-collection report with the sync token from its last report, and the server answers with the members changed or removed since then, plus a new token. A PROPFIND request lists every member with its ETag, and a calendar-multiget report fetches several members’ content in one request. The adapter sends these requests through python-caldav, a Python CalDAV client library.
Why a CalDAV scope is one calendar collection#
Each collection has its own sync token (RFC 6578), so one collection is one scope.
calendaring-sync |
CalDAV |
|---|---|
Scope |
A calendar collection |
Object id |
The path of the member’s href, always spelled one way, since python-caldav can spell one member’s address two ways ( |
Version |
|
Sync token |
|
Content and fingerprint |
The iCalendar data ( |
Containers |
The collection the member is in |
Logical key |
The |
Why the adapter checks supported reports first#
Before a collection’s first sync, the adapter reads DAV:supported-report-set (RFC 3253 Section 3.1.5), the list of reports the collection supports, so it knows whether sync-collection works before any report fails. A collection without it gets a full listing every time, and the adapter fetches only changed members. So does a collection that names no DAV:sync-token, which one that supports the report must (RFC 6578 Section 4), since a report from an empty token would ask for every member.
Why a full listing reads the token first#
A full listing reads the collection’s DAV:sync-token property, which holds the token a report would return at that moment (RFC 6578 Section 4), and then lists each member and its ETag with a PROPFIND (RFC 4918 Section 9.1) of Depth: 1, which covers the collection and the members directly inside it (RFC 4918 Section 10.2). Members whose ETag matches the stored one are reported unchanged, and the rest are fetched. Anything that changes during the listing appears in the next report from that token.
The adapter never lists with a report sent with an empty token, which asks for every member. A server may truncate any report (RFC 6578 Section 3.6), and a tested server truncates an empty-token report at a checkpoint that skips the remaining members, so a listing would infer that objects were removed when they weren’t. A server can cap a PROPFIND too: a tested server lists at most its configured number of results and marks the cut with a 507 for the collection. That cut is visible and never comes with a token that skips members, but a listing cut short still can’t tell a missing member from a removed one. So when a PROPFIND comes back with a 507, the adapter raises an error instead of ingesting it, and the server’s limit has to be raised. That leaves a known gap: a calendar larger than the limit can’t be synced from such a server. Listing with calendar-query split into time ranges, each under the limit, could close it later.
Why changes come from python-caldav’s sync report#
Changes since a token come from python-caldav’s get_objects_by_sync_token(). Up to 3.6.0, python-caldav sends sync-collection with Depth: 1, although the report is only defined for Depth: 0 (RFC 6578 Section 3.2); every tested server answers it all the same, and later releases send Depth: 0. When a server truncates a report with a 507 status and a checkpoint token (RFC 6578 Section 3.6), python-caldav follows the checkpoints, up to 100 reports, and the adapter hands the core the whole result as one change set. If the result is still cut after that, the change set is a truncated page that ends at the last checkpoint, nothing is pushed, and the next sync continues from there. The adapter turns python-caldav’s fallback off: it would answer a failed report with a full listing under a client-side fake- token.
A removed member has a 404 status and no DAV:propstat, the element that would carry its properties (RFC 6578 Section 3.2), and python-caldav lists it in the result’s deleted_urls, which the adapter reads removals from. Some servers report a member deleted and created again at one href only as removed, so the adapter fetches removed members too, in multigets of their own: one that comes back with data is reported changed, the rest removed. A changed member whose reported ETag is the one already stored hasn’t changed since the adapter fetched it, so it isn’t fetched again. A server can leave the new token out of its answer; the adapter then keeps the scope’s token, so nothing in that answer is lost, and the next sync asks again from the same point.
The adapter never sends DAV:limit (RFC 6578 Section 3.7), which asks the server to return at most a given number of results: a server can apply it by dropping the remaining changes and returning its newest token.
Why only an invalid token resets the scope#
A server that won’t accept a token answers with the DAV:valid-sync-token precondition, an error element saying the token isn’t valid (RFC 6578 Section 3.2), and the client falls back to a full sync, without downloading unchanged cached resources. RFC 4791 Section 1.3 specifies 403 or 409 for a failed precondition, and a tested server sends 412.
The adapter resets the scope only when the error names DAV:valid-sync-token, whatever its status. Any other failure, such as a refused login or a server that’s down, is raised, so a flaky server doesn’t cause a full listing on every error. The adapter only sends tokens the server issued, so a 500 means the server failed, not that the token was malformed. python-caldav keeps the precondition on every error, so the adapter reads it from the error’s preconditions, whatever the status.
A tested server answers a token it no longer accepts with a report that leaves out older changes and raises no error. The adapter can’t tell that from an ordinary report, so it says whether a scope needs regular full listings, from python-caldav’s sync-token.invalid-token feature. Your application sets that feature by naming the server or by probing it with caldav-server-tester, so a new server or an upgrade needs no list of names here. How often to run them is up to your application; a full listing only fetches members whose ETag changed, so a frequent one is cheap.
Why content comes from a multiget, with its own ETag#
Changed members are fetched with calendar-multiget (RFC 4791 Section 7.9), asking for DAV:getetag and calendar-data together, with no Depth header as that section asks. The stored version is the ETag from this response, not from the sync report: an object can change in between (RFC 6578 Section 3.1), and an older ETag would cause a needless fetch on the next sync and a wrong base for the next write.
A member that comes back without data, as python-caldav returns a 404 entry, or is missing from the response although RFC 4791 Section 7.9 requires an entry for every href, was removed after the report. It’s left out of the change set, and the next report carries its removal. A server can also answer the whole calendar-multiget with 404 when one member is gone; the adapter then checks that the calendar is still there and fetches each member on its own. A member that comes back with data but no ETag is kept with no version, so the next listing fetches it again.
Why the fingerprint leaves out three properties#
Tested servers rewrite what they store: they add DTSTAMP, reorder properties and parameters, fold lines at other points, and reorder the parts of a recurrence rule, which may come in any order (RFC 5545 Section 3.3.10). A server sets DTSTAMP and LAST-MODIFIED itself (RFC 5545 Section 3.8.7.2, RFC 5545 Section 3.8.7.3), and PRODID names the product that created the object (RFC 5545 Section 3.7.3). Leaving those three out and sorting the rest makes a server’s copy of your own write fingerprint the same as what you sent, on every tested server. UID is kept, since it’s the logical key, and so is everything else, because any other difference can be a real edit. Calendar data is UTF-8 unless a charset says otherwise (RFC 5545 Section 3.1.4): python-caldav hands the adapter what a server sent as UTF-8, and the adapter refuses an edit that isn’t.
Why writes are conditional#
Updates and deletes send If-Match with the base ETag (RFC 9110 Section 13.1.1), so the server applies them only if the member still has that ETag, and creates send If-None-Match: * (RFC 9110 Section 13.1.2), so a create never replaces an existing member. A mismatch gets status 412, and the edit stays pending for the next ingest. If-Match compares ETags strongly, so a weak one (W/) never matches; an edit to a member whose stored ETag is weak waits, without being sent, until a sync stores a strong one, and the adapter’s waiting() lists such edits. The ETag comes from the DAV:getetag property of the server’s answer, not from an HTTP header a proxy can weaken, so edits wait only on a server that reports a weak ETag itself.
Creates and updates use python-caldav’s save() with raw_data, which sends the bytes you recorded unchanged, and precondition="etag", which sends the stored ETag. Updates also pass increase_seqno=False, because save() otherwise increments the event’s SEQUENCE revision number and the server wouldn’t receive the edit you recorded. Creates pass no_overwrite=True, which sends If-None-Match: *. Deletes use delete(if_match=True, precondition="etag"). A 404 on an update or a delete means the member is gone; a tested server answers a delete of a gone member with 412 instead, which leaves the delete pending until the next ingest reports the removal.
A create uses an href of unreserved characters only (RFC 3986 Section 2.3), a UUID and .ics, so it can’t come back spelled differently. A server that already holds the UID answers with an error element, CALDAV:no-uid-conflict (RFC 4791 Section 5.3.2.1) or CALDAV:unique-scheduling-object-resource (RFC 6638 Section 3.2.4.1); python-caldav raises UidConflictError for either, whatever the status, so the adapter never mistakes it for a plain precondition failure. For a create or an update, that’s a conflict with the object that holds the UID, named when the error element holds its DAV:href and that href is in the calendar; otherwise the core finds the holder by its UID.
Unless the client is configured with the server’s profile, python-caldav searches the calendar for the UID of a create before sending it, which costs a request per create and refuses a UID that’s taken even on a server that would accept it. Configuring the client’s features with the server’s name lets the server check instead.
A server MUST NOT return a strong ETag for data it stored differently (RFC 4791 Section 5.3.4), but tested servers do, so an applied push doesn’t keep the ETag it got back.
Why the adapter needs python-caldav 3.6.0 or later#
The adapter needs python-caldav 3.6.0 or later. It relies on following truncated sync reports, listing removed members in deleted_urls, keeping preconditions on every error, UidConflictError, honoring increase_seqno on every save, raw_data holding calendar data as bytes, save(raw_data=...) sending bytes unchanged, precondition="etag", and If-None-Match: * on save(no_overwrite=True).
JMAP#
JMAP is a protocol for reading and changing data on a server with JSON requests over HTTP. A server holds one or more accounts, and each account holds records of several types, such as calendars, events, and tasks. Every record has an id. For each record type in an account, the server keeps a state string that changes whenever any record of that type changes. A client calls methods on a record type, written as the type’s name and the method, such as CalendarEvent/get:
/changesreturns the ids created, updated, or destroyed since a state you give it, and the new state/getfetches records by id/querylists the ids that match a filter/setcreates, updates, and destroys records
Events are JSCalendar objects, a JSON format for calendar data. The adapter makes these calls through calendaring-jmap, a Python JMAP client library.
Why a JMAP scope is one account and one record type#
A record type is the kind of data a JMAP record holds, such as CalendarEvent for events or Task for tasks. A state string covers all records of one type in one account (RFC 8620 Section 5.1), so CalendarEvent and Task in one account are two scopes.
calendaring-sync |
JMAP |
|---|---|
Scope |
One account and one record type |
Object id |
The record’s |
Version |
None: JMAP has no per-record version |
Sync token |
The type’s |
Content and fingerprint |
The JSCalendar JSON, encoded as UTF-8, and SHA-256 of it serialized with sorted keys, without the properties the server sets, and with |
Containers |
The event’s |
Logical key |
|
Content stays JSCalendar, since converting to iCalendar would lose data on each round trip; converting is left to your application.
With no per-record version, every object /changes reports is fetched and compared by fingerprint, and a full listing fetches everything.
Why the fingerprint leaves out what the server sets#
The server sets id, baseEventId and isOrigin, and works out utcStart and utcEnd when asked (draft-ietf-jmap-calendars-32 section 5). It also sets created, updated and sequence itself (draft-ietf-jmap-calendars-32 section 5.9), and prodId names the product that last changed the event (RFC 8984 Section 4.1.4). The fingerprint leaves all of those out and fills in isDraft, useDefaultAlerts and version with their defaults, because a server MUST return isDraft and useDefaultAlerts even when you never sent them (draft-ietf-jmap-calendars-32 section 5.7), and a tested server leaves out version. A server’s copy of your own write then fingerprints the same as what you sent, on both tested servers. The fingerprint is always taken from a full /get, since a /set answer only carries some properties.
Why some errors reset the scope#
On cannotCalculateChanges, the error a server returns when it can’t work out the changes since the state given, the client MUST invalidate its cache for the type (RFC 8620 Section 5.2), so the adapter resets the scope. It also resets on two answers seen on a tested server: an oldState that doesn’t repeat the sinceState the adapter sent, and invalidArguments from /changes. The adapter only sends states the server issued and a valid maxChanges, so the state is the only argument that can be wrong.
Why pages follow the new state#
When a /changes answer leaves out some changes, it sets hasMoreChanges to true, and the client may call /changes again with the newState it received (RFC 8620 Section 5.2). Each answer is a truncated page that advances the scope to its newState.
Objects are fetched with /get after /changes, so one can be newer than the newState it came with. The next /changes reports it again, and the matching fingerprint makes that a no-op.
A full listing reads the type’s state first, then pages through /query and /get. That state becomes the token, so anything changed during the listing appears in the next /changes.
Why every create carries its own UID#
A server generates a uid only when a create omits it (draft-ietf-jmap-calendars-32 section 5.9). The adapter always sends one, so a create whose result was lost is recognized when it comes back under a server-assigned id.
Why writes use ifInState#
A /set with ifInState, the state the client expects, fails with stateMismatch if the type’s state has moved (RFC 8620 Section 5.3). That state covers the whole account, so a rejection means something changed, not necessarily this object. The edits stay pending for the next ingest.
Why events come before tasks#
Tasks fit the model as their own scope, but no tested JMAP server advertises JMAP Tasks (urn:ietf:params:jmap:tasks). The JMAP adapter starts with CalendarEvent and adds tasks once a server supports them.
Why the adapter needs JSCalendar writes#
The adapter pushes the JSCalendar you recorded unchanged, so it needs a client call that writes JSCalendar without converting it. Guarded writes need ifInState on those calls, and paging needs each /changes page’s newState. Without ifInState, a push can overwrite a change made since the last sync. Without paging, a truncated /changes answer resets the scope instead, which is correct but downloads everything again.