Arc Editor plans for automatic export / backup?

Arc App Support
1

(I am aware I am asking questions about a beta app…)

I have built an application (just for myself) that reads the automatic Arc exports (daily/monthly gzipped JSON) and stores it in a local database. As the new Arc Editor is getting better and better, I am making preparations to import data from the new Arc app.

I see there’s documentation on the Locokit2 based export: LocoKit2/docs/export at main · sobri909/LocoKit2 · GitHub

Nice!

The documentation talks about two types (single file and bucketed), compression, incremental, etc.

The current functionality is (from what I can see): full export only (timestamped), bucketed, uncompressed.

I understand not everything is built yet, but I would be very interested in what is planned. E.g. the plans for automatic, incremental bucketed / single file, compressed.

For a Place, the documentation mentions secondsFromGMT. Is that not linked to a date (i.e. with/without daylight savings)? So, I would expect a timezone name, not an offset number.

For a Timeline item where uncertainActivityType: false, I don’t always see classifiedActivityType?

    "trip" : {
      "lastSaved" : 783809402,
      "speed" : 0.9819105654208016,
      "itemId" : "123A8996-9737-42F9-9832-1AEEB3CBD314",
      "distance" : 153.3498826248254,
      "uncertainActivityType" : true
    }

vs an unconfirmed one that does have one:

    "trip" : {
      "speed" : 2.565635193438946,
      "uncertainActivityType" : true,
      "itemId" : "2CFB0DD3-DF58-4C5B-854B-E178A38AB2F5",
      "classifiedActivityType" : 4,
      "lastSaved" : 784469164,
      "distance" : 3163.869482893171
    }
2

Yep, currently I’ve only implemented the full export variety. Because that’s what I’ve needed during testing, doing full exports, delete app, reinstall app (because of database schema change or some such), then reimport everything again.

But daily incremental backups are almost top of my todos list at the moment! So those will be coming in a build soon. (Though the next build is going to be about improvements to daily/weekly/monthly/yearly timeline views, to clean up the loose edges there).

To know daylight savings status you’d need a date, but Places aren’t date specific. For regions with daylight savings time you’d need to store two timezones in the Place. But then you’d also need to know which timezone to use, which would require knowing not just the two timezones but also their applicable date ranges.

Also timezone name often isn’t available, for example if doing a reverse geocode lookup for lat,lon coordinates. In those cases the geocode server will only give you the offset from UTC, not the specific timezone name. So you end up with something that’s no more informationally useful than secondsFromGMT, but is more difficult to parse and use in code. That leaves you with secondsFromGMT being the most space and processing efficient way to store the available info.

This one confuses me every time I run into it. I’ll check what the constraints are…


table.column("uncertainActivityType", .boolean).notNull().defaults(to: true)

    .check { $0 == true || Column("classifiedActivityType") != nil || Column("confirmedActivityType") != nil }

    .check { $0 == false || Column("confirmedActivityType") == nil }

Ok so, even that’s confusing to read. But… it’s allowed to be true if there’s a classifiedActivityType OR not a confirmedActivityType.

Looks like the constraints leave it open to being either true or false if there’s no classifiedActivityType. In that case it doesn’t care what value it has. Arguably either value can make sense.

classifiedActivityType isn’t set until a classifier is run over the samples. Which typically should happen quite soon after sample creation, but not always. Ah and now I remember: not allowed to run the CoreML classifiers in the background. So that field will only get populated in the db when the app is in the foreground. Which means that there could be hours of samples accumulated with no value yet.