I used to use Arc back in 2018, and stopped using the app due to battery usage. I recently reinstalled Arc, and it detected and started restoring data from 2018, I could see places etc on the map, but after the restore completed all of the data from 2018 vanished. Any idea how I can get this back?
Hm. That’s odd. I don’t know how that could happen. Arc basically doesn’t delete timeline data ever (with exception of excess sleep mode samples recorded during long stationary periods). Possibly some weird item merging went on, and there’s a mega item somewhere, with a large amount of data merged into it?
I’m wondering if there’s something in the timeline view code that prevents to from skipping large sections of time, or make it very slow to seek backwards?
The UI will let me tap the back button to go to date before my most recent install period. But then I get a spinner that never stops. Additionally the cal view will momentarily show dates before my current install period, then they’ll vanish after a second or two, and finally the data does still seem to so of be in the more recent backups. There’s Locomotion Samples from 2018, but there do seem to be timeline items.
Ah, yeah, that sounds like there’s some stuff up with the processing, and there’s timeline items getting merged into a mega item. The slow / stalled UI and spinners will indicate that a massive amount of data is trying to be loaded.
If you can get into the timeline view for a day where there’s one of the mega items (eg a visit with an absurdly long duration), and can then get into the details view and Individual Segments view for that item, you’ll be able to start cleaning up the mess from there, by splitting out the pieces into sensible separate timeline items.
Though that’ll be quite a long struggle. Another option is to wait for the old backup system to finish restoring, then make sure the new backup system has finished backing up. Then you’ll have all your LocomotionSample data safe in the new backup system, in visible JSON files. You could then do a fresh install, but with the TimelineItem JSON files for the broken timeline period all deleted.
When restoring from backup, the only vitally important ingredient is the LocomotionSamples. The rest can be rebuilt sensibly by the processing engine (albeit without your manual corrections, such as place assignments). Samples will be imported as orphans, with no parent TimelineItem. Then the processing engine creates new TimelineItems for the samples, and reprocesses those new items the same as newly recorded data is. So basically it rebuilds the timelines, based on the underlying sample data. It’s a slow process, but a reliable one, and possibly less troublesome than trying to do cleanup on the mystery mess / mega items that are in your database now.
Let me know any questions you have! The most important detail though is to make sure you’ve got all the underlying LocomotionSample data restored from the old backup system, and then finished being backed up into the new backup system (ie the JSON files in iCloud Drive/Arc App/Backups). Once you’ve got those (and copies of them, for extra safety!) your data is safe, and can be manipulated and restored more tidily.
Unfortunately I haven’t been able to do this, the app never manages to complete its work before the phone sleeps, or I need to do something else.
Ah, that’s a bit concerning. Would I expect the 2018 data to appear in a 2018 dated locomotion samples files?
The old backup has finished restoring, and a new backup created. I started the restore process on the 1st, and it took 3 or days to complete.
The earliest dated locomotion file in the new backup is 2021-W52.json.gz. But I’ve the noticed the 2022-W01.json.gz (which is the week the restore would have completed), is 2MB large. So an order of magnitude larger than the other locomotion file. Will this file contain all my old data then?
Could you expand on this a little more? Is there any way I can split up this mega item by manipulating the Timeline items in the backup? The data was recorded along time ago, I’m not sure I remember enough to redo the corrections accurately.
Hm. That’s a worry. The W is the week-of-year, so W52 is the last week of the year. Each file contains all the samples dated within that week. You should have LocomotionSample files for 2018.
You could, though it would be essentially the same as what you’d achieve doing it in the UI after restore, but more difficult.
A mega item is what happens when somehow multiple timeline items get merged together, so instead of being “home → train → work → train → home” it all gets merged into one big “home” visit, for example. If that manages to happen, then other adjacent items will get pulled in to the mega item due to falling within its radius, and the problem compounds.
It only very rarely happens in normal operation, but has been known to happen from dodgy restores from the old backup system.
Looking through the locomotion samples it seems everything from 2018 is gone 
Is there anyway to retrieve the original backup data and have another shot at restoring it?
Not sure if this is helpful, but digging through the Arc logs, it seems the restore process for my old data stopped just after I installed ArcMini. I can see FINISHED_SLOW_RESTORING_SAMPLES log lines until the time I install ArcMini, and Arc hands over recording. After that there’s no more FINISHED_SLOW_RESTORING_SAMPLES lines, but a set of new logs files with crashes in them, followed by more normal logs, but zero additional restore lines.
Is it possible that installing ArcMini interrupted the restore process and prevented the locomotion samples from being committed?
Unlikely at this stage 
Once Arc has completed a full backup with the new backup system, it deletes all data from the old backup system on iCloud CloudKit. So by now it will be gone.
Curious. It shouldn’t be possible. Arc Mini doesn’t yet have any knowledge of or involvement in the backups system. But maybe there’s some interaction I’m not thinking of…
How are you seeing those log lines? Those would be in Arc App’s logs, yeah? Not Arc Mini?
Yeah I arrived at the same conclusion last night after trying a load of reinstalls and restores in the hope of getting Arc to pull data out of CloudKit. I still have the GPX exports, but I’ll need to come up with a way of converting them into ArcJSON to import them, got some ideas on how I can do that.
I’ve now seen the behaviour happen twice. My partner also reinstalled the app, it originally showed timeline data from 2018 and gave a backup restore percentage. But after installing Arc Mini, all the 2018 data being restored vanished, and the backup message disappeared. Is it possible that Arc Mini is deleting files out of the App Group space that Arc needs for the old CloudKit restore process? Or possibly updating the Arc app DB so that Arc thinks that there is no CloudKit data to restore anymore?
Can’t say too much about how I’m seeing them unfortunately. But yeah, I pulled them out of the Arc’s (not ArcMini’s) Library/ApplicationSupport/Logs directory in the application container. I’ve now ended up wiping those logs unfortunately, after a doing a bunch or reinstalls and restores, I forgot to take a copy before hand.