JSON Import Format

Some pretty simple precautions:

  • Make sure your ARC backup is up-to-date. You can force a backup by disabling backups, waiting a little while (there’s a thread somewhere with exact of time you need to wait) then re-enabling backups. That force an immediate foreground backup to happen
  • Makes sure you backup is outside of iCloud, so don’t accidentally overwrite while importing
  • Keep a separate backup of the data you wanna merge. The merging process can get fiddly, you don’t want to accidentally overwrite the data you’re merging during the processing process
  • Make sure you give iCloud enough to sync. It can take a while for everything to sync up, and it easy to make a mess by trying change stuff before iCloud has got all your devices in sync.

Beyond that, the importing works quite well. It can take a little while to complete, and transferring many small files via iCloud is slow. You really need to give it time.

Also the process of downloading the files onto your phone is a little unreliable. Arc asks iOS to download everything, then waits for confirmation. With many small files, iOS frequently forgets to actually download files, so the process gets stuck. Force quitting Arc, and re-starting the import is needed. After Arc requests a file to be downloaded enough times, iOS eventually sorts itself out. Thankfully killing an import and restarting is completely safe, you’ll probably need to do that a couple of time. You can also help this process along by manually downloading the files using the Files app.

Finally make sure that to force a new backup once you’ve got all your data imported and you’ve checked it! You don’t want to do this process more than once, it can take a while for Arc to chug through all the data.

2 Likes

All excellent advice @AvianLyric! Thanks!

The grace period is 30 seconds. It waits 30 seconds to see if the user turns backups back on or not. Then if backups are still turned off, everything in the database is marked as not backed up, and it starts again from scratch. Oh and before starting a fresh backup it will rename the existing backup to “Previous Backups [some_random_text]”, to preserve it.

Important advice! Arc does work on copies when doing a managed restore, and doesn’t delete anything if doing manual imports, but you never want to trust your data solely to the brains of a computer - they’re untrustworthy beasts. Always work on a copy, not the original :+1:t3:

iOS 16 has a long overdue new feature in the Files app - you can now long press on a folder (in this case the Backups or Restore folder, for example) and choose “Download Now”. That will queue up and download the entire contents of the folder, and do it seemingly much faster and more reliably than when Arc asks iOS to do the same. So that trick helps to avoid a lot of wasted time, waiting for iOS to pay attention to Arc’s requests to download individual files.

Have you managed to tweak this yet?

Looking for this to bring in a years worth of data that I’ve been waiting to import for years :smiley:

I haven’t, honestly wasn’t sure there was anyone who wanted to use it.

I would say that importing years worth of GPX data might not be a good idea. It takes quite a bit of time to clean up after the import, even for a small amount of data.

If you’re still interested I’ll dig it out and have another look at it. But be prepared, you’ll need some technical chops to figure it all out, and I won’t be able to provide much support.

1 Like

For built-in importing of Arc’s ArcJSON export format, that’s very high on my todos.

I probably won’t get it into the next update. I want to get Dark Mode and a couple of other long overdue smaller features shipped, and ArcJSON import is quite fiddly and time consuming, so it’d hold up the release. But it’s definitely coming.

I haven’t had a chance to get around to it but indeed I have four or five years of old data to import. It seemed like the way with the current script is to run it on each of the monthly JSON exports and then import one at a time?

I can wait if you think sometime you might have a chance to improve it. Or if Matt might get around to import in a few months, perhaps that would be a better time.

It’s been a long time since I last looked at this. But I’m pretty sure the script will let you do a large batch conversion. No need to convert files one at a time. I personally won’t be making any improves to this, I really don’t have time, and I suspect that there wont be any changes to Arc import feature anytime soon either.

So TL;DR, this is script is probably as good as it’s gonna get. So you should just try it and see what happens.

Having said all that, I’m happy to provide some guidance if you have specific questions or issues. But im not gonna do any handholding im afraid.

Yeah unfortunately in the feature request polls there’s little to no enthusiasm for ArcJSON import :disappointed:

It’s one of those things that really should already be there, but if there’s little to no interest from users, it’s hard to justify the time on it.