Hi! I’m trying to import historical GPX or JSON files (2014-2021) from an old Arc App > Export folder into the latest version’s Arc App > Import folder. It doesn’t seem to work. Could you point me in the right direction, please? Thanks a lot.
Hi @plantemathieu!
For importing GPX files, that’s currently done from the Import menu, from the more/ellipsis menu on Timeline view.
Importing complex files with potentially multiple different kinds of data in them requires more UI, allowing you to decide which parts of the file to import, and how to import those parts.
The Import folder isn’t used in this case, due to importing from the Import folder being a one-step process of selecting the file and then importing it as a whole. Which can often result in large messes that need cleaning up afterwards.
Importing of JSON files unfortunately isn’t supported yet. It periodically comes up, but there’s not enough enthusiasm (votes) for it to push it up the priorities so that I can justify working on it sooner rather than later.
Thanks for that. I imported about 6 months of data last night via the Import GPX function. The earliest months (November 2013-April 2014) appear fine in daily view, but don’t have any data bubbles around the dates in calendar view. Other months (May-June 2014) do have data bubbles, and look like any other more recent months. Did you change the way data appears around these dates? How do I know if the importer is still analyzing data? File Importer doesn’t show anything.
Mathieu Plante
plante.mathieu@gmail.com
www.assemblagerequis.com
In the calendar view there’s four different colour states for days:
- Light grey number: before the earliest data, so can’t be viewed / nothing to view
- Black number: Normal day with data, that can be viewed, and requires no cleanup
- Black number within grey circle: Normal day with data, that can be viewed, and has some data that needs cleaning up
- White number within orange circle: Favourited day with data
It might be that you’re seeing the grey circle, which means that there’s at least one item in that day that’s marked as “needs confirmation”, due to the classifier not having high enough confidence in its decision.
I forget what the GPX view shows exactly, but once you’ve tapped import, it’ll finish up in a second or two, and all be done. After that the processing engine might do a little bit of extra work once you navigate to that day in Timeline view, perhaps cleaning up around the edges, refining the start/end times. But that should be over in a second or two.
The 6 months I imported all have black numbers and no bubbles, both those that have data and those that don’t. Some days in 2024 have gray bubbles even though they’ve been cleaned up, so maybe it’s ok. At least the import seems to have worked.
If you end up reworking the import system, would it override the data that has already been imported? I want to import the rest of my data now, but don’t want to end up having duplicates in the future.
The GPX importer will warn you that the selected import data overlaps existing data. It’ll show a little indicator on the row in the list. Then you can tap into that row and see on the details view map the existing data and GPX data.
If there’s overlapping data it’ll for sure slow down the import process, because you’ll need to keep an eye out for it and can’t just “select all”. But yeah, at least the app will warn you and let you choose what to do.
Oh, and if you import over the top of existing data, the existing data will be kept but disabled. Then if the imported data is deleted from one of the Edit views, the previous data will be reenabled.
Indicator in list
Details view for inspecting the overlap
You thought of everything, you’re a genius, haha!
Almost everything is working, except that a few months I’m trying to import just completely crash Arc. Most of these files are over 2 MB and contain about 1500+ items each. Is there a way to cut them in half or separate them into months?
Haha, I wish! There’s always something I’ve missed, no matter how carefully I try to plan.
Hmm. When does it crash? Does it crash when trying to first open the GPX file? Or when trying to import the GPX segments? If it’s the latter, then you can try only selecting a few of them at a time, and importing in stages.
If it’s the former, then… we’re going to have to get creative. I tested on a bunch of very large GPX files when building the importer, but it’s possible you’ve got some files even bigger than my test files!
It’s kind of a mix of both. 11 .gpx files out of 88 make the app crash. It happens as soon as I choose Select All in the importer. A few months have a few hundred items, which would be manageable to check by hand, but others have thousands of items, and some are greyed out and can’t even be checked.
For the ones that are greyed out, those will be ones that either haven’t finished loading yet, or that can’t be imported due to missing information.
If it crashes when you tap Select All then I’m fairly certain it’ll be a memory problem. It will be trying to load too many items into memory, and then iOS terminates the app due to using too much memory.
So unfortunately yeah, the only way around it will be avoid the Select All button, and to import in batches. Which is going to be a bit tedious!
Actually, if you could send me one of the larger files (matt@bigpaua.com) that causes crashes on Select All, I could experiment with it here and see if I can find a way to get around the memory problem, and get a fix into the next Arc update. Worth a try!