Hi Matt, I really do like the new interface, but it doesnt really work as I thought it would. I was kind of hoping that confirming it would clean up the underlying data, but I see that it only confirms the overall segment, but doesnt overwrite the underlying data to be the same data type.
this is post using the button, my impression was that the clearly wrong tram segments would have been made into car after using the button. As it is, I have no confirmed something as a car, but it still shows up as a tram when I click into it for a more detailed view. I now have to go back and hit “clean up” on all the confirmed segments, and there is also no longer an indication of which ones these are so I have to search through all segments
Would it be possible to have the new confirm interface also convert all the points to the same type of data (e.g. car), or have it as a toggle in a settings menu? As it is right now it is the same or maybe a bit worse than the previous behavior for how I clean the data
Also if it confirms two segments to be the same type (besides the underlying data not matching the overall type as I talked about above) they no longer merge if they are the same type
I had similar sentiments to you, but it ended up making sense to do it this way for a few reasons. To put things in context, I’ll list out the current various confirm/cleanup actions and what they do, from most aggressive to least aggressive. Apologies in advance - this is going to be one of my far-too-long replies!
The Confirm button on an item’s Edit view: This one will do what you’re wanting. It will convert all of the segments within the item to the chosen type, regardless of what other type they might currently be, and regardless of whether they’re been confirmed as that other type previously. It will clobber all previous type assignments. This is the most aggressive action, and works this way on the Edit view for any item, regardless of whether it was listed as Uncertain or Unconfirmed.
The Clean Up button from any of the various viewed it’s offered in: This will convert all samples in the item to the item’s most common type and confirm them, as long as they’re not already that type, and as long as they haven’t already been confirmed as a different type. So for example if the item has overall been classified as walking but there’s a smattering of segments of other types, only those other segments will be converted to and confirmed as walking, and only if they haven’t already been confirmed as that other type.
This allows for cleaning up for example a train item where you’ve already confirmed bits of walking on either end. Tapping Clean Up will convert any stray segments in the middle to train, while not messing with the walking bits you’d already confirmed on either end of the train trip.
The new Confirm All button on the new Confirm List view: This new action is the least aggressive of all. It will only confirm the samples that are already classified as the target type, and does nothing else.
This action is only taken when tapping the Confirm All button on the new Confirm List view. If you tap on the item in the Confirm List view to get to its Edit view, then the Confirm button there will do the first action (ie the most aggressive one) instead.
Ok so now to the why of it. The most aggressive action is available only on an item’s Edit view, ie the view where you’re looking at that item and only that item. It’s not a batch action and there’s no ambiguity or vague context about it - you’re looking at that item zoomed on the map, you can see that item’s segments on the map, and you’re making a decision about only that item.
The Clean Up action is its own kind of thing - it’s not a “confirm” action and it’s named to match what it does - clean up some mess you can see. And it’s only available from views where you can definitely see that mess either in the segments list or on the map or both.
The new Confirm All action is only available on this new view, and is done for a group of items rather than a single item. On that view you can’t see each item’s segments - they’re not shown on the map and you’re not in a segments view that lists them out. If it did the most aggressive action to each of those items, eg overwriting the confirmed types of some inner segments that you’d previously carefully edited, you wouldn’t see that happen and it wouldn’t be obvious that that’s what you’d be doing. You could be accidentally overwriting a bunch of carefully edited segments inside a bunch of items, realising that’s what you were doing.
So basically that means that button has to be restricted to only doing what it says and nothing else, ie confirming the items. Any more than that and it could be undoing previously carefully tweaked segment splits and type changes, without any visibility of it happening.
That would mean that the only way to “confirm all” would be to wipe out any previous careful edits. You’d be left with a new batch edit button that you can’t use if you’ve done any previous editing that you want to preserve.
The way it is now, it’s always safe to tap Confirm All, because it’s only going to do what it says. You get the items’ classified types locked in for all of the items in the list, with one quick tap. And if you had done any previous careful editing within any of those items, that work won’t be lost.
I think that makes sense for cases where there are conflicting edit types
But in most cases I haven’t edited any of the points, especially if it’s a new day, I think it would make more sense if it only appeared for items that haven’t had any human interaction and changes all points to the correct one
The way it is right now, assumes a lot more human cleaning of data than likely occurs, especially as this feature gets applied to new data of data
Also I frustratingly cant combine these segments, even after changing the type and then back to the same type, I can’t marge them because they’ve both had endpoints confirmed in the new interface
Then it wouldn’t be possible to have a “Confirm All” button, because some unconfirmed items would have to be left out.
I think the distinction here is between “confirming” and “cleaning up”. The confirm button on the Item Edit view will effectively do both, but in the other cases it’s one or the other, not both. And it basically has to be that way, otherwise you can’t safely have a “Confirm All” button.
Oh sorry I forgot to address this one! This one is entirely separate, and likely a bug.
There’s long been an edge case where two adjacent items of same type will mysteriously not merge. But then going into one of the items and reconfirming it would trigger a successful merge. In the latest update I found one cause of that bug and fixed it, reducing the cases where it would happen, but there’s still a case left where it can happen.
I’m not quite sure what that final case is (which is why I didn’t get it fixed!) But I’m still on the lookout for it.
Was going to post a new post on this, but it looks like it’s the same issue as Hutima has above with the car — took a flight and now I’ve got 17 different “Airplane” segments all next to each other and not merging (I counted!) along with a few “Thinking…” in the middle there.
This is just a small segment of it, but I’ve tried clicking every single one to re-reclassify as Airplane, as well as going into the individual segments (which are each 1 segment each) and re-reclassifying as Airplane, but they won’t merge.
Yep, exactly. I 99% know what’s causing this, and can get it fixed. Coincidentally I’ve just had the exact same thing happen with a flight this week too, so it’s staring at me, reminding me I have to fix the bug!
Once I’ve fixed the bug, you should be able to do exactly as you already tried - go in and set the type to airplane again, which then triggers a reprocessing run, even though the type didn’t technically “change”. At that point the items will happily merge.
Aside: This new bug is actually a misfiring feature. Many years ago I built a merge rule into the processing engine to stop adjacent path items being merged if that merge couldn’t plausibly have happened in the physical world. For example if there’s a car item with average speed of 50 km/h, then a gap of 10 seconds and 50 kilometres, and another car item with speed of 50 km/h, that couldn’t possibly have physically happened. you can’t travel at 50 km/h and cover 50 kilometres in 10 seconds. So the merge isn’t allowed.
The catch is that I made a typo in that merge rule and it was never applied. I did a dumb. I also somehow never noticed that it wasn’t ever being applied, and pretty much no one else did either - we were all perfectly happy with merges happening that wouldn’t have been possible in the real world.
(Oh, the point of that rule was largely to avoid imported data getting merged in with existing data in nonsensical ways. And for a long time the only way to import data was from Moves app data, so we also didn’t really run into these physically impossible merge situations often anyway.)
Now, flash forward to today, and in the latest update I “fixed” that broken logic that was stopping that merge rule from being applied. Suddenly it’s working! Or rather “working”. It turns out that there’s a whole lot of situations where the data can be dodgy in various ways that make that physical speed/distance test not hold up well. Airplanes are a common case. During flight it’s common for a lot of “nolo” data to be recorded, ie “no location data”. That makes it very hard to accurately determine the location where one path item ends and another starts, which then messes up the speed/distance maths and the merge rule goes from being a helpful sanity check to being just purely a nuisance.
I actually did account for that when I “fixed” it just the other week. But… clearly I didn’t account for the nolos problem well enough! Because it’s still happening. But next time I’ll get it right
All that makes sense and I appreciate the detailed response!
Do you have a rough guess of when it might be fixed (are we talking days, weeks, months etc.?)
When it is fixed, will it automatically clean itself up without me knowing, or do I need to remember to go back to that day and “trigger” it to clean itself up?
Right now I’m deep in some work on the upcoming Arc Editor app, trying to get that into beta as soon as possible. But the same bug/misfeature exists in the new code too, so I’m seeing the same problem pop up there every few days - it’s hard to miss!
So as soon as I next lose my temper with it and dive in to attempt to unravel the broken logic I’ll be essentially fixing it in both apps (it’s the same logic in both) which will mean I’ll have the fix ready to ship in Arc Timeline too.
You’ll need to go back and trigger it. You’ll be able to do that by going into the Edit view for one of the items that haven’t merged and reconfirming their type. That’ll trigger the processing engine to reassess the potential merges.
Not yet, sorry! I’m still working hard on getting Arc Editor ready. But haven’t been working on that particular code again in the past couple of weeks, so it hasn’t come up again yet.
I think it’s just chance that I haven’t had a situation that’s caused the same data problem in a while, so I haven’t been forced to address it yet while working on Arc Editor. But it’s inevitable. And Arc Editor itself is getting closer to being ready for at least private beta. I’d say public beta could potentially be some time next month, if all goes well.
Is there another post somewhere about what exactly Arc Editor is? It’d be really great to see some sort of guide on which of all these apps (Arc, Arc Mini, Arc Recorder, Arc Editor) I should keep vs delete and which I should be actively using vs just letting run in the background.
I think I’ve talked about it a lot, but not in any single place. Arc Editor is basically a full rewrite / rebuild of Arc Timeline app, to get rid of a decade of cruft, mystery bugs and crashes, weird timeline processing, etc.
It’s being initially called Arc Timeline Editor because at first it won’t have the Activity and Overview tabs, only the Timeline tab, so initially it’ll just be good at viewing and editing the timelines.
But eventually it’ll also get those tabs, and end up completely replacing old Arc Timeline app - that’s the goal. A fully fresh, clean start, with that decade of various accumulated problems cleaned away.
Right now the best combo is Arc Timeline and Arc Recorder. Arc Mini doesn’t have much use anymore, except perhaps for free users who don’t want to pay for Arc Timeline. I’ll eventually remove Arc Mini from the App Store - perhaps in the next month or two.
Once Arc Editor is ready, I’ll personally probably run Arc Timeline (for the Activity and Overview tabs), Arc Editor (for the cleaned up and bug free timeline editing), and Arc Recorder (for the reliable recording with extremely low risk of the app being terminated, thus great protection from data gaps).
Once Arc Editor is fully ready to take over from Arc Timeline, perhaps early next year, the best combo will be Arc Editor and Arc Recorder. At which point Arc Editor will probably also be renamed Arc Timeline, with the old Arc Timeline removed from the App Store.
Personally yeah I’d just delete Arc Mini, unless you like using it at all (I don’t!). I only have Arc Timeline and Arc Recorder on my main phone.
I keep Mini on my other test devices, because it’s still on the App Store so I’ve got to keep testing it to make sure it keeps working. But I’m looking forward to deleting it from everywhere once I remove Mini from the App Store.
Actually… I’m not sure why I haven’t already removed it from the App Store. Hmm. I think maybe just because I should put a message in Mini first, to tell people to switch over to Arc Recorder. Yeah, that’ll be why.
Hey Matt – wanted to check in on the status of this update, as I’ve had another flight or two that looks like lots of broken little segments that aren’t joining together even if I go back and re-assign as “Airplane”. From the history here, you’d last said back in early October that it would be more like “days” rather than “weeks” for a fix but I haven’t seen any App Store updates in a few months (and hence it’s still not fixed).
The temp fix for this issue I found is just to make it stationary, delete the visit, and usually the segment will join together and you can reassign the stationary segment back to the original
Kinda sucks it’s manual like that but at least it’ll be fixed
In truth, I just haven’t run into one of these cases again. Even for a few recent flights that looked like they were going to fail to merge… they did merge. So I didn’t have test data, and it once again slipped my mind. A bit of a case of good luck for me, bad luck for everyone else! My successful merges meant I forgot the problem was there and needed fixing
I need to prepare a new build/release within the next few days, for a few other reasons, so I’ll make sure to get the fix for this in to that.
I’ve been deep in Arc Editor work, still churning away trying to get that ready for beta, which has been taking up all my mental space. But a new Arc Timeline release really is overdue, so I’ll set aside today for purely working on that, which will get the ball rolling.
Oh this is a clever hack @Hutima! I hadn’t thought of that one. Though I’d be terrified to try, in fear of it merging into an adjacent visit and creating a more confusing mess to clean up. But certainly it sounds like a workable way around the trip merge problem, which is something I thought was otherwise impossible.
Anyway… today, no more Arc Editor work, only Arc Timeline! I’ll get that new release started. All the items I have for inclusion in it are minor fixes, and this one also counts as a minor fix (either I figure out the proper fix or I just turn off the merge rule altogether). So it shouldn’t take more than a few days to prep the build and release. Fingers crossed I don’t get distracted again
Good news! When working on the bug yesterday I found a minor logic weakness, and fixed it. I wasn’t confident it’d help much, so was going to send the people in this thread a TestFlight build to check, rather than waiting until the App Store release. But then going back through my data I found a recent flight that hadn’t merged properly. And… just viewing that day’s timeline, it self corrected and merged up all perfectly!
So hopefully that means it’s finally fixed!
I’m going to do a couple of other minor unrelated fixes, then submit a build to the App Store in the next day or two.
