Creating a new topic because the most recent similar topic I could find (“ Almost six month of unconfirmed activity”) was two years old and seemed to be related to backups more than normal use.
Hi Matt! I hope you’re doing well and I appreciate your support as always.
I’ve noticed that dozens of old days throughout my history will, sometimes many months after they’ve been confirmed, suddenly have a few unconfirmed activities. I’ll check the calendar view, and see sometimes most of a week or even a month full of the gray “unconfirmed activity” bubbles. I’ve noticed that sometimes more of these will appear on the following date or two, immediately after I view the initial date to make the corrections (though with persistence and a lot of time, you can eventually work through them all; at some point a date won’t ask for more confirmations and you can move on).
That said, it does seem to be the case that revisiting old dates often prompts more unconfirmed activities to appear, usually (but not always) on that date or those immediately following. Sometimes a correction/confirmation on one date will cause new unconfirmed activities to pop up on dates in past months. It’s usually silly corrections. For example, it’ll ask if a 50-mile trip with an average speed of 75mph is walking, or if a segment I’m fairly sure I already marked as cycling in the past is driving. Most of these reconfirmations (and maybe some are very late 1st confirmations?) are movements that are not routine, but I think some are.
For help in diagnosing: this isn’t a new problem (I think I’ve needed to make old confirmations maybe for two years now), though it’s stood out to me lately how many unconfirmed activities there are, and how consistently I have to keep making corrections whenever I’ll visit an older date, because it prompts more to appear.
My only theory is that I may have changed how precisely I categorize movements now…I’ll sometimes separate the 1-minute walk through a parking lot from the 10-min drive over, when maybe I didn’t used to (because I think the questions might’ve been less precise in the past), and the learning model is now confused about what exactly walking vs driving is or something? Not sure.
Anyway, key issue: viewing or correcting past activities seems to spawn more confirmation requests that add up over time.
Let me know if you might have an idea of what might be going on here. I appreciate your time.
Thanks for raising this. I’ve often wondered the same over the past couple of years, and never understood why a date full of confirmed activities and locations will sometimes return to needing reconfirmation of a small number of places or activities.
When you clean up today’s timeline (ie do the confirms/corrects) you’re getting classifier results based on the state of the activity types model today. The model is based on all your previous confirms/corrects prior to today. But then let’s say two months later you’ve subsequently done two months worth of confirms/corrects, changing the state of the model(s) significantly.
You might have changed your daily routines, for example changed from taking the train to work to cycling to work, or changed the time when you go for morning or afternoon runs, or even started running or cycling much faster - there’s all sorts of changes in your behaviours and routines (and subsequently in the model) that will happen over time.
That means that if the classifier/model looks back at some old timeline data that it previously thought was one thing, it might now think it looks like something else. Although if you had previously confirmed an item then it won’t be marked as needing confirmation again - confirmed is confirmed, set in stone. But there will be items in the timeline that the classifier/model had previously concluded didn’t need classification, because it could confidently (at the time) conclude that it was cycling or train or car or whichever. But now, with more detailed and different knowledge of your behaviour, those old items don’t look so certain anymore, and might now fall below the confidence threshold and require confirmation.
I do believe though that there is an Arc bug hiding in there somewhere. Often I will see days marked in grey in the calendar, then tap through to do cleanup, then after a few seconds of thinking the classifier will remove the “needing confirmation” button and conclude that actually they’re all fine - false alarm. So that seems at least superficially like something that shouldn’t happen. If it first says “there’s stuff to clean up on this day”, then after a brief think through I changes its mind to “nah, actually it’s all good”, that can’t be right.
So for that weird case I do have a todo logged, to hunt down why it’s doing that.
For the rest, yeah, it’s the case described above - the model has learnt more over time, and perhaps you’ve also changed your routines and behaviours, such that the conclusions drawn by the new model state disagree with the conclusions drawn by the old.
The trick is that it’s not asking about the ones you did confirm, it’s asking about the ones it thought didn’t need confirmation back then.
So for example if there were 10 items in the day, and at that time it asked you to confirm 3 of them, that means 7 of them fell into the zone where the classifier/model was confident enough in its automatic classifications that it didn’t ask. But then several months later, the newer, more fleshed out model looks back on those 7 and disagrees, thinking that now some of those do need confirmation.
Part of this is simply a case of the model learning more about us. For example if in the very first week in a new neighbourhood the only data the model sees in your timelines is walking and car, it will believe the world is very simple and everything is either walking or car. But then after living in that neighbourhood for 6 months it will know that actually you also cycle, run, take taxis, sometimes buses, etc, etc. So looking back on the old data it now realises it was naive to believe that everything was safely either walking or car.
This feature is pretty irritating, if you spent time in the past “confirming” (or in some cases, “accepting” rather than confirming, as no confirmation is needed) the timelines of each day. It is happening for vacations with me, and years after, I can’t remember (or research) which bar I visited.
I suggest a “confirm whole day” button which locks/confirms the whole day and basically sets also the accepted (not manually confirmed) ones into “confirmed” state, so nothing must be re-evaluated by the ML engine at a later stage.
I may be mistaken but I feel like there are days where I have manually verified every single interaction (travel mode and location) to give them all a thumbs up even when they didn’t prompt me … and then had to reapprove something, maybe 1-2 aspects.
This is not evidenced because I can’t point to a specific day when it’s happened but that’s my memory. I’ll keep an eye on it in future to see if I can replicate it.
I also like the idea of the confirm everything button as a way to “lock in” the day and tell the model to not mess with it (and take it as definitely correct training data)
I usually check my timeline at the end of the day and correct everything that needs correcting, so it would be cool to just say “yep, that’s exactly how it was, no need to overthink it” to the model
In the new neighborhood analogy - maybe I really did just walk and drive for the whole first week, so why make the model needlessly second-guess itself when I can just say that the current state is the ground truth
Gotcha! Thanks for the explanation. That does make sense that it’s reevaluating as the model changes. I can recognize now that some of the trips asking for a review are the sort of trips that I’ve categorized more precisely over time. And I have noticed that as well, where it’ll “pretend” to need confirmation for a second but then seem to “decide” it’s fine after all.
As far as the way unconfirmed activities pop up, I assume it’s set up such that the activities of nearby dates will be run by the model when a date gets viewed? Just out of curiosity, are they checked against the model at other times?
I think I’m a fan of the suggestions being made in this thread; I generally tend to review my history often enough to check that everything looks correct on each day, and wonder if it would be reasonable to allow the model to assume a day is correct in its entirety if it’s been thoroughly reviewed (or even approved with an “everything looks good” button, as some have mentioned).
I gave up on reconfirming stuff in the past long ago. it looks like Arc is messing with me.
As mentioned my other people here, fully verified days pop up unconfirmed. Confirmation Wizard is strenuous for me. I’m a visual person and reading and comprehending what I need to confirm takes too much time. I have an idea to make confirmations more visual and I’ll open a separate topic for that.
If you claim that confirmed items are “set in stone”, please add a function to lock the data on all items in the day (confirm everything as is) after being checked by a user. The first step would be to just go through all items and mark as confirmed.
That will make it very easy to verify your assumption that confirmed data is not automatically changed later.
If it turns out items get unconfirmed later, you can perhaps investigate.
The second level would require to add a “day_confirmed flag” in the table so it is marked on the day level, not item level.
That also could help you hunt for that problem.
I thing there are enough people here in this thread here to help you with finding the problem.
I made a change a while back, to make sure the activity types classifier wouldn’t decide to reclassify old data. It previously was configured to consider data older than maybe 6 months as needing of reassessment, on the basis that the classifier will be smarter now and do a better job of it. But in practice that just made life more difficult, so I changed it to never consider old data due for reassessment.
Which is why I think there’s a bug in there somewhere, with it still somehow getting triggered into reassessing old data, thus still causing this problem.
The intention with the current code is that once the day is past, it’s not up for reassessment (unless you manually start tapping through the items, making changes yourself, at which point the classifier will get involved again). So the fact that it’s not seeming to act that way, that it’s seeming to reassess old data, even when just viewing the calendar view, strongly implies there’s a bug somewhere. It’s in my todos to hunt that out.
This is happening to my data as well. For some reason, starting from October 1, 2019, full months of records have changed (I just went to check today) and according to the current data I live in a supermarket and visit a furniture store daily. I go through Arc history regularly to assign locations while I still remember what happened, so all those months were previously cleared (there are some days where I traveled and I see that some places are correct), but largely it’s dominated by three locations. Very frustrating; I’ll have to go through almost four years of data now and guess where I’ve been.
It’s an ongoing issue because I see manually confirmed (eg yesterday) places and activities reset 1-2 times and I keep going back to reconfirm them daily now.
Another thing I’m seeing with these re-unconfirmed activities is that sometimes going to the day will show an unconfirmed activity, then once I tap on the purple “unconfirmed items” bar it disappears.
And sometimes I go to a day, and I see places change.
This is the one thing I doubt. Once a visit has a confirmed place, it can’t be changed. The ones that Arc will be asking you to confirm will be ones that were never previously confirmed.
If it truely were the case that previously confirmed place assignments were being nulled out, then that would indicate actual database corruption. Which would be a much more serious matter than the classifier deciding to reassess its previous decisions on visits that had never been confirmed.
I’ve never seen the latter happen, and would be extremely concerned if it did. It would also take unequivocal proof to convince me that it has happened!
This was discussed above, and is definitely a bug, which I will hopefully find the cause of soon. But the above scenario you describe where a visit with a confirmed place has become uncofirmed, that’s something altogether else, and if proven, would raise serious concerns about data corruption.
It is happening for sure with activities - which, if I understand, could be the classifier going back and reassessing? I might be lumping activities and places confirmations into one pile, I’ll look specifically for a place requiring reconfirmation.
It will only be happening for activities in the above described bug situation, where it at first thinks there’s some that need confirming, but then either a second later when the view reloads, or when you start the confirmation flow, it will realise it was mistaken.
That’s a specific bug where the timeline view is showing incorrect UI, while the underlying data is still correct.
There’s basically two different situations:
The bug where the UI says some trip items need confirming, then shortly after it realises they don’t.
Previously unconfirmed items (either visit or trip items) that the classifier originally concluded didn’t need confirmation, are now marked as requiring confirmation, due to the model having more detailed data now, thus less certainty on its original choice.
The first one is a minor annoyance that self resolves, but still needs to be fixed. The second one is a larger annoyance, because it does end up forcing us to go through and confirm things that were previously fine as they were.
However there shouldn’t be any case where previously confirmed items become unconfirmed. That would indicate database corruption of some sort, and would be a “drop everything and go into panic mode” for me!
There’s no indication we’re looking at anything like that. Just one minor annoyance and one less minor annoyance, neither of which indicate actual data corruption (phew!), just the app being a bit unnecessarily annoying
This is slightly different than the original question, but I actually have noticed that one-time visit names will sometimes disappear while categorizing a new day with lots of new locations. Maybe it’s unsure of the visit duration/details again after you change the following activity so it adjusts and drops the name in the process? I’ve noticed this a few times recently.
Yeah I’ve seen this bug too. A visit having a one-time title marks it as the same as having a confirmed place assignment, so it shouldn’t be possible for that title to get lost. But it does sometimes. I suspect something in the timeline processing isn’t respecting that “as good as confirmed” condition.
By the way, does arc process past these when it’s open or do you have to go through each day in the UI for you to trigger processing? That is, is there any point in having arc open for a while?
Arc avoids doing any processing in the background outside of what’s absolutely necessary. So in the background it’ll only be processing the current item / some portion of the current day.
When the app is in the foreground, it will only do processing on the timeline data visible in the UI, ie if you’re viewing today, it’ll do processing / metadata updating on today’s timeline items.
So to get it to reassess any past data you would need to navigate to that day from the calendar view (or swiping back). And even then, it shouldn’t be reassessing the actual classifier results for any of that older data, unless you go into the Edit/Confirm view for that item.
That last part is where I think the bug (or one of the bugs, if there’s more than one!) lies. I suspect somehow the classifier is having another go at old data even when just viewing the calendar view (or the timeline view but without tapping into Edit/Confirm). It really shouldn’t do that, but it does very much look like sometimes it does
