Activity summaries incorrect

Any idea why these activity summaries don’t sum up properly

Arc app mini says that the summaries were calculated two hours ago so I thought they would fix itself but it doesn’t seem to be the case

It gets even more inaccurate when you pull the yearly summary

Hm, yeah, those don’t look very added up!

The scheduled background task for updating the activity summaries will update all the summaries that’ve been queued for update, but summaries are only queued for update after you’ve viewed them. This is to avoid the app having to update activity summaries that you never view. For example if you never view weekly summaries, those won’t get queued for updating, saving precious background processing time (of which iOS gives Arc very little allowance of).

Try flicking back and forth between previous and next date ranges of the summaries, to trigger an immediate update. You should see the top part of the summary grey out, which means it’s updating right now. After it fades back in (meaning it finished updating), if the numbers still don’t add up right, let me know! That would definitely be something weird and wrong.

Oh, the top section updates first, then the individual activities beneath each update individually. So you might see loading spinners on some of the activity boxes for a little bit after the top section has finished updating.

Just playing around with my own summaries now, I got my yearly to update to latest by flicking to previous year then back, and it adds up properly (I think it added up properly before that, but I didn’t check properly). My current week is also adding up correctly, but last week’s is off by about half an hour, which seems too large a discrepancy to blame on rounding. I’ll dig into the code and see if I can figure out why there might be going on there…

Yeah it seems like when I do that the last month or so looks correct, but the further back it seems more incorrect

It looks like the calculation isn’t propagating down (iOS is probably stopping it and the app may or may not be detecting that’s the case)

Days I’ve viewed before seem calculated and show up at higher orders of dates, but calculating the higher order of date doesn’t necessarily mean the lower ones are calculated properly

Not sure if it matters but I’m having the most issues with 2020 and 2021 and the first half of 2022, it looks like 2017-2019 sum up, this might be because I’ve manually viewed the dates in calendar to correct import mistakes so it might also be days that never got viewed in timeline aren’t processed properly

It might be this is only an issue for people who restore from backup and don’t go through every single day for timeline calculations, I just find it odd the yearly map works but the yearly activity doesn’t in which case it might not be worth the time to fix

I need to give this stuff another going over at some stage. I also find it disconcerting, going back to older date ranges and seeing outdated numbers, then flicking back and forth to see if I can trigger an update (which has probably already been queued on first view, but really I want to see it updating immediately, while I wait).

So yeah, if you view an older date range and it looks wrong, it should have been queued for update the moment you viewed it, and should then later end up with correctly updated numbers. But there’s no visual feedback on that “queued for update later”, so you’re left with flicking around between dates until you see it being live updated in front of your eyes. Really not ideal :smirk:

I think there’s still an occasional error though

I do notice it turns grey when calculating and confirms, but sometimes after summing up an activity type it just doesn’t show up

An example is below here taxi was calculated, but it then disappeared in the final summary

Yeah I’ve seen similar sometimes. Though it’s hard to catch for debugging, because it tends to come right on its own in the end, and frustratingly doesn’t happen when I’m trying to debug the problem!

Was there meant to be taxi time in that day?

Ah kindof, it was supposed to be kayaking but I’ve been toggling it back and forth to get this specific 50 minute segment to show up

I can get it to show up if I change one of the start/end points and make it a new activity type.

I can get it to keep showing up if I change it back to kayaking, But if I change the start point back to the original it disappears, and this new activity type is also “infected” with this bug so it no longer shows up.

The activity is also properly calculated when changed to “walking” which also showed up correctly

So it seems to be tied to a specific start / stop journey

It might be that it most frequently occurs with short trips that start and end in the same location because I have quite a few of these in 2020 when my travel was infrequent and I often walked from home to home just to keep active

I do know that occasionally the whole day is recalculated because when the activity summary task is done in arcmini this type of error sometimes disappears

So it updated an hour ago, added some screenshots from arc mini for debugging

1 Like

Ugh, that all sounds quite a struggle!

I can’t quite make sense of what might be going on in the background there. A lot of what you describe doesn’t make sense. But then if it did all make sense I’d have found the bug by now and fixed it :joy:

In your second screenshot I can see the activitySummaryUpdates hasn’t completed for 9 days, which will be part of the problem. If an activity summary isn’t getting updated in real time while you’re viewing the Activities tab then it’ll get updated later, during that activitySummaryUpdates task. But being 9 days behind means there must be some backlog it’s not getting through.

It looks like you might have monthly auto GPX/JSON exports turned on? Which is also 9 days behind. That task (and the daily auto exports) are the heaviest tasks, taking the longest to finish each day. So it’s likely that that one is chewing up most of Arc’s allotted background processing time each day, leaving not much for the rest. I recommend turning it off for a few days, to see if the others can catch up.

Basically I think there’s two distinct things going on:

  1. There’s some weird bug with the real time activity summary updating that happens while we’re viewing the Activities tab. I think they’re all getting queued up properly, but for some reason the ones we’re viewing aren’t always getting updated properly while viewing, even if we’re on the tab for quite a while.

  2. The activity summaries that have been queued for update should all get updated later, by that activitySummaryUpdates background task. But if that task falls too far behind, we’re going to start noticing more stuff that’s not updated, falling further behind reality.

For problem 2 there’s not much I can do. iOS will do what it does, limiting Arc’s background processing time as its mood takes it, and it can’t be negotiated with.

For problem 1, hopefully I can catch it in the act of being weird, and find the bug! It’s all still looking super random and weird though, as your experiments show. But eventually these things shake out. I have hope :joy:

Quick update on this: I forgot to add it to the release notes for Arc 3.7.0, but I’ve significantly improved the speed of activity summary updating in 3.7.0! I’m seeing near instant updating when swiping back through days/weeks/etc now :tada:

1 Like

I did see that thanks!

It didn’t seem to solve the issue with certain activities not being counted though, like on this day car isn’t showing up (again if I change it to a new activity it shows up, but if I change it back to car it disappears)

Honestly couldn’t tell you how to replicate this though since it seems random

I haven’t seen this on any new data though so it might be fixed if you forced a recalc of all summaries with whatever changes you made

1 Like

Yeah I suspected there’d still some be the weirdness you’d identified before. I still haven’t had a chance to hunt down causes for those yet. I’ll keep hunting!

I think I got one of the edge cases and why my 2020 is so weird, though I don’t think it’ll solve every case

Bogus data is being counted I had ten days where my phone was out for repair and it’s counting the 10 days with bogus activity as real activity

Bogus data as a type is excluded from the activity list but still counts as an activity time

This will fix some of the over counting issues I saw while not moving a lot in 2020 but probably won’t fix the odd undercounting issues I’ve experienced

1 Like

Oooh, that sounds very plausible. I’ll look into it!

As a side note, the bogus data is being counted across months, so any activity that spans midnight on the 31st might also duplicated, but that’s probably hard to fix with Time logic

1 Like

That part should hopefully already be accounted for. When it calculates totals for the summaries it fetches all Timeline Items that fall within the date range (either partially or wholly) but for items that bridge the boundaries it only counts the portion of the item inside the range.

But yeah, it seems very likely that I’m not accounting for “bogus”. I bet I added the bogus type after the whole activity summaries system was built.

I went back to the dates of the screenshots posted above and I think your last database fix also fixed those errors, the time seems calculated properly and the missing activity segments are also there, so those might have been fixed indirectly from your last patch

My 2020 though, definitely messed up from all the bogus or missing data from lockdowns

Yeah some of my old data is still messed up too in Activity tab. In my case I think it’s because I recently did a fresh install and full restore from backup.

I think there’s a bug where data restored from backup doesn’t get included in the activity summaries until it’s been viewed at least once in timeline view. Which makes no sense, but that’s what I’m seeing :man_shrugging:t2:

Yay for building a complex system with lots of moving parts over a bunch of years, then gradually forgetting how each part works :grimacing:

I am wondering how does Arc app handle automatic time zone changes and automatic region specific daylight saving time or maybe leap-days/leap years for all these continuous data recordings and calculations/statistics while potentially travelling across time zone borders with region specific time/date measurement. :thinking::earth_americas::earth_africa::earth_asia:

Times are always shown in local, it’s kind of annoying for cross world trips because I am have days cutting off at like 2 PM

But I also know that this is a very hard fix on the coding side

1 Like