If you're interested in this concept, it's not new and the alarm has been sounded since the android Facebook app required motion sensor permissions in android 4.
> alarm has been sounded since the android Facebook app required motion sensor permissions in android 4.
Serves as a useful reminder that just because someone may not care if these companies collect this data now, they are storing it, sometimes indefinitely, and as technology advances, will be able to make more use of it than they were at the time you agreed to share it with them.
It's like all the ransomware gangs hoarding the encrypted data they stole, waiting for a quantum computing breakthrough to be able to decrypt it.
Not sure what to do about it, if anything, but the average person is severely under-equipped and undereducated to deal with this and protect themselves from the levels of surveillance that are soon to come.
There are exceptions, but for the most part, I'm not sure that knowing specifically what I was doing five years ago has much value to me or anyone else?
You are not always the target, but leaking data like this empowers those that may use it against someone else. I mean to say, you are providing training data. People who move-like-this-at-these-times also have these other tendencies... It's like fusion sensing.
Same with secure messaging. You might not care that [insert boogeyman here] know what you're doing all day, but people you interact with may be harmed by you being leaky.
Anyway, the whole point is your lack of imagination is not a good reason to embrace surveillance. Rejecting surveillance on principal slows down our descent into a panopticonical hellscape.
Firstly, I looked up panopticon, and it does indeed seem like that is how modern societies are being structured now, and secondly... It is true that people generally seem to display a lack of imagination regarding all of this. The whole "ive got nothing to hide so what do i care if they surveil me" attitude comes across as selfish and naive in this light.
I wonder what ad-tech or whatever org would do if we got to a state where they just couldnt track or id people. Would we still have free entertainment on the net? would the net even still be mainstream?
I tried denying the Sensor perm to most apps and my battery tanked. My guess is there are a few that sit in a busy loop trying to get the data with no handling of the permission not being granted, because it's expected on 99.99999% of devices
And that's by design, because it waits for a larger window of data to gain confidence. Notice how you ~never get false positives on it starting an exercise session.
I see comments like GP's often enough that I think Apple just does a bad job explaining how/why it works.
> The researchers ran the audio and motion data through smaller models that generated text captions and class predictions, then fed those outputs into different LLMs (Gemini-2.5-pro and Qwen-32B) to see how well they could identify the activity.
Maybe I'm not understanding it, but as I get it, LLMs weren't really important: all they did was further interpreting outputs of a fronting audio-to-text classifier model.
Doesn't the smartphone already far surpass the Telescreen's capabilities and presence? It does more and we carry them in our pockets.
Do people not realize we're beyond 1984? In 1984 the tech wasn't always listening, rather it had the capacity to. Much of it was about how not knowing meant you'd act as if you were just in case. It was making reference to totalitarian states where you don't know if you can freely talk to your neighbor or if they'd turn you in, where people end up creating a double speak
In 1984 the idea was there were not enough people to listen to everyone, all the time, but the mere possibility was enough. Of course, for us with AI, things are considerable worse. Also, tele screens were mandatory. We are not there with cell phones in a de jure sense, but certainly there in a de facto sense. Of course, if enough people carry phones, it doesn't matter if a few stragglers don't, they will get caught in the net unless they live as hermits, in which case who cares about them. All the pieces are in place, there is no reason we cannot have a global North Korea.
Something to note here that annoys me about the title is that the LLMs aren't taking in the raw data (LLM's are for text, after all). The raw data is fed through audio and motion models that then produce natural language descriptions, that are then fed to the LLM.
Unrelated: yeah, this article is a little creepy, but damn is it interesting technically.
The tinfoil interpretatio that LLMs can spy on you is shortsighted and a bit paranoid, it would require LLM providers to actually run a prompt asking what you are doing.
However, any system with a mic, like your cellphone listening for a "Hey Siri" prompt, or your fridge, could theoretically be coupled with an llm on an adhoc basis to get a fuller picture of what's going on.
Pretty cool, if an attacker or govt force with a warrant can get an audio stream they can get some clues although of course not probatory evidence.
we'll inevitably have universal tracking for everything like this (good luck privacy), it's essentially machine learning around a bunch of vibration patterns... ideal for a device that hundreds of millions of people are carrying everywhere daily
One more positive interpretation of Apple's research interests here is that devices like the Watch can better differentiate between "the wearer just fell and we should call 911" and "the wearer is playing with their kids".
If you're interested in this concept, it's not new and the alarm has been sounded since the android Facebook app required motion sensor permissions in android 4.
https://par.nsf.gov/servlets/purl/10028982
https://arxiv.org/pdf/2109.13834.pdf
> alarm has been sounded since the android Facebook app required motion sensor permissions in android 4.
Serves as a useful reminder that just because someone may not care if these companies collect this data now, they are storing it, sometimes indefinitely, and as technology advances, will be able to make more use of it than they were at the time you agreed to share it with them.
It's like all the ransomware gangs hoarding the encrypted data they stole, waiting for a quantum computing breakthrough to be able to decrypt it.
Not sure what to do about it, if anything, but the average person is severely under-equipped and undereducated to deal with this and protect themselves from the levels of surveillance that are soon to come.
There are exceptions, but for the most part, I'm not sure that knowing specifically what I was doing five years ago has much value to me or anyone else?
I was probably sitting in front of a computer.
You are not always the target, but leaking data like this empowers those that may use it against someone else. I mean to say, you are providing training data. People who move-like-this-at-these-times also have these other tendencies... It's like fusion sensing.
Same with secure messaging. You might not care that [insert boogeyman here] know what you're doing all day, but people you interact with may be harmed by you being leaky.
Anyway, the whole point is your lack of imagination is not a good reason to embrace surveillance. Rejecting surveillance on principal slows down our descent into a panopticonical hellscape.
Firstly, I looked up panopticon, and it does indeed seem like that is how modern societies are being structured now, and secondly... It is true that people generally seem to display a lack of imagination regarding all of this. The whole "ive got nothing to hide so what do i care if they surveil me" attitude comes across as selfish and naive in this light.
I wonder what ad-tech or whatever org would do if we got to a state where they just couldnt track or id people. Would we still have free entertainment on the net? would the net even still be mainstream?
My point was more that this data loses its value fast. I'd worry more about it knowing where I was recently.
If only activists and journalists hide, while everybody else provides all data to the government/Facebook, then it becomes easy to track them.
I tried denying the Sensor perm to most apps and my battery tanked. My guess is there are a few that sit in a busy loop trying to get the data with no handling of the permission not being granted, because it's expected on 99.99999% of devices
Maybe the 2026 Apple Watch will be able to auto detect running as reliably as my 2015 Samsung Gear S2. My 2022 Series 8 is certainly not there yet.
That’s weird, I have perfect running detection on an old(er) (Apple) watch. Detection does start late (but is retroactive, so it’s not an issue).
And that's by design, because it waits for a larger window of data to gain confidence. Notice how you ~never get false positives on it starting an exercise session.
I see comments like GP's often enough that I think Apple just does a bad job explaining how/why it works.
Personally, I understand perfectly how it works. But even with the retroactive start, it tends to miss the first five or so minutes.
There are also stats on the screen during tracking that ideally should appear as early as possible.
> Notice how you ~never get false positives on it starting an exercise session.
Oh, I absolutely do.
Why do you need LLM to interpret patterns?
> The researchers ran the audio and motion data through smaller models that generated text captions and class predictions, then fed those outputs into different LLMs (Gemini-2.5-pro and Qwen-32B) to see how well they could identify the activity.
Maybe I'm not understanding it, but as I get it, LLMs weren't really important: all they did was further interpreting outputs of a fronting audio-to-text classifier model.
The same reason you need transistors to make computers.
You don't need them, but they are one way to do it that people know how to implement.
Identifying patterns is fairly amenable to analytic approaches, interpreting them, less so.
A better analogy is: same reason you need FPGAs to run x86 code. You dont, but if FPGAs are hot and good for your career you gonna use them.
To ensure you drank your Verification Can of Mountain Dew, of course.
AI will finally allow us to bring 1984's Telescreens into existence, at scale.
Doesn't the smartphone already far surpass the Telescreen's capabilities and presence? It does more and we carry them in our pockets.
Do people not realize we're beyond 1984? In 1984 the tech wasn't always listening, rather it had the capacity to. Much of it was about how not knowing meant you'd act as if you were just in case. It was making reference to totalitarian states where you don't know if you can freely talk to your neighbor or if they'd turn you in, where people end up creating a double speak
In 1984 the idea was there were not enough people to listen to everyone, all the time, but the mere possibility was enough. Of course, for us with AI, things are considerable worse. Also, tele screens were mandatory. We are not there with cell phones in a de jure sense, but certainly there in a de facto sense. Of course, if enough people carry phones, it doesn't matter if a few stragglers don't, they will get caught in the net unless they live as hermits, in which case who cares about them. All the pieces are in place, there is no reason we cannot have a global North Korea.
> Doesn't the smartphone already far surpass the Telescreen's capabilities and presence? It does more and we carry them in our pockets.
Maybe in data collection, but no one's watching.
With AI, someone can always be watching you. It's really the killer app for AI.
Something to note here that annoys me about the title is that the LLMs aren't taking in the raw data (LLM's are for text, after all). The raw data is fed through audio and motion models that then produce natural language descriptions, that are then fed to the LLM.
Unrelated: yeah, this article is a little creepy, but damn is it interesting technically.
https://abc7.com/post/student-handcuffed-doritos-bag-mistake...
In about:config (Firefox) would
device.sensors.enabled = false
have any effect for browser based access, or is this strictly the app?
The tinfoil interpretatio that LLMs can spy on you is shortsighted and a bit paranoid, it would require LLM providers to actually run a prompt asking what you are doing.
However, any system with a mic, like your cellphone listening for a "Hey Siri" prompt, or your fridge, could theoretically be coupled with an llm on an adhoc basis to get a fuller picture of what's going on.
Pretty cool, if an attacker or govt force with a warrant can get an audio stream they can get some clues although of course not probatory evidence.
we'll inevitably have universal tracking for everything like this (good luck privacy), it's essentially machine learning around a bunch of vibration patterns... ideal for a device that hundreds of millions of people are carrying everywhere daily
Time to ditch the Apple Watch then
One more positive interpretation of Apple's research interests here is that devices like the Watch can better differentiate between "the wearer just fell and we should call 911" and "the wearer is playing with their kids".
nuclear power generation is pretty beneficial, but that doesn't justify the existence of nuclear weapons.
"Can" does not necessarily mean "will", especially for Apple. I wouldn't be surprised if you're describing a feature slated for release in 2037.
https://support.apple.com/en-us/108896