RISKS Forum mailing list archives

RISKS-LIST: Risks-Forum Digest  Friday 18 April 2025  Volume 34 : Issue 61

ACM FORUM ON RISKS TO THE PUBLIC IN COMPUTERS AND RELATED SYSTEMS (comp.risks)
Peter G. Neumann, founder and still moderator

***** See last item for further information, disclaimers, caveats, etc. *****
This issue is archived at <http://www.risks.org> as
  <http://catless.ncl.ac.uk/Risks/34.61>
The current issue can also be found at
  <http://www.csl.sri.com/users/risko/risks.txt>

  Contents: Way-Backlogged...  Taking a few at a time
Gov IT whistleblower threatened at home (ArsTechnica)
Starliner crew post-return interview; Important Lessons (ArsTechica)
DOGE Plans to Rebuild SSA Code Base in Months, Risking Benefits and System
 Collapse (WiReD)
The DOGE Axe Comes for Libraries and Museums (WiReD)
DOGE reportedly using Google Docs in violation of vetting and chains of
 custody (Lauren Weinstein)
Another Masterful Gambit: DOGE Moves From Secure, Reliable Tape Archives to
 Hackable Digital Records (404Media)
Ireland probes Musk's X for feeding Europeans' data to its AI model Grok
 (Politico)
Silicon Valley crosswalk buttons apparently hacked to imitate Musk,
 Zuckerberg voices (Palo Alto Online)
Hacked pedestrian crossings play fake messages from Musk and Zuckerberg
 (BBC)
Em-dashes considered a sign of AI-written text -- not joking, but hilarious
 (Lauren Weinstein)
A little nerd humor from Sunday's Demonstration. (Boston, via P M Wexelblat)
NATO acquires AI military system from Palantir (FT)
AI models still struggle to debug software, Microsoft study shows
 (TechCrunch)
Tariffs and AI (NY Times via Jim Geissman)
TLS certs to expire at 47 days by 2029 (Cliff Kilby)
Abridged info on RISKS (comp.risks)

----------------------------------------------------------------------

Date: Thu, 17 Apr 2025 12:07:47 PDT
From: Peter Neumann <neumann () csl sri com>
Subject: Gov IT whistleblower threatened at home (ArsTechnica)

https://arstechnica.com/tech-policy/2025/04/government-it-whistleblower-calls-out-doge-says-he-was-threatened-at-home/

The person logging in from Russia apparently had the correct credentials for
a DOGE account, according to Berulis. "Whoever was attempting to log in was
using one of the newly created accounts that were used in the other
DOGE-related activities, and it appeared they had the correct username and
password due to the authentication flow only stopping them due to our
no-out-of-country logins policy activating," he wrote. "There were more than
20 such attempts, and what is particularly concerning is that many of these
login attempts occurred within 15 minutes of the accounts being created by
DOGE engineers."

An assistant chief information officer (ACIO) was given instructions that IT
employees "were not to adhere to SOP [standard operating procedure] with the
DOGE account creation in regards to creating records," Berulis wrote. "He
specifically was told that there were to be no logs or records made of the
accounts created for DOGE employees."

DOGE officials were to be given "the highest level of access and
unrestricted access to internal systems," specifically "tenant owner"
accounts in Microsoft Azure that come "with essentially unrestricted
permission to read, copy, and alter data," Berulis wrote. These "permissions
are above even my CIO's access level to our systems" and "well above what
level of access is required to pull metrics, efficiency reports, and any
other details that would be needed to assess utilization or usage of systems
in our agency."

Berulis described several more suspicious events that followed DOGE's
arrival. There was a new container that he described as "basically an
opaque, virtual node that has the ability to build and run programs or
scripts without revealing its activities to the rest of the network."  There
was also a token that "was configured to expire quickly after creation and
use, making it harder to gain insight into what it was used for during its
lifetime."

On March 6, various users "reported login issues to the service desk and,
upon inspection, I found some conditional access policies were updated
recently," he wrote. This was odd because "policies that had been in place
for over a year were suddenly found to have been changed with no
corresponding documentation or approvals," he wrote. "Upon my discovery of
these changes, I asked the security personnel and information assurance team
about it, but they had no knowledge of any planned changes or approvals."

On March 7, Berulis says he "started tracking what appeared to be sensitive
data leaving the secured location." About 10GB of data was exfiltrated, but
it was "unclear which files were copied and removed," he wrote. On that same
day, Berulis says he reported his concerns about sensitive data being
exfiltrated to CIO Prem Aburvasmy.

On March 10, Berulis found that controls in Microsoft Purview to prevent
insecure or unauthorized access from mobile devices had been disabled, he
wrote. "In addition, outside of expected baselines and with no corresponding
approvals or records I could find I noted the following: an interface
exposed to the public Internet, a few internal alerting and monitoring
systems in the off state, and multi-factor authentication changed," he
wrote.

The team observed more odd activity in the ensuing weeks, Berulis
wrote. Data was sent to "an unknown external endpoint," but the network team
was unable to obtain connection logs or determine what data was removed, he
wrote. There were also "spikes in billing in Mission Systems related to
storage input/output" associated with projects that could no longer be found
in the NLRB system, indicating that "resources may have been deleted or
short-lived," he wrote.

"Accordingly, we launched a formal review and I provided all evidence of
what we deemed to be a serious, ongoing security breach or potentially
illegal removal of personally identifiable information," he wrote.

But on April 3 or 4, the assistant CIO "and I were informed that
instructions had come down to drop the US-CERT reporting and investigation
and we were directed not to move forward or create an official report,"
Berulis wrote.

------------------------------

Date: Thu, 17 Apr 2025 12:07:47 PDT
From: Peter Neumann <neumann () csl sri com>
Subject: Starliner crew post-return interview; Important Lessons
 (ArsTechica)

An ArsTechnica article based on an interview with Astronauts Butch Willmore
and Suni Williams describes the partial timeline of thruster problems
experienced on the maiden crewed Starliner flight. Some good lessons about
"mission rules" and what to do when things do not go as planned.

https://arstechnica.com/space/2025/04/the-harrowing-story-of-what-flying-starliner-was-like-when-its-thrusters-failed/

------------------------------

Date: Mon, 31 Mar 2025 01:44:04 -0400
From: Gabe Goldberg <gabe () gabegold com>
Subject: DOGE Plans to Rebuild SSA Code Base in Months, Risking Benefits and
 System Collapse (WiReD)

Social Security systems contain tens of millions of lines of code written in
COBOL, an archaic programming language. Safely rewriting that code would
take years -— DOGE wants it done in months.  ...

In order to migrate all COBOL code into a more modern language within a few
months, DOGE would likely need to employ some form of generative artificial
intelligence to help translate the millions of lines of code, sources tell
WIRED. “DOGE thinks if they can say they got rid of all the COBOL in months,
then their way is the right way, and we all just suck for not breaking
sh*t,” says the SSA technologist.

DOGE would also need to develop tests to ensure the nesw system’s outputs
match the previous one. It would be difficult to resolve all of the possible
edge cases over the course of several years, let alone months,

“This is an environment that is held together with bail wire and duct tape,”
the former senior SSA technologist working in the office of the chief
information officer tells WIRED. “The leaders need to understand that
they’re dealing with a house of cards or Jenga. If they start pulling pieces
out, which they’ve already stated they're doing, things can break.”

https://www.wired.com/story/doge-rebuild-social-security-administration-cobol-benefits/

------------------------------

Date: Wed, 2 Apr 2025 15:42:11 -0400
From: Gabe Goldberg <gabe () gabegold com>
Subject: The DOGE Axe Comes for Libraries and Museums (WiReD)

The Institute of Museum and Library Services has long received bipartisan
support. But after years of trying, President Donald Trump has delivered it
a crushing blow.

https://www.wired.com/story/institute-museum-library-services-layoffs

------------------------------

Date: Tue, 8 Apr 2025 07:55:16 -0700
From: Lauren Weinstein <lauren () vortex com>
Subject: DOGE reportedly using Google Docs in violation of vetting and
 chains of custody

------------------------------

Date: Tue, 8 Apr 2025 13:00:38 -0700
From: "Jim" <jgeissman () socal rr com>
Subject: Another Masterful Gambit: DOGE Moves From Secure, Reliable Tape
 Archives to Hackable Digital Records

https://www.404media.co/doge-gsa-magnetic-tape-archives-digital-storage/

------------------------------

Date: Fri, 11 Apr 2025 10:22:03 -0700
From: Steve Bacher <sebmb1 () verizon net>
Subject: Ireland probes Musk's X for feeding Europeans' data to its AI model
 Grok (Politico)

The investigation threatens to stoke further tensions between the EU and
U.S. over tech rules.

Ireland's privacy regulator launched an investigation on Friday into how
social media platform X has used Europeans' personal data to train its
artificial intelligence model Grok.

The move to target the platform owned by Elon Musk, tech billionaire and
right-hand man to United States President Donald Trump, is likely to stoke
further tensions between the EU and U.S. over Europe's tech rules and
regulations.

The probe by Ireland's Data Protection Commission (DPC) looks into how
personal data "in publicly-accessible posts" on X were processed to train
Grok, the regulator said in a statement on Friday.

Musk's AI startup xAI has been developing a group of AI models under the
name Grok, which are used to power things like the AI chatbot available on
the X platform.

Grok's gobbling of EU data was already the subject of scrutiny from the
Irish regulator last year, when X — after a battle in the Irish courts -—
agreed to suspend the use of EU citizens' data to train its AI models.

The Irish regulator said on Friday that its new investigation will examine
whether X has been complying with the EU's General Data Protection
Regulation (GDPR), including whether data was processed lawfully and
according to transparency rules.

X did not immediately respond to a request for comment.

https://www.politico.eu/article/irish-dpc-launches-investigation-into-xs-use-of-eu-data-to-train-ai/ 

------------------------------

Date: Sun, 13 Apr 2025 16:07:14 -0400
From: Gabe Goldberg <gabe () gabegold com>
Subject: Silicon Valley crosswalk buttons apparently hacked to
 imitate Musk, Zuckerberg voices (Palo Alto Online)

Crosswalk buttons along the mid-Peninsula appear to have been hacked, so
that when pressed, voices professing to be Mark Zuckerberg or Elon Musk
begin speaking.

Videos taken at locations in Redwood City, Menlo Park and Palo Alto show
various messages that begin to play when crosswalk buttons are hit. The
voices appear to imitate how Zuckerberg and Musk sound.

In one video, taken on Saturday morning at the corner of Arguello Street,
Broadway and Marshall Street in Redwood City, a voice claiming to be
Zuckerberg says that “it’s normal to feel uncomfortable or even violated as
we forcefully insert AI into every facet of your conscious experience. And I
just want to assure you, you don’t need to worry because there's absolutely
nothing you can do to stop it.”

In another video, taken in downtown Palo Alto early on Saturday morning, a
voice claiming to be Musk says that he would “like to personally welcome you
to Palo Alto.”

vhttps://www.paloaltoonline.com/technology/2025/04/12/silicon-valley-crosswalk-buttons-apparently-hacked-to-imitate-musk-zuckerberg-voices/

------------------------------

Date: Tue, 15 Apr 2025 21:37:34 -0600
From: Matthew Kruk <mkrukg () gmail com>
Subject: Hacked pedestrian crossings play fake messages from Musk
 and Zuckerberg (BBC)

https://www.bbc.com/news/articles/ckgejgd0d3ro

Pedestrian crossings in several areas of northern California have been
hacked with fake greetings mocking the tech billionaires Elon Musk and Mark
Zuckerberg.  Officials in Silicon Valley are investigating and have disabled
the audio feature on the crossings which usually plays instructions to
"walk" or "wait".  The surprise message were noticed over the weekend in
Palo Alto, Redwood City and Menlo Park -- which is home to Zuckerberg's
sprawling Meta campus.

One Musk impersonation offered to buy passing pedestrians a Tesla
Cybertruck if they agreed to be his friend. Another from a false Zuckerberg
said "real ones call me The Zuck".

  [Jan Wolitzky noted an article in the LA Times.  A lot of media editors
  seem to need a little levity.  PGN]

------------------------------

Date: Tue, 15 Apr 2025 08:11:41 -0700
From: Lauren Weinstein <lauren () vortex com>
Subject: Em-dashes considered a sign of AI-written text -- not joking, but
 hilarious

I have -- basically since the start of my writing -- extensively used "--",
probably more than I should, but it's a habit and narrative style
punctuation I prefer. I never actually use em dashes myself, though some
platforms will automatically convert "--" to an em dash by default. I mainly
edit in ASCII editors, and of course em-dash isn't even supported there. I
also prefer "--" since I know for sure how it will be displayed to the
reader, while there is still less assurance with em-dashes. If em-dashes are
now considered a sign of AI-written text due to their use by ChatGPT, etc.,
that's fairly hilarious. -L

------------------------------

Date: Mon, 7 Apr 2025 12:54:08 -0400
From: P M Wexelblat <wex () mac com>
Subject: A little nerd humor from Sunday's Demonstration. (Boston, of course)

         [PGN's representation of the snapshot:
     An eating place display: BREAKFAST and LUNCH
     A hand-made banner: HANDS OFF: WORKING COBOL CODE  
         ]

------------------------------

Date: Mon, 14 Apr 2025 19:06:34 -0700
From: geoff goodfellow <geoff () iconia com>
Subject: NATO acquires AI military system from Palantir (FT)

NATO has acquired an artificial intelligence-powered military system from
Palantir, the US software company chaired by Donald Trump-backer Peter
Thiel and with strong Pentagon connections.

The alliance's choice comes amid rising anxiety among European members over
a potential US withdrawal after Trump threatened to stop protecting the
continent if capitals did not drastically increase defence spending. Nato is
also racing to keep up with the development of rivals' AI military
capabilities such as China.

Palantir's Maven Smart System (MSS NATO) uses generative AI, machine
learning and large language models to provide Ccommanders with a secure,
common operational capability and will be used to support ongoing NATO
operations, the alliance said on Monday.

Such battle-space management systems allow 20-50 soldiers to do the work
sifting through battlefield data that teams of hundreds or even thousands
did in recent conflicts such as Afghanistan and Iraq.

``It's able to take the place of entire teams doing these rather dull
tasks,'' said Noah Sylvia, analyst at Royal United Services Institute, a
London-based think-tank.

France has developed Artemis, which Sylvia said was a domestic alternative,
but not a competitor to Palantir's Maven system, so as not to be reliant on
the US. [...]

 https://on.ft.com/4j2G9fU

------------------------------

Date: Sat, 12 Apr 2025 08:01:24 -0700
From: Steve Bacher <sebmb1 () verizon net>
Subject: AI models still struggle to debug software, Microsoft study shows
 (TechCrunch)

AI models from OpenAI, Anthropic, and other top AI labs are 
increasingly being used to assist with programming tasks. Google CEO 
Sundar Pichai said in October that 25% of new code at the company is 
generated by AI, and Meta CEO Mark Zuckerberg has expressed ambitions to 
widely deploy AI coding models within the social media giant.

Yet even some of the best models today struggle to resolve software bugs 
that wouldn't trip up experienced devs.

A new study from Microsoft Research, Microsoft’s R&D division, reveals 
that models, including Anthropic’s Claude 3.7 Sonnet and OpenAI’s 
o3-mini, fail to debug many issues in a software development benchmark 
called SWE-bench Lite. The results are a sobering reminder that, despite 
bold pronouncements from companies like OpenAI, AI is still no match for 
human experts in domains such as coding.

The study's co-authors tested nine different models as the backbone for 
a “single prompt-based agent” that had access to a number of debugging 
tools, including a Python debugger. They tasked this agent with solving 
a curated set of 300 software debugging tasks from SWE-bench Lite.

According to the co-authors, even when equipped with stronger and more
recent models, their agent rarely completed more than half of the debugging
tasks successfully. Claude 3.7 Sonnet had the highest average success rate
(48.4%), followed by OpenAI’s o1 (30.2%), and o3-mini (22.1%).  [...]

https://techcrunch.com/2025/04/10/ai-models-still-struggle-to-debug-software-microsoft-study-shows/ 

------------------------------

Date: Sat, 5 Apr 2025 08:06:40 -0700
From: "Jim" <jgeissman () socal rr com>
Subject: Tariffs and AI

NYTimes chat, Ezra Klein and Paul Krugman, 5 Apr 2025

  [Klein:] One of the things flying around social media has been that if you
  went and you asked the various leading artificial intelligence programs,
  ChatGPT and Gemini and Claude: What's a pretty simple way to calculate
  tariffs on all other countries? -- it will offer you basically the
  calculation [Trump administration] used [when calculating other countries'
  tariffs].

  [Krugman:] This is part of the problem with what we're calling AI, with
  large language models. They pick up what's out there without necessarily
  being able to discriminate what is sensible and what is not.

There's certainly no paper I would imagine in any economics journal saying:
Do this. Maybe some people out there are saying something like this. But it
really is not something you would recommend, if you know anything about how
trade works -- which ChatGPT does not. So it really is weird that it would
come up with this.

------------------------------

Date: Tue, 15 Apr 2025 01:33:17 +0000
From: Cliff Kilby <cliffjkilby () gmail com>
Subject: TLS certs to expire at 47 days by 2029

Newer piece
https://www.theregister.com/2025/04/14/ssl_tls_certificates
Slightly older piece
https://www.theregister.com/2024/10/15/apples_security_cert_lifespan/

"And while it's generally agreed that shorter lifespans improve Internet
security overall -- longer certificate terms mean criminals have more time
to exploit compromised website certificates -- the burden of managing these
expired certs will fall squarely on the shoulders of website and systems
administrators."

No.

47 days is security theatre. It will remove certificate invalidation as a
control mechanism. There will be little point in maintaining the certificate
revocation list (CRL) as the attitude will be "well, it will just expire
anyway".

It's hard to fake a successful response from a revocation check that
indicates the certificate has not been invalidated, but what happens when
the attacker gains control of the clock?

Sure, it's difficult to grab the clock from the browser, but, browsers
aren't the only place that certificate validity is checked. The other ones
are slightly more critical. For example, driver signing.

Actual security would be limiting SAN to within the same second level
domain, instead of the current process which allows someone to cut a cert
with a dozen seemingly unrelated domains attached. Or, having issuers
automatically add expired certs to the CRL. Or expanding support for Name
Constraints. Or, changing Certification Authority Authorization (CAA) policy
to default deny for domains that have no CAA records at all where the
current policy is default allow. Or, actually removing TLS<1.3. Or,
rejecting certificates that were issued with less than 128 bit entropy
(i.e. <3072-bit RSA).

I think the most direct evidence this is all made up is this quote from
Tim Callan, chief compliance officer at Sectigo and vice-chair of the CA/B
Forum. "This pivotal and positive advancement for our industry underscores
the importance of agility and proactive risk management in today's threat
landscape while preparing for the risks of the quantum era."

TLS1.2+ with AES-256 is quantum resistant. And it's already available. And
it's built in to all of these browsers.

------------------------------

Date: Sat, 28 Oct 2023 11:11:11 -0800
From: RISKS-request () csl sri com
Subject: Abridged info on RISKS (comp.risks)

 The ACM RISKS Forum is a MODERATED digest.  Its Usenet manifestation is
 comp.risks, the feed for which is donated by panix.com as of June 2011.
=> SUBSCRIPTIONS: The mailman Web interface can be used directly to
 subscribe and unsubscribe:
   http://mls.csl.sri.com/mailman/listinfo/risks

=> SUBMISSIONS: to risks () CSL sri com with meaningful SUBJECT: line that
   includes the string `notsp'.  Otherwise your message may not be read.
 *** This attention-string has never changed, but might if spammers use it.
=> SPAM challenge-responses will not be honored.  Instead, use an alternative
 address from which you never send mail where the address becomes public!
=> The complete INFO file (submissions, default disclaimers, archive sites,
 copyright policy, etc.) has moved to the ftp.sri.com site:
   <risksinfo.html>.
 *** Contributors are assumed to have read the full info file for guidelines!

=> OFFICIAL ARCHIVES:  http://www.risks.org takes you to Lindsay Marshall's
    delightfully searchable html archive at newcastle:
  http://catless.ncl.ac.uk/Risks/VL.IS --> VoLume, ISsue.
  Also, ftp://ftp.sri.com/risks for the current volume/previous directories
     or ftp://ftp.sri.com/VL/risks-VL.IS for previous VoLume
  If none of those work for you, the most recent issue is always at
     http://www.csl.sri.com/users/risko/risks.txt, and index at /risks-34.00
  ALTERNATIVE ARCHIVES: http://seclists.org/risks/ (only since mid-2001)
 *** NOTE: If a cited URL fails, we do not try to update them.  Try
  browsing on the keywords in the subject line or cited article leads.
  Apologies for what Office365 and SafeLinks may have done to URLs.
==> Special Offer to Join ACM for readers of the ACM RISKS Forum:
    <http://www.acm.org/joinacm1>

------------------------------

End of RISKS-FORUM Digest 34.61
************************

Current thread:

• Risks Digest 34.61 RISKS List Owner (Apr 18)